- 28 Aug, 2023 3 commits
-
-
Marc Zyngier authored
* kvm-arm64/6.6/pmu-fixes: : . : Another set of PMU fixes, coutrtesy of Reiji Watanabe. : From the cover letter: : : "This series fixes a couple of PMUver related handling of : vPMU support. : : On systems where the PMUVer is not uniform across all PEs, : KVM currently does not advertise PMUv3 to the guest, : even if userspace successfully runs KVM_ARM_VCPU_INIT with : KVM_ARM_VCPU_PMU_V3." : : Additionally, a fix for an obscure counter oversubscription : issue happening when the hsot profines the guest's EL0. : . KVM: arm64: pmu: Guard PMU emulation definitions with CONFIG_KVM KVM: arm64: pmu: Resync EL0 state on counter rotation KVM: arm64: PMU: Don't advertise STALL_SLOT_{FRONTEND,BACKEND} KVM: arm64: PMU: Don't advertise the STALL_SLOT event KVM: arm64: PMU: Avoid inappropriate use of host's PMUVer KVM: arm64: PMU: Disallow vPMU on non-uniform PMUVer Signed-off-by:Marc Zyngier <maz@kernel.org>
-
Marc Zyngier authored
* kvm-arm64/tlbi-range: : . : FEAT_TLBIRANGE support, courtesy of Raghavendra Rao Ananta. : From the cover letter: : : "In certain code paths, KVM/ARM currently invalidates the entire VM's : page-tables instead of just invalidating a necessary range. For example, : when collapsing a table PTE to a block PTE, instead of iterating over : each PTE and flushing them, KVM uses 'vmalls12e1is' TLBI operation to : flush all the entries. This is inefficient since the guest would have : to refill the TLBs again, even for the addresses that aren't covered : by the table entry. The performance impact would scale poorly if many : addresses in the VM is going through this remapping. : : For architectures that implement FEAT_TLBIRANGE, KVM can replace such : inefficient paths by performing the invalidations only on the range of : addresses that are in scope. This series tries to achieve the same in : the areas of stage-2 map, unmap and write-protecting the pages." : . KVM: arm64: Use TLBI range-based instructions for unmap KVM: arm64: Invalidate the table entries upon a range KVM: arm64: Flush only the memslot after write-protect KVM: arm64: Implement kvm_arch_flush_remote_tlbs_range() KVM: arm64: Define kvm_tlb_flush_vmid_range() KVM: arm64: Implement __kvm_tlb_flush_vmid_range() arm64: tlb: Implement __flush_s2_tlb_range_op() arm64: tlb: Refactor the core flush algorithm of __flush_tlb_range KVM: Move kvm_arch_flush_remote_tlbs_memslot() to common code KVM: Allow range-based TLB invalidation from common code KVM: Remove CONFIG_HAVE_KVM_ARCH_TLB_FLUSH_ALL KVM: arm64: Use kvm_arch_flush_remote_tlbs() KVM: Declare kvm_arch_flush_remote_tlbs() globally KVM: Rename kvm_arch_flush_remote_tlb() to kvm_arch_flush_remote_tlbs() Signed-off-by: -
Marc Zyngier authored
* kvm-arm64/nv-trap-forwarding: (30 commits) : . : This implements the so called "trap forwarding" infrastructure, which : gets used when we take a trap from an L2 guest and that the L1 guest : wants to see the trap for itself. : . KVM: arm64: nv: Add trap description for SPSR_EL2 and ELR_EL2 KVM: arm64: nv: Select XARRAY_MULTI to fix build error KVM: arm64: nv: Add support for HCRX_EL2 KVM: arm64: Move HCRX_EL2 switch to load/put on VHE systems KVM: arm64: nv: Expose FGT to nested guests KVM: arm64: nv: Add switching support for HFGxTR/HDFGxTR KVM: arm64: nv: Expand ERET trap forwarding to handle FGT KVM: arm64: nv: Add SVC trap forwarding KVM: arm64: nv: Add trap forwarding for HDFGxTR_EL2 KVM: arm64: nv: Add trap forwarding for HFGITR_EL2 KVM: arm64: nv: Add trap forwarding for HFGxTR_EL2 KVM: arm64: nv: Add fine grained trap forwarding infrastructure KVM: arm64: nv: Add trap forwarding for CNTHCTL_EL2 KVM: arm64: nv: Add trap forwarding for MDCR_EL2 KVM: arm64: nv: Expose FEAT_EVT to nested guests KVM: arm64: nv: Add trap forwarding for HCR_EL2 KVM: arm64: nv: Add trap forwarding infrastructure KVM: arm64: Restructure FGT register switching KVM: arm64: nv: Add FGT registers KVM: arm64: Add missing HCR_EL2 trap bits ... Signed-off-by:
-
- 23 Aug, 2023 2 commits
-
-
Marc Zyngier authored
Having carved a hole for SP_EL1, we are now missing the entries for SPSR_EL2 and ELR_EL2. Add them back. Reported-by:
Miguel Luis <miguel.luis@oracle.com> Signed-off-by:
-
Marc Zyngier authored
Most of the internal definitions for PMU emulation are guarded with CONFIG_HW_PERF_EVENTS. However, this isn't enough, and leads to these definitions leaking if CONFIG_KVM isn't enabled. This leads to some compilation breakage in this exact configuration. Fix it by falling back to the dummy stubs if either perf or KVM isn't selected. Reported-by:
Alexander Stein <alexander.stein@ew.tq-group.com> Tested-by:
-
- 22 Aug, 2023 1 commit
-
-
Marc Zyngier authored
Huang Shijie reports that, when profiling a guest from the host with a number of events that exceeds the number of available counters, the reported counts are wildly inaccurate. Without the counter oversubscription, the reported counts are correct. Their investigation indicates that upon counter rotation (which takes place on the back of a timer interrupt), we fail to re-apply the guest EL0 enabling, leading to the counting of host events instead of guest events. In order to solve this, add yet another hook between the host PMU driver and KVM, re-applying the guest EL0 configuration if the right conditions apply (the host is VHE, we are in interrupt context, and we interrupted a running vcpu). This triggers a new vcpu request which will apply the correct configuration on guest reentry. With this, we have the correct counts, even when the counters are oversubscribed. Reported-by:
Huang Shijie <shijie@os.amperecomputing.com> Suggested-by:
Oliver Upton <oliver.upton@linux.dev> Tested_by: Huang Shijie <shijie@os.amperecomputing.com> Signed-off-by:
Mark Rutland <mark.rutland@arm.com> Link: https://lore.kernel.org/r/20230820090108.177817-1-maz@kernel.org
-
- 20 Aug, 2023 4 commits
-
-
Reiji Watanabe authored
Don't advertise STALL_SLOT_{FRONT,BACK}END events to the guest, similar to STALL_SLOT event, as when any of these three events are implemented, all three of them should be implemented, according to the Arm ARM. Suggested-by:Reiji Watanabe <reijiw@google.com> Signed-off-by:
-
Reiji Watanabe authored
Currently, KVM hides the STALL_SLOT event for guests if the host PMU version is PMUv3p4 or newer, as PMMIR_EL1 is handled as RAZ for the guests. But, this should be based on the guests' PMU version (instead of the host PMU version), as an older PMU that doesn't support PMMIR_EL1 could support the STALL_SLOT event, according to the Arm ARM. Exposing the STALL_SLOT event without PMMIR_EL1 won't be very useful anyway though. Stop advertising the STALL_SLOT event for guests unconditionally, rather than fixing or keeping the inaccurate checking to advertise the event for the case, where it is not very useful. Suggested-by:
-
Reiji Watanabe authored
Avoid using the PMUVer of the host's PMU hardware to determine the PMU event mask, except in one case, as the value of host's PMUVer may differ from the value of ID_AA64DFR0_EL1.PMUVer for the guest. The exception case is when using the PMUVer to determine the valid range of events for KVM_ARM_VCPU_PMU_V3_FILTER, as it has been allowing userspace to specify events that are valid for the PMU hardware, regardless of the value of the guest's ID_AA64DFR0_EL1.PMUVer. KVM will use a valid range of events based on the value of the guest's ID_AA64DFR0_EL1.PMUVer, in order to effectively filter events that the guest attempts to program though. Signed-off-by:
-
Reiji Watanabe authored
Disallow userspace from configuring vPMU for guests on systems where the PMUVer is not uniform across all PEs. KVM has not been advertising PMUv3 to the guests with vPMU on such systems anyway, and such systems would be extremely uncommon and unlikely to even use KVM. Signed-off-by:
-
- 17 Aug, 2023 30 commits
-
-
Randy Dunlap authored
populate_nv_trap_config() uses xa_store_range(), which is only built when XARRAY_MULTI is set, so select that symbol to prevent the build error. aarch64-linux-ld: arch/arm64/kvm/emulate-nested.o: in function `populate_nv_trap_config': emulate-nested.c:(.init.text+0x17c): undefined reference to `xa_store_range' Fixes: e58ec47b ("KVM: arm64: nv: Add trap forwarding infrastructure") Signed-off-by:
Randy Dunlap <rdunlap@infradead.org> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oliver.upton@linux.dev> Cc: kvmarm@lists.linux.dev Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: linux-arm-kernel@lists.infradead.org Signed-off-by:
-
Marc Zyngier authored
HCRX_EL2 has an interesting effect on HFGITR_EL2, as it conditions the traps of TLBI*nXS. Expand the FGT support to add a new Fine Grained Filter that will get checked when the instruction gets trapped, allowing the shadow register to override the trap as needed. Reviewed-by:
Eric Auger <eric.auger@redhat.com> Signed-off-by:
Jing Zhang <jingzhangos@google.com> Link: https://lore.kernel.org/r/20230815183903.2735724-29-maz@kernel.org
-
Marc Zyngier authored
Although the nVHE behaviour requires HCRX_EL2 to be switched on each switch between host and guest, there is nothing in this register that would affect a VHE host. It is thus possible to save/restore this register on load/put on VHE systems, avoiding unnecessary sysreg access on the hot path. Additionally, it avoids unnecessary traps when running with NV. To achieve this, simply move the read/writes to the *_common() helpers, which are called on load/put on VHE, and more eagerly on nVHE. Reviewed-by:
-
Marc Zyngier authored
Now that we have FGT support, expose the feature to NV guests. Reviewed-by:
-
Marc Zyngier authored
Now that we can evaluate the FGT registers, allow them to be merged with the hypervisor's own configuration (in the case of HFG{RW}TR_EL2) or simply set for HFGITR_EL2, HDGFRTR_EL2 and HDFGWTR_EL2. Reviewed-by: -
Marc Zyngier authored
We already handle ERET being trapped from a L1 guest in hyp context. However, with FGT, we can also have ERET being trapped from L2, and this needs to be reinjected into L1. Add the required exception routing. Signed-off-by:
-
Marc Zyngier authored
HFGITR_EL2 allows the trap of SVC instructions to EL2. Allow these traps to be forwarded. Take this opportunity to deny any 32bit activity when NV is enabled. Reviewed-by:
-
Marc Zyngier authored
... and finally, the Debug version of FGT, with its *enormous* list of trapped registers. Reviewed-by:
Suzuki K Poulose <suzuki.poulose@arm.com> Reviewed-by:
-
Marc Zyngier authored
Similarly, implement the trap forwarding for instructions affected by HFGITR_EL2. Note that the TLBI*nXS instructions should be affected by HCRX_EL2, which will be dealt with down the line. Also, ERET* and SVC traps are handled separately. Reviewed-by:
-
Marc Zyngier authored
Implement the trap forwarding for traps described by HFGxTR_EL2, reusing the Fine Grained Traps infrastructure previously implemented. Reviewed-by:
-
Marc Zyngier authored
Fine Grained Traps are fun. Not. Implement the fine grained trap forwarding, reusing the Coarse Grained Traps infrastructure previously implemented. Each sysreg/instruction inserted in the xarray gets a FGT group (vaguely equivalent to a register number), a bit number in that register, and a polarity. It is then pretty easy to check the FGT state at handling time, just like we do for the coarse version (it is just faster). Reviewed-by:
-
Marc Zyngier authored
Describe the CNTHCTL_EL2 register, and associate it with all the sysregs it allows to trap. Reviewed-by:
-
Marc Zyngier authored
Describe the MDCR_EL2 register, and associate it with all the sysregs it allows to trap. Reviewed-by:
-
Marc Zyngier authored
Now that we properly implement FEAT_EVT (as we correctly forward traps), expose it to guests. Reviewed-by:
-
Marc Zyngier authored
Describe the HCR_EL2 register, and associate it with all the sysregs it allows to trap. Reviewed-by:
-
Marc Zyngier authored
A significant part of what a NV hypervisor needs to do is to decide whether a trap from a L2+ guest has to be forwarded to a L1 guest or handled locally. This is done by checking for the trap bits that the guest hypervisor has set and acting accordingly, as described by the architecture. A previous approach was to sprinkle a bunch of checks in all the system register accessors, but this is pretty error prone and doesn't help getting an overview of what is happening. Instead, implement a set of global tables that describe a trap bit, combinations of trap bits, behaviours on trap, and what bits must be evaluated on a system register trap. Although this is painful to describe, this allows to specify each and every control bit in a static manner. To make it efficient, the table is inserted in an xarray that is global to the system, and checked each time we trap a system register while running a L2 guest. Add the basic infrastructure for now, while additional patches will implement configuration registers. Signed-off-by:
-
Marc Zyngier authored
As we're about to majorly extend the handling of FGT registers, restructure the code to actually save/restore the registers as required. This is made easy thanks to the previous addition of the EL2 registers, allowing us to use the host context for this purpose. Reviewed-by:
-
Marc Zyngier authored
Add the 5 registers covering FEAT_FGT. The AMU-related registers are currently left out as we don't have a plan for them. Yet. Reviewed-by:
-
Marc Zyngier authored
We're still missing a handfull of HCR_EL2 trap bits. Add them. Reviewed-by:
-
Marc Zyngier authored
As we blindly reset some HFGxTR_EL2 bits to 0, we also randomly trap unsuspecting sysregs that have their trap bits with a negative polarity. ACCDATA_EL1 is one such register that can be accessed by the guest, causing a splat on the host as we don't have a proper handler for it. Adding such handler addresses the issue, though there are a number of other registers missing as the current architecture documentation doesn't describe them yet. Reviewed-by:
-
Mark Brown authored
In order to allow us to have shared code for managing fine grained traps for KVM guests add it as a detected feature rather than relying on it being a dependency of other features. Acked-by:
Catalin Marinas <catalin.marinas@arm.com> Reviewed-by:
Mark Brown <broonie@kernel.org> [maz: converted to ARM64_CPUID_FIELDS()] Link: https://lore.kernel.org/r/20230301-kvm-arm64-fgt-v4-1-1bf8d235ac1f@kernel.orgReviewed-by:
Zenghui Yu <yuzenghui@huawei.com> Reviewed-by:
-
Marc Zyngier authored
As we're about to implement full support for FEAT_FGT, add the full HDFGRTR_EL2 and HDFGWTR_EL2 layouts. Reviewed-by:
-
Marc Zyngier authored
HFGITR_EL2 traps a bunch of instructions for which we don't have encodings yet. Add them. Reviewed-by:
-
Marc Zyngier authored
The HDFGxTR_EL2 registers trap a (huge) set of debug and trace related registers. Add their encodings (and only that, because we really don't care about what these registers actually do at this stage). Acked-by:
-
Marc Zyngier authored
Add the encodings for the AT operation that are usable from NS. Reviewed-by:
-
Marc Zyngier authored
Add all the TLBI encodings that are usable from Non-Secure. Reviewed-by:
-
Marc Zyngier authored
Add the missing DC *VA encodings. Reviewed-by:
-
Marc Zyngier authored
We only describe a few of the ERX*_EL1 registers. Add the missing ones (ERXPFGF_EL1, ERXPFGCTL_EL1, ERXPFGCDN_EL1, ERXMISC2_EL1 and ERXMISC3_EL1). Reviewed-by:
-
Marc Zyngier authored
Add the missing VA-based CMOs encodings. Reviewed-by:
-
Raghavendra Rao Ananta authored
The current implementation of the stage-2 unmap walker traverses the given range and, as a part of break-before-make, performs TLB invalidations with a DSB for every PTE. A multitude of this combination could cause a performance bottleneck on some systems. Hence, if the system supports FEAT_TLBIRANGE, defer the TLB invalidations until the entire walk is finished, and then use range-based instructions to invalidate the TLBs in one go. Condition deferred TLB invalidation on the system supporting FWB, as the optimization is entirely pointless when the unmap walker needs to perform CMOs. Rename stage2_put_pte() to stage2_unmap_put_pte() as the function now serves the stage-2 unmap walker specifically, rather than acting generic. Signed-off-by:
Raghavendra Rao Ananta <rananta@google.com> Reviewed-by:
Shaoqin Huang <shahuang@redhat.com> Signed-off-by:
-
