- 03 Dec, 2020 40 commits
-
-
Christophe Leroy authored
SPRN_VTB and CPU_FTR_ARCH_207S are always defined, no need of an ifdef. Signed-off-by:
Christophe Leroy <christophe.leroy@csgroup.eu> Signed-off-by:
Michael Ellerman <mpe@ellerman.id.au> Link: https://lore.kernel.org/r/a0fc81cd85121407726bcf480fc9a0d8e7617fce.1601549933.git.christophe.leroy@csgroup.eu
-
Christophe Leroy authored
Use SPRN_SPRG_SCRATCH2 as a third scratch register in exception prologs in order to simplify them and avoid data going back and forth from/to CR. Signed-off-by:
-
Christophe Leroy authored
Use SPRN_SPRG_SCRATCH2 as an alternative scratch register in the early part of DSI prolog in order to avoid clobbering SPRN_SPRG_SCRATCH0/1 used by other prologs. The 603 doesn't like a jump from DataLoadTLBMiss to the 10 nops that are now in the beginning of DSI exception as a result of the feature section. To workaround this, add a jump as alternative. It also avoids fetching 10 nops for nothing. Signed-off-by:
-
Christophe Leroy authored
Make code more readable with a clear CONFIG_VMAP_STACK section and a clear non CONFIG_VMAP_STACK section. Signed-off-by:
-
Christophe Leroy authored
On the 603, SDR1 is not used. In order to free SPRN_SPRG2, use SPRN_SDR1 to store the pgdir phys addr. But only some bits of SDR1 can be used (0xffff01ff). As the pgdir is 4k aligned, rotate it by 4 bits to the left. Signed-off-by:
-
Christophe Leroy authored
SPRN_SPRG_PGDIR is there mainly to speedup SW TLB miss handlers for powerpc 603. We need to free SPRN_SPRG2 to reduce the mess with CONFIG_VMAP_STACK. In hash_page(), reading PGDIR from thread_struct will be in the noise performance wise. Signed-off-by:
-
Christophe Leroy authored
An FTR_SECTION_ELSE is in the middle of BEGIN_MMU_FTR_SECTION/ALT_MMU_FTR_SECTION_END_IFSET Change it to MMU_FTR_SECTION_ELSE Signed-off-by:
-
Christophe Leroy authored
We now always map kernel text with BATs. Neither need to preload hash with kernel text addresses nor ensure they are never evicted. This is more or less a revert of commit ee4f2ea4 ("[POWERPC] Fix 32-bit mm operations when not using BATs") Signed-off-by:
-
Christophe Leroy authored
Since commit 2b279c03 ("powerpc/32s: Allow mapping with BATs with DEBUG_PAGEALLOC"), there is no real situation where mapping without BATs is required. In order to simplify memory handling, always map kernel text and rodata with BATs even when "nobats" kernel parameter is set. Also fix the 603 TLB miss exceptions that don't require anymore kernel page table if DEBUG_PAGEALLOC. Signed-off-by:
-
Christophe Lombard authored
Add specific kernel traces which provide information on mmu notifier and on pages range. Signed-off-by:
Christophe Lombard <clombard@linux.vnet.ibm.com> Acked-by:
Frederic Barrat <fbarrat@linux.ibm.com> Signed-off-by:
-
Christophe Lombard authored
Add invalidate_range mmu notifier, when required (ATSD access of MMIO registers is available), to initiate TLB invalidation commands. For the time being, the ATSD0 set of registers is used by default. The pasid and bdf values have to be configured in the Process Element Entry. The PEE must be set up to match the BDF/PASID of the AFU. Signed-off-by:
-
Christophe Lombard authored
To complete the MMIO based mechanism, the fields: PASID, bus, device and function of the Process Element Entry have to be filled. (See OpenCAPI Power Platform Architecture document) Hypervisor Process Element Entry Word 0 1 .... 7 8 ...... 12 13 ..15 16.... 19 20 ........... 31 0 OSL Configuration State (0:31) 1 OSL Configuration State (32:63) 2 PASID | Reserved 3 Bus | Device |Function | Reserved 4 Reserved 5 Reserved 6 .... Signed-off-by: -
Christophe Lombard authored
When a TLB Invalidate is required for the Logical Partition, the following sequence has to be performed: 1. Load MMIO ATSD AVA register with the necessary value, if required. 2. Write the MMIO ATSD launch register to initiate the TLB Invalidate command. 3. Poll the MMIO ATSD status register to determine when the TLB Invalidate has been completed. Signed-off-by:
-
Christophe Lombard authored
Platform specific function to assign a register set to a Logical Partition. The "ibm,mmio-atsd" property, provided by the firmware, contains the 16 base ATSD physical addresses (ATSD0 through ATSD15) of the set of MMIO registers (XTS MMIO ATSDx LPARID/AVA/launch/status register). For the time being, the ATSD0 set of registers is used by default. Signed-off-by:
-
Athira Rajeev authored
PowerISA v3.1 introduces new control bit (PMCCEXT) for restricting access to group B PMU registers in problem state when MMCR0 PMCC=0b00. In problem state and when MMCR0 PMCC=0b00, setting the Monitor Mode Control Register bit 54 (MMCR0 PMCCEXT), will restrict read permission on Group B Performance Monitor Registers (SIER, SIAR, SDAR and MMCR1). When this bit is set to zero, group B registers will be readable. In other platforms (like power9), the older behaviour is retained where group B PMU SPRs are readable. Patch adds support for MMCR0 PMCCEXT bit in power10 by enabling this bit during boot and during the PMU event enable/disable callback functions. Signed-off-by:
Athira Rajeev <atrajeev@linux.vnet.ibm.com> Signed-off-by:
-
Athira Rajeev authored
Export l2l3 events (PM_L2_ST_MISS and PM_L2_ST) and LLC-prefetches (PM_L3_PF_MISS_L3) via sysfs, and also add these to list of cache_events. Signed-off-by:
-
Athira Rajeev authored
Fix the event code for events: branch-instructions (to PM_BR_FIN), branch-misses (to PM_MPRED_BR_FIN) and cache-misses (to PM_LD_DEMAND_MISS_L1_FIN) for power10 PMU. Update the list of generic events with this modified event code. Signed-off-by:
-
Athira Rajeev authored
There are event code updates for some of the generic events and cache events for power10. Inorder to maintain the current event codes work with DD1 also, create a new array of generic_events, cache_events and pmu_attr_groups with suffix _dd1, example, power10_events_attr_dd1. So that further updates to event codes can be made in the original list, ie, power10_events_attr. Update the power10 pmu init code to pick the dd1 list while registering the power PMU, based on the pvr (Processor Version Register) value. Signed-off-by:
-
Athira Rajeev authored
The PMU group constraints mask for threshold events covers all thresholding bits which includes threshold control value (start/stop), select value as well as thresh_cmp value (MMCRA[9:18]. In power9, thresh_cmp bits were part of the event code. But in case of power10, thresh_cmp bits are not part of event code due to inclusion of MMCR3 bits. Hence thresh_cmp is not valid for group constraints for power10. Fix the PMU group constraints checking for threshold events in power10 by using constraint mask and value for only threshold control and select bits. Fixes: a64e697c ("powerpc/perf: power10 Performance Monitoring support") Signed-off-by:
-
Athira Rajeev authored
In Power9, L2/L3 bus events are always available as a "bank" of 4 events. To obtain the counts for any of the l2/l3 bus events in a given bank, the user will have to program PMC4 with corresponding l2/l3 bus event for that bank. Commit 59029136 ("powerpc/perf: Add constraints for power9 l2/l3 bus events") enforced this rule in Power9. But this is not valid for Power10, since in Power10 Monitor Mode Control Register2 (MMCR2) has bits to configure l2/l3 event bits. Hence remove this PMC4 constraint check from power10. Since the l2/l3 bits in MMCR2 are not per-pmc, patch handles group constrints checks for l2/l3 bits in MMCR2. Fixes: a64e697c ("powerpc/perf: power10 Performance Monitoring support") Signed-off-by:
-
Athira Rajeev authored
power10 uses bit 9 of the raw event code as RADIX_SCOPE_QUAL. This bit is used for enabling the radix process events. Patch fixes the PMU counter support functions to program bit 18 of MMCR1 ( Monitor Mode Control Register1 ) with the RADIX_SCOPE_QUAL bit value. Since this field is not per-pmc, add this to PMU group constraints to make sure events in a group will have same bit value for this field. Use bit 21 as constraint bit field for radix_scope_qual. Patch also updates the power10 raw event encoding layout information, format field and constraints bit layout to include the radix_scope_qual bit. Fixes: a64e697c ("powerpc/perf: power10 Performance Monitoring support") Signed-off-by:
-
Aneesh Kumar K.V authored
If FTR_BOOK3S_KUAP is disabled, kernel will continue to run with the same AMR value with which it was entered. Hence there is a high chance that we can return without restoring the AMR value. This also helps the case when applications are not using the pkey feature. In this case, different applications will have the same AMR values and hence we can avoid restoring AMR in this case too. Also avoid isync() if not really needed. Do the same for IAMR. null-syscall benchmark results: With smap/smep disabled: Without patch: 957.95 ns 2778.17 cycles With patch: 858.38 ns 2489.30 cycles With smap/smep enabled: Without patch: 1017.26 ns 2950.36 cycles With patch: 1021.51 ns 2962.44 cycles Signed-off-by:
Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com> Signed-off-by:
-
Aneesh Kumar K.V authored
Don't enable KUEP/KUAP if we support less than or equal to 3 keys. Signed-off-by:
-
Aneesh Kumar K.V authored
Signed-off-by:
Sandipan Das <sandipan@linux.ibm.com> Signed-off-by:
-
Aneesh Kumar K.V authored
Signed-off-by:
-
Aneesh Kumar K.V authored
Radix use IAMR Key 0 and hash translation use IAMR key 3. Signed-off-by:
-
Aneesh Kumar K.V authored
Radix use AMR Key 0 and hash translation use AMR key 3. Signed-off-by:
-
Aneesh Kumar K.V authored
With hash translation use DSISR_KEYFAULT to identify a wrong access. With Radix we look at the AMR value and type of fault. Signed-off-by:
-
Aneesh Kumar K.V authored
If an application has configured address protection such that read/write is denied using pkey even the kernel should receive a FAULT on accessing the same. This patch use user AMR value stored in pt_regs.amr to achieve the same. Signed-off-by:
-
Aneesh Kumar K.V authored
Now that kernel correctly store/restore userspace AMR/IAMR values, avoid manipulating AMR and IAMR from the kernel on behalf of userspace. Signed-off-by:
-
Aneesh Kumar K.V authored
We will remove thread.amr/iamr/uamor in a later patch Signed-off-by:
-
Aneesh Kumar K.V authored
On fork, we inherit from the parent and on exec, we should switch to default_amr values. Also, avoid changing the AMR register value within the kernel. The kernel now runs with different AMR values. Signed-off-by:
-
Aneesh Kumar K.V authored
Child thread.kuap value is inherited from the parent in copy_thread_tls. We still need to make sure when the child returns from a fork in the kernel we start with the kernel default AMR value. Signed-off-by:
-
Aneesh Kumar K.V authored
This prepare kernel to operate with a different value than userspace AMR/IAMR. For this, AMR/IAMR need to be saved and restored on entry and return from the kernel. With KUAP we modify kernel AMR when accessing user address from the kernel via copy_to/from_user interfaces. We don't need to modify IAMR value in similar fashion. If MMU_FTR_PKEY is enabled we need to save AMR/IAMR in pt_regs on entering kernel from userspace. If not we can assume that AMR/IAMR is not modified from userspace. We need to save AMR if we have MMU_FTR_BOOK3S_KUAP feature enabled and we are interrupted within kernel. This is required so that if we get interrupted within copy_to/from_user we continue with the right AMR value. If we hae MMU_FTR_BOOK3S_KUEP enabled we need to restore IAMR on return to userspace beause kernel will be running with a different IAMR value. Signed-off-by:
-
Aneesh Kumar K.V authored
In later patches during exec, we would like to access default regs.amr to control access to the user mapping. Having thread.regs set early makes the code changes simpler. Signed-off-by:
-
Aneesh Kumar K.V authored
This patch updates kernel hash page table entries to use storage key 3 for its mapping. This implies all kernel access will now use key 3 to control READ/WRITE. The patch also prevents the allocation of key 3 from userspace and UAMOR value is updated such that userspace cannot modify key 3. Signed-off-by:
-
Aneesh Kumar K.V authored
This is in preparation to adding support for kuap with hash translation. In preparation for that rename/move kuap related functions to non radix names. Also move the feature bit closer to MMU_FTR_KUEP. MMU_FTR_KUEP is renamed to MMU_FTR_BOOK3S_KUEP to indicate the feature is only relevant to BOOK3S_64 Signed-off-by:
-
Aneesh Kumar K.V authored
The next set of patches adds support for kuep with hash translation. In preparation for that rename/move kuap related functions to non radix names. Also set MMU_FTR_KUEP and add the missing isync(). Signed-off-by:
-
Aneesh Kumar K.V authored
The next set of patches adds support for kuap with hash translation. In preparation for that rename/move kuap related functions to non radix names. Signed-off-by:
-
Aneesh Kumar K.V authored
This patch consolidates UAMOR update across pkey, kuap and kuep features. The boot cpu initialize UAMOR via pkey init and both radix/hash do the secondary cpu UAMOR init in early_init_mmu_secondary. We don't check for mmu_feature in radix secondary init because UAMOR is a supported SPRN with all CPUs supporting radix translation. The old code was not updating UAMOR if we had smap disabled and smep enabled. This change handles that case. Signed-off-by:
-
