Commit 103916ff authored by Linus Torvalds's avatar Linus Torvalds

Merge tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux

Pull arm64 updates from Will Deacon:
 "The most interesting parts are probably the mm changes from Ryan which
  optimise the creation of the linear mapping at boot and (separately)
  implement write-protect support for userfaultfd.

  Outside of our usual directories, the Kbuild-related changes under
  scripts/ have been acked by Masahiro whilst the drivers/acpi/ parts
  have been acked by Rafael and the addition of cpumask_any_and_but()
  has been acked by Yury.

  ACPI:

   - Support for the Firmware ACPI Control Structure (FACS) signature
     feature which is used to reboot out of hibernation on some systems

  Kbuild:

   - Support for building Flat Image Tree (FIT) images, where the kernel
     Image is compressed alongside a set of devicetree blobs

  Memory management:

   - Optimisation of our early page-table manipulation for creation of
     the linear mapping

   - Support for userfaultfd write protection, which brings along some
     nice cleanups to our handling of invalid but present ptes

   - Extend our use of range TLBI invalidation at EL1

  Perf and PMUs:

   - Ensure that the 'pmu->parent' pointer is correctly initialised by
     PMU drivers

   - Avoid allocating 'cpumask_t' types on the stack in some PMU drivers

   - Fix parsing of the CPU PMU "version" field in assembly code, as it
     doesn't follow the usual architectural rules

   - Add best-effort unwinding support for USER_STACKTRACE

   - Minor driver fixes and cleanups

  Selftests:

   - Minor cleanups to the arm64 selftests (missing NULL check, unused
     variable)

  Miscellaneous:

   - Add a command-line alias for disabling 32-bit application support

   - Add part number for Neoverse-V2 CPUs

   - Minor fixes and cleanups"

* tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (64 commits)
  arm64/mm: Fix pud_user_accessible_page() for PGTABLE_LEVELS <= 2
  arm64/mm: Add uffd write-protect support
  arm64/mm: Move PTE_PRESENT_INVALID to overlay PTE_NG
  arm64/mm: Remove PTE_PROT_NONE bit
  arm64/mm: generalize PMD_PRESENT_INVALID for all levels
  arm64: simplify arch_static_branch/_jump function
  arm64: Add USER_STACKTRACE support
  arm64: Add the arm64.no32bit_el0 command line option
  drivers/perf: hisi: hns3: Actually use devm_add_action_or_reset()
  drivers/perf: hisi: hns3: Fix out-of-bound access when valid event group
  drivers/perf: hisi_pcie: Fix out-of-bound access when valid event group
  kselftest: arm64: Add a null pointer check
  arm64: defer clearing DAIF.D
  arm64: assembler: update stale comment for disable_step_tsk
  arm64/sysreg: Update PIE permission encodings
  kselftest/arm64: Remove unused parameters in abi test
  perf/arm-spe: Assign parents for event_source device
  perf/arm-smmuv3: Assign parents for event_source device
  perf/arm-dsu: Assign parents for event_source device
  perf/arm-dmc620: Assign parents for event_source device
  ...
parents 1338acfe 54e1a2aa
...@@ -431,6 +431,9 @@ ...@@ -431,6 +431,9 @@
arcrimi= [HW,NET] ARCnet - "RIM I" (entirely mem-mapped) cards arcrimi= [HW,NET] ARCnet - "RIM I" (entirely mem-mapped) cards
Format: <io>,<irq>,<nodeID> Format: <io>,<irq>,<nodeID>
arm64.no32bit_el0 [ARM64] Unconditionally disable the execution of
32 bit applications.
arm64.nobti [ARM64] Unconditionally disable Branch Target arm64.nobti [ARM64] Unconditionally disable Branch Target
Identification support Identification support
......
...@@ -20,7 +20,6 @@ interrupt, and the PMU driver shall register perf PMU drivers like L3C, ...@@ -20,7 +20,6 @@ interrupt, and the PMU driver shall register perf PMU drivers like L3C,
HHA and DDRC etc. The available events and configuration options shall HHA and DDRC etc. The available events and configuration options shall
be described in the sysfs, see: be described in the sysfs, see:
/sys/devices/hisi_sccl{X}_<l3c{Y}/hha{Y}/ddrc{Y}>/, or
/sys/bus/event_source/devices/hisi_sccl{X}_<l3c{Y}/hha{Y}/ddrc{Y}>. /sys/bus/event_source/devices/hisi_sccl{X}_<l3c{Y}/hha{Y}/ddrc{Y}>.
The "perf list" command shall list the available events from sysfs. The "perf list" command shall list the available events from sysfs.
......
...@@ -16,7 +16,7 @@ HNS3 PMU driver ...@@ -16,7 +16,7 @@ HNS3 PMU driver
The HNS3 PMU driver registers a perf PMU with the name of its sicl id.:: The HNS3 PMU driver registers a perf PMU with the name of its sicl id.::
/sys/devices/hns3_pmu_sicl_<sicl_id> /sys/bus/event_source/devices/hns3_pmu_sicl_<sicl_id>
PMU driver provides description of available events, filter modes, format, PMU driver provides description of available events, filter modes, format,
identifier and cpumask in sysfs. identifier and cpumask in sysfs.
...@@ -40,9 +40,9 @@ device. ...@@ -40,9 +40,9 @@ device.
Example usage of checking event code and subevent code:: Example usage of checking event code and subevent code::
$# cat /sys/devices/hns3_pmu_sicl_0/events/dly_tx_normal_to_mac_time $# cat /sys/bus/event_source/devices/hns3_pmu_sicl_0/events/dly_tx_normal_to_mac_time
config=0x00204 config=0x00204
$# cat /sys/devices/hns3_pmu_sicl_0/events/dly_tx_normal_to_mac_packet_num $# cat /sys/bus/event_source/devices/hns3_pmu_sicl_0/events/dly_tx_normal_to_mac_packet_num
config=0x10204 config=0x10204
Each performance statistic has a pair of events to get two values to Each performance statistic has a pair of events to get two values to
...@@ -60,7 +60,7 @@ computation to calculate real performance data is::: ...@@ -60,7 +60,7 @@ computation to calculate real performance data is:::
Example usage of checking supported filter mode:: Example usage of checking supported filter mode::
$# cat /sys/devices/hns3_pmu_sicl_0/filtermode/bw_ssu_rpu_byte_num $# cat /sys/bus/event_source/devices/hns3_pmu_sicl_0/filtermode/bw_ssu_rpu_byte_num
filter mode supported: global/port/port-tc/func/func-queue/ filter mode supported: global/port/port-tc/func/func-queue/
Example usage of perf:: Example usage of perf::
......
...@@ -10,7 +10,7 @@ There is one logical L2 PMU exposed, which aggregates the results from ...@@ -10,7 +10,7 @@ There is one logical L2 PMU exposed, which aggregates the results from
the physical PMUs. the physical PMUs.
The driver provides a description of its available events and configuration The driver provides a description of its available events and configuration
options in sysfs, see /sys/devices/l2cache_0. options in sysfs, see /sys/bus/event_source/devices/l2cache_0.
The "format" directory describes the format of the events. The "format" directory describes the format of the events.
......
...@@ -9,7 +9,7 @@ PMU with device name l3cache_<socket>_<instance>. User space is responsible ...@@ -9,7 +9,7 @@ PMU with device name l3cache_<socket>_<instance>. User space is responsible
for aggregating across slices. for aggregating across slices.
The driver provides a description of its available events and configuration The driver provides a description of its available events and configuration
options in sysfs, see /sys/devices/l3cache*. Given that these are uncore PMUs options in sysfs, see /sys/bus/event_source/devices/l3cache*. Given that these are uncore PMUs
the driver also exposes a "cpumask" sysfs attribute which contains a mask the driver also exposes a "cpumask" sysfs attribute which contains a mask
consisting of one CPU per socket which will be used to handle all the PMU consisting of one CPU per socket which will be used to handle all the PMU
events on that socket. events on that socket.
......
...@@ -22,7 +22,7 @@ The thunderx2_pmu driver registers per-socket perf PMUs for the DMC and ...@@ -22,7 +22,7 @@ The thunderx2_pmu driver registers per-socket perf PMUs for the DMC and
L3C devices. Each PMU can be used to count up to 4 (DMC/L3C) or up to 8 L3C devices. Each PMU can be used to count up to 4 (DMC/L3C) or up to 8
(CCPI2) events simultaneously. The PMUs provide a description of their (CCPI2) events simultaneously. The PMUs provide a description of their
available events and configuration options under sysfs, see available events and configuration options under sysfs, see
/sys/devices/uncore_<l3c_S/dmc_S/ccpi2_S/>; S is the socket id. /sys/bus/event_source/devices/uncore_<l3c_S/dmc_S/ccpi2_S/>; S is the socket id.
The driver does not support sampling, therefore "perf record" will not The driver does not support sampling, therefore "perf record" will not
work. Per-task perf sessions are also not supported. work. Per-task perf sessions are also not supported.
......
...@@ -13,7 +13,7 @@ PMU (perf) driver ...@@ -13,7 +13,7 @@ PMU (perf) driver
The xgene-pmu driver registers several perf PMU drivers. Each of the perf The xgene-pmu driver registers several perf PMU drivers. Each of the perf
driver provides description of its available events and configuration options driver provides description of its available events and configuration options
in sysfs, see /sys/devices/<l3cX/iobX/mcbX/mcX>/. in sysfs, see /sys/bus/event_source/devices/<l3cX/iobX/mcbX/mcX>/.
The "format" directory describes format of the config (event ID), The "format" directory describes format of the config (event ID),
config1 (agent ID) fields of the perf_event_attr structure. The "events" config1 (agent ID) fields of the perf_event_attr structure. The "events"
......
...@@ -62,6 +62,7 @@ Sphinx\ [#f1]_ 2.4.4 sphinx-build --version ...@@ -62,6 +62,7 @@ Sphinx\ [#f1]_ 2.4.4 sphinx-build --version
cpio any cpio --version cpio any cpio --version
GNU tar 1.28 tar --version GNU tar 1.28 tar --version
gtags (optional) 6.6.5 gtags --version gtags (optional) 6.6.5 gtags --version
mkimage (optional) 2017.01 mkimage --version
====================== =============== ======================================== ====================== =============== ========================================
.. [#f1] Sphinx is needed only to build the Kernel documentation .. [#f1] Sphinx is needed only to build the Kernel documentation
...@@ -189,6 +190,14 @@ The kernel build requires GNU GLOBAL version 6.6.5 or later to generate ...@@ -189,6 +190,14 @@ The kernel build requires GNU GLOBAL version 6.6.5 or later to generate
tag files through ``make gtags``. This is due to its use of the gtags tag files through ``make gtags``. This is due to its use of the gtags
``-C (--directory)`` flag. ``-C (--directory)`` flag.
mkimage
-------
This tool is used when building a Flat Image Tree (FIT), commonly used on ARM
platforms. The tool is available via the ``u-boot-tools`` package or can be
built from the U-Boot source code. See the instructions at
https://docs.u-boot.org/en/latest/build/tools.html#building-tools-for-linux
System utilities System utilities
**************** ****************
......
...@@ -3073,6 +3073,13 @@ F: drivers/mmc/host/sdhci-of-arasan.c ...@@ -3073,6 +3073,13 @@ F: drivers/mmc/host/sdhci-of-arasan.c
N: zynq N: zynq
N: xilinx N: xilinx
ARM64 FIT SUPPORT
M: Simon Glass <sjg@chromium.org>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
F: arch/arm64/boot/Makefile
F: scripts/make_fit.py
ARM64 PORT (AARCH64 ARCHITECTURE) ARM64 PORT (AARCH64 ARCHITECTURE)
M: Catalin Marinas <catalin.marinas@arm.com> M: Catalin Marinas <catalin.marinas@arm.com>
M: Will Deacon <will@kernel.org> M: Will Deacon <will@kernel.org>
......
...@@ -255,9 +255,11 @@ config ARM64 ...@@ -255,9 +255,11 @@ config ARM64
select SYSCTL_EXCEPTION_TRACE select SYSCTL_EXCEPTION_TRACE
select THREAD_INFO_IN_TASK select THREAD_INFO_IN_TASK
select HAVE_ARCH_USERFAULTFD_MINOR if USERFAULTFD select HAVE_ARCH_USERFAULTFD_MINOR if USERFAULTFD
select HAVE_ARCH_USERFAULTFD_WP if USERFAULTFD
select TRACE_IRQFLAGS_SUPPORT select TRACE_IRQFLAGS_SUPPORT
select TRACE_IRQFLAGS_NMI_SUPPORT select TRACE_IRQFLAGS_NMI_SUPPORT
select HAVE_SOFTIRQ_ON_OWN_STACK select HAVE_SOFTIRQ_ON_OWN_STACK
select USER_STACKTRACE_SUPPORT
help help
ARM 64-bit (AArch64) Linux support. ARM 64-bit (AArch64) Linux support.
......
...@@ -154,6 +154,10 @@ libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a ...@@ -154,6 +154,10 @@ libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
# Default target when executing plain make # Default target when executing plain make
boot := arch/arm64/boot boot := arch/arm64/boot
BOOT_TARGETS := Image vmlinuz.efi image.fit
PHONY += $(BOOT_TARGETS)
ifeq ($(CONFIG_EFI_ZBOOT),) ifeq ($(CONFIG_EFI_ZBOOT),)
KBUILD_IMAGE := $(boot)/Image.gz KBUILD_IMAGE := $(boot)/Image.gz
else else
...@@ -162,8 +166,10 @@ endif ...@@ -162,8 +166,10 @@ endif
all: $(notdir $(KBUILD_IMAGE)) all: $(notdir $(KBUILD_IMAGE))
vmlinuz.efi: Image image.fit: dtbs
Image vmlinuz.efi: vmlinux
vmlinuz.efi image.fit: Image
$(BOOT_TARGETS): vmlinux
$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@ $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
Image.%: Image Image.%: Image
...@@ -215,6 +221,7 @@ virtconfig: ...@@ -215,6 +221,7 @@ virtconfig:
define archhelp define archhelp
echo '* Image.gz - Compressed kernel image (arch/$(ARCH)/boot/Image.gz)' echo '* Image.gz - Compressed kernel image (arch/$(ARCH)/boot/Image.gz)'
echo ' Image - Uncompressed kernel image (arch/$(ARCH)/boot/Image)' echo ' Image - Uncompressed kernel image (arch/$(ARCH)/boot/Image)'
echo ' image.fit - Flat Image Tree (arch/$(ARCH)/boot/image.fit)'
echo ' install - Install uncompressed kernel' echo ' install - Install uncompressed kernel'
echo ' zinstall - Install compressed kernel' echo ' zinstall - Install compressed kernel'
echo ' Install using (your) ~/bin/installkernel or' echo ' Install using (your) ~/bin/installkernel or'
......
...@@ -2,3 +2,4 @@ ...@@ -2,3 +2,4 @@
Image Image
Image.gz Image.gz
vmlinuz* vmlinuz*
image.fit
...@@ -16,7 +16,8 @@ ...@@ -16,7 +16,8 @@
OBJCOPYFLAGS_Image :=-O binary -R .note -R .note.gnu.build-id -R .comment -S OBJCOPYFLAGS_Image :=-O binary -R .note -R .note.gnu.build-id -R .comment -S
targets := Image Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo Image.zst targets := Image Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo \
Image.zst image.fit
$(obj)/Image: vmlinux FORCE $(obj)/Image: vmlinux FORCE
$(call if_changed,objcopy) $(call if_changed,objcopy)
...@@ -39,6 +40,9 @@ $(obj)/Image.lzo: $(obj)/Image FORCE ...@@ -39,6 +40,9 @@ $(obj)/Image.lzo: $(obj)/Image FORCE
$(obj)/Image.zst: $(obj)/Image FORCE $(obj)/Image.zst: $(obj)/Image FORCE
$(call if_changed,zstd) $(call if_changed,zstd)
$(obj)/image.fit: $(obj)/Image $(obj)/dts/dtbs-list FORCE
$(call if_changed,fit)
EFI_ZBOOT_PAYLOAD := Image EFI_ZBOOT_PAYLOAD := Image
EFI_ZBOOT_BFD_TARGET := elf64-littleaarch64 EFI_ZBOOT_BFD_TARGET := elf64-littleaarch64
EFI_ZBOOT_MACH_TYPE := ARM64 EFI_ZBOOT_MACH_TYPE := ARM64
......
...@@ -50,16 +50,12 @@ ...@@ -50,16 +50,12 @@
msr daif, \flags msr daif, \flags
.endm .endm
.macro enable_dbg
msr daifclr, #8
.endm
.macro disable_step_tsk, flgs, tmp .macro disable_step_tsk, flgs, tmp
tbz \flgs, #TIF_SINGLESTEP, 9990f tbz \flgs, #TIF_SINGLESTEP, 9990f
mrs \tmp, mdscr_el1 mrs \tmp, mdscr_el1
bic \tmp, \tmp, #DBG_MDSCR_SS bic \tmp, \tmp, #DBG_MDSCR_SS
msr mdscr_el1, \tmp msr mdscr_el1, \tmp
isb // Synchronise with enable_dbg isb // Take effect before a subsequent clear of DAIF.D
9990: 9990:
.endm .endm
...@@ -480,9 +476,10 @@ alternative_endif ...@@ -480,9 +476,10 @@ alternative_endif
*/ */
.macro reset_pmuserenr_el0, tmpreg .macro reset_pmuserenr_el0, tmpreg
mrs \tmpreg, id_aa64dfr0_el1 mrs \tmpreg, id_aa64dfr0_el1
sbfx \tmpreg, \tmpreg, #ID_AA64DFR0_EL1_PMUVer_SHIFT, #4 ubfx \tmpreg, \tmpreg, #ID_AA64DFR0_EL1_PMUVer_SHIFT, #4
cmp \tmpreg, #1 // Skip if no PMU present cmp \tmpreg, #ID_AA64DFR0_EL1_PMUVer_NI
b.lt 9000f ccmp \tmpreg, #ID_AA64DFR0_EL1_PMUVer_IMP_DEF, #4, ne
b.eq 9000f // Skip if no PMU present or IMP_DEF
msr pmuserenr_el0, xzr // Disable PMU access from EL0 msr pmuserenr_el0, xzr // Disable PMU access from EL0
9000: 9000:
.endm .endm
......
...@@ -86,6 +86,7 @@ ...@@ -86,6 +86,7 @@
#define ARM_CPU_PART_CORTEX_X2 0xD48 #define ARM_CPU_PART_CORTEX_X2 0xD48
#define ARM_CPU_PART_NEOVERSE_N2 0xD49 #define ARM_CPU_PART_NEOVERSE_N2 0xD49
#define ARM_CPU_PART_CORTEX_A78C 0xD4B #define ARM_CPU_PART_CORTEX_A78C 0xD4B
#define ARM_CPU_PART_NEOVERSE_V2 0xD4F
#define APM_CPU_PART_XGENE 0x000 #define APM_CPU_PART_XGENE 0x000
#define APM_CPU_VAR_POTENZA 0x00 #define APM_CPU_VAR_POTENZA 0x00
...@@ -159,6 +160,7 @@ ...@@ -159,6 +160,7 @@
#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2) #define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2)
#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2) #define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C) #define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
#define MIDR_NEOVERSE_V2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V2)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX) #define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX) #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX) #define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
......
...@@ -59,13 +59,14 @@ ...@@ -59,13 +59,14 @@
.macro __init_el2_debug .macro __init_el2_debug
mrs x1, id_aa64dfr0_el1 mrs x1, id_aa64dfr0_el1
sbfx x0, x1, #ID_AA64DFR0_EL1_PMUVer_SHIFT, #4 ubfx x0, x1, #ID_AA64DFR0_EL1_PMUVer_SHIFT, #4
cmp x0, #1 cmp x0, #ID_AA64DFR0_EL1_PMUVer_NI
b.lt .Lskip_pmu_\@ // Skip if no PMU present ccmp x0, #ID_AA64DFR0_EL1_PMUVer_IMP_DEF, #4, ne
b.eq .Lskip_pmu_\@ // Skip if no PMU present or IMP_DEF
mrs x0, pmcr_el0 // Disable debug access traps mrs x0, pmcr_el0 // Disable debug access traps
ubfx x0, x0, #11, #5 // to EL2 and allow access to ubfx x0, x0, #11, #5 // to EL2 and allow access to
.Lskip_pmu_\@: .Lskip_pmu_\@:
csel x2, xzr, x0, lt // all PMU counters from EL1 csel x2, xzr, x0, eq // all PMU counters from EL1
/* Statistical profiling */ /* Statistical profiling */
ubfx x0, x1, #ID_AA64DFR0_EL1_PMSVer_SHIFT, #4 ubfx x0, x1, #ID_AA64DFR0_EL1_PMSVer_SHIFT, #4
......
...@@ -5,7 +5,6 @@ ...@@ -5,7 +5,6 @@
#ifndef __ASM_IRQFLAGS_H #ifndef __ASM_IRQFLAGS_H
#define __ASM_IRQFLAGS_H #define __ASM_IRQFLAGS_H
#include <asm/alternative.h>
#include <asm/barrier.h> #include <asm/barrier.h>
#include <asm/ptrace.h> #include <asm/ptrace.h>
#include <asm/sysreg.h> #include <asm/sysreg.h>
......
...@@ -15,17 +15,23 @@ ...@@ -15,17 +15,23 @@
#define JUMP_LABEL_NOP_SIZE AARCH64_INSN_SIZE #define JUMP_LABEL_NOP_SIZE AARCH64_INSN_SIZE
#define JUMP_TABLE_ENTRY(key, label) \
".pushsection __jump_table, \"aw\"\n\t" \
".align 3\n\t" \
".long 1b - ., %l["#label"] - .\n\t" \
".quad %c0 - .\n\t" \
".popsection\n\t" \
: : "i"(key) : : label
static __always_inline bool arch_static_branch(struct static_key * const key, static __always_inline bool arch_static_branch(struct static_key * const key,
const bool branch) const bool branch)
{ {
char *k = &((char *)key)[branch];
asm goto( asm goto(
"1: nop \n\t" "1: nop \n\t"
" .pushsection __jump_table, \"aw\" \n\t" JUMP_TABLE_ENTRY(k, l_yes)
" .align 3 \n\t" );
" .long 1b - ., %l[l_yes] - . \n\t"
" .quad %c0 - . \n\t"
" .popsection \n\t"
: : "i"(&((char *)key)[branch]) : : l_yes);
return false; return false;
l_yes: l_yes:
...@@ -35,15 +41,11 @@ static __always_inline bool arch_static_branch(struct static_key * const key, ...@@ -35,15 +41,11 @@ static __always_inline bool arch_static_branch(struct static_key * const key,
static __always_inline bool arch_static_branch_jump(struct static_key * const key, static __always_inline bool arch_static_branch_jump(struct static_key * const key,
const bool branch) const bool branch)
{ {
char *k = &((char *)key)[branch];
asm goto( asm goto(
"1: b %l[l_yes] \n\t" "1: b %l[l_yes] \n\t"
" .pushsection __jump_table, \"aw\" \n\t" JUMP_TABLE_ENTRY(k, l_yes)
" .align 3 \n\t" );
" .long 1b - ., %l[l_yes] - . \n\t"
" .quad %c0 - . \n\t"
" .popsection \n\t"
: : "i"(&((char *)key)[branch]) : : l_yes);
return false; return false;
l_yes: l_yes:
return true; return true;
......
...@@ -18,14 +18,21 @@ ...@@ -18,14 +18,21 @@
#define PTE_DIRTY (_AT(pteval_t, 1) << 55) #define PTE_DIRTY (_AT(pteval_t, 1) << 55)
#define PTE_SPECIAL (_AT(pteval_t, 1) << 56) #define PTE_SPECIAL (_AT(pteval_t, 1) << 56)
#define PTE_DEVMAP (_AT(pteval_t, 1) << 57) #define PTE_DEVMAP (_AT(pteval_t, 1) << 57)
#define PTE_PROT_NONE (_AT(pteval_t, 1) << 58) /* only when !PTE_VALID */
/* /*
* This bit indicates that the entry is present i.e. pmd_page() * PTE_PRESENT_INVALID=1 & PTE_VALID=0 indicates that the pte's fields should be
* still points to a valid huge page in memory even if the pmd * interpreted according to the HW layout by SW but any attempted HW access to
* has been invalidated. * the address will result in a fault. pte_present() returns true.
*/ */
#define PMD_PRESENT_INVALID (_AT(pteval_t, 1) << 59) /* only when !PMD_SECT_VALID */ #define PTE_PRESENT_INVALID (PTE_NG) /* only when !PTE_VALID */
#ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP
#define PTE_UFFD_WP (_AT(pteval_t, 1) << 58) /* uffd-wp tracking */
#define PTE_SWP_UFFD_WP (_AT(pteval_t, 1) << 3) /* only for swp ptes */
#else
#define PTE_UFFD_WP (_AT(pteval_t, 0))
#define PTE_SWP_UFFD_WP (_AT(pteval_t, 0))
#endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */
#define _PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) #define _PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED)
#define _PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) #define _PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S)
...@@ -103,7 +110,7 @@ static inline bool __pure lpa2_is_enabled(void) ...@@ -103,7 +110,7 @@ static inline bool __pure lpa2_is_enabled(void)
__val; \ __val; \
}) })
#define PAGE_NONE __pgprot(((_PAGE_DEFAULT) & ~PTE_VALID) | PTE_PROT_NONE | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN) #define PAGE_NONE __pgprot(((_PAGE_DEFAULT) & ~PTE_VALID) | PTE_PRESENT_INVALID | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN)
/* shared+writable pages are clean by default, hence PTE_RDONLY|PTE_WRITE */ /* shared+writable pages are clean by default, hence PTE_RDONLY|PTE_WRITE */
#define PAGE_SHARED __pgprot(_PAGE_SHARED) #define PAGE_SHARED __pgprot(_PAGE_SHARED)
#define PAGE_SHARED_EXEC __pgprot(_PAGE_SHARED_EXEC) #define PAGE_SHARED_EXEC __pgprot(_PAGE_SHARED_EXEC)
......
...@@ -105,7 +105,7 @@ static inline pteval_t __phys_to_pte_val(phys_addr_t phys) ...@@ -105,7 +105,7 @@ static inline pteval_t __phys_to_pte_val(phys_addr_t phys)
/* /*
* The following only work if pte_present(). Undefined behaviour otherwise. * The following only work if pte_present(). Undefined behaviour otherwise.
*/ */
#define pte_present(pte) (!!(pte_val(pte) & (PTE_VALID | PTE_PROT_NONE))) #define pte_present(pte) (pte_valid(pte) || pte_present_invalid(pte))
#define pte_young(pte) (!!(pte_val(pte) & PTE_AF)) #define pte_young(pte) (!!(pte_val(pte) & PTE_AF))
#define pte_special(pte) (!!(pte_val(pte) & PTE_SPECIAL)) #define pte_special(pte) (!!(pte_val(pte) & PTE_SPECIAL))
#define pte_write(pte) (!!(pte_val(pte) & PTE_WRITE)) #define pte_write(pte) (!!(pte_val(pte) & PTE_WRITE))
...@@ -132,6 +132,8 @@ static inline pteval_t __phys_to_pte_val(phys_addr_t phys) ...@@ -132,6 +132,8 @@ static inline pteval_t __phys_to_pte_val(phys_addr_t phys)
#define pte_dirty(pte) (pte_sw_dirty(pte) || pte_hw_dirty(pte)) #define pte_dirty(pte) (pte_sw_dirty(pte) || pte_hw_dirty(pte))
#define pte_valid(pte) (!!(pte_val(pte) & PTE_VALID)) #define pte_valid(pte) (!!(pte_val(pte) & PTE_VALID))
#define pte_present_invalid(pte) \
((pte_val(pte) & (PTE_VALID | PTE_PRESENT_INVALID)) == PTE_PRESENT_INVALID)
/* /*
* Execute-only user mappings do not have the PTE_USER bit set. All valid * Execute-only user mappings do not have the PTE_USER bit set. All valid
* kernel mappings have the PTE_UXN bit set. * kernel mappings have the PTE_UXN bit set.
...@@ -261,6 +263,13 @@ static inline pte_t pte_mkpresent(pte_t pte) ...@@ -261,6 +263,13 @@ static inline pte_t pte_mkpresent(pte_t pte)
return set_pte_bit(pte, __pgprot(PTE_VALID)); return set_pte_bit(pte, __pgprot(PTE_VALID));
} }
static inline pte_t pte_mkinvalid(pte_t pte)
{
pte = set_pte_bit(pte, __pgprot(PTE_PRESENT_INVALID));
pte = clear_pte_bit(pte, __pgprot(PTE_VALID));
return pte;
}
static inline pmd_t pmd_mkcont(pmd_t pmd) static inline pmd_t pmd_mkcont(pmd_t pmd)
{ {
return __pmd(pmd_val(pmd) | PMD_SECT_CONT); return __pmd(pmd_val(pmd) | PMD_SECT_CONT);
...@@ -271,9 +280,31 @@ static inline pte_t pte_mkdevmap(pte_t pte) ...@@ -271,9 +280,31 @@ static inline pte_t pte_mkdevmap(pte_t pte)
return set_pte_bit(pte, __pgprot(PTE_DEVMAP | PTE_SPECIAL)); return set_pte_bit(pte, __pgprot(PTE_DEVMAP | PTE_SPECIAL));
} }
static inline void __set_pte(pte_t *ptep, pte_t pte) #ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP
static inline int pte_uffd_wp(pte_t pte)
{
return !!(pte_val(pte) & PTE_UFFD_WP);
}
static inline pte_t pte_mkuffd_wp(pte_t pte)
{
return pte_wrprotect(set_pte_bit(pte, __pgprot(PTE_UFFD_WP)));
}
static inline pte_t pte_clear_uffd_wp(pte_t pte)
{
return clear_pte_bit(pte, __pgprot(PTE_UFFD_WP));
}
#endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */
static inline void __set_pte_nosync(pte_t *ptep, pte_t pte)
{ {
WRITE_ONCE(*ptep, pte); WRITE_ONCE(*ptep, pte);
}
static inline void __set_pte(pte_t *ptep, pte_t pte)
{
__set_pte_nosync(ptep, pte);
/* /*
* Only if the new pte is valid and kernel, otherwise TLB maintenance * Only if the new pte is valid and kernel, otherwise TLB maintenance
...@@ -463,13 +494,39 @@ static inline pte_t pte_swp_clear_exclusive(pte_t pte) ...@@ -463,13 +494,39 @@ static inline pte_t pte_swp_clear_exclusive(pte_t pte)
return clear_pte_bit(pte, __pgprot(PTE_SWP_EXCLUSIVE)); return clear_pte_bit(pte, __pgprot(PTE_SWP_EXCLUSIVE));
} }
#ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP
static inline pte_t pte_swp_mkuffd_wp(pte_t pte)
{
return set_pte_bit(pte, __pgprot(PTE_SWP_UFFD_WP));
}
static inline int pte_swp_uffd_wp(pte_t pte)
{
return !!(pte_val(pte) & PTE_SWP_UFFD_WP);
}
static inline pte_t pte_swp_clear_uffd_wp(pte_t pte)
{
return clear_pte_bit(pte, __pgprot(PTE_SWP_UFFD_WP));
}
#endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */
#ifdef CONFIG_NUMA_BALANCING #ifdef CONFIG_NUMA_BALANCING
/* /*
* See the comment in include/linux/pgtable.h * See the comment in include/linux/pgtable.h
*/ */
static inline int pte_protnone(pte_t pte) static inline int pte_protnone(pte_t pte)
{ {
return (pte_val(pte) & (PTE_VALID | PTE_PROT_NONE)) == PTE_PROT_NONE; /*
* pte_present_invalid() tells us that the pte is invalid from HW
* perspective but present from SW perspective, so the fields are to be
* interpretted as per the HW layout. The second 2 checks are the unique
* encoding that we use for PROT_NONE. It is insufficient to only use
* the first check because we share the same encoding scheme with pmds
* which support pmd_mkinvalid(), so can be present-invalid without
* being PROT_NONE.
*/
return pte_present_invalid(pte) && !pte_user(pte) && !pte_user_exec(pte);
} }
static inline int pmd_protnone(pmd_t pmd) static inline int pmd_protnone(pmd_t pmd)
...@@ -478,12 +535,7 @@ static inline int pmd_protnone(pmd_t pmd) ...@@ -478,12 +535,7 @@ static inline int pmd_protnone(pmd_t pmd)
} }
#endif #endif
#define pmd_present_invalid(pmd) (!!(pmd_val(pmd) & PMD_PRESENT_INVALID)) #define pmd_present(pmd) pte_present(pmd_pte(pmd))
static inline int pmd_present(pmd_t pmd)
{
return pte_present(pmd_pte(pmd)) || pmd_present_invalid(pmd);
}
/* /*
* THP definitions. * THP definitions.
...@@ -508,14 +560,16 @@ static inline int pmd_trans_huge(pmd_t pmd) ...@@ -508,14 +560,16 @@ static inline int pmd_trans_huge(pmd_t pmd)
#define pmd_mkclean(pmd) pte_pmd(pte_mkclean(pmd_pte(pmd))) #define pmd_mkclean(pmd) pte_pmd(pte_mkclean(pmd_pte(pmd)))
#define pmd_mkdirty(pmd) pte_pmd(pte_mkdirty(pmd_pte(pmd))) #define pmd_mkdirty(pmd) pte_pmd(pte_mkdirty(pmd_pte(pmd)))
#define pmd_mkyoung(pmd) pte_pmd(pte_mkyoung(pmd_pte(pmd))) #define pmd_mkyoung(pmd) pte_pmd(pte_mkyoung(pmd_pte(pmd)))
#define pmd_mkinvalid(pmd) pte_pmd(pte_mkinvalid(pmd_pte(pmd)))
static inline pmd_t pmd_mkinvalid(pmd_t pmd) #ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP
{ #define pmd_uffd_wp(pmd) pte_uffd_wp(pmd_pte(pmd))
pmd = set_pmd_bit(pmd, __pgprot(PMD_PRESENT_INVALID)); #define pmd_mkuffd_wp(pmd) pte_pmd(pte_mkuffd_wp(pmd_pte(pmd)))
pmd = clear_pmd_bit(pmd, __pgprot(PMD_SECT_VALID)); #define pmd_clear_uffd_wp(pmd) pte_pmd(pte_clear_uffd_wp(pmd_pte(pmd)))
#define pmd_swp_uffd_wp(pmd) pte_swp_uffd_wp(pmd_pte(pmd))
return pmd; #define pmd_swp_mkuffd_wp(pmd) pte_pmd(pte_swp_mkuffd_wp(pmd_pte(pmd)))
} #define pmd_swp_clear_uffd_wp(pmd) \
pte_pmd(pte_swp_clear_uffd_wp(pmd_pte(pmd)))
#endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */
#define pmd_thp_or_huge(pmd) (pmd_huge(pmd) || pmd_trans_huge(pmd)) #define pmd_thp_or_huge(pmd) (pmd_huge(pmd) || pmd_trans_huge(pmd))
...@@ -760,6 +814,7 @@ static inline pmd_t *pud_pgtable(pud_t pud) ...@@ -760,6 +814,7 @@ static inline pmd_t *pud_pgtable(pud_t pud)
#else #else
#define pud_valid(pud) false
#define pud_page_paddr(pud) ({ BUILD_BUG(); 0; }) #define pud_page_paddr(pud) ({ BUILD_BUG(); 0; })
#define pud_user_exec(pud) pud_user(pud) /* Always 0 with folding */ #define pud_user_exec(pud) pud_user(pud) /* Always 0 with folding */
...@@ -1005,6 +1060,8 @@ static inline p4d_t *p4d_offset_kimg(pgd_t *pgdp, u64 addr) ...@@ -1005,6 +1060,8 @@ static inline p4d_t *p4d_offset_kimg(pgd_t *pgdp, u64 addr)
static inline bool pgtable_l5_enabled(void) { return false; } static inline bool pgtable_l5_enabled(void) { return false; }
#define p4d_index(addr) (((addr) >> P4D_SHIFT) & (PTRS_PER_P4D - 1))
/* Match p4d_offset folding in <asm/generic/pgtable-nop4d.h> */ /* Match p4d_offset folding in <asm/generic/pgtable-nop4d.h> */
#define p4d_set_fixmap(addr) NULL #define p4d_set_fixmap(addr) NULL
#define p4d_set_fixmap_offset(p4dp, addr) ((p4d_t *)p4dp) #define p4d_set_fixmap_offset(p4dp, addr) ((p4d_t *)p4dp)
...@@ -1027,8 +1084,8 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) ...@@ -1027,8 +1084,8 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
* in MAIR_EL1. The mask below has to include PTE_ATTRINDX_MASK. * in MAIR_EL1. The mask below has to include PTE_ATTRINDX_MASK.
*/ */
const pteval_t mask = PTE_USER | PTE_PXN | PTE_UXN | PTE_RDONLY | const pteval_t mask = PTE_USER | PTE_PXN | PTE_UXN | PTE_RDONLY |
PTE_PROT_NONE | PTE_VALID | PTE_WRITE | PTE_GP | PTE_PRESENT_INVALID | PTE_VALID | PTE_WRITE |
PTE_ATTRINDX_MASK; PTE_GP | PTE_ATTRINDX_MASK;
/* preserve the hardware dirty information */ /* preserve the hardware dirty information */
if (pte_hw_dirty(pte)) if (pte_hw_dirty(pte))
pte = set_pte_bit(pte, __pgprot(PTE_DIRTY)); pte = set_pte_bit(pte, __pgprot(PTE_DIRTY));
...@@ -1076,17 +1133,17 @@ static inline int pgd_devmap(pgd_t pgd) ...@@ -1076,17 +1133,17 @@ static inline int pgd_devmap(pgd_t pgd)
#ifdef CONFIG_PAGE_TABLE_CHECK #ifdef CONFIG_PAGE_TABLE_CHECK
static inline bool pte_user_accessible_page(pte_t pte) static inline bool pte_user_accessible_page(pte_t pte)
{ {
return pte_present(pte) && (pte_user(pte) || pte_user_exec(pte)); return pte_valid(pte) && (pte_user(pte) || pte_user_exec(pte));
} }
static inline bool pmd_user_accessible_page(pmd_t pmd) static inline bool pmd_user_accessible_page(pmd_t pmd)
{ {
return pmd_leaf(pmd) && !pmd_present_invalid(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd)); return pmd_valid(pmd) && !pmd_table(pmd) && (pmd_user(pmd) || pmd_user_exec(pmd));
} }
static inline bool pud_user_accessible_page(pud_t pud) static inline bool pud_user_accessible_page(pud_t pud)
{ {
return pud_leaf(pud) && (pud_user(pud) || pud_user_exec(pud)); return pud_valid(pud) && !pud_table(pud) && (pud_user(pud) || pud_user_exec(pud));
} }
#endif #endif
...@@ -1248,15 +1305,16 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, ...@@ -1248,15 +1305,16 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,
* Encode and decode a swap entry: * Encode and decode a swap entry:
* bits 0-1: present (must be zero) * bits 0-1: present (must be zero)
* bits 2: remember PG_anon_exclusive * bits 2: remember PG_anon_exclusive
* bits 3-7: swap type * bit 3: remember uffd-wp state
* bits 8-57: swap offset * bits 6-10: swap type
* bit 58: PTE_PROT_NONE (must be zero) * bit 11: PTE_PRESENT_INVALID (must be zero)
* bits 12-61: swap offset
*/ */
#define __SWP_TYPE_SHIFT 3 #define __SWP_TYPE_SHIFT 6
#define __SWP_TYPE_BITS 5 #define __SWP_TYPE_BITS 5
#define __SWP_OFFSET_BITS 50
#define __SWP_TYPE_MASK ((1 << __SWP_TYPE_BITS) - 1) #define __SWP_TYPE_MASK ((1 << __SWP_TYPE_BITS) - 1)
#define __SWP_OFFSET_SHIFT (__SWP_TYPE_BITS + __SWP_TYPE_SHIFT) #define __SWP_OFFSET_SHIFT 12
#define __SWP_OFFSET_BITS 50
#define __SWP_OFFSET_MASK ((1UL << __SWP_OFFSET_BITS) - 1) #define __SWP_OFFSET_MASK ((1UL << __SWP_OFFSET_BITS) - 1)
#define __swp_type(x) (((x).val >> __SWP_TYPE_SHIFT) & __SWP_TYPE_MASK) #define __swp_type(x) (((x).val >> __SWP_TYPE_SHIFT) & __SWP_TYPE_MASK)
......
...@@ -1036,18 +1036,18 @@ ...@@ -1036,18 +1036,18 @@
* Permission Indirection Extension (PIE) permission encodings. * Permission Indirection Extension (PIE) permission encodings.
* Encodings with the _O suffix, have overlays applied (Permission Overlay Extension). * Encodings with the _O suffix, have overlays applied (Permission Overlay Extension).
*/ */
#define PIE_NONE_O 0x0 #define PIE_NONE_O UL(0x0)
#define PIE_R_O 0x1 #define PIE_R_O UL(0x1)
#define PIE_X_O 0x2 #define PIE_X_O UL(0x2)
#define PIE_RX_O 0x3 #define PIE_RX_O UL(0x3)
#define PIE_RW_O 0x5 #define PIE_RW_O UL(0x5)
#define PIE_RWnX_O 0x6 #define PIE_RWnX_O UL(0x6)
#define PIE_RWX_O 0x7 #define PIE_RWX_O UL(0x7)
#define PIE_R 0x8 #define PIE_R UL(0x8)
#define PIE_GCS 0x9 #define PIE_GCS UL(0x9)
#define PIE_RX 0xa #define PIE_RX UL(0xa)
#define PIE_RW 0xc #define PIE_RW UL(0xc)
#define PIE_RWX 0xe #define PIE_RWX UL(0xe)
#define PIRx_ELx_PERM(idx, perm) ((perm) << ((idx) * 4)) #define PIRx_ELx_PERM(idx, perm) ((perm) << ((idx) * 4))
......
...@@ -142,17 +142,24 @@ static inline unsigned long get_trans_granule(void) ...@@ -142,17 +142,24 @@ static inline unsigned long get_trans_granule(void)
* EL1, Inner Shareable". * EL1, Inner Shareable".
* *
*/ */
#define __TLBI_VADDR_RANGE(baddr, asid, scale, num, ttl) \ #define TLBIR_ASID_MASK GENMASK_ULL(63, 48)
({ \ #define TLBIR_TG_MASK GENMASK_ULL(47, 46)
unsigned long __ta = (baddr); \ #define TLBIR_SCALE_MASK GENMASK_ULL(45, 44)
unsigned long __ttl = (ttl >= 1 && ttl <= 3) ? ttl : 0; \ #define TLBIR_NUM_MASK GENMASK_ULL(43, 39)
__ta &= GENMASK_ULL(36, 0); \ #define TLBIR_TTL_MASK GENMASK_ULL(38, 37)
__ta |= __ttl << 37; \ #define TLBIR_BADDR_MASK GENMASK_ULL(36, 0)
__ta |= (unsigned long)(num) << 39; \
__ta |= (unsigned long)(scale) << 44; \ #define __TLBI_VADDR_RANGE(baddr, asid, scale, num, ttl) \
__ta |= get_trans_granule() << 46; \ ({ \
__ta |= (unsigned long)(asid) << 48; \ unsigned long __ta = 0; \
__ta; \ unsigned long __ttl = (ttl >= 1 && ttl <= 3) ? ttl : 0; \
__ta |= FIELD_PREP(TLBIR_BADDR_MASK, baddr); \
__ta |= FIELD_PREP(TLBIR_TTL_MASK, __ttl); \
__ta |= FIELD_PREP(TLBIR_NUM_MASK, num); \
__ta |= FIELD_PREP(TLBIR_SCALE_MASK, scale); \
__ta |= FIELD_PREP(TLBIR_TG_MASK, get_trans_granule()); \
__ta |= FIELD_PREP(TLBIR_ASID_MASK, asid); \
__ta; \
}) })
/* These macros are used by the TLBI RANGE feature. */ /* These macros are used by the TLBI RANGE feature. */
...@@ -439,11 +446,11 @@ static inline void __flush_tlb_range_nosync(struct vm_area_struct *vma, ...@@ -439,11 +446,11 @@ static inline void __flush_tlb_range_nosync(struct vm_area_struct *vma,
* When not uses TLB range ops, we can handle up to * When not uses TLB range ops, we can handle up to
* (MAX_DVM_OPS - 1) pages; * (MAX_DVM_OPS - 1) pages;
* When uses TLB range ops, we can handle up to * When uses TLB range ops, we can handle up to
* (MAX_TLBI_RANGE_PAGES - 1) pages. * MAX_TLBI_RANGE_PAGES pages.
*/ */
if ((!system_supports_tlb_range() && if ((!system_supports_tlb_range() &&
(end - start) >= (MAX_DVM_OPS * stride)) || (end - start) >= (MAX_DVM_OPS * stride)) ||
pages >= MAX_TLBI_RANGE_PAGES) { pages > MAX_TLBI_RANGE_PAGES) {
flush_tlb_mm(vma->vm_mm); flush_tlb_mm(vma->vm_mm);
return; return;
} }
......
...@@ -26,6 +26,7 @@ ...@@ -26,6 +26,7 @@
#include <linux/libfdt.h> #include <linux/libfdt.h>
#include <linux/smp.h> #include <linux/smp.h>
#include <linux/serial_core.h> #include <linux/serial_core.h>
#include <linux/suspend.h>
#include <linux/pgtable.h> #include <linux/pgtable.h>
#include <acpi/ghes.h> #include <acpi/ghes.h>
...@@ -227,6 +228,15 @@ void __init acpi_boot_table_init(void) ...@@ -227,6 +228,15 @@ void __init acpi_boot_table_init(void)
if (earlycon_acpi_spcr_enable) if (earlycon_acpi_spcr_enable)
early_init_dt_scan_chosen_stdout(); early_init_dt_scan_chosen_stdout();
} else { } else {
#ifdef CONFIG_HIBERNATION
struct acpi_table_header *facs = NULL;
acpi_get_table(ACPI_SIG_FACS, 1, &facs);
if (facs) {
swsusp_hardware_signature =
((struct acpi_table_facs *)facs)->hardware_signature;
acpi_put_table(facs);
}
#endif
acpi_parse_spcr(earlycon_acpi_spcr_enable, true); acpi_parse_spcr(earlycon_acpi_spcr_enable, true);
if (IS_ENABLED(CONFIG_ACPI_BGRT)) if (IS_ENABLED(CONFIG_ACPI_BGRT))
acpi_table_parse(ACPI_SIG_BGRT, acpi_parse_bgrt); acpi_table_parse(ACPI_SIG_BGRT, acpi_parse_bgrt);
......
...@@ -10,94 +10,12 @@ ...@@ -10,94 +10,12 @@
#include <asm/pointer_auth.h> #include <asm/pointer_auth.h>
struct frame_tail { static bool callchain_trace(void *data, unsigned long pc)
struct frame_tail __user *fp;
unsigned long lr;
} __attribute__((packed));
/*
* Get the return address for a single stackframe and return a pointer to the
* next frame tail.
*/
static struct frame_tail __user *
user_backtrace(struct frame_tail __user *tail,
struct perf_callchain_entry_ctx *entry)
{
struct frame_tail buftail;
unsigned long err;
unsigned long lr;
/* Also check accessibility of one struct frame_tail beyond */
if (!access_ok(tail, sizeof(buftail)))
return NULL;
pagefault_disable();
err = __copy_from_user_inatomic(&buftail, tail, sizeof(buftail));
pagefault_enable();
if (err)
return NULL;
lr = ptrauth_strip_user_insn_pac(buftail.lr);
perf_callchain_store(entry, lr);
/*
* Frame pointers should strictly progress back up the stack
* (towards higher addresses).
*/
if (tail >= buftail.fp)
return NULL;
return buftail.fp;
}
#ifdef CONFIG_COMPAT
/*
* The registers we're interested in are at the end of the variable
* length saved register structure. The fp points at the end of this
* structure so the address of this struct is:
* (struct compat_frame_tail *)(xxx->fp)-1
*
* This code has been adapted from the ARM OProfile support.
*/
struct compat_frame_tail {
compat_uptr_t fp; /* a (struct compat_frame_tail *) in compat mode */
u32 sp;
u32 lr;
} __attribute__((packed));
static struct compat_frame_tail __user *
compat_user_backtrace(struct compat_frame_tail __user *tail,
struct perf_callchain_entry_ctx *entry)
{ {
struct compat_frame_tail buftail; struct perf_callchain_entry_ctx *entry = data;
unsigned long err;
/* Also check accessibility of one struct frame_tail beyond */
if (!access_ok(tail, sizeof(buftail)))
return NULL;
pagefault_disable();
err = __copy_from_user_inatomic(&buftail, tail, sizeof(buftail));
pagefault_enable();
if (err)
return NULL;
perf_callchain_store(entry, buftail.lr);
/*
* Frame pointers should strictly progress back up the stack
* (towards higher addresses).
*/
if (tail + 1 >= (struct compat_frame_tail __user *)
compat_ptr(buftail.fp))
return NULL;
return (struct compat_frame_tail __user *)compat_ptr(buftail.fp) - 1; return perf_callchain_store(entry, pc) == 0;
} }
#endif /* CONFIG_COMPAT */
void perf_callchain_user(struct perf_callchain_entry_ctx *entry, void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
struct pt_regs *regs) struct pt_regs *regs)
...@@ -107,35 +25,7 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry, ...@@ -107,35 +25,7 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
return; return;
} }
perf_callchain_store(entry, regs->pc); arch_stack_walk_user(callchain_trace, entry, regs);
if (!compat_user_mode(regs)) {
/* AARCH64 mode */
struct frame_tail __user *tail;
tail = (struct frame_tail __user *)regs->regs[29];
while (entry->nr < entry->max_stack &&
tail && !((unsigned long)tail & 0x7))
tail = user_backtrace(tail, entry);
} else {
#ifdef CONFIG_COMPAT
/* AARCH32 compat mode */
struct compat_frame_tail __user *tail;
tail = (struct compat_frame_tail __user *)regs->compat_fp - 1;
while ((entry->nr < entry->max_stack) &&
tail && !((unsigned long)tail & 0x3))
tail = compat_user_backtrace(tail, entry);
#endif
}
}
static bool callchain_trace(void *data, unsigned long pc)
{
struct perf_callchain_entry_ctx *entry = data;
return perf_callchain_store(entry, pc) == 0;
} }
void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
......
...@@ -108,6 +108,7 @@ static const struct ftr_set_desc pfr0 __prel64_initconst = { ...@@ -108,6 +108,7 @@ static const struct ftr_set_desc pfr0 __prel64_initconst = {
.override = &id_aa64pfr0_override, .override = &id_aa64pfr0_override,
.fields = { .fields = {
FIELD("sve", ID_AA64PFR0_EL1_SVE_SHIFT, pfr0_sve_filter), FIELD("sve", ID_AA64PFR0_EL1_SVE_SHIFT, pfr0_sve_filter),
FIELD("el0", ID_AA64PFR0_EL1_EL0_SHIFT, NULL),
{} {}
}, },
}; };
...@@ -223,6 +224,7 @@ static const struct { ...@@ -223,6 +224,7 @@ static const struct {
{ "nokaslr", "arm64_sw.nokaslr=1" }, { "nokaslr", "arm64_sw.nokaslr=1" },
{ "rodata=off", "arm64_sw.rodataoff=1" }, { "rodata=off", "arm64_sw.rodataoff=1" },
{ "arm64.nolva", "id_aa64mmfr2.varange=0" }, { "arm64.nolva", "id_aa64mmfr2.varange=0" },
{ "arm64.no32bit_el0", "id_aa64pfr0.el0=1" },
}; };
static int __init parse_hexdigit(const char *p, u64 *v) static int __init parse_hexdigit(const char *p, u64 *v)
......
...@@ -298,8 +298,15 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p) ...@@ -298,8 +298,15 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
dynamic_scs_init(); dynamic_scs_init();
/* /*
* Unmask SError as soon as possible after initializing earlycon so * The primary CPU enters the kernel with all DAIF exceptions masked.
* that we can report any SErrors immediately. *
* We must unmask Debug and SError before preemption or scheduling is
* possible to ensure that these are consistently unmasked across
* threads, and we want to unmask SError as soon as possible after
* initializing earlycon so that we can report any SErrors immediately.
*
* IRQ and FIQ will be unmasked after the root irqchip has been
* detected and initialized.
*/ */
local_daif_restore(DAIF_PROCCTX_NOIRQ); local_daif_restore(DAIF_PROCCTX_NOIRQ);
......
...@@ -264,6 +264,13 @@ asmlinkage notrace void secondary_start_kernel(void) ...@@ -264,6 +264,13 @@ asmlinkage notrace void secondary_start_kernel(void)
set_cpu_online(cpu, true); set_cpu_online(cpu, true);
complete(&cpu_running); complete(&cpu_running);
/*
* Secondary CPUs enter the kernel with all DAIF exceptions masked.
*
* As with setup_arch() we must unmask Debug and SError exceptions, and
* as the root irqchip has already been detected and initialized we can
* unmask IRQ and FIQ at the same time.
*/
local_daif_restore(DAIF_PROCCTX); local_daif_restore(DAIF_PROCCTX);
/* /*
......
...@@ -324,3 +324,123 @@ void show_stack(struct task_struct *tsk, unsigned long *sp, const char *loglvl) ...@@ -324,3 +324,123 @@ void show_stack(struct task_struct *tsk, unsigned long *sp, const char *loglvl)
dump_backtrace(NULL, tsk, loglvl); dump_backtrace(NULL, tsk, loglvl);
barrier(); barrier();
} }
/*
* The struct defined for userspace stack frame in AARCH64 mode.
*/
struct frame_tail {
struct frame_tail __user *fp;
unsigned long lr;
} __attribute__((packed));
/*
* Get the return address for a single stackframe and return a pointer to the
* next frame tail.
*/
static struct frame_tail __user *
unwind_user_frame(struct frame_tail __user *tail, void *cookie,
stack_trace_consume_fn consume_entry)
{
struct frame_tail buftail;
unsigned long err;
unsigned long lr;
/* Also check accessibility of one struct frame_tail beyond */
if (!access_ok(tail, sizeof(buftail)))
return NULL;
pagefault_disable();
err = __copy_from_user_inatomic(&buftail, tail, sizeof(buftail));
pagefault_enable();
if (err)
return NULL;
lr = ptrauth_strip_user_insn_pac(buftail.lr);
if (!consume_entry(cookie, lr))
return NULL;
/*
* Frame pointers should strictly progress back up the stack
* (towards higher addresses).
*/
if (tail >= buftail.fp)
return NULL;
return buftail.fp;
}
#ifdef CONFIG_COMPAT
/*
* The registers we're interested in are at the end of the variable
* length saved register structure. The fp points at the end of this
* structure so the address of this struct is:
* (struct compat_frame_tail *)(xxx->fp)-1
*
* This code has been adapted from the ARM OProfile support.
*/
struct compat_frame_tail {
compat_uptr_t fp; /* a (struct compat_frame_tail *) in compat mode */
u32 sp;
u32 lr;
} __attribute__((packed));
static struct compat_frame_tail __user *
unwind_compat_user_frame(struct compat_frame_tail __user *tail, void *cookie,
stack_trace_consume_fn consume_entry)
{
struct compat_frame_tail buftail;
unsigned long err;
/* Also check accessibility of one struct frame_tail beyond */
if (!access_ok(tail, sizeof(buftail)))
return NULL;
pagefault_disable();
err = __copy_from_user_inatomic(&buftail, tail, sizeof(buftail));
pagefault_enable();
if (err)
return NULL;
if (!consume_entry(cookie, buftail.lr))
return NULL;
/*
* Frame pointers should strictly progress back up the stack
* (towards higher addresses).
*/
if (tail + 1 >= (struct compat_frame_tail __user *)
compat_ptr(buftail.fp))
return NULL;
return (struct compat_frame_tail __user *)compat_ptr(buftail.fp) - 1;
}
#endif /* CONFIG_COMPAT */
void arch_stack_walk_user(stack_trace_consume_fn consume_entry, void *cookie,
const struct pt_regs *regs)
{
if (!consume_entry(cookie, regs->pc))
return;
if (!compat_user_mode(regs)) {
/* AARCH64 mode */
struct frame_tail __user *tail;
tail = (struct frame_tail __user *)regs->regs[29];
while (tail && !((unsigned long)tail & 0x7))
tail = unwind_user_frame(tail, cookie, consume_entry);
} else {
#ifdef CONFIG_COMPAT
/* AARCH32 compat mode */
struct compat_frame_tail __user *tail;
tail = (struct compat_frame_tail __user *)regs->compat_fp - 1;
while (tail && !((unsigned long)tail & 0x3))
tail = unwind_compat_user_frame(tail, cookie, consume_entry);
#endif
}
}
...@@ -109,28 +109,12 @@ EXPORT_SYMBOL(phys_mem_access_prot); ...@@ -109,28 +109,12 @@ EXPORT_SYMBOL(phys_mem_access_prot);
static phys_addr_t __init early_pgtable_alloc(int shift) static phys_addr_t __init early_pgtable_alloc(int shift)
{ {
phys_addr_t phys; phys_addr_t phys;
void *ptr;
phys = memblock_phys_alloc_range(PAGE_SIZE, PAGE_SIZE, 0, phys = memblock_phys_alloc_range(PAGE_SIZE, PAGE_SIZE, 0,
MEMBLOCK_ALLOC_NOLEAKTRACE); MEMBLOCK_ALLOC_NOLEAKTRACE);
if (!phys) if (!phys)
panic("Failed to allocate page table page\n"); panic("Failed to allocate page table page\n");
/*
* The FIX_{PGD,PUD,PMD} slots may be in active use, but the FIX_PTE
* slot will be free, so we can (ab)use the FIX_PTE slot to initialise
* any level of table.
*/
ptr = pte_set_fixmap(phys);
memset(ptr, 0, PAGE_SIZE);
/*
* Implicit barriers also ensure the zeroed page is visible to the page
* table walker
*/
pte_clear_fixmap();
return phys; return phys;
} }
...@@ -172,16 +156,25 @@ bool pgattr_change_is_safe(u64 old, u64 new) ...@@ -172,16 +156,25 @@ bool pgattr_change_is_safe(u64 old, u64 new)
return ((old ^ new) & ~mask) == 0; return ((old ^ new) & ~mask) == 0;
} }
static void init_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, static void init_clear_pgtable(void *table)
phys_addr_t phys, pgprot_t prot)
{ {
pte_t *ptep; clear_page(table);
ptep = pte_set_fixmap_offset(pmdp, addr); /* Ensure the zeroing is observed by page table walks. */
dsb(ishst);
}
static void init_pte(pte_t *ptep, unsigned long addr, unsigned long end,
phys_addr_t phys, pgprot_t prot)
{
do { do {
pte_t old_pte = __ptep_get(ptep); pte_t old_pte = __ptep_get(ptep);
__set_pte(ptep, pfn_pte(__phys_to_pfn(phys), prot)); /*
* Required barriers to make this visible to the table walker
* are deferred to the end of alloc_init_cont_pte().
*/
__set_pte_nosync(ptep, pfn_pte(__phys_to_pfn(phys), prot));
/* /*
* After the PTE entry has been populated once, we * After the PTE entry has been populated once, we
...@@ -192,8 +185,6 @@ static void init_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, ...@@ -192,8 +185,6 @@ static void init_pte(pmd_t *pmdp, unsigned long addr, unsigned long end,
phys += PAGE_SIZE; phys += PAGE_SIZE;
} while (ptep++, addr += PAGE_SIZE, addr != end); } while (ptep++, addr += PAGE_SIZE, addr != end);
pte_clear_fixmap();
} }
static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr,
...@@ -204,6 +195,7 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, ...@@ -204,6 +195,7 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr,
{ {
unsigned long next; unsigned long next;
pmd_t pmd = READ_ONCE(*pmdp); pmd_t pmd = READ_ONCE(*pmdp);
pte_t *ptep;
BUG_ON(pmd_sect(pmd)); BUG_ON(pmd_sect(pmd));
if (pmd_none(pmd)) { if (pmd_none(pmd)) {
...@@ -214,10 +206,14 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, ...@@ -214,10 +206,14 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr,
pmdval |= PMD_TABLE_PXN; pmdval |= PMD_TABLE_PXN;
BUG_ON(!pgtable_alloc); BUG_ON(!pgtable_alloc);
pte_phys = pgtable_alloc(PAGE_SHIFT); pte_phys = pgtable_alloc(PAGE_SHIFT);
ptep = pte_set_fixmap(pte_phys);
init_clear_pgtable(ptep);
ptep += pte_index(addr);
__pmd_populate(pmdp, pte_phys, pmdval); __pmd_populate(pmdp, pte_phys, pmdval);
pmd = READ_ONCE(*pmdp); } else {
BUG_ON(pmd_bad(pmd));
ptep = pte_set_fixmap_offset(pmdp, addr);
} }
BUG_ON(pmd_bad(pmd));
do { do {
pgprot_t __prot = prot; pgprot_t __prot = prot;
...@@ -229,20 +225,26 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, ...@@ -229,20 +225,26 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr,
(flags & NO_CONT_MAPPINGS) == 0) (flags & NO_CONT_MAPPINGS) == 0)
__prot = __pgprot(pgprot_val(prot) | PTE_CONT); __prot = __pgprot(pgprot_val(prot) | PTE_CONT);
init_pte(pmdp, addr, next, phys, __prot); init_pte(ptep, addr, next, phys, __prot);
ptep += pte_index(next) - pte_index(addr);
phys += next - addr; phys += next - addr;
} while (addr = next, addr != end); } while (addr = next, addr != end);
/*
* Note: barriers and maintenance necessary to clear the fixmap slot
* ensure that all previous pgtable writes are visible to the table
* walker.
*/
pte_clear_fixmap();
} }
static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, static void init_pmd(pmd_t *pmdp, unsigned long addr, unsigned long end,
phys_addr_t phys, pgprot_t prot, phys_addr_t phys, pgprot_t prot,
phys_addr_t (*pgtable_alloc)(int), int flags) phys_addr_t (*pgtable_alloc)(int), int flags)
{ {
unsigned long next; unsigned long next;
pmd_t *pmdp;
pmdp = pmd_set_fixmap_offset(pudp, addr);
do { do {
pmd_t old_pmd = READ_ONCE(*pmdp); pmd_t old_pmd = READ_ONCE(*pmdp);
...@@ -268,8 +270,6 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, ...@@ -268,8 +270,6 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end,
} }
phys += next - addr; phys += next - addr;
} while (pmdp++, addr = next, addr != end); } while (pmdp++, addr = next, addr != end);
pmd_clear_fixmap();
} }
static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr,
...@@ -279,6 +279,7 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, ...@@ -279,6 +279,7 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr,
{ {
unsigned long next; unsigned long next;
pud_t pud = READ_ONCE(*pudp); pud_t pud = READ_ONCE(*pudp);
pmd_t *pmdp;
/* /*
* Check for initial section mappings in the pgd/pud. * Check for initial section mappings in the pgd/pud.
...@@ -292,10 +293,14 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, ...@@ -292,10 +293,14 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr,
pudval |= PUD_TABLE_PXN; pudval |= PUD_TABLE_PXN;
BUG_ON(!pgtable_alloc); BUG_ON(!pgtable_alloc);
pmd_phys = pgtable_alloc(PMD_SHIFT); pmd_phys = pgtable_alloc(PMD_SHIFT);
pmdp = pmd_set_fixmap(pmd_phys);
init_clear_pgtable(pmdp);
pmdp += pmd_index(addr);
__pud_populate(pudp, pmd_phys, pudval); __pud_populate(pudp, pmd_phys, pudval);
pud = READ_ONCE(*pudp); } else {
BUG_ON(pud_bad(pud));
pmdp = pmd_set_fixmap_offset(pudp, addr);
} }
BUG_ON(pud_bad(pud));
do { do {
pgprot_t __prot = prot; pgprot_t __prot = prot;
...@@ -307,10 +312,13 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, ...@@ -307,10 +312,13 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr,
(flags & NO_CONT_MAPPINGS) == 0) (flags & NO_CONT_MAPPINGS) == 0)
__prot = __pgprot(pgprot_val(prot) | PTE_CONT); __prot = __pgprot(pgprot_val(prot) | PTE_CONT);
init_pmd(pudp, addr, next, phys, __prot, pgtable_alloc, flags); init_pmd(pmdp, addr, next, phys, __prot, pgtable_alloc, flags);
pmdp += pmd_index(next) - pmd_index(addr);
phys += next - addr; phys += next - addr;
} while (addr = next, addr != end); } while (addr = next, addr != end);
pmd_clear_fixmap();
} }
static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end, static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end,
...@@ -330,12 +338,15 @@ static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end, ...@@ -330,12 +338,15 @@ static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end,
p4dval |= P4D_TABLE_PXN; p4dval |= P4D_TABLE_PXN;
BUG_ON(!pgtable_alloc); BUG_ON(!pgtable_alloc);
pud_phys = pgtable_alloc(PUD_SHIFT); pud_phys = pgtable_alloc(PUD_SHIFT);
pudp = pud_set_fixmap(pud_phys);
init_clear_pgtable(pudp);
pudp += pud_index(addr);
__p4d_populate(p4dp, pud_phys, p4dval); __p4d_populate(p4dp, pud_phys, p4dval);
p4d = READ_ONCE(*p4dp); } else {
BUG_ON(p4d_bad(p4d));
pudp = pud_set_fixmap_offset(p4dp, addr);
} }
BUG_ON(p4d_bad(p4d));
pudp = pud_set_fixmap_offset(p4dp, addr);
do { do {
pud_t old_pud = READ_ONCE(*pudp); pud_t old_pud = READ_ONCE(*pudp);
...@@ -385,12 +396,15 @@ static void alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long end, ...@@ -385,12 +396,15 @@ static void alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long end,
pgdval |= PGD_TABLE_PXN; pgdval |= PGD_TABLE_PXN;
BUG_ON(!pgtable_alloc); BUG_ON(!pgtable_alloc);
p4d_phys = pgtable_alloc(P4D_SHIFT); p4d_phys = pgtable_alloc(P4D_SHIFT);
p4dp = p4d_set_fixmap(p4d_phys);
init_clear_pgtable(p4dp);
p4dp += p4d_index(addr);
__pgd_populate(pgdp, p4d_phys, pgdval); __pgd_populate(pgdp, p4d_phys, pgdval);
pgd = READ_ONCE(*pgdp); } else {
BUG_ON(pgd_bad(pgd));
p4dp = p4d_set_fixmap_offset(pgdp, addr);
} }
BUG_ON(pgd_bad(pgd));
p4dp = p4d_set_fixmap_offset(pgdp, addr);
do { do {
p4d_t old_p4d = READ_ONCE(*p4dp); p4d_t old_p4d = READ_ONCE(*p4dp);
...@@ -457,11 +471,10 @@ void create_kpti_ng_temp_pgd(pgd_t *pgdir, phys_addr_t phys, unsigned long virt, ...@@ -457,11 +471,10 @@ void create_kpti_ng_temp_pgd(pgd_t *pgdir, phys_addr_t phys, unsigned long virt,
static phys_addr_t __pgd_pgtable_alloc(int shift) static phys_addr_t __pgd_pgtable_alloc(int shift)
{ {
void *ptr = (void *)__get_free_page(GFP_PGTABLE_KERNEL); /* Page is zeroed by init_clear_pgtable() so don't duplicate effort. */
BUG_ON(!ptr); void *ptr = (void *)__get_free_page(GFP_PGTABLE_KERNEL & ~__GFP_ZERO);
/* Ensure the zeroed page is visible to the page table walker */ BUG_ON(!ptr);
dsb(ishst);
return __pa(ptr); return __pa(ptr);
} }
......
...@@ -135,14 +135,6 @@ SYM_FUNC_START(cpu_do_resume) ...@@ -135,14 +135,6 @@ SYM_FUNC_START(cpu_do_resume)
msr tcr_el1, x8 msr tcr_el1, x8
msr vbar_el1, x9 msr vbar_el1, x9
/*
* __cpu_setup() cleared MDSCR_EL1.MDE and friends, before unmasking
* debug exceptions. By restoring MDSCR_EL1 here, we may take a debug
* exception. Mask them until local_daif_restore() in cpu_suspend()
* resets them.
*/
disable_daif
msr mdscr_el1, x10 msr mdscr_el1, x10
msr sctlr_el1, x12 msr sctlr_el1, x12
...@@ -466,8 +458,6 @@ SYM_FUNC_START(__cpu_setup) ...@@ -466,8 +458,6 @@ SYM_FUNC_START(__cpu_setup)
msr cpacr_el1, xzr // Reset cpacr_el1 msr cpacr_el1, xzr // Reset cpacr_el1
mov x1, #1 << 12 // Reset mdscr_el1 and disable mov x1, #1 << 12 // Reset mdscr_el1 and disable
msr mdscr_el1, x1 // access to the DCC from EL0 msr mdscr_el1, x1 // access to the DCC from EL0
isb // Unmask debug exceptions now,
enable_dbg // since this is per-cpu
reset_pmuserenr_el0 x1 // Disable PMU access from EL0 reset_pmuserenr_el0 x1 // Disable PMU access from EL0
reset_amuserenr_el0 x1 // Disable AMU access from EL0 reset_amuserenr_el0 x1 // Disable AMU access from EL0
......
...@@ -315,23 +315,19 @@ void acpi_tb_parse_fadt(void) ...@@ -315,23 +315,19 @@ void acpi_tb_parse_fadt(void)
ACPI_TABLE_ORIGIN_INTERNAL_PHYSICAL, ACPI_TABLE_ORIGIN_INTERNAL_PHYSICAL,
NULL, FALSE, TRUE, &acpi_gbl_dsdt_index); NULL, FALSE, TRUE, &acpi_gbl_dsdt_index);
/* If Hardware Reduced flag is set, there is no FACS */ if (acpi_gbl_FADT.facs) {
acpi_tb_install_standard_table((acpi_physical_address)
if (!acpi_gbl_reduced_hardware) { acpi_gbl_FADT.facs,
if (acpi_gbl_FADT.facs) { ACPI_TABLE_ORIGIN_INTERNAL_PHYSICAL,
acpi_tb_install_standard_table((acpi_physical_address) NULL, FALSE, TRUE,
acpi_gbl_FADT.facs, &acpi_gbl_facs_index);
ACPI_TABLE_ORIGIN_INTERNAL_PHYSICAL, }
NULL, FALSE, TRUE, if (acpi_gbl_FADT.Xfacs) {
&acpi_gbl_facs_index); acpi_tb_install_standard_table((acpi_physical_address)
} acpi_gbl_FADT.Xfacs,
if (acpi_gbl_FADT.Xfacs) { ACPI_TABLE_ORIGIN_INTERNAL_PHYSICAL,
acpi_tb_install_standard_table((acpi_physical_address) NULL, FALSE, TRUE,
acpi_gbl_FADT.Xfacs, &acpi_gbl_xfacs_index);
ACPI_TABLE_ORIGIN_INTERNAL_PHYSICAL,
NULL, FALSE, TRUE,
&acpi_gbl_xfacs_index);
}
} }
} }
......
...@@ -36,12 +36,7 @@ acpi_status acpi_tb_initialize_facs(void) ...@@ -36,12 +36,7 @@ acpi_status acpi_tb_initialize_facs(void)
{ {
struct acpi_table_facs *facs; struct acpi_table_facs *facs;
/* If Hardware Reduced flag is set, there is no FACS */ if (acpi_gbl_FADT.Xfacs &&
if (acpi_gbl_reduced_hardware) {
acpi_gbl_FACS = NULL;
return (AE_OK);
} else if (acpi_gbl_FADT.Xfacs &&
(!acpi_gbl_FADT.facs (!acpi_gbl_FADT.facs
|| !acpi_gbl_use32_bit_facs_addresses)) { || !acpi_gbl_use32_bit_facs_addresses)) {
(void)acpi_get_table_by_index(acpi_gbl_xfacs_index, (void)acpi_get_table_by_index(acpi_gbl_xfacs_index,
......
...@@ -709,6 +709,7 @@ static int ali_drw_pmu_probe(struct platform_device *pdev) ...@@ -709,6 +709,7 @@ static int ali_drw_pmu_probe(struct platform_device *pdev)
drw_pmu->pmu = (struct pmu) { drw_pmu->pmu = (struct pmu) {
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = &pdev->dev,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.event_init = ali_drw_pmu_event_init, .event_init = ali_drw_pmu_event_init,
.add = ali_drw_pmu_add, .add = ali_drw_pmu_add,
...@@ -746,18 +747,14 @@ static int ali_drw_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) ...@@ -746,18 +747,14 @@ static int ali_drw_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
struct ali_drw_pmu_irq *irq; struct ali_drw_pmu_irq *irq;
struct ali_drw_pmu *drw_pmu; struct ali_drw_pmu *drw_pmu;
unsigned int target; unsigned int target;
int ret;
cpumask_t node_online_cpus;
irq = hlist_entry_safe(node, struct ali_drw_pmu_irq, node); irq = hlist_entry_safe(node, struct ali_drw_pmu_irq, node);
if (cpu != irq->cpu) if (cpu != irq->cpu)
return 0; return 0;
ret = cpumask_and(&node_online_cpus, target = cpumask_any_and_but(cpumask_of_node(cpu_to_node(cpu)),
cpumask_of_node(cpu_to_node(cpu)), cpu_online_mask); cpu_online_mask, cpu);
if (ret) if (target >= nr_cpu_ids)
target = cpumask_any_but(&node_online_cpus, cpu);
else
target = cpumask_any_but(cpu_online_mask, cpu); target = cpumask_any_but(cpu_online_mask, cpu);
if (target >= nr_cpu_ids) if (target >= nr_cpu_ids)
......
...@@ -492,6 +492,7 @@ int meson_ddr_pmu_create(struct platform_device *pdev) ...@@ -492,6 +492,7 @@ int meson_ddr_pmu_create(struct platform_device *pdev)
*pmu = (struct ddr_pmu) { *pmu = (struct ddr_pmu) {
.pmu = { .pmu = {
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = &pdev->dev,
.capabilities = PERF_PMU_CAP_NO_EXCLUDE, .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.attr_groups = attr_groups, .attr_groups = attr_groups,
......
...@@ -1409,6 +1409,7 @@ static int cci_pmu_init(struct cci_pmu *cci_pmu, struct platform_device *pdev) ...@@ -1409,6 +1409,7 @@ static int cci_pmu_init(struct cci_pmu *cci_pmu, struct platform_device *pdev)
cci_pmu->pmu = (struct pmu) { cci_pmu->pmu = (struct pmu) {
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = &pdev->dev,
.name = cci_pmu->model->name, .name = cci_pmu->model->name,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.pmu_enable = cci_pmu_enable, .pmu_enable = cci_pmu_enable,
......
...@@ -1265,6 +1265,7 @@ static int arm_ccn_pmu_init(struct arm_ccn *ccn) ...@@ -1265,6 +1265,7 @@ static int arm_ccn_pmu_init(struct arm_ccn *ccn)
/* Perf driver registration */ /* Perf driver registration */
ccn->dt.pmu = (struct pmu) { ccn->dt.pmu = (struct pmu) {
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = ccn->dev,
.attr_groups = arm_ccn_pmu_attr_groups, .attr_groups = arm_ccn_pmu_attr_groups,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.event_init = arm_ccn_pmu_event_init, .event_init = arm_ccn_pmu_event_init,
......
...@@ -1950,20 +1950,20 @@ static int arm_cmn_pmu_offline_cpu(unsigned int cpu, struct hlist_node *cpuhp_no ...@@ -1950,20 +1950,20 @@ static int arm_cmn_pmu_offline_cpu(unsigned int cpu, struct hlist_node *cpuhp_no
struct arm_cmn *cmn; struct arm_cmn *cmn;
unsigned int target; unsigned int target;
int node; int node;
cpumask_t mask;
cmn = hlist_entry_safe(cpuhp_node, struct arm_cmn, cpuhp_node); cmn = hlist_entry_safe(cpuhp_node, struct arm_cmn, cpuhp_node);
if (cpu != cmn->cpu) if (cpu != cmn->cpu)
return 0; return 0;
node = dev_to_node(cmn->dev); node = dev_to_node(cmn->dev);
if (cpumask_and(&mask, cpumask_of_node(node), cpu_online_mask) &&
cpumask_andnot(&mask, &mask, cpumask_of(cpu))) target = cpumask_any_and_but(cpumask_of_node(node), cpu_online_mask, cpu);
target = cpumask_any(&mask); if (target >= nr_cpu_ids)
else
target = cpumask_any_but(cpu_online_mask, cpu); target = cpumask_any_but(cpu_online_mask, cpu);
if (target < nr_cpu_ids) if (target < nr_cpu_ids)
arm_cmn_migrate(cmn, target); arm_cmn_migrate(cmn, target);
return 0; return 0;
} }
...@@ -2482,6 +2482,7 @@ static int arm_cmn_probe(struct platform_device *pdev) ...@@ -2482,6 +2482,7 @@ static int arm_cmn_probe(struct platform_device *pdev)
cmn->cpu = cpumask_local_spread(0, dev_to_node(cmn->dev)); cmn->cpu = cpumask_local_spread(0, dev_to_node(cmn->dev));
cmn->pmu = (struct pmu) { cmn->pmu = (struct pmu) {
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = cmn->dev,
.attr_groups = arm_cmn_attr_groups, .attr_groups = arm_cmn_attr_groups,
.capabilities = PERF_PMU_CAP_NO_EXCLUDE, .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
......
...@@ -1206,6 +1206,7 @@ static int arm_cspmu_register_pmu(struct arm_cspmu *cspmu) ...@@ -1206,6 +1206,7 @@ static int arm_cspmu_register_pmu(struct arm_cspmu *cspmu)
cspmu->pmu = (struct pmu){ cspmu->pmu = (struct pmu){
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.module = cspmu->impl.module, .module = cspmu->impl.module,
.parent = cspmu->dev,
.pmu_enable = arm_cspmu_enable, .pmu_enable = arm_cspmu_enable,
.pmu_disable = arm_cspmu_disable, .pmu_disable = arm_cspmu_disable,
.event_init = arm_cspmu_event_init, .event_init = arm_cspmu_event_init,
...@@ -1322,8 +1323,7 @@ static int arm_cspmu_cpu_online(unsigned int cpu, struct hlist_node *node) ...@@ -1322,8 +1323,7 @@ static int arm_cspmu_cpu_online(unsigned int cpu, struct hlist_node *node)
static int arm_cspmu_cpu_teardown(unsigned int cpu, struct hlist_node *node) static int arm_cspmu_cpu_teardown(unsigned int cpu, struct hlist_node *node)
{ {
int dst; unsigned int dst;
struct cpumask online_supported;
struct arm_cspmu *cspmu = struct arm_cspmu *cspmu =
hlist_entry_safe(node, struct arm_cspmu, cpuhp_node); hlist_entry_safe(node, struct arm_cspmu, cpuhp_node);
...@@ -1333,9 +1333,8 @@ static int arm_cspmu_cpu_teardown(unsigned int cpu, struct hlist_node *node) ...@@ -1333,9 +1333,8 @@ static int arm_cspmu_cpu_teardown(unsigned int cpu, struct hlist_node *node)
return 0; return 0;
/* Choose a new CPU to migrate ownership of the PMU to */ /* Choose a new CPU to migrate ownership of the PMU to */
cpumask_and(&online_supported, &cspmu->associated_cpus, dst = cpumask_any_and_but(&cspmu->associated_cpus,
cpu_online_mask); cpu_online_mask, cpu);
dst = cpumask_any_but(&online_supported, cpu);
if (dst >= nr_cpu_ids) if (dst >= nr_cpu_ids)
return 0; return 0;
......
...@@ -673,6 +673,7 @@ static int dmc620_pmu_device_probe(struct platform_device *pdev) ...@@ -673,6 +673,7 @@ static int dmc620_pmu_device_probe(struct platform_device *pdev)
dmc620_pmu->pmu = (struct pmu) { dmc620_pmu->pmu = (struct pmu) {
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = &pdev->dev,
.capabilities = PERF_PMU_CAP_NO_EXCLUDE, .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.event_init = dmc620_pmu_event_init, .event_init = dmc620_pmu_event_init,
......
...@@ -230,15 +230,6 @@ static const struct attribute_group *dsu_pmu_attr_groups[] = { ...@@ -230,15 +230,6 @@ static const struct attribute_group *dsu_pmu_attr_groups[] = {
NULL, NULL,
}; };
static int dsu_pmu_get_online_cpu_any_but(struct dsu_pmu *dsu_pmu, int cpu)
{
struct cpumask online_supported;
cpumask_and(&online_supported,
&dsu_pmu->associated_cpus, cpu_online_mask);
return cpumask_any_but(&online_supported, cpu);
}
static inline bool dsu_pmu_counter_valid(struct dsu_pmu *dsu_pmu, u32 idx) static inline bool dsu_pmu_counter_valid(struct dsu_pmu *dsu_pmu, u32 idx)
{ {
return (idx < dsu_pmu->num_counters) || return (idx < dsu_pmu->num_counters) ||
...@@ -751,6 +742,7 @@ static int dsu_pmu_device_probe(struct platform_device *pdev) ...@@ -751,6 +742,7 @@ static int dsu_pmu_device_probe(struct platform_device *pdev)
dsu_pmu->pmu = (struct pmu) { dsu_pmu->pmu = (struct pmu) {
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.parent = &pdev->dev,
.module = THIS_MODULE, .module = THIS_MODULE,
.pmu_enable = dsu_pmu_enable, .pmu_enable = dsu_pmu_enable,
.pmu_disable = dsu_pmu_disable, .pmu_disable = dsu_pmu_disable,
...@@ -827,14 +819,16 @@ static int dsu_pmu_cpu_online(unsigned int cpu, struct hlist_node *node) ...@@ -827,14 +819,16 @@ static int dsu_pmu_cpu_online(unsigned int cpu, struct hlist_node *node)
static int dsu_pmu_cpu_teardown(unsigned int cpu, struct hlist_node *node) static int dsu_pmu_cpu_teardown(unsigned int cpu, struct hlist_node *node)
{ {
int dst; struct dsu_pmu *dsu_pmu;
struct dsu_pmu *dsu_pmu = hlist_entry_safe(node, struct dsu_pmu, unsigned int dst;
cpuhp_node);
dsu_pmu = hlist_entry_safe(node, struct dsu_pmu, cpuhp_node);
if (!cpumask_test_and_clear_cpu(cpu, &dsu_pmu->active_cpu)) if (!cpumask_test_and_clear_cpu(cpu, &dsu_pmu->active_cpu))
return 0; return 0;
dst = dsu_pmu_get_online_cpu_any_but(dsu_pmu, cpu); dst = cpumask_any_and_but(&dsu_pmu->associated_cpus,
cpu_online_mask, cpu);
/* If there are no active CPUs in the DSU, leave IRQ disabled */ /* If there are no active CPUs in the DSU, leave IRQ disabled */
if (dst >= nr_cpu_ids) if (dst >= nr_cpu_ids)
return 0; return 0;
......
...@@ -196,6 +196,7 @@ int arm_pmu_device_probe(struct platform_device *pdev, ...@@ -196,6 +196,7 @@ int arm_pmu_device_probe(struct platform_device *pdev,
if (!pmu) if (!pmu)
return -ENOMEM; return -ENOMEM;
pmu->pmu.parent = &pdev->dev;
pmu->plat_device = pdev; pmu->plat_device = pdev;
ret = pmu_parse_irqs(pmu); ret = pmu_parse_irqs(pmu);
......
...@@ -860,6 +860,7 @@ static int smmu_pmu_probe(struct platform_device *pdev) ...@@ -860,6 +860,7 @@ static int smmu_pmu_probe(struct platform_device *pdev)
smmu_pmu->pmu = (struct pmu) { smmu_pmu->pmu = (struct pmu) {
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = &pdev->dev,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.pmu_enable = smmu_pmu_enable, .pmu_enable = smmu_pmu_enable,
.pmu_disable = smmu_pmu_disable, .pmu_disable = smmu_pmu_disable,
......
...@@ -932,6 +932,7 @@ static int arm_spe_pmu_perf_init(struct arm_spe_pmu *spe_pmu) ...@@ -932,6 +932,7 @@ static int arm_spe_pmu_perf_init(struct arm_spe_pmu *spe_pmu)
spe_pmu->pmu = (struct pmu) { spe_pmu->pmu = (struct pmu) {
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = &spe_pmu->pdev->dev,
.capabilities = PERF_PMU_CAP_EXCLUSIVE | PERF_PMU_CAP_ITRACE, .capabilities = PERF_PMU_CAP_EXCLUSIVE | PERF_PMU_CAP_ITRACE,
.attr_groups = arm_spe_pmu_attr_groups, .attr_groups = arm_spe_pmu_attr_groups,
/* /*
......
...@@ -690,9 +690,8 @@ static int dwc_pcie_pmu_offline_cpu(unsigned int cpu, struct hlist_node *cpuhp_n ...@@ -690,9 +690,8 @@ static int dwc_pcie_pmu_offline_cpu(unsigned int cpu, struct hlist_node *cpuhp_n
{ {
struct dwc_pcie_pmu *pcie_pmu; struct dwc_pcie_pmu *pcie_pmu;
struct pci_dev *pdev; struct pci_dev *pdev;
int node;
cpumask_t mask;
unsigned int target; unsigned int target;
int node;
pcie_pmu = hlist_entry_safe(cpuhp_node, struct dwc_pcie_pmu, cpuhp_node); pcie_pmu = hlist_entry_safe(cpuhp_node, struct dwc_pcie_pmu, cpuhp_node);
/* Nothing to do if this CPU doesn't own the PMU */ /* Nothing to do if this CPU doesn't own the PMU */
...@@ -702,10 +701,9 @@ static int dwc_pcie_pmu_offline_cpu(unsigned int cpu, struct hlist_node *cpuhp_n ...@@ -702,10 +701,9 @@ static int dwc_pcie_pmu_offline_cpu(unsigned int cpu, struct hlist_node *cpuhp_n
pcie_pmu->on_cpu = -1; pcie_pmu->on_cpu = -1;
pdev = pcie_pmu->pdev; pdev = pcie_pmu->pdev;
node = dev_to_node(&pdev->dev); node = dev_to_node(&pdev->dev);
if (cpumask_and(&mask, cpumask_of_node(node), cpu_online_mask) &&
cpumask_andnot(&mask, &mask, cpumask_of(cpu))) target = cpumask_any_and_but(cpumask_of_node(node), cpu_online_mask, cpu);
target = cpumask_any(&mask); if (target >= nr_cpu_ids)
else
target = cpumask_any_but(cpu_online_mask, cpu); target = cpumask_any_but(cpu_online_mask, cpu);
if (target >= nr_cpu_ids) { if (target >= nr_cpu_ids) {
......
...@@ -651,6 +651,7 @@ static int ddr_perf_init(struct ddr_pmu *pmu, void __iomem *base, ...@@ -651,6 +651,7 @@ static int ddr_perf_init(struct ddr_pmu *pmu, void __iomem *base,
*pmu = (struct ddr_pmu) { *pmu = (struct ddr_pmu) {
.pmu = (struct pmu) { .pmu = (struct pmu) {
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = dev,
.capabilities = PERF_PMU_CAP_NO_EXCLUDE, .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.attr_groups = attr_groups, .attr_groups = attr_groups,
......
...@@ -350,15 +350,27 @@ static bool hisi_pcie_pmu_validate_event_group(struct perf_event *event) ...@@ -350,15 +350,27 @@ static bool hisi_pcie_pmu_validate_event_group(struct perf_event *event)
return false; return false;
for (num = 0; num < counters; num++) { for (num = 0; num < counters; num++) {
/*
* If we find a related event, then it's a valid group
* since we don't need to allocate a new counter for it.
*/
if (hisi_pcie_pmu_cmp_event(event_group[num], sibling)) if (hisi_pcie_pmu_cmp_event(event_group[num], sibling))
break; break;
} }
/*
* Otherwise it's a new event but if there's no available counter,
* fail the check since we cannot schedule all the events in
* the group simultaneously.
*/
if (num == HISI_PCIE_MAX_COUNTERS)
return false;
if (num == counters) if (num == counters)
event_group[counters++] = sibling; event_group[counters++] = sibling;
} }
return counters <= HISI_PCIE_MAX_COUNTERS; return true;
} }
static int hisi_pcie_pmu_event_init(struct perf_event *event) static int hisi_pcie_pmu_event_init(struct perf_event *event)
...@@ -673,7 +685,6 @@ static int hisi_pcie_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) ...@@ -673,7 +685,6 @@ static int hisi_pcie_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
{ {
struct hisi_pcie_pmu *pcie_pmu = hlist_entry_safe(node, struct hisi_pcie_pmu, node); struct hisi_pcie_pmu *pcie_pmu = hlist_entry_safe(node, struct hisi_pcie_pmu, node);
unsigned int target; unsigned int target;
cpumask_t mask;
int numa_node; int numa_node;
/* Nothing to do if this CPU doesn't own the PMU */ /* Nothing to do if this CPU doesn't own the PMU */
...@@ -684,10 +695,10 @@ static int hisi_pcie_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) ...@@ -684,10 +695,10 @@ static int hisi_pcie_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
/* Choose a local CPU from all online cpus. */ /* Choose a local CPU from all online cpus. */
numa_node = dev_to_node(&pcie_pmu->pdev->dev); numa_node = dev_to_node(&pcie_pmu->pdev->dev);
if (cpumask_and(&mask, cpumask_of_node(numa_node), cpu_online_mask) &&
cpumask_andnot(&mask, &mask, cpumask_of(cpu))) target = cpumask_any_and_but(cpumask_of_node(numa_node),
target = cpumask_any(&mask); cpu_online_mask, cpu);
else if (target >= nr_cpu_ids)
target = cpumask_any_but(cpu_online_mask, cpu); target = cpumask_any_but(cpu_online_mask, cpu);
if (target >= nr_cpu_ids) { if (target >= nr_cpu_ids) {
...@@ -807,6 +818,7 @@ static int hisi_pcie_alloc_pmu(struct pci_dev *pdev, struct hisi_pcie_pmu *pcie_ ...@@ -807,6 +818,7 @@ static int hisi_pcie_alloc_pmu(struct pci_dev *pdev, struct hisi_pcie_pmu *pcie_
pcie_pmu->pmu = (struct pmu) { pcie_pmu->pmu = (struct pmu) {
.name = name, .name = name,
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = &pdev->dev,
.event_init = hisi_pcie_pmu_event_init, .event_init = hisi_pcie_pmu_event_init,
.pmu_enable = hisi_pcie_pmu_enable, .pmu_enable = hisi_pcie_pmu_enable,
.pmu_disable = hisi_pcie_pmu_disable, .pmu_disable = hisi_pcie_pmu_disable,
......
...@@ -504,7 +504,6 @@ int hisi_uncore_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) ...@@ -504,7 +504,6 @@ int hisi_uncore_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
{ {
struct hisi_pmu *hisi_pmu = hlist_entry_safe(node, struct hisi_pmu, struct hisi_pmu *hisi_pmu = hlist_entry_safe(node, struct hisi_pmu,
node); node);
cpumask_t pmu_online_cpus;
unsigned int target; unsigned int target;
if (!cpumask_test_and_clear_cpu(cpu, &hisi_pmu->associated_cpus)) if (!cpumask_test_and_clear_cpu(cpu, &hisi_pmu->associated_cpus))
...@@ -518,9 +517,8 @@ int hisi_uncore_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) ...@@ -518,9 +517,8 @@ int hisi_uncore_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
hisi_pmu->on_cpu = -1; hisi_pmu->on_cpu = -1;
/* Choose a new CPU to migrate ownership of the PMU to */ /* Choose a new CPU to migrate ownership of the PMU to */
cpumask_and(&pmu_online_cpus, &hisi_pmu->associated_cpus, target = cpumask_any_and_but(&hisi_pmu->associated_cpus,
cpu_online_mask); cpu_online_mask, cpu);
target = cpumask_any_but(&pmu_online_cpus, cpu);
if (target >= nr_cpu_ids) if (target >= nr_cpu_ids)
return 0; return 0;
...@@ -538,6 +536,7 @@ void hisi_pmu_init(struct hisi_pmu *hisi_pmu, struct module *module) ...@@ -538,6 +536,7 @@ void hisi_pmu_init(struct hisi_pmu *hisi_pmu, struct module *module)
struct pmu *pmu = &hisi_pmu->pmu; struct pmu *pmu = &hisi_pmu->pmu;
pmu->module = module; pmu->module = module;
pmu->parent = hisi_pmu->dev;
pmu->task_ctx_nr = perf_invalid_context; pmu->task_ctx_nr = perf_invalid_context;
pmu->event_init = hisi_uncore_pmu_event_init; pmu->event_init = hisi_uncore_pmu_event_init;
pmu->pmu_enable = hisi_uncore_pmu_enable; pmu->pmu_enable = hisi_uncore_pmu_enable;
......
...@@ -1085,15 +1085,27 @@ static bool hns3_pmu_validate_event_group(struct perf_event *event) ...@@ -1085,15 +1085,27 @@ static bool hns3_pmu_validate_event_group(struct perf_event *event)
return false; return false;
for (num = 0; num < counters; num++) { for (num = 0; num < counters; num++) {
/*
* If we find a related event, then it's a valid group
* since we don't need to allocate a new counter for it.
*/
if (hns3_pmu_cmp_event(event_group[num], sibling)) if (hns3_pmu_cmp_event(event_group[num], sibling))
break; break;
} }
/*
* Otherwise it's a new event but if there's no available counter,
* fail the check since we cannot schedule all the events in
* the group simultaneously.
*/
if (num == HNS3_PMU_MAX_HW_EVENTS)
return false;
if (num == counters) if (num == counters)
event_group[counters++] = sibling; event_group[counters++] = sibling;
} }
return counters <= HNS3_PMU_MAX_HW_EVENTS; return true;
} }
static u32 hns3_pmu_get_filter_condition(struct perf_event *event) static u32 hns3_pmu_get_filter_condition(struct perf_event *event)
...@@ -1419,6 +1431,7 @@ static int hns3_pmu_alloc_pmu(struct pci_dev *pdev, struct hns3_pmu *hns3_pmu) ...@@ -1419,6 +1431,7 @@ static int hns3_pmu_alloc_pmu(struct pci_dev *pdev, struct hns3_pmu *hns3_pmu)
hns3_pmu->pmu = (struct pmu) { hns3_pmu->pmu = (struct pmu) {
.name = name, .name = name,
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = &pdev->dev,
.event_init = hns3_pmu_event_init, .event_init = hns3_pmu_event_init,
.pmu_enable = hns3_pmu_enable, .pmu_enable = hns3_pmu_enable,
.pmu_disable = hns3_pmu_disable, .pmu_disable = hns3_pmu_disable,
...@@ -1515,7 +1528,7 @@ static int hns3_pmu_irq_register(struct pci_dev *pdev, ...@@ -1515,7 +1528,7 @@ static int hns3_pmu_irq_register(struct pci_dev *pdev,
return ret; return ret;
} }
ret = devm_add_action(&pdev->dev, hns3_pmu_free_irq, pdev); ret = devm_add_action_or_reset(&pdev->dev, hns3_pmu_free_irq, pdev);
if (ret) { if (ret) {
pci_err(pdev, "failed to add free irq action, ret = %d.\n", ret); pci_err(pdev, "failed to add free irq action, ret = %d.\n", ret);
return ret; return ret;
......
...@@ -801,9 +801,8 @@ static int l2cache_pmu_online_cpu(unsigned int cpu, struct hlist_node *node) ...@@ -801,9 +801,8 @@ static int l2cache_pmu_online_cpu(unsigned int cpu, struct hlist_node *node)
static int l2cache_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) static int l2cache_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
{ {
struct cluster_pmu *cluster;
struct l2cache_pmu *l2cache_pmu; struct l2cache_pmu *l2cache_pmu;
cpumask_t cluster_online_cpus; struct cluster_pmu *cluster;
unsigned int target; unsigned int target;
l2cache_pmu = hlist_entry_safe(node, struct l2cache_pmu, node); l2cache_pmu = hlist_entry_safe(node, struct l2cache_pmu, node);
...@@ -820,9 +819,8 @@ static int l2cache_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) ...@@ -820,9 +819,8 @@ static int l2cache_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
cluster->on_cpu = -1; cluster->on_cpu = -1;
/* Any other CPU for this cluster which is still online */ /* Any other CPU for this cluster which is still online */
cpumask_and(&cluster_online_cpus, &cluster->cluster_cpus, target = cpumask_any_and_but(&cluster->cluster_cpus,
cpu_online_mask); cpu_online_mask, cpu);
target = cpumask_any_but(&cluster_online_cpus, cpu);
if (target >= nr_cpu_ids) { if (target >= nr_cpu_ids) {
disable_irq(cluster->irq); disable_irq(cluster->irq);
return 0; return 0;
...@@ -904,6 +902,7 @@ static int l2_cache_pmu_probe(struct platform_device *pdev) ...@@ -904,6 +902,7 @@ static int l2_cache_pmu_probe(struct platform_device *pdev)
l2cache_pmu->pmu = (struct pmu) { l2cache_pmu->pmu = (struct pmu) {
/* suffix is instance id for future use with multiple sockets */ /* suffix is instance id for future use with multiple sockets */
.name = "l2cache_0", .name = "l2cache_0",
.parent = &pdev->dev,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.pmu_enable = l2_cache_pmu_enable, .pmu_enable = l2_cache_pmu_enable,
.pmu_disable = l2_cache_pmu_disable, .pmu_disable = l2_cache_pmu_disable,
......
...@@ -748,6 +748,7 @@ static int qcom_l3_cache_pmu_probe(struct platform_device *pdev) ...@@ -748,6 +748,7 @@ static int qcom_l3_cache_pmu_probe(struct platform_device *pdev)
return -ENOMEM; return -ENOMEM;
l3pmu->pmu = (struct pmu) { l3pmu->pmu = (struct pmu) {
.parent = &pdev->dev,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.pmu_enable = qcom_l3_cache__pmu_enable, .pmu_enable = qcom_l3_cache__pmu_enable,
......
...@@ -136,6 +136,7 @@ static int pmu_legacy_device_probe(struct platform_device *pdev) ...@@ -136,6 +136,7 @@ static int pmu_legacy_device_probe(struct platform_device *pdev)
pmu = riscv_pmu_alloc(); pmu = riscv_pmu_alloc();
if (!pmu) if (!pmu)
return -ENOMEM; return -ENOMEM;
pmu->pmu.parent = &pdev->dev;
pmu_legacy_init(pmu); pmu_legacy_init(pmu);
return 0; return 0;
......
...@@ -1043,7 +1043,6 @@ static struct ctl_table sbi_pmu_sysctl_table[] = { ...@@ -1043,7 +1043,6 @@ static struct ctl_table sbi_pmu_sysctl_table[] = {
.extra1 = SYSCTL_ZERO, .extra1 = SYSCTL_ZERO,
.extra2 = SYSCTL_TWO, .extra2 = SYSCTL_TWO,
}, },
{ }
}; };
static int pmu_sbi_device_probe(struct platform_device *pdev) static int pmu_sbi_device_probe(struct platform_device *pdev)
...@@ -1081,6 +1080,7 @@ static int pmu_sbi_device_probe(struct platform_device *pdev) ...@@ -1081,6 +1080,7 @@ static int pmu_sbi_device_probe(struct platform_device *pdev)
} }
pmu->pmu.attr_groups = riscv_pmu_attr_groups; pmu->pmu.attr_groups = riscv_pmu_attr_groups;
pmu->pmu.parent = &pdev->dev;
pmu->cmask = cmask; pmu->cmask = cmask;
pmu->ctr_start = pmu_sbi_ctr_start; pmu->ctr_start = pmu_sbi_ctr_start;
pmu->ctr_stop = pmu_sbi_ctr_stop; pmu->ctr_stop = pmu_sbi_ctr_stop;
......
...@@ -504,24 +504,19 @@ static void tx2_uncore_event_update(struct perf_event *event) ...@@ -504,24 +504,19 @@ static void tx2_uncore_event_update(struct perf_event *event)
static enum tx2_uncore_type get_tx2_pmu_type(struct acpi_device *adev) static enum tx2_uncore_type get_tx2_pmu_type(struct acpi_device *adev)
{ {
int i = 0; struct acpi_device_id devices[] = {
struct acpi_tx2_pmu_device {
__u8 id[ACPI_ID_LEN];
enum tx2_uncore_type type;
} devices[] = {
{"CAV901D", PMU_TYPE_L3C}, {"CAV901D", PMU_TYPE_L3C},
{"CAV901F", PMU_TYPE_DMC}, {"CAV901F", PMU_TYPE_DMC},
{"CAV901E", PMU_TYPE_CCPI2}, {"CAV901E", PMU_TYPE_CCPI2},
{"", PMU_TYPE_INVALID} {}
}; };
const struct acpi_device_id *id;
while (devices[i].type != PMU_TYPE_INVALID) { id = acpi_match_acpi_device(devices, adev);
if (!strcmp(acpi_device_hid(adev), devices[i].id)) if (!id)
break; return PMU_TYPE_INVALID;
i++;
}
return devices[i].type; return (enum tx2_uncore_type)id->driver_data;
} }
static bool tx2_uncore_validate_event(struct pmu *pmu, static bool tx2_uncore_validate_event(struct pmu *pmu,
...@@ -729,6 +724,7 @@ static int tx2_uncore_pmu_register( ...@@ -729,6 +724,7 @@ static int tx2_uncore_pmu_register(
/* Perf event registration */ /* Perf event registration */
tx2_pmu->pmu = (struct pmu) { tx2_pmu->pmu = (struct pmu) {
.module = THIS_MODULE, .module = THIS_MODULE,
.parent = tx2_pmu->dev,
.attr_groups = tx2_pmu->attr_groups, .attr_groups = tx2_pmu->attr_groups,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.event_init = tx2_uncore_event_init, .event_init = tx2_uncore_event_init,
...@@ -932,9 +928,8 @@ static int tx2_uncore_pmu_online_cpu(unsigned int cpu, ...@@ -932,9 +928,8 @@ static int tx2_uncore_pmu_online_cpu(unsigned int cpu,
static int tx2_uncore_pmu_offline_cpu(unsigned int cpu, static int tx2_uncore_pmu_offline_cpu(unsigned int cpu,
struct hlist_node *hpnode) struct hlist_node *hpnode)
{ {
int new_cpu;
struct tx2_uncore_pmu *tx2_pmu; struct tx2_uncore_pmu *tx2_pmu;
struct cpumask cpu_online_mask_temp; unsigned int new_cpu;
tx2_pmu = hlist_entry_safe(hpnode, tx2_pmu = hlist_entry_safe(hpnode,
struct tx2_uncore_pmu, hpnode); struct tx2_uncore_pmu, hpnode);
...@@ -945,11 +940,8 @@ static int tx2_uncore_pmu_offline_cpu(unsigned int cpu, ...@@ -945,11 +940,8 @@ static int tx2_uncore_pmu_offline_cpu(unsigned int cpu,
if (tx2_pmu->hrtimer_callback) if (tx2_pmu->hrtimer_callback)
hrtimer_cancel(&tx2_pmu->hrtimer); hrtimer_cancel(&tx2_pmu->hrtimer);
cpumask_copy(&cpu_online_mask_temp, cpu_online_mask); new_cpu = cpumask_any_and_but(cpumask_of_node(tx2_pmu->node),
cpumask_clear_cpu(cpu, &cpu_online_mask_temp); cpu_online_mask, cpu);
new_cpu = cpumask_any_and(
cpumask_of_node(tx2_pmu->node),
&cpu_online_mask_temp);
tx2_pmu->cpu = new_cpu; tx2_pmu->cpu = new_cpu;
if (new_cpu >= nr_cpu_ids) if (new_cpu >= nr_cpu_ids)
......
...@@ -1102,6 +1102,7 @@ static int xgene_init_perf(struct xgene_pmu_dev *pmu_dev, char *name) ...@@ -1102,6 +1102,7 @@ static int xgene_init_perf(struct xgene_pmu_dev *pmu_dev, char *name)
/* Perf driver registration */ /* Perf driver registration */
pmu_dev->pmu = (struct pmu) { pmu_dev->pmu = (struct pmu) {
.parent = pmu_dev->parent->dev,
.attr_groups = pmu_dev->attr_groups, .attr_groups = pmu_dev->attr_groups,
.task_ctx_nr = perf_invalid_context, .task_ctx_nr = perf_invalid_context,
.pmu_enable = xgene_perf_pmu_enable, .pmu_enable = xgene_perf_pmu_enable,
......
...@@ -405,6 +405,29 @@ unsigned int cpumask_any_but(const struct cpumask *mask, unsigned int cpu) ...@@ -405,6 +405,29 @@ unsigned int cpumask_any_but(const struct cpumask *mask, unsigned int cpu)
return i; return i;
} }
/**
* cpumask_any_and_but - pick a "random" cpu from *mask1 & *mask2, but not this one.
* @mask1: the first input cpumask
* @mask2: the second input cpumask
* @cpu: the cpu to ignore
*
* Returns >= nr_cpu_ids if no cpus set.
*/
static inline
unsigned int cpumask_any_and_but(const struct cpumask *mask1,
const struct cpumask *mask2,
unsigned int cpu)
{
unsigned int i;
cpumask_check(cpu);
i = cpumask_first_and(mask1, mask2);
if (i != cpu)
return i;
return cpumask_next_and(cpu, mask1, mask2);
}
/** /**
* cpumask_nth - get the Nth cpu in a cpumask * cpumask_nth - get the Nth cpu in a cpumask
* @srcp: the cpumask pointer * @srcp: the cpumask pointer
......
...@@ -504,6 +504,22 @@ quiet_cmd_uimage = UIMAGE $@ ...@@ -504,6 +504,22 @@ quiet_cmd_uimage = UIMAGE $@
-a $(UIMAGE_LOADADDR) -e $(UIMAGE_ENTRYADDR) \ -a $(UIMAGE_LOADADDR) -e $(UIMAGE_ENTRYADDR) \
-n '$(UIMAGE_NAME)' -d $< $@ -n '$(UIMAGE_NAME)' -d $< $@
# Flat Image Tree (FIT)
# This allows for packaging of a kernel and all devicetrees files, using
# compression.
# ---------------------------------------------------------------------------
MAKE_FIT := $(srctree)/scripts/make_fit.py
# Use this to override the compression algorithm
FIT_COMPRESSION ?= gzip
quiet_cmd_fit = FIT $@
cmd_fit = $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \
--name '$(UIMAGE_NAME)' \
$(if $(findstring 1,$(KBUILD_VERBOSE)),-v) \
--compress $(FIT_COMPRESSION) -k $< @$(word 2,$^)
# XZ # XZ
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Use xzkern to compress the kernel image and xzmisc to compress other things. # Use xzkern to compress the kernel image and xzmisc to compress other things.
......
#!/usr/bin/env python3
# SPDX-License-Identifier: GPL-2.0+
#
# Copyright 2024 Google LLC
# Written by Simon Glass <sjg@chromium.org>
#
"""Build a FIT containing a lot of devicetree files
Usage:
make_fit.py -A arm64 -n 'Linux-6.6' -O linux
-o arch/arm64/boot/image.fit -k /tmp/kern/arch/arm64/boot/image.itk
@arch/arm64/boot/dts/dtbs-list -E -c gzip
Creates a FIT containing the supplied kernel and a set of devicetree files,
either specified individually or listed in a file (with an '@' prefix).
Use -E to generate an external FIT (where the data is placed after the
FIT data structure). This allows parsing of the data without loading
the entire FIT.
Use -c to compress the data, using bzip2, gzip, lz4, lzma, lzo and
zstd algorithms.
The resulting FIT can be booted by bootloaders which support FIT, such
as U-Boot, Linuxboot, Tianocore, etc.
Note that this tool does not yet support adding a ramdisk / initrd.
"""
import argparse
import collections
import os
import subprocess
import sys
import tempfile
import time
import libfdt
# Tool extension and the name of the command-line tools
CompTool = collections.namedtuple('CompTool', 'ext,tools')
COMP_TOOLS = {
'bzip2': CompTool('.bz2', 'bzip2'),
'gzip': CompTool('.gz', 'pigz,gzip'),
'lz4': CompTool('.lz4', 'lz4'),
'lzma': CompTool('.lzma', 'lzma'),
'lzo': CompTool('.lzo', 'lzop'),
'zstd': CompTool('.zstd', 'zstd'),
}
def parse_args():
"""Parse the program ArgumentParser
Returns:
Namespace object containing the arguments
"""
epilog = 'Build a FIT from a directory tree containing .dtb files'
parser = argparse.ArgumentParser(epilog=epilog, fromfile_prefix_chars='@')
parser.add_argument('-A', '--arch', type=str, required=True,
help='Specifies the architecture')
parser.add_argument('-c', '--compress', type=str, default='none',
help='Specifies the compression')
parser.add_argument('-E', '--external', action='store_true',
help='Convert the FIT to use external data')
parser.add_argument('-n', '--name', type=str, required=True,
help='Specifies the name')
parser.add_argument('-o', '--output', type=str, required=True,
help='Specifies the output file (.fit)')
parser.add_argument('-O', '--os', type=str, required=True,
help='Specifies the operating system')
parser.add_argument('-k', '--kernel', type=str, required=True,
help='Specifies the (uncompressed) kernel input file (.itk)')
parser.add_argument('-v', '--verbose', action='store_true',
help='Enable verbose output')
parser.add_argument('dtbs', type=str, nargs='*',
help='Specifies the devicetree files to process')
return parser.parse_args()
def setup_fit(fsw, name):
"""Make a start on writing the FIT
Outputs the root properties and the 'images' node
Args:
fsw (libfdt.FdtSw): Object to use for writing
name (str): Name of kernel image
"""
fsw.INC_SIZE = 65536
fsw.finish_reservemap()
fsw.begin_node('')
fsw.property_string('description', f'{name} with devicetree set')
fsw.property_u32('#address-cells', 1)
fsw.property_u32('timestamp', int(time.time()))
fsw.begin_node('images')
def write_kernel(fsw, data, args):
"""Write out the kernel image
Writes a kernel node along with the required properties
Args:
fsw (libfdt.FdtSw): Object to use for writing
data (bytes): Data to write (possibly compressed)
args (Namespace): Contains necessary strings:
arch: FIT architecture, e.g. 'arm64'
fit_os: Operating Systems, e.g. 'linux'
name: Name of OS, e.g. 'Linux-6.6.0-rc7'
compress: Compression algorithm to use, e.g. 'gzip'
"""
with fsw.add_node('kernel'):
fsw.property_string('description', args.name)
fsw.property_string('type', 'kernel_noload')
fsw.property_string('arch', args.arch)
fsw.property_string('os', args.os)
fsw.property_string('compression', args.compress)
fsw.property('data', data)
fsw.property_u32('load', 0)
fsw.property_u32('entry', 0)
def finish_fit(fsw, entries):
"""Finish the FIT ready for use
Writes the /configurations node and subnodes
Args:
fsw (libfdt.FdtSw): Object to use for writing
entries (list of tuple): List of configurations:
str: Description of model
str: Compatible stringlist
"""
fsw.end_node()
seq = 0
with fsw.add_node('configurations'):
for model, compat in entries:
seq += 1
with fsw.add_node(f'conf-{seq}'):
fsw.property('compatible', bytes(compat))
fsw.property_string('description', model)
fsw.property_string('fdt', f'fdt-{seq}')
fsw.property_string('kernel', 'kernel')
fsw.end_node()
def compress_data(inf, compress):
"""Compress data using a selected algorithm
Args:
inf (IOBase): Filename containing the data to compress
compress (str): Compression algorithm, e.g. 'gzip'
Return:
bytes: Compressed data
"""
if compress == 'none':
return inf.read()
comp = COMP_TOOLS.get(compress)
if not comp:
raise ValueError(f"Unknown compression algorithm '{compress}'")
with tempfile.NamedTemporaryFile() as comp_fname:
with open(comp_fname.name, 'wb') as outf:
done = False
for tool in comp.tools.split(','):
try:
subprocess.call([tool, '-c'], stdin=inf, stdout=outf)
done = True
break
except FileNotFoundError:
pass
if not done:
raise ValueError(f'Missing tool(s): {comp.tools}\n')
with open(comp_fname.name, 'rb') as compf:
comp_data = compf.read()
return comp_data
def output_dtb(fsw, seq, fname, arch, compress):
"""Write out a single devicetree to the FIT
Args:
fsw (libfdt.FdtSw): Object to use for writing
seq (int): Sequence number (1 for first)
fmame (str): Filename containing the DTB
arch: FIT architecture, e.g. 'arm64'
compress (str): Compressed algorithm, e.g. 'gzip'
Returns:
tuple:
str: Model name
bytes: Compatible stringlist
"""
with fsw.add_node(f'fdt-{seq}'):
# Get the compatible / model information
with open(fname, 'rb') as inf:
data = inf.read()
fdt = libfdt.FdtRo(data)
model = fdt.getprop(0, 'model').as_str()
compat = fdt.getprop(0, 'compatible')
fsw.property_string('description', model)
fsw.property_string('type', 'flat_dt')
fsw.property_string('arch', arch)
fsw.property_string('compression', compress)
fsw.property('compatible', bytes(compat))
with open(fname, 'rb') as inf:
compressed = compress_data(inf, compress)
fsw.property('data', compressed)
return model, compat
def build_fit(args):
"""Build the FIT from the provided files and arguments
Args:
args (Namespace): Program arguments
Returns:
tuple:
bytes: FIT data
int: Number of configurations generated
size: Total uncompressed size of data
"""
seq = 0
size = 0
fsw = libfdt.FdtSw()
setup_fit(fsw, args.name)
entries = []
# Handle the kernel
with open(args.kernel, 'rb') as inf:
comp_data = compress_data(inf, args.compress)
size += os.path.getsize(args.kernel)
write_kernel(fsw, comp_data, args)
for fname in args.dtbs:
# Ignore overlay (.dtbo) files
if os.path.splitext(fname)[1] == '.dtb':
seq += 1
size += os.path.getsize(fname)
model, compat = output_dtb(fsw, seq, fname, args.arch, args.compress)
entries.append([model, compat])
finish_fit(fsw, entries)
# Include the kernel itself in the returned file count
return fsw.as_fdt().as_bytearray(), seq + 1, size
def run_make_fit():
"""Run the tool's main logic"""
args = parse_args()
out_data, count, size = build_fit(args)
with open(args.output, 'wb') as outf:
outf.write(out_data)
ext_fit_size = None
if args.external:
mkimage = os.environ.get('MKIMAGE', 'mkimage')
subprocess.check_call([mkimage, '-E', '-F', args.output],
stdout=subprocess.DEVNULL)
with open(args.output, 'rb') as inf:
data = inf.read()
ext_fit = libfdt.FdtRo(data)
ext_fit_size = ext_fit.totalsize()
if args.verbose:
comp_size = len(out_data)
print(f'FIT size {comp_size:#x}/{comp_size / 1024 / 1024:.1f} MB',
end='')
if ext_fit_size:
print(f', header {ext_fit_size:#x}/{ext_fit_size / 1024:.1f} KB',
end='')
print(f', {count} files, uncompressed {size / 1024 / 1024:.1f} MB')
if __name__ == "__main__":
sys.exit(run_make_fit())
...@@ -701,18 +701,18 @@ ...@@ -701,18 +701,18 @@
* Permission Indirection Extension (PIE) permission encodings. * Permission Indirection Extension (PIE) permission encodings.
* Encodings with the _O suffix, have overlays applied (Permission Overlay Extension). * Encodings with the _O suffix, have overlays applied (Permission Overlay Extension).
*/ */
#define PIE_NONE_O 0x0 #define PIE_NONE_O UL(0x0)
#define PIE_R_O 0x1 #define PIE_R_O UL(0x1)
#define PIE_X_O 0x2 #define PIE_X_O UL(0x2)
#define PIE_RX_O 0x3 #define PIE_RX_O UL(0x3)
#define PIE_RW_O 0x5 #define PIE_RW_O UL(0x5)
#define PIE_RWnX_O 0x6 #define PIE_RWnX_O UL(0x6)
#define PIE_RWX_O 0x7 #define PIE_RWX_O UL(0x7)
#define PIE_R 0x8 #define PIE_R UL(0x8)
#define PIE_GCS 0x9 #define PIE_GCS UL(0x9)
#define PIE_RX 0xa #define PIE_RX UL(0xa)
#define PIE_RW 0xc #define PIE_RW UL(0xc)
#define PIE_RWX 0xe #define PIE_RWX UL(0xe)
#define PIRx_ELx_PERM(idx, perm) ((perm) << ((idx) * 4)) #define PIRx_ELx_PERM(idx, perm) ((perm) << ((idx) * 4))
......
...@@ -262,7 +262,7 @@ static int write_clone_read(void) ...@@ -262,7 +262,7 @@ static int write_clone_read(void)
int main(int argc, char **argv) int main(int argc, char **argv)
{ {
int ret, i; int ret;
putstr("TAP version 13\n"); putstr("TAP version 13\n");
putstr("1.."); putstr("1..");
......
...@@ -6,6 +6,7 @@ ...@@ -6,6 +6,7 @@
#include <stdint.h> #include <stdint.h>
#include <sys/prctl.h> #include <sys/prctl.h>
#include <sys/utsname.h> #include <sys/utsname.h>
#include "../../kselftest.h"
#define SHIFT_TAG(tag) ((uint64_t)(tag) << 56) #define SHIFT_TAG(tag) ((uint64_t)(tag) << 56)
#define SET_TAG(ptr, tag) (((uint64_t)(ptr) & ~SHIFT_TAG(0xff)) | \ #define SET_TAG(ptr, tag) (((uint64_t)(ptr) & ~SHIFT_TAG(0xff)) | \
...@@ -21,6 +22,9 @@ int main(void) ...@@ -21,6 +22,9 @@ int main(void)
if (prctl(PR_SET_TAGGED_ADDR_CTRL, PR_TAGGED_ADDR_ENABLE, 0, 0, 0) == 0) if (prctl(PR_SET_TAGGED_ADDR_CTRL, PR_TAGGED_ADDR_ENABLE, 0, 0, 0) == 0)
tbi_enabled = 1; tbi_enabled = 1;
ptr = (struct utsname *)malloc(sizeof(*ptr)); ptr = (struct utsname *)malloc(sizeof(*ptr));
if (!ptr)
ksft_exit_fail_msg("Failed to allocate utsname buffer\n");
if (tbi_enabled) if (tbi_enabled)
tag = 0x42; tag = 0x42;
ptr = (struct utsname *)SET_TAG(ptr, tag); ptr = (struct utsname *)SET_TAG(ptr, tag);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment