In order to allow the simplification of way we declare hwcaps annotate
most of the unsigned fields in the identification registers as such. This
is not a complete annotation, it does cover all the cases where we already
annotate signedness of the field in the hwcaps and some others which I
happened to look at and seemed clear but there will be more and nothing
outside the identification registers was even looked at.

Other fields can be annotated as incrementally as people have the time and
need to do so.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221207-arm64-sysreg-helpers-v4-3-25b6b3fb9d18@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

We currently annotate a few bitfields as signed in hwcaps, update all of
these to be SignedEnum in the sysreg generation.  Further signed bitfields
can be done incrementally, this is the minimum required for the conversion
of the hwcaps to use token pasting to simplify their declaration.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221207-arm64-sysreg-helpers-v4-2-25b6b3fb9d18@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Many of our enumerations follow a standard scheme where the values can be
treated as signed however there are some where the value must be treated
as signed and others that are simple enumerations where there is no clear
ordering to the values. Provide new field types SignedEnum and
UnsignedEnum which allows the signedness to be specified in the sysreg
definition and emit a REG_FIELD_SIGNED define for these which is a
boolean corresponding to our current FTR_UNSIGNED and FTR_SIGNED macros.

Existing Enums will need to be converted, since these do not have a
define generated anyone wishing to use the sign of one of these will
need to explicitly annotate that field so nothing should start going
wrong by default.
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221207-arm64-sysreg-helpers-v4-1-25b6b3fb9d18@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Merge branches 'for-next/sysreg', 'for-next/compat-hwcap' and 'for-next/sme2' into for-next/sysreg-hwcaps

Patches on this branch depend on the branches merged above.

The newly added zt-test program copied the pattern from the other FP
stress test programs of having a redundant _start label which is
rejected by clang, as we did in a parallel series for the other tests
remove the label so we can build with clang.

No functional change.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20230130-arm64-fix-sme2-clang-v1-1-3ce81d99ea8f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

This hwcap was added for 32-bit native arm kernel by commit fea53546
("ARM: 9274/1: Add hwcap for Speculative Store Bypassing Safe") and hence
the corresponding changes added in 32-bit compat arm64 for similar user
interfaces.

Speculative Store Bypass Safe is a feature(FEAT_SSBS) present in
AArch32/AArch64 state for Armv8 and can be identified by PFR2.SSBS
identification register. This hwcap is already advertised in native arm64
kernel.
Signed-off-by: Amit Daniel Kachhap <amit.kachhap@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20230111053706.13994-8-amit.kachhap@arm.comSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

This hwcap was added for 32-bit native arm kernel by commit
3bda6d88 ("ARM: 9273/1: Add hwcap for Speculation Barrier(SB)")
and hence the corresponding changes added in 32-bit compat arm64 kernel.

Speculation Barrier is a feature(FEAT_SB) present in both AArch32 and
AArch64 state. This hwcap is already advertised in native arm64 kernel.
Signed-off-by: Amit Daniel Kachhap <amit.kachhap@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20230111053706.13994-7-amit.kachhap@arm.comSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

This hwcap was added earlier for 32-bit native arm kernel by commit
956ca3a4 ("ARM: 9272/1: vfp: Add hwcap for FEAT_AA32I8MM") and hence
the corresponding changes added in 32-bit compat arm64 kernel for similar
user interfaces.

Int8 matrix multiplication is a feature (FEAT_AA32I8MM) present in AArch32
state of Armv8 and is identified by ISAR6.I8MM register. Similar
feature(FEAT_I8MM) exist for AArch64 state and is already advertised in
arm64 kernel.
Signed-off-by: Amit Daniel Kachhap <amit.kachhap@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20230111053706.13994-6-amit.kachhap@arm.comSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

This hwcap was added earlier for 32-bit native arm kernel by commit
23b6d4ad ("ARM: 9271/1: vfp: Add hwcap for FEAT_AA32BF16") and hence
the corresponding changes added in 32-bit compat arm64 kernel.

Brain 16-bit floating-point storage format is a feature (FEAT_AA32BF16)
present in AArch32 state for Armv8 and is represented by ISAR6.BF16
identification register. Similar feature (FEAT_BF16) exist for AArch64
state and is already advertised in native arm64 kernel.
Signed-off-by: Amit Daniel Kachhap <amit.kachhap@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20230111053706.13994-5-amit.kachhap@arm.comSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

This hwcap was added earlier for 32-bit native arm kernel by commit
ce483549 ("ARM: 9270/1: vfp: Add hwcap for FEAT_FHM") and hence the
corresponding changes added in 32-bit compat arm64 kernel for similar user
interfaces.

Floating-point half-precision multiplication (FHM) is a feature present
in AArch32/AArch64 state for Armv8. This hwcap is already advertised in
native arm64 kernel.
Signed-off-by: Amit Daniel Kachhap <amit.kachhap@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20230111053706.13994-4-amit.kachhap@arm.comSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

This hwcap was added earlier for 32-bit native arm kernel by commit
62ea0d87 ("ARM: 9269/1: vfp: Add hwcap for FEAT_DotProd") and hence the
corresponding changes added in 32-bit compat arm64 kernel for similar user
interfaces.

Advanced Dot product is a feature (FEAT_DotProd) present in both
AArch32/AArch64 state for Armv8 and is already advertised in native arm64
kernel.
Signed-off-by: Amit Daniel Kachhap <amit.kachhap@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20230111053706.13994-3-amit.kachhap@arm.comSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

These hwcaps were added earlier for 32-bit native arm kernel by commit
c00a19c8 ("ARM: 9268/1: vfp: Add hwcap FPHP and ASIMDHP for FEAT_FP16")
and hence the corresponding changes added in 32-bit compat arm64 kernel for
similar userspace interfaces.

Floating point half-precision (FPHP) and Advanced SIMD half-precision
(ASIMDHP) represents the Armv8 FP16 feature extension and is already
advertised in native arm64 kernel.
Signed-off-by: Amit Daniel Kachhap <amit.kachhap@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20230111053706.13994-2-amit.kachhap@arm.comSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Add the hwcaps defined by SME 2 and 2.1 to the hwcaps test.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-21-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Add coverage of the ZT ptrace interface.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-20-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Verify that ZT0 is preserved over syscalls when it is present and
PSTATE.ZA is set.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-19-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

We should have a ZT register frame with an expected size when ZA is enabled
and have no ZT frame when ZA is disabled. Since we don't load any data into
ZT we expect the data to all be zeros since the architecture guarantees it
will be set to 0 as ZA is enabled.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-18-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Add ZT to the set of signal contexts that the shared code understands and
validates the form of.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-17-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Support test cases for SME2 by adding it to the set of features that we
enumerate so test cases can check for it.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-16-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Hook up the newly added zt-test program in the FPSIMD stress tests, start
a copy per CPU when SME2 is supported.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-15-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Following the pattern for the other register sets add a stress test program
for ZT0 which continually loads and verifies patterns in the register in
an effort to discover context switching problems.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-14-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

In order to allow userspace to discover the presence of the new SME features
add hwcaps for them.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-13-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Implement support for a new note type NT_ARM64_ZT providing access to
ZT0 when implemented. Since ZT0 is a register with constant size this is
much simpler than for other SME state.

As ZT0 is only accessible when PSTATE.ZA is set writes to ZT0 cause
PSTATE.ZA to be set, the main alternative would be to return -EBUSY in
this case but this seemed more constructive. Practical users are also
going to be working with ZA anyway and have some understanding of the
state.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-12-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Add a new signal context type for ZT which is present in the signal frame
when ZA is enabled and ZT is supported by the system. In order to account
for the possible addition of further ZT registers in the future we make the
number of registers variable in the ABI, though currently the only possible
number is 1. We could just use a bare list head for the context since the
number of registers can be inferred from the size of the context but for
usability and future extensibility we define a header with the number of
registers and some reserved fields in it.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-11-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

When the system supports SME2 the ZT0 register must be context switched as
part of the floating point state. This register is stored immediately
after ZA in memory and is only accessible when PSTATE.ZA is set so we
handle it in the same functions we use to save and restore ZA.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-10-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

When the system supports SME2 there is an additional register ZT0 which
we must store when the task is using SME. Since ZT0 is accessible only
when PSTATE.ZA is set just like ZA we allocate storage for it along with
ZA, increasing the allocation size for the memory region where we store
ZA and storing the data for ZT after that for ZA.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-9-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Add basic feature detection for SME2, detecting that the feature is present
and disabling traps for ZT0.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-8-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

The new register ZT0 introduced by SME2 comes with a new trap, disable it
for the host kernel so that we can implement support for it.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-7-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

In order to avoid unrealistic toolchain requirements we manually encode the
instructions for loading and storing ZT0.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-6-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

SME2 defines a new ISS code for use when trapping acesses to ZT0, add a
definition for it.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-5-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

As well as a number of simple features which only add new instructions and
require corresponding hwcaps SME2 introduces a new register ZT0 for which
we must define ABI. Fortunately this is a fixed size 512 bits and therefore
much more straightforward than the base SME state, the only wrinkle is that
it is only accessible when ZA is accessible.

While there is only a single register the architecture is written with a
view to exensibility, including a number in the name, so follow this in the
ABI.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-4-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

FEAT_SME2 and FEAT_SME2P1 introduce several new SME features which can
be enumerated via ID_AA64SMFR0_EL1 and a new register ZT0 access to
which is controlled via SMCR_ELn, add the relevant register description.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-3-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

SME 2 introduces the new ZT0 register, we require that access to this
reigster is not trapped when we identify that the feature is supported.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-2-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

In preparation for adding support for storage for ZT0 to the thread_struct
rename za_state to sme_state. Since ZT0 is accessible when PSTATE.ZA is
set just like ZA itself we will extend the allocation done for ZA to
cover it, avoiding the need to further expand task_struct for non-SME
tasks.

No functional changes.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-sme2-v4-1-f2fa0aef982f@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Now that we are generating ISR_EL1 we have acquired a constant for
ISR_EL1.A, use it rather than the magic number we had been using in the KVM
entry code.
Suggested-by: Marc Zyngier <maz@kernel.org>
Acked-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-isr-el1-v2-3-89f7073a1ca9@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Add a definition of ISR_EL1 as per DDI0487I.a. This register was not
previously defined in sysreg.h, no functional changes.
Acked-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-isr-el1-v2-2-89f7073a1ca9@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

FEAT_NMI adds a new interrupt status register for NMIs, ICC_NMIAR1_EL1.
Add the definition for this register as per IHI0069H.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221208-arm64-isr-el1-v2-1-89f7073a1ca9@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

The ARM64_FEATURE_MASK(), used extensively by KVM, assumes that all ID
register fields are 4 bits wide but this is not the case any more, for
example there are several 1 bit fields in ID_AA64SMFR0_EL1. Fortunately
we now have generated constants for all the ID mask registers which can
be used instead. Rather than create churn from updating existing users
update the macro to reference the generated constants and replace the
comment with a note advising against adding new users.

There are also users of ARM64_FEATURE_FIELD_BITS in the pKVM code which
will need to be fixed separately, since no relevant feature is planned to
be exposed to protected guests in the immediate future there is no
immediate issue with them assuming fields are 4 bits wide.
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221222-arm64-arm64-feature-mask-v1-1-c34c1e177f90@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

The recently converted 32 bit ID registers have errors in MVFR0_EL1.FPSP,
MVFR0_EL1.SIMDReg and MVFR1_EL1.SIMDHP where enumeration values which
should be 0b0010 are specified as 0b0001. Correct these.

Fixes: e79c94a2 ("arm64/sysreg: Convert MVFR0_EL1 to automatic generation")
Fixes: c9b718ed ("arm64/sysreg: Convert MVFR1_EL1 to automatic generation")
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221207-arm64-sysreg-helpers-v3-2-0d71a7b174a8@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>

Since it was added our hwcap for DIT has specified that DIT is a signed
field but this appears to be incorrect, the two values for the enumeration
are:

	0b0000	NI
	0b0001	IMP

which look like a normal unsigned enumeration and the in-kernel DIT usage
added by 01ab991f ("arm64: Enable data independent timing (DIT) in the
kernel") detects the feature with an unsigned enum. Fix the hwcap to specify
the field as unsigned.

Fixes: 7206dc93 ("arm64: Expose Arm v8.4 features")
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20221207-arm64-sysreg-helpers-v3-1-0d71a7b174a8@kernel.orgSigned-off-by: Catalin Marinas <catalin.marinas@arm.com>