Commits · 7a8e7da42250138adf202ba2596ef7f68651060a · Kirill Smelkov / linux

03 Apr, 2018 23 commits

RISC-V: Fixes to module loading · 7a8e7da4

Palmer Dabbelt authored Apr 02, 2018

This cleans up the module support that was commited earlier to work with
what's actually emitted from our GCC port as it lands upstream.  Most of
the work here is adding new relocations to the kernel.

There's some limitations on module loading imposed by the kernel:

* The kernel doesn't support linker relaxation, which is necessary to
  support R_RISCV_ALIGN.  In order to get reliable module building
  you're going to need to a GCC that supports the new '-mno-relax',
  which IIRC isn't going to be out until 8.1.0.  It's somewhat unlikely
  that R_RISCV_ALIGN will appear in a module even without '-mno-relax'
  support, so issues shouldn't be common.

* There is no large code model for RISC-V, which means modules must be
  loaded within a 32-bit signed offset of the kernel.  We don't
  currently have any mechanism for ensuring this memory remains free or
  moving pages around, so issues here might be common.

I fixed a singcle merge conflict in arch/riscv/kernel/Makefile.

7a8e7da4

RISC-V: Assorted memory model fixes · 2c9046b7

Palmer Dabbelt authored Apr 02, 2018

These fixes fall into three categories

* The definiton of __smp_{store_release,load_acquire}, which allow us to
  emit a full fence when unnecessary.
* Fixes to avoid relying on the behavior of "*.aqrl" atomics, as those
  are specified in the currently released RISC-V memory model draft in
  a way that makes them useless for Linux.  This might change in the
  future, but now the code matches the memory model spec as it's written
  so at least we're getting closer to something sane.  The actual fix is
  to delete the RISC-V specific atomics and drop back to generic
  versions that use the new fences from above.
* Cleanups to our atomic macros, which are mostly non-functional
  changes.

Unfortunately I haven't given these as thorough of a testing as I
probably should have, but I've poked through the code and they seem
generally OK.

2c9046b7

RISC-V: Add dynamic ftrace support for RISC-V platforms · 1cead2d7

Palmer Dabbelt authored Apr 02, 2018

This patch set includes the building blocks of dynamic ftrace features
for RISC-V machines. I'm afraid I'm not very familiar with ftrace, but
the code looks OK to me. It's been used to track down a performance
problem in our SPI driver and appears to work acceptably, but we haven't
given it a whole lot of banging yet so there might still be some bugs
lurking around somewhere.

1cead2d7

RISC-V: Add definition of relocation types · e21d5421

Zong Li authored Mar 15, 2018

Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

e21d5421

RISC-V: Enable module support in defconfig · 4a632cec

Zong Li authored Mar 15, 2018

Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

4a632cec

RISC-V: Support SUB32 relocation type in kernel module · 4aad074c
Zong Li authored Mar 15, 2018
```
Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
```
4aad074c
RISC-V: Support ADD32 relocation type in kernel module · 8e691b16
Zong Li authored Mar 15, 2018
```
Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
```
8e691b16

RISC-V: Support ALIGN relocation type in kernel module · 29e405cd

Zong Li authored Mar 15, 2018

Just fail on align type. Kernel modules loader didn't do relax
like linker, it is difficult to remove or migrate the code,
but the remnant nop instructions harm the performaace of module.
We expect the building module with the no-relax option.
Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

29e405cd

RISC-V: Support RVC_BRANCH/JUMP relocation type in kernel modulewq · 56ea45ae
Zong Li authored Mar 15, 2018
```
Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
```
56ea45ae

RISC-V: Support HI20/LO12_I/LO12_S relocation type in kernel module · e7456e69

Zong Li authored Mar 15, 2018

HI20 and LO12_I/LO12_S relocate the absolute address, the range of
offset must in 32-bit.
Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

e7456e69

RISC-V: Support CALL relocation type in kernel module · e1910c72
Zong Li authored Mar 15, 2018
```
Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
```
e1910c72

RISC-V: Support GOT_HI20/CALL_PLT relocation type in kernel module · da975dd4

Zong Li authored Mar 15, 2018

For CALL_PLT, emit the plt entry only when offset is more than 32-bit.

For PCREL_LO12, it uses the location of corresponding HI20 to
get the address of external symbol. It should check the HI20 type
is the PCREL_HI20 or GOT_HI20, because sometime the location will
have two or more relocation types.
For example:
0:   00000797                auipc   a5,0x0
                     0: R_RISCV_ALIGN        *ABS*
                     0: R_RISCV_GOT_HI20     SYMBOL
4:   0007b783                ld      a5,0(a5) # 0 <SYMBOL>
                     4: R_RISCV_PCREL_LO12_I .L0
                     4: R_RISCV_RELAX        *ABS*
Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

da975dd4

RISC-V: Add section of GOT.PLT for kernel module · b8bde0ef

Zong Li authored Mar 15, 2018

Separate the function symbol address from .plt to .got.plt section.

The original plt entry has trampoline code with symbol address,
there is a 32-bit padding bwtween jar instruction and symbol address.

Extract the symbol address to .got.plt to reduce the module size.
Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

b8bde0ef

RISC-V: Add sections of PLT and GOT for kernel module · ab1ef68e

Zong Li authored Mar 15, 2018

The address of external symbols will locate more than 32-bit offset
in 64-bit kernel with sv39 or sv48 virtual addressing.

Module loader emits the GOT and PLT entries for data symbols and
function symbols respectively.

The PLT entry is a trampoline code for jumping to the 64-bit
real address. The GOT entry is just the data symbol address.
Signed-off-by: Zong Li <zong@andestech.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

ab1ef68e

riscv/atomic: Strengthen implementations with fences · 5ce6c1f3

Andrea Parri authored Mar 09, 2018

Atomics present the same issue with locking: release and acquire
variants need to be strengthened to meet the constraints defined
by the Linux-kernel memory consistency model [1].

Atomics present a further issue: implementations of atomics such
as atomic_cmpxchg() and atomic_add_unless() rely on LR/SC pairs,
which do not give full-ordering with .aqrl; for example, current
implementations allow the "lr-sc-aqrl-pair-vs-full-barrier" test
below to end up with the state indicated in the "exists" clause.

In order to "synchronize" LKMM and RISC-V's implementation, this
commit strengthens the implementations of the atomics operations
by replacing .rl and .aq with the use of ("lightweigth") fences,
and by replacing .aqrl LR/SC pairs in sequences such as:

  0:      lr.w.aqrl  %0, %addr
          bne        %0, %old, 1f
          ...
          sc.w.aqrl  %1, %new, %addr
          bnez       %1, 0b
  1:

with sequences of the form:

  0:      lr.w       %0, %addr
          bne        %0, %old, 1f
          ...
          sc.w.rl    %1, %new, %addr   /* SC-release   */
          bnez       %1, 0b
          fence      rw, rw            /* "full" fence */
  1:

following Daniel's suggestion.

These modifications were validated with simulation of the RISC-V
memory consistency model.

C lr-sc-aqrl-pair-vs-full-barrier

{}

P0(int *x, int *y, atomic_t *u)
{
	int r0;
	int r1;

	WRITE_ONCE(*x, 1);
	r0 = atomic_cmpxchg(u, 0, 1);
	r1 = READ_ONCE(*y);
}

P1(int *x, int *y, atomic_t *v)
{
	int r0;
	int r1;

	WRITE_ONCE(*y, 1);
	r0 = atomic_cmpxchg(v, 0, 1);
	r1 = READ_ONCE(*x);
}

exists (u=1 /\ v=1 /\ 0:r1=0 /\ 1:r1=0)

[1] https://marc.info/?l=linux-kernel&m=151930201102853&w=2
    https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/hKywNHBkAXM
    https://marc.info/?l=linux-kernel&m=151633436614259&w=2Suggested-by: Daniel Lustig <dlustig@nvidia.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Cc: Palmer Dabbelt <palmer@sifive.com>
Cc: Albert Ou <albert@sifive.com>
Cc: Daniel Lustig <dlustig@nvidia.com>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Jade Alglave <j.alglave@ucl.ac.uk>
Cc: Luc Maranget <luc.maranget@inria.fr>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Akira Yokosawa <akiyks@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: linux-riscv@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

5ce6c1f3

riscv/spinlock: Strengthen implementations with fences · 0123f4d7

Andrea Parri authored Mar 09, 2018

Current implementations map locking operations using .rl and .aq
annotations.  However, this mapping is unsound w.r.t. the kernel
memory consistency model (LKMM) [1]:

Referring to the "unlock-lock-read-ordering" test reported below,
Daniel wrote:

  "I think an RCpc interpretation of .aq and .rl would in fact
   allow the two normal loads in P1 to be reordered [...]

   The intuition would be that the amoswap.w.aq can forward from
   the amoswap.w.rl while that's still in the store buffer, and
   then the lw x3,0(x4) can also perform while the amoswap.w.rl
   is still in the store buffer, all before the l1 x1,0(x2)
   executes.  That's not forbidden unless the amoswaps are RCsc,
   unless I'm missing something.

   Likewise even if the unlock()/lock() is between two stores.
   A control dependency might originate from the load part of
   the amoswap.w.aq, but there still would have to be something
   to ensure that this load part in fact performs after the store
   part of the amoswap.w.rl performs globally, and that's not
   automatic under RCpc."

Simulation of the RISC-V memory consistency model confirmed this
expectation.

In order to "synchronize" LKMM and RISC-V's implementation, this
commit strengthens the implementations of the locking operations
by replacing .rl and .aq with the use of ("lightweigth") fences,
resp., "fence rw,  w" and "fence r , rw".

C unlock-lock-read-ordering

{}
/* s initially owned by P1 */

P0(int *x, int *y)
{
        WRITE_ONCE(*x, 1);
        smp_wmb();
        WRITE_ONCE(*y, 1);
}

P1(int *x, int *y, spinlock_t *s)
{
        int r0;
        int r1;

        r0 = READ_ONCE(*y);
        spin_unlock(s);
        spin_lock(s);
        r1 = READ_ONCE(*x);
}

exists (1:r0=1 /\ 1:r1=0)

[1] https://marc.info/?l=linux-kernel&m=151930201102853&w=2
    https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/hKywNHBkAXM
    https://marc.info/?l=linux-kernel&m=151633436614259&w=2Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Cc: Palmer Dabbelt <palmer@sifive.com>
Cc: Albert Ou <albert@sifive.com>
Cc: Daniel Lustig <dlustig@nvidia.com>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Jade Alglave <j.alglave@ucl.ac.uk>
Cc: Luc Maranget <luc.maranget@inria.fr>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Akira Yokosawa <akiyks@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: linux-riscv@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

0123f4d7

riscv/barrier: Define __smp_{store_release,load_acquire} · 8d235b17

Andrea Parri authored Feb 27, 2018

Introduce __smp_{store_release,load_acquire}, and rely on the generic
definitions for smp_{store_release,load_acquire}. This avoids the use
of full ("rw,rw") fences on SMP.
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>

8d235b17