1. 11 Nov, 2016 13 commits
    • Mark Rutland's avatar
      arm64: split thread_info from task stack · c02433dd
      Mark Rutland authored
      This patch moves arm64's struct thread_info from the task stack into
      task_struct. This protects thread_info from corruption in the case of
      stack overflows, and makes its address harder to determine if stack
      addresses are leaked, making a number of attacks more difficult. Precise
      detection and handling of overflow is left for subsequent patches.
      
      Largely, this involves changing code to store the task_struct in sp_el0,
      and acquire the thread_info from the task struct. Core code now
      implements current_thread_info(), and as noted in <linux/sched.h> this
      relies on offsetof(task_struct, thread_info) == 0, enforced by core
      code.
      
      This change means that the 'tsk' register used in entry.S now points to
      a task_struct, rather than a thread_info as it used to. To make this
      clear, the TI_* field offsets are renamed to TSK_TI_*, with asm-offsets
      appropriately updated to account for the structural change.
      
      Userspace clobbers sp_el0, and we can no longer restore this from the
      stack. Instead, the current task is cached in a per-cpu variable that we
      can safely access from early assembly as interrupts are disabled (and we
      are thus not preemptible).
      
      Both secondary entry and idle are updated to stash the sp and task
      pointer separately.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Tested-by: default avatarLaura Abbott <labbott@redhat.com>
      Cc: AKASHI Takahiro <takahiro.akashi@linaro.org>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
      Cc: James Morse <james.morse@arm.com>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      c02433dd
    • Mark Rutland's avatar
      arm64: assembler: introduce ldr_this_cpu · 1b7e2296
      Mark Rutland authored
      Shortly we will want to load a percpu variable in the return from
      userspace path. We can save an instruction by folding the addition of
      the percpu offset into the load instruction, and this patch adds a new
      helper to do so.
      
      At the same time, we clean up this_cpu_ptr for consistency. As with
      {adr,ldr,str}_l, we change the template to take the destination register
      first, and name this dst. Secondly, we rename the macro to adr_this_cpu,
      following the scheme of adr_l, and matching the newly added
      ldr_this_cpu.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Tested-by: default avatarLaura Abbott <labbott@redhat.com>
      Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
      Cc: James Morse <james.morse@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      1b7e2296
    • Mark Rutland's avatar
      arm64: make cpu number a percpu variable · 57c82954
      Mark Rutland authored
      In the absence of CONFIG_THREAD_INFO_IN_TASK, core code maintains
      thread_info::cpu, and low-level architecture code can access this to
      build raw_smp_processor_id(). With CONFIG_THREAD_INFO_IN_TASK, core code
      maintains task_struct::cpu, which for reasons of hte header soup is not
      accessible to low-level arch code.
      
      Instead, we can maintain a percpu variable containing the cpu number.
      
      For both the old and new implementation of raw_smp_processor_id(), we
      read a syreg into a GPR, add an offset, and load the result. As the
      offset is now larger, it may not be folded into the load, but otherwise
      the assembly shouldn't change much.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Tested-by: default avatarLaura Abbott <labbott@redhat.com>
      Cc: James Morse <james.morse@arm.com>
      Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      57c82954
    • Mark Rutland's avatar
      arm64: smp: prepare for smp_processor_id() rework · 580efaa7
      Mark Rutland authored
      Subsequent patches will make smp_processor_id() use a percpu variable.
      This will make smp_processor_id() dependent on the percpu offset, and
      thus we cannot use smp_processor_id() to figure out what to initialise
      the offset to.
      
      Prepare for this by initialising the percpu offset based on
      current::cpu, which will work regardless of how smp_processor_id() is
      implemented. Also, make this relationship obvious by placing this code
      together at the start of secondary_start_kernel().
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Tested-by: default avatarLaura Abbott <labbott@redhat.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      580efaa7
    • Mark Rutland's avatar
      arm64: move sp_el0 and tpidr_el1 into cpu_suspend_ctx · 623b476f
      Mark Rutland authored
      When returning from idle, we rely on the fact that thread_info lives at
      the end of the kernel stack, and restore this by masking the saved stack
      pointer. Subsequent patches will sever the relationship between the
      stack and thread_info, and to cater for this we must save/restore sp_el0
      explicitly, storing it in cpu_suspend_ctx.
      
      As cpu_suspend_ctx must be doubleword aligned, this leaves us with an
      extra slot in cpu_suspend_ctx. We can use this to save/restore tpidr_el1
      in the same way, which simplifies the code, avoiding pointer chasing on
      the restore path (as we no longer need to load thread_info::cpu followed
      by the relevant slot in __per_cpu_offset based on this).
      
      This patch stashes both registers in cpu_suspend_ctx.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Tested-by: default avatarLaura Abbott <labbott@redhat.com>
      Cc: James Morse <james.morse@arm.com>
      Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      623b476f
    • Mark Rutland's avatar
      arm64: prep stack walkers for THREAD_INFO_IN_TASK · 9bbd4c56
      Mark Rutland authored
      When CONFIG_THREAD_INFO_IN_TASK is selected, task stacks may be freed
      before a task is destroyed. To account for this, the stacks are
      refcounted, and when manipulating the stack of another task, it is
      necessary to get/put the stack to ensure it isn't freed and/or re-used
      while we do so.
      
      This patch reworks the arm64 stack walking code to account for this.
      When CONFIG_THREAD_INFO_IN_TASK is not selected these perform no
      refcounting, and this should only be a structural change that does not
      affect behaviour.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Tested-by: default avatarLaura Abbott <labbott@redhat.com>
      Cc: AKASHI Takahiro <takahiro.akashi@linaro.org>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: James Morse <james.morse@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      9bbd4c56
    • Mark Rutland's avatar
      arm64: unexport walk_stackframe · 2020a5ae
      Mark Rutland authored
      The walk_stackframe functions is architecture-specific, with a varying
      prototype, and common code should not use it directly. None of its
      current users can be built as modules. With THREAD_INFO_IN_TASK, users
      will also need to hold a stack reference before calling it.
      
      There's no reason for it to be exported, and it's very easy to misuse,
      so unexport it for now.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      2020a5ae
    • Mark Rutland's avatar
      arm64: traps: simplify die() and __die() · 876e7a38
      Mark Rutland authored
      In arm64's die and __die routines we pass around a thread_info, and
      subsequently use this to determine the relevant task_struct, and the end
      of the thread's stack. Subsequent patches will decouple thread_info from
      the stack, and this approach will no longer work.
      
      To figure out the end of the stack, we can use the new generic
      end_of_stack() helper. As we only call __die() from die(), and die()
      always deals with the current task, we can remove the parameter and have
      both acquire current directly, which also makes it clear that __die
      can't be called for arbitrary tasks.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Tested-by: default avatarLaura Abbott <labbott@redhat.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      876e7a38
    • Mark Rutland's avatar
      arm64: factor out current_stack_pointer · a9ea0017
      Mark Rutland authored
      We define current_stack_pointer in <asm/thread_info.h>, though other
      files and header relying upon it do not have this necessary include, and
      are thus fragile to changes in the header soup.
      
      Subsequent patches will affect the header soup such that directly
      including <asm/thread_info.h> may result in a circular header include in
      some of these cases, so we can't simply include <asm/thread_info.h>.
      
      Instead, factor current_thread_info into its own header, and have all
      existing users include this explicitly.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Tested-by: default avatarLaura Abbott <labbott@redhat.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      a9ea0017
    • Mark Rutland's avatar
      arm64: asm-offsets: remove unused definitions · 3fe12da4
      Mark Rutland authored
      Subsequent patches will move the thread_info::{task,cpu} fields, and the
      current TI_{TASK,CPU} offset definitions are not used anywhere.
      
      This patch removes the redundant definitions.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Tested-by: default avatarLaura Abbott <labbott@redhat.com>
      Cc: James Morse <james.morse@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      3fe12da4
    • Mark Rutland's avatar
      arm64: thread_info remove stale items · dcbe0285
      Mark Rutland authored
      We have a comment claiming __switch_to() cares about where cpu_context
      is located relative to cpu_domain in thread_info. However arm64 has
      never had a thread_info::cpu_domain field, and neither __switch_to nor
      cpu_switch_to care where the cpu_context field is relative to others.
      
      Additionally, the init_thread_info alias is never used anywhere in the
      kernel, and will shortly become problematic when thread_info is moved
      into task_struct.
      
      This patch removes both.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Tested-by: default avatarLaura Abbott <labbott@redhat.com>
      Cc: James Morse <james.morse@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      dcbe0285
    • Mark Rutland's avatar
      thread_info: include <current.h> for THREAD_INFO_IN_TASK · dc3d2a67
      Mark Rutland authored
      When CONFIG_THREAD_INFO_IN_TASK is selected, the current_thread_info()
      macro relies on current having been defined prior to its use. However,
      not all users of current_thread_info() include <asm/current.h>, and thus
      current is not guaranteed to be defined.
      
      When CONFIG_THREAD_INFO_IN_TASK is not selected, it's possible that
      get_current() / current are based upon current_thread_info(), and
      <asm/current.h> includes <asm/thread_info.h>. Thus always including
      <asm/current.h> would result in circular dependences on some platforms.
      
      To ensure both cases work, this patch includes <asm/current.h>, but only
      when CONFIG_THREAD_INFO_IN_TASK is selected.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Acked-by: default avatarHeiko Carstens <heiko.carstens@de.ibm.com>
      Reviewed-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Kees Cook <keescook@chromium.org>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      dc3d2a67
    • Mark Rutland's avatar
      thread_info: factor out restart_block · 53d74d05
      Mark Rutland authored
      Since commit f56141e3 ("all arches, signal: move restart_block
      to struct task_struct"), thread_info and restart_block have been
      logically distinct, yet struct restart_block is still defined in
      <linux/thread_info.h>.
      
      At least one architecture (erroneously) uses restart_block as part of
      its thread_info, and thus the definition of restart_block must come
      before the include of <asm/thread_info>. Subsequent patches in this
      series need to shuffle the order of includes and definitions in
      <linux/thread_info.h>, and will make this ordering fragile.
      
      This patch moves the definition of restart_block out to its own header.
      This serves as generic cleanup, logically separating thread_info and
      restart_block, and also makes it easier to avoid fragility.
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Reviewed-by: default avatarAndy Lutomirski <luto@kernel.org>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
      Cc: Kees Cook <keescook@chromium.org>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      53d74d05
  2. 09 Nov, 2016 4 commits
  3. 07 Nov, 2016 19 commits
  4. 05 Nov, 2016 4 commits