1. 26 Sep, 2017 15 commits
    • Eric Biggers's avatar
      x86/fpu: Eliminate the 'xfeatures' local variable in copy_user_to_xstate() · 3d703477
      Eric Biggers authored
      We now have this field in hdr.xfeatures.
      Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: Kevin Hao <haokexin@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Michael Halcrow <mhalcrow@google.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Wanpeng Li <wanpeng.li@hotmail.com>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Cc: kernel-hardening@lists.openwall.com
      Link: http://lkml.kernel.org/r/20170924105913.9157-9-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      3d703477
    • Eric Biggers's avatar
      x86/fpu: Copy the full header in copy_user_to_xstate() · af2c4322
      Eric Biggers authored
      This is in preparation to verify the full xstate header as supplied by user-space.
      Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: Kevin Hao <haokexin@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Michael Halcrow <mhalcrow@google.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Wanpeng Li <wanpeng.li@hotmail.com>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Cc: kernel-hardening@lists.openwall.com
      Link: http://lkml.kernel.org/r/20170924105913.9157-8-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      af2c4322
    • Eric Biggers's avatar
      x86/fpu: Use validate_xstate_header() to validate the xstate_header in copy_kernel_to_xstate() · af95774b
      Eric Biggers authored
      Tighten the checks in copy_kernel_to_xstate().
      Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: Kevin Hao <haokexin@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Michael Halcrow <mhalcrow@google.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Wanpeng Li <wanpeng.li@hotmail.com>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Cc: kernel-hardening@lists.openwall.com
      Link: http://lkml.kernel.org/r/20170924105913.9157-7-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      af95774b
    • Eric Biggers's avatar
      x86/fpu: Eliminate the 'xfeatures' local variable in copy_kernel_to_xstate() · b89eda48
      Eric Biggers authored
      We have this information in the xstate_header.
      Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: Kevin Hao <haokexin@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Michael Halcrow <mhalcrow@google.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Wanpeng Li <wanpeng.li@hotmail.com>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Cc: kernel-hardening@lists.openwall.com
      Link: http://lkml.kernel.org/r/20170924105913.9157-6-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      b89eda48
    • Eric Biggers's avatar
      x86/fpu: Copy the full state_header in copy_kernel_to_xstate() · 80d8ae86
      Eric Biggers authored
      This is in preparation to verify the full xstate header as supplied by user-space.
      Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: Kevin Hao <haokexin@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Michael Halcrow <mhalcrow@google.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Wanpeng Li <wanpeng.li@hotmail.com>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Cc: kernel-hardening@lists.openwall.com
      Link: http://lkml.kernel.org/r/20170924105913.9157-5-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      80d8ae86
    • Eric Biggers's avatar
      x86/fpu: Use validate_xstate_header() to validate the xstate_header in __fpu__restore_sig() · b11e2e18
      Eric Biggers authored
      Tighten the checks in __fpu__restore_sig() and update comments.
      Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: Kevin Hao <haokexin@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Michael Halcrow <mhalcrow@google.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Wanpeng Li <wanpeng.li@hotmail.com>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Cc: kernel-hardening@lists.openwall.com
      Link: http://lkml.kernel.org/r/20170924105913.9157-4-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      b11e2e18
    • Eric Biggers's avatar
      x86/fpu: Use validate_xstate_header() to validate the xstate_header in xstateregs_set() · cf9df81b
      Eric Biggers authored
      Tighten the checks in xstateregs_set().
      Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: Kevin Hao <haokexin@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Michael Halcrow <mhalcrow@google.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Wanpeng Li <wanpeng.li@hotmail.com>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Cc: kernel-hardening@lists.openwall.com
      Link: http://lkml.kernel.org/r/20170924105913.9157-3-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      cf9df81b
    • Eric Biggers's avatar
      x86/fpu: Introduce validate_xstate_header() · e63e5d5c
      Eric Biggers authored
      Move validation of user-supplied xstate_header into a helper function,
      in preparation of calling it from both the ptrace and sigreturn syscall
      paths.
      
      The new function also considers it to be an error if *any* reserved bits
      are set, whereas before we were just clearing most of them silently.
      
      This should reduce the chance of bugs that fail to correctly validate
      user-supplied XSAVE areas.  It also will expose any broken userspace
      programs that set the other reserved bits; this is desirable because
      such programs will lose compatibility with future CPUs and kernels if
      those bits are ever used for anything.  (There shouldn't be any such
      programs, and in fact in the case where the compacted format is in use
      we were already validating xfeatures.  But you never know...)
      Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: Kevin Hao <haokexin@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Michael Halcrow <mhalcrow@google.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Wanpeng Li <wanpeng.li@hotmail.com>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Cc: kernel-hardening@lists.openwall.com
      Link: http://lkml.kernel.org/r/20170924105913.9157-2-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      e63e5d5c
    • Ingo Molnar's avatar
      x86/fpu: Rename fpu__activate_fpstate_read/write() to fpu__prepare_[read|write]() · 369a036d
      Ingo Molnar authored
      As per the new nomenclature we don't 'activate' the FPU state
      anymore, we initialize it. So drop the _activate_fpstate name
      from these functions, which were a bit of a mouthful anyway,
      and name them:
      
      	fpu__prepare_read()
      	fpu__prepare_write()
      
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
      369a036d
    • Ingo Molnar's avatar
      x86/fpu: Rename fpu__activate_curr() to fpu__initialize() · 2ce03d85
      Ingo Molnar authored
      Rename this function to better express that it's all about
      initializing the FPU state of a task which goes hand in hand
      with the fpu::initialized field.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-33-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      2ce03d85
    • Ingo Molnar's avatar
      x86/fpu: Simplify and speed up fpu__copy() · e10078eb
      Ingo Molnar authored
      fpu__copy() has a preempt_disable()/enable() pair, which it had to do to
      be able to atomically unlazy the current task when doing an FNSAVE.
      
      But we don't unlazy tasks anymore, we always do direct saves/restores of
      FPU context.
      
      So remove both the unnecessary critical section, and update the comments.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-32-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      e10078eb
    • Ingo Molnar's avatar
      x86/fpu: Fix stale comments about lazy FPU logic · 7f1487c5
      Ingo Molnar authored
      We don't do any lazy restore anymore, what we have are two pieces of optimization:
      
       - no-FPU tasks that don't save/restore the FPU context (kernel threads are such)
      
       - cached FPU registers maintained via the fpu->last_cpu field. This means that
         if an FPU task context switches to a non-FPU task then we can maintain the
         FPU registers as an in-FPU copies (cache), and skip the restoration of them
         once we switch back to the original FPU-using task.
      
      Update all the comments that still referred to old 'lazy' and 'unlazy' concepts.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-31-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      7f1487c5
    • Ingo Molnar's avatar
      x86/fpu: Rename fpu::fpstate_active to fpu::initialized · e4a81bfc
      Ingo Molnar authored
      The x86 FPU code used to have a complex state machine where both the FPU
      registers and the FPU state context could be 'active' (or inactive)
      independently of each other - which enabled features like lazy FPU restore.
      
      Much of this complexity is gone in the current code: now we basically can
      have FPU-less tasks (kernel threads) that don't use (and save/restore) FPU
      state at all, plus full FPU users that save/restore directly with no laziness
      whatsoever.
      
      But the fpu::fpstate_active still carries bits of the old complexity - meanwhile
      this flag has become a simple flag that shows whether the FPU context saving
      area in the thread struct is initialized and used, or not.
      
      Rename it to fpu::initialized to express this simplicity in the name as well.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-30-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      e4a81bfc
    • Ingo Molnar's avatar
      x86/fpu: Remove fpu__current_fpstate_write_begin/end() · 685c930d
      Ingo Molnar authored
      These functions are not used anymore, so remove them.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Bobby Powers <bobbypowers@gmail.com>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-29-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      685c930d
    • Ingo Molnar's avatar
      x86/fpu: Fix fpu__activate_fpstate_read() and update comments · 4618e909
      Ingo Molnar authored
      fpu__activate_fpstate_read() can be called for the current task
      when coredumping - or for stopped tasks when ptrace-ing.
      
      Implement this properly in the code and update the comments.
      
      This also fixes an incorrect (but harmless) warning introduced by
      one of the earlier patches.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-28-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      4618e909
  2. 25 Sep, 2017 2 commits
    • Eric Biggers's avatar
      x86/fpu: Reinitialize FPU registers if restoring FPU state fails · d5c8028b
      Eric Biggers authored
      Userspace can change the FPU state of a task using the ptrace() or
      rt_sigreturn() system calls.  Because reserved bits in the FPU state can
      cause the XRSTOR instruction to fail, the kernel has to carefully
      validate that no reserved bits or other invalid values are being set.
      
      Unfortunately, there have been bugs in this validation code.  For
      example, we were not checking that the 'xcomp_bv' field in the
      xstate_header was 0.  As-is, such bugs are exploitable to read the FPU
      registers of other processes on the system.  To do so, an attacker can
      create a task, assign to it an invalid FPU state, then spin in a loop
      and monitor the values of the FPU registers.  Because the task's FPU
      registers are not being restored, sometimes the FPU registers will have
      the values from another process.
      
      This is likely to continue to be a problem in the future because the
      validation done by the CPU instructions like XRSTOR is not immediately
      visible to kernel developers.  Nor will invalid FPU states ever be
      encountered during ordinary use --- they will only be seen during
      fuzzing or exploits.  There can even be reserved bits outside the
      xstate_header which are easy to forget about.  For example, the MXCSR
      register contains reserved bits, which were not validated by the
      KVM_SET_XSAVE ioctl until commit a575813b ("KVM: x86: Fix load
      damaged SSEx MXCSR register").
      
      Therefore, mitigate this class of vulnerability by restoring the FPU
      registers from init_fpstate if restoring from the task's state fails.
      
      We actually used to do this, but it was (perhaps unwisely) removed by
      commit 9ccc27a5 ("x86/fpu: Remove error return values from
      copy_kernel_to_*regs() functions").  This new patch is also a bit
      different.  First, it only clears the registers, not also the bad
      in-memory state; this is simpler and makes it easier to make the
      mitigation cover all callers of __copy_kernel_to_fpregs().  Second, it
      does the register clearing in an exception handler so that no extra
      instructions are added to context switches.  In fact, we *remove*
      instructions, since previously we were always zeroing the register
      containing 'err' even if CONFIG_X86_DEBUG_FPU was disabled.
      Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
      Reviewed-by: default avatarRik van Riel <riel@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Kevin Hao <haokexin@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Michael Halcrow <mhalcrow@google.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Wanpeng Li <wanpeng.li@hotmail.com>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Cc: kernel-hardening@lists.openwall.com
      Link: http://lkml.kernel.org/r/20170922174156.16780-4-ebiggers3@gmail.com
      Link: http://lkml.kernel.org/r/20170923130016.21448-27-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      d5c8028b
    • Eric Biggers's avatar
      x86/fpu: Don't let userspace set bogus xcomp_bv · 814fb7bb
      Eric Biggers authored
      On x86, userspace can use the ptrace() or rt_sigreturn() system calls to
      set a task's extended state (xstate) or "FPU" registers.  ptrace() can
      set them for another task using the PTRACE_SETREGSET request with
      NT_X86_XSTATE, while rt_sigreturn() can set them for the current task.
      In either case, registers can be set to any value, but the kernel
      assumes that the XSAVE area itself remains valid in the sense that the
      CPU can restore it.
      
      However, in the case where the kernel is using the uncompacted xstate
      format (which it does whenever the XSAVES instruction is unavailable),
      it was possible for userspace to set the xcomp_bv field in the
      xstate_header to an arbitrary value.  However, all bits in that field
      are reserved in the uncompacted case, so when switching to a task with
      nonzero xcomp_bv, the XRSTOR instruction failed with a #GP fault.  This
      caused the WARN_ON_FPU(err) in copy_kernel_to_xregs() to be hit.  In
      addition, since the error is otherwise ignored, the FPU registers from
      the task previously executing on the CPU were leaked.
      
      Fix the bug by checking that the user-supplied value of xcomp_bv is 0 in
      the uncompacted case, and returning an error otherwise.
      
      The reason for validating xcomp_bv rather than simply overwriting it
      with 0 is that we want userspace to see an error if it (incorrectly)
      provides an XSAVE area in compacted format rather than in uncompacted
      format.
      
      Note that as before, in case of error we clear the task's FPU state.
      This is perhaps non-ideal, especially for PTRACE_SETREGSET; it might be
      better to return an error before changing anything.  But it seems the
      "clear on error" behavior is fine for now, and it's a little tricky to
      do otherwise because it would mean we couldn't simply copy the full
      userspace state into kernel memory in one __copy_from_user().
      
      This bug was found by syzkaller, which hit the above-mentioned
      WARN_ON_FPU():
      
          WARNING: CPU: 1 PID: 0 at ./arch/x86/include/asm/fpu/internal.h:373 __switch_to+0x5b5/0x5d0
          CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.13.0 #453
          Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
          task: ffff9ba2bc8e42c0 task.stack: ffffa78cc036c000
          RIP: 0010:__switch_to+0x5b5/0x5d0
          RSP: 0000:ffffa78cc08bbb88 EFLAGS: 00010082
          RAX: 00000000fffffffe RBX: ffff9ba2b8bf2180 RCX: 00000000c0000100
          RDX: 00000000ffffffff RSI: 000000005cb10700 RDI: ffff9ba2b8bf36c0
          RBP: ffffa78cc08bbbd0 R08: 00000000929fdf46 R09: 0000000000000001
          R10: 0000000000000000 R11: 0000000000000000 R12: ffff9ba2bc8e42c0
          R13: 0000000000000000 R14: ffff9ba2b8bf3680 R15: ffff9ba2bf5d7b40
          FS:  00007f7e5cb10700(0000) GS:ffff9ba2bf400000(0000) knlGS:0000000000000000
          CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
          CR2: 00000000004005cc CR3: 0000000079fd5000 CR4: 00000000001406e0
          Call Trace:
          Code: 84 00 00 00 00 00 e9 11 fd ff ff 0f ff 66 0f 1f 84 00 00 00 00 00 e9 e7 fa ff ff 0f ff 66 0f 1f 84 00 00 00 00 00 e9 c2 fa ff ff <0f> ff 66 0f 1f 84 00 00 00 00 00 e9 d4 fc ff ff 66 66 2e 0f 1f
      
      Here is a C reproducer.  The expected behavior is that the program spin
      forever with no output.  However, on a buggy kernel running on a
      processor with the "xsave" feature but without the "xsaves" feature
      (e.g. Sandy Bridge through Broadwell for Intel), within a second or two
      the program reports that the xmm registers were corrupted, i.e. were not
      restored correctly.  With CONFIG_X86_DEBUG_FPU=y it also hits the above
      kernel warning.
      
          #define _GNU_SOURCE
          #include <stdbool.h>
          #include <inttypes.h>
          #include <linux/elf.h>
          #include <stdio.h>
          #include <sys/ptrace.h>
          #include <sys/uio.h>
          #include <sys/wait.h>
          #include <unistd.h>
      
          int main(void)
          {
              int pid = fork();
              uint64_t xstate[512];
              struct iovec iov = { .iov_base = xstate, .iov_len = sizeof(xstate) };
      
              if (pid == 0) {
                  bool tracee = true;
                  for (int i = 0; i < sysconf(_SC_NPROCESSORS_ONLN) && tracee; i++)
                      tracee = (fork() != 0);
                  uint32_t xmm0[4] = { [0 ... 3] = tracee ? 0x00000000 : 0xDEADBEEF };
                  asm volatile("   movdqu %0, %%xmm0\n"
                               "   mov %0, %%rbx\n"
                               "1: movdqu %%xmm0, %0\n"
                               "   mov %0, %%rax\n"
                               "   cmp %%rax, %%rbx\n"
                               "   je 1b\n"
                               : "+m" (xmm0) : : "rax", "rbx", "xmm0");
                  printf("BUG: xmm registers corrupted!  tracee=%d, xmm0=%08X%08X%08X%08X\n",
                         tracee, xmm0[0], xmm0[1], xmm0[2], xmm0[3]);
              } else {
                  usleep(100000);
                  ptrace(PTRACE_ATTACH, pid, 0, 0);
                  wait(NULL);
                  ptrace(PTRACE_GETREGSET, pid, NT_X86_XSTATE, &iov);
                  xstate[65] = -1;
                  ptrace(PTRACE_SETREGSET, pid, NT_X86_XSTATE, &iov);
                  ptrace(PTRACE_CONT, pid, 0, 0);
                  wait(NULL);
              }
              return 1;
          }
      
      Note: the program only tests for the bug using the ptrace() system call.
      The bug can also be reproduced using the rt_sigreturn() system call, but
      only when called from a 32-bit program, since for 64-bit programs the
      kernel restores the FPU state from the signal frame by doing XRSTOR
      directly from userspace memory (with proper error checking).
      Reported-by: default avatarDmitry Vyukov <dvyukov@google.com>
      Signed-off-by: default avatarEric Biggers <ebiggers@google.com>
      Reviewed-by: default avatarKees Cook <keescook@chromium.org>
      Reviewed-by: default avatarRik van Riel <riel@redhat.com>
      Acked-by: default avatarDave Hansen <dave.hansen@linux.intel.com>
      Cc: <stable@vger.kernel.org> [v3.17+]
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Kevin Hao <haokexin@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Michael Halcrow <mhalcrow@google.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Wanpeng Li <wanpeng.li@hotmail.com>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Cc: kernel-hardening@lists.openwall.com
      Fixes: 0b29643a ("x86/xsaves: Change compacted format xsave area header")
      Link: http://lkml.kernel.org/r/20170922174156.16780-2-ebiggers3@gmail.com
      Link: http://lkml.kernel.org/r/20170923130016.21448-25-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      814fb7bb
  3. 24 Sep, 2017 23 commits
    • Andi Kleen's avatar
      x86/fpu: Turn WARN_ON() in context switch into WARN_ON_FPU() · 03eaec81
      Andi Kleen authored
      copy_xregs_to_kernel checks if the alternatives have been already
      patched.
      
      This WARN_ON() is always executed in every context switch.
      
      All the other checks in fpu internal.h are WARN_ON_FPU(), but
      this one is plain WARN_ON(). I assume it was forgotten to switch it.
      
      So switch it to WARN_ON_FPU() too to avoid some unnecessary code
      in the context switch, and a potentially expensive cache line miss for the
      global variable.
      Signed-off-by: default avatarAndi Kleen <ak@linux.intel.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170329062605.4970-1-andi@firstfloor.org
      Link: http://lkml.kernel.org/r/20170923130016.21448-24-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      03eaec81
    • kbuild test robot's avatar
      x86/fpu: Fix boolreturn.cocci warnings · 4f8cef59
      kbuild test robot authored
      arch/x86/kernel/fpu/xstate.c:931:9-10: WARNING: return of 0/1 in function 'xfeatures_mxcsr_quirk' with return type bool
      
       Return statements in functions returning bool should use true/false instead of 1/0.
      
      Generated by: scripts/coccinelle/misc/boolreturn.cocci
      Signed-off-by: default avatarFengguang Wu <fengguang.wu@intel.com>
      Signed-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Borislav Petkov <bp@suse.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Cc: kbuild-all@01.org
      Cc: tipbuild@zytor.com
      Link: http://lkml.kernel.org/r/20170306004553.GA25764@lkp-wsm-ep1
      Link: http://lkml.kernel.org/r/20170923130016.21448-23-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      4f8cef59
    • Rik van Riel's avatar
      x86/fpu: Add FPU state copying quirk to handle XRSTOR failure on Intel Skylake CPUs · 0852b374
      Rik van Riel authored
      On Skylake CPUs I noticed that XRSTOR is unable to deal with states
      created by copyout_from_xsaves() if the xstate has only SSE/YMM state, and
      no FP state. That is, xfeatures had XFEATURE_MASK_SSE set, but not
      XFEATURE_MASK_FP.
      
      The reason is that part of the SSE/YMM state lives in the MXCSR and
      MXCSR_FLAGS fields of the FP state.
      
      Ensure that whenever we copy SSE or YMM state around, the MXCSR and
      MXCSR_FLAGS fields are also copied around.
      Signed-off-by: default avatarRik van Riel <riel@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Borislav Petkov <bp@suse.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170210085445.0f1cc708@annuminas.surriel.com
      Link: http://lkml.kernel.org/r/20170923130016.21448-22-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      0852b374
    • Ingo Molnar's avatar
      x86/fpu: Remove struct fpu::fpregs_active · 99dc26bd
      Ingo Molnar authored
      The previous changes paved the way for the removal of the
      fpu::fpregs_active state flag - we now only have the
      fpu::fpstate_active and fpu::last_cpu fields left.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-21-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      99dc26bd
    • Ingo Molnar's avatar
      x86/fpu: Decouple fpregs_activate()/fpregs_deactivate() from fpu->fpregs_active · 6cf4edbe
      Ingo Molnar authored
      The fpregs_activate()/fpregs_deactivate() are currently called in such a pattern:
      
      	if (!fpu->fpregs_active)
      		fpregs_activate(fpu);
      
      	...
      
      	if (fpu->fpregs_active)
      		fpregs_deactivate(fpu);
      
      But note that it's actually safe to call them without checking the flag first.
      
      This further decouples the fpu->fpregs_active flag from actual FPU logic.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-20-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      6cf4edbe
    • Ingo Molnar's avatar
      x86/fpu: Change fpu->fpregs_active users to fpu->fpstate_active · f1c8cd01
      Ingo Molnar authored
      We want to simplify the FPU state machine by eliminating fpu->fpregs_active,
      and we can do that because the two state flags (::fpregs_active and
      ::fpstate_active) are set essentially together.
      
      The old lazy FPU switching code used to make a distinction - but there's
      no lazy switching code anymore, we always switch in an 'eager' fashion.
      
      Do this by first changing all substantial uses of fpu->fpregs_active
      to fpu->fpstate_active and adding a few debug checks to double check
      our assumption is correct.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-19-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      f1c8cd01
    • Ingo Molnar's avatar
      x86/fpu: Split the state handling in fpu__drop() · b6aa8555
      Ingo Molnar authored
      Prepare fpu__drop() to use fpu->fpregs_active.
      
      There are two distinct usecases for fpu__drop() in this context:
      exit_thread() when called for 'current' in exit(), and when called
      for another task in fork().
      
      This patch does not change behavior, it only adds a couple of
      debug checks and structures the code to make the ->fpregs_active
      change more obviously correct.
      
      All the complications will be removed later on.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-18-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      b6aa8555
    • Ingo Molnar's avatar
      x86/fpu: Make the fpu state change in fpu__clear() scheduler-atomic · a10b6a16
      Ingo Molnar authored
      Do this temporarily only, to make it easier to change the FPU state machine,
      in particular this change couples the fpu->fpregs_active and fpu->fpstate_active
      states: they are only set/cleared together (as far as the scheduler sees them).
      
      This will be removed by later patches.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-17-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      a10b6a16
    • Ingo Molnar's avatar
      x86/fpu: Simplify fpu->fpregs_active use · b3a16308
      Ingo Molnar authored
      The fpregs_active() inline function is pretty pointless - in almost
      all the callsites it can be replaced with a direct fpu->fpregs_active
      access.
      
      Do so and eliminate the extra layer of obfuscation.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-16-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      b3a16308
    • Ingo Molnar's avatar
      x86/fpu: Flip the parameter order in copy_*_to_xstate() · 6d7f7da5
      Ingo Molnar authored
      Make it more consistent with regular memcpy() semantics, where the destination
      argument comes first.
      
      No change in functionality.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-15-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      6d7f7da5
    • Ingo Molnar's avatar
      x86/fpu: Remove 'kbuf' parameter from the copy_user_to_xstate() API · 7b9094c6
      Ingo Molnar authored
      No change in functionality.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-14-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      7b9094c6
    • Ingo Molnar's avatar
      x86/fpu: Remove 'ubuf' parameter from the copy_kernel_to_xstate() API · 59dffa4e
      Ingo Molnar authored
      No change in functionality.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-13-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      59dffa4e
    • Ingo Molnar's avatar
      x86/fpu: Split copy_user_to_xstate() into copy_kernel_to_xstate() & copy_user_to_xstate() · 79fecc2b
      Ingo Molnar authored
      Similar to:
      
        x86/fpu: Split copy_xstate_to_user() into copy_xstate_to_kernel() & copy_xstate_to_user()
      
      No change in functionality.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-12-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      79fecc2b
    • Ingo Molnar's avatar
      x86/fpu: Simplify __copy_xstate_to_kernel() return values · 8c0817f4
      Ingo Molnar authored
      __copy_xstate_to_kernel() can only return 0 (because kernel copies cannot fail),
      simplify the code throughout.
      
      No change in functionality.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-11-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      8c0817f4
    • Ingo Molnar's avatar
      x86/fpu: Change 'size_total' parameter to unsigned and standardize the size... · 6ff15f8d
      Ingo Molnar authored
      x86/fpu: Change 'size_total' parameter to unsigned and standardize the size checks in copy_xstate_to_*()
      
      'size_total' is derived from an unsigned input parameter - and then converted
      to 'int' and checked for negative ranges:
      
      	if (size_total < 0 || offset < size_total) {
      
      This conversion and the checks are unnecessary obfuscation, reject overly
      large requested copy sizes outright and simplify the underlying code.
      Reported-by: default avatarRik van Riel <riel@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-10-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      6ff15f8d
    • Ingo Molnar's avatar
      x86/fpu: Clarify parameter names in the copy_xstate_to_*() methods · 56583c9a
      Ingo Molnar authored
      Right now there's a confusing mixture of 'offset' and 'size' parameters:
      
       - __copy_xstate_to_*() input parameter 'end_pos' not not really an offset,
         but the full size of the copy to be performed.
      
       - input parameter 'count' to copy_xstate_to_*() shadows that of
         __copy_xstate_to_*()'s 'count' parameter name - but the roles
         are different: the first one is the total number of bytes to
         be copied, while the second one is a partial copy size.
      
      To unconfuse all this, use a consistent set of parameter names:
      
       - 'size' is the partial copy size within a single xstate component
       - 'size_total' is the total copy requested
       - 'offset_start' is the requested starting offset.
       - 'offset' is the offset within an xstate component.
      
      No change in functionality.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-9-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      56583c9a
    • Ingo Molnar's avatar
      x86/fpu: Remove the 'start_pos' parameter from the __copy_xstate_to_*() functions · 8a5b7318
      Ingo Molnar authored
      'start_pos' is always 0, so remove it and remove the pointless check of 'pos < 0'
      which can not ever be true as 'pos' is unsigned ...
      
      No change in functionality.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-8-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      8a5b7318
    • Ingo Molnar's avatar
      x86/fpu: Clean up the parameter definitions of copy_xstate_to_*() · becb2bb7
      Ingo Molnar authored
      Remove pointless 'const' of non-pointer input parameter.
      
      Remove unnecessary parenthesis that shows uncertainty about arithmetic operator precedence.
      
      Clarify copy_xstate_to_user() description.
      
      No change in functionality.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-7-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      becb2bb7
    • Ingo Molnar's avatar
      x86/fpu: Clean up parameter order in the copy_xstate_to_*() APIs · d7eda6c9
      Ingo Molnar authored
      Parameter ordering is weird:
      
        int copy_xstate_to_kernel(unsigned int pos, unsigned int count, void *kbuf, struct xregs_state *xsave);
        int copy_xstate_to_user(unsigned int pos, unsigned int count, void __user *ubuf, struct xregs_state *xsave);
      
      'pos' and 'count', which are attributes of the destination buffer, are listed before the destination
      buffer itself ...
      
      List them after the primary arguments instead.
      
      This makes the code more similar to regular memcpy() variant APIs.
      
      No change in functionality.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-6-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      d7eda6c9
    • Ingo Molnar's avatar
      x86/fpu: Remove 'kbuf' parameter from the copy_xstate_to_user() APIs · a69c158f
      Ingo Molnar authored
      The 'kbuf' parameter is unused in the _user() side of the API, remove it.
      
      This simplifies the code and makes it easier to think about.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-5-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      a69c158f
    • Ingo Molnar's avatar
      x86/fpu: Remove 'ubuf' parameter from the copy_xstate_to_kernel() APIs · 4d981cf2
      Ingo Molnar authored
      The 'ubuf' parameter is unused in the _kernel() side of the API, remove it.
      
      This simplifies the code and makes it easier to think about.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-4-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      4d981cf2
    • Ingo Molnar's avatar
      x86/fpu: Split copy_xstate_to_user() into copy_xstate_to_kernel() & copy_xstate_to_user() · f0d4f30a
      Ingo Molnar authored
      copy_xstate_to_user() is a weird API - in part due to a bad API inherited
      from the regset APIs.
      
      But don't propagate that bad API choice into the FPU code - so as a first
      step split the API into kernel and user buffer handling routines.
      
      (Also split the xstate_copyout() internal helper.)
      
      The split API is a dumb duplication that should be obviously correct, the
      real splitting will be done in the next patch.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-3-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      f0d4f30a
    • Ingo Molnar's avatar
      x86/fpu: Rename copyin_to_xsaves()/copyout_from_xsaves() to... · 656f0831
      Ingo Molnar authored
      x86/fpu: Rename copyin_to_xsaves()/copyout_from_xsaves() to copy_user_to_xstate()/copy_xstate_to_user()
      
      The 'copyin/copyout' nomenclature needlessly departs from what the modern FPU code
      uses, which is:
      
       copy_fpregs_to_fpstate()
       copy_fpstate_to_sigframe()
       copy_fregs_to_user()
       copy_fxregs_to_kernel()
       copy_fxregs_to_user()
       copy_kernel_to_fpregs()
       copy_kernel_to_fregs()
       copy_kernel_to_fxregs()
       copy_kernel_to_xregs()
       copy_user_to_fregs()
       copy_user_to_fxregs()
       copy_user_to_xregs()
       copy_xregs_to_kernel()
       copy_xregs_to_user()
      
      I.e. according to this pattern, the following rename should be done:
      
        copyin_to_xsaves()    -> copy_user_to_xstate()
        copyout_from_xsaves() -> copy_xstate_to_user()
      
      or, if we want to be pedantic, denote that that the user-space format is ptrace:
      
        copyin_to_xsaves()    -> copy_user_ptrace_to_xstate()
        copyout_from_xsaves() -> copy_xstate_to_user_ptrace()
      
      But I'd suggest the shorter, non-pedantic name.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andy Lutomirski <luto@kernel.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Dave Hansen <dave.hansen@linux.intel.com>
      Cc: Eric Biggers <ebiggers3@gmail.com>
      Cc: Fenghua Yu <fenghua.yu@intel.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
      Link: http://lkml.kernel.org/r/20170923130016.21448-2-mingo@kernel.orgSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
      656f0831