• Peter Zijlstra's avatar
    perf/x86: Further optimize copy_from_user_nmi() · e00b12e6
    Peter Zijlstra authored
    Now that we can deal with nested NMI due to IRET re-enabling NMIs and
    can deal with faults from NMI by making sure we preserve CR2 over NMIs
    we can in fact simply access user-space memory from NMI context.
    
    So rewrite copy_from_user_nmi() to use __copy_from_user_inatomic() and
    rework the fault path to do the minimal required work before taking
    the in_atomic() fault handler.
    
    In particular avoid perf_sw_event() which would make perf recurse on
    itself (it should be harmless as our recursion protections should be
    able to deal with this -- but why tempt fate).
    
    Also rename notify_page_fault() to kprobes_fault() as that is a much
    better name; there is no notifier in it and its specific to kprobes.
    
    Don measured that his worst case NMI path shrunk from ~300K cycles to
    ~150K cycles.
    
    Cc: Stephane Eranian <eranian@google.com>
    Cc: jmario@redhat.com
    Cc: Arnaldo Carvalho de Melo <acme@infradead.org>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Andi Kleen <ak@linux.intel.com>
    Cc: dave.hansen@linux.intel.com
    Tested-by: default avatarDon Zickus <dzickus@redhat.com>
    Signed-off-by: default avatarPeter Zijlstra <peterz@infradead.org>
    Link: http://lkml.kernel.org/r/20131024105206.GM2490@laptop.programming.kicks-ass.netSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    e00b12e6
fault.c 29.2 KB