• Linus Torvalds's avatar
    x86: inline the 'rep movs' in user copies for the FSRM case · 577e6a7f
    Linus Torvalds authored
    This does the same thing for the user copies as commit 0db7058e
    ("x86/clear_user: Make it faster") did for clear_user().  In other
    words, it inlines the "rep movs" case when X86_FEATURE_FSRM is set,
    avoiding the function call entirely.
    
    In order to do that, it makes the calling convention for the out-of-line
    case ("copy_user_generic_unrolled") match the 'rep movs' calling
    convention, although it does also end up clobbering a number of
    additional registers.
    
    Also, to simplify code sharing in the low-level assembly with the
    __copy_user_nocache() function (that uses the normal C calling
    convention), we end up with a kind of mixed return value for the
    low-level asm code: it will return the result in both %rcx (to work as
    an alternative for the 'rep movs' case), _and_ in %rax (for the nocache
    case).
    
    We could avoid this by wrapping __copy_user_nocache() callers in an
    inline asm, but since the cost is just an extra register copy, it's
    probably not worth it.
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    577e6a7f
copy_user_64.S 8.17 KB