• Nick Terrell's avatar
    lz4: fix kernel decompression speed · b1a3e75e
    Nick Terrell authored
    This patch replaces all memcpy() calls with LZ4_memcpy() which calls
    __builtin_memcpy() so the compiler can inline it.
    
    LZ4 relies heavily on memcpy() with a constant size being inlined.  In x86
    and i386 pre-boot environments memcpy() cannot be inlined because memcpy()
    doesn't get defined as __builtin_memcpy().
    
    An equivalent patch has been applied upstream so that the next import
    won't lose this change [1].
    
    I've measured the kernel decompression speed using QEMU before and after
    this patch for the x86_64 and i386 architectures.  The speed-up is about
    10x as shown below.
    
    Code	Arch	Kernel Size	Time	Speed
    v5.8	x86_64	11504832 B	148 ms	 79 MB/s
    patch	x86_64	11503872 B	 13 ms	885 MB/s
    v5.8	i386	 9621216 B	 91 ms	106 MB/s
    patch	i386	 9620224 B	 10 ms	962 MB/s
    
    I also measured the time to decompress the initramfs on x86_64, i386, and
    arm.  All three show the same decompression speed before and after, as
    expected.
    
    [1] https://github.com/lz4/lz4/pull/890Signed-off-by: default avatarNick Terrell <terrelln@fb.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Cc: Yann Collet <yann.collet.73@gmail.com>
    Cc: Gao Xiang <gaoxiang25@huawei.com>
    Cc: Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
    Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
    Cc: Ingo Molnar <mingo@kernel.org>
    Cc: Arvind Sankar <nivedita@alum.mit.edu>
    Link: http://lkml.kernel.org/r/20200803194022.2966806-1-nickrterrell@gmail.comSigned-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    b1a3e75e
lz4_decompress.c 20.2 KB