• Akira Tsukamoto's avatar
    riscv: __asm_copy_to-from_user: Optimize unaligned memory access and pipeline stall · ca6eaaa2
    Akira Tsukamoto authored
    This patch will reduce cpu usage dramatically in kernel space especially
    for application which use sys-call with large buffer size, such as
    network applications. The main reason behind this is that every
    unaligned memory access will raise exceptions and switch between s-mode
    and m-mode causing large overhead.
    
    First copy in bytes until reaches the first word aligned boundary in
    destination memory address. This is the preparation before the bulk
    aligned word copy.
    
    The destination address is aligned now, but oftentimes the source
    address is not in an aligned boundary. To reduce the unaligned memory
    access, it reads the data from source in aligned boundaries, which will
    cause the data to have an offset, and then combines the data in the next
    iteration by fixing offset with shifting before writing to destination.
    The majority of the improving copy speed comes from this shift copy.
    
    In the lucky situation that the both source and destination address are
    on the aligned boundary, perform load and store with register size to
    copy the data. Without the unrolling, it will reduce the speed since the
    next store instruction for the same register using from the load will
    stall the pipeline.
    
    At last, copying the remainder in one byte at a time.
    Signed-off-by: default avatarAkira Tsukamoto <akira.tsukamoto@gmail.com>
    Signed-off-by: default avatarPalmer Dabbelt <palmerdabbelt@google.com>
    ca6eaaa2
uaccess.S 5.04 KB