• Jens Axboe's avatar
    io_uring: support for user allocated memory for rings/sqes · 03d89a2d
    Jens Axboe authored
    Currently io_uring applications must call mmap(2) twice to map the rings
    themselves, and the sqes array. This works fine, but it does not support
    using huge pages to back the rings/sqes.
    
    Provide a way for the application to pass in pre-allocated memory for
    the rings/sqes, which can then suitably be allocated from shmfs or
    via mmap to get huge page support.
    
    Particularly for larger rings, this reduces the TLBs needed.
    
    If an application wishes to take advantage of that, it must pre-allocate
    the memory needed for the sq/cq ring, and the sqes. The former must
    be passed in via the io_uring_params->cq_off.user_data field, while the
    latter is passed in via the io_uring_params->sq_off.user_data field. Then
    it must set IORING_SETUP_NO_MMAP in the io_uring_params->flags field,
    and io_uring will then map the existing memory into the kernel for shared
    use. The application must not call mmap(2) to map rings as it otherwise
    would have, that will now fail with -EINVAL if this setup flag was used.
    
    The pages used for the rings and sqes must be contigious. The intent here
    is clearly that huge pages should be used, otherwise the normal setup
    procedure works fine as-is. The application may use one huge page for
    both the rings and sqes.
    
    Outside of those initialization changes, everything works like it did
    before.
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    03d89a2d
io_uring.c 120 KB