• Israel Rukshin's avatar
    nvme-fc: Avoid preallocating big SGL for data · b1ae1a23
    Israel Rukshin authored
    nvme_fc_create_io_queues() preallocates a big buffer for the IO SGL based
    on SG_CHUNK_SIZE.
    
    Modern DMA engines are often capable of dealing with very big segments so
    the SG_CHUNK_SIZE is often too big. SG_CHUNK_SIZE results in a static 4KB
    SGL allocation per command.
    
    If a controller has lots of deep queues, preallocation for the sg list can
    consume substantial amounts of memory. For nvme-fc, nr_hw_queues can be
    128 and each queue's depth 128. This means the resulting preallocation
    for the data SGL is 128*128*4K = 64MB per controller.
    
    Switch to runtime allocation for SGL for lists longer than 2 entries. This
    is the approach used by NVMe PCI so it should be reasonable for NVMeOF as
    well. Runtime SGL allocation has always been the case for the legacy I/O
    path so this is nothing new.
    Reviewed-by: default avatarMax Gurtovoy <maxg@mellanox.com>
    Reviewed-by: default avatarJames Smart <james.smart@broadcom.com>
    Signed-off-by: default avatarIsrael Rukshin <israelr@mellanox.com>
    Signed-off-by: default avatarKeith Busch <kbusch@kernel.org>
    b1ae1a23
fc.c 94.2 KB