• Xiang Chen's avatar
    scsi: hisi_sas: Pre-allocate slot DMA buffers · 2ba5afb6
    Xiang Chen authored
    Currently the driver spends much time allocating and freeing the slot DMA
    buffer for command delivery/completion. To boost the performance,
    pre-allocate the buffers for all IPTT. The downside of this approach is
    that we are reallocating all buffer memory upfront, so hog memory which we
    may not need.
    
    However, the current method - DMA buffer pool - also caches all buffers and
    does not free them until the pool is destroyed, so is not exactly efficient
    either.
    
    On top of this, since the slot DMA buffer is slightly bigger than a 4K
    page, we need to allocate 2x4K pages per buffer (for 4K page kernel), which
    is quite wasteful. For 64K page size this is not such an issue.
    
    So, for the 4K page case, in order to make memory usage more efficient,
    pre-allocating larger blocks of DMA memory for the buffers can be more
    efficient.
    
    To make DMA memory usage most efficient, we would choose a single
    contiguous DMA memory block, but this could use up all the DMA memory in
    the system (when CMA enabled and no IOMMU), or we may just not be able to
    allocate a DMA buffer large enough when no CMA or IOMMU.
    
    To decide the block size we use the LCM (least common multiple) of the
    buffer size and the page size. We roundup(64) to ensure the LCM is not too
    large, even though a little memory may be wasted per block.
    
    So, with this, the total memory requirement is about is about 17MB for 4096
    max IPTT.
    
    Previously (for 4K pages case), it would be 32MB (for all slots
    allocated).
    
    With this change, the relative increase of IOPS for bs=4K read when
    PAGE_SIZE=4K and PAGE_SIZE=64K is as follows:
        IODEPTH     4K PAGE_SIZE      64K PAGE_SIZE
        32          56%               47%
        64          53%               44%
        128         64%               43%
        256         67%               45%
    Signed-off-by: default avatarXiang Chen <chenxiang66@hisilicon.com>
    Signed-off-by: default avatarJohn Garry <john.garry@huawei.com>
    Signed-off-by: default avatarMartin K. Petersen <martin.petersen@oracle.com>
    2ba5afb6
hisi_sas.h 12.6 KB