• Sumit.Saxena@avagotech.com's avatar
    megaraid_sas : Use Block layer tag support for internal command indexing · 4026e9aa
    Sumit.Saxena@avagotech.com authored
    megaraid_sas driver will use block layer provided tag for indexing internal
    MPT frames to get any unique MPT frame tied with tag.  Each IO request
    submitted from SCSI mid layer will get associated MPT frame from MPT framepool
    (retrieved and return back using spinlock inside megaraid_sas driver's
    submission/completion call back). Getting MPT frame from MPT Frame pool is
    very expensive operation because of associated spin lock operation (spinlock
    overhead increase on multi NUMA node). This type of locking in driver is very
    expensive call considering each IO request need - Acquire and Release of the
    same lock.
    
    With this support, in IO path driver will directly provide the unique command
    index(which is based on block layer tag) and will get the MPT frame tied to
    the tag and this way driver can get rid off lock, which synchronizes the
    access to MPT frame pool while fetching and returning MPT frame from the pool.
    
    This support in driver provides siginificant performance improvement(on multi
    NUMA node system)on latest upstream with SCSI.MQ as well as on existing linux
    distributions.
    
    Here is the data for test executed at Avago-
    - IO Tool- FIO
    - 4 Socket SMC server. (4 NUMA node server)
    - 12 SSDs in JBOD mode .
    - 4K Rand READ, QD=32
    - SCSI MQ x86_64 (Latest Upstream kernel)
    - upto 300% Performance Improvement.
    
    If IOs are running on single Node, perfromance gain is less, but as soon as
    increase number of nodes, performance improvement is significant. IOs running
    on all 4 NUMA nodes, with this patch applied IOPs observed was 1170K vs 344K
    IOPs seen without this patch.
    
    Logically, there are two parts of this patch- 1) Block layer tag support 2)
    changes in calling convention of return_cmd.  part 2 will revert the changes
    done by patch- 90dc9d98 megaraid_sas : MFI MPT linked list corruption fix
    because changes done in part 1 has fixed the problem of MFI MPT linked list
    corruption. part 2 is very much dependent on part 1, so we decided to have
    single patch for these two logical changes.
    
    [jejb: remove chatty printk pointed out by hch]
    Signed-off-by: default avatarKashyap Desai <kashyap.desai@avagotech.com>
    Signed-off-by: default avatarSumit Saxena <sumit.saxena@avagotech.com>
    Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
    Signed-off-by: default avatarJames Bottomley <JBottomley@Odin.com>
    4026e9aa
megaraid_sas_fusion.c 85.1 KB