• Logan Gunthorpe's avatar
    NTB: Add MSI interrupt support to ntb_transport · 2b0569b3
    Logan Gunthorpe authored
    Introduce the module parameter 'use_msi' which, when set, uses
    MSI interrupts instead of doorbells for each queue pair (QP). The
    parameter is only available if NTB MSI support is configured into
    the kernel. We also require there to be more than one memory window
    (MW) so that an extra one is available to forward the APIC region.
    
    To use MSIs, we request one interrupt per QP and forward the MSI address
    and data to the peer using scratch pad registers (SPADS) above the MW
    SPADS. (If there are not enough SPADS the MSI interrupt will not be used.)
    
    Once registered, we simply use ntb_msi_peer_trigger and the receiving
    ISR simply queues up the rxc_db_work for the queue.
    
    This addition can significantly improve performance of ntb_transport.
    In a simple, untuned, apples-to-apples comparision using ntb_netdev
    and iperf with switchtec hardware, I see 3.88Gb/s without MSI
    interrupts and 14.1Gb/s wit MSI, which is a more than 3x improvement.
    Signed-off-by: default avatarLogan Gunthorpe <logang@deltatee.com>
    Cc: Dave Jiang <dave.jiang@intel.com>
    Cc: Allen Hubbe <allenbh@gmail.com>
    Signed-off-by: default avatarJon Mason <jdmason@kudzu.us>
    2b0569b3
ntb_transport.c 61 KB