Skip to content

L
linux
Switch branch/tag
  1. 22 Jul, 2011 3 commits
    • Roland Dreier's avatar
      Merge branches 'cma', 'cxgb4', 'ipath', 'misc', 'mlx4', 'mthca', 'qib' and 'srp' into for-next · 44602075
      Roland Dreier authored
      44602075
    • Mike Marciniszyn's avatar
      IB/qib: Defer HCA error events to tasklet · e67306a3
      Mike Marciniszyn authored 
      With ib_qib options:

    options ib_qib krcvqs=1 pcie_caps=0x51 rcvhdrcnt=4096 singleport=1 ibmtu=4

a run of ib_write_bw -a yields the following:

    ------------------------------------------------------------------
     #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]
     1048576   5000           2910.64            229.80
    ------------------------------------------------------------------

The top cpu use in a profile is:

    CPU: Intel Architectural Perfmon, speed 2400.15 MHz (estimated)
    Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask
    of 0x00 (No unit mask) count 1002300
    Counted LLC_MISSES events (Last level cache demand requests from this core that
    missed the LLC) with a unit mask of 0x41 (No unit mask) count 10000
    samples  %        samples  %        app name                 symbol name
    15237    29.2642  964      17.1195  ib_qib.ko                qib_7322intr
    12320    23.6618  1040     18.4692  ib_qib.ko                handle_7322_errors
    4106      7.8860  0              0  vmlinux                  vsnprintf


Analysis of the stats, profile, the code, and the annotated profile indicate:
 - All of the overflow interrupts (one per packet overflow) are
   serviced on CPU0 with no mitigation on the frequency.
 - All of the receive interrupts are being serviced by CPU0.  (That is
   the way truescale.cmds statically allocates the kctx IRQs to CPU)
 - The code is spending all of its time servicing QIB_I_C_ERROR
   RcvEgrFullErr interrupts on CPU0, starving the packet receive
   processing.
 - The decode_err routine is very inefficient, using a printf variant
   to format a "%s" and continues to loop when the errs mask has been
   cleared.
 - Both qib_7322intr and handle_7322_errors read pci registers, which
   is very inefficient.

The fix does the following:
 - Adds a tasklet to service QIB_I_C_ERROR
 - Replaces the very inefficient scnprintf() with a memcpy().  A field
   is added to qib_hwerror_msgs to save the sizeof("string") at
   compile time so that a strlen is not needed during err_decode().
 - The most frequent errors (Overflows) are serviced first to exit the
   loop as early as possible.
 - The loop now exits as soon as the errs mask is clear rather than
   fruitlessly looping through the msp array.

With this fix the performance changes to:

    ------------------------------------------------------------------
     #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]
     1048576   5000           2990.64            2941.35
    ------------------------------------------------------------------

During testing of the error handling overflow patch, it was determined
that some CPU's were slower when servicing both overflow and receive
interrupts on CPU0 with different MSI interrupt vectors.

This patch adds an option (krcvq01_no_msi) to not use a dedicated MSI
interrupt for kctx's < 2 and to service them on the default interrupt.
For some CPUs, the cost of the interrupt enter/exit is more costly
than then the additional PCI read in the default handler.
Signed-off-by: default avatarMike Marciniszyn <mike.marciniszyn@qlogic.com>
Signed-off-by: default avatarRoland Dreier <roland@purestorage.com>
      e67306a3
    • Dotan Barak's avatar
      mlx4_core: Bump the driver version to 1.0 · e1892fa8
      Dotan Barak authored 
      Many features were added to this driver, so the driver version should change too.
Signed-off-by: default avatarDotan Barak <dotanb@dev.mellanox.co.il>
Signed-off-by: default avatarRoland Dreier <roland@purestorage.com>
      e1892fa8
  3. 19 Jul, 2011 10 commits
  5. 18 Jul, 2011 11 commits
  7. 15 Jul, 2011 1 commit
    • Goldwyn Rodrigues's avatar
      IB/mthca: Stop returning separate error and status from FW commands · cdb73db0
      Goldwyn Rodrigues authored 
      Instead of having firmware command functions return an error and also
a status, leading to code like:

	err = mthca_FW_COMMAND(..., &status);
	if (err)
		goto out;
        if (status) {
		err = -E...;
		goto out;
	}

all over the place, just handle the FW status inside the FW command
handling code (the way mlx4 does it), so we can simply write:

	err = mthca_FW_COMMAND(...);
	if (err)
		goto out;

In addition to simplifying the source code, this also saves a healthy
chunk of text:

    add/remove: 0/0 grow/shrink: 10/88 up/down: 510/-3357 (-2847)
    function                                     old     new   delta
    static.trans_table                           324     584    +260
    mthca_cmd_poll                               352     477    +125
    mthca_cmd_wait                               511     567     +56
    mthca_table_put                              213     240     +27
    mthca_cleanup_db_tab                         372     387     +15
    __mthca_remove_one                           314     323      +9
    mthca_cleanup_user_db_tab                    275     283      +8
    __mthca_init_one                            1738    1746      +8
    mthca_cleanup                                 20      21      +1
    mthca_MAD_IFC                               1081    1082      +1
    mthca_MGID_HASH                               43      40      -3
    mthca_MAP_ICM_AUX                             23      20      -3
    mthca_MAP_ICM                                 19      16      -3
    mthca_MAP_FA                                  23      20      -3
    mthca_READ_MGM                                43      38      -5
    mthca_QUERY_SRQ                               43      38      -5
    mthca_QUERY_QP                                59      54      -5
    mthca_HW2SW_SRQ                               43      38      -5
    mthca_HW2SW_MPT                               60      55      -5
    mthca_HW2SW_EQ                                43      38      -5
    mthca_HW2SW_CQ                                43      38      -5
    mthca_free_icm_table                         120     114      -6
    mthca_query_srq                              214     206      -8
    mthca_free_qp                                662     654      -8
    mthca_cmd                                     38      28     -10
    mthca_alloc_db                              1321    1311     -10
    mthca_setup_hca                             1067    1055     -12
    mthca_WRITE_MTT                               35      22     -13
    mthca_WRITE_MGM                               40      27     -13
    mthca_UNMAP_ICM_AUX                           36      23     -13
    mthca_UNMAP_FA                                36      23     -13
    mthca_SYS_DIS                                 36      23     -13
    mthca_SYNC_TPT                                36      23     -13
    mthca_SW2HW_SRQ                               35      22     -13
    mthca_SW2HW_MPT                               35      22     -13
    mthca_SW2HW_EQ                                35      22     -13
    mthca_SW2HW_CQ                                35      22     -13
    mthca_RUN_FW                                  36      23     -13
    mthca_DISABLE_LAM                             36      23     -13
    mthca_CLOSE_IB                                36      23     -13
    mthca_CLOSE_HCA                               38      25     -13
    mthca_ARM_SRQ                                 39      26     -13
    mthca_free_icms                              178     164     -14
    mthca_QUERY_DDR                              389     375     -14
    mthca_resize_cq                             1063    1048     -15
    mthca_unmap_eq_icm                           123     107     -16
    mthca_map_eq_icm                             396     380     -16
    mthca_cmd_box                                 90      74     -16
    mthca_SET_IB                                 433     417     -16
    mthca_RESIZE_CQ                              369     353     -16
    mthca_MAP_ICM_page                           240     224     -16
    mthca_MAP_EQ                                 183     167     -16
    mthca_INIT_IB                                473     457     -16
    mthca_INIT_HCA                               745     729     -16
    mthca_map_user_db                            816     798     -18
    mthca_SYS_EN                                 157     139     -18
    mthca_cleanup_qp_table                        78      59     -19
    mthca_cleanup_eq_table                       168     149     -19
    mthca_UNMAP_ICM                              143     121     -22
    mthca_modify_srq                             172     149     -23
    mthca_unmap_fmr                              198     174     -24
    mthca_query_qp                               814     790     -24
    mthca_query_pkey                             343     319     -24
    mthca_SET_ICM_SIZE                            34      10     -24
    mthca_QUERY_DEV_LIM                         1870    1846     -24
    mthca_map_cmd                               1130    1105     -25
    mthca_ENABLE_LAM                             401     375     -26
    mthca_modify_port                            247     220     -27
    mthca_query_device                           884     850     -34
    mthca_NOP                                     75      41     -34
    mthca_table_get                              287     249     -38
    mthca_init_qp_table                          333     293     -40
    mthca_MODIFY_QP                              348     308     -40
    mthca_close_hca                              131      89     -42
    mthca_free_eq                                435     390     -45
    mthca_query_port                             755     705     -50
    mthca_free_cq                                581     528     -53
    mthca_alloc_icm_table                        578     524     -54
    mthca_multicast_attach                      1041     986     -55
    mthca_init_hca                               326     271     -55
    mthca_query_gid                              487     431     -56
    mthca_free_srq                               524     468     -56
    mthca_free_mr                                168     111     -57
    mthca_create_eq                             1560    1501     -59
    mthca_multicast_detach                       790     728     -62
    mthca_write_mtt                              918     854     -64
    mthca_register_device                       1406    1342     -64
    mthca_fmr_alloc                              947     883     -64
    mthca_mr_alloc                               652     582     -70
    mthca_process_mad                           1242    1164     -78
    mthca_dev_lim                                910     830     -80
    find_mgm                                     482     400     -82
    mthca_modify_qp                             3852    3753     -99
    mthca_init_cq                               1281    1181    -100
    mthca_alloc_srq                             1719    1610    -109
    mthca_init_eq_table                         1807    1679    -128
    mthca_init_tavor                             761     491    -270
    mthca_init_arbel                            2617    2098    -519
Signed-off-by: default avatarGoldwyn Rodrigues <rgoldwyn@suse.de>
      cdb73db0
  9. 13 Jul, 2011 2 commits
  11. 11 Jul, 2011 13 commits
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7