• Jason Gunthorpe's avatar
    RDMA/mlx5: Use irq xarray locking for mkey_table · 1524b12a
    Jason Gunthorpe authored
    The mkey_table xarray is touched by the reg_mr_callback() function which
    is called from a hard irq. Thus all other uses of xa_lock must use the
    _irq variants.
    
      WARNING: inconsistent lock state
      5.4.0-rc1 #12 Not tainted
      --------------------------------
      inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
      python3/343 [HC0[0]:SC0[0]:HE1:SE1] takes:
      ffff888182be1d40 (&(&xa->xa_lock)->rlock#3){?.-.}, at: xa_erase+0x12/0x30
      {IN-HARDIRQ-W} state was registered at:
        lock_acquire+0xe1/0x200
        _raw_spin_lock_irqsave+0x35/0x50
        reg_mr_callback+0x2dd/0x450 [mlx5_ib]
        mlx5_cmd_exec_cb_handler+0x2c/0x70 [mlx5_core]
        mlx5_cmd_comp_handler+0x355/0x840 [mlx5_core]
       [..]
    
       Possible unsafe locking scenario:
    
             CPU0
             ----
        lock(&(&xa->xa_lock)->rlock#3);
        <Interrupt>
          lock(&(&xa->xa_lock)->rlock#3);
    
       *** DEADLOCK ***
    
      2 locks held by python3/343:
       #0: ffff88818eb4bd38 (&uverbs_dev->disassociate_srcu){....}, at: ib_uverbs_ioctl+0xe5/0x1e0 [ib_uverbs]
       #1: ffff888176c76d38 (&file->hw_destroy_rwsem){++++}, at: uobj_destroy+0x2d/0x90 [ib_uverbs]
    
      stack backtrace:
      CPU: 3 PID: 343 Comm: python3 Not tainted 5.4.0-rc1 #12
      Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
      Call Trace:
       dump_stack+0x86/0xca
       print_usage_bug.cold.50+0x2e5/0x355
       mark_lock+0x871/0xb50
       ? match_held_lock+0x20/0x250
       ? check_usage_forwards+0x240/0x240
       __lock_acquire+0x7de/0x23a0
       ? __kasan_check_read+0x11/0x20
       ? mark_lock+0xae/0xb50
       ? mark_held_locks+0xb0/0xb0
       ? find_held_lock+0xca/0xf0
       lock_acquire+0xe1/0x200
       ? xa_erase+0x12/0x30
       _raw_spin_lock+0x2a/0x40
       ? xa_erase+0x12/0x30
       xa_erase+0x12/0x30
       mlx5_ib_dealloc_mw+0x55/0xa0 [mlx5_ib]
       uverbs_dealloc_mw+0x3c/0x70 [ib_uverbs]
       uverbs_free_mw+0x1a/0x20 [ib_uverbs]
       destroy_hw_idr_uobject+0x49/0xa0 [ib_uverbs]
       [..]
    
    Fixes: 04177915 ("RDMA/mlx5: Add missing synchronize_srcu() for MW cases")
    Link: https://lore.kernel.org/r/20191024234910.GA9038@ziepe.caAcked-by: default avatarLeon Romanovsky <leonro@mellanox.com>
    Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
    1524b12a
mr.c 58 KB