• Kaike Wan's avatar
    IB/hfi1: Ensure r_tid_ack is valid before building TID RDMA ACK packet · c1abd865
    Kaike Wan authored
    The index r_tid_ack is used to indicate the next TID RDMA WRITE request to
    acknowledge in the ring s_ack_queue[] on the responder side and should be
    set to a valid index other than its initial value before r_tid_tail is
    advanced to the next TID RDMA WRITE request and particularly before a TID
    RDMA ACK is built. Otherwise, a NULL pointer dereference may result:
    
      BUG: unable to handle kernel paging request at ffff9a32d27abff8
      IP: [<ffffffffc0d87ea6>] hfi1_make_tid_rdma_pkt+0x476/0xcb0 [hfi1]
      PGD 2749032067 PUD 0
      Oops: 0000 1 SMP
      Modules linked in: osp(OE) ofd(OE) lfsck(OE) ost(OE) mgc(OE) osd_zfs(OE) lquota(OE) lustre(OE) lmv(OE) mdc(OE) lov(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) ib_ipoib(OE) hfi1(OE) rdmavt(OE) nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ib_isert iscsi_target_mod target_core_mod ib_ucm dm_mirror dm_region_hash dm_log mlx5_ib dm_mod zfs(POE) rpcrdma sunrpc rdma_ucm ib_uverbs opa_vnic ib_iser zunicode(POE) ib_umad zavl(POE) icp(POE) sb_edac intel_powerclamp coretemp rdma_cm intel_rapl iosf_mbi iw_cm libiscsi scsi_transport_iscsi kvm ib_cm iTCO_wdt mxm_wmi iTCO_vendor_support irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd zcommon(POE) znvpair(POE) pcspkr spl(OE) mei_me
      sg mei ioatdma lpc_ich joydev i2c_i801 shpchp ipmi_si ipmi_devintf ipmi_msghandler wmi acpi_power_meter ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic mgag200 mlx5_core drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ixgbe ahci ttm mlxfw ib_core libahci devlink mdio crct10dif_pclmul crct10dif_common drm ptp libata megaraid_sas crc32c_intel i2c_algo_bit pps_core i2c_core dca [last unloaded: rdmavt]
      CPU: 15 PID: 68691 Comm: kworker/15:2H Kdump: loaded Tainted: P W OE ------------ 3.10.0-862.2.3.el7_lustre.x86_64 #1
      Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS SE5C610.86B.01.01.0016.033120161139 03/31/2016
      Workqueue: hfi0_0 _hfi1_do_tid_send [hfi1]
      task: ffff9a01f47faf70 ti: ffff9a11776a8000 task.ti: ffff9a11776a8000
      RIP: 0010:[<ffffffffc0d87ea6>] [<ffffffffc0d87ea6>] hfi1_make_tid_rdma_pkt+0x476/0xcb0 [hfi1]
      RSP: 0018:ffff9a11776abd08 EFLAGS: 00010002
      RAX: ffff9a32d27abfc0 RBX: ffff99f2d27aa000 RCX: 00000000ffffffff
      RDX: 0000000000000000 RSI: 0000000000000220 RDI: ffff99f2ffc05300
      RBP: ffff9a11776abd88 R08: 000000000001c310 R09: ffffffffc0d87ad4
      R10: 0000000000000000 R11: 0000000000000000 R12: ffff9a117a423c00
      R13: ffff9a117a423c00 R14: ffff9a03500c0000 R15: ffff9a117a423cb8
      FS: 0000000000000000(0000) GS:ffff9a117e9c0000(0000) knlGS:0000000000000000
      CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
      CR2: ffff9a32d27abff8 CR3: 0000002748a0e000 CR4: 00000000001607e0
      Call Trace:
      [<ffffffffc0d88874>] _hfi1_do_tid_send+0x194/0x320 [hfi1]
      [<ffffffffaf0b2dff>] process_one_work+0x17f/0x440
      [<ffffffffaf0b3ac6>] worker_thread+0x126/0x3c0
      [<ffffffffaf0b39a0>] ? manage_workers.isra.24+0x2a0/0x2a0
      [<ffffffffaf0bae31>] kthread+0xd1/0xe0
      [<ffffffffaf0bad60>] ? insert_kthread_work+0x40/0x40
      [<ffffffffaf71f5f7>] ret_from_fork_nospec_begin+0x21/0x21
      [<ffffffffaf0bad60>] ? insert_kthread_work+0x40/0x40
      hfi1 0000:05:00.0: hfi1_0: reserved_op: opcode 0xf2, slot 2, rsv_used 1, rsv_ops 1
      Code: 00 00 41 8b 8d d8 02 00 00 89 c8 48 89 45 b0 48 c1 65 b0 06 48 8b 83 a0 01 00 00 48 01 45 b0 48 8b 45 b0 41 80 bd 10 03 00 00 00 <48> 8b 50 38 4c 8d 7a 50 74 45 8b b2 d0 00 00 00 85 f6 0f 85 72
      RIP [<ffffffffc0d87ea6>] hfi1_make_tid_rdma_pkt+0x476/0xcb0 [hfi1]
      RSP <ffff9a11776abd08>
      CR2: ffff9a32d27abff8
    
    This problem can happen if a RESYNC request is received before r_tid_ack
    is modified.
    
    This patch fixes the issue by making sure that r_tid_ack is set to a valid
    value before a TID RDMA ACK is built. Functions are defined to simplify
    the code.
    
    Fixes: 07b92370 ("IB/hfi1: Add functions to receive TID RDMA WRITE request")
    Fixes: 7cf0ad67 ("IB/hfi1: Add a function to receive TID RDMA RESYNC packet")
    Link: https://lore.kernel.org/r/20191025195830.106825.44022.stgit@awfm-01.aw.intel.com
    Cc: <stable@vger.kernel.org>
    Reviewed-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
    Reviewed-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
    Signed-off-by: default avatarKaike Wan <kaike.wan@intel.com>
    Signed-off-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
    Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
    c1abd865
tid_rdma.c 156 KB