• Jason Wang's avatar
    vdpa: mlx5: support per virtqueue dma device · 36871fb9
    Jason Wang authored
    This patch implements per virtqueue dma device for mlx5_vdpa. This is
    needed for virtio_vdpa to work for CVQ which is backed by vringh but
    not DMA. We simply advertise the vDPA device itself as the DMA device
    for CVQ then DMA API can simply use PA so the identical mapping for
    CVQ can still be used. Otherwise the identical (1:1) mapping won't
    work when platform IOMMU is enabled since the IOVA is allocated on
    demand which is not necessarily the PA.
    
    This fixes the following crash when mlx5 vDPA device is bound to
    virtio-vdpa with platform IOMMU enabled but not in passthrough mode:
    
    BUG: unable to handle page fault for address: ff2fb3063deb1002
    #PF: supervisor read access in kernel mode
    #PF: error_code(0x0000) - not-present page
    PGD 1393001067 P4D 1393002067 PUD 0
    Oops: 0000 [#1] PREEMPT SMP NOPTI
    CPU: 55 PID: 8923 Comm: kworker/u112:3 Kdump: loaded Not tainted 6.1.0+ #7
    Hardware name: Dell Inc. PowerEdge R750/0PJ80M, BIOS 1.5.4 12/17/2021
    Workqueue: mlx5_vdpa_wq mlx5_cvq_kick_handler [mlx5_vdpa]
    RIP: 0010:vringh_getdesc_iotlb+0x93/0x1d0 [vringh]
    Code: 14 25 40 ef 01 00 83 82 c0 0a 00 00 01 48 2b 05 93 5a 1b ea 8b 4c 24 14 48 c1 f8 06 48 c1 e0 0c 48 03 05 90 5a 1b ea 48 01 c8 <0f> b7 00 83 aa c0 0a 00 00 01 65 ff 0d bc e4 41 3f 0f 84 05 01 00
    RSP: 0018:ff46821ba664fdf8 EFLAGS: 00010282
    RAX: ff2fb3063deb1002 RBX: 0000000000000a20 RCX: 0000000000000002
    RDX: ff2fb318d2f94380 RSI: 0000000000000002 RDI: 0000000000000001
    RBP: ff2fb3065e832410 R08: ff46821ba664fe00 R09: 0000000000000001
    R10: 0000000000000000 R11: 000000000000000d R12: ff2fb3065e832488
    R13: ff2fb3065e8324a8 R14: ff2fb3065e8324c8 R15: ff2fb3065e8324a8
    FS:  0000000000000000(0000) GS:ff2fb3257fac0000(0000) knlGS:0000000000000000
    CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    CR2: ff2fb3063deb1002 CR3: 0000001392010006 CR4: 0000000000771ee0
    DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
    DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
    PKRU: 55555554
    Call Trace:
    <TASK>
      mlx5_cvq_kick_handler+0x89/0x2b0 [mlx5_vdpa]
      process_one_work+0x1e2/0x3b0
      ? rescuer_thread+0x390/0x390
      worker_thread+0x50/0x3a0
      ? rescuer_thread+0x390/0x390
      kthread+0xd6/0x100
      ? kthread_complete_and_exit+0x20/0x20
      ret_from_fork+0x1f/0x30
      </TASK>
    Reviewed-by: default avatarEli Cohen <elic@nvidia.com>
    Tested-by: default avatarEli Cohen <elic@nvidia.com>
    Signed-off-by: default avatarJason Wang <jasowang@redhat.com>
    Message-Id: <20230119061525.75068-6-jasowang@redhat.com>
    Signed-off-by: default avatarMichael S. Tsirkin <mst@redhat.com>
    36871fb9
mlx5_vnet.c 87.3 KB