• Matthew Brost's avatar
    drm/sched: Convert drm scheduler to use a work queue rather than kthread · a6149f03
    Matthew Brost authored
    In Xe, the new Intel GPU driver, a choice has made to have a 1 to 1
    mapping between a drm_gpu_scheduler and drm_sched_entity. At first this
    seems a bit odd but let us explain the reasoning below.
    
    1. In Xe the submission order from multiple drm_sched_entity is not
    guaranteed to be the same completion even if targeting the same hardware
    engine. This is because in Xe we have a firmware scheduler, the GuC,
    which allowed to reorder, timeslice, and preempt submissions. If a using
    shared drm_gpu_scheduler across multiple drm_sched_entity, the TDR falls
    apart as the TDR expects submission order == completion order. Using a
    dedicated drm_gpu_scheduler per drm_sched_entity solve this problem.
    
    2. In Xe submissions are done via programming a ring buffer (circular
    buffer), a drm_gpu_scheduler provides a limit on number of jobs, if the
    limit of number jobs is set to RING_SIZE / MAX_SIZE_PER_JOB we get flow
    control on the ring for free.
    
    A problem with this design is currently a drm_gpu_scheduler uses a
    kthread for submission / job cleanup. This doesn't scale if a large
    number of drm_gpu_scheduler are used. To work around the scaling issue,
    use a worker rather than kthread for submission / job cleanup.
    
    v2:
      - (Rob Clark) Fix msm build
      - Pass in run work queue
    v3:
      - (Boris) don't have loop in worker
    v4:
      - (Tvrtko) break out submit ready, stop, start helpers into own patch
    v5:
      - (Boris) default to ordered work queue
    v6:
      - (Luben / checkpatch) fix alignment in msm_ringbuffer.c
      - (Luben) s/drm_sched_submit_queue/drm_sched_wqueue_enqueue
      - (Luben) Update comment for drm_sched_wqueue_enqueue
      - (Luben) Positive check for submit_wq in drm_sched_init
      - (Luben) s/alloc_submit_wq/own_submit_wq
    v7:
      - (Luben) s/drm_sched_wqueue_enqueue/drm_sched_run_job_queue
    v8:
      - (Luben) Adjust var names / comments
    Signed-off-by: default avatarMatthew Brost <matthew.brost@intel.com>
    Reviewed-by: default avatarLuben Tuikov <luben.tuikov@amd.com>
    Link: https://lore.kernel.org/r/20231031032439.1558703-3-matthew.brost@intel.comSigned-off-by: default avatarLuben Tuikov <ltuikov89@gmail.com>
    a6149f03
gpu_scheduler.h 20.2 KB