These 2 functions are coded by multiple person in multiple patches. The
'return' and 'goto err' are mix-used in same place, which cause the
function looks disorder. Unify to use only 'goto' so that the gvt lock
is acquired in one place and released in one place.
Signed-off-by: Pei Zhang <pei.zhang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

In the process of dmabuf_obj cleanup, the dmabuf_obj might be freed during
dmabuf_obj_put leaking intel_gvt_hypervisor_put_vfio_device.

Move intel_gvt_hypervisor_put_vfio_device and all the other dmabuf_obj ops
in front of dmabuf_obj_put and let every dmabuf_obj have a chance to call
intel_gvt_hypervisor_put_vfio_device to fix this leaking issue.

Fixes: e3a0d7976c53 ("drm/i915/gvt: Handle orphan dmabuf_objs")
Signed-off-by: Tina Zhang <tina.zhang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Both ROM/VGA region are not supported for vGPU in GVT. But if the device
model want to get those region, we should return the correct information
but not leave the structure with random data. Change to same operation
of BAR3-BAR5 which are also not supported by vGPU.

Refer to function @intel_vgpu_rw.
Signed-off-by: Pei Zhang <pei.zhang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Rename the files to reflect their real role - to switch the mmio context of
each vGPU engine.

v2: update Makefile.
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

After engine mmio switched, software still need write workload
submission registers. So we can remove the MMIO barriar in MMIO
switch.
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Select appropriate mmio list at initialization time, so we don't need to
do duplicated work at where requires the mmio list.

V2:
  - Add a termination mark of mmio list.
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

To improve the readability, let's remove the hard code for each mmio
definition. The raw offset remained as a comment, which give us an
offset based view.

This refine is to make it convenient for new platform enabling.
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

The ring_id maximum boundary is being compared using the > operator
instead of >=, leading to an off-by-one error and an out of bounds
write into array vgpu->hws_pga[].  Fix this by simply using the
correct comparison operator. Also re-work another comparison that
uses the comparison > I915_NUM_ENGINES - 1 to use the >= idiom using
to keep this consistent in this code.

Detected by CoverityScan, CID#1462404 ("Out-of-bounds write")

Fixes: a2ae95af ("drm/i915/gvt: update CSB and CSB write pointer in virtual HWSP")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

The switch statement is missing breaks for the cases of
GVT_FAILSAFE_INSUFFICIENT_RESOURCE and GVT_FAILSAFE_GUEST_ERR. Add them
in.

Detected by CoverityScan, CID#1462416 ("Missing break in switch")

Fixes: e011c6ce ("drm/i915/gvt: Add VM healthy check for workload_thread")
Fixes: a33fc7a0 ("drm/i915/gvt: enter failsafe mode when guest requires more resources")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

This is to workaround guest driver hang regression after
preemption enable that gvt hasn't enabled handling of that
for guest workload. So in effect this disables preemption
for gvt context now.
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

We shouldn't mark inactive for vGPU context if preempted,
which would still be re-scheduled later. So keep active state.

Fixes: d6c05113 ("drm/i915/execlists: Distinguish the incomplete context notifies")
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

The current schedule policy rely on a 1ms timer to execute workload. This
can introduce maximum 1ms unnecessary latency. This is especially bad for
small media workloads.

And I don't think we need this timer for QoS, but the change is not simply
remove the code. So I made a new API intel_gvt_kick_schedule() for future
change.
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Convert the macro to a function which should always be preferred.
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

The per vGPU dmabuf_obj list should be released in intel_vgpu_dmabuf_
cleanup, which is invoked either in the process of closing a VM or in
the process of removing a vGPU.

Fixes: e3a0d7976c53 ("drm/i915/gvt: Handle orphan dmabuf_objs")
Signed-off-by: Tina Zhang <tina.zhang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

This patch introduces KBL platform to dma-buf on Gvt-g.
Signed-off-by: Tina Zhang <tina.zhang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

dmabuf_obj's destruction relys on GEM release operation, which is managed
in i915 driver. And there is a time window between vgpu's destruction and
its dmabuf_objs' destruction. This patch is to free the orphan dmabuf_objs
correctly after the vgpu passes away.
Signed-off-by: Tina Zhang <tina.zhang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

This patch introduces a guest's framebuffer sharing mechanism based on
dma-buf subsystem. With this sharing mechanism, guest's framebuffer can
be shared between guest VM and host.

v17:
- modify VFIO_DEVICE_GET_GFX_DMABUF interface. (Alex)

v16:
- add x_hot and y_hot. (Gerd)
- add flag validation for VFIO_DEVICE_GET_GFX_DMABUF. (Alex)
- rebase 4.14.0-rc6.

v15:
- add VFIO_DEVICE_GET_GFX_DMABUF ABI. (Gerd)
- add intel_vgpu_dmabuf_cleanup() to clean up the vGPU's dmabuf. (Gerd)

v14:
- add PROBE, DMABUF and REGION flags. (Alex)

v12:
- refine the lifecycle of dmabuf.

v9:
- remove dma-buf management. (Alex)
- track the dma-buf create and release in kernel mode. (Gerd) (Daniel)

v8:
- refine the dma-buf ioctl definition.(Alex)
- add a lock to protect the dmabuf list. (Alex)

v7:
- release dma-buf related allocations in dma-buf's associated release
  function. (Alex)
- refine ioctl interface for querying plane info or create dma-buf.
  (Alex)

v6:
- align the dma-buf life cycle with the vfio device. (Alex)
- add the dma-buf related operations in a separate patch. (Gerd)
- i915 related changes. (Chris)

v5:
- fix bug while checking whether the gem obj is gvt's dma-buf when user
  change caching mode or domains. Add a helper function to do it.
  (Xiaoguang)
- add definition for the query plane and create dma-buf. (Xiaoguang)

v4:
- fix bug while checking whether the gem obj is gvt's dma-buf when set
  caching mode or doamins. (Xiaoguang)

v3:
- declare a new flag I915_GEM_OBJECT_IS_GVT_DMABUF in drm_i915_gem_object
  to represent the gem obj for gvt's dma-buf. The tiling mode, caching
  mode and domains can not be changed for this kind of gem object. (Alex)
- change dma-buf related information to be more generic. So other vendor
  can use the same interface. (Alex)

v2:
- create a management fd for dma-buf operations. (Alex)
- alloc gem object's backing storage in gem obj's get_pages() callback.
  (Chris)
Signed-off-by: Tina Zhang <tina.zhang@intel.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Add VFIO_DEVICE_QUERY_GFX_PLANE ioctl command to let user query and get
a plane and its information. So far, two types of buffers are supported:
buffers based on dma-buf and buffers based on region.

This ioctl can be invoked with:
1) Either DMABUF or REGION flag. Vendor driver returns a plane_info
successfully only when the specific kind of buffer is supported.
2) Flag PROBE. And at the same time either DMABUF or REGION must be set,
so that vendor driver returns success only when the specific kind of
buffer is supported.

Add VFIO_DEVICE_GET_GFX_DMABUF ioctl command to let user get a specific
dma-buf fd of an exposed MDEV buffer provided by dmabuf_id which was
returned in VFIO_DEVICE_QUERY_GFX_PLANE ioctl command.

The life cycle of an exposed MDEV buffer is handled by userspace and
tracked by kernel space. The returned dmabuf_id in struct vfio_device_
query_gfx_plane can be a new id of a new exposed buffer or an old id of
a re-exported buffer. Host user can check the value of dmabuf_id to see
if it needs to create new resources according to the new exposed buffer
or just re-use the existing resource related to the old buffer.

v18:
- update comments for VFIO_DEVICE_GET_GFX_DMABUF. (Alex)

v17:
- modify VFIO_DEVICE_GET_GFX_DMABUF interface. (Alex)

v16:
- add x_hot and y_hot fields. (Gerd)
- add comments for VFIO_DEVICE_GET_GFX_DMABUF. (Alex)
- rebase to 4.14.0-rc6.

v15:
- add a ioctl to get a dmabuf for a given dmabuf id. (Gerd)

v14:
- add PROBE, DMABUF and REGION flags. (Alex)

v12:
- add drm_format_mod back. (Gerd and Zhenyu)
- add region_index. (Gerd)

v11:
- rename plane_type to drm_plane_type. (Gerd)
- move fields of vfio_device_query_gfx_plane to vfio_device_gfx_plane_info.
  (Gerd)
- remove drm_format_mod, start fields. (Daniel)
- remove plane_id.

v10:
- refine the ABI API VFIO_DEVICE_QUERY_GFX_PLANE. (Alex) (Gerd)

v3:
- add a field gvt_plane_info in the drm_i915_gem_obj structure to save
  the decoded plane information to avoid look up while need the plane
  info. (Gerd)
Signed-off-by: Tina Zhang <tina.zhang@intel.com>
Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Kirti Wankhede <kwankhede@nvidia.com>
Acked-by: Alex Williamson <alex.williamson@redhat.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

This patch is to introduce the framebuffer decoder which can decode guest
OS's framebuffer information, including primary, cursor and sprite plane.

v16:
- rebase to 4.14.0-rc6.

v14:
- refine pixel format table. (Zhenyu)

v9:
- move drm format change to a separate patch. (Xiaoguang)

v8:
- fix a bug in decoding primary plane. (Tina)

v7:
- refine framebuffer decoder code. (Zhenyu)
Signed-off-by: Tina Zhang <tina.zhang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Windows guest driver needs vbt in opregion, to configure the setting
for display. Without opregion support, the display registers won't
be set and this blocks display model to get the correct information
of the guest display plane.

This patch is to provide a virtual opregion for guest. The original
author of this patch is Xiaoguang Chen.

This patch is split from the "Dma-buf support for GVT-g" patch set,
with being rebased to the latest gvt-staging branch.

v3:
- add checking region index during intel_vgpu_rw. (Xiong)

v2:
- refine intel_vgpu_reg_release_opregion. (Xiong)

Here are the previous version comments:

v18:
- unmap vgpu's opregion when destroying vgpu.

v16:
- rebase to 4.14.0-rc6.
Signed-off-by: Bing Niu <bing.niu@intel.com>
Signed-off-by: Tina Zhang <tina.zhang@intel.com>
Tested-by: Xiong Zhang <xiong.y.zhang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Currently guest opregion is allocated and initialised when guest
write opregion base register. This is too late for kvmgt, so
move it to vgpu_create time.
Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com>
Tested-by: Tina Zhang <tina.zhang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

sparse spots

drivers/gpu/drm/i915/gvt/opregion.c:234 alloc_and_init_virt_opregion() error: memcpy() 'header->signature' too small (16 vs 17)

as gvt is indeed trying to memcpy a string longer than the signature[].

Fixes: b2d6ef70 ("drm/i915/gvt: Let each vgpu has separate opregion memory")
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Xiong Zhang <xiong.y.zhang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: intel-gvt-dev@lists.freedesktop.org
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

drivers/gpu/drm/i915/gvt/execlist.c:531:6: warning: symbol 'clean_execlist' was not declared. Should it be static?
drivers/gpu/drm/i915/gvt/execlist.c:545:6: warning: symbol 'reset_execlist' was not declared. Should it be static?
drivers/gpu/drm/i915/gvt/execlist.c:556:5: warning: symbol 'init_execlist' was not declared. Should it be static?
drivers/gpu/drm/i915/gvt/scheduler.c:248:6: warning: symbol 'release_shadow_wa_ctx' was not declared. Should it be static?

References: 06bb372f ("drm/i915/gvt: Introduce intel_vgpu_reset_submission")
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: intel-gvt-dev@lists.freedesktop.org
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Previously the performance is improved through the workload auditing
and shadowing ahead of vGPU scheduling, however, there is the case that
more requests are allocated in submit_context before the previous request
is added, the timeline will hold its seqno which is later.

This patch is to move the request alloc to dispatch_workload function,
where is the same place as request is added.

It will fix the issue of kernel BUG for (timeline->seqno != request->fence.seqno)
check when add_request.

Fixes: 89ea20b9 ("drm/i915/gvt: Factor out scan and shadow from workload dispatch")
Signed-off-by: Chuanxiao Dong <chuanxiao.dong@intel.com>
Signed-off-by: fred gao <fred.gao@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Currently every vgpu share a common gvt opregion memory, but
it is freed at vgpu destroy, then the later vgpu doesn't have
opregion memory once the first vgpu is destroyed. This cause
guest function failure like reboot, second or later boot.

This patch allocate and init virt opregion memory for each
vgpu, so this memory could be freed at vgpu destroy.
Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

mmio_read_from_hw() let vgpu could read hw reg, if vgpu's workload
is running on hw, things is good. Otherwise vgpu will get other
vgpu's reg val, it is unsafe.

This patch limit such hw access to active vgpu. If vgpu isn't
running on hw, the reg read of this vgpu will get the last active
val which saved at schedule_out.

v2: ring timestamp is walking continuously even if the ring is idle.
    so read hw directly. (Zhenyu)
Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

This reverts commit b20d09886fd1b74cd2255d846029a049e524db14.

This caused windows driver boot errors for invalid page address.
Revert for now.
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Our vGPU doesn't have a device ROM, we need follow the PCI spec to
report this info to drivers. Otherwise, we would see below errors.

Inspecting possible rom at 0xfe049000 (vd=8086:1912 bdf=00:10.0)
qemu-system-x86_64: vfio-pci: Cannot read device rom at 00000000-0000-0000-0000-000000000001
Device option ROM contents are probably invalid (check dmesg).
Skip option ROM probe with rombar=0, or load from file with romfile=No option rom signature (got 4860)

I will also send a improvement patch to PCI subsystem related to PCI ROM.
But no idea to omit below error, since no pattern to detect vbios shadow
without touch its content.
0000:00:10.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0x0000
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

gvt_vgpu_err means something goes wrong. We need the error propagates to
kernel message by default.
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

I have seen the cmd parser dump partial odd info. Stop that and only dump
the full verbose info when debug enabled.
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Use I915_WRITE_FW instead of I915_WRITE to reduce overhead.
The overall mmio switch latency lowers from ~600us to ~180us.
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

gvt-linux.git repo is moved for its new place under
https://github.com/intel/gvt-linux.git

Old https://github.com/01org/gvt-linux.git is set
only for redirect now.
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

This new debugfs entry is used to figure out which registers of vGPU
is different to host. It is a useful tool for new platform enabling
and debugging. When read this entry, all the diff mmio are recognized
and sorted by mmio offset. Besides, the bit positions of different
value are listed in 'Diff' column. Here is a show:

$ sudo cat ./mmio_diff
Offset   HW       vGPU     Diff
00002030 000025f8 00000000 3-8,10,13
00002034 012025f8 00000000 3-8,10,13,21,24
00002038 027fb000 00000000 12-13,15-22,25
0000203c 00003000 00000000 12-13
00002054 0000000a 00000040 1,3,6
00002074 012025f8 00000000 3-8,10,13,21,24
00002080 fffe6000 00000000 13-14,17-31
000020a8 fffffeff ffffffff 8
000020d4 00000004 00000000 2
....
00145974 eb42718c 010c11b0 2-5,13-14,17-19,22,25,27,29-31
00145978 0000002f 0000002a 0,2
0014597c 0000002f 0000002a 0,2
00145980 0000002b 00000028 0-1
00145984 a5a87c9e b27d20c0 1-4,6,10-12,14,16,18,20,22-26,28
001459c0 88390000 883c0000 16,18
00146200 88350000 883a0000 16-19
Total: 72432, Diff: 901

v3: fix a typo.
v2: add mmio_hw_access_pre/post().
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

This patch add a function intel_gvt_for_each_tracked_mmio() to
iterate each tracked mmio. The caller don't be aware of how the
tracked mmios are presented internally.

v2: remove snapshot_hw_mmio_registers().
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

this is an enhanced opregion emulation for win guest support
by initializing more data members including opregion header
size, version and child device propertity for display port.
for simplicity, redefined child_device_config structure.
Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

The engine provides a mirror of the CSB and CSB write pointer in the HWSP.
Read these status from virtual HWSP in VM can reduce CPU utilization while
applications have much more short GPU workloads. Here we update the
corresponding data in virtual HWSP as it in virtual MMIO.

Before read these status from HWSP in GVT-g VM, please ensure the host
support it by checking the BIT(3) of caps in PVINFO.

Virtual HWSP only support GEN8+ platform, since the HWSP MMIO may change
follow the platform update, please add the corresponding MMIO emulation
when enable new platforms in GVT-g.

v3 : Add address audit in HWSP address update.

v4 :
     Separate this patch with enalbe virtual HWSP in VM.
     Use intel_gvt_render_mmio_to_ring_id() to determine ring_id by offset.

v5 : Remove unnessary check about Gen8, GVT-g only support Gen8+.
Signed-off-by: Weinan Li <weinan.z.li@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>

Refine previously broken PPGTT scratch. Scratch PTE was no correctly
handled and also the handling of scratch entries in page table walk was
not well organized, which brings gaps of introducing lazy shadow.
Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>

We need ops->set_present() during generating a new scratch page table
entry.
Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>

Need to figure out page table type of current level by GTT entry type
during getting a scratch page table entry.
Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>

During a vGPU reset, the scratch page table shouldn't be cleared, what
needs to be cleared should be the scratch page.
Signed-off-by: Zhi Wang <zhi.a.wang@intel.com>