- 18 Oct, 2023 4 commits
-
-
Jonathan Cavitt authored
In case of GT is suspended, don't allow submission of new TLB invalidation request and cancel all pending requests. The TLB entries will be invalidated either during GuC reload or on system resume. Signed-off-by:
Fei Yang <fei.yang@intel.com> Signed-off-by:
Jonathan Cavitt <jonathan.cavitt@intel.com> CC: John Harrison <john.c.harrison@intel.com> Reviewed-by:
Andi Shyti <andi.shyti@linux.intel.com> Acked-by:
Tvrtko Ursulin <tvrtko.ursulin@intel.com> Acked-by:
Nirmoy Das <nirmoy.das@intel.com> Signed-off-by:
-
Prathap Kumar Valsan authored
The GuC firmware had defined the interface for Translation Look-Aside Buffer (TLB) invalidation. We should use this interface when invalidating the engine and GuC TLBs. Add additional functionality to intel_gt_invalidate_tlb, invalidating the GuC TLBs and falling back to GT invalidation when the GuC is disabled. The invalidation is done by sending a request directly to the GuC tlb_lookup that invalidates the table. The invalidation is submitted as a wait request and is performed in the CT event handler. This means we cannot perform this TLB invalidation path if the CT is not enabled. If the request isn't fulfilled in two seconds, this would constitute an error in the invalidation as that would constitute either a lost request or a severe GuC overload. With this new invalidation routine, we can perform GuC-based GGTT invalidations. GuC-based GGTT invalidation is incompatible with MMIO invalidation so we should not perform MMIO invalidation when GuC-based GGTT invalidation is expected. The additional complexity incurred in this patch will be necessary for range-based tlb invalidations, which will be platformed in the future. Signed-off-by:
Prathap Kumar Valsan <prathap.kumar.valsan@intel.com> Signed-off-by:
Bruce Chang <yu.bruce.chang@intel.com> Signed-off-by:
Chris Wilson <chris.p.wilson@intel.com> Signed-off-by:
Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com> Signed-off-by:
Aravind Iddamsetty <aravind.iddamsetty@intel.com> Signed-off-by:
John Harrison <John.C.Harrison@Intel.com> Signed-off-by:
-
Jonathan Cavitt authored
As of now, there is no mechanism for tracking a given request's progress through the queue. Instead, add a helper that returns an estimated maximum time the queue should take to drain if completely full. Suggested-by:
-
Jonathan Cavitt authored
Add device info flags for if GuC TLB Invalidation is enabled. Signed-off-by:
-
- 17 Oct, 2023 2 commits
-
-
Ville Syrjälä authored
Just use a simple {} to zero initialize arrays/structs instead of the hodgepodge of stuff we are using currently. Signed-off-by:Ville Syrjälä <ville.syrjala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20231012122442.15718-6-ville.syrjala@linux.intel.comReviewed-by:
Jani Nikula <jani.nikula@intel.com>
-
Ville Syrjälä authored
Just use a simple {} to zero initialize arrays/structs instead of the hodgepodge of stuff we are using currently. Signed-off-by:
-
- 16 Oct, 2023 1 commit
-
-
Ville Syrjälä authored
If we can't find a free fence register to handle a fault in the GMADR range just return VM_FAULT_NOPAGE without populating the PTE so that userspace will retry the access and trigger another fault. Eventually we should find a free fence and the fault will get properly handled. A further improvement idea might be to reserve a fence (or one per CPU?) for the express purpose of handling faults without having to retry. But that would require some additional work. Looks like this may have gotten broken originally by commit 39965b37 ("drm/i915: don't trash the gtt when running out of fences") as that changed the errno to -EDEADLK which wasn't handle by the gtt fault code either. But later in commit 2feeb528 ("drm/i915/gt: Fix -EDEADLK handling regression") I changed it again to -ENOBUFS as -EDEADLK was now getting used for the ww mutex dance. So this fix only makes sense after that last commit. Cc: stable@vger.kernel.org Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/9479 Fixes: 2feeb528 ("drm/i915/gt: Fix -EDEADLK handling regression") Signed-off-by:
-
- 10 Oct, 2023 2 commits
-
-
John Harrison authored
Update a bunch of GT related print messages in non-GT files to use the GT specific helpers. Signed-off-by:
-
John Harrison authored
A bunch of print messages got missed in the update to using sub-system specific helpers. So update those. Signed-off-by:
-
- 09 Oct, 2023 2 commits
-
-
Daniele Ceraolo Spurio authored
The GuC handles the WA, the KMD just needs to set the flag to enable it on the appropriate platforms. Signed-off-by:
Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Reviewed-by:
Vinay Belgaumkar <vinay.belgaumkar@intel.com> Reviewed-by:
-
Kees Cook authored
Prepare for the coming implementation by GCC and Clang of the __counted_by attribute. Flexible array members annotated with __counted_by can have their accesses bounds-checked at run-time via CONFIG_UBSAN_BOUNDS (for array indexing) and CONFIG_FORTIFY_SOURCE (for strcpy/memcpy-family functions). As found with Coccinelle[1], add __counted_by for struct ct_incoming_msg. Cc: Jani Nikula <jani.nikula@linux.intel.com> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Cc: David Airlie <airlied@gmail.com> Cc: Daniel Vetter <daniel@ffwll.ch> Cc: "Gustavo A. R. Silva" <gustavoars@kernel.org> Cc: John Harrison <John.C.Harrison@Intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: Matt Roper <matthew.d.roper@intel.com> Cc: intel-gfx@lists.freedesktop.org Cc: dri-devel@lists.freedesktop.org Cc: linux-hardening@vger.kernel.org Link: https://github.com/kees/kernel-tools/blob/trunk/coccinelle/examples/counted_by.cocci [1] Signed-off-by:
Kees Cook <keescook@chromium.org> Reviewed-by:
Gustavo A. R. Silva <gustavoars@kernel.org> Reviewed-by:
-
- 07 Oct, 2023 1 commit
-
-
John Harrison authored
The latest GuC has new features and new workarounds that we wish to enable. So let the universe know that it is useful to update their firmware. Signed-off-by:
-
- 04 Oct, 2023 1 commit
-
-
John Harrison authored
If an active context has been banned (e.g. Ctrl+C killed) then it is likely to be reset as part of evicting it from the hardware. That results in a 'ignoring context reset notification: banned = 1' message at info level. This confuses/concerns people and makes them think something has gone wrong when it hasn't. There is already a debug level message with essentially the same information. So drop the 'ignore' info level one and just add the 'ignore' flag to the debug level one instead (which will therefore not appear by default but will still show up in CI runs). Signed-off-by:
-
- 03 Oct, 2023 3 commits
-
-
Chris Wilson authored
With multi-GT devices, the object may have been bound on each GT and so we need to invalidate the TLBs across all GT before releasing the pages back to the system. Fixes: d6c531ab ("drm/i915: Invalidate the TLBs on each GT") Signed-off-by:
-
Mathias Krause authored
Chaining user engines happens in multiple passes during driver initialization, mutating its type along the way. It starts off with a simple lock-less linked list (struct llist_node/head) populated by intel_engine_add_user() which later gets sorted and converted to an intermediate regular list (struct list_head) just to be converted once more to its final rb-tree structure (struct rb_node/root) in intel_engines_driver_register(). All of these types overlay the uabi_node/uabi_engines members which is unfortunate but safe if one takes care about using the rb-tree based structure only after the conversion has completed. However, mistakes happen and commit 1ec23ed7 ("drm/i915: Use uabi engines for the default engine map") violated that assumption, as the multiple type evolution was all to easy hidden behind casts papering over it. Make the type evolution of uabi_node/uabi_engines more visible by putting all members into an anonymous union and use the correctly typed member in its various users. This allows us to drop quite some ugly casts and, hopefully, make the evolution of the members better recognisable to avoid future mistakes. Signed-off-by:
Mathias Krause <minipli@grsecurity.net> Reviewed-by:
-
Mathias Krause authored
Commit 1ec23ed7 ("drm/i915: Use uabi engines for the default engine map") switched from using for_each_engine() to for_each_uabi_engine() to iterate over the user engines. While this seems to be a sensible change, it's only safe to do when the engines are actually chained using the rb-tree structure which is not the case during early driver initialization where it can be either a lock-less list or regular double-linked list. In fact, the modesetting initialization code may end up calling default_engines() through the fb helper code while the engines list is still llist_node-based: i915_driver_probe() -> intel_display_driver_probe() -> intel_fbdev_init() -> drm_fb_helper_init() -> drm_client_init() -> drm_client_open() -> drm_file_alloc() -> i915_driver_open() -> i915_gem_open() -> i915_gem_context_open() -> i915_gem_create_context() -> default_engines() Using for_each_uabi_engine() in default_engines() is therefore wrong, as it would try to interpret the llist as rb-tree, making it find no engine at all, as the rb_left and rb_right members will still be NULL, as they haven't been initialized yet. To fix this type confusion register the engines earlier and at the same time reduce the amount of code that has to deal with the intermediate llist state. Reported-by: sanity checks in grsecurity Suggested-by:
-
- 30 Sep, 2023 8 commits
-
-
Nirmoy Das authored
MTL can hang because of a HW bug while parallel reading/writing from/to LMEM/GTTMMADR BAR so try to reduce GGTT update related pci transactions with blitter command as recommended for Wa_13010847436 and Wa_14019519902. Signed-off-by:
Oak Zeng <oak.zeng@intel.com> Reviewed-by:
-
Nirmoy Das authored
Toggle binder context ready status when needed. To issue gpu commands, the driver must be primed to receive requests. Maintain binder-based GGTT update disablement until driver probing completes. Moreover, implement a temporary disablement of blitter prior to entering suspend, followed by re-enablement post-resume. This is acceptable as those transition periods are mostly single threaded. v2: move changes to lower levels from i915_driver.c(Jani). use new function for setting context ready status. Signed-off-by: -
Nirmoy Das authored
Implement GGTT update method with blitter command, MI_UPDATE_GTT and install those handlers if a platform requires that. v2: Make sure we hold the GT wakeref and Blitter engine wakeref before we call mutex_lock/intel_context_enter below. When GT/engine are not awake, the intel_context_enter calls into some runtime pm function which can end up with kmalloc/fs_reclaim. But trigger fs_reclaim holding a mutex lock is not allowed because shrinker can also try to hold the same mutex lock. It is a circular lock. So hold the GT/blitter engine wakeref before calling mutex_lock, to fix the circular lock. Signed-off-by:
-
Nirmoy Das authored
Add i915_ggtt_require_binder() to indicate that i915 needs to create binder context which will be used by subsequent patch to enable i915_address_space vfuncs that will use GPU commands to update GGTT. Signed-off-by:
-
Nirmoy Das authored
Implement a way to iterate over sgt with pre-initialized sgt_iter state. Signed-off-by:
-
Nirmoy Das authored
Create a separate kernel context if a platform requires GGTT updates using MI_UPDATE_GTT blitter command. Subsequent patch will introduce methods to update GGTT using this bind context and MI_UPDATE_GTT blitter command. v2: fix context leak on err(Oak) v3: improve err handling and improve function names(Andi) add docs for few functions. Signed-off-by: -
Chris Wilson authored
When runtime pm is first woken, it will synchronously call the registered callbacks for the device. These callbacks may pull in their own forest of locks, which we do not want to conflate with the intel_wakeref.mutex. A second minor benefit to reducing the coverage of the mutex, is that it will reduce contention for frequent sleeps and wakes (such as when being used for soft-rc6). v2: remove usage of fetch_and_zero() and other improvements(Jani) Signed-off-by:
-
Jonathan Cavitt authored
Where applicable, use for_each_gt instead of to_gt in the i915_gem_shrinker functions to make them apply to more than just the primary GT. Specifically, this ensure i915_gem_shrink_all retires all requests across all GTs, and this makes i915_gem_shrinker_vmap unmap VMAs from all GTs. v2: Pass correct GT to intel_gt_retire_requests(Andrzej). v3: Remove unnecessary braces(Andi) v4: Undo v3 to fix build failure. Signed-off-by:
Andrzej Hajda <andrzej.hajda@intel.com> Reviewed-by:
-
- 29 Sep, 2023 4 commits
-
-
Nirmoy Das authored
On MTL GEN12_RING_FAULT_REG is not replicated so don't do mcr based operation for this register. v2: use MEDIA_VER() instead of GRAPHICS_VER()(Matt). v3: s/"MEDIA_VER(i915) == 13"/"MEDIA_VER(i915) >= 13"(Matt) improve comment. v4: improve the comment further(Andi) Signed-off-by:Matt Roper <matthew.d.roper@intel.com> Reviewed-by:
-
Nirmoy Das authored
During resume, the steer semaphore on GT1 was observed to be held. The hardware team has confirmed the safety of clearing steer semaphores for all GTs during driver load/resume, as no lock acquisitions can occur in this process by other agents. v2: reset on resume not in intel_gt_init(). v3: do the reset on intel_gt_resume_early() v4: do general sanitization for all GTs(Matt) Signed-off-by:
-
Nirmoy Das authored
Move early resume functions of gt to a proper file. Signed-off-by:
-
Nirmoy Das authored
Implement intel_gt_mcr_lock_sanitize() to provide a mechanism for cleaning the steer semaphore when absolutely necessary. v2: remove unnecessary lock(Andi, Matt) improve the kernel doc(Matt) s/intel_gt_mcr_lock_clear/intel_gt_mcr_lock_sanitize Signed-off-by:
-
- 28 Sep, 2023 1 commit
-
-
Nirmoy Das authored
PIPE_CONTROL_FLUSH_L3 is not needed for aux invalidation so don't set that. Fixes: ad8ebf12 ("drm/i915/gt: Ensure memory quiesced before invalidation") Cc: Jonathan Cavitt <jonathan.cavitt@intel.com> Cc: Andi Shyti <andi.shyti@linux.intel.com> Cc: <stable@vger.kernel.org> # v5.8+ Cc: Andrzej Hajda <andrzej.hajda@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Matt Roper <matthew.d.roper@intel.com> Cc: Tejas Upadhyay <tejas.upadhyay@intel.com> Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com> Cc: Tapani Pälli <tapani.palli@intel.com> Cc: Mark Janes <mark.janes@intel.com> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com> Signed-off-by:
Tapani Pälli <tapani.palli@intel.com> Reviewed-by:
-
- 27 Sep, 2023 1 commit
-
-
Tvrtko Ursulin authored
Commit ade8a0f5 ("drm/i915: Make all GPU resets atomic") added a preempt disable section over the hardware reset callback to prepare the driver for being able to reset from atomic contexts. In retrospect I can see that the work item at a time was about removing the struct mutex from the reset path. Code base also briefly entertained the idea of doing the reset under stop_machine in order to serialize userspace mmap and temporary glitch in the fence registers (see eb8d0f5a ("drm/i915: Remove GPU reset dependence on struct_mutex"), but that never materialized and was soon removed in 2caffbf1 ("drm/i915: Revoke mmaps and prevent access to fence registers across reset") and replaced with a SRCU based solution. As such, as far as I can see, today we still have a requirement that resets must not sleep (invoked from submission tasklets), but no need to support invoking them from a truly atomic context. Given that the preemption section is problematic on RT kernels, since the uncore lock becomes a sleeping lock and so is invalid in such section, lets try and remove it. Potential downside is that our short waits on GPU to complete the reset may get extended if CPU scheduling interferes, but in practice that probably isn't a deal breaker. In terms of mechanics, since the preemption disabled block is being removed we just need to replace a few of the wait_for_atomic macros into busy looping versions which will work (and not complain) when called from non-atomic sections. v2: * Fix timeouts which are now in us. (Andi) * Update one comment as a drive by. (Andi) Signed-off-by:
-
- 26 Sep, 2023 2 commits
-
-
Umesh Nerlige Ramappa authored
Ideally the busyness worker should take a gt pm wakeref because the worker only needs to be active while gt is awake. However, the gt_park path cancels the worker synchronously and this complicates the flow if the worker is also running at the same time. The cancel waits for the worker and when the worker releases the wakeref, that would call gt_park and would lead to a deadlock. The resolution is to take the global pm wakeref if runtime pm is already active. If not, we don't need to update the busyness stats as the stats would already be updated when the gt was parked. Note: - We do not requeue the worker if we cannot take a reference to runtime pm since intel_guc_busyness_unpark would requeue the worker in the resume path. - If the gt was parked longer than time taken for GT timestamp to roll over, we ignore those rollovers since we don't care about tracking the exact GT time. We only care about roll overs when the gt is active and running workloads. - There is a window of time between gt_park and runtime suspend, where the worker may run. This is acceptable since the worker will not find any new data to update busyness. v2: (Daniele) - Edit commit message and code comment - Use runtime pm in the worker - Put runtime pm after enabling the worker - Use Link tag and add Fixes tag v3: (Daniele) - Reword commit and comments and add details Link: https://gitlab.freedesktop.org/drm/intel/-/issues/7077 Fixes: 77cdd054 ("drm/i915/pmu: Connect engine busyness stats from GuC to pmu") Signed-off-by:
-
Ashutosh Dixit authored
There is no reason to add gtt_offset to the cached head/tail pointers stream->oa_buffer.head and stream->oa_buffer.tail. This causes the code to constantly add gtt_offset and subtract gtt_offset and is error prone. It is much simpler to maintain stream->oa_buffer.head and stream->oa_buffer.tail without adding gtt_offset to them and just allow for the gtt_offset when reading/writing from/to HW registers. v2: Minor tweak to commit message due to dropping patch in previous series Signed-off-by:
Ashutosh Dixit <ashutosh.dixit@intel.com> Reviewed-by:
-
- 25 Sep, 2023 3 commits
-
-
Shekhar Chauhan authored
Drop UGM per set fragment threshold to 3 BSpec: 54833 Signed-off-by:
Shekhar Chauhan <shekhar.chauhan@intel.com> Reviewed-by:
-
Jani Nikula authored
Just let the compiler decide what's best. Turns out absolutely nothing changes in the output with the inlines removed. Signed-off-by:
Rodrigo Vivi <rodrigo.vivi@intel.com> Reviewed-by:
-
Jani Nikula authored
It's actively harmful to add static inlines in headers that require you to pull in more headers. Remove the include added in commit f1530f91 ("drm/i915/gt: Apply workaround 22016122933 correctly"). We see that there's already an implicit dependency on the i915_drv.h that we need to address too. Cc: Andi Shyti <andi.shyti@linux.intel.com> Cc: Fei Yang <fei.yang@intel.com> Cc: Jonathan Cavitt <jonathan.cavitt@intel.com> Cc: Matt Roper <matthew.d.roper@intel.com> Signed-off-by:
-
- 21 Sep, 2023 3 commits
-
-
Javier Pello authored
There is an assertion in ggtt_reserve_guc_top that the global GTT is of size at least GUC_GGTT_TOP, which is not the case on a 32-bit platform; see commit 562d55d9 ("drm/i915/bdw: Only use 2g GGTT for 32b platforms"). If GEM_BUG_ON is enabled, this triggers a BUG(); if GEM_BUG_ON is disabled, the subsequent reservation fails and the driver fails to initialise the device: i915 0000:00:02.0: [drm:i915_init_ggtt [i915]] Failed to reserve top of GGTT for GuC i915 0000:00:02.0: Device initialization failed (-28) i915 0000:00:02.0: Please file a bug on drm/i915; see https://gitlab.freedesktop.org/drm/intel/-/wikis/How-to-file-i915-bugs for details. i915: probe of 0000:00:02.0 failed with error -28 Make the reservation at the top of the available space, whatever that is, instead of assuming that the top will be GUC_GGTT_TOP. Fixes: 91180076 ("drm/i915/uc: Reserve upper range of GGTT") Link: https://gitlab.freedesktop.org/drm/intel/-/issues/9080Signed-off-by:
Javier Pello <devel@otheo.eu> Reviewed-by:
-
Badal Nilawar authored
It has been observed sometimes RC6 status register's unused bits are being set by h/w, without affecting RC6 functionality therefore updating the mask with used bits accordingly. As mtl_drpc is debugfs function, removing MISSING_CASE from default case as it doesn't make sense to panic (panic_on_warn=1) the CI system if register is reporting unsupported state. Cc: Anshuman Gupta <anshuman.gupta@intel.com> Signed-off-by:
Badal Nilawar <badal.nilawar@intel.com> Reviewed-by:
Anshuman Gupta <anshuman.gupta@intel.com> Signed-off-by:
-
Matthew Wilcox (Oracle) authored
The folio conversion changed the behaviour of shmem_sg_alloc_table() to put the entire length of the last folio into the sg list, even if the sg list should have been shorter. gen8_ggtt_insert_entries() relied on the list being the right length and would overrun the end of the page tables. Other functions may also have been affected. Clamp the length of the last entry in the sg list to be the expected length. Signed-off-by:
Matthew Wilcox (Oracle) <willy@infradead.org> Fixes: 0b62af28 ("i915: convert shmem_sg_free_table() to use a folio_batch") Cc: stable@vger.kernel.org # 6.5.x Link: https://gitlab.freedesktop.org/drm/intel/-/issues/9256 Link: https://lore.kernel.org/lkml/6287208.lOV4Wx5bFT@natalenko.name/Reported-by:
Oleksandr Natalenko <oleksandr@natalenko.name> Tested-by:
-
- 20 Sep, 2023 1 commit
-
-
Justin Stitt authored
`strncpy` is deprecated for use on NUL-terminated destination strings [1]. We should prefer more robust and less ambiguous string interfaces. A suitable replacement is `strscpy` [2] due to the fact that it guarantees NUL-termination on the destination buffer without unnecessarily NUL-padding. `ctx` is zero allocated and as such strncpy's NUL-padding behavior was strictly a performance hit which is now resolved. Link: https://www.kernel.org/doc/html/latest/process/deprecated.html#strncpy-on-nul-terminated-strings [1] Link: https://manpages.debian.org/testing/linux-manual-4.8/strscpy.9.en.html [2] Link: https://github.com/KSPP/linux/issues/90 Cc: linux-hardening@vger.kernel.org Signed-off-by:
Justin Stitt <justinstitt@google.com> Reviewed-by:
-
- 19 Sep, 2023 1 commit
-
-
Alan Previn authored
On Meteorlake onwards, HW specs require that all user contexts that run on render or compute engines and require PXP must enforce run-alone bit in lrc. Add this enforcement for protected contexts. Signed-off-by:
Alan Previn <alan.previn.teres.alexis@intel.com> Reviewed-by:
Vivaik Balasubrawmanian <vivaik.balasubrawmanian@intel.com> Signed-off-by:
-
