- 26 Oct, 2023 8 commits
-
-
Jason Gunthorpe authored
In many cases the dev argument will now be !NULL so we should use it to finalize the domain at allocation. Make apple_dart_finalize_domain() accept the correct type. Reviewed-by:
Janne Grunau <j@jannau.net> Signed-off-by:
Jason Gunthorpe <jgg@nvidia.com> Acked-by:
Sven Peter <sven@svenpeter.dev> Link: https://lore.kernel.org/r/8-v2-bff223cf6409+282-dart_paging_jgg@nvidia.comSigned-off-by:
Joerg Roedel <jroedel@suse.de>
-
Jason Gunthorpe authored
Since the IDENTITY and BLOCKED behaviors were moved to global statics all that remains is the paging domain. Rename to apple_dart_attach_dev_paging() and remove the left over type check. Reviewed-by:
-
Jason Gunthorpe authored
Move to the new static global for blocked domains. Move the blocked specific code to apple_dart_attach_dev_blocked(). Reviewed-by:
-
Jason Gunthorpe authored
Move to the new static global for identity domains. Move the identity specific code to apple_dart_attach_dev_identity(). Reviewed-by:
-
Jason Gunthorpe authored
Move the global static blocked domain to the ops and convert the unmanaged domain to domain_alloc_paging. Signed-off-by:
Kevin Tian <kevin.tian@intel.com> Acked-by:
-
Jason Gunthorpe authored
Trivially migrate to the ops->blocked_domain for the existing global static. Reviewed-by:
Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by:
-
Jason Gunthorpe authored
The global static should pre-define the type and the NOP free function can be now left as NULL. Reviewed-by:
-
Jason Gunthorpe authored
Following the pattern of identity domains, just assign the BLOCKED domain global statics to a value in ops. Update the core code to use the global static directly. Update powerpc to use the new scheme and remove its empty domain_alloc callback. Reviewed-by:
-
- 16 Oct, 2023 1 commit
-
-
Dan Carpenter authored
The iommu_map_sgtable() function returns ssize_t and negative error codes but it's declared as size_t instead. I think that static checkers would have complained if this caused a bug, but even though it doesn't cause a bug, it's definitely worth fixing. Signed-off-by:
Dan Carpenter <dan.carpenter@linaro.org> Reviewed-by:
-
- 06 Oct, 2023 1 commit
-
-
Jason Gunthorpe authored
POWER throws a splat at boot, it looks like the DMA ops were probably changed while a driver was attached. Something is still weird about how power sequences its bootup. Previously this was hidden since the core iommu code did nothing during probe, now it calls spapr_tce_platform_iommu_attach_dev(). Make spapr_tce_platform_iommu_attach_dev() do nothing on the probe time call like it did before. WARNING: CPU: 0 PID: 8 at arch/powerpc/kernel/iommu.c:407 __iommu_free+0x1e4/0x1f0 Modules linked in: sd_mod t10_pi crc64_rocksoft crc64 sg ibmvfc mlx5_core(+) scsi_transport_fc ibmveth mlxfw psample dm_multipath dm_mirror dm_region_hash dm_log dm_mod fuse CPU: 0 PID: 8 Comm: kworker/0:0 Not tainted 6.6.0-rc3-next-20230929-auto #1 Hardware name: IBM,9080-HEX POWER10 (raw) 0x800200 0xf000006 of:IBM,FW1030.30 (NH1030_062) hv:phyp pSeries Workqueue: events work_for_cpu_fn NIP: c00000000005f6d4 LR: c00000000005f6d0 CTR: 00000000005ca81c REGS: c000000003a27890 TRAP: 0700 Not tainted (6.6.0-rc3-next-20230929-auto) MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 48000824 XER: 00000008 CFAR: c00000000020f738 IRQMASK: 0 GPR00: c00000000005f6d0 c000000003a27b30 c000000001481800 000000000000017 GPR04: 00000000ffff7fff c000000003a27950 c000000003a27948 0000000000000027 GPR08: c000000c18c07c10 0000000000000001 0000000000000027 c000000002ac8a08 GPR12: 0000000000000000 c000000002ff0000 c00000000019cc88 c000000003042300 GPR16: 0000000000000000 0000000000000000 0000000000000000 c000000003071ab0 GPR20: c00000000349f80d c000000003215440 c000000003215480 61c8864680b583eb GPR24: 0000000000000000 000000007fffffff 0800000020000000 0000000000000010 GPR28: 0000000000020000 0000800000020000 c00000000c5dc800 c00000000c5dc880 NIP [c00000000005f6d4] __iommu_free+0x1e4/0x1f0 LR [c00000000005f6d0] __iommu_free+0x1e0/0x1f0 Call Trace: [c000000003a27b30] [c00000000005f6d0] __iommu_free+0x1e0/0x1f0 (unreliable) [c000000003a27bc0] [c00000000005f848] iommu_free+0x28/0x70 [c000000003a27bf0] [c000000000061518] iommu_free_coherent+0x68/0xa0 [c000000003a27c20] [c00000000005e8d4] dma_iommu_free_coherent+0x24/0x40 [c000000003a27c40] [c00000000024698c] dma_free_attrs+0x10c/0x140 [c000000003a27c90] [c008000000dcb8d4] mlx5_cmd_cleanup+0x5c/0x90 [mlx5_core] [c000000003a27cc0] [c008000000dc45a0] mlx5_mdev_uninit+0xc8/0x100 [mlx5_core] [c000000003a27d00] [c008000000dc4ac4] probe_one+0x3ec/0x530 [mlx5_core] [c000000003a27d90] [c0000000008c5edc] local_pci_probe+0x6c/0x110 [c000000003a27e10] [c000000000189c98] work_for_cpu_fn+0x38/0x60 [c000000003a27e40] [c00000000018d1d0] process_scheduled_works+0x230/0x4f0 [c000000003a27f10] [c00000000018ff14] worker_thread+0x1e4/0x500 [c000000003a27f90] [c00000000019cdb8] kthread+0x138/0x140 [c000000003a27fe0] [c00000000000df98] start_kernel_thread+0x14/0x18 Code: 481b004d 60000000 e89e0028 3c62ffe0 3863dd20 481b0039 60000000 e89e0038 3c62ffe0 3863dd38 481b0025 60000000 <0fe00000> 4bffff20 60000000 3c4c0142 ---[ end trace 0000000000000000 ]--- iommu_free: invalid entry entry = 0x8000000203d0 dma_addr = 0x8000000203d0000 Table = 0xc00000000c5dc800 bus# = 0x1 size = 0x20000 startOff = 0x800000000000 index = 0x70200016 Fixes: 2ad56efa ("powerpc/iommu: Setup a default domain and remove set_platform_dma_ops") Reported-by:Tasmiya Nalatwad <tasmiya@linux.vnet.ibm.com> Link: https://lore.kernel.org/r/d06cee81-c47f-9d62-dfc6-4c77b60058db@linux.vnet.ibm.comTested-by:
-
- 05 Oct, 2023 2 commits
-
-
Jason Gunthorpe authored
This function returns NULL on errors, not ERR_PTR. Fixes: 1c68cbc6 ("iommu: Add IOMMU_DOMAIN_PLATFORM") Reported-by:
-
Jason Gunthorpe authored
msm_iommu platforms do not select either CONFIG_IOMMU_DMA or CONFIG_ARM_DMA_USE_IOMMU so they create a IOMMU_DOMAIN_DMA domain by default and never populate it. This acts like a BLOCKED domain and breaks the GPU driver on the platform. Detect this and force use of IDENTITY instead. Fixes: 98ac73f9 ("iommu: Require a default_domain for all iommu drivers") Reported-by:
Dmitry Baryshkov <dmitry.baryshkov@linaro.org> Link: https://lore.kernel.org/linux-iommu/CAA8EJprz7VVmBG68U9zLuqPd0UdSRHYoLDJSP6tCj6H6qanuTQ@mail.gmail.com/Signed-off-by:
Jerry Snitselaar <jsnitsel@redhat.com> Tested-by:
-
- 25 Sep, 2023 28 commits
-
-
Jiapeng Chong authored
./drivers/iommu/iommu.c: iommu-priv.h is included more than once. Reported-by:
Abaci Robot <abaci@linux.alibaba.com> Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=6186Signed-off-by:
Jiapeng Chong <jiapeng.chong@linux.alibaba.com> Link: https://lore.kernel.org/r/20230818092620.91748-1-jiapeng.chong@linux.alibaba.comSigned-off-by:
-
Robin Murphy authored
Automatically scaling the depot up to suit the peak capacity of a workload is all well and good, but it would be nice to have a way to scale it back down again if the workload changes. To that end, add backround reclaim that will gradually free surplus magazines if the depot size remains above a reasonable threshold for long enough. Reviewed-by:
Robin Murphy <robin.murphy@arm.com> Link: https://lore.kernel.org/r/03170665c56d89c6ce6081246b47f68d4e483308.1694535580.git.robin.murphy@arm.comSigned-off-by:
-
Robin Murphy authored
The algorithm in the original paper specifies the storage of full magazines in the depot as an unbounded list rather than a fixed-size array. It turns out to be pretty straightforward to do this in our implementation with no significant loss of efficiency. This allows the depot to scale up to the working set sizes of larger systems, while also potentially saving some memory on smaller ones too. Since this involves touching struct iova_magazine with the requisite care, we may as well reinforce the comment with a proper assertion too. Reviewed-by:
John Garry <john.g.garry@oracle.com> Reviewed-by:
-
Robin Murphy authored
The current checks for the __IOMMU_DOMAIN_PAGING capability seem a bit stifled, since it is quite likely now that a non-paging domain won't have a pgsize_bitmap and/or mapping ops, and thus get caught by the earlier condition anyway. Swap them around to test the more fundamental condition first, then we can reasonably also upgrade the other to a WARN_ON, since if a driver does ever expose a paging domain without the means to actually page, it's clearly very broken. Signed-off-by:
-
Robin Murphy authored
With everyone now implementing the new interfaces, clean up the last remnants of the old map/unmap ops and simplify the calling logic again. Signed-off-by:
-
Robin Murphy authored
Trivially update map/unmap to the new interface, which is quite happy for drivers to still process just one page per call. Signed-off-by:
-
Robin Murphy authored
Trivially update map/unmap to the new interface, which is quite happy for drivers to still process just one page per call. Signed-off-by:
-
Robin Murphy authored
Trivially update map/unmap to the new interface, which is quite happy for drivers to still process just one page per call. Signed-off-by:
-
Robin Murphy authored
Trivially update map/unmap to the new interface, which is quite happy for drivers to still process just one page per call. Signed-off-by:
-
Robin Murphy authored
Trivially update map/unmap to the new interface, which is quite happy for drivers to still process just one page per call. Signed-off-by:
-
Jason Gunthorpe authored
Use the new helper. For some reason omap will probe its driver even if it doesn't load an iommu driver. Keep this working by keeping a bool to track if the iommu driver was started. Signed-off-by:
-
Jason Gunthorpe authored
Use the new helper. This driver is kind of weird since in ARM mode it pretends it has per-device groups, but ARM64 mode does not. Signed-off-by:
-
Jason Gunthorpe authored
Use the new helper. Signed-off-by:
-
Jason Gunthorpe authored
Use the new helper. Signed-off-by:
-
Jason Gunthorpe authored
Use the new helper. Signed-off-by:
Jernej Skrabec <jernej.skrabec@gmail.com> Link: https://lore.kernel.org/r/3-v1-c869a95191f2+5e8-iommu_single_grp_jgg@nvidia.comSigned-off-by:
-
Jason Gunthorpe authored
This implements the common pattern seen in drivers of a single iommu_group for the entire iommu driver instance. Implement this in core code so the drivers that want this can select it from their ops. Reviewed-by:
-
Jason Gunthorpe authored
Several functions obtain the group reference and then release it before returning. This gives the impression that the refcount is protecting something for the duration of the function. In truth all of these functions are called in places that know a device driver is probed to the device and our locking rules already require that dev->iommu_group cannot change while a driver is attached to the struct device. If this was not the case then this code is already at risk of triggering UAF as it is racy if the dev->iommu_group is concurrently going to NULL/free. refcount debugging will throw a WARN if kobject_get() is called on a 0 refcount object to highlight the bug. Remove the confusing refcounting and leave behind a comment about the restriction. Reviewed-by:
-
Jason Gunthorpe authored
These drivers don't support IOMMU_DOMAIN_DMA, so this commit effectively allows them to support that mode. The prior work to require default_domains makes this safe because every one of these drivers is either compilation incompatible with dma-iommu.c, or already establishing a default_domain. In both cases alloc_domain() will never be called with IOMMU_DOMAIN_DMA for these drivers so it is safe to drop the test. Removing these tests clarifies that the domain allocation path is only about the functionality of a paging domain and has nothing to do with policy of how the paging domain is used for UNMANAGED/DMA/DMA_FQ. Tested-by:
Niklas Schnelle <schnelle@linux.ibm.com> Tested-by:
Steven Price <steven.price@arm.com> Tested-by:
Marek Szyprowski <m.szyprowski@samsung.com> Tested-by:
Nicolin Chen <nicolinc@nvidia.com> Reviewed-by:
-
Jason Gunthorpe authored
These drivers are all trivially converted since the function is only called if the domain type is going to be IOMMU_DOMAIN_UNMANAGED/DMA. Tested-by:
Heiko Stuebner <heiko@sntech.de> Tested-by:
-
Jason Gunthorpe authored
This callback requests the driver to create only a __IOMMU_DOMAIN_PAGING domain, so it saves a few lines in a lot of drivers needlessly checking the type. More critically, this allows us to sweep out all the IOMMU_DOMAIN_UNMANAGED and IOMMU_DOMAIN_DMA checks from a lot of the drivers, simplifying what is going on in the code and ultimately removing the now-unused special cases in drivers where they did not support IOMMU_DOMAIN_DMA. domain_alloc_paging() should return a struct iommu_domain that is functionally compatible with ARM_DMA_USE_IOMMU, dma-iommu.c and iommufd. Be forwards looking and pass in a 'struct device *' argument. We can provide this when allocating the default_domain. No drivers will look at this. Tested-by:
-
Jason Gunthorpe authored
Allocate a domain from a group. Automatically obtains the iommu_ops to use from the device list of the group. Convert the internal callers to use it. Tested-by:
-
Jason Gunthorpe authored
At this point every iommu driver will cause a default_domain to be selected, so we can finally remove this gap from the core code. The following table explains what each driver supports and what the resulting default_domain will be: ops->defaut_domain IDENTITY DMA PLATFORM v ARM32 dma-iommu ARCH amd/iommu.c Y Y N/A either apple-dart.c Y Y N/A either arm-smmu.c Y Y IDENTITY either qcom_iommu.c G Y IDENTITY either arm-smmu-v3.c Y Y N/A either exynos-iommu.c G Y IDENTITY either fsl_pamu_domain.c Y Y N/A N/A PLATFORM intel/iommu.c Y Y N/A either ipmmu-vmsa.c G Y IDENTITY either msm_iommu.c G IDENTITY N/A mtk_iommu.c G Y IDENTITY either mtk_iommu_v1.c G IDENTITY N/A omap-iommu.c G IDENTITY N/A rockchip-iommu.c G Y IDENTITY either s390-iommu.c Y Y N/A N/A PLATFORM sprd-iommu.c Y N/A DMA sun50i-iommu.c G Y IDENTITY either tegra-smmu.c G Y IDENTITY IDENTITY virtio-iommu.c Y Y N/A either spapr Y Y N/A N/A PLATFORM * G means ops->identity_domain is used * N/A means the driver will not compile in this configuration ARM32 drivers select an IDENTITY default domain through either the ops->identity_domain or directly requesting an IDENTIY domain through alloc_domain(). In ARM64 mode tegra-smmu will still block the use of dma-iommu.c and forces an IDENTITY domain. S390 uses a PLATFORM domain to represent when the dma_ops are set to the s390 iommu code. fsl_pamu uses an PLATFORM domain. POWER SPAPR uses PLATFORM and blocking to enable its weird VFIO mode. The x86 drivers continue unchanged. After this patch group->default_domain is only NULL for a short period during bus iommu probing while all the groups are constituted. Otherwise it is always !NULL. This completes changing the iommu subsystem driver contract to a system where the current iommu_domain always represents some form of translation and the driver is continuously asserting a definable translation mode. It resolves the confusion that the original ops->detach_dev() caused around what translation, exactly, is the IOMMU performing after detach. There were at least three different answers to that question in the tree, they are all now clearly named with domain types. Tested-by: -
Jason Gunthorpe authored
Prior to commit 1b932ced ("iommu: Remove detach_dev callbacks") the sun50i_iommu_detach_device() function was being called by ops->detach_dev(). This is an IDENTITY domain so convert sun50i_iommu_detach_device() into sun50i_iommu_identity_attach() and a full IDENTITY domain and thus hook it back up the same was as the old ops->detach_dev(). Reviewed-by:
-
Jason Gunthorpe authored
This brings back the ops->detach_dev() code that commit 1b932ced ("iommu: Remove detach_dev callbacks") deleted and turns it into an IDENTITY domain. Reviewed-by:
-
Jason Gunthorpe authored
This brings back the ops->detach_dev() code that commit 1b932ced ("iommu: Remove detach_dev callbacks") deleted and turns it into an IDENTITY domain. Also reverts commit 584d334b ("iommu/ipmmu-vmsa: Remove ipmmu_utlb_disable()") Reviewed-by:
-
Jason Gunthorpe authored
This brings back the ops->detach_dev() code that commit 1b932ced ("iommu: Remove detach_dev callbacks") deleted and turns it into an IDENTITY domain. Reviewed-by:
-
Jason Gunthorpe authored
All drivers are now using IDENTITY or PLATFORM domains for what this did, we can remove it now. It is no longer possible to attach to a NULL domain. Tested-by:
-
Jason Gunthorpe authored
What msm does during msm_iommu_set_platform_dma() is actually putting the iommu into identity mode. Move to the new core support for ARM_DMA_USE_IOMMU by defining ops->identity_domain. This driver does not support IOMMU_DOMAIN_DMA, however it cannot be compiled on ARM64 either. Most likely it is fine to support dma-iommu.c Reviewed-by:
-
