Commit 0d50f7b1 authored by Joerg Roedel's avatar Joerg Roedel Committed by Greg Kroah-Hartman

iommu/amd: Fix race in increase_address_space()

[ Upstream commit 754265bc ]

After the conversion to lock-less dma-api call the
increase_address_space() function can be called without any
locking. Multiple CPUs could potentially race for increasing
the address space, leading to invalid domain->mode settings
and invalid page-tables. This has been happening in the wild
under high IO load and memory pressure.

Fix the race by locking this operation. The function is
called infrequently so that this does not introduce
a performance regression in the dma-api path again.
Reported-by: default avatarQian Cai <cai@lca.pw>
Fixes: 256e4621 ('iommu/amd: Make use of the generic IOVA allocator')
Signed-off-by: default avatarJoerg Roedel <jroedel@suse.de>
Signed-off-by: default avatarSasha Levin <sashal@kernel.org>
parent 52f32e4a
......@@ -1340,18 +1340,21 @@ static void domain_flush_devices(struct protection_domain *domain)
* another level increases the size of the address space by 9 bits to a size up
* to 64 bits.
*/
static bool increase_address_space(struct protection_domain *domain,
static void increase_address_space(struct protection_domain *domain,
gfp_t gfp)
{
unsigned long flags;
u64 *pte;
if (domain->mode == PAGE_MODE_6_LEVEL)
spin_lock_irqsave(&domain->lock, flags);
if (WARN_ON_ONCE(domain->mode == PAGE_MODE_6_LEVEL))
/* address space already 64 bit large */
return false;
goto out;
pte = (void *)get_zeroed_page(gfp);
if (!pte)
return false;
goto out;
*pte = PM_LEVEL_PDE(domain->mode,
iommu_virt_to_phys(domain->pt_root));
......@@ -1359,7 +1362,10 @@ static bool increase_address_space(struct protection_domain *domain,
domain->mode += 1;
domain->updated = true;
return true;
out:
spin_unlock_irqrestore(&domain->lock, flags);
return;
}
static u64 *alloc_pte(struct protection_domain *domain,
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment