• Petr Tesarik's avatar
    swiotlb: allocate a new memory pool when existing pools are full · 1aaa7368
    Petr Tesarik authored
    When swiotlb_find_slots() cannot find suitable slots, schedule the
    allocation of a new memory pool. It is not possible to allocate the pool
    immediately, because this code may run in interrupt context, which is not
    suitable for large memory allocations. This means that the memory pool will
    be available too late for the currently requested mapping, but the stress
    on the software IO TLB allocator is likely to continue, and subsequent
    allocations will benefit from the additional pool eventually.
    
    Keep all memory pools for an allocator in an RCU list to avoid locking on
    the read side. For modifications, add a new spinlock to struct io_tlb_mem.
    
    The spinlock also protects updates to the total number of slabs (nslabs in
    struct io_tlb_mem), but not reads of the value. Readers may therefore
    encounter a stale value, but this is not an issue:
    
    - swiotlb_tbl_map_single() and is_swiotlb_active() only check for non-zero
      value. This is ensured by the existence of the default memory pool,
      allocated at boot.
    
    - The exact value is used only for non-critical purposes (debugfs, kernel
      messages).
    Signed-off-by: default avatarPetr Tesarik <petr.tesarik.ext@huawei.com>
    Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
    1aaa7368
swiotlb.c 45.7 KB