linux/kernel/dma
Petr Tesarik 1aaa736815 swiotlb: allocate a new memory pool when existing pools are full
When swiotlb_find_slots() cannot find suitable slots, schedule the
allocation of a new memory pool. It is not possible to allocate the pool
immediately, because this code may run in interrupt context, which is not
suitable for large memory allocations. This means that the memory pool will
be available too late for the currently requested mapping, but the stress
on the software IO TLB allocator is likely to continue, and subsequent
allocations will benefit from the additional pool eventually.

Keep all memory pools for an allocator in an RCU list to avoid locking on
the read side. For modifications, add a new spinlock to struct io_tlb_mem.

The spinlock also protects updates to the total number of slabs (nslabs in
struct io_tlb_mem), but not reads of the value. Readers may therefore
encounter a stale value, but this is not an issue:

- swiotlb_tbl_map_single() and is_swiotlb_active() only check for non-zero
  value. This is ensured by the existence of the default memory pool,
  allocated at boot.

- The exact value is used only for non-critical purposes (debugfs, kernel
  messages).

Signed-off-by: Petr Tesarik <petr.tesarik.ext@huawei.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2023-08-01 18:02:27 +02:00
..
coherent.c dma-mapping: Add dma_release_coherent_memory to DMA API 2022-06-24 09:30:54 -06:00
contiguous.c dma-contiguous: check for memory region overlap 2023-07-31 17:54:56 +02:00
debug.c dma-debug: Use %pa to format phys_addr_t 2023-03-30 08:51:52 +09:00
debug.h dma-debug: teach add_dma_entry() about DMA_ATTR_SKIP_CPU_SYNC 2021-10-18 12:46:45 +02:00
direct.c swiotlb: if swiotlb is full, fall back to a transient memory pool 2023-08-01 18:02:20 +02:00
direct.h dma-mapping: force bouncing if the kmalloc() size is not cache-line-aligned 2023-06-19 16:19:23 -07:00
dummy.c dma-mapping: return error code from dma_dummy_map_sg() 2021-08-09 17:13:06 +02:00
Kconfig swiotlb: add a flag whether SWIOTLB is allowed to grow 2023-08-01 18:02:17 +02:00
Makefile dma-mapping: remove CONFIG_DMA_REMAP 2022-03-03 14:00:57 +03:00
map_benchmark.c dma-mapping: benchmark: remove MODULE_LICENSE in non-modules 2023-04-13 13:13:51 -07:00
mapping.c dma-mapping: move arch_dma_set_mask() declaration to header 2023-07-31 17:54:28 +02:00
ops_helpers.c dma-mapping: handle vmalloc addresses in dma_common_{mmap,get_sgtable} 2021-07-16 11:30:26 +02:00
pool.c mm, treewide: redefine MAX_ORDER sanely 2023-04-05 19:42:46 -07:00
remap.c dma-remap: use kvmalloc_array/kvfree for larger dma memory remap 2023-06-07 15:06:28 +02:00
swiotlb.c swiotlb: allocate a new memory pool when existing pools are full 2023-08-01 18:02:27 +02:00