Minchan Kim b475d42d2c zsmalloc: replace per zpage lock with pool->migrate_lock ... The zsmalloc has used a bit for spin_lock in zpage handle to keep zpage object alive during several operations. However, it causes the problem for PREEMPT_RT as well as introducing too complicated. This patch replaces the bit spin_lock with pool->migrate_lock rwlock. It could make the code simple as well as zsmalloc work under PREEMPT_RT. The drawback is the pool->migrate_lock is bigger granuarity than per zpage lock so the contention would be higher than old when both IO-related operations(i.e., zsmalloc, zsfree, zs_[map|unmap]) and compaction(page/zpage migration) are going in parallel(*, the migrate_lock is rwlock and IO related functions are all read side lock so there is no contention). However, the write-side is fast enough(dominant overhead is just page copy) so it wouldn't affect much. If the lock granurity becomes more problem later, we could introduce table locks based on handle as a hash value. Link: https://lkml.kernel.org/r/20211115185909.3949505-9-minchan@kernel.org Signed-off-by: Minchan Kim <minchan@kernel.org> Acked-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Tested-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Mike Galbraith <umgwanakikbuti@gmail.com> Cc: Peter Zijlstra (Intel) <peterz@infradead.org> Cc: Sergey Senozhatsky <senozhatsky@chromium.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>		2022-01-22 08:33:37 +02:00
..
damon	mm/damon: hide kernel pointer from tracepoint event	2022-01-15 16:30:33 +02:00
kasan	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
kfence	slab changes for 5.17	2022-01-10 11:58:12 -08:00
backing-dev.c	mm: bdi: initialize bdi_min_ratio when bdi is unregistered	2021-12-10 17:10:56 -08:00
balloon_compaction.c	mm: fix typos in comments	2021-05-07 00:26:35 -07:00
bootmem_info.c	bootmem: Use page->index instead of page->freelist	2022-01-06 12:27:03 +01:00
cleancache.c
cma_debug.c
cma_sysfs.c
cma.c	memblock: rename memblock_free to memblock_phys_free	2021-11-06 13:30:41 -07:00
cma.h
compaction.c	mm: compaction: fix the migration stats in trace_mm_compaction_migratepages()	2022-01-15 16:30:30 +02:00
debug_page_ref.c
debug_vm_pgtable.c	mm: ptep_clear() page table helper	2022-01-15 16:30:28 +02:00
debug.c	mm,fs: split dump_mapping() out from dump_page()	2022-01-15 16:30:26 +02:00
dmapool.c	mm/dmapool.c: revert "make dma pool to use kmalloc_node"	2022-01-15 16:30:28 +02:00
early_ioremap.c	mm/early_ioremap.c: remove redundant early_ioremap_shutdown()	2021-09-08 11:50:24 -07:00
fadvise.c
failslab.c
filemap.c	mm/migrate.c: rework migration_entry_wait() to not take a pageref	2022-01-22 08:33:34 +02:00
folio-compat.c	filemap: Add filemap_release_folio()	2022-01-04 13:15:34 -05:00
frontswap.c	mm/frontswap.c: use non-atomic '__set_bit()' when possible	2022-01-15 16:30:26 +02:00
gup_test.c
gup_test.h
gup.c	mm/gup.c: stricter check on THP migration entry during follow_pmd_mask	2022-01-15 16:30:26 +02:00
highmem.c	Fixes for 5.16 folios:	2021-11-25 10:13:56 -08:00
hmm.c	mm/hmm.c: allow VM_MIXEDMAP to work with hmm_range_fault	2022-01-15 16:30:31 +02:00
huge_memory.c	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
hugetlb_cgroup.c	hugetlb: add hugetlb.*.numa_stat file	2022-01-15 16:30:29 +02:00
hugetlb_vmemmap.c	mm: hugetlb: introduce CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON	2021-06-30 20:47:26 -07:00
hugetlb_vmemmap.h	mm: hugetlb: introduce nr_free_vmemmap_pages in the struct hstate	2021-06-30 20:47:25 -07:00
hugetlb.c	mm: change page type prior to adding page table entry	2022-01-15 16:30:28 +02:00
hwpoison-inject.c	mm: hwpoison: don't drop slab caches for offlining non-LRU page	2021-09-03 09:58:15 -07:00
init-mm.c	mm: add setup_initial_init_mm() helper	2021-07-08 11:48:21 -07:00
internal.h	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
interval_tree.c
io-mapping.c
ioremap.c	mm: move ioremap_page_range to vmalloc.c	2021-09-08 11:50:24 -07:00
Kconfig	Merge branch 'akpm' (patches from Andrew)	2022-01-20 10:41:01 +02:00
Kconfig.debug	mm: page table check	2022-01-15 16:30:28 +02:00
khugepaged.c	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
kmemleak.c	kmemleak: fix kmemleak false positive report with HW tag-based kasan enable	2022-01-15 16:30:25 +02:00
ksm.c	mm: ksm: fix use-after-free kasan report in ksm_might_need_to_copy	2022-01-15 16:30:31 +02:00
list_lru.c	mm: list_lru: only add memcg-aware lrus to the global lru list	2021-11-06 13:30:35 -07:00
maccess.c	ARM: 9115/1: mm/maccess: fix unaligned copy_{from,to}_kernel_nofault	2021-08-20 11:39:25 +01:00
madvise.c	mm: move anon_vma declarations to linux/mm_inline.h	2022-01-15 16:30:27 +02:00
Makefile	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
mapping_dirty_helpers.c	mm: move tlb_flush_pending inline helpers to mm_inline.h	2022-01-15 16:30:27 +02:00
memblock.c	arm64 fixes for -rc1	2021-11-10 11:29:30 -08:00
memcontrol.c	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
memfd.c	mm,hugetlb: remove mlock ulimit for SHM_HUGETLB	2021-11-09 10:02:48 -08:00
memory_hotplug.c	treewide: Add missing includes masked by cgroup -> bpf dependency	2021-12-03 10:58:13 -08:00
memory-failure.c	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
memory.c	Merge branch 'akpm' (patches from Andrew)	2022-01-20 10:41:01 +02:00
mempolicy.c	mm/mempolicy: fix all kernel-doc warnings	2022-01-15 16:30:30 +02:00
mempool.c	mm: remove spurious blkdev.h includes	2021-10-18 06:17:01 -06:00
memremap.c	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
memtest.c
migrate.c	mm/migrate.c: rework migration_entry_wait() to not take a pageref	2022-01-22 08:33:34 +02:00
mincore.c
mlock.c	mm: add a field to store names for private anonymous memory	2022-01-15 16:30:27 +02:00
mm_init.c
mmap_lock.c	mm: mmap_lock: fix disabling preemption directly	2021-07-23 17:43:28 -07:00
mmap.c	Merge branch 'signal-for-v5.17' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace	2022-01-17 05:49:30 +02:00
mmu_gather.c	mm: move tlb_flush_pending inline helpers to mm_inline.h	2022-01-15 16:30:27 +02:00
mmu_notifier.c
mmzone.c
mprotect.c	mm: add a field to store names for private anonymous memory	2022-01-15 16:30:27 +02:00
mremap.c	mm, hugepages: add mremap() support for hugepage backed vma	2021-11-06 13:30:39 -07:00
msync.c
nommu.c	Merge branch 'akpm' (patches from Andrew)	2021-11-06 14:08:17 -07:00
oom_kill.c	Merge branch 'signal-for-v5.17' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace	2022-01-17 05:49:30 +02:00
page_alloc.c	Merge branch 'akpm' (patches from Andrew)	2022-01-20 10:41:01 +02:00
page_counter.c	mm/page_counter: remove an incorrect call to propagate_protected_usage()	2022-01-15 16:30:27 +02:00
page_ext.c	mm: make some vars and functions static or __init	2022-01-15 16:30:31 +02:00
page_idle.c	mm/idle_page_tracking: make PG_idle reusable	2021-09-08 11:50:24 -07:00
page_io.c	delayacct: support swapin delay accounting for swapping without blkio	2022-01-20 08:52:55 +02:00
page_isolation.c	mm/page_isolation: unset migratetype directly for non Buddy page	2022-01-15 16:30:30 +02:00
page_owner.c	mm: make some vars and functions static or __init	2022-01-15 16:30:31 +02:00
page_poison.c
page_reporting.c	mm/page_reporting: allow driver to specify reporting order	2021-06-29 10:53:47 -07:00
page_reporting.h	mm/page_reporting: export reporting order as module parameter	2021-06-29 10:53:47 -07:00
page_table_check.c	mm: page table check	2022-01-15 16:30:28 +02:00
page_vma_mapped.c	mm: device exclusive memory access	2021-07-01 11:06:03 -07:00
page-writeback.c	mm/writeback: Improve __folio_mark_dirty() comment	2022-01-02 20:28:57 -05:00
pagewalk.c	mm: pagewalk: fix walk for hugepage tables	2021-06-29 10:53:49 -07:00
percpu-internal.h	mm: memcg/percpu: account extra objcg space to memory cgroups	2022-01-15 16:30:31 +02:00
percpu-km.c	percpu: flush tlb in pcpu_reclaim_populated()	2021-07-04 18:30:17 +00:00
percpu-stats.c	percpu: rework memcg accounting	2021-06-05 20:43:15 +00:00
percpu-vm.c	percpu: flush tlb in pcpu_reclaim_populated()	2021-07-04 18:30:17 +00:00
percpu.c	Merge branch 'akpm' (patches from Andrew)	2022-01-20 10:41:01 +02:00
pgalloc-track.h	mm: fix typos in comments	2021-05-07 00:26:35 -07:00
pgtable-generic.c	mm: move tlb_flush_pending inline helpers to mm_inline.h	2022-01-15 16:30:27 +02:00
process_vm_access.c
ptdump.c
readahead.c	readahead: Convert page_cache_ra_unbounded to folios	2022-01-04 13:15:34 -05:00
rmap.c	mm/rmap: fix potential batched TLB flush race	2022-01-15 16:30:31 +02:00
rodata_test.c
secretmem.c	mm/secretmem: avoid letting secretmem_users drop to zero	2021-10-28 17:18:55 -07:00
shmem.c	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
shuffle.c
shuffle.h	mm/shuffle: fix section mismatch warning	2021-05-22 15:09:07 -10:00
slab_common.c	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
slab.c	mm/kasan: Convert to struct folio and struct slab	2022-01-06 12:26:14 +01:00
slab.h	slab changes for 5.17 - part 2	2022-01-18 06:40:47 +02:00
slob.c	mm/slob: Remove unnecessary page_mapcount_reset() function call	2022-01-06 12:27:28 +01:00
slub.c	mm/slub: Define struct slab fields for CONFIG_SLUB_CPU_PARTIAL only when enabled	2022-01-06 12:26:53 +01:00
sparse-vmemmap.c	mm: remove redundant smp_wmb()	2021-11-06 13:30:36 -07:00
sparse.c	bootmem: Use page->index instead of page->freelist	2022-01-06 12:27:03 +01:00
swap_cgroup.c
swap_slots.c	treewide: Add missing includes masked by cgroup -> bpf dependency	2021-12-03 10:58:13 -08:00
swap_state.c	mm/workingset: Convert workingset_refault() to take a folio	2021-10-18 07:49:40 -04:00
swap.c	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
swapfile.c	mm: remove the total_mapcount argument from page_trans_huge_mapcount()	2022-01-15 16:30:28 +02:00
truncate.c	Merge branch 'akpm' (patches from Andrew)	2022-01-15 20:37:06 +02:00
usercopy.c	mm: Convert check_heap_object() to use struct slab	2022-01-06 12:25:51 +01:00
userfaultfd.c	mm: shmem: don't truncate page if memory failure happens	2022-01-15 16:30:26 +02:00
util.c	mm: allow !GFP_KERNEL allocations for kvmalloc	2022-01-15 16:30:29 +02:00
vmacache.c
vmalloc.c	mm/vmalloc: be more explicit about supported gfp flags.	2022-01-15 16:30:28 +02:00
vmpressure.c	mm/vmpressure: fix data-race with memcg->socket_pressure	2021-11-06 13:30:40 -07:00
vmscan.c	vmscan: make drop_slab_node static	2022-01-15 16:30:30 +02:00
vmstat.c	mm/vmstat: add events for THP max_ptes_* exceeds	2022-01-15 16:30:29 +02:00
workingset.c	Merge branch 'akpm' (patches from Andrew)	2021-11-09 10:11:53 -08:00
z3fold.c	mm/z3fold: add kerneldoc fields for z3fold_pool	2021-07-01 11:06:03 -07:00
zbud.c	mm/zbud: add kerneldoc fields for zbud_pool	2021-07-01 11:06:03 -07:00
zpool.c	zpool: remove the list of pools_head	2022-01-15 16:30:31 +02:00
zsmalloc.c	zsmalloc: replace per zpage lock with pool->migrate_lock	2022-01-22 08:33:37 +02:00
zswap.c	mm/zswap.c: fix two bugs in zswap_writeback_entry()	2021-06-30 20:47:31 -07:00