A mirror of the official Linux kernel repository just in case
Go to file
Nicholas Piggin 2655421ae6 lazy tlb: shoot lazies, non-refcounting lazy tlb mm reference handling scheme
On big systems, the mm refcount can become highly contented when doing a
lot of context switching with threaded applications.  user<->idle switch
is one of the important cases.  Abandoning lazy tlb entirely slows this
switching down quite a bit in the common uncontended case, so that is not
viable.

Implement a scheme where lazy tlb mm references do not contribute to the
refcount, instead they get explicitly removed when the refcount reaches
zero.

The final mmdrop() sends IPIs to all CPUs in the mm_cpumask and they
switch away from this mm to init_mm if it was being used as the lazy tlb
mm.  Enabling the shoot lazies option therefore requires that the arch
ensures that mm_cpumask contains all CPUs that could possibly be using mm.
A DEBUG_VM option IPIs every CPU in the system after this to ensure there
are no references remaining before the mm is freed.

Shootdown IPIs cost could be an issue, but they have not been observed to
be a serious problem with this scheme, because short-lived processes tend
not to migrate CPUs much, therefore they don't get much chance to leave
lazy tlb mm references on remote CPUs.  There are a lot of options to
reduce them if necessary, described in comments.

The near-worst-case can be benchmarked with will-it-scale:

  context_switch1_threads -t $(($(nproc) / 2))

This will create nproc threads (nproc / 2 switching pairs) all sharing the
same mm that spread over all CPUs so each CPU does thread->idle->thread
switching.

[ Rik came up with basically the same idea a few years ago, so credit
  to him for that. ]

Link: https://lore.kernel.org/linux-mm/20230118080011.2258375-1-npiggin@gmail.com/
Link: https://lore.kernel.org/all/20180728215357.3249-11-riel@surriel.com/
Link: https://lkml.kernel.org/r/20230203071837.1136453-5-npiggin@gmail.com
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nadav Amit <nadav.amit@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-03-28 16:20:08 -07:00
arch lazy tlb: shoot lazies, non-refcounting lazy tlb mm reference handling scheme 2023-03-28 16:20:08 -07:00
block block: remove obsolete config BLOCK_COMPAT 2023-03-16 09:35:44 -06:00
certs Kbuild updates for v6.3 2023-02-26 11:53:25 -08:00
crypto asymmetric_keys: log on fatal failures in PE/pkcs7 2023-03-21 16:23:56 +00:00
Documentation lazy tlb: allow lazy tlb mm refcounting to be configurable 2023-03-28 16:20:08 -07:00
drivers USB/Thunderbolt driver fixes for 6.3-rc4 2023-03-26 10:22:44 -07:00
fs lazy tlb: introduce lazy tlb mm refcount helper functions 2023-03-28 16:20:08 -07:00
include lazy tlb: allow lazy tlb mm refcounting to be configurable 2023-03-28 16:20:08 -07:00
init bootconfig: Change message if no bootconfig with CONFIG_BOOT_CONFIG_FORCE=y 2023-03-22 22:21:43 +09:00
io_uring block-6.3-2023-03-24 2023-03-24 14:10:39 -07:00
ipc Merge branch 'work.namespace' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2023-02-24 19:20:07 -08:00
kernel lazy tlb: shoot lazies, non-refcounting lazy tlb mm reference handling scheme 2023-03-28 16:20:08 -07:00
lib lazy tlb: shoot lazies, non-refcounting lazy tlb mm reference handling scheme 2023-03-28 16:20:08 -07:00
LICENSES LICENSES: Add the copyleft-next-0.3.1 license 2022-11-08 15:44:01 +01:00
mm mm/zswap: try to avoid worst-case scenario on same element pages 2023-03-28 16:20:07 -07:00
net nfsd-6.3 fixes: 2023-03-25 13:32:43 -07:00
rust Rust fixes for 6.3-rc1 2023-03-03 14:51:15 -08:00
samples LoongArch changes for v6.3 2023-03-01 09:27:00 -08:00
scripts checksyscalls: ignore fstat to silence build warning on LoongArch 2023-03-23 17:18:32 -07:00
security keys: Do not cache key in task struct if key is requested from kernel thread 2023-03-21 16:22:40 +00:00
sound ALSA: hda/ca0132: fixup buffer overrun at tuning_ctl_set() 2023-03-14 17:04:53 +01:00
tools - Add a AMX ptrace self test 2023-03-26 09:01:24 -07:00
usr usr/gen_init_cpio.c: remove unnecessary -1 values from int file 2022-10-03 14:21:44 -07:00
virt KVM/riscv changes for 6.3 2023-02-15 12:33:28 -05:00
.clang-format cpumask: re-introduce constant-sized cpumask optimizations 2023-03-05 14:30:34 -08:00
.cocciconfig
.get_maintainer.ignore get_maintainer: add Alan to .get_maintainer.ignore 2022-08-20 15:17:44 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for *.dtso files 2023-02-26 15:28:23 +09:00
.gitignore kbuild: rpm-pkg: move source components to rpmbuild/SOURCES 2023-03-16 22:45:56 +09:00
.mailmap mailmap: add an entry for Leonard Crestez 2023-03-28 15:24:32 -07:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING
CREDITS There is no particular theme here - mainly quick hits all over the tree. 2023-02-23 17:55:40 -08:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS USB/Thunderbolt driver fixes for 6.3-rc4 2023-03-26 10:22:44 -07:00
Makefile Linux 6.3-rc4 2023-03-26 14:40:20 -07:00
README

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.