linux/kernel/rcu
Frederic Weisbecker fbb94cbd70 rcu/nocb: Invoke rcu_core() at the start of deoffloading
On PREEMPT_RT, if rcu_core() is preempted by the de-offloading process,
some work, such as callbacks acceleration and invocation, may be left
unattended due to the volatile checks on the offloaded state.

In the worst case this work is postponed until the next rcu_pending()
check that can take a jiffy to reach, which can be a problem in case
of callbacks flooding.

Solve that with invoking rcu_core() early in the de-offloading process.
This way any work dismissed by an ongoing rcu_core() call fooled by
a preempting deoffloading process will be caught up by a nearby future
recall to rcu_core(), this time fully aware of the de-offloading state.

Tested-by: Valentin Schneider <valentin.schneider@arm.com>
Tested-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Neeraj Upadhyay <neeraju@codeaurora.org>
Cc: Uladzislau Rezki <urezki@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
2021-12-07 16:24:44 -08:00
..
Kconfig Merge branches 'doc.2021.01.06a', 'fixes.2021.01.04b', 'kfree_rcu.2021.01.04a', 'mmdumpobj.2021.01.22a', 'nocb.2021.01.06a', 'rt.2021.01.04a', 'stall.2021.01.06a', 'torture.2021.01.12a' and 'tortureall.2021.01.06a' into HEAD 2021-01-22 15:26:44 -08:00
Kconfig.debug rcu: Restrict RCU_STRICT_GRACE_PERIOD to at most four CPUs 2021-05-10 16:22:54 -07:00
Makefile rcuperf: Change rcuperf to rcuscale 2020-08-24 18:39:24 -07:00
rcu_segcblist.c rcu/nocb: Invoke rcu_core() at the start of deoffloading 2021-12-07 16:24:44 -08:00
rcu_segcblist.h rcu/nocb: Prepare state machine for a new step 2021-12-07 16:24:44 -08:00
rcu.h Merge branches 'bitmaprange.2021.05.10c', 'doc.2021.05.10c', 'fixes.2021.05.13a', 'kvfree_rcu.2021.05.10c', 'mmdumpobj.2021.05.10c', 'nocb.2021.05.12a', 'srcu.2021.05.12a', 'tasks.2021.05.18a' and 'torture.2021.05.10c' into HEAD 2021-05-18 10:56:19 -07:00
rcuscale.c rcuscale: Warn on individual rcu_scale_init() error conditions 2021-09-13 16:36:16 -07:00
rcutorture.c Merge branches 'fixes.2021.10.07a', 'scftorture.2021.09.16a', 'tasks.2021.09.15a', 'torture.2021.09.13b' and 'torturescript.2021.09.16a' into HEAD 2021-10-07 10:02:17 -07:00
refscale.c refscale: Warn on individual ref_scale_init() error conditions 2021-09-13 16:36:16 -07:00
srcutiny.c srcutiny: Mark read-side data races 2021-08-06 13:41:48 -07:00
srcutree.c Merge branches 'bitmaprange.2021.05.10c', 'doc.2021.05.10c', 'fixes.2021.05.13a', 'kvfree_rcu.2021.05.10c', 'mmdumpobj.2021.05.10c', 'nocb.2021.05.12a', 'srcu.2021.05.12a', 'tasks.2021.05.18a' and 'torture.2021.05.10c' into HEAD 2021-05-18 10:56:19 -07:00
sync.c rcu: Fix various typos in comments 2021-05-12 12:11:05 -07:00
tasks.h RCU pull request for v5.16 2021-11-01 20:25:38 -07:00
tiny.c srcu: Initialize SRCU after timers 2021-05-10 16:03:35 -07:00
tree_exp.h rcu: Fix existing exp request check in sync_sched_exp_online_cleanup() 2021-09-13 16:32:46 -07:00
tree_nocb.h rcu/nocb: Invoke rcu_core() at the start of deoffloading 2021-12-07 16:24:44 -08:00
tree_plugin.h rcu: Always inline rcu_dynticks_task*_{enter,exit}() 2021-10-07 09:59:41 -07:00
tree_stall.h sched,rcu: Rework try_invoke_on_locked_down_task() 2021-10-07 13:51:15 +02:00
tree.c rcu/nocb: Invoke rcu_core() at the start of deoffloading 2021-12-07 16:24:44 -08:00
tree.h rcu/nocb: Make local rcu_nocb_lock_irqsave() safe against concurrent deoffloading 2021-12-07 16:24:44 -08:00
update.c RCU pull request for v5.16 2021-11-01 20:25:38 -07:00