During testing Sedat Dilek hit a "suspicious RCU usage" splat that pointed

out a real bug. During suspend and resume the tlb_flush tracepoint is
 called when the CPU is going offline. As the CPU has been noted as offline,
 RCU is ignoring that CPU, which means that it can not use RCU protected
 locks. When tracepoints are activated, they require RCU locking, and
 if RCU is ignoring a CPU that runs a tracepoint, there is a chance that
 the tracepoint could cause corruption.
 
 The solution was to change the tracepoint into a TRACE_EVENT_CONDITION()
 which allows us to check a condition to determine if the tracepoint
 should be called or not. If the condition is not met, the rcu protected
 code will not be executed. By adding the condition
 "cpu_online(smp_processor_id())", this will prevent the RCU protected
 code from being executed if the CPU is marked offline.
 
 After adding this, another bug was discovered. As RCU checks rcu callers,
 if a rcu call is not done, there is no check (obviously). We found that
 tracepoints could be added in RCU ignored locations and not have lockdep
 complain until the tracepoint is activated. This missed places where
 tracepoints were added in places they should not have been. To fix this,
 code was added in 3.18 that if lockdep is enabled, any tracepoint will
 still call the rcu checks even if the tracepoint is not enabled. The bug
 here, is that the check does not take the CONDITION into account. As the
 condition may prevent tracepoints from being activated in RCU ignored
 areas (as the one patch does), we get false positives when we enable
 lockdep and hit a tracepoint that the condition prevents it from being
 called in a RCU ignored location. The fix for this is to add the
 CONDITION to the rcu checks, even if the tracepoint is not enabled.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQEcBAABAgAGBQJU1rQfAAoJEEjnJuOKh9ld19UH/juFLZFjpYBgtRmbCZa/54Zk
 i2Fa3U8jQe8MHHEYOjCLT9MQTYo/42btmJhr7kWKtIoUgDEli4lkOpbs+H0qar5y
 Vv9+1cLeNFQzgIE3nwV7cjAw7Jufoyzd1lstDqIQvcmzZnQ5sNyyVeigMcxGv8Ls
 4FyqzG6zCVgiDL4LyYNHdNcMr6qLs3KTFDEqp+kQreeO7R1r3ZEpq3JoWaEUgoPP
 qrYv/rqVosLBUGA0pd7RmiGOxhjeKm15qz1GkiPeeus6DDWC6bvPC8cAc/FfkXH0
 hYpoQghSZVnXGy0LzVsd44gj7tYx1FHEpYy1s8G6d5WJcNOGZ6OoZOdOZMyjPVw=
 =PeL2
 -----END PGP SIGNATURE-----

Merge tag 'trace-fixes-v3.19-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull ftrace fixes from Steven Rostedt:
 "During testing Sedat Dilek hit a "suspicious RCU usage" splat that
  pointed out a real bug.  During suspend and resume the tlb_flush
  tracepoint is called when the CPU is going offline.  As the CPU has
  been noted as offline, RCU is ignoring that CPU, which means that it
  can not use RCU protected locks.  When tracepoints are activated, they
  require RCU locking, and if RCU is ignoring a CPU that runs a
  tracepoint, there is a chance that the tracepoint could cause
  corruption.

  The solution was to change the tracepoint into a
  TRACE_EVENT_CONDITION() which allows us to check a condition to
  determine if the tracepoint should be called or not.  If the condition
  is not met, the rcu protected code will not be executed.  By adding
  the condition "cpu_online(smp_processor_id())", this will prevent the
  RCU protected code from being executed if the CPU is marked offline.

  After adding this, another bug was discovered.  As RCU checks rcu
  callers, if a rcu call is not done, there is no check (obviously).  We
  found that tracepoints could be added in RCU ignored locations and not
  have lockdep complain until the tracepoint is activated.  This missed
  places where tracepoints were added in places they should not have
  been.  To fix this, code was added in 3.18 that if lockdep is enabled,
  any tracepoint will still call the rcu checks even if the tracepoint
  is not enabled.  The bug here, is that the check does not take the
  CONDITION into account.  As the condition may prevent tracepoints from
  being activated in RCU ignored areas (as the one patch does), we get
  false positives when we enable lockdep and hit a tracepoint that the
  condition prevents it from being called in a RCU ignored location.

  The fix for this is to add the CONDITION to the rcu checks, even if
  the tracepoint is not enabled"

* tag 'trace-fixes-v3.19-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace:
  x86/tlb/trace: Do not trace on CPU that is offline
  tracing: Add condition check to RCU lockdep checks
This commit is contained in:
Linus Torvalds 2015-02-08 18:08:14 -08:00
commit 4e02370f64
2 changed files with 4 additions and 2 deletions

View File

@ -173,7 +173,7 @@ extern void syscall_unregfunc(void);
TP_PROTO(data_proto), \
TP_ARGS(data_args), \
TP_CONDITION(cond),,); \
if (IS_ENABLED(CONFIG_LOCKDEP)) { \
if (IS_ENABLED(CONFIG_LOCKDEP) && (cond)) { \
rcu_read_lock_sched_notrace(); \
rcu_dereference_sched(__tracepoint_##name.funcs);\
rcu_read_unlock_sched_notrace(); \

View File

@ -13,11 +13,13 @@
{ TLB_LOCAL_SHOOTDOWN, "local shootdown" }, \
{ TLB_LOCAL_MM_SHOOTDOWN, "local mm shootdown" }
TRACE_EVENT(tlb_flush,
TRACE_EVENT_CONDITION(tlb_flush,
TP_PROTO(int reason, unsigned long pages),
TP_ARGS(reason, pages),
TP_CONDITION(cpu_online(smp_processor_id())),
TP_STRUCT__entry(
__field( int, reason)
__field(unsigned long, pages)