linux/drivers/md/bcache
Coly Li a1a2d8f016 bcache: avoid unnecessary soft lockup in kworker update_writeback_rate()
The kworker routine update_writeback_rate() is schedued to update the
writeback rate in every 5 seconds by default. Before calling
__update_writeback_rate() to do real job, semaphore dc->writeback_lock
should be held by the kworker routine.

At the same time, bcache writeback thread routine bch_writeback_thread()
also needs to hold dc->writeback_lock before flushing dirty data back
into the backing device. If the dirty data set is large, it might be
very long time for bch_writeback_thread() to scan all dirty buckets and
releases dc->writeback_lock. In such case update_writeback_rate() can be
starved for long enough time so that kernel reports a soft lockup warn-
ing started like:
  watchdog: BUG: soft lockup - CPU#246 stuck for 23s! [kworker/246:31:179713]

Such soft lockup condition is unnecessary, because after the writeback
thread finishes its job and releases dc->writeback_lock, the kworker
update_writeback_rate() may continue to work and everything is fine
indeed.

This patch avoids the unnecessary soft lockup by the following method,
- Add new member to struct cached_dev
  - dc->rate_update_retry (0 by default)
- In update_writeback_rate() call down_read_trylock(&dc->writeback_lock)
  firstly, if it fails then lock contention happens.
- If dc->rate_update_retry <= BCH_WBRATE_UPDATE_MAX_SKIPS (15), doesn't
  acquire the lock and reschedules the kworker for next try.
- If dc->rate_update_retry > BCH_WBRATE_UPDATE_MAX_SKIPS, no retry
  anymore and call down_read(&dc->writeback_lock) to wait for the lock.

By the above method, at worst case update_writeback_rate() may retry for
1+ minutes before blocking on dc->writeback_lock by calling down_read().
For a 4TB cache device with 1TB dirty data, 90%+ of the unnecessary soft
lockup warning message can be avoided.

When retrying to acquire dc->writeback_lock in update_writeback_rate(),
of course the writeback rate cannot be updated. It is fair, because when
the kworker is blocked on the lock contention of dc->writeback_lock, the
writeback rate cannot be updated neither.

This change follows Jens Axboe's suggestion to a more clear and simple
version.

Signed-off-by: Coly Li <colyli@suse.de>
Link: https://lore.kernel.org/r/20220528124550.32834-2-colyli@suse.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-05-28 06:48:26 -06:00
..
alloc.c block: decouple REQ_OP_SECURE_ERASE from REQ_OP_DISCARD 2022-04-17 19:49:59 -06:00
bcache_ondisk.h bcache: move uapi header bcache.h to bcache code directory 2021-10-29 06:43:21 -06:00
bcache.h bcache: avoid unnecessary soft lockup in kworker update_writeback_rate() 2022-05-28 06:48:26 -06:00
bset.c bcache: Avoid comma separated statements 2021-02-10 08:06:00 -07:00
bset.h bcache: move uapi header bcache.h to bcache code directory 2021-10-29 06:43:21 -06:00
btree.c bcache: memset on stack variables in bch_btree_check() and bch_sectors_dirty_init() 2022-05-27 09:49:48 -06:00
btree.h bcache: improve multithreaded bch_btree_check() 2022-05-24 06:19:33 -06:00
closure.c bcache: Convert to DEFINE_SHOW_ATTRIBUTE 2020-10-02 14:25:29 -06:00
closure.h bcache: fix typo in code comments of closure_return_with_destructor() 2018-10-08 08:19:43 -06:00
debug.c block: turn bio_kmalloc into a simple kmalloc wrapper 2022-04-17 19:30:41 -06:00
debug.h
extents.c bcache: remove PTR_CACHE 2021-04-11 08:37:55 -06:00
extents.h
features.c bcache: move uapi header bcache.h to bcache code directory 2021-10-29 06:43:21 -06:00
features.h bcache: move uapi header bcache.h to bcache code directory 2021-10-29 06:43:21 -06:00
io.c block: pass a block_device and opf to bio_init 2022-02-02 07:49:59 -07:00
journal.c bcache: avoid journal no-space deadlock by reserving 1 journal bucket 2022-05-24 06:19:33 -06:00
journal.h bcache: avoid journal no-space deadlock by reserving 1 journal bucket 2022-05-24 06:19:33 -06:00
Kconfig block: make the block holder code optional 2021-08-09 11:50:42 -06:00
Makefile bcache: add sysfs file to display feature sets information of cache set 2020-07-25 07:38:21 -06:00
movinggc.c block: pass a block_device and opf to bio_init 2022-02-02 07:49:59 -07:00
request.c md: bcache: check the return value of kzalloc() in detached_dev_do_request() 2022-05-27 09:49:48 -06:00
request.h block: switch polling to be bio based 2021-10-18 06:17:36 -06:00
stats.c bcache: use default_groups in kobj_type 2022-01-26 15:56:18 +01:00
stats.h bcache: remove bcache device self-defined readahead 2021-06-08 15:06:03 -06:00
super.c bcache: avoid journal no-space deadlock by reserving 1 journal bucket 2022-05-24 06:19:33 -06:00
sysfs.c block: remove QUEUE_FLAG_DISCARD 2022-04-17 19:49:59 -06:00
sysfs.h bcache: use default_groups in kobj_type 2022-01-26 15:56:18 +01:00
trace.c
util.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
util.h for-5.16/bdev-size-2021-10-29 2021-11-01 09:50:37 -07:00
writeback.c bcache: avoid unnecessary soft lockup in kworker update_writeback_rate() 2022-05-28 06:48:26 -06:00
writeback.h bcache: improve multithreaded bch_sectors_dirty_init() 2022-05-24 06:19:33 -06:00