rtrs_clt_sess is used for paths and not sessions on the client side. This
creates confusion so let's rename it to rtrs_clt_path. Also, rename
related variables and functions.
Coccinelle is used to do the transformations for most of the occurrences
and remaining ones were handled manually.
Link: https://lore.kernel.org/r/20220105180708.7774-4-jinpu.wang@ionos.com
Signed-off-by: Vaishali Thakkar <vaishali.thakkar@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
With preemption enabled (CONFIG_DEBUG_PREEMPT=y), the following appeared
when rnbd client tries to map remote block device.
BUG: using smp_processor_id() in preemptible [00000000] code: bash/1733
caller is debug_smp_processor_id+0x17/0x20
CPU: 0 PID: 1733 Comm: bash Not tainted 5.16.0-rc1 #5
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.14.0-0-g155821a-rebuilt.opensuse.org 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x5d/0x78
dump_stack+0x10/0x12
check_preemption_disabled+0xe4/0xf0
debug_smp_processor_id+0x17/0x20
rtrs_clt_update_all_stats+0x3b/0x70 [rtrs_client]
rtrs_clt_read_req+0xc3/0x380 [rtrs_client]
? rtrs_clt_init_req+0xe3/0x120 [rtrs_client]
rtrs_clt_request+0x1a7/0x320 [rtrs_client]
? 0xffffffffc0ab1000
send_usr_msg+0xbf/0x160 [rnbd_client]
? rnbd_clt_put_sess+0x60/0x60 [rnbd_client]
? send_usr_msg+0x160/0x160 [rnbd_client]
? sg_alloc_table+0x27/0xb0
? sg_zero_buffer+0xd0/0xd0
send_msg_sess_info+0xe9/0x180 [rnbd_client]
? rnbd_clt_put_sess+0x60/0x60 [rnbd_client]
? blk_mq_alloc_tag_set+0x2ef/0x370
rnbd_clt_map_device+0xba8/0xcd0 [rnbd_client]
? send_msg_open+0x200/0x200 [rnbd_client]
rnbd_clt_map_device_store+0x3e5/0x620 [rnbd_client
To supress the calltrace, let's call get_cpu_ptr/put_cpu_ptr pair in
rtrs_clt_update_rdma_stats to disable preemption when accessing per-cpu
variable.
While at it, let's make the similar change in rtrs_clt_update_wc_stats.
And for rtrs_clt_inc_failover_cnt, though it was only called inside rcu
section, but it still can be preempted in case CONFIG_PREEMPT_RCU is
enabled, so change it to {get,put}_cpu_ptr pair either.
Link: https://lore.kernel.org/r/20211128133501.38710-1-guoqing.jiang@linux.dev
Signed-off-by: Guoqing Jiang <guoqing.jiang@linux.dev>
Reviewed-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
There are mis-match at counting inflight IO after changing the multipath
policy.
For example, we started fio test with round-robin policy and then we
changed the policy to min-inflight. IOs created under the RR policy is
finished under the min-inflight policy and inflight counter only
decreased. So the counter would be negative value. And also we started
fio test with min-inflight policy and changed the policy to the
round-robin. IOs created under the min-inflight policy increased the
inflight IO counter but the inflight IO counter was not decreased because
the policy was the round-robin when IO was finished.
So it should count IOs only if the IO is created under the min-inflight
policy. It should not care the policy when the IO is finished.
This patch adds a field mp_policy in struct rtrs_clt_io_req and stores the
multipath policy when an object of rtrs_clt_io_req is created. Then
rtrs-clt checks the mp_policy of only struct rtrs_clt_io_req instead of
the struct rtrs_clt.
Link: https://lore.kernel.org/r/20210806112112.124313-6-haris.iqbal@ionos.com
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Reviewed-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
The IO performance test with fio after swapping the likely and unlikely
macros in all if-statement shows no difference. They do not help for the
performance of rtrs.
Thanks to Haakon Bugge for the test scenario.
The fio test did random read on 32 rnbd devices and 64 processes.
Test environment:
- Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
- 376G memory
- kernel version: 5.4.86
- gcc version: gcc (Debian 8.3.0-6) 8.3.0
- Infiniband controller: Mellanox Technologies MT27800 Family [ConnectX-5]
Test result:
- before swapping: IOPS=829k, BW=3239MiB/s
- after swapping: IOPS=829k, BW=3238MiB/s
- remove all (un)likely: IOPS=829k, BW=3238MiB/s
Link: https://lore.kernel.org/r/20210806112112.124313-5-haris.iqbal@ionos.com
Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Reviewed-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
