- 29 May, 2020 10 commits
-
-
Ming Lei authored
Most of blk-mq drivers depend on managed IRQ's auto-affinity to setup up queue mapping. Thomas mentioned the following point[1]: "That was the constraint of managed interrupts from the very beginning: The driver/subsystem has to quiesce the interrupt line and the associated queue _before_ it gets shutdown in CPU unplug and not fiddle with it until it's restarted by the core when the CPU is plugged in again." However, current blk-mq implementation doesn't quiesce hw queue before the last CPU in the hctx is shutdown. Even worse, CPUHP_BLK_MQ_DEAD is a cpuhp state handled after the CPU is down, so there isn't any chance to quiesce the hctx before shutting down the CPU. Add new CPUHP_AP_BLK_MQ_ONLINE state to stop allocating from blk-mq hctxs where the last CPU goes away, and wait for completion of in-flight requests. This guarantees that there is no inflight I/O before shutting down the managed IRQ. Add a BLK_MQ_F_STACKING and set it for dm-rq and loop, so we don't need to wait for completion of in-flight requests from these drivers to avoid a potential dead-lock. It is safe to do this for stacking drivers as those do not use interrupts at all and their I/O completions are triggered by underlying devices I/O completion. [1] https://lore.kernel.org/linux-block/alpine.DEB.2.21.1904051331270.1802@nanos.tec.linutronix.de/ [hch: different retry mechanism, merged two patches, minor cleanups] Signed-off-by:
Ming Lei <ming.lei@redhat.com> Signed-off-by:
Christoph Hellwig <hch@lst.de> Reviewed-by:
Hannes Reinecke <hare@suse.de> Reviewed-by:
Daniel Wagner <dwagner@suse.de> Signed-off-by:
Jens Axboe <axboe@kernel.dk>
-
Ming Lei authored
Add a new blk_mq_all_tag_iter function to iterate over all allocated scheduler tags and driver tags. This is more flexible than the existing blk_mq_all_tag_busy_iter function as it allows the callers to do whatever they want on allocated request instead of being limited to started requests. It will be used to implement draining allocated requests on specified hctx in this patchset. [hch: switch from the two booleans to a more readable flags field and consolidate the tags iter functions] Signed-off-by:
Bart van Assche <bvanassche@acm.org> Signed-off-by:
-
Christoph Hellwig authored
blk_mq_alloc_request_hctx is only used for NVMeoF connect commands, so tailor it to the specific requirements, and don't bother the general fast path code with its special twinkles. Signed-off-by:
-
Christoph Hellwig authored
Replace various magic -1 constants for tags with BLK_MQ_NO_TAG. Signed-off-by:
Johannes Thumshirn <johannes.thumshirn@wdc.com> Reviewed-by:
-
Christoph Hellwig authored
To prepare for wider use of this constant give it a more applicable name. Signed-off-by:
-
Christoph Hellwig authored
Don't split request initialization between __blk_mq_alloc_request and blk_mq_rq_ctx_init. Also remove the op argument as it can be derived from the blk_mq_alloc_data structure. Signed-off-by:
-
Christoph Hellwig authored
The bio argument is entirely unused, and the request_queue can be passed through the alloc_data, given that it needs to be filled out for the low-level tag allocation anyway. Also rename the function to __blk_mq_alloc_request as the switch between get and alloc in the call chains is rather confusing. Signed-off-by:
-
Christoph Hellwig authored
None of the I/O schedulers actually needs it. Signed-off-by:
-
Keith Busch authored
Use blk_mq_foce_complete_rq() to bypass fake timeout error injection so that request reclaim may proceed. Signed-off-by:
Keith Busch <kbusch@kernel.org> Reviewed-by:
-
Keith Busch authored
Drivers may need to bypass error injection for error recovery. Rename __blk_mq_complete_request() to blk_mq_force_complete_rq() and export that function so drivers may skip potential fake timeouts after they've reclaimed lost requests. Signed-off-by:
-
- 28 May, 2020 1 commit
-
-
Christoph Hellwig authored
disk_start_io_acct and disk_end_io_acct need at least a struct gendisk forward declaration, but for weird historic reasons much of blkdev.h is stubbed out for CONFIG_BLOCK=n. Fix this by stubbing more out for now, but eventually this header will need a massive cleanup. Fixes: 956d510e ("block: add disk/bio-based accounting helpers") Reported-by:
Stephen Rothwell <sfr@canb.auug.org.au> Signed-off-by:
-
- 27 May, 2020 17 commits
-
-
Colin Ian King authored
The variable err is being initialized with a value that is never read and it is being updated later with a new value. The initialization is redundant and can be removed. Signed-off-by:
Colin Ian King <colin.king@canonical.com> Reviewed-by:
Eric Biggers <ebiggers@google.com> Reviewed-by:
Satya Tangirala <satyat@google.com> Addresses-Coverity: ("Unused value") Signed-off-by:
-
Christoph Hellwig authored
We only need the stats lock (aka preempt_disable()) for updating the states, not for looking up or dropping the hd_struct reference. Signed-off-by:
Konstantin Khlebnikov <khlebnikov@yandex-team.ru> Signed-off-by:
-
Konstantin Khlebnikov authored
Most architectures have fast path to access percpu for current cpu. The required preempt_disable() is provided by part_stat_lock(). [hch: rebased] Signed-off-by:
-
Konstantin Khlebnikov authored
The RCU lock is required only in disk_map_sector_rcu() to lookup the partition. After that request holds reference to related hd_struct. Replace get_cpu() with preempt_disable() - returned cpu index is unused. [hch: rebased] Signed-off-by:
-
Konstantin Khlebnikov authored
Move the non-"new_io" branch of blk_account_io_start() into separate function. Fix merge accounting for discards (they were counted as write merges). The new blk_account_io_merge_bio() doesn't call update_io_ticks() unlike blk_account_io_start(), as there is no reason for that. [hch: rebased] Signed-off-by:
-
Konstantin Khlebnikov authored
Also rename blk_account_io_merge() into blk_account_io_merge_request() to distinguish it from merging request and bio. [hch: rebased] Signed-off-by:
-
Christoph Hellwig authored
percpu variables have a perfectly fine working stub implementation for UP kernels, so use that. Signed-off-by:
-
Christoph Hellwig authored
All callers are in blk-core.c, so move update_io_ticks over. Signed-off-by:
-
Christoph Hellwig authored
Remove these now unused functions. Signed-off-by:
-
Christoph Hellwig authored
Switch zram to use the nicer bio accounting helpers, and as part of that ensure each bio is counted as a single I/O request. Signed-off-by:
-
Christoph Hellwig authored
Switch dm to use the nicer bio accounting helpers. Signed-off-by:
-
Christoph Hellwig authored
Switch dm to use the nicer bio accounting helpers. Signed-off-by:
-
Christoph Hellwig authored
Switch bcache to use the nicer bio accounting helpers, and call the routines where we also sample the start time to give coherent accounting results. Signed-off-by:
Coly Li <colyli@suse.de> Signed-off-by:
-
Christoph Hellwig authored
Switch rsxx to use the nicer bio accounting helpers. Signed-off-by:
-
Christoph Hellwig authored
Switch rsxx to use the nicer bio accounting helpers. Signed-off-by:
-
Christoph Hellwig authored
Switch drbd to use the nicer bio accounting helpers. Signed-off-by:
-
Christoph Hellwig authored
Add two new helpers to simplify I/O accounting for bio based drivers. Currently these drivers use the generic_start_io_acct and generic_end_io_acct helpers which have very cumbersome calling conventions, don't actually return the time they started accounting, and try to deal with accounting for partitions, which can't happen for bio based drivers. The new helpers will be used to subsequently replace uses of the old helpers. The main API is the bio based wrappes in blkdev.h, but for zram which wants to account rw_page based I/O lower level routines are provided as well. Signed-off-by:
-
- 22 May, 2020 2 commits
-
-
Christoph Hellwig authored
Both of these never can be NULL for a live block device. Signed-off-by:
-
Christoph Hellwig authored
The argument isn't used by any caller, and drivers don't fill out bi_sector for flush requests either. Signed-off-by:
-
- 19 May, 2020 10 commits
-
-
Baolin Wang authored
The flush_queue_delayed was introdued to hold queue if flush is running for non-queueable flush drive by commit 3ac0cc45 ("hold queue if flush is running for non-queueable flush drive"), but the non mq parts of the flush code had been removed by commit 7e992f84 ("block: remove non mq parts from the flush code"), as well as removing the usage of the flush_queue_delayed flag. Thus remove the unused flush_queue_delayed flag. Signed-off-by:
Baolin Wang <baolin.wang7@gmail.com> Reviewed-by:
-
Bart Van Assche authored
This patch suppresses an uninteresting KMSAN complaint without affecting performance of the null_blk driver if CONFIG_KMSAN is disabled. Reported-by:
Alexander Potapenko <glider@google.com> Signed-off-by:
-
Bart Van Assche authored
Since it is nontrivial that nth_page() does not have to be used for a bio_vec, document this. Signed-off-by:
-
Bart Van Assche authored
This change makes it possible to pass 'const struct bio *' arguments to these functions. Signed-off-by:
Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com> Cc: Ming Lei <ming.lei@redhat.com> Cc: Damien Le Moal <damien.lemoal@wdc.com> Cc: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com> Cc: Alexander Potapenko <glider@google.com> Signed-off-by:
-
Bart Van Assche authored
This patch fixes the following sparse warnings: block/ioctl.c:209:16: warning: incorrect type in argument 1 (different address spaces) block/ioctl.c:209:16: expected void const volatile [noderef] <asn:1> * block/ioctl.c:209:16: got signed int [usertype] *argp block/ioctl.c:214:16: warning: incorrect type in argument 1 (different address spaces) block/ioctl.c:214:16: expected void const volatile [noderef] <asn:1> * block/ioctl.c:214:16: got unsigned int [usertype] *argp block/ioctl.c:666:40: warning: incorrect type in argument 1 (different address spaces) block/ioctl.c:666:40: expected signed int [usertype] *argp block/ioctl.c:666:40: got void [noderef] <asn:1> *argp block/ioctl.c:672:41: warning: incorrect type in argument 1 (different address spaces) block/ioctl.c:672:41: expected unsigned int [usertype] *argp block/ioctl.c:672:41: got void [noderef] <asn:1> *argp Fixes: 9b81648c ("compat_ioctl: simplify up block/ioctl.c") Signed-off-by:
Arnd Bergmann <arnd@arndb.de> Cc: Arnd Bergmann <arnd@arndb.de> Signed-off-by:
-
Christoph Hellwig authored
part_inc_in_flight and part_dec_in_flight only have one caller each, and those callers are purely for bio based drivers. Merge each function into the only caller, and remove the superflous blk-mq checks. Signed-off-by:
-
Christoph Hellwig authored
part_inc_in_flight and part_dec_in_flight are no-ops for blk-mq queues, so remove the calls in purely blk-mq callers. Signed-off-by:
-
Christoph Hellwig authored
Don't bother to call part_in_flight / part_in_flight_rw on blk-mq devices, just call the blk-mq versions directly. Signed-off-by:
-
Christoph Hellwig authored
Signed-off-by:
-
Christoph Hellwig authored
blk_mq_make_request currently needs to grab an q_usage_counter reference when allocating a request. This is because the block layer grabs one before calling blk_mq_make_request, but also releases it as soon as blk_mq_make_request returns. Remove the blk_queue_exit call after blk_mq_make_request returns, and instead let it consume the reference. This works perfectly fine for the block layer caller, just device mapper needs an extra reference as the old problem still persists there. Open code blk_queue_enter_live in device mapper, as there should be no other callers and this allows better documenting why we do a non-try get. Signed-off-by:
-
