- 29 Nov, 2021 40 commits
-
-
Christoph Hellwig authored
This reverts commit 4896c4e64ba5d5d5acdbcf68c5910dd4f6d8fa62. The helper is not needed any more. Signed-off-by:
Christoph Hellwig <hch@lst.de> Link: https://lore.kernel.org/r/20211126115817.2087431-6-hch@lst.deSigned-off-by:
Jens Axboe <axboe@kernel.dk>
-
Christoph Hellwig authored
No need to create a new I/O context if there is none present yet in ->limit_depth. Signed-off-by:
-
Christoph Hellwig authored
Remove the unused bfqd argument, and hardcode ioc to current->io_context. Signed-off-by:
-
Christoph Hellwig authored
Move the copying of the I/O context to the block layer as that is where we can use the proper low-level interfaces. Signed-off-by:
-
Christoph Hellwig authored
Add the proper module prefix to avoid conflicts with a function in the scheduler. Signed-off-by:
-
Ming Lei authored
bio->bi_opf isn't finalized before checking the bio, so use it after submit_bio_checks() returns. Fixes: 5b13bc8a ("blk-mq: cleanup request allocation") Cc: Christoph Hellwig <hch@lst.de> Signed-off-by:
Ming Lei <ming.lei@redhat.com> Signed-off-by:
-
Jan Kara authored
Commit 7cc4ffc5 ("block, bfq: put reqs of waker and woken in dispatch list") added a condition to bfq_insert_request() which added waker's requests directly to dispatch list. The rationale was that completing waker's IO is needed to get more IO for the current queue. Although this rationale is valid, there is a hole in it. The waker does not necessarily serve the IO only for the current queue and maybe it's current IO is not needed for current queue to make progress. Furthermore injecting IO like this completely bypasses any service accounting within bfq and thus we do not properly track how much service is waker's queue getting or that the waker is actually doing any IO. Depending on the conditions this can result in the waker getting too much or too few service. Consider for example the following job file: [global] directory=/mnt/repro/ rw=write size=8g time_based runtime=30 ramp_time=10 blocksize=1m direct=0 ioengine=sync [slowwriter] numjobs=1 prioclass=2 prio=7 fsync=200 [fastwriter] numjobs=1 prioclass=2 prio=0 fsync=200 Despite processes have very different IO priorities, they get the same about of service. The reason is that bfq identifies these processes as having waker-wakee relationship and once that happens, IO from fastwriter gets injected during slowwriter's time slice. As a result bfq is not aware that fastwriter has any IO to do and constantly schedules only slowwriter's queue. Thus fastwriter is forced to compete with slowwriter's IO all the time instead of getting its share of time based on IO priority. Drop the special injection condition from bfq_insert_request(). As a result, requests will be tracked and queued in a normal way and on next dispatch bfq_select_queue() can decide whether the waker's inserted requests should be injected during the current queue's timeslice or not. Fixes: 7cc4ffc5 ("block, bfq: put reqs of waker and woken in dispatch list") Acked-by:
Paolo Valente <paolo.valente@linaro.org> Signed-off-by:
Jan Kara <jack@suse.cz> Link: https://lore.kernel.org/r/20211125133645.27483-8-jack@suse.czSigned-off-by:
-
Jan Kara authored
Waker - wakee relationships are important in deciding whether one queue can preempt the other one. Print information about detected waker-wakee relationships so that scheduling decisions can be better understood from block traces. Acked-by:
-
Jan Kara authored
Instead of having helper formating bfqq pid, provide a helper to generate full bfqq name as used in the traces. It saves some code duplication and will save more in the coming tracepoints. Acked-by:
-
Jan Kara authored
Currently, when process A starts issuing requests shortly after process B has completed some IO three times in a row, we decide that B is a "waker" of A meaning that completing IO of B is needed for A to make progress and generally stop separating A's and B's IO much. This logic is useful to avoid unnecessary idling and thus throughput loss for cases where workload needs to switch e.g. between the process and the journaling thread doing IO. However the detection heuristic tends to frequently give false positives when A and B are fighting IO bandwidth and other processes aren't doing much IO as we are basically deemed to eventually accumulate three occurences of a situation where one process starts issuing requests after the other has completed some IO. To reduce these false positives, cancel the waker detection also if we didn't accumulate three detected wakeups within given timeout. The rationale is that if wakeups are really rare, the pointless idling doesn't hurt throughput that much anyway. This significantly reduces false waker detection for workload like: [global] directory=/mnt/repro/ rw=write size=8g time_based runtime=30 ramp_time=10 blocksize=1m direct=0 ioengine=sync [slowwriter] numjobs=1 fsync=200 [fastwriter] numjobs=1 fsync=200 Acked-by:
-
Jan Kara authored
When cgroup IO scheduling is used with BFQ it does not really provide service differentiation if the cgroup drives a big IO depth. That for example happens with writeback which asynchronously submits lots of IO but it can happen with AIO as well. The problem is that if we have two cgroups that submit IO with different weights, the cgroup with higher weight properly gets more IO time and is able to dispatch more IO. However this causes lower weight cgroup to accumulate more requests inside BFQ and eventually lower weight cgroup consumes most of IO scheduler tags. At that point higher weight cgroup stops getting better service as it is mostly blocked waiting for a scheduler tag while its queues inside BFQ are empty and thus lower weight cgroup gets served. Check how many requests submitting cgroup has allocated in bfq_limit_depth() and if it consumes more requests than what would correspond to its weight limit available depth to 1 so that the cgroup cannot consume many more requests. With this limitation the higher weight cgroup gets proper service even with writeback. Reviewed-by:
Michal Koutný <mkoutny@suse.com> Acked-by:
-
Jan Kara authored
Store bitmap depth shift inside bfq_data so that we can use it in bfq_limit_depth() for proportioning when limiting number of available request tags for a cgroup. Acked-by:
-
Jan Kara authored
When we want to limit number of requests used by each bfqq and also cgroup, we need to track also number of requests used by each cgroup. So track number of allocated requests for each bfq_entity. Acked-by:
-
Jan Kara authored
Currently we lookup ICQ only after the request is allocated. However BFQ will want to decide how many scheduler tags it allows a given bfq queue (effectively a process) to consume based on cgroup weight. So provide a function blk_mq_sched_get_icq() so that BFQ can lookup ICQ earlier. Acked-by:
-
Sebastian Andrzej Siewior authored
The completion callback for the sdhci-pci device is invoked from a kworker. I couldn't identify in which context is mmc_blk_mq_req_done() invoke but the remaining caller are from invoked from preemptible context. Here it would make sense to complete the request directly instead scheduling ksoftirqd for its completion. Signed-off-by:
Sebastian Andrzej Siewior <bigeasy@linutronix.de> Reviewed-by:
Ulf Hansson <ulf.hansson@linaro.org> Acked-by:
Adrian Hunter <adrian.hunter@intel.com> Link: https://lore.kernel.org/r/20211025070658.1565848-3-bigeasy@linutronix.deSigned-off-by:
-
Sebastian Andrzej Siewior authored
Add blk_mq_complete_request_direct() which completes the block request directly instead deferring it to softirq for single queue devices. This is useful for devices which complete the requests in preemptible context and raising softirq from means scheduling ksoftirqd. Signed-off-by:
-
Eric Biggers authored
This function is trivial and is only used in one place. Having this function is misleading because it implies that blk_crypto_register() needs to be paired with blk_crypto_unregister(), which is not the case. Just set disk->queue->crypto_profile to NULL directly. Signed-off-by:
Eric Biggers <ebiggers@google.com> Reviewed-by:
-
Christoph Hellwig authored
Refactor the request alloction so that blk_mq_get_cached_request tries to find a cached request first, and the entirely separate and now self contained blk_mq_get_new_requests allocates one or more requests if that is not possible. There is a small change in behavior as submit_bio_checks is called twice now if a cached request is present but can't be used, but that is a small price to pay for unwinding this code. Signed-off-by:
-
Christoph Hellwig authored
Not needed, shift it into the source files that need it instead. Signed-off-by:
-
Christoph Hellwig authored
Not needed. Signed-off-by:
-
Christoph Hellwig authored
Not needed. Signed-off-by:
-
Christoph Hellwig authored
No needed, shift a blk-stat.h include into the source file that needs it instead. Signed-off-by:
-
Christoph Hellwig authored
No needed, shift it into the source files that need it instead. Signed-off-by:
-
Christoph Hellwig authored
All callers pass q->elevator. Signed-off-by:
-
Christoph Hellwig authored
Open code elevator_exit in it's only caller, and rename __elevator_exit to elevator_exit. Signed-off-by:
-
Christoph Hellwig authored
blk_get_flush_queue is only used in blk-flush.c, so move it there. Signed-off-by:
-
Guo Zhengkui authored
Remove a repeated "#include<linux/sched/sysctl.h>". Signed-off-by:
Guo Zhengkui <guozhengkui@vivo.com> Reviewed-by:
Chaitanya Kulkarni <kch@nvidia.com> Link: https://lore.kernel.org/r/20211123063340.25882-1-guozhengkui@vivo.comSigned-off-by:
-
Jens Axboe authored
The only user of the io_context for IO is BFQ, yet we put the checking and logic of it into the normal IO path. Put the creation into blk_mq_sched_assign_ioc(), and have BFQ use that helper. Reviewed-by:
Johannes Thumshirn <johannes.thumshirn@wdc.com> Reviewed-by:
-
Jens Axboe authored
This is essentially never used, yet it's about 1/3rd of the total queue size. Allocate it when needed, and don't embed it in the queue. Kill the queue flag for this while at it, since we can just check the assigned pointer now. Reviewed-by:
-
Jens Axboe authored
We don't need to write to the bio if: 1) No ioprio value has ever been assigned to the blkcg 2) We wouldn't anyway, depending on bio and blkcg IO priority Reviewed-by:
-
Christoph Hellwig authored
Keep all the functionality for adding a request to a plug in a single place. Signed-off-by:
-
Christoph Hellwig authored
blk_mq_submit_bio has two different plug cases, one that uses full plugging and a limited plugging one. The limited plugging case is only used for a corner case that does not matter in real life: - no ->commit_rqs (so not NVMe) - no shared tags (so not SCSI) - not rotational (so no old disk or floppy driver) - must have multiple queues (so no eMMC) Remove the limited merging case and all the related junk to simplify blk_mq_submit_bio and the functions called from it. Signed-off-by:
-
Christoph Hellwig authored
Set up GENHD_FL_REMOVABLE together with the rest of the gendisk fields. Signed-off-by:
-
Christoph Hellwig authored
Switch to an enum and tidy up the documentation. Signed-off-by:
-
Christoph Hellwig authored
Hidden gendisks can't be opened using blkdev_get_*, so we can't really reach any of the partition scanning paths or partitioning ioctls except for the initial partition scan from add_disk. Signed-off-by:
-
Christoph Hellwig authored
All modern drivers can support extra partitions using the extended dev_t. In fact except for the ioctl method drivers never even see partitions in normal operation. So remove the GENHD_FL_EXT_DEVT and allow extra partitions for all block devices that do support partitions, and require those that do not support partitions to explicit disallow them using GENHD_FL_NO_PART. Signed-off-by:
-
Christoph Hellwig authored
This flag is not set directly anywhere and only inherited from GENHD_FL_HIDDEN. Just check for GENHD_FL_HIDDEN instead. Signed-off-by:
-
Christoph Hellwig authored
This manually reverts 07b652cdbec3 ("mmc: card: Don't show eMMC RPMB and BOOT areas in /proc/partitions"). Based on the commit description that change was purely cosmetic. mmc is the last driver that sets this flag and thus prevents it from being removed. Signed-off-by: -
Christoph Hellwig authored
This manually reverts commit 27290b469051 ("null_blk: suppress invalid partition info"). The message in that commit log can't appearch as the flag is never checked during probing, and there is no good reason to treat null_blk special in /proc/partitions. Signed-off-by: -
Christoph Hellwig authored
Hidden gendisks never hash the block device inode, so this can't happen. Signed-off-by:
-
