Commit 9054650f authored by Jens Axboe's avatar Jens Axboe

block: relax jiffies rounding for timeouts

In doing high IOPS testing, blk-mq is generally pretty well optimized.
There are a few things that stuck out as using more CPU than what is
really warranted, and one thing is the round_jiffies_up() that we do
twice for each request. That accounts for about 0.8% of the CPU in
my testing.

We can make this cheaper by avoiding an integer division, by just adding
a rough HZ mask that we can AND with instead. The timeouts are only on a
second granularity already, we don't have to be that accurate here and
this patch barely changes that. All we care about is nice grouping.
Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
parent 87890092
...@@ -88,11 +88,29 @@ void blk_abort_request(struct request *req) ...@@ -88,11 +88,29 @@ void blk_abort_request(struct request *req)
} }
EXPORT_SYMBOL_GPL(blk_abort_request); EXPORT_SYMBOL_GPL(blk_abort_request);
static unsigned long blk_timeout_mask __read_mostly;
int __init blk_timeout_init(void)
{
blk_timeout_mask = roundup_pow_of_two(HZ) - 1;
return 0;
}
late_initcall(blk_timeout_init);
/*
* Just a rough estimate, we don't care about specific values for timeouts.
*/
static inline unsigned long blk_round_jiffies(unsigned long j)
{
return (j + blk_timeout_mask) + 1;
}
unsigned long blk_rq_timeout(unsigned long timeout) unsigned long blk_rq_timeout(unsigned long timeout)
{ {
unsigned long maxt; unsigned long maxt;
maxt = round_jiffies_up(jiffies + BLK_MAX_TIMEOUT); maxt = blk_round_jiffies(jiffies + BLK_MAX_TIMEOUT);
if (time_after(timeout, maxt)) if (time_after(timeout, maxt))
timeout = maxt; timeout = maxt;
...@@ -129,7 +147,7 @@ void blk_add_timer(struct request *req) ...@@ -129,7 +147,7 @@ void blk_add_timer(struct request *req)
* than an existing one, modify the timer. Round up to next nearest * than an existing one, modify the timer. Round up to next nearest
* second. * second.
*/ */
expiry = blk_rq_timeout(round_jiffies_up(expiry)); expiry = blk_rq_timeout(blk_round_jiffies(expiry));
if (!timer_pending(&q->timeout) || if (!timer_pending(&q->timeout) ||
time_before(expiry, q->timeout.expires)) { time_before(expiry, q->timeout.expires)) {
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment