Commit c1cecf88 authored by Sebastian Andrzej Siewior's avatar Sebastian Andrzej Siewior Committed by Peter Zijlstra

sched: Cache task_struct::flags in sched_submit_work()

sched_submit_work() is considered to be a hot path. The preempt_disable()
instruction is a compiler barrier and forces the compiler to load
task_struct::flags for the second comparison.
By using a local variable, the compiler can load the value once and keep it in
a register for the second comparison.

Verified on x86-64 with gcc-10.
Signed-off-by: default avatarSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20200819200025.lqvmyefqnbok5i4f@linutronix.de
parent 01ccf592
...@@ -4551,9 +4551,12 @@ void __noreturn do_task_dead(void) ...@@ -4551,9 +4551,12 @@ void __noreturn do_task_dead(void)
static inline void sched_submit_work(struct task_struct *tsk) static inline void sched_submit_work(struct task_struct *tsk)
{ {
unsigned int task_flags;
if (!tsk->state) if (!tsk->state)
return; return;
task_flags = tsk->flags;
/* /*
* If a worker went to sleep, notify and ask workqueue whether * If a worker went to sleep, notify and ask workqueue whether
* it wants to wake up a task to maintain concurrency. * it wants to wake up a task to maintain concurrency.
...@@ -4562,9 +4565,9 @@ static inline void sched_submit_work(struct task_struct *tsk) ...@@ -4562,9 +4565,9 @@ static inline void sched_submit_work(struct task_struct *tsk)
* in the possible wakeup of a kworker and because wq_worker_sleeping() * in the possible wakeup of a kworker and because wq_worker_sleeping()
* requires it. * requires it.
*/ */
if (tsk->flags & (PF_WQ_WORKER | PF_IO_WORKER)) { if (task_flags & (PF_WQ_WORKER | PF_IO_WORKER)) {
preempt_disable(); preempt_disable();
if (tsk->flags & PF_WQ_WORKER) if (task_flags & PF_WQ_WORKER)
wq_worker_sleeping(tsk); wq_worker_sleeping(tsk);
else else
io_wq_worker_sleeping(tsk); io_wq_worker_sleeping(tsk);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment