• Peter Zijlstra's avatar
    sched/core: Fix remote wakeups · b7e7ade3
    Peter Zijlstra authored
    Commit:
    
      b5179ac7 ("sched/fair: Prepare to fix fairness problems on migration")
    
    ... introduced a bug: Mike Galbraith found that it introduced a
    performance regression, while Paul E. McKenney reported lost
    wakeups and bisected it to this commit.
    
    The reason is that I mis-read ttwu_queue() such that I assumed any
    wakeup that got a remote queue must have had the task migrated.
    
    Since this is not so; we need to transfer this information between
    queueing the wakeup and actually doing the wakeup. Use a new
    task_struct::sched_flag for this, we already write to
    sched_contributes_to_load in the wakeup path so this is a hot and
    modified cacheline.
    Reported-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    Reported-by: default avatarMike Galbraith <umgwanakikbuti@gmail.com>
    Tested-by: default avatarMike Galbraith <umgwanakikbuti@gmail.com>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Andrew Hunter <ahh@google.com>
    Cc: Andy Lutomirski <luto@amacapital.net>
    Cc: Ben Segall <bsegall@google.com>
    Cc: Borislav Petkov <bp@alien8.de>
    Cc: Brian Gerst <brgerst@gmail.com>
    Cc: Dave Hansen <dave.hansen@linux.intel.com>
    Cc: Denys Vlasenko <dvlasenk@redhat.com>
    Cc: Fenghua Yu <fenghua.yu@intel.com>
    Cc: H. Peter Anvin <hpa@zytor.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Matt Fleming <matt@codeblueprint.co.uk>
    Cc: Morten Rasmussen <morten.rasmussen@arm.com>
    Cc: Oleg Nesterov <oleg@redhat.com>
    Cc: Paul Turner <pjt@google.com>
    Cc: Pavan Kondeti <pkondeti@codeaurora.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: byungchul.park@lge.com
    Fixes: b5179ac7 ("sched/fair: Prepare to fix fairness problems on migration")
    Link: http://lkml.kernel.org/r/20160523091907.GD15728@worktop.ger.corp.intel.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    b7e7ade3
core.c 210 KB