• Peter Zijlstra's avatar
    sched/fair: Fix effective_load() to consistently use smoothed load · 7dd49125
    Peter Zijlstra authored
    Starting with the following commit:
    
      fde7d22e ("sched/fair: Fix overly small weight for interactive group entities")
    
    calc_tg_weight() doesn't compute the right value as expected by effective_load().
    
    The difference is in the 'correction' term. In order to ensure \Sum
    rw_j >= rw_i we cannot use tg->load_avg directly, since that might be
    lagging a correction on the current cfs_rq->avg.load_avg value.
    Therefore we use tg->load_avg - cfs_rq->tg_load_avg_contrib +
    cfs_rq->avg.load_avg.
    
    Now, per the referenced commit, calc_tg_weight() doesn't use
    cfs_rq->avg.load_avg, as is later used in @w, but uses
    cfs_rq->load.weight instead.
    
    So stop using calc_tg_weight() and do it explicitly.
    
    The effects of this bug are wake_affine() making randomly
    poor choices in cgroup-intense workloads.
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: <stable@vger.kernel.org> # v4.3+
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Fixes: fde7d22e ("sched/fair: Fix overly small weight for interactive group entities")
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
    7dd49125
fair.c 229 KB