• Pan Xinhui's avatar
    locking/osq: Break out of spin-wait busy waiting loop for a preempted vCPU in osq_lock() · 5aff60a1
    Pan Xinhui authored
    An over-committed guest with more vCPUs than pCPUs has a heavy overload
    in osq_lock().
    
    This is because if vCPU-A holds the osq lock and yields out, vCPU-B ends
    up waiting for per_cpu node->locked to be set. IOW, vCPU-B waits for
    vCPU-A to run and unlock the osq lock.
    
    Use the new vcpu_is_preempted(cpu) interface to detect if a vCPU is
    currently running or not, and break out of the spin-loop if so.
    
    test case:
    
     $ perf record -a perf bench sched messaging -g 400 -p && perf report
    
     before patch:
     18.09%  sched-messaging  [kernel.vmlinux]  [k] osq_lock
     12.28%  sched-messaging  [kernel.vmlinux]  [k] rwsem_spin_on_owner
      5.27%  sched-messaging  [kernel.vmlinux]  [k] mutex_unlock
      3.89%  sched-messaging  [kernel.vmlinux]  [k] wait_consider_task
      3.64%  sched-messaging  [kernel.vmlinux]  [k] _raw_write_lock_irq
      3.41%  sched-messaging  [kernel.vmlinux]  [k] mutex_spin_on_owner.is
      2.49%  sched-messaging  [kernel.vmlinux]  [k] system_call
    
     after patch:
     20.68%  sched-messaging  [kernel.vmlinux]  [k] mutex_spin_on_owner
      8.45%  sched-messaging  [kernel.vmlinux]  [k] mutex_unlock
      4.12%  sched-messaging  [kernel.vmlinux]  [k] system_call
      3.01%  sched-messaging  [kernel.vmlinux]  [k] system_call_common
      2.83%  sched-messaging  [kernel.vmlinux]  [k] copypage_power7
      2.64%  sched-messaging  [kernel.vmlinux]  [k] rwsem_spin_on_owner
      2.00%  sched-messaging  [kernel.vmlinux]  [k] osq_lock
    Suggested-by: default avatarBoqun Feng <boqun.feng@gmail.com>
    Tested-by: default avatarJuergen Gross <jgross@suse.com>
    Signed-off-by: default avatarPan Xinhui <xinhui.pan@linux.vnet.ibm.com>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Acked-by: default avatarChristian Borntraeger <borntraeger@de.ibm.com>
    Acked-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
    Cc: David.Laight@ACULAB.COM
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: benh@kernel.crashing.org
    Cc: bsingharora@gmail.com
    Cc: dave@stgolabs.net
    Cc: kernellwp@gmail.com
    Cc: konrad.wilk@oracle.com
    Cc: linuxppc-dev@lists.ozlabs.org
    Cc: mpe@ellerman.id.au
    Cc: paulmck@linux.vnet.ibm.com
    Cc: paulus@samba.org
    Cc: rkrcmar@redhat.com
    Cc: virtualization@lists.linux-foundation.org
    Cc: will.deacon@arm.com
    Cc: xen-devel-request@lists.xenproject.org
    Cc: xen-devel@lists.xenproject.org
    Link: http://lkml.kernel.org/r/1478077718-37424-3-git-send-email-xinhui.pan@linux.vnet.ibm.com
    [ Translated to English. ]
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
    5aff60a1
osq_lock.c 5.27 KB