• Jiebin Sun's avatar
    ipc/msg: mitigate the lock contention with percpu counter · 72d1e611
    Jiebin Sun authored
    The msg_bytes and msg_hdrs atomic counters are frequently updated when IPC
    msg queue is in heavy use, causing heavy cache bounce and overhead. 
    Change them to percpu_counter greatly improve the performance.  Since
    there is one percpu struct per namespace, additional memory cost is
    minimal.  Reading of the count done in msgctl call, which is infrequent. 
    So the need to sum up the counts in each CPU is infrequent.
    
    Apply the patch and test the pts/stress-ng-1.4.0
    -- system v message passing (160 threads).
    
    Score gain: 3.99x
    
    CPU: ICX 8380 x 2 sockets
    Core number: 40 x 2 physical cores
    Benchmark: pts/stress-ng-1.4.0
    -- system v message passing (160 threads)
    
    [akpm@linux-foundation.org: coding-style cleanups]
    [jiebin.sun@intel.com: avoid negative value by overflow in msginfo]
      Link: https://lkml.kernel.org/r/20220920150809.4014944-1-jiebin.sun@intel.com
    [akpm@linux-foundation.org: fix min() warnings]
    Link: https://lkml.kernel.org/r/20220913192538.3023708-3-jiebin.sun@intel.comSigned-off-by: default avatarJiebin Sun <jiebin.sun@intel.com>
    Reviewed-by: default avatarTim Chen <tim.c.chen@linux.intel.com>
    Cc: Alexander Mikhalitsyn <alexander.mikhalitsyn@virtuozzo.com>
    Cc: Alexey Gladkov <legion@kernel.org>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: Davidlohr Bueso <dave@stgolabs.net>
    Cc: Dennis Zhou <dennis@kernel.org>
    Cc: "Eric W . Biederman" <ebiederm@xmission.com>
    Cc: Manfred Spraul <manfred@colorfullife.com>
    Cc: Shakeel Butt <shakeelb@google.com>
    Cc: Tejun Heo <tj@kernel.org>
    Cc: Vasily Averin <vasily.averin@linux.dev>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    72d1e611
util.h 8.87 KB