• Martin KaFai Lau's avatar
    kprobes: Optimize hot path by using percpu counter to collect 'nhit' statistics · a7636d9e
    Martin KaFai Lau authored
    When doing ebpf+kprobe on some hot TCP functions (e.g.
    tcp_rcv_established), the kprobe_dispatcher() function
    shows up in 'perf report'.
    
    In kprobe_dispatcher(), there is a lot of cache bouncing
    on 'tk->nhit++'.  'tk->nhit' and 'tk->tp.flags' also share
    the same cacheline.
    
    perf report (cycles:pp):
    
    	8.30%  ipv4_dst_check
    	4.74%  copy_user_enhanced_fast_string
    	3.93%  dst_release
    	2.80%  tcp_v4_rcv
    	2.31%  queued_spin_lock_slowpath
    	2.30%  _raw_spin_lock
    	1.88%  mlx4_en_process_rx_cq
    	1.84%  eth_get_headlen
    	1.81%  ip_rcv_finish
    	~~~~
    	1.71%  kprobe_dispatcher
    	~~~~
    	1.55%  mlx4_en_xmit
    	1.09%  __probe_kernel_read
    
    perf report after patch:
    
    	9.15%  ipv4_dst_check
    	5.00%  copy_user_enhanced_fast_string
    	4.12%  dst_release
    	2.96%  tcp_v4_rcv
    	2.50%  _raw_spin_lock
    	2.39%  queued_spin_lock_slowpath
    	2.11%  eth_get_headlen
    	2.03%  mlx4_en_process_rx_cq
    	1.69%  mlx4_en_xmit
    	1.19%  ip_rcv_finish
    	1.12%  __probe_kernel_read
    	1.02%  ehci_hcd_cleanup
    Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
    Acked-by: default avatarMasami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
    Acked-by: default avatarSteven Rostedt <rostedt@goodmis.org>
    Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com>
    Cc: Josef Bacik <jbacik@fb.com>
    Cc: Kernel Team <kernel-team@fb.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Link: http://lkml.kernel.org/r/1454531308-2441898-1-git-send-email-kafai@fb.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    a7636d9e
trace_kprobe.c 36.3 KB