• Sukadev Bhattiprolu's avatar
    perf trace: Fix race condition at the end of started workloads · 7951722d
    Sukadev Bhattiprolu authored
    I get following crash on multiple systems and across several releases
    (at least since v3.18).
    
    	Core was generated by `/tmp/perf trace sleep 0.2 '.
    	Program terminated with signal SIGSEGV, Segmentation fault.
    	#0  perf_mmap__read_head (mm=0x3fff9bf30070) at util/evlist.h:195
    	195		u64 head = ACCESS_ONCE(pc->data_head);
    	(gdb) bt
    	#0  perf_mmap__read_head (mm=0x3fff9bf30070) at util/evlist.h:195
    	#1  perf_evlist__mmap_read (evlist=0x10027f11910, idx=<optimized out>)
    	    at util/evlist.c:637
    	#2  0x000000001003ce4c in trace__run (argv=<optimized out>,
    	    argc=<optimized out>, trace=0x3fffd7b28288) at builtin-trace.c:2259
    	#3  cmd_trace (argc=<optimized out>, argv=<optimized out>,
    	    prefix=<optimized out>) at builtin-trace.c:2799
    	#4  0x00000000100657b8 in run_builtin (p=0x10176798 <commands+480>, argc=3,
    	    argv=0x3fffd7b2b550) at perf.c:370
    	#5  0x00000000100063e8 in handle_internal_command (argv=0x3fffd7b2b550, argc=3)
    	    at perf.c:429
    	#6  run_argv (argv=0x3fffd7b2af70, argcp=0x3fffd7b2af7c) at perf.c:473
    	#7  main (argc=3, argv=0x3fffd7b2b550) at perf.c:588
    
    The problem seems to be a race condition, when the application has just
    exited.  Some/all fds associated with the perf-events (tracepoints) go
    into a POLLHUP/ POLLERR state and the mmap region associated with those
    events are unmapped (in perf_evlist__filter_pollfd()).
    
    But we go back and do a perf_evlist__mmap_read() which assumes that the
    mmaps are still valid and we hit the crash.
    
    If the mapping for an event is released, its refcnt is 0 (and ->base
    is NULL), so ensure we have non-zero refcount before accessing the map.
    
    Note that perf-record has a similar logic but unlike perf-trace, the
    record__mmap_read_all() checks the evlist->mmap[i].base before accessing
    the map.
    Signed-off-by: default avatarSukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
    Cc: Jiri Olsa <jolsa@redhat.com>
    Cc: Li Zhang <zhlcindy@linux.vnet.ibm.com>
    Link: http://lkml.kernel.org/r/20150612060003.GA19913@us.ibm.com
    [ Fixed it up to use atomic_read() ]
    Signed-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
    7951722d
evlist.c 37.8 KB