1. 12 Aug, 2015 17 commits
  2. 10 Aug, 2015 7 commits
  3. 08 Aug, 2015 3 commits
  4. 07 Aug, 2015 13 commits
    • Wang Nan's avatar
      perf tests: Add LLVM test for eBPF on-the-fly compiling · 9bc898c7
      Wang Nan authored
      Previous patches introduce llvm__compile_bpf() to compile source file to
      eBPF object. This patch adds testcase to test it. It also tests libbpf
      by opening generated object after applying next patch which introduces
      HAVE_LIBBPF_SUPPORT option.
      
      Since llvm__compile_bpf() prints long messages which users who don't
      explicitly test llvm doesn't care, this patch set verbose to -1 to
      suppress all debug, warning and error message, and hint user use 'perf
      test -v' to see the full output.
      
      For the same reason, if clang is not found in PATH and there's no [llvm]
      section in .perfconfig, skip this test.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/n/1436445342-1402-17-git-send-email-wangnan0@huawei.com
      [ Add tools/lib/bpf/ to tools/perf/MANIFEST, so that the tarball targets build ]
      Signed-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      9bc898c7
    • Wang Nan's avatar
      perf tools: Auto detecting kernel include options · 0c6d18bf
      Wang Nan authored
      To help user find correct kernel include options, this patch extracts
      them from kbuild system by an embedded script kinc_fetch_script, which
      creates a temporary directory, generates Makefile and an empty dummy.o
      then use the Makefile to fetch $(NOSTDINC_FLAGS), $(LINUXINCLUDE) and
      $(EXTRA_CFLAGS) options. The result is passed to compiler script using
      'KERNEL_INC_OPTIONS' environment variable.
      
      Because options from kbuild contains relative path like
      'Iinclude/generated/uapi', the work directory must be changed. This is
      done by previous patch.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/r/1436445342-1402-16-git-send-email-wangnan0@huawei.comSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      0c6d18bf
    • Wang Nan's avatar
      perf tools: Auto detecting kernel build directory · d325d788
      Wang Nan authored
      This patch detects kernel build directory by checking the existence of
      include/generated/autoconf.h.
      
      clang working directory is changed to kbuild directory if it is found,
      to help user use relative include path. Following patch will detect
      kernel include directory, which contains relative include patch so this
      workdir changing is needed.
      
      Users are allowed to set 'kbuild-dir = ""' manually to disable this
      checking.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/n/tip-owyfwfbemrjn0tlj6tgk2nf5@git.kernel.orgSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      d325d788
    • Wang Nan's avatar
      perf tools: Call clang to compile C source to object code · 4cea3a9c
      Wang Nan authored
      This is the core patch for supporting eBPF on-the-fly compiling, does
      the following work:
      
       1. Search clang compiler using search_program().
      
       2. Run command template defined in llvm-bpf-cmd-template option in
          [llvm] config section using read_from_pipe(). Patch of clang and
          source code path is injected into shell command using environment
          variable using force_set_env().
      
        Commiter notice:
      
        When building with DEBUG=1 we get a compiler error that gets fixed with
        the same approach described in commit b2365122:
      
          perf kmem: Fix compiler warning about may be accessing uninitialized variable
      
          The last argument to strtok_r doesn't need to be initialized, its
          just a placeholder to make this routine reentrant, but gcc doesn't know
          about that and complains, breaking the build, fix it by setting it to
          NULL.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/n/1436445342-1402-14-git-send-email-wangnan0@huawei.comSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      4cea3a9c
    • Wang Nan's avatar
      perf tools: Introduce llvm config options · aa61fd05
      Wang Nan authored
      This patch introduces [llvm] config section with 5 options. Following
      patches will use then to config llvm dynamica compiling.
      
      'llvm-utils.[ch]' is introduced in this patch for holding all
      llvm/clang related stuffs.
      
      Example:
      
        [llvm]
              # Path to clang. If omit, search it from $PATH.
      	clang-path = "/path/to/clang"
      
              # Cmdline template. Following line shows its default value.
              # Environment variable is used to passing options.
              #
              # *NOTE*: -D__KERNEL__ MUST appears before $CLANG_OPTIONS,
              # so user have a chance to use -U__KERNEL__ in $CLANG_OPTIONS
              # to cancel it.
      	clang-bpf-cmd-template = "$CLANG_EXEC -D__KERNEL__ $CLANG_OPTIONS \
      				  $KERNEL_INC_OPTIONS -Wno-unused-value \
      				  -Wno-pointer-sign -working-directory \
      				  $WORKING_DIR  -c $CLANG_SOURCE -target \
      				  bpf -O2 -o -"
      
              # Options passed to clang, will be passed to cmdline by
              # $CLANG_OPTIONS.
      	clang-opt = "-Wno-unused-value -Wno-pointer-sign"
      
              # kbuild directory. If not set, use /lib/modules/`uname -r`/build.
              # If set to "" deliberately, skip kernel header auto-detector.
      	kbuild-dir = "/path/to/kernel/build"
      
              # Options passed to 'make' when detecting kernel header options.
      	kbuild-opts = "ARCH=x86_64"
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/r/1437477214-149684-1-git-send-email-wangnan0@huawei.comSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      aa61fd05
    • Wang Nan's avatar
      bpf tools: Link all bpf objects onto a list · 9a208eff
      Wang Nan authored
      To allow enumeration of all bpf_objects, keep them in a list (hidden to
      caller). bpf_object__for_each_safe() is introduced to do this iteration.
      It is safe even user close the object during iteration.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/r/1435716878-189507-23-git-send-email-wangnan0@huawei.comSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      9a208eff
    • Wang Nan's avatar
      bpf tools: Introduce accessors for struct bpf_program · aa9b1ac3
      Wang Nan authored
      This patch introduces accessors for user of libbpf to retrieve section
      name and fd of a opened/loaded eBPF program. 'struct bpf_prog_handler'
      is used for that purpose. Accessors of programs section name and file
      descriptor are provided. Set/get private data are also impelmented.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Link: http://lkml.kernel.org/r/1435716878-189507-21-git-send-email-wangnan0@huawei.comSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      aa9b1ac3
    • Wang Nan's avatar
      bpf tools: Load eBPF programs in object files into kernel · 55cffde2
      Wang Nan authored
      This patch utilizes previous introduced bpf_load_program to load
      programs in the ELF file into kernel. Result is stored in 'fd' field in
      'struct bpf_program'.
      
      During loading, it allocs a log buffer and free it before return.  Note
      that that buffer is not passed to bpf_load_program() if the first
      loading try is successful. Doesn't use a statically allocated log buffer
      to avoid potention multi-thread problem.
      
      Instructions collected during opening is cleared after loading.
      
      load_program() is created for loading a 'struct bpf_insn' array into
      kernel, bpf_program__load() calls it. By this design we have a function
      loads instructions into kernel. It will be used by further patches,
      which creates different instances from a program and load them into
      kernel.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/r/1435716878-189507-20-git-send-email-wangnan0@huawei.comSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      55cffde2
    • Wang Nan's avatar
      bpf tools: Introduce bpf_load_program() to bpf.c · 7bf98369
      Wang Nan authored
      bpf_load_program() can be used to load bpf program into kernel. To make
      loading faster, first try to load without logbuf. Try again with logbuf
      if the first try failed.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/r/1435716878-189507-19-git-send-email-wangnan0@huawei.comSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      7bf98369
    • Wang Nan's avatar
      bpf tools: Relocate eBPF programs · 8a47a6c5
      Wang Nan authored
      If an eBPF program accesses a map, LLVM generates a load instruction
      which loads an absolute address into a register, like this:
      
        ld_64   r1, <MCOperand Expr:(mymap)>
        ...
        call    2
      
      That ld_64 instruction will be recorded in relocation section.
      To enable the usage of that map, relocation must be done by replacing
      the immediate value by real map file descriptor so it can be found by
      eBPF map functions.
      
      This patch to the relocation work based on information collected by
      patches:
      
      'bpf tools: Collect symbol table from SHT_SYMTAB section',
      'bpf tools: Collect relocation sections from SHT_REL sections'
      and
      'bpf tools: Record map accessing instructions for each program'.
      
      For each instruction which needs relocation, it inject corresponding
      file descriptor to imm field. As a part of protocol, src_reg is set to
      BPF_PSEUDO_MAP_FD to notify kernel this is a map loading instruction.
      
      This is the final part of map relocation patch. The principle of map
      relocation is described in commit message of 'bpf tools: Collect symbol
      table from SHT_SYMTAB section'.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/r/1435716878-189507-18-git-send-email-wangnan0@huawei.comSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      8a47a6c5
    • Wang Nan's avatar
      bpf tools: Create eBPF maps defined in an object file · 52d3352e
      Wang Nan authored
      This patch creates maps based on 'map' section in object file using
      bpf_create_map(), and stores the fds into an array in 'struct
      bpf_object'.
      
      Previous patches parse ELF object file and collects required data, but
      doesn't play with the kernel. They belong to the 'opening' phase. This
      patch is the first patch in 'loading' phase. The 'loaded' field is
      introduced in 'struct bpf_object' to avoid loading an object twice,
      because the loading phase clears resources collected during the opening
      which becomes useless after loading. In this patch, maps_buf is cleared.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/r/1435716878-189507-17-git-send-email-wangnan0@huawei.comSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      52d3352e
    • Wang Nan's avatar
      bpf tools: Add bpf.c/h for common bpf operations · e3ed2fef
      Wang Nan authored
      This patch introduces bpf.c and bpf.h, which hold common functions
      issuing bpf syscall. The goal of these two files is to hide syscall
      completely from user. Note that bpf.c and bpf.h deal with kernel
      interface only. Things like structure of 'map' section in the ELF object
      is not cared by of bpf.[ch].
      
      We first introduce bpf_create_map().
      
      Note that, since functions in bpf.[ch] are wrapper of sys_bpf, they
      don't use OO style naming.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/r/1435716878-189507-16-git-send-email-wangnan0@huawei.comSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      e3ed2fef
    • Wang Nan's avatar
      bpf tools: Record map accessing instructions for each program · 34090915
      Wang Nan authored
      This patch records the indices of instructions which are needed to be
      relocated. That information is saved in the 'reloc_desc' field in
      'struct bpf_program'. In the loading phase (this patch takes effect in
      the opening phase), the collected instructions will be replaced by map
      loading instructions.
      
      Since we are going to close the ELF file and clear all data at the end
      of the 'opening' phase, the ELF information will no longer be valid in
      the 'loading' phase. We have to locate the instructions before maps are
      loaded, instead of directly modifying the instruction.
      
      'struct bpf_map_def' is introduced in this patch to let us know how many
      maps are defined in the object.
      
      This is the third part of map relocation. The principle of map relocation
      is described in commit message of 'bpf tools: Collect symbol table from
      SHT_SYMTAB section'.
      Signed-off-by: default avatarWang Nan <wangnan0@huawei.com>
      Acked-by: default avatarAlexei Starovoitov <ast@plumgrid.com>
      Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: David Ahern <dsahern@gmail.com>
      Cc: He Kuang <hekuang@huawei.com>
      Cc: Jiri Olsa <jolsa@kernel.org>
      Cc: Kaixu Xia <xiakaixu@huawei.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Namhyung Kim <namhyung@kernel.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Zefan Li <lizefan@huawei.com>
      Cc: pi3orama@163.com
      Link: http://lkml.kernel.org/r/1435716878-189507-15-git-send-email-wangnan0@huawei.comSigned-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      34090915