1. 09 Dec, 2018 6 commits
    • Martin KaFai Lau's avatar
      bpf: libbpf: Add btf_line_info support to libbpf · 3d650141
      Martin KaFai Lau authored
      This patch adds bpf_line_info support to libbpf:
      1) Parsing the line_info sec from ".BTF.ext"
      2) Relocating the line_info.  If the main prog *_info relocation
         fails, it will ignore the remaining subprog line_info and continue.
         If the subprog *_info relocation fails, it will bail out.
      3) BPF_PROG_LOAD a prog with line_info
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      3d650141
    • Martin KaFai Lau's avatar
      bpf: libbpf: Refactor and bug fix on the bpf_func_info loading logic · f0187f0b
      Martin KaFai Lau authored
      This patch refactor and fix a bug in the libbpf's bpf_func_info loading
      logic.  The bug fix and refactoring are targeting the same
      commit 2993e051 ("tools/bpf: add support to read .BTF.ext sections")
      which is in the bpf-next branch.
      
      1) In bpf_load_program_xattr(), it should retry when errno == E2BIG
         regardless of log_buf and log_buf_sz.  This patch fixes it.
      
      2) btf_ext__reloc_init() and btf_ext__reloc() are essentially
         the same except btf_ext__reloc_init() always has insns_cnt == 0.
         Hence, btf_ext__reloc_init() is removed.
      
         btf_ext__reloc() is also renamed to btf_ext__reloc_func_info()
         to get ready for the line_info support in the next patch.
      
      3) Consolidate func_info section logic from "btf_ext_parse_hdr()",
         "btf_ext_validate_func_info()" and "btf_ext__new()" to
         a new function "btf_ext_copy_func_info()" such that similar
         logic can be reused by the later libbpf's line_info patch.
      
      4) The next line_info patch will store line_info_cnt instead of
         line_info_len in the bpf_program because the kernel is taking
         line_info_cnt also.  It will save a few "len" to "cnt" conversions
         and will also save some function args.
      
         Hence, this patch also makes bpf_program to store func_info_cnt
         instead of func_info_len.
      
      5) btf_ext depends on btf.  e.g. the func_info's type_id
         in ".BTF.ext" is not useful when ".BTF" is absent.
         This patch only init the obj->btf_ext pointer after
         it has successfully init the obj->btf pointer.
      
         This can avoid always checking "obj->btf && obj->btf_ext"
         together for accessing ".BTF.ext".  Checking "obj->btf_ext"
         alone will do.
      
      6) Move "struct btf_sec_func_info" from btf.h to btf.c.
         There is no external usage outside btf.c.
      
      Fixes: 2993e051 ("tools/bpf: add support to read .BTF.ext sections")
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      f0187f0b
    • Martin KaFai Lau's avatar
      bpf: Add unit tests for bpf_line_info · 4d6304c7
      Martin KaFai Lau authored
      Add unit tests for bpf_line_info for both BPF_PROG_LOAD and
      BPF_OBJ_GET_INFO_BY_FD.
      
      jit enabled:
      [root@arch-fb-vm1 bpf]# ./test_btf -k 0
      BTF prog info raw test[5] (line_info (No subprog)): OK
      BTF prog info raw test[6] (line_info (No subprog. insn_off >= prog->len)): OK
      BTF prog info raw test[7] (line_info (No subprog. zero tailing line_info): OK
      BTF prog info raw test[8] (line_info (No subprog. nonzero tailing line_info)): OK
      BTF prog info raw test[9] (line_info (subprog)): OK
      BTF prog info raw test[10] (line_info (subprog + func_info)): OK
      BTF prog info raw test[11] (line_info (subprog. missing 1st func line info)): OK
      BTF prog info raw test[12] (line_info (subprog. missing 2nd func line info)): OK
      BTF prog info raw test[13] (line_info (subprog. unordered insn offset)): OK
      
      jit disabled:
      BTF prog info raw test[5] (line_info (No subprog)): not jited. skipping jited_line_info check. OK
      BTF prog info raw test[6] (line_info (No subprog. insn_off >= prog->len)): OK
      BTF prog info raw test[7] (line_info (No subprog. zero tailing line_info): not jited. skipping jited_line_info check. OK
      BTF prog info raw test[8] (line_info (No subprog. nonzero tailing line_info)): OK
      BTF prog info raw test[9] (line_info (subprog)): not jited. skipping jited_line_info check. OK
      BTF prog info raw test[10] (line_info (subprog + func_info)): not jited. skipping jited_line_info check. OK
      BTF prog info raw test[11] (line_info (subprog. missing 1st func line info)): OK
      BTF prog info raw test[12] (line_info (subprog. missing 2nd func line info)): OK
      BTF prog info raw test[13] (line_info (subprog. unordered insn offset)): OK
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      4d6304c7
    • Martin KaFai Lau's avatar
      bpf: Refactor and bug fix in test_func_type in test_btf.c · 05687352
      Martin KaFai Lau authored
      1) bpf_load_program_xattr() is absorbing the EBIG error
         which makes testing this case impossible.  It is replaced
         with a direct syscall(__NR_bpf, BPF_PROG_LOAD,...).
      2) The test_func_type() is renamed to test_info_raw() to
         prepare for the new line_info test in the next patch.
      3) The bpf_obj_get_info_by_fd() testing for func_info
         is refactored to test_get_finfo().  A new
         test_get_linfo() will be added in the next patch
         for testing line_info purpose.
      4) The test->func_info_cnt is checked instead of
         a static value "2".
      5) Remove unnecessary "\n" in error message.
      6) Adding back info_raw_test_num to the cmd arg such
         that a specific test case can be tested, like
         all other existing tests.
      
      7) Fix a bug in handling expected_prog_load_failure.
         A test could pass even if prog_fd != -1 while
         expected_prog_load_failure is true.
      8) The min rec_size check should be < 8 instead of < 4.
      
      Fixes: 4798c4ba ("tools/bpf: extends test_btf to test load/retrieve func_type info")
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      05687352
    • Martin KaFai Lau's avatar
      bpf: tools: Sync uapi bpf.h · ee491d8d
      Martin KaFai Lau authored
      Sync uapi bpf.h to tools/include/uapi/linux for
      the new bpf_line_info.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      ee491d8d
    • Martin KaFai Lau's avatar
      bpf: Add bpf_line_info support · c454a46b
      Martin KaFai Lau authored
      This patch adds bpf_line_info support.
      
      It accepts an array of bpf_line_info objects during BPF_PROG_LOAD.
      The "line_info", "line_info_cnt" and "line_info_rec_size" are added
      to the "union bpf_attr".  The "line_info_rec_size" makes
      bpf_line_info extensible in the future.
      
      The new "check_btf_line()" ensures the userspace line_info is valid
      for the kernel to use.
      
      When the verifier is translating/patching the bpf_prog (through
      "bpf_patch_insn_single()"), the line_infos' insn_off is also
      adjusted by the newly added "bpf_adj_linfo()".
      
      If the bpf_prog is jited, this patch also provides the jited addrs (in
      aux->jited_linfo) for the corresponding line_info.insn_off.
      "bpf_prog_fill_jited_linfo()" is added to fill the aux->jited_linfo.
      It is currently called by the x86 jit.  Other jits can also use
      "bpf_prog_fill_jited_linfo()" and it will be done in the followup patches.
      In the future, if it deemed necessary, a particular jit could also provide
      its own "bpf_prog_fill_jited_linfo()" implementation.
      
      A few "*line_info*" fields are added to the bpf_prog_info such
      that the user can get the xlated line_info back (i.e. the line_info
      with its insn_off reflecting the translated prog).  The jited_line_info
      is available if the prog is jited.  It is an array of __u64.
      If the prog is not jited, jited_line_info_cnt is 0.
      
      The verifier's verbose log with line_info will be done in
      a follow up patch.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      c454a46b
  2. 07 Dec, 2018 9 commits
  3. 06 Dec, 2018 5 commits
    • Alexei Starovoitov's avatar
      Merge branch 'bpf_func_info-improvements' · a06aef4e
      Alexei Starovoitov authored
      Martin KaFai Lau says:
      
      ====================
      The patchset has a few improvements on bpf_func_info:
      1. Improvements on the behaviors of info.func_info, info.func_info_cnt
         and info.func_info_rec_size.
      2. Name change: s/insn_offset/insn_off/
      
      Please see individual patch for details.
      ====================
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      a06aef4e
    • Martin KaFai Lau's avatar
      bpf: Expect !info.func_info and insn_off name changes in test_btf/libbpf/bpftool · 84ecc1f9
      Martin KaFai Lau authored
      Similar to info.jited_*, info.func_info could be 0 if
      bpf_dump_raw_ok() == false.
      
      This patch makes changes to test_btf and bpftool to expect info.func_info
      could be 0.
      
      This patch also makes the needed changes for s/insn_offset/insn_off/.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      84ecc1f9
    • Martin KaFai Lau's avatar
      bpf: tools: Sync uapi bpf.h for the name changes in bpf_func_info · 555249df
      Martin KaFai Lau authored
      This patch sync the name changes in bpf_func_info to
      the tools/.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      555249df
    • Martin KaFai Lau's avatar
      bpf: Change insn_offset to insn_off in bpf_func_info · d30d42e0
      Martin KaFai Lau authored
      The later patch will introduce "struct bpf_line_info" which
      has member "line_off" and "file_off" referring back to the
      string section in btf.  The line_"off" and file_"off"
      are more consistent to the naming convention in btf.h that
      means "offset" (e.g. name_off in "struct btf_type").
      
      The to-be-added "struct bpf_line_info" also has another
      member, "insn_off" which is the same as the "insn_offset"
      in "struct bpf_func_info".  Hence, this patch renames "insn_offset"
      to "insn_off" for "struct bpf_func_info".
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      d30d42e0
    • Martin KaFai Lau's avatar
      bpf: Improve the info.func_info and info.func_info_rec_size behavior · 7337224f
      Martin KaFai Lau authored
      1) When bpf_dump_raw_ok() == false and the kernel can provide >=1
         func_info to the userspace, the current behavior is setting
         the info.func_info_cnt to 0 instead of setting info.func_info
         to 0.
      
         It is different from the behavior in jited_func_lens/nr_jited_func_lens,
         jited_ksyms/nr_jited_ksyms...etc.
      
         This patch fixes it. (i.e. set func_info to 0 instead of
         func_info_cnt to 0 when bpf_dump_raw_ok() == false).
      
      2) When the userspace passed in info.func_info_cnt == 0, the kernel
         will set the expected func_info size back to the
         info.func_info_rec_size.  It is a way for the userspace to learn
         the kernel expected func_info_rec_size introduced in
         commit 838e9690 ("bpf: Introduce bpf_func_info").
      
         An exception is the kernel expected size is not set when
         func_info is not available for a bpf_prog.  This makes the
         returned info.func_info_rec_size has different values
         depending on the returned value of info.func_info_cnt.
      
         This patch sets the kernel expected size to info.func_info_rec_size
         independent of the info.func_info_cnt.
      
      3) The current logic only rejects invalid func_info_rec_size if
         func_info_cnt is non zero.  This patch also rejects invalid
         nonzero info.func_info_rec_size and not equal to the kernel
         expected size.
      
      4) Set info.btf_id as long as prog->aux->btf != NULL.  That will
         setup the later copy_to_user() codes look the same as others
         which then easier to understand and maintain.
      
         prog->aux->btf is not NULL only if prog->aux->func_info_cnt > 0.
      
         Breaking up info.btf_id from prog->aux->func_info_cnt is needed
         for the later line info patch anyway.
      
         A similar change is made to bpf_get_prog_name().
      
      Fixes: 838e9690 ("bpf: Introduce bpf_func_info")
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      7337224f
  4. 05 Dec, 2018 4 commits
    • Quentin Monnet's avatar
      tools: bpftool: add a command to dump the trace pipe · 30da46b5
      Quentin Monnet authored
      BPF programs can use the bpf_trace_printk() helper to print debug
      information into the trace pipe. Add a subcommand
      "bpftool prog tracelog" to simply dump this pipe to the console.
      
      This is for a good part copied from iproute2, where the feature is
      available with "tc exec bpf dbg". Changes include dumping pipe content
      to stdout instead of stderr and adding JSON support (content is dumped
      as an array of strings, one per line read from the pipe). This version
      is dual-licensed, with Daniel's permission.
      
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Suggested-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Signed-off-by: default avatarQuentin Monnet <quentin.monnet@netronome.com>
      Reviewed-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      30da46b5
    • Daniel Borkmann's avatar
      Merge branch 'bpf-jit-overridable-alloc' · 41888179
      Daniel Borkmann authored
      Ard Biesheuvel says:
      
      ====================
      On arm64, modules are allocated from a 128 MB window which is close to
      the core kernel, so that relative direct branches are guaranteed to be
      in range (except in some KASLR configurations). Also, module_alloc()
      is in charge of allocating KASAN shadow memory when running with KASAN
      enabled.
      
      This means that the way BPF reuses module_alloc()/module_memfree() is
      undesirable on arm64 (and potentially other architectures as well),
      and so this series refactors BPF's use of those functions to permit
      architectures to change this behavior.
      
      Patch #1 breaks out the module_alloc() and module_memfree() calls into
      __weak functions so they can be overridden.
      
      Patch #2 implements the new alloc/free overrides for arm64
      
      Changes since v3:
      - drop 'const' modifier for free() hook void* argument
      - move the dedicated BPF region to before the module region, putting it
        within 4GB of the module and kernel regions on non-KASLR kernels
      
      Changes since v2:
      - properly build time and runtime tested this time (log after the diffstat)
      - create a dedicated 128 MB region at the top of the vmalloc space for BPF
        programs, ensuring that the programs will be in branching range of each
        other (which we currently rely upon) but at an arbitrary distance from
        the kernel and modules (which we don't care about)
      
      Changes since v1:
      - Drop misguided attempt to 'fix' and refactor the free path. Instead,
        just add another __weak wrapper for the invocation of module_memfree()
      ====================
      
      Cc: Daniel Borkmann <daniel@iogearbox.net>
      Cc: Alexei Starovoitov <ast@kernel.org>
      Cc: Rick Edgecombe <rick.p.edgecombe@intel.com>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Jann Horn <jannh@google.com>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: Jessica Yu <jeyu@kernel.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Catalin Marinas <catalin.marinas@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: "David S. Miller" <davem@davemloft.net>
      Cc: linux-arm-kernel@lists.infradead.org
      Cc: linux-kernel@vger.kernel.org
      Cc: netdev@vger.kernel.org
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      41888179
    • Ard Biesheuvel's avatar
      arm64/bpf: don't allocate BPF JIT programs in module memory · 91fc957c
      Ard Biesheuvel authored
      The arm64 module region is a 128 MB region that is kept close to
      the core kernel, in order to ensure that relative branches are
      always in range. So using the same region for programs that do
      not have this restriction is wasteful, and preferably avoided.
      
      Now that the core BPF JIT code permits the alloc/free routines to
      be overridden, implement them by vmalloc()/vfree() calls from a
      dedicated 128 MB region set aside for BPF programs. This ensures
      that BPF programs are still in branching range of each other, which
      is something the JIT currently depends upon (and is not guaranteed
      when using module_alloc() on KASLR kernels like we do currently).
      It also ensures that placement of BPF programs does not correlate
      with the placement of the core kernel or modules, making it less
      likely that leaking the former will reveal the latter.
      
      This also solves an issue under KASAN, where shadow memory is
      needlessly allocated for all BPF programs (which don't require KASAN
      shadow pages since they are not KASAN instrumented)
      Signed-off-by: default avatarArd Biesheuvel <ard.biesheuvel@linaro.org>
      Acked-by: default avatarWill Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      91fc957c
    • Ard Biesheuvel's avatar
      bpf: add __weak hook for allocating executable memory · dc002bb6
      Ard Biesheuvel authored
      By default, BPF uses module_alloc() to allocate executable memory,
      but this is not necessary on all arches and potentially undesirable
      on some of them.
      
      So break out the module_alloc() and module_memfree() calls into __weak
      functions to allow them to be overridden in arch code.
      Signed-off-by: default avatarArd Biesheuvel <ard.biesheuvel@linaro.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      dc002bb6
  5. 04 Dec, 2018 5 commits
  6. 03 Dec, 2018 4 commits
  7. 02 Dec, 2018 1 commit
  8. 01 Dec, 2018 6 commits
    • Alexei Starovoitov's avatar
      Merge branch 'xdp1-improvements' · 71fc156f
      Alexei Starovoitov authored
      Matteo Croce says:
      
      ====================
      Small improvements to improve the readability and easiness
      to use of the xdp1 sample.
      ====================
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      71fc156f
    • Matteo Croce's avatar
      samples: bpf: get ifindex from ifname · dc378a1a
      Matteo Croce authored
      Find the ifindex with if_nametoindex() instead of requiring the
      numeric ifindex.
      Signed-off-by: default avatarMatteo Croce <mcroce@redhat.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      dc378a1a
    • Matteo Croce's avatar
      samples: bpf: improve xdp1 example · d606ee5c
      Matteo Croce authored
      Store only the total packet count for every protocol, instead of the
      whole per-cpu array.
      Use bpf_map_get_next_key() to iterate the map, instead of looking up
      all the protocols.
      Signed-off-by: default avatarMatteo Croce <mcroce@redhat.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      d606ee5c
    • Alexei Starovoitov's avatar
      Merge branch 'improve-test-coverage-sparc' · 9ffd05d9
      Alexei Starovoitov authored
      David Miller says:
      
      ====================
      On sparc64 a ton of test cases in test_verifier.c fail because
      the memory accesses in the test case are unaligned (or cannot
      be proven to be aligned by the verifier).
      
      Perhaps we can eventually try to (carefully) modify each test case
      which has this problem to not use unaligned accesses but:
      
      1) That is delicate work.
      
      2) The changes might not fully respect the original
         intention of the testcase.
      
      3) In some cases, such a transformation might not even
         be feasible at all.
      
      So add an "any alignment" flag to tell the verifier to forcefully
      disable it's alignment checks completely.
      
      test_verifier.c is then annotated to use this flag when necessary.
      
      The presence of the flag in each test case is good documentation to
      anyone who wants to actually tackle the job of eliminating the
      unaligned memory accesses in the test cases.
      
      I've also seen several weird things in test cases, like trying to
      access __skb->mark in a packet buffer.
      
      This gets rid of 104 test_verifier.c failures on sparc64.
      
      Changes since v1:
      
      1) Explain the new BPF_PROG_LOAD flag in easier to understand terms.
         Suggested by Alexei.
      
      2) Make bpf_verify_program() just take a __u32 prog_flags instead of
         just accumulating boolean arguments over and over.  Also suggested
         by Alexei.
      
      Changes since RFC:
      
      1) Only the admin can allow the relaxation of alignment restrictions
         on inefficient unaligned access architectures.
      
      2) Use F_NEEDS_EFFICIENT_UNALIGNED_ACCESS instead of making a new
         flag.
      
      3) Annotate in the output, when we have a test case that the verifier
         accepted but we did not try to execute because we are on an
         inefficient unaligned access platform.  Maybe with some arch
         machinery we can avoid this in the future.
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      ====================
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      9ffd05d9
    • David Miller's avatar
      bpf: Apply F_NEEDS_EFFICIENT_UNALIGNED_ACCESS to more ACCEPT test cases. · 0a686324
      David Miller authored
      If a testcase has alignment problems but is expected to be ACCEPT,
      verify it using F_NEEDS_EFFICIENT_UNALIGNED_ACCESS too.
      
      Maybe in the future if we add some architecture specific code to elide
      the unaligned memory access warnings during the test, we can execute
      these as well.
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      0a686324
    • David Miller's avatar
      bpf: Make more use of 'any' alignment in test_verifier.c · 2acc5fd5
      David Miller authored
      Use F_NEEDS_EFFICIENT_UNALIGNED_ACCESS in more tests where the
      expected result is REJECT.
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      2acc5fd5