1. 24 Aug, 2020 7 commits
    • Martin KaFai Lau's avatar
      bpf: tcp: Add bpf_skops_hdr_opt_len() and bpf_skops_write_hdr_opt() · 331fca43
      Martin KaFai Lau authored
      The bpf prog needs to parse the SYN header to learn what options have
      been sent by the peer's bpf-prog before writing its options into SYNACK.
      This patch adds a "syn_skb" arg to tcp_make_synack() and send_synack().
      This syn_skb will eventually be made available (as read-only) to the
      bpf prog.  This will be the only SYN packet available to the bpf
      prog during syncookie.  For other regular cases, the bpf prog can
      also use the saved_syn.
      
      When writing options, the bpf prog will first be called to tell the
      kernel its required number of bytes.  It is done by the new
      bpf_skops_hdr_opt_len().  The bpf prog will only be called when the new
      BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG is set in tp->bpf_sock_ops_cb_flags.
      When the bpf prog returns, the kernel will know how many bytes are needed
      and then update the "*remaining" arg accordingly.  4 byte alignment will
      be included in the "*remaining" before this function returns.  The 4 byte
      aligned number of bytes will also be stored into the opts->bpf_opt_len.
      "bpf_opt_len" is a newly added member to the struct tcp_out_options.
      
      Then the new bpf_skops_write_hdr_opt() will call the bpf prog to write the
      header options.  The bpf prog is only called if it has reserved spaces
      before (opts->bpf_opt_len > 0).
      
      The bpf prog is the last one getting a chance to reserve header space
      and writing the header option.
      
      These two functions are half implemented to highlight the changes in
      TCP stack.  The actual codes preparing the bpf running context and
      invoking the bpf prog will be added in the later patch with other
      necessary bpf pieces.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Reviewed-by: default avatarEric Dumazet <edumazet@google.com>
      Link: https://lore.kernel.org/bpf/20200820190052.2885316-1-kafai@fb.com
      331fca43
    • Martin KaFai Lau's avatar
      bpf: tcp: Add bpf_skops_parse_hdr() · 00d211a4
      Martin KaFai Lau authored
      The patch adds a function bpf_skops_parse_hdr().
      It will call the bpf prog to parse the TCP header received at
      a tcp_sock that has at least reached the ESTABLISHED state.
      
      For the packets received during the 3WHS (SYN, SYNACK and ACK),
      the received skb will be available to the bpf prog during the callback
      in bpf_skops_established() introduced in the previous patch and
      in the bpf_skops_write_hdr_opt() that will be added in the
      next patch.
      
      Calling bpf prog to parse header is controlled by two new flags in
      tp->bpf_sock_ops_cb_flags:
      BPF_SOCK_OPS_PARSE_UNKNOWN_HDR_OPT_CB_FLAG and
      BPF_SOCK_OPS_PARSE_ALL_HDR_OPT_CB_FLAG.
      
      When BPF_SOCK_OPS_PARSE_UNKNOWN_HDR_OPT_CB_FLAG is set,
      the bpf prog will only be called when there is unknown
      option in the TCP header.
      
      When BPF_SOCK_OPS_PARSE_ALL_HDR_OPT_CB_FLAG is set,
      the bpf prog will be called on all received TCP header.
      
      This function is half implemented to highlight the changes in
      TCP stack.  The actual codes preparing the bpf running context and
      invoking the bpf prog will be added in the later patch with other
      necessary bpf pieces.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Reviewed-by: default avatarEric Dumazet <edumazet@google.com>
      Link: https://lore.kernel.org/bpf/20200820190046.2885054-1-kafai@fb.com
      00d211a4
    • Martin KaFai Lau's avatar
      bpf: tcp: Add bpf_skops_established() · 72be0fe6
      Martin KaFai Lau authored
      In tcp_init_transfer(), it currently calls the bpf prog to give it a
      chance to handle the just "ESTABLISHED" event (e.g. do setsockopt
      on the newly established sk).  Right now, it is done by calling the
      general purpose tcp_call_bpf().
      
      In the later patch, it also needs to pass the just-received skb which
      concludes the 3 way handshake. E.g. the SYNACK received at the active side.
      The bpf prog can then learn some specific header options written by the
      peer's bpf-prog and potentially do setsockopt on the newly established sk.
      Thus, instead of reusing the general purpose tcp_call_bpf(), a new function
      bpf_skops_established() is added to allow passing the "skb" to the bpf
      prog.  The actual skb passing from bpf_skops_established() to the bpf prog
      will happen together in a later patch which has the necessary bpf pieces.
      
      A "skb" arg is also added to tcp_init_transfer() such that
      it can then be passed to bpf_skops_established().
      
      Calling the new bpf_skops_established() instead of tcp_call_bpf()
      should be a noop in this patch.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
      Link: https://lore.kernel.org/bpf/20200820190039.2884750-1-kafai@fb.com
      72be0fe6
    • Martin KaFai Lau's avatar
      tcp: Add saw_unknown to struct tcp_options_received · 7656d684
      Martin KaFai Lau authored
      In a later patch, the bpf prog only wants to be called to handle
      a header option if that particular header option cannot be handled by
      the kernel.  This unknown option could be written by the peer's bpf-prog.
      It could also be a new standard option that the running kernel does not
      support it while a bpf-prog can handle it.
      
      This patch adds a "saw_unknown" bit to "struct tcp_options_received"
      and it uses an existing one byte hole to do that.  "saw_unknown" will
      be set in tcp_parse_options() if it sees an option that the kernel
      cannot handle.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Reviewed-by: default avatarEric Dumazet <edumazet@google.com>
      Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
      Link: https://lore.kernel.org/bpf/20200820190033.2884430-1-kafai@fb.com
      7656d684
    • Martin KaFai Lau's avatar
      tcp: bpf: Add TCP_BPF_RTO_MIN for bpf_setsockopt · ca584ba0
      Martin KaFai Lau authored
      This patch adds bpf_setsockopt(TCP_BPF_RTO_MIN) to allow bpf prog
      to set the min rto of a connection.  It could be used together
      with the earlier patch which has added bpf_setsockopt(TCP_BPF_DELACK_MAX).
      
      A later selftest patch will communicate the max delay ack in a
      bpf tcp header option and then the receiving side can use
      bpf_setsockopt(TCP_BPF_RTO_MIN) to set a shorter rto.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Reviewed-by: default avatarEric Dumazet <edumazet@google.com>
      Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
      Link: https://lore.kernel.org/bpf/20200820190027.2884170-1-kafai@fb.com
      ca584ba0
    • Martin KaFai Lau's avatar
      tcp: bpf: Add TCP_BPF_DELACK_MAX setsockopt · 2b8ee4f0
      Martin KaFai Lau authored
      This change is mostly from an internal patch and adapts it from sysctl
      config to the bpf_setsockopt setup.
      
      The bpf_prog can set the max delay ack by using
      bpf_setsockopt(TCP_BPF_DELACK_MAX).  This max delay ack can be communicated
      to its peer through bpf header option.  The receiving peer can then use
      this max delay ack and set a potentially lower rto by using
      bpf_setsockopt(TCP_BPF_RTO_MIN) which will be introduced
      in the next patch.
      
      Another later selftest patch will also use it like the above to show
      how to write and parse bpf tcp header option.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Reviewed-by: default avatarEric Dumazet <edumazet@google.com>
      Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
      Link: https://lore.kernel.org/bpf/20200820190021.2884000-1-kafai@fb.com
      2b8ee4f0
    • Martin KaFai Lau's avatar
      tcp: Use a struct to represent a saved_syn · 70a217f1
      Martin KaFai Lau authored
      The TCP_SAVE_SYN has both the network header and tcp header.
      The total length of the saved syn packet is currently stored in
      the first 4 bytes (u32) of an array and the actual packet data is
      stored after that.
      
      A later patch will add a bpf helper that allows to get the tcp header
      alone from the saved syn without the network header.  It will be more
      convenient to have a direct offset to a specific header instead of
      re-parsing it.  This requires to separately store the network hdrlen.
      The total header length (i.e. network + tcp) is still needed for the
      current usage in getsockopt.  Although this total length can be obtained
      by looking into the tcphdr and then get the (th->doff << 2), this patch
      chooses to directly store the tcp hdrlen in the second four bytes of
      this newly created "struct saved_syn".  By using a new struct, it can
      give a readable name to each individual header length.
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Reviewed-by: default avatarEric Dumazet <edumazet@google.com>
      Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
      Link: https://lore.kernel.org/bpf/20200820190014.2883694-1-kafai@fb.com
      70a217f1
  2. 21 Aug, 2020 17 commits
  3. 20 Aug, 2020 10 commits
    • Andrii Nakryiko's avatar
      selftests/bpf: List newest Clang built-ins needed for some CO-RE selftests · 149cb339
      Andrii Nakryiko authored
      Record which built-ins are optional and needed for some of recent BPF CO-RE
      subtests. Document Clang diff that fixed corner-case issue with
      __builtin_btf_type_id().
      Suggested-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20200820061411.1755905-4-andriin@fb.com
      149cb339
    • Andrii Nakryiko's avatar
      selftests/bpf: Fix two minor compilation warnings reported by GCC 4.9 · 37a6a9e7
      Andrii Nakryiko authored
      GCC 4.9 seems to be more strict in some regards. Fix two minor issue it
      reported.
      
      Fixes: 1c1052e0 ("tools/testing/selftests/bpf: Add self-tests for new helper bpf_get_ns_current_pid_tgid.")
      Fixes: 2d7824ff ("selftests: bpf: Add test for sk_assign")
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20200820061411.1755905-3-andriin@fb.com
      37a6a9e7
    • Andrii Nakryiko's avatar
      libbpf: Fix libbpf build on compilers missing __builtin_mul_overflow · dda1ec9f
      Andrii Nakryiko authored
      GCC compilers older than version 5 don't support __builtin_mul_overflow yet.
      Given GCC 4.9 is the minimal supported compiler for building kernel and the
      fact that libbpf is a dependency of resolve_btfids, which is dependency of
      CONFIG_DEBUG_INFO_BTF=y, this needs to be handled. This patch fixes the issue
      by falling back to slower detection of integer overflow in such cases.
      
      Fixes: 029258d7 ("libbpf: Remove any use of reallocarray() in libbpf")
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20200820061411.1755905-2-andriin@fb.com
      dda1ec9f
    • Andrii Nakryiko's avatar
      libbpf: Fix detection of BPF helper call instruction · 9b2f6fec
      Andrii Nakryiko authored
      BPF_CALL | BPF_JMP32 is explicitly not allowed by verifier for BPF helper
      calls, so don't detect it as a valid call. Also drop the check on func_id
      pointer, as it's currently always non-null.
      
      Fixes: 109cea5a ("libbpf: Sanitize BPF program code for bpf_probe_read_{kernel, user}[_str]")
      Reported-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20200820061411.1755905-1-andriin@fb.com
      9b2f6fec
    • Daniel Borkmann's avatar
      Merge branch 'bpf-umd-debug' · 0bc23a1d
      Daniel Borkmann authored
      Alexei Starovoitov says:
      
      ====================
      This patch set is the first real user of user mode driver facility. The
      general use case for user mode driver is to ship vmlinux with preloaded BPF
      programs. In this particular case the user mode driver populates bpffs instance
      with two BPF iterators. In several months BPF_LSM project would need to preload
      the kernel with its own set of BPF programs and attach to LSM hooks instead of
      bpffs. BPF iterators and BPF_LSM are unstable from uapi perspective. They are
      tracing based and peek into arbitrary kernel data structures. One can question
      why a kernel module cannot embed BPF programs inside. The reason is that libbpf
      is necessary to load them. First libbpf loads BPF Type Format, then creates BPF
      maps, populates them. Then it relocates code sections inside BPF programs,
      loads BPF programs, and finally attaches them to events. Theoretically libbpf
      can be rewritten to work in the kernel, but that is massive undertaking. The
      maintenance of in-kernel libbpf and user space libbpf would be another
      challenge. Another obstacle to embedding BPF programs into kernel module is
      sys_bpf api. Loading of programs, BTF, maps goes through the verifier. It
      validates and optimizes the code. It's possible to provide in-kernel api to all
      of sys_bpf commands (load progs, create maps, update maps, load BTF, etc), but
      that is huge amount of work and forever maintenance headache.
      Hence the decision is to ship vmlinux with user mode drivers that load
      BPF programs. Just like kernel modules extend vmlinux BPF programs
      are safe extensions of the kernel and some of them need to ship with vmlinux.
      
      This patch set adds a kernel module with user mode driver that populates bpffs
      with two BPF iterators.
      
      $ mount bpffs /my/bpffs/ -t bpf
      $ ls -la /my/bpffs/
      total 4
      drwxrwxrwt  2 root root    0 Jul  2 00:27 .
      drwxr-xr-x 19 root root 4096 Jul  2 00:09 ..
      -rw-------  1 root root    0 Jul  2 00:27 maps.debug
      -rw-------  1 root root    0 Jul  2 00:27 progs.debug
      
      The user mode driver will load BPF Type Formats, create BPF maps, populate BPF
      maps, load two BPF programs, attach them to BPF iterators, and finally send two
      bpf_link IDs back to the kernel.
      The kernel will pin two bpf_links into newly mounted bpffs instance under
      names "progs.debug" and "maps.debug". These two files become human readable.
      
      $ cat /my/bpffs/progs.debug
        id name            attached
        11 dump_bpf_map    bpf_iter_bpf_map
        12 dump_bpf_prog   bpf_iter_bpf_prog
        27 test_pkt_access
        32 test_main       test_pkt_access test_pkt_access
        33 test_subprog1   test_pkt_access_subprog1 test_pkt_access
        34 test_subprog2   test_pkt_access_subprog2 test_pkt_access
        35 test_subprog3   test_pkt_access_subprog3 test_pkt_access
        36 new_get_skb_len get_skb_len test_pkt_access
        37 new_get_skb_ifindex get_skb_ifindex test_pkt_access
        38 new_get_constant get_constant test_pkt_access
      
      The BPF program dump_bpf_prog() in iterators.bpf.c is printing this data about
      all BPF programs currently loaded in the system. This information is unstable
      and will change from kernel to kernel.
      
      In some sence this output is similar to 'bpftool prog show' that is using
      stable api to retreive information about BPF programs. The BPF subsytems grows
      quickly and there is always demand to show as much info about BPF things as
      possible. But we cannot expose all that info via stable uapi of bpf syscall,
      since the details change so much. Right now a BPF program can be attached to
      only one other BPF program. Folks are working on patches to enable
      multi-attach, but for debugging it's necessary to see the current state. There
      is no uapi for that, but above output shows it:
        37 new_get_skb_ifindex  get_skb_ifindex test_pkt_access
        38 new_get_constant     get_constant    test_pkt_access
           [1]                  [2]             [3]
      [1] is the full name of BPF prog from BTF.
      [2] is the name of function inside target BPF prog.
      [3] is the name of target BPF prog.
      
      [2] and [3] are not exposed via uapi, since they will change from single to
      multi soon. There are many other cases where bpf internals are useful for
      debugging, but shouldn't be exposed via uapi due to high rate of changes.
      
      systemd mounts /sys/fs/bpf at the start, so this kernel module with user mode
      driver needs to be available early. BPF_LSM most likely would need to preload
      BPF programs even earlier.
      
      Few interesting observations:
      - though bpffs comes with two human readble files "progs.debug" and
        "maps.debug" they can be removed. 'rm -f /sys/fs/bpf/progs.debug' will remove
        bpf_link and kernel will automatically unload corresponding BPF progs, maps,
        BTFs. In the future '-o remount' will be able to restore them. This is not
        implemented yet.
      
      - 'ps aux|grep bpf_preload' shows nothing. User mode driver loaded BPF
        iterators and exited. Nothing is lingering in user space at this point.
      
      - We can consider giving 0644 permissions to "progs.debug" and "maps.debug"
        to allow unprivileged users see BPF things loaded in the system.
        We cannot do so with "bpftool prog show", since it's using cap_sys_admin
        parts of bpf syscall.
      
      - The functionality split between core kernel, bpf_preload kernel module and
        user mode driver is very similar to bpfilter style of interaction.
      
      - Similar BPF iterators can be used as unstable extensions to /proc.
        Like mounting /proc can prepopolate some subdirectory in there with
        a BPF iterator that will print QUIC sockets instead of tcp and udp.
      
      Changelog:
      
      v5->v6:
      - refactored Makefiles with Andrii's help
        - switched to explicit $(MAKE) style
        - switched to userldlibs instead of userldflags
        - fixed build issue with libbpf Makefile due to invocation from kbuild
      - fixed menuconfig order as spotted by Daniel
      - introduced CONFIG_USERMODE_DRIVER bool that is selected by bpfilter and bpf_preload
      
      v4->v5:
      - addressed Song and Andrii feedback. s/pages/max_entries/
      
      v3->v4:
      - took THIS_MODULE in patch 3 as suggested by Daniel to simplify the code.
      - converted BPF iterator to use BTF (when available) to print full BPF program name
      instead of 16-byte truncated version.
      This is something I've been using drgn scripts for.
      Take a look at get_name() in iterators.bpf.c to see how short it is comparing
      to what user space bpftool would have to do to print the same full name:
      . get prog info via obj_info_by_fd
      . do get_fd_by_id from info->btf_id
      . fetch potentially large BTF of the program from the kernel
      . parse that BTF in user space to figure out all type boundaries and string section
      . read info->func_info to get btf_id of func_proto from there
      . find that btf_id in the parsed BTF
      That's quite a bit work for bpftool comparing to few lines in get_name().
      I guess would be good to make bpftool do this info extraction anyway.
      While doing this BTF reading in the kernel realized that the verifier is not smart
      enough to follow double pointers (added to my todo list), otherwise get_name()
      would have been even shorter.
      
      v2->v3:
      - fixed module unload race (Daniel)
      - added selftest (Daniel)
      - fixed build bot warning
      
      v1->v2:
      - changed names to 'progs.debug' and 'maps.debug' to hopefully better indicate
        instability of the text output. Having dot in the name also guarantees
        that these special files will not conflict with normal bpf objects pinned
        in bpffs, since dot is disallowed for normal pins.
      - instead of hard coding link_name in the core bpf moved into UMD.
      - cleanedup error handling.
      - addressed review comments from Yonghong and Andrii.
      ====================
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      0bc23a1d
    • Alexei Starovoitov's avatar
      selftests/bpf: Add bpffs preload test. · edb65ee5
      Alexei Starovoitov authored
      Add a test that mounts two bpffs instances and checks progs.debug
      and maps.debug for sanity data.
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Link: https://lore.kernel.org/bpf/20200819042759.51280-5-alexei.starovoitov@gmail.com
      edb65ee5
    • Alexei Starovoitov's avatar
      bpf: Add kernel module with user mode driver that populates bpffs. · d71fa5c9
      Alexei Starovoitov authored
      Add kernel module with user mode driver that populates bpffs with
      BPF iterators.
      
      $ mount bpffs /my/bpffs/ -t bpf
      $ ls -la /my/bpffs/
      total 4
      drwxrwxrwt  2 root root    0 Jul  2 00:27 .
      drwxr-xr-x 19 root root 4096 Jul  2 00:09 ..
      -rw-------  1 root root    0 Jul  2 00:27 maps.debug
      -rw-------  1 root root    0 Jul  2 00:27 progs.debug
      
      The user mode driver will load BPF Type Formats, create BPF maps, populate BPF
      maps, load two BPF programs, attach them to BPF iterators, and finally send two
      bpf_link IDs back to the kernel.
      The kernel will pin two bpf_links into newly mounted bpffs instance under
      names "progs.debug" and "maps.debug". These two files become human readable.
      
      $ cat /my/bpffs/progs.debug
        id name            attached
        11 dump_bpf_map    bpf_iter_bpf_map
        12 dump_bpf_prog   bpf_iter_bpf_prog
        27 test_pkt_access
        32 test_main       test_pkt_access test_pkt_access
        33 test_subprog1   test_pkt_access_subprog1 test_pkt_access
        34 test_subprog2   test_pkt_access_subprog2 test_pkt_access
        35 test_subprog3   test_pkt_access_subprog3 test_pkt_access
        36 new_get_skb_len get_skb_len test_pkt_access
        37 new_get_skb_ifindex get_skb_ifindex test_pkt_access
        38 new_get_constant get_constant test_pkt_access
      
      The BPF program dump_bpf_prog() in iterators.bpf.c is printing this data about
      all BPF programs currently loaded in the system. This information is unstable
      and will change from kernel to kernel as ".debug" suffix conveys.
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Link: https://lore.kernel.org/bpf/20200819042759.51280-4-alexei.starovoitov@gmail.com
      d71fa5c9
    • Alexei Starovoitov's avatar
      bpf: Add BPF program and map iterators as built-in BPF programs. · f0fdfefb
      Alexei Starovoitov authored
      The program and map iterators work similar to seq_file-s.
      Once the program is pinned in bpffs it can be read with "cat" tool
      to print human readable output. In this case about BPF programs and maps.
      For example:
      $ cat /sys/fs/bpf/progs.debug
        id name            attached
         5 dump_bpf_map    bpf_iter_bpf_map
         6 dump_bpf_prog   bpf_iter_bpf_prog
      $ cat /sys/fs/bpf/maps.debug
        id name            max_entries
         3 iterator.rodata     1
      
      To avoid kernel build dependency on clang 10 separate bpf skeleton generation
      into manual "make" step and instead check-in generated .skel.h into git.
      
      Unlike 'bpftool prog show' in-kernel BTF name is used (when available)
      to print full name of BPF program instead of 16-byte truncated name.
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Link: https://lore.kernel.org/bpf/20200819042759.51280-3-alexei.starovoitov@gmail.com
      f0fdfefb
    • Alexei Starovoitov's avatar
      bpf: Factor out bpf_link_by_id() helper. · 005142b8
      Alexei Starovoitov authored
      Refactor the code a bit to extract bpf_link_by_id() helper.
      It's similar to existing bpf_prog_by_id().
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      Acked-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Link: https://lore.kernel.org/bpf/20200819042759.51280-2-alexei.starovoitov@gmail.com
      005142b8
    • Xu Wang's avatar
  4. 19 Aug, 2020 6 commits
    • Alexei Starovoitov's avatar
      Merge branch 'type-and-enum-value-relos' · c1447efd
      Alexei Starovoitov authored
      Andrii Nakryiko says:
      
      ====================
      This patch set adds libbpf support for two new classes of CO-RE relocations:
      type-based (TYPE_EXISTS/TYPE_SIZE/TYPE_ID_LOCAL/TYPE_ID_TARGET) and enum
      value-vased (ENUMVAL_EXISTS/ENUMVAL_VALUE):
        - TYPE_EXISTS allows to detect presence in kernel BTF of a locally-recorded
          BTF type. Useful for feature detection (new functionality often comes with
          new internal kernel types), as well as handling type renames and bigger
          refactorings.
        - TYPE_SIZE allows to get the real size (in bytes) of a specified kernel
          type. Useful for dumping internal structure as-is through perfbuf or
          ringbuf.
        - TYPE_ID_LOCAL/TYPE_ID_TARGET allow to capture BTF type ID of a BTF type in
          program's BTF or kernel BTF, respectively. These could be used for
          high-performance and space-efficient generic data dumping/logging by
          relying on small and cheap BTF type ID as a data layout descriptor, for
          post-processing on user-space side.
        - ENUMVAL_EXISTS can be used for detecting the presence of enumerator value
          in kernel's enum type. Most direct application is to detect BPF helper
          support in kernel.
        - ENUMVAL_VALUE allows to relocate real integer value of kernel enumerator
          value, which is subject to change (e.g., always a potential issue for
          internal, non-UAPI, kernel enums).
      
      I've indicated potential applications for these relocations, but relocations
      themselves are generic and unassuming and are designed to work correctly even
      in unintended applications. Furthermore, relocated values become constants,
      known to the verifier and could and would be used for dead branch code
      detection and elimination. This makes them ideal to do all sorts of feature
      detection and guarding functionality that's not available on some older (but
      still supported by BPF program) kernels, while having to compile and maintain
      one unified source code.
      
      Selftests are added for all the new features. Selftests utilizing new Clang
      built-ins are designed such that they will compile with older Clangs and will
      be skipped during test runs. So this shouldn't cause any build and test
      failures on systems with slightly outdated Clang compiler.
      
      LLVM patches adding these relocation in Clang:
        - __builtin_btf_type_id() ([0], [1], [2]);
        - __builtin_preserve_type_info(), __builtin_preserve_enum_value() ([3], [4]).
      
        [0] https://reviews.llvm.org/D74572
        [1] https://reviews.llvm.org/D74668
        [2] https://reviews.llvm.org/D85174
        [3] https://reviews.llvm.org/D83878
        [4] https://reviews.llvm.org/D83242
      
      v2->v3:
        - fix feature detection for __builtin_btf_type_id() test (Yonghong);
        - fix extra empty lines at the end of files (Yonghong);
      
      v1->v2:
        - selftests detect built-in support and are skipped if not found (Alexei).
      ====================
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      c1447efd
    • Andrii Nakryiko's avatar
      selftests/bpf: Add tests for ENUMVAL_EXISTS/ENUMVAL_VALUE relocations · 33574905
      Andrii Nakryiko authored
      Add tests validating existence and value relocations for enum value-based
      relocations. If __builtin_preserve_enum_value() built-in is not supported,
      skip tests.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20200819194519.3375898-6-andriin@fb.com
      33574905
    • Andrii Nakryiko's avatar
      libbpf: Implement enum value-based CO-RE relocations · eacaaed7
      Andrii Nakryiko authored
      Implement two relocations of a new enumerator value-based CO-RE relocation
      kind: ENUMVAL_EXISTS and ENUMVAL_VALUE.
      
      First, ENUMVAL_EXISTS, allows to detect the presence of a named enumerator
      value in the target (kernel) BTF. This is useful to do BPF helper/map/program
      type support detection from BPF program side. bpf_core_enum_value_exists()
      macro helper is provided to simplify built-in usage.
      
      Second, ENUMVAL_VALUE, allows to capture enumerator integer value and relocate
      it according to the target BTF, if it changes. This is useful to have
      a guarantee against intentional or accidental re-ordering/re-numbering of some
      of the internal (non-UAPI) enumerations, where kernel developers don't care
      about UAPI backwards compatiblity concerns. bpf_core_enum_value() allows to
      capture this succinctly and use correct enum values in code.
      
      LLVM uses ldimm64 instruction to capture enumerator value-based relocations,
      so add support for ldimm64 instruction patching as well.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20200819194519.3375898-5-andriin@fb.com
      eacaaed7
    • Andrii Nakryiko's avatar
      selftests/bpf: Add CO-RE relo test for TYPE_ID_LOCAL/TYPE_ID_TARGET · 4836bf5e
      Andrii Nakryiko authored
      Add tests for BTF type ID relocations. To allow testing this, enhance
      core_relo.c test runner to allow dynamic initialization of test inputs.
      If Clang doesn't have necessary support for new functionality, test is
      skipped.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20200819194519.3375898-4-andriin@fb.com
      4836bf5e
    • Andrii Nakryiko's avatar
      selftests/bpf: Test TYPE_EXISTS and TYPE_SIZE CO-RE relocations · 124a892d
      Andrii Nakryiko authored
      Add selftests for TYPE_EXISTS and TYPE_SIZE relocations, testing correctness
      of relocations and handling of type compatiblity/incompatibility.
      
      If __builtin_preserve_type_info() is not supported by compiler, skip tests.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20200819194519.3375898-3-andriin@fb.com
      124a892d
    • Andrii Nakryiko's avatar
      libbpf: Implement type-based CO-RE relocations support · 3fc32f40
      Andrii Nakryiko authored
      Implement support for TYPE_EXISTS/TYPE_SIZE/TYPE_ID_LOCAL/TYPE_ID_REMOTE
      relocations. These are examples of type-based relocations, as opposed to
      field-based relocations supported already. The difference is that they are
      calculating relocation values based on the type itself, not a field within
      a struct/union.
      
      Type-based relos have slightly different semantics when matching local types
      to kernel target types, see comments in bpf_core_types_are_compat() for
      details. Their behavior on failure to find target type in kernel BTF also
      differs. Instead of "poisoning" relocatable instruction and failing load
      subsequently in kernel, they return 0 (which is rarely a valid return result,
      so user BPF code can use that to detect success/failure of the relocation and
      deal with it without extra "guarding" relocations). Also, it's always possible
      to check existence of the type in target kernel with TYPE_EXISTS relocation,
      similarly to a field-based FIELD_EXISTS.
      
      TYPE_ID_LOCAL relocation is a bit special in that it always succeeds (barring
      any libbpf/Clang bugs) and resolved to BTF ID using **local** BTF info of BPF
      program itself. Tests in subsequent patches demonstrate the usage and
      semantics of new relocations.
      Signed-off-by: default avatarAndrii Nakryiko <andriin@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20200819194519.3375898-2-andriin@fb.com
      3fc32f40