1. 25 Sep, 2018 1 commit
  2. 21 Sep, 2018 2 commits
  3. 19 Sep, 2018 1 commit
    • Willem de Bruijn's avatar
      flow_dissector: fix build failure without CONFIG_NET · 2dfd184a
      Willem de Bruijn authored
      If boolean CONFIG_BPF_SYSCALL is enabled, kernel/bpf/syscall.c will
      call flow_dissector functions from net/core/flow_dissector.c.
      
      This causes this build failure if CONFIG_NET is disabled:
      
          kernel/bpf/syscall.o: In function `__x64_sys_bpf':
          syscall.c:(.text+0x3278): undefined reference to
          `skb_flow_dissector_bpf_prog_attach'
          syscall.c:(.text+0x3310): undefined reference to
          `skb_flow_dissector_bpf_prog_detach'
          kernel/bpf/syscall.o:(.rodata+0x3f0): undefined reference to
          `flow_dissector_prog_ops'
          kernel/bpf/verifier.o:(.rodata+0x250): undefined reference to
          `flow_dissector_verifier_ops'
      
      Analogous to other optional BPF program types in syscall.c, add stubs
      if the relevant functions are not compiled and move the BPF_PROG_TYPE
      definition in the #ifdef CONFIG_NET block.
      
      Fixes: d58e468b ("flow_dissector: implements flow dissector BPF hook")
      Reported-by: default avatarRandy Dunlap <rdunlap@infradead.org>
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Acked-by: Randy Dunlap <rdunlap@infradead.org> # build-tested
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      2dfd184a
  4. 18 Sep, 2018 3 commits
    • Yonghong Song's avatar
      samples/bpf: fix a compilation failure · 534e0e52
      Yonghong Song authored
      samples/bpf build failed with the following errors:
      
        $ make samples/bpf/
        ...
        HOSTCC  samples/bpf/sockex3_user.o
        /data/users/yhs/work/net-next/samples/bpf/sockex3_user.c:16:8: error: redefinition of ‘struct bpf_flow_keys’
         struct bpf_flow_keys {
                ^
        In file included from /data/users/yhs/work/net-next/samples/bpf/sockex3_user.c:4:0:
        ./usr/include/linux/bpf.h:2338:9: note: originally defined here
          struct bpf_flow_keys *flow_keys;
                 ^
        make[3]: *** [samples/bpf/sockex3_user.o] Error 1
      
      Commit d58e468b ("flow_dissector: implements flow dissector BPF hook")
      introduced struct bpf_flow_keys in include/uapi/linux/bpf.h and hence
      caused the naming conflict with samples/bpf/sockex3_user.c.
      
      The fix is to rename struct bpf_flow_keys in samples/bpf/sockex3_user.c
      to flow_keys to avoid the conflict.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      534e0e52
    • YueHaibing's avatar
      samples/bpf: remove duplicated includes · 664e7878
      YueHaibing authored
      Remove duplicated includes.
      Signed-off-by: default avatarYueHaibing <yuehaibing@huawei.com>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      664e7878
    • Yonghong Song's avatar
      tools/bpf: bpftool: improve output format for bpftool net · 7900efc1
      Yonghong Song authored
      This is a followup patch for Commit f6f3bac0
      ("tools/bpf: bpftool: add net support").
      Some improvements are made for the bpftool net output.
      Specially, plain output is more concise such that
      per attachment should nicely fit in one line.
      Compared to previous output, the prog tag is removed
      since it can be easily obtained with program id.
      Similar to xdp attachments, the device name is added
      to tc attachments.
      
      The bpf program attached through shared block
      mechanism is supported as well.
        $ ip link add dev v1 type veth peer name v2
        $ tc qdisc add dev v1 ingress_block 10 egress_block 20 clsact
        $ tc qdisc add dev v2 ingress_block 10 egress_block 20 clsact
        $ tc filter add block 10 protocol ip prio 25 bpf obj bpf_shared.o sec ingress flowid 1:1
        $ tc filter add block 20 protocol ip prio 30 bpf obj bpf_cyclic.o sec classifier flowid 1:1
        $ bpftool net
        xdp:
      
        tc:
        v2(7) clsact/ingress bpf_shared.o:[ingress] id 23
        v2(7) clsact/egress bpf_cyclic.o:[classifier] id 24
        v1(8) clsact/ingress bpf_shared.o:[ingress] id 23
        v1(8) clsact/egress bpf_cyclic.o:[classifier] id 24
      
      The documentation and "bpftool net help" are updated
      to make it clear that current implementation only
      supports xdp and tc attachments. For programs
      attached to cgroups, "bpftool cgroup" can be used
      to dump attachments. For other programs e.g.
      sk_{filter,skb,msg,reuseport} and lwt/seg6,
      iproute2 tools should be used.
      
      The new output:
        $ bpftool net
        xdp:
        eth0(2) driver id 198
      
        tc:
        eth0(2) clsact/ingress fbflow_icmp id 335 act [{icmp_action id 336}]
        eth0(2) clsact/egress fbflow_egress id 334
        $ bpftool -jp net
        [{
              "xdp": [{
                      "devname": "eth0",
                      "ifindex": 2,
                      "mode": "driver",
                      "id": 198
                  }
              ],
              "tc": [{
                      "devname": "eth0",
                      "ifindex": 2,
                      "kind": "clsact/ingress",
                      "name": "fbflow_icmp",
                      "id": 335,
                      "act": [{
                              "name": "icmp_action",
                              "id": 336
                          }
                      ]
                  },{
                      "devname": "eth0",
                      "ifindex": 2,
                      "kind": "clsact/egress",
                      "name": "fbflow_egress",
                      "id": 334
                  }
              ]
          }
        ]
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
      7900efc1
  5. 14 Sep, 2018 7 commits
    • Alexei Starovoitov's avatar
      selftests/bpf: fix bpf_flow.c build · 70e88c75
      Alexei Starovoitov authored
      fix the following build error:
      clang -I. -I./include/uapi -I../../../include/uapi -idirafter /usr/local/include -idirafter /data/users/ast/llvm/bld/lib/clang/7.0.0/include -idirafter /usr/include -Wno-compare-distinct-pointer-types \
      	 -O2 -target bpf -emit-llvm -c bpf_flow.c -o - |      \
      llc -march=bpf -mcpu=generic  -filetype=obj -o /data/users/ast/bpf-next/tools/testing/selftests/bpf/bpf_flow.o
      LLVM ERROR: 'dissect' label emitted multiple times to assembly file
      make: *** [/data/users/ast/bpf-next/tools/testing/selftests/bpf/bpf_flow.o] Error 1
      
      Fixes: 9c98b13c ("flow_dissector: implements eBPF parser")
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      70e88c75
    • Alexei Starovoitov's avatar
      Merge branch 'bpf-flow-dissector' · 4a9f42c9
      Alexei Starovoitov authored
      Petar Penkov says:
      
      ====================
      This patch series hardens the RX stack by allowing flow dissection in BPF,
      as previously discussed [1]. Because of the rigorous checks of the BPF
      verifier, this provides significant security guarantees. In particular, the
      BPF flow dissector cannot get inside of an infinite loop, as with
      CVE-2013-4348, because BPF programs are guaranteed to terminate. It cannot
      read outside of packet bounds, because all memory accesses are checked.
      Also, with BPF the administrator can decide which protocols to support,
      reducing potential attack surface. Rarely encountered protocols can be
      excluded from dissection and the program can be updated without kernel
      recompile or reboot if a bug is discovered.
      
      Patch 1 adds infrastructure to execute a BPF program in __skb_flow_dissect.
      This includes a new BPF program and attach type.
      
      Patch 2 adds the new BPF flow dissector definitions to tools/uapi.
      
      Patch 3 adds support for the new BPF program type to libbpf and bpftool.
      
      Patch 4 adds a flow dissector program in BPF. This parses most protocols in
      __skb_flow_dissect in BPF for a subset of flow keys (basic, control, ports,
      and address types).
      
      Patch 5 adds a selftest that attaches the BPF program to the flow dissector
      and sends traffic with different levels of encapsulation.
      
      Performance Evaluation:
      The in-kernel implementation was compared against the demo program from
      patch 4 using the test in patch 5 with IPv4/UDP traffic over 10 seconds.
      	$perf record -a -C 4 taskset -c 4 ./test_flow_dissector -i 4 -f 8 \
      		-t 10
      
      In-kernel Dissector:
      	__skb_flow_dissect overhead: 2.12%
      	Total Packets: 3,272,597 (from output of ./test_flow_dissector)
      
      BPF Dissector:
      	__skb_flow_dissect overhead: 1.63%
      	Total Packets: 3,232,356 (from output of ./test_flow_dissector)
      
      No-op BPF Dissector:
      	__skb_flow_dissect overhead: 1.52%
      	Total Packets: 3,330,635 (from output of ./test_flow_dissector)
      
      Changes since v3:
      1/ struct bpf_flow_keys reorganized to remove holes in patch 1 and patch 2.
      
      Changes since v2:
      1/ Changes to tools/include/uapi pulled into a separate patch 2
      2/ Changes to tools/lib and tools/bpftool pulled into a separate patch 3
      3/ Changed flow_keys in __sk_buff from __u32 to struct bpf_flow_keys *
      4/ Added nhoff field in struct bpf_flow_keys to pass initial offset
      5/ Saving all of the modified control block, rather than just the qdisc
      6/ Sample BPF program in patch 4 modified to use the changes above
      
      Changes since v1:
      1/ LD_ABS instructions now disallowed for the new BPF prog type
      2/ now checks if skb is NULL in __skb_flow_dissect()
      3/ fixed incorrect accesses in flow_dissector_is_valid_access()
      	- writes to the flow_keys field now disallowed
      	- reads/writes to tc_classid and data_meta now disallowed
      4/ headers pulled with bpf_skb_load_data if direct access fails
      
      Changes since RFC:
      1/ Flow dissector hook changed from global to per-netns
      2/ Defined struct bpf_flow_keys to be used in BPF flow dissector
      programs instead of exposing the internal flow keys layout. Added a
      function to translate from bpf_flow_keys to the internal layout after BPF
      dissection is complete. The pointer to this struct is stored in
      qdisc_skb_cb rather than inside of the 20 byte control block which
      simplifies verification and allows access to all 20 bytes of the cb.
      3/ Removed GUE parsing as it relied on a hardcoded port
      4/ MPLS parsing now stops at the first label which is consistent
      with the in-kernel flow dissector
      5/ Refactored to use direct packet access and to write out to
      struct bpf_flow_keys
      
      [1] http://vger.kernel.org/netconf2017_files/rx_hardening_and_udp_gso.pdf
      ====================
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      4a9f42c9
    • Petar Penkov's avatar
      selftests/bpf: test bpf flow dissection · 50b3ed57
      Petar Penkov authored
      Adds a test that sends different types of packets over multiple
      tunnels and verifies that valid packets are dissected correctly.  To do
      so, a tc-flower rule is added to drop packets on UDP src port 9, and
      packets are sent from ports 8, 9, and 10. Only the packets on port 9
      should be dropped. Because tc-flower relies on the flow dissector to
      match flows, correct classification demonstrates correct dissection.
      
      Also add support logic to load the BPF program and to inject the test
      packets.
      Signed-off-by: default avatarPetar Penkov <ppenkov@google.com>
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      50b3ed57
    • Petar Penkov's avatar
      flow_dissector: implements eBPF parser · 9c98b13c
      Petar Penkov authored
      This eBPF program extracts basic/control/ip address/ports keys from
      incoming packets. It supports recursive parsing for IP encapsulation,
      and VLAN, along with IPv4/IPv6 and extension headers.  This program is
      meant to show how flow dissection and key extraction can be done in
      eBPF.
      
      Link: http://vger.kernel.org/netconf2017_files/rx_hardening_and_udp_gso.pdfSigned-off-by: default avatarPetar Penkov <ppenkov@google.com>
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      9c98b13c
    • Petar Penkov's avatar
      bpf: support flow dissector in libbpf and bpftool · c22fbae7
      Petar Penkov authored
      This patch extends libbpf and bpftool to work with programs of type
      BPF_PROG_TYPE_FLOW_DISSECTOR.
      Signed-off-by: default avatarPetar Penkov <ppenkov@google.com>
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      c22fbae7
    • Petar Penkov's avatar
      bpf: sync bpf.h uapi with tools/ · 2f965e3f
      Petar Penkov authored
      This patch syncs tools/include/uapi/linux/bpf.h with the flow dissector
      definitions from include/uapi/linux/bpf.h
      Signed-off-by: default avatarPetar Penkov <ppenkov@google.com>
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      2f965e3f
    • Petar Penkov's avatar
      flow_dissector: implements flow dissector BPF hook · d58e468b
      Petar Penkov authored
      Adds a hook for programs of type BPF_PROG_TYPE_FLOW_DISSECTOR and
      attach type BPF_FLOW_DISSECTOR that is executed in the flow dissector
      path. The BPF program is per-network namespace.
      Signed-off-by: default avatarPetar Penkov <ppenkov@google.com>
      Signed-off-by: default avatarWillem de Bruijn <willemb@google.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      d58e468b
  6. 11 Sep, 2018 5 commits
    • Anders Roxell's avatar
      net/core/filter: fix unused-variable warning · 1edb6e03
      Anders Roxell authored
      Building with CONFIG_INET=n will show the warning below:
      net/core/filter.c: In function ‘____bpf_getsockopt’:
      net/core/filter.c:4048:19: warning: unused variable ‘tp’ [-Wunused-variable]
        struct tcp_sock *tp;
                         ^~
      net/core/filter.c:4046:31: warning: unused variable ‘icsk’ [-Wunused-variable]
        struct inet_connection_sock *icsk;
                                     ^~~~
      Move the variable declarations inside the {} block where they are used.
      
      Fixes: 1e215300 ("bpf: add TCP_SAVE_SYN/TCP_SAVED_SYN options for bpf_(set|get)sockopt")
      Signed-off-by: default avatarAnders Roxell <anders.roxell@linaro.org>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      1edb6e03
    • Yonghong Song's avatar
      tools/bpf: fix a netlink recv issue · 9d0b3c1f
      Yonghong Song authored
      Commit f7010770 ("tools/bpf: move bpf/lib netlink related
      functions into a new file") introduced a while loop for the
      netlink recv path. This while loop is needed since the
      buffer in recv syscall may not be enough to hold all the
      information and in such cases multiple recv calls are needed.
      
      There is a bug introduced by the above commit as
      the while loop may block on recv syscall if there is no
      more messages are expected. The netlink message header
      flag NLM_F_MULTI is used to indicate that more messages
      are expected and this patch fixed the bug by doing
      further recv syscall only if multipart message is expected.
      
      The patch added another fix regarding to message length of 0.
      When netlink recv returns message length of 0, there will be
      no more messages for returning data so the while loop
      can end.
      
      Fixes: f7010770 ("tools/bpf: move bpf/lib netlink related functions into a new file")
      Reported-by: default avatarBjörn Töpel <bjorn.topel@intel.com>
      Tested-by: default avatarBjörn Töpel <bjorn.topel@intel.com>
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      9d0b3c1f
    • Alexei Starovoitov's avatar
      Merge branch 'progarray_mapinmap_dump' · 2e2a0c96
      Alexei Starovoitov authored
      Yonghong Song says:
      
      ====================
      The support to dump program array and map_in_map maps
      for bpffs and bpftool is added. Patch #1 added bpffs support
      and Patch #2 added bpftool support. Please see
      individual patches for example output.
      ====================
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      2e2a0c96
    • Yonghong Song's avatar
      tools/bpf: bpftool: support prog array map and map of maps · ad3338d2
      Yonghong Song authored
      Currently, prog array map and map of maps are not supported
      in bpftool. This patch added the support.
      Different from other map types, for prog array map and
      map of maps, the key returned bpf_get_next_key() may not
      point to a valid value. So for these two map types,
      no error will be printed out when such a scenario happens.
      
      The following is the plain and json dump if btf is not available:
        $ ./bpftool map dump id 10
          key: 08 00 00 00  value: 5c 01 00 00
          Found 1 element
        $ ./bpftool -jp map dump id 10
          [{
              "key": ["0x08","0x00","0x00","0x00"
              ],
              "value": ["0x5c","0x01","0x00","0x00"
              ]
          }]
      
      If the BTF is available, the dump looks below:
        $ ./bpftool map dump id 2
          [{
                  "key": 0,
                  "value": 7
              }
          ]
        $ ./bpftool -jp map dump id 2
          [{
              "key": ["0x00","0x00","0x00","0x00"
              ],
              "value": ["0x07","0x00","0x00","0x00"
              ],
              "formatted": {
                  "key": 0,
                  "value": 7
              }
          }]
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      ad3338d2
    • Yonghong Song's avatar
      bpf: add bpffs pretty print for program array map · a7c19db3
      Yonghong Song authored
      Added bpffs pretty print for program array map. For a particular
      array index, if the program array points to a valid program,
      the "<index>: <prog_id>" will be printed out like
         0: 6
      which means bpf program with id "6" is installed at index "0".
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      a7c19db3
  7. 07 Sep, 2018 9 commits
    • Yonghong Song's avatar
      tools/bpf: bpftool: add net support · f6f3bac0
      Yonghong Song authored
      Add "bpftool net" support. Networking devices are enumerated
      to dump device index/name associated with xdp progs.
      
      For each networking device, tc classes and qdiscs are enumerated
      in order to check their bpf filters.
      In addition, root handle and clsact ingress/egress are also checked for
      bpf filters.  Not all filter information is printed out. Only ifindex,
      kind, filter name, prog_id and tag are printed out, which are good
      enough to show attachment information. If the filter action
      is a bpf action, its bpf program id, bpf name and tag will be
      printed out as well.
      
      For example,
        $ ./bpftool net
        xdp [
        ifindex 2 devname eth0 prog_id 198
        ]
        tc_filters [
        ifindex 2 kind qdisc_htb name prefix_matcher.o:[cls_prefix_matcher_htb]
                  prog_id 111727 tag d08fe3b4319bc2fd act []
        ifindex 2 kind qdisc_clsact_ingress name fbflow_icmp
                  prog_id 130246 tag 3f265c7f26db62c9 act []
        ifindex 2 kind qdisc_clsact_egress name prefix_matcher.o:[cls_prefix_matcher_clsact]
                  prog_id 111726 tag 99a197826974c876
        ifindex 2 kind qdisc_clsact_egress name cls_fg_dscp
                  prog_id 108619 tag dc4630674fd72dcc act []
        ifindex 2 kind qdisc_clsact_egress name fbflow_egress
                  prog_id 130245 tag 72d2d830d6888d2c
        ]
        $ ./bpftool -jp net
        [{
              "xdp": [{
                      "ifindex": 2,
                      "devname": "eth0",
                      "prog_id": 198
                  }
              ],
              "tc_filters": [{
                      "ifindex": 2,
                      "kind": "qdisc_htb",
                      "name": "prefix_matcher.o:[cls_prefix_matcher_htb]",
                      "prog_id": 111727,
                      "tag": "d08fe3b4319bc2fd",
                      "act": []
                  },{
                      "ifindex": 2,
                      "kind": "qdisc_clsact_ingress",
                      "name": "fbflow_icmp",
                      "prog_id": 130246,
                      "tag": "3f265c7f26db62c9",
                      "act": []
                  },{
                      "ifindex": 2,
                      "kind": "qdisc_clsact_egress",
                      "name": "prefix_matcher.o:[cls_prefix_matcher_clsact]",
                      "prog_id": 111726,
                      "tag": "99a197826974c876"
                  },{
                      "ifindex": 2,
                      "kind": "qdisc_clsact_egress",
                      "name": "cls_fg_dscp",
                      "prog_id": 108619,
                      "tag": "dc4630674fd72dcc",
                      "act": []
                  },{
                      "ifindex": 2,
                      "kind": "qdisc_clsact_egress",
                      "name": "fbflow_egress",
                      "prog_id": 130245,
                      "tag": "72d2d830d6888d2c"
                  }
              ]
          }
        ]
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      f6f3bac0
    • Yonghong Song's avatar
      tools/bpf: add more netlink functionalities in lib/bpf · 36f1678d
      Yonghong Song authored
      This patch added a few netlink attribute parsing functions
      and the netlink API functions to query networking links, tc classes,
      tc qdiscs and tc filters. For example, the following API is
      to get networking links:
        int nl_get_link(int sock, unsigned int nl_pid,
                        dump_nlmsg_t dump_link_nlmsg,
                        void *cookie);
      
      Note that when the API is called, the user also provided a
      callback function with the following signature:
        int (*dump_nlmsg_t)(void *cookie, void *msg, struct nlattr **tb);
      
      The "cookie" is the parameter the user passed to the API and will
      be available for the callback function.
      The "msg" is the information about the result, e.g., ifinfomsg or
      tcmsg. The "tb" is the parsed netlink attributes.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      36f1678d
    • Yonghong Song's avatar
      tools/bpf: move bpf/lib netlink related functions into a new file · f7010770
      Yonghong Song authored
      There are no functionality change for this patch.
      
      In the subsequent patches, more netlink related library functions
      will be added and a separate file is better than cluttering bpf.c.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      f7010770
    • Yonghong Song's avatar
      tools/bpf: sync kernel uapi header if_link.h to tools · 52b7b784
      Yonghong Song authored
      Among others, this header will be used later for
      bpftool net support.
      Signed-off-by: default avatarYonghong Song <yhs@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      52b7b784
    • Mauricio Vasquez B's avatar
      selftests/bpf/test_progs: do not check errno == 0 · f5bd3948
      Mauricio Vasquez B authored
      The errno man page states: "The value in errno is significant only when
      the return value of the call indicated an error..." then it is not correct
      to check it, it could be different than zero even if the function
      succeeded.
      
      It causes some false positives if errno is set by a previous function.
      Signed-off-by: default avatarMauricio Vasquez B <mauricio.vasquez@polito.it>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      f5bd3948
    • Mauricio Vasquez B's avatar
    • Jesper Dangaard Brouer's avatar
      xdp: split code for map vs non-map redirect · 47b123ed
      Jesper Dangaard Brouer authored
      The compiler does an efficient job of inlining static C functions.
      Perf top clearly shows that almost everything gets inlined into the
      function call xdp_do_redirect.
      
      The function xdp_do_redirect end-up containing and interleaving the
      map and non-map redirect code.  This is sub-optimal, as it would be
      strange for an XDP program to use both types of redirect in the same
      program. The two use-cases are separate, and interleaving the code
      just cause more instruction-cache pressure.
      
      I would like to stress (again) that the non-map variant bpf_redirect
      is very slow compared to the bpf_redirect_map variant, approx half the
      speed.  Measured with driver i40e the difference is:
      
      - map     redirect: 13,250,350 pps
      - non-map redirect:  7,491,425 pps
      
      For this reason, the function name of the non-map variant of redirect
      have been called xdp_do_redirect_slow.  This hopefully gives a hint
      when using perf, that this is not the optimal XDP redirect operating mode.
      Signed-off-by: default avatarJesper Dangaard Brouer <brouer@redhat.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      47b123ed
    • Jesper Dangaard Brouer's avatar
      xdp: explicit inline __xdp_map_lookup_elem · 2a68d85f
      Jesper Dangaard Brouer authored
      The compiler chooses to not-inline the function __xdp_map_lookup_elem,
      because it can see that it is used by both Generic-XDP and native-XDP
      do redirect calls (xdp_do_generic_redirect_map and xdp_do_redirect_map).
      
      The compiler cannot know that this is a bad choice, as it cannot know
      that a net device cannot run both XDP modes (Generic or Native) at the
      same time.  Thus, mark this function inline, even-though we normally
      leave this up-to the compiler.
      Signed-off-by: default avatarJesper Dangaard Brouer <brouer@redhat.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      2a68d85f
    • Jesper Dangaard Brouer's avatar
      xdp: unlikely instrumentation for xdp map redirect · e1302542
      Jesper Dangaard Brouer authored
      Notice the compiler generated ASM code layout was suboptimal.  It
      assumed map enqueue errors as the likely case, which is shouldn't.
      It assumed that xdp_do_flush_map() was a likely case, due to maps
      changing between packets, which should be very unlikely.
      Signed-off-by: default avatarJesper Dangaard Brouer <brouer@redhat.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      e1302542
  8. 06 Sep, 2018 4 commits
    • Alexei Starovoitov's avatar
      bpf/verifier: fix verifier instability · a9c676bc
      Alexei Starovoitov authored
      Edward Cree says:
      In check_mem_access(), for the PTR_TO_CTX case, after check_ctx_access()
      has supplied a reg_type, the other members of the register state are set
      appropriately.  Previously reg.range was set to 0, but as it is in a
      union with reg.map_ptr, which is larger, upper bytes of the latter were
      left in place.  This then caused the memcmp() in regsafe() to fail,
      preventing some branches from being pruned (and occasionally causing the
      same program to take a varying number of processed insns on repeated
      verifier runs).
      
      Fix the instability by clearing bpf_reg_state in __mark_reg_[un]known()
      
      Fixes: f1174f77 ("bpf/verifier: rework value tracking")
      Debugged-by: default avatarEdward Cree <ecree@solarflare.com>
      Acked-by: default avatarEdward Cree <ecree@solarflare.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      a9c676bc
    • Taeung Song's avatar
      libbpf: Remove the duplicate checking of function storage · 69495d2a
      Taeung Song authored
      After the commit eac7d845 ("tools: libbpf: don't return '.text'
      as a program for multi-function programs"), bpf_program__next()
      in bpf_object__for_each_program skips the function storage such as .text,
      so eliminate the duplicate checking.
      
      Cc: Jakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarTaeung Song <treeze.taeung@gmail.com>
      Acked-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      69495d2a
    • Dmitry Safonov's avatar
      netlink: Make groups check less stupid in netlink_bind() · 428f944b
      Dmitry Safonov authored
      As Linus noted, the test for 0 is needless, groups type can follow the
      usual kernel style and 8*sizeof(unsigned long) is BITS_PER_LONG:
      
      > The code [..] isn't technically incorrect...
      > But it is stupid.
      > Why stupid? Because the test for 0 is pointless.
      >
      > Just doing
      >        if (nlk->ngroups < 8*sizeof(groups))
      >                groups &= (1UL << nlk->ngroups) - 1;
      >
      > would have been fine and more understandable, since the "mask by shift
      > count" already does the right thing for a ngroups value of 0. Now that
      > test for zero makes me go "what's special about zero?". It turns out
      > that the answer to that is "nothing".
      [..]
      > The type of "groups" is kind of silly too.
      >
      > Yeah, "long unsigned int" isn't _technically_ wrong. But we normally
      > call that type "unsigned long".
      
      Cleanup my piece of pointlessness.
      
      Cc: "David S. Miller" <davem@davemloft.net>
      Cc: Herbert Xu <herbert@gondor.apana.org.au>
      Cc: Steffen Klassert <steffen.klassert@secunet.com>
      Cc: netdev@vger.kernel.org
      Fairly-blamed-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Signed-off-by: default avatarDmitry Safonov <dima@arista.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      428f944b
    • Vincent Whitchurch's avatar
      packet: add sockopt to ignore outgoing packets · fa788d98
      Vincent Whitchurch authored
      Currently, the only way to ignore outgoing packets on a packet socket is
      via the BPF filter.  With MSG_ZEROCOPY, packets that are looped into
      AF_PACKET are copied in dev_queue_xmit_nit(), and this copy happens even
      if the filter run from packet_rcv() would reject them.  So the presence
      of a packet socket on the interface takes away the benefits of
      MSG_ZEROCOPY, even if the packet socket is not interested in outgoing
      packets.  (Even when MSG_ZEROCOPY is not used, the skb is unnecessarily
      cloned, but the cost for that is much lower.)
      
      Add a socket option to allow AF_PACKET sockets to ignore outgoing
      packets to solve this.  Note that the *BSDs already have something
      similar: BIOCSSEESENT/BIOCSDIRECTION and BIOCSDIRFILT.
      
      The first intended user is lldpd.
      Signed-off-by: default avatarVincent Whitchurch <vincent.whitchurch@axis.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      fa788d98
  9. 05 Sep, 2018 8 commits