1. 13 Nov, 2020 7 commits
  2. 12 Nov, 2020 3 commits
  3. 11 Nov, 2020 9 commits
  4. 10 Nov, 2020 10 commits
    • Alexei Starovoitov's avatar
      Merge branch 'Remove unused test_ipip.sh test and add missed' · 9600d623
      Alexei Starovoitov authored
      Hangbin Liu says:
      
      ====================
      
      In comment 173ca26e ("samples/bpf: add comprehensive ipip, ipip6,
      ip6ip6 test") we added some bpf tunnel tests. In commit 933a741e
      ("selftests/bpf: bpf tunnel test.") when we moved it to the current
      folder, we missed some points:
      
      1. ip6ip6 test is not added
      2. forgot to remove test_ipip.sh in sample folder
      3. TCP test code is not removed in test_tunnel_kern.c
      
      In this patch set I add back ip6ip6 test and remove unused code. I'm not sure
      if this should be net or net-next, so just set to net.
      
      Here is the test result:
      ```
      Testing IP6IP6 tunnel...
      PING ::11(::11) 56 data bytes
      
      --- ::11 ping statistics ---
      3 packets transmitted, 3 received, 0% packet loss, time 63ms
      rtt min/avg/max/mdev = 0.014/1028.308/2060.906/841.361 ms, pipe 2
      PING 1::11(1::11) 56 data bytes
      
      --- 1::11 ping statistics ---
      3 packets transmitted, 3 received, 0% packet loss, time 48ms
      rtt min/avg/max/mdev = 0.026/0.029/0.036/0.006 ms
      PING 1::22(1::22) 56 data bytes
      
      --- 1::22 ping statistics ---
      3 packets transmitted, 3 received, 0% packet loss, time 47ms
      rtt min/avg/max/mdev = 0.030/0.048/0.067/0.016 ms
      PASS: ip6ip6tnl
      ```
      
      v3:
      Add back ICMP check as Martin suggested.
      
      v2: Keep ip6ip6 section in test_tunnel_kern.c.
      ====================
      Acked-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      9600d623
    • Hangbin Liu's avatar
      samples/bpf: Remove unused test_ipip.sh · e2215b05
      Hangbin Liu authored
      The tcbpf2_kern.o and related kernel sections are moved to bpf
      selftest folder since b05cd740 ("samples/bpf: remove the bpf tunnel
      testsuite."). Remove this one as well.
      
      Fixes: b05cd740 ("samples/bpf: remove the bpf tunnel testsuite.")
      Signed-off-by: default avatarHangbin Liu <liuhangbin@gmail.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Link: https://lore.kernel.org/bpf/20201110015013.1570716-3-liuhangbin@gmail.com
      e2215b05
    • Hangbin Liu's avatar
      selftest/bpf: Add missed ip6ip6 test back · 58cfa49c
      Hangbin Liu authored
      In comment 173ca26e ("samples/bpf: add comprehensive ipip, ipip6,
      ip6ip6 test") we added ip6ip6 test for bpf tunnel testing. But in commit
      933a741e ("selftests/bpf: bpf tunnel test.") when we moved it to
      the current folder, we didn't add it.
      
      This patch add the ip6ip6 test back to bpf tunnel test. Update the ipip6's
      topology for both IPv4 and IPv6 testing. Since iperf test is removed as
      currect framework simplified it in purpose, I also removed unused tcp
      checkings in test_tunnel_kern.c.
      
      Fixes: 933a741e ("selftests/bpf: bpf tunnel test.")
      Signed-off-by: default avatarHangbin Liu <liuhangbin@gmail.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20201110015013.1570716-2-liuhangbin@gmail.com
      58cfa49c
    • Alexei Starovoitov's avatar
      Merge branch 'Integrate kernel module BTF support' · 0e6f601e
      Alexei Starovoitov authored
      Andrii Nakryiko says:
      
      ====================
      
      This patch set adds BTF generation for kernel modules using a compact split
      BTF approach. Respective patches have all the details.
      
      Kernel module BTFs rely on pahole's split BTF support, which is added in [0]
      and will be available starting from v1.19. Support for it is detected
      automatically during kernel build time.
      
      This patch set implements in-kernel support for split BTF loading and
      validation. It also extends GET_OBJ_INFO API for BTFs to return BTF's module
      name and a flag whether BTF itself is in-kernel or user-provided. vmlinux BTF
      is also exposed to user-space through the same BTF object iteration APIs.
      
      Follow up patch set will utilize the fact that vmlinux and module BTFs now
      have associated ID to provide ability to attach BPF fentry/fexit/etc programs
      to functions defined in kernel modules.
      
      bpftool is also extended to show module/vmlinux BTF's name.
      
        [0] https://patchwork.kernel.org/project/netdevbpf/list/?series=378699&state=*
      
      v3->v4:
        - copy_to_user() on ENOSPC in btf_get_info_by_fd() (Martin);
      v2->v3:
        - get rid of unnecessary gotos (Song);
      v2->v1:
        - drop WARNs, add fewer pr_warn()'s instead (Greg);
        - properly initialize sysfs binary attribute structure (Greg);
        - add __maybe_unused to any_section_objs, used conditionally by module BTF;
      rfc->v1:
        - CONFIG_DEBUG_INFO_BTF_MODULES is derived automatically (Alexei);
        - vmlinux BTF now has explicit "vmlinux" name (Alexei);
        - added sysfs ABI documentation for /sys/kernel/btf/<module> (Greg).
      
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
      ====================
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      0e6f601e
    • Andrii Nakryiko's avatar
      tools/bpftool: Add support for in-kernel and named BTF in `btf show` · cecaf4a0
      Andrii Nakryiko authored
      Display vmlinux BTF name and kernel module names when listing available BTFs
      on the system.
      
      In human-readable output mode, module BTFs are reported with "name
      [module-name]", while vmlinux BTF will be reported as "name [vmlinux]".
      Square brackets are added by bpftool and follow kernel convention when
      displaying modules in human-readable text outputs.
      
      [vmuser@archvm bpf]$ sudo ../../../bpf/bpftool/bpftool btf s
      1: name [vmlinux]  size 4082281B
      6: size 2365B  prog_ids 8,6  map_ids 3
      7: name [button]  size 46895B
      8: name [pcspkr]  size 42328B
      9: name [serio_raw]  size 39375B
      10: name [floppy]  size 57185B
      11: name [i2c_core]  size 76186B
      12: name [crc32c_intel]  size 16036B
      13: name [i2c_piix4]  size 50497B
      14: name [irqbypass]  size 14124B
      15: name [kvm]  size 197985B
      16: name [kvm_intel]  size 123564B
      17: name [cryptd]  size 42466B
      18: name [crypto_simd]  size 17187B
      19: name [glue_helper]  size 39205B
      20: name [aesni_intel]  size 41034B
      25: size 36150B
              pids bpftool(2519)
      
      In JSON mode, two fields (boolean "kernel" and string "name") are reported for
      each BTF object. vmlinux BTF is reported with name "vmlinux" (kernel itself
      returns and empty name for vmlinux BTF).
      
      [vmuser@archvm bpf]$ sudo ../../../bpf/bpftool/bpftool btf s -jp
      [{
              "id": 1,
              "size": 4082281,
              "prog_ids": [],
              "map_ids": [],
              "kernel": true,
              "name": "vmlinux"
          },{
              "id": 6,
              "size": 2365,
              "prog_ids": [8,6
              ],
              "map_ids": [3
              ],
              "kernel": false
          },{
              "id": 7,
              "size": 46895,
              "prog_ids": [],
              "map_ids": [],
              "kernel": true,
              "name": "button"
          },{
      
      ...
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Tested-by: default avatarAlan Maguire <alan.maguire@oracle.com>
      Link: https://lore.kernel.org/bpf/20201110011932.3201430-6-andrii@kernel.org
      cecaf4a0
    • Andrii Nakryiko's avatar
      bpf: Load and verify kernel module BTFs · 36e68442
      Andrii Nakryiko authored
      Add kernel module listener that will load/validate and unload module BTF.
      Module BTFs gets ID generated for them, which makes it possible to iterate
      them with existing BTF iteration API. They are given their respective module's
      names, which will get reported through GET_OBJ_INFO API. They are also marked
      as in-kernel BTFs for tooling to distinguish them from user-provided BTFs.
      
      Also, similarly to vmlinux BTF, kernel module BTFs are exposed through
      sysfs as /sys/kernel/btf/<module-name>. This is convenient for user-space
      tools to inspect module BTF contents and dump their types with existing tools:
      
      [vmuser@archvm bpf]$ ls -la /sys/kernel/btf
      total 0
      drwxr-xr-x  2 root root       0 Nov  4 19:46 .
      drwxr-xr-x 13 root root       0 Nov  4 19:46 ..
      
      ...
      
      -r--r--r--  1 root root     888 Nov  4 19:46 irqbypass
      -r--r--r--  1 root root  100225 Nov  4 19:46 kvm
      -r--r--r--  1 root root   35401 Nov  4 19:46 kvm_intel
      -r--r--r--  1 root root     120 Nov  4 19:46 pcspkr
      -r--r--r--  1 root root     399 Nov  4 19:46 serio_raw
      -r--r--r--  1 root root 4094095 Nov  4 19:46 vmlinux
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Reviewed-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      Link: https://lore.kernel.org/bpf/20201110011932.3201430-5-andrii@kernel.org
      36e68442
    • Andrii Nakryiko's avatar
      kbuild: Build kernel module BTFs if BTF is enabled and pahole supports it · 5f9ae91f
      Andrii Nakryiko authored
      Detect if pahole supports split BTF generation, and generate BTF for each
      selected kernel module, if it does. This is exposed to Makefiles and C code as
      CONFIG_DEBUG_INFO_BTF_MODULES flag.
      
      Kernel module BTF has to be re-generated if either vmlinux's BTF changes or
      module's .ko changes. To achieve that, I needed a helper similar to
      if_changed, but that would allow to filter out vmlinux from the list of
      updated dependencies for .ko building. I've put it next to the only place that
      uses and needs it, but it might be a better idea to just add it along the
      other if_changed variants into scripts/Kbuild.include.
      
      Each kernel module's BTF deduplication is pretty fast, as it does only
      incremental BTF deduplication on top of already deduplicated vmlinux BTF. To
      show the added build time, I've first ran make only just built kernel (to
      establish the baseline) and then forced only BTF re-generation, without
      regenerating .ko files. The build was performed with -j60 parallelization on
      56-core machine. The final time also includes bzImage building, so it's not
      a pure BTF overhead.
      
      $ time make -j60
      ...
      make -j60  27.65s user 10.96s system 782% cpu 4.933 total
      $ touch ~/linux-build/default/vmlinux && time make -j60
      ...
      make -j60  123.69s user 27.85s system 1566% cpu 9.675 total
      
      So 4.6 seconds real time, with noticeable part spent in compressed vmlinux and
      bzImage building.
      
      To show size savings, I've built my kernel configuration with about 700 kernel
      modules with full BTF per each kernel module (without deduplicating against
      vmlinux) and with split BTF against deduplicated vmlinux (approach in this
      patch). Below are top 10 modules with biggest BTF sizes. And total size of BTF
      data across all kernel modules.
      
      It shows that split BTF "compresses" 115MB down to 5MB total. And the biggest
      kernel modules get a downsize from 500-570KB down to 200-300KB.
      
      FULL BTF
      ========
      
      $ for f in $(find . -name '*.ko'); do size -A -d $f | grep BTF | awk '{print $2}'; done | awk '{ s += $1 } END { print s }'
      115710691
      
      $ for f in $(find . -name '*.ko'); do printf "%s %d\n" $f $(size -A -d $f | grep BTF | awk '{print $2}'); done | sort -nr -k2 | head -n10
      ./drivers/gpu/drm/i915/i915.ko 570570
      ./drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 520240
      ./drivers/gpu/drm/radeon/radeon.ko 503849
      ./drivers/infiniband/hw/mlx5/mlx5_ib.ko 491777
      ./fs/xfs/xfs.ko 411544
      ./drivers/net/ethernet/intel/i40e/i40e.ko 403904
      ./drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 398754
      ./drivers/infiniband/core/ib_core.ko 397224
      ./fs/cifs/cifs.ko 386249
      ./fs/nfsd/nfsd.ko 379738
      
      SPLIT BTF
      =========
      
      $ for f in $(find . -name '*.ko'); do size -A -d $f | grep BTF | awk '{print $2}'; done | awk '{ s += $1 } END { print s }'
      5194047
      
      $ for f in $(find . -name '*.ko'); do printf "%s %d\n" $f $(size -A -d $f | grep BTF | awk '{print $2}'); done | sort -nr -k2 | head -n10
      ./drivers/gpu/drm/i915/i915.ko 293206
      ./drivers/gpu/drm/radeon/radeon.ko 282103
      ./fs/xfs/xfs.ko 222150
      ./drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 198503
      ./drivers/infiniband/hw/mlx5/mlx5_ib.ko 198356
      ./drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 113444
      ./fs/cifs/cifs.ko 109379
      ./arch/x86/kvm/kvm.ko 100225
      ./drivers/gpu/drm/drm.ko 94827
      ./drivers/infiniband/core/ib_core.ko 91188
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Link: https://lore.kernel.org/bpf/20201110011932.3201430-4-andrii@kernel.org
      5f9ae91f
    • Andrii Nakryiko's avatar
      bpf: Assign ID to vmlinux BTF and return extra info for BTF in GET_OBJ_INFO · 53297220
      Andrii Nakryiko authored
      Allocate ID for vmlinux BTF. This makes it visible when iterating over all BTF
      objects in the system. To allow distinguishing vmlinux BTF (and later kernel
      module BTF) from user-provided BTFs, expose extra kernel_btf flag, as well as
      BTF name ("vmlinux" for vmlinux BTF, will equal to module's name for module
      BTF).  We might want to later allow specifying BTF name for user-provided BTFs
      as well, if that makes sense. But currently this is reserved only for
      in-kernel BTFs.
      
      Having in-kernel BTFs exposed IDs will allow to extend BPF APIs that require
      in-kernel BTF type with ability to specify BTF types from kernel modules, not
      just vmlinux BTF. This will be implemented in a follow up patch set for
      fentry/fexit/fmod_ret/lsm/etc.
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Link: https://lore.kernel.org/bpf/20201110011932.3201430-3-andrii@kernel.org
      53297220
    • Andrii Nakryiko's avatar
      bpf: Add in-kernel split BTF support · 951bb646
      Andrii Nakryiko authored
      Adjust in-kernel BTF implementation to support a split BTF mode of operation.
      Changes are mostly mirroring libbpf split BTF changes, with the exception of
      start_id being 0 for in-kernel implementation due to simpler read-only mode.
      
      Otherwise, for split BTF logic, most of the logic of jumping to base BTF,
      where necessary, is encapsulated in few helper functions. Type numbering and
      string offset in a split BTF are logically continuing where base BTF ends, so
      most of the high-level logic is kept without changes.
      
      Type verification and size resolution is only doing an added resolution of new
      split BTF types and relies on already cached size and type resolution results
      in the base BTF.
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarSong Liu <songliubraving@fb.com>
      Link: https://lore.kernel.org/bpf/20201110011932.3201430-2-andrii@kernel.org
      951bb646
    • Martin KaFai Lau's avatar
      bpf: selftest: Use static globals in tcp_hdr_options and btf_skc_cls_ingress · f52b8fd3
      Martin KaFai Lau authored
      Some globals in the tcp_hdr_options test and btf_skc_cls_ingress test
      are not using static scope.  This patch fixes it.
      
      Targeting bpf-next branch as an improvement since it currently does not
      break the build.
      
      Fixes: ad2f8eb0 ("bpf: selftests: Tcp header options")
      Fixes: 9a856cae ("bpf: selftest: Add test_btf_skc_cls_ingress")
      Signed-off-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
      Link: https://lore.kernel.org/bpf/20201106225402.4135741-1-kafai@fb.com
      f52b8fd3
  5. 09 Nov, 2020 2 commits
  6. 06 Nov, 2020 9 commits