1. 11 Mar, 2022 13 commits
  2. 10 Mar, 2022 12 commits
  3. 09 Mar, 2022 10 commits
    • Alexei Starovoitov's avatar
      Merge branch 'Add support for transmitting packets using XDP in bpf_prog_run()' · de55c9a1
      Alexei Starovoitov authored
      Toke Høiland-Jørgensen says:
      
      ====================
      
      This series adds support for transmitting packets using XDP in
      bpf_prog_run(), by enabling a new mode "live packet" mode which will handle
      the XDP program return codes and redirect the packets to the stack or other
      devices.
      
      The primary use case for this is testing the redirect map types and the
      ndo_xdp_xmit driver operation without an external traffic generator. But it
      turns out to also be useful for creating a programmable traffic generator
      in XDP, as well as injecting frames into the stack. A sample traffic
      generator, which was included in previous versions of the series, but now
      moved to xdp-tools, transmits up to 9 Mpps/core on my test machine.
      
      To transmit the frames, the new mode instantiates a page_pool structure in
      bpf_prog_run() and initialises the pages to contain XDP frames with the
      data passed in by userspace. These frames can then be handled as though
      they came from the hardware XDP path, and the existing page_pool code takes
      care of returning and recycling them. The setup is optimised for high
      performance with a high number of repetitions to support stress testing and
      the traffic generator use case; see patch 1 for details.
      
      v11:
      - Fix override of return code in xdp_test_run_batch()
      - Add Martin's ACKs to remaining patches
      
      v10:
      - Only propagate memory allocation errors from xdp_test_run_batch()
      - Get rid of BPF_F_TEST_XDP_RESERVED; batch_size can be used to probe
      - Check that batch_size is unset in non-XDP test_run funcs
      - Lower the number of repetitions in the selftest to 10k
      - Count number of recycled pages in the selftest
      - Fix a few other nits from Martin, carry forward ACKs
      
      v9:
      - XDP_DROP packets in the selftest to ensure pages are recycled
      - Fix a few issues reported by the kernel test robot
      - Rewrite the documentation of the batch size to make it a bit clearer
      - Rebase to newest bpf-next
      
      v8:
      - Make the batch size configurable from userspace
      - Don't interrupt the packet loop on errors in do_redirect (this can be
        caught from the tracepoint)
      - Add documentation of the feature
      - Add reserved flag userspace can use to probe for support (kernel didn't
        check flags previously)
      - Rebase to newest bpf-next, disallow live mode for jumbo frames
      
      v7:
      - Extend the local_bh_disable() to cover the full test run loop, to prevent
        running concurrently with the softirq. Fixes a deadlock with veth xmit.
      - Reinstate the forwarding sysctl setting in the selftest, and bump up the
        number of packets being transmitted to trigger the above bug.
      - Update commit message to make it clear that user space can select the
        ingress interface.
      
      v6:
      - Fix meta vs data pointer setting and add a selftest for it
      - Add local_bh_disable() around code passing packets up the stack
      - Create a new netns for the selftest and use a TC program instead of the
        forwarding hack to count packets being XDP_PASS'ed from the test prog.
      - Check for the correct ingress ifindex in the selftest
      - Rebase and drop patches 1-5 that were already merged
      
      v5:
      - Rebase to current bpf-next
      
      v4:
      - Fix a few code style issues (Alexei)
      - Also handle the other return codes: XDP_PASS builds skbs and injects them
        into the stack, and XDP_TX is turned into a redirect out the same
        interface (Alexei).
      - Drop the last patch adding an xdp_trafficgen program to samples/bpf; this
        will live in xdp-tools instead (Alexei).
      - Add a separate bpf_test_run_xdp_live() function to test_run.c instead of
        entangling the new mode in the existing bpf_test_run().
      
      v3:
      - Reorder patches to make sure they all build individually (Patchwork)
      - Remove a couple of unused variables (Patchwork)
      - Remove unlikely() annotation in slow path and add back John's ACK that I
        accidentally dropped for v2 (John)
      
      v2:
      - Split up up __xdp_do_redirect to avoid passing two pointers to it (John)
      - Always reset context pointers before each test run (John)
      - Use get_mac_addr() from xdp_sample_user.h instead of rolling our own (Kumar)
      - Fix wrong offset for metadata pointer
      ====================
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      de55c9a1
    • Toke Høiland-Jørgensen's avatar
      selftests/bpf: Add selftest for XDP_REDIRECT in BPF_PROG_RUN · 55fcacca
      Toke Høiland-Jørgensen authored
      This adds a selftest for the XDP_REDIRECT facility in BPF_PROG_RUN, that
      redirects packets into a veth and counts them using an XDP program on the
      other side of the veth pair and a TC program on the local side of the veth.
      Signed-off-by: default avatarToke Høiland-Jørgensen <toke@redhat.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Link: https://lore.kernel.org/bpf/20220309105346.100053-6-toke@redhat.com
      55fcacca
    • Toke Høiland-Jørgensen's avatar
      selftests/bpf: Move open_netns() and close_netns() into network_helpers.c · a3033884
      Toke Høiland-Jørgensen authored
      These will also be used by the xdp_do_redirect test being added in the next
      commit.
      Signed-off-by: default avatarToke Høiland-Jørgensen <toke@redhat.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Link: https://lore.kernel.org/bpf/20220309105346.100053-5-toke@redhat.com
      a3033884
    • Toke Høiland-Jørgensen's avatar
      libbpf: Support batch_size option to bpf_prog_test_run · 24592ad1
      Toke Høiland-Jørgensen authored
      Add support for setting the new batch_size parameter to BPF_PROG_TEST_RUN
      to libbpf; just add it as an option and pass it through to the kernel.
      Signed-off-by: default avatarToke Høiland-Jørgensen <toke@redhat.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Link: https://lore.kernel.org/bpf/20220309105346.100053-4-toke@redhat.com
      24592ad1
    • Toke Høiland-Jørgensen's avatar
      Documentation/bpf: Add documentation for BPF_PROG_RUN · 1a7551f1
      Toke Høiland-Jørgensen authored
      This adds documentation for the BPF_PROG_RUN command; a short overview of
      the command itself, and a more verbose description of the "live packet"
      mode for XDP introduced in the previous commit.
      Signed-off-by: default avatarToke Høiland-Jørgensen <toke@redhat.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Link: https://lore.kernel.org/bpf/20220309105346.100053-3-toke@redhat.com
      1a7551f1
    • Toke Høiland-Jørgensen's avatar
      bpf: Add "live packet" mode for XDP in BPF_PROG_RUN · b530e9e1
      Toke Høiland-Jørgensen authored
      This adds support for running XDP programs through BPF_PROG_RUN in a mode
      that enables live packet processing of the resulting frames. Previous uses
      of BPF_PROG_RUN for XDP returned the XDP program return code and the
      modified packet data to userspace, which is useful for unit testing of XDP
      programs.
      
      The existing BPF_PROG_RUN for XDP allows userspace to set the ingress
      ifindex and RXQ number as part of the context object being passed to the
      kernel. This patch reuses that code, but adds a new mode with different
      semantics, which can be selected with the new BPF_F_TEST_XDP_LIVE_FRAMES
      flag.
      
      When running BPF_PROG_RUN in this mode, the XDP program return codes will
      be honoured: returning XDP_PASS will result in the frame being injected
      into the networking stack as if it came from the selected networking
      interface, while returning XDP_TX and XDP_REDIRECT will result in the frame
      being transmitted out that interface. XDP_TX is translated into an
      XDP_REDIRECT operation to the same interface, since the real XDP_TX action
      is only possible from within the network drivers themselves, not from the
      process context where BPF_PROG_RUN is executed.
      
      Internally, this new mode of operation creates a page pool instance while
      setting up the test run, and feeds pages from that into the XDP program.
      The setup cost of this is amortised over the number of repetitions
      specified by userspace.
      
      To support the performance testing use case, we further optimise the setup
      step so that all pages in the pool are pre-initialised with the packet
      data, and pre-computed context and xdp_frame objects stored at the start of
      each page. This makes it possible to entirely avoid touching the page
      content on each XDP program invocation, and enables sending up to 9
      Mpps/core on my test box.
      
      Because the data pages are recycled by the page pool, and the test runner
      doesn't re-initialise them for each run, subsequent invocations of the XDP
      program will see the packet data in the state it was after the last time it
      ran on that particular page. This means that an XDP program that modifies
      the packet before redirecting it has to be careful about which assumptions
      it makes about the packet content, but that is only an issue for the most
      naively written programs.
      
      Enabling the new flag is only allowed when not setting ctx_out and data_out
      in the test specification, since using it means frames will be redirected
      somewhere else, so they can't be returned.
      Signed-off-by: default avatarToke Høiland-Jørgensen <toke@redhat.com>
      Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
      Acked-by: default avatarMartin KaFai Lau <kafai@fb.com>
      Link: https://lore.kernel.org/bpf/20220309105346.100053-2-toke@redhat.com
      b530e9e1
    • Andrii Nakryiko's avatar
      Merge branch 'BPF test_progs tests improvement' · 3399dd9f
      Andrii Nakryiko authored
      Mykola Lysenko says:
      
      ====================
      
      First patch reduces the sample_freq to 1000 to ensure test will
      work even when kernel.perf_event_max_sample_rate was reduced to 1000.
      
      Patches for send_signal and find_vma tune the test implementation to
      make sure needed thread is scheduled. Also, both tests will finish as
      soon as possible after the test condition is met.
      ====================
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      3399dd9f
    • Mykola Lysenko's avatar
      Improve stability of find_vma BPF test · ba83af05
      Mykola Lysenko authored
      Remove unneeded spleep and increase length of dummy CPU
      intensive computation to guarantee test process execution.
      Also, complete aforemention computation as soon as
      test success criteria is met
      Signed-off-by: default avatarMykola Lysenko <mykolal@fb.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20220308200449.1757478-4-mykolal@fb.com
      ba83af05
    • Mykola Lysenko's avatar
      Improve send_signal BPF test stability · 1fd49864
      Mykola Lysenko authored
      Substitute sleep with dummy CPU intensive computation.
      Finish aforemention computation as soon as signal was
      delivered to the test process. Make the BPF code to
      only execute when PID global variable is set
      Signed-off-by: default avatarMykola Lysenko <mykolal@fb.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20220308200449.1757478-3-mykolal@fb.com
      1fd49864
    • Mykola Lysenko's avatar
      Improve perf related BPF tests (sample_freq issue) · d4b54054
      Mykola Lysenko authored
      Linux kernel may automatically reduce kernel.perf_event_max_sample_rate
      value when running tests in parallel on slow systems. Linux kernel checks
      against this limit when opening perf event with freq=1 parameter set.
      The lower bound is 1000. This patch reduces sample_freq value to 1000
      in all BPF tests that use sample_freq to ensure they always can open
      perf event.
      Signed-off-by: default avatarMykola Lysenko <mykolal@fb.com>
      Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
      Acked-by: default avatarYonghong Song <yhs@fb.com>
      Link: https://lore.kernel.org/bpf/20220308200449.1757478-2-mykolal@fb.com
      d4b54054
  4. 08 Mar, 2022 5 commits