- 27 Jan, 2022 9 commits
-
-
Maciej Fijalkowski authored
Apply the logic that was done for regular XDP from commit 9610bd98 ("ice: optimize XDP_TX workloads") to the ZC side of the driver. On top of that, introduce batching to Tx that is inspired by i40e's implementation with adjustments to the cleaning logic - take into the account NAPI budget in ice_clean_xdp_irq_zc(). Separating the stats structs onto separate cache lines seemed to improve the performance. Signed-off-by:
Maciej Fijalkowski <maciej.fijalkowski@intel.com> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Reviewed-by:
Alexander Lobakin <alexandr.lobakin@intel.com> Link: https://lore.kernel.org/bpf/20220125160446.78976-8-maciej.fijalkowski@intel.com
-
Maciej Fijalkowski authored
Commit 9610bd98 ("ice: optimize XDP_TX workloads") introduced @next_dd and @next_rs to ice_tx_ring struct. Currently, their state is not restored in ice_clean_tx_ring(), which was not causing any troubles as the XDP rings are gone after we're done with XDP prog on interface. For upcoming usage of mentioned fields in AF_XDP, this might expose us to a potential dead Tx side. Scenario would look like following (based on xdpsock): - two xdpsock instances are spawned in Tx mode - one of them is killed - XDP prog is kept on interface due to the other xdpsock still running * this means that XDP rings stayed in place - xdpsock is launched again on same queue id that was terminated on - @next_dd and @next_rs setting is bogus, therefore transmit side is broken To protect us from the above, restore the initial @next_rs and @next_dd values when cleaning the Tx ring. Signed-off-by:
Magnus Karlsson <magnus.karlsson@intel.com> Link: https://lore.kernel.org/bpf/20220125160446.78976-7-maciej.fijalkowski@intel.com
-
Magnus Karlsson authored
Move desc_array from the driver to the pool. The reason behind this is that we can then reuse this array as a temporary storage for descriptors in all zero-copy drivers that use the batched interface. This will make it easier to add batching to more drivers. i40e is the only driver that has a batched Tx zero-copy implementation, so no need to touch any other driver. Signed-off-by:
-
Maciej Fijalkowski authored
XDP_TX workloads use a concept of Tx threshold that indicates the interval of setting RS bit on descriptors which in turn tells the HW to generate an interrupt to signal the completion of Tx on HW side. It is currently based on a constant value of 32 which might not work out well for various sizes of ring combined with for example batch size that can be set via SO_BUSY_POLL_BUDGET. Internal tests based on AF_XDP showed that most convenient setup of mentioned threshold is when it is equal to quarter of a ring length. Make use of recently introduced ICE_RING_QUARTER macro and use this value as a substitute for ICE_TX_THRESH. Align also ethtool -G callback so that next_dd/next_rs fields are up to date in terms of the ring size. Signed-off-by:
-
Maciej Fijalkowski authored
Currently, if ice_clean_rx_irq_zc() processed the whole ring and next_to_use != 0, then ice_alloc_rx_buf_zc() would not refill the whole ring even if the XSK buffer pool would have enough free entries (either from fill ring or the internal recycle mechanism) - it is because ring wrap is not handled. Improve the logic in ice_alloc_rx_buf_zc() to address the problem above. Do not clamp the count of buffers that is passed to xsk_buff_alloc_batch() in case when next_to_use + buffer count >= rx_ring->count, but rather split it and have two calls to the mentioned function - one for the part up until the wrap and one for the part after the wrap. Signed-off-by:
-
Maciej Fijalkowski authored
With the upcoming introduction of batching to XSK data path, performance wise it will be the best to have the ring descriptor count to be aligned to power of 2. Check if ring sizes that user is going to attach the XSK socket fulfill the condition above. For Tx side, although check is being done against the Tx queue and in the end the socket will be attached to the XDP queue, it is fine since XDP queues get the ring->count setting from Tx queues. Suggested-by:
-
Maciej Fijalkowski authored
Remove the likely before napi_complete_done as this is the unlikely case when busy-poll is used. Removing this has a positive performance impact for busy-poll and no negative impact to the regular case. Signed-off-by:
-
Jakub Kicinski authored
Remove two dead stubs, sk_msg_clear_meta() was never used, use of xskq_cons_is_full() got replaced by xsk_tx_writeable() in v5.10. Signed-off-by:
Jakub Kicinski <kuba@kernel.org> Link: https://lore.kernel.org/r/20220126185412.2776254-1-kuba@kernel.orgSigned-off-by:
Alexei Starovoitov <ast@kernel.org>
-
Andrii Nakryiko authored
Fix how selftests determine relative offset of a function that is uprobed. Previously, there was an assumption that uprobed function is always in the first executable region, which is not always the case (libbpf CI hits this case now). So get_base_addr() approach in isolation doesn't work anymore. So teach get_uprobe_offset() to determine correct memory mapping and calculate uprobe offset correctly. While at it, I merged together two implementations of get_uprobe_offset() helper, moving powerpc64-specific logic inside (had to add extra {} block to avoid unused variable error for insn). Also ensured that uprobed functions are never inlined, but are still static (and thus local to each selftest), by using a no-op asm volatile block internally. I didn't want to keep them global __weak, because some tests use uprobe's ref counter offset (to test USDT-like logic) which is not compatible with non-refcounted uprobe. So it's nicer to have each test uprobe target local to the file and guaranteed to not be inlined or skipped by the compiler (which can happen with static functions, especially if compiling selftests with -O2). Signed-off-by:Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20220126193058.3390292-1-andrii@kernel.orgSigned-off-by:
-
- 26 Jan, 2022 11 commits
-
-
Yonghong Song authored
Compiling kernel and selftests/bpf with latest llvm like blow: make -j LLVM=1 make -C tools/testing/selftests/bpf -j LLVM=1 I hit the following compilation error: /.../prog_tests/log_buf.c:215:6: error: variable 'log_buf' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (!ASSERT_OK_PTR(raw_btf_data, "raw_btf_data_good")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /.../prog_tests/log_buf.c:264:7: note: uninitialized use occurs here free(log_buf); ^~~~~~~ /.../prog_tests/log_buf.c:215:2: note: remove the 'if' if its condition is always false if (!ASSERT_OK_PTR(raw_btf_data, "raw_btf_data_good")) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /.../prog_tests/log_buf.c:205:15: note: initialize the variable 'log_buf' to silence this warning char *log_buf; ^ = NULL 1 error generated. Compiler rightfully detected that log_buf is uninitialized in one of failure path as indicated in the above. Proper initialization of 'log_buf' variable fixed the issue. Signed-off-by:Yonghong Song <yhs@fb.com> Signed-off-by:
-
Stanislav Fomichev authored
Commit dee872e1 ("bpf: Populate kfunc BTF ID sets in struct btf") breaks loading of some modules when CONFIG_DEBUG_INFO_BTF is not set. register_btf_kfunc_id_set returns -ENOENT to the callers when there is no module btf. Let's return 0 (success) instead to let those modules work in !CONFIG_DEBUG_INFO_BTF cases. Acked-by:
Kumar Kartikeya Dwivedi <memxor@gmail.com> Fixes: dee872e1 ("bpf: Populate kfunc BTF ID sets in struct btf") Signed-off-by:
Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20220126001340.1573649-1-sdf@google.comSigned-off-by:
-
Felix Maurer authored
Originally, the kernel strictly checked the size of the optval in getsockopt(TCP_ZEROCOPY_RECEIVE) to be equal to sizeof(struct tcp_zerocopy_receive). With c8856c05, this was changed to allow optvals of different sizes. The bpf code in the sockopt_sk test was still performing the strict size check. This fix adapts the kernel behavior from c8856c05 in the selftest, i.e., just check if the required fields are there. Fixes: 9cacf81f ("bpf: Remove extra lock_sock for TCP_ZEROCOPY_RECEIVE") Signed-off-by:
Felix Maurer <fmaurer@redhat.com> Reviewed-by:
-
Alexei Starovoitov authored
Andrii Nakryiko says: ==================== Another batch of simple deprecations. One of the last few, hopefully, as we are getting close to deprecating all the planned APIs/features. See individual patches for details. v1->v2: - rebased on latest bpf-next, fixed Closes: reference. ==================== Signed-off-by: -
Andrii Nakryiko authored
bpf_program__set_<type>() APIs are deprecated, use generic bpf_program__set_type() APIs instead. Signed-off-by:
-
Andrii Nakryiko authored
Use preferred setter and getter APIs instead of deprecated ones. Signed-off-by:
-
Andrii Nakryiko authored
Switch to using preferred setters and getters instead of deprecated ones. Signed-off-by:
-
Andrii Nakryiko authored
Use bpf_program__type() instead of discouraged bpf_program__get_type(). Also switch to bpf_map__set_max_entries() instead of bpf_map__resize(). Signed-off-by:
-
Andrii Nakryiko authored
Not sure why these APIs were added in the first place instead of a completely generic (and not requiring constantly adding new APIs with each new BPF program type) bpf_program__type() and bpf_program__set_type() APIs. But as it is right now, there are 13 such specialized is_type/set_type APIs, while latest kernel is already at 30+ BPF program types. Instead of completing the set of APIs and keep chasing kernel's bpf_prog_type enum, deprecate existing subset and recommend generic bpf_program__type() and bpf_program__set_type() APIs. Signed-off-by:
-
Andrii Nakryiko authored
Deprecated bpf_map__resize() in favor of bpf_map__set_max_entries() setter. In addition to having a surprising name (users often don't realize that they need to use bpf_map__resize()), the name also implies some magic way of resizing BPF map after it is created, which is clearly not the case. Another minor annoyance is that bpf_map__resize() disallows 0 value for max_entries, which in some cases is totally acceptable (e.g., like for BPF perf buf case to let libbpf auto-create one buffer per each available CPU core). [0] Closes: https://github.com/libbpf/libbpf/issues/304Signed-off-by:
-
Andrii Nakryiko authored
Move a bunch of "getters" into libbpf_legacy.h to keep them there in libbpf 1.0. See [0] for discussion of "Discouraged APIs". These getters don't add any maintenance burden and are simple alias, but they are inconsistent in naming. So keep them in libbpf_legacy.h instead of libbpf.h to "hide" them in favor of preferred getters ([1]). Also add two missing getters: bpf_program__type() and bpf_program__expected_attach_type(). [0] https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#handling-deprecation-of-apis-and-functionality [1] Closes: https://github.com/libbpf/libbpf/issues/307Signed-off-by:
-
- 25 Jan, 2022 13 commits
-
-
Andrii Nakryiko authored
Kenta Tada says: ==================== Currently, rcx is read as the fourth parameter of syscall on x86_64. But x86_64 Linux System Call convention uses r10 actually. This commit adds the wrapper for users who want to access to syscall params to analyze the user space. Changelog: ---------- v1 -> v2: - Rebase to current bpf-next https://lore.kernel.org/bpf/20211222213924.1869758-1-andrii@kernel.org/ v2 -> v3: - Modify the definition of SYSCALL macros for only targeted archs. - Define __BPF_TARGET_MISSING variants for completeness. - Remove CORE variants. These macros will not be used. - Add a selftest. v3 -> v4: - Modify a selftest not to use serial tests. - Modify a selftest to use ASSERT_EQ(). - Extract syscall wrapper for all the other tests. - Add CORE variants. v4 -> v5: - Modify the CORE variant macro not to read memory directly. - Remove the unnecessary comment. - Add a selftest for the CORE variant. ==================== Signed-off-by:
-
Kenta Tada authored
Add a selftest to verify the behavior of PT_REGS_xxx and the CORE variant. Signed-off-by:
Kenta Tada <Kenta.Tada@sony.com> Signed-off-by:
-
Kenta Tada authored
Currently, rcx is read as the fourth parameter of syscall on x86_64. But x86_64 Linux System Call convention uses r10 actually. This commit adds the wrapper for users who want to access to syscall params to analyze the user space. Signed-off-by:
-
Kenta Tada authored
Extract the helper to set up SYS_PREFIX for fentry and kprobe selftests that use __x86_sys_* attach functions. Suggested-by:
-
Christy Lee authored
Mark bpf_object__open_xattr() as deprecated, use bpf_object__open_file() instead. [0] Closes: https://github.com/libbpf/libbpf/issues/287Signed-off-by:
Christy Lee <christylee@fb.com> Signed-off-by:
-
Andrii Nakryiko authored
Christy Lee says: ==================== Deprecate bpf_object__open_buffer() API, replace all usage with bpf_object__open_mem(). ==================== Signed-off-by: -
Christy Lee authored
bpf_object__open_buffer() API is deprecated, use the unified opts bpf_object__open_mem() API in perf instead. This requires at least libbpf 0.0.6. Signed-off-by:
-
Christy Lee authored
Deprecate bpf_object__open_buffer() API in favor of the unified opts-based bpf_object__open_mem() API. Signed-off-by:
-
Alexei Starovoitov authored
Kenny Yu says: ==================== This patch series makes the following changes: * Adds a new bpf helper `bpf_copy_from_user_task` to read user space memory from a different task. * Adds the ability to create sleepable bpf iterator programs. As an example of how this will be used, at Meta we are using bpf task iterator programs and this new bpf helper to read C++ async stack traces of a running process for debugging C++ binaries in production. Changes since v6: * Split first patch into two patches: first patch to add support for bpf iterators to use sleepable helpers, and the second to add the new bpf helper. * Simplify implementation of `bpf_copy_from_user_task` based on feedback. * Add to docs that the destination buffer will be zero-ed on error. Changes since v5: * Rename `bpf_access_process_vm` to `bpf_copy_from_user_task`. * Change return value to be all-or-nothing. If we get a partial read, memset all bytes to 0 and return -EFAULT. * Add to docs that the helper can only be used by sleepable BPF programs. * Fix nits in selftests. Changes since v4: * Make `flags` into u64. * Use `user_ptr` arg name to be consistent with `bpf_copy_from_user`. * Add an extra check in selftests to verify access_process_vm calls succeeded. Changes since v3: * Check if `flags` is 0 and return -EINVAL if not. * Rebase on latest bpf-next branch and fix merge conflicts. Changes since v2: * Reorder arguments in `bpf_access_process_vm` to match existing related bpf helpers (e.g. `bpf_probe_read_kernel`, `bpf_probe_read_user`, `bpf_copy_from_user`). * `flags` argument is provided for future extensibility and is not currently used, and we always invoke `access_process_vm` with no flags. * Merge bpf helper patch and `bpf_iter_run_prog` patch together for better bisectability in case of failures. * Clean up formatting and comments in selftests. Changes since v1: * Fixed "Invalid wait context" issue in `bpf_iter_run_prog` by using `rcu_read_lock_trace()` for sleepable bpf iterator programs. ==================== Signed-off-by: -
Kenny Yu authored
This adds a test for bpf iterator programs to make use of sleepable bpf helpers. Signed-off-by:
Kenny Yu <kennyyu@fb.com> Acked-by:
-
Kenny Yu authored
This adds a new bpf section "iter.s" to allow bpf iterator programs to be sleepable. Signed-off-by:
-
Kenny Yu authored
This adds a helper for bpf programs to read the memory of other tasks. As an example use case at Meta, we are using a bpf task iterator program and this new helper to print C++ async stack traces for all threads of a given process. Signed-off-by:
-
Kenny Yu authored
This patch allows bpf iterator programs to use sleepable helpers by changing `bpf_iter_run_prog` to use the appropriate synchronization. With sleepable bpf iterator programs, we can no longer use `rcu_read_lock()` and must use `rcu_read_lock_trace()` instead to protect the bpf program. Signed-off-by:
-
- 24 Jan, 2022 7 commits
-
-
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-nextJakub Kicinski authored
Daniel Borkmann says: ==================== pull-request: bpf-next 2022-01-24 We've added 80 non-merge commits during the last 14 day(s) which contain a total of 128 files changed, 4990 insertions(+), 895 deletions(-). The main changes are: 1) Add XDP multi-buffer support and implement it for the mvneta driver, from Lorenzo Bianconi, Eelco Chaudron and Toke Høiland-Jørgensen. 2) Add unstable conntrack lookup helpers for BPF by using the BPF kfunc infra, from Kumar Kartikeya Dwivedi. 3) Extend BPF cgroup programs to export custom ret value to userspace via two helpers bpf_get_retval() and bpf_set_retval(), from YiFei Zhu. 4) Add support for AF_UNIX iterator batching, from Kuniyuki Iwashima. 5) Complete missing UAPI BPF helper description and change bpf_doc.py script to enforce consistent & complete helper documentation, from Usama Arif. 6) Deprecate libbpf's legacy BPF map definitions and streamline XDP APIs to follow tc-based APIs, from Andrii Nakryiko. 7) Support BPF_PROG_QUERY for BPF programs attached to sockmap, from Di Zhu. 8) Deprecate libbpf's bpf_map__def() API and replace users with proper getters and setters, from Christy Lee. 9) Extend libbpf's btf__add_btf() with an additional hashmap for strings to reduce overhead, from Kui-Feng Lee. 10) Fix bpftool and libbpf error handling related to libbpf's hashmap__new() utility function, from Mauricio Vásquez. 11) Add support to BTF program names in bpftool's program dump, from Raman Shukhau. 12) Fix resolve_btfids build to pick up host flags, from Connor O'Brien. * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (80 commits) selftests, bpf: Do not yet switch to new libbpf XDP APIs selftests, xsk: Fix rx_full stats test bpf: Fix flexible_array.cocci warnings xdp: disable XDP_REDIRECT for xdp frags bpf: selftests: add CPUMAP/DEVMAP selftests for xdp frags bpf: selftests: introduce bpf_xdp_{load,store}_bytes selftest net: xdp: introduce bpf_xdp_pointer utility routine bpf: generalise tail call map compatibility check libbpf: Add SEC name for xdp frags programs bpf: selftests: update xdp_adjust_tail selftest to include xdp frags bpf: test_run: add xdp_shared_info pointer in bpf_test_finish signature bpf: introduce frags support to bpf_prog_test_run_xdp() bpf: move user_size out of bpf_test_init bpf: add frags support to xdp copy helpers bpf: add frags support to the bpf_xdp_adjust_tail() API bpf: introduce bpf_xdp_get_buff_len helper net: mvneta: enable jumbo frames if the loaded XDP program support frags bpf: introduce BPF_F_XDP_HAS_FRAGS flag in prog_flags loading the ebpf program net: mvneta: add frags support to XDP_TX xdp: add frags support to xdp_return_{buff/frame} ... ==================== Link: https://lore.kernel.org/r/20220124221235.18993-1-daniel@iogearbox.netSigned-off-by: -
Daniel Borkmann authored
Revert commit 54435652 ("selftests/bpf: switch to new libbpf XDP APIs") for now given this will heavily conflict with 4b27480d ("bpf/selftests: convert xdp_link test to ASSERT_* macros") upon merge. Andrii agreed to redo the conversion cleanly after trees merged. Signed-off-by:
-
Jakub Kicinski authored
Merge tag 'linux-can-fixes-for-5.17-20220124' of git://git.kernel.org/pub/scm/linux/kernel/git/mkl/linux-can Marc Kleine-Budde says: ==================== pull-request: can 2022-01-24 The first patch updates the email address of Brian Silverman from his former employer to his private address. The next patch fixes DT bindings information for the tcan4x5x SPI CAN driver. The following patch targets the m_can driver and fixes the introduction of FIFO bulk read support. Another patch for the tcan4x5x driver, which fixes the max register value for the regmap config. The last patch for the flexcan driver marks the RX mailbox support for the MCF5441X as support. * tag 'linux-can-fixes-for-5.17-20220124' of git://git.kernel.org/pub/scm/linux/kernel/git/mkl/linux-can: can: flexcan: mark RX via mailboxes as supported on MCF5441X can: tcan4x5x: regmap: fix max register value can: m_can: m_can_fifo_{read,write}: don't read or write from/to FIFO if length is 0 dt-bindings: can: tcan4x5x: fix mram-cfg RX FIFO config mailmap: update email address of Brian Silverman ==================== Link: https://lore.kernel.org/r/20220124175955.3464134-1-mkl@pengutronix.deSigned-off-by:
-
Marc Kleine-Budde authored
Most flexcan IP cores support 2 RX modes: - FIFO - mailbox The flexcan IP core on the MCF5441X cannot receive CAN RTR messages via mailboxes. However the mailbox mode is more performant. The commit | 1c45f577 ("can: flexcan: add ethtool support to change rx-rtr setting during runtime") added support to switch from FIFO to mailbox mode on these cores. After testing the mailbox mode on the MCF5441X by Angelo Dureghello, this patch marks it (without RTR capability) as supported. Further the IP core overview table is updated, that RTR reception via mailboxes is not supported. Link: https://lore.kernel.org/all/20220121084425.3141218-1-mkl@pengutronix.deTested-by:
Angelo Dureghello <angelo@kernel-space.org> Signed-off-by:
Marc Kleine-Budde <mkl@pengutronix.de>
-
Marc Kleine-Budde authored
The MRAM of the tcan4x5x has a size of 2K and starts at 0x8000. There are no further registers in the tcan4x5x making 0x87fc the biggest addressable register. This patch fixes the max register value of the regmap config from 0x8ffc to 0x87fc. Fixes: 6e1caaf8 ("can: tcan4x5x: fix max register value") Link: https://lore.kernel.org/all/20220119064011.2943292-1-mkl@pengutronix.deSigned-off-by:
-
Marc Kleine-Budde authored
In order to optimize FIFO access, especially on m_can cores attached to slow busses like SPI, in patch | e3938177 ("can: m_can: Disable IRQs on FIFO bus errors") bulk read/write support has been added to the m_can_fifo_{read,write} functions. That change leads to the tcan driver to call regmap_bulk_{read,write}() with a length of 0 (for CAN frames with 0 data length). regmap treats this as an error: | tcan4x5x spi1.0 tcan4x5x0: FIFO write returned -22 This patch fixes the problem by not calling the cdev->ops->{read,write)_fifo() in case of a 0 length read/write. Fixes: e3938177 ("can: m_can: Disable IRQs on FIFO bus errors") Link: https://lore.kernel.org/all/20220114155751.2651888-1-mkl@pengutronix.de Cc: stable@vger.kernel.org Cc: Matt Kline <matt@bitbashing.io> Cc: Chandrasekar Ramakrishnan <rcsekar@samsung.com> Reported-by:
Michael Anochin <anochin@photo-meter.com> Signed-off-by:
-
Marc Kleine-Budde authored
This tcan4x5x only comes with 2K of MRAM, a RX FIFO with a dept of 32 doesn't fit into the MRAM. Use a depth of 16 instead. Fixes: 4edd396a ("dt-bindings: can: tcan4x5x: Add DT bindings for TCAN4x5X driver") Link: https://lore.kernel.org/all/20220119062951.2939851-1-mkl@pengutronix.deSigned-off-by:
-
