- 12 May, 2022 3 commits
-
-
Alexei Starovoitov authored
Feng zhou says: ==================== From: Feng Zhou <zhoufeng.zf@bytedance.com> Trace some functions, such as enqueue_task_fair, need to access the corresponding cpu, not the current cpu, and bpf_map_lookup_elem percpu map cannot do it. So add bpf_map_lookup_percpu_elem to accomplish it for percpu_array_map, percpu_hash_map, lru_percpu_hash_map. v1->v2: Addressed comments from Alexei Starovoitov. - add a selftest for bpf_map_lookup_percpu_elem. ==================== Signed-off-by:Alexei Starovoitov <ast@kernel.org>
-
Feng Zhou authored
test_progs: Tests new ebpf helpers bpf_map_lookup_percpu_elem. Signed-off-by:
Feng Zhou <zhoufeng.zf@bytedance.com> Link: https://lore.kernel.org/r/20220511093854.411-3-zhoufeng.zf@bytedance.comSigned-off-by:
-
Feng Zhou authored
Add new ebpf helpers bpf_map_lookup_percpu_elem. The implementation method is relatively simple, refer to the implementation method of map_lookup_elem of percpu map, increase the parameters of cpu, and obtain it according to the specified cpu. Signed-off-by:
-
- 11 May, 2022 25 commits
-
-
Alexei Starovoitov authored
Kumar Kartikeya Dwivedi says: ==================== Fix a build time warning, and address comments from Alexei on the merged version [0]. [0]: https://lore.kernel.org/bpf/20220424214901.2743946-1-memxor@gmail.com Changelog: ---------- v1 -> v2 v1: https://lore.kernel.org/bpf/20220510211727.575686-1-memxor@gmail.com * Add Fixes tag to patch 1 * Fix test_progs-noalu32 failure in CI due to different alloc_insn (Alexei) * Remove per-CPU struct, use global struct (Alexei) ==================== Signed-off-by:
-
Kumar Kartikeya Dwivedi authored
Check at runtime how various operations for kptr_ref affect its refcount and verify against the actual count. Signed-off-by:
Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20220511194654.765705-5-memxor@gmail.comSigned-off-by:
-
Kumar Kartikeya Dwivedi authored
This uses the newly added SEC("?foo") naming to disable autoload of programs, and then loads them one by one for the object and verifies that loading fails and matches the returned error string from verifier. This is similar to already existing verifier tests but provides coverage for BPF C. Signed-off-by: -
Kumar Kartikeya Dwivedi authored
In an effort to actually test the refcounting logic at runtime, add a refcount_t member to prog_test_ref_kfunc and use it in selftests to verify and test the whole logic more exhaustively. The kfunc calls for prog_test_member do not require runtime refcounting, as they are only used for verifier selftests, not during runtime execution. Hence, their implementation now has a WARN_ON_ONCE as it is not meant to be reachable code at runtime. It is strictly used in tests triggering failure cases in the verifier. bpf_kfunc_call_memb_release is called from map free path, since prog_test_member is embedded in map value for some verifier tests, so we skip WARN_ON_ONCE for it. Signed-off-by:
-
Kumar Kartikeya Dwivedi authored
Kernel Test Robot complained about missing static storage class annotation for bpf_kptr_xchg_proto variable. sparse: symbol 'bpf_kptr_xchg_proto' was not declared. Should it be static? This caused by missing extern definition in the header. Add it to suppress the sparse warning. Fixes: c0a5a21c ("bpf: Allow storing referenced kptr in map") Reported-by:
kernel test robot <lkp@intel.com> Signed-off-by:
-
Yonghong Song authored
With latest clang, I got the following compilation errors: .../prog_tests/test_tunnel.c:291:6: error: variable 'local_ip_map_fd' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (attach_tc_prog(&tc_hook, -1, set_dst_prog_fd)) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .../bpf/prog_tests/test_tunnel.c:312:6: note: uninitialized use occurs here if (local_ip_map_fd >= 0) ^~~~~~~~~~~~~~~ ... .../prog_tests/kprobe_multi_test.c:346:6: error: variable 'err' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized] if (IS_ERR(map)) ^~~~~~~~~~~ .../prog_tests/kprobe_multi_test.c:388:6: note: uninitialized use occurs here if (err) { ^~~ This patch fixed the above compilation errors. Signed-off-by:Yonghong Song <yhs@fb.com> Acked-by:
David Vernet <void@manifault.com> Link: https://lore.kernel.org/r/20220511184735.3670214-1-yhs@fb.comSigned-off-by:
-
Daniel Müller authored
Some of the BPF selftests are failing when running with a rather bare bones configuration based on tools/testing/selftests/bpf/config. Specifically, we see a bunch of failures due to errno 95: > test_attach_api:PASS:fentry_raw_skel_load 0 nsec > libbpf: prog 'test_kprobe_manual': failed to attach: Operation not supported > test_attach_api:FAIL:bpf_program__attach_kprobe_multi_opts unexpected error: -95 > 79 /6 kprobe_multi_test/attach_api_syms:FAIL The cause of these is that CONFIG_FPROBE is missing. With this change we add this configuration value to the BPF selftests config. Signed-off-by:
Daniel Müller <deso@posteo.net> Acked-by:
-
Alexei Starovoitov authored
Magnus Karlsson says: ==================== This patch set adds busy-poll testing to the xsk selftests. It runs exactly the same tests as with regular softirq processing, but with busy-poll enabled. I have also included a number of fixes to the selftests that have been bugging me for a while or was discovered while implementing the busy-poll support. In summary these are: * Fix the error reporting of failed tests. Each failed test used to be reported as both failed and passed, messing up things. * Added a summary test printout at the end of the test suite so that users do not have to scroll up and look at the result of both the softirq run and the busy_poll run. * Added a timeout to the tests, so that if a test locks up, we report a fail and still get to run all the other tests. * Made the stats test just look and feel like all the other tests. Makes the code simpler and the test reporting more consistent. These are the 3 last commits. * Replaced zero length packets with packets of 64 byte length. This so that some of the tests will pass after commit 726e2c59 ("veth: Ensure eth header is in skb's linear part"). * Added clean-up of the veth pair when terminating the test run. * Some smaller clean-ups of unused stuff. Note, to pass the busy-poll tests commit 8de8b71b ("xsk: Fix l2fwd for copy mode + busy poll combo") need to be present. It is present in bpf but not yet in bpf-next. Thanks: Magnus ==================== Acked-by:
Björn Töpel <bjorn@kernel.org> Signed-off-by:
-
Magnus Karlsson authored
Convert the stats tests from spinning on the getsockopt to just check getsockopt once when the Rx thread has received all the packets. The actual completion of receiving the last packet forms a natural point in time when the receiver is ready to call the getsockopt to check the stats. In the previous version , we just span on the getsockopt until we received the right answer. This could be forever or just getting the "correct" answer by shear luck. The pacing_on variable can now be dropped since all test can now handle pacing properly. Signed-off-by:
Magnus Karlsson <magnus.karlsson@intel.com> Link: https://lore.kernel.org/r/20220510115604.8717-10-magnus.karlsson@gmail.comSigned-off-by:
-
Magnus Karlsson authored
Make the stats tests look and feel just like normal tests instead of bunched under the umbrella of TEST_STATS. This means we will always run each of them even if one fails. Also gets rid of some special case code. Signed-off-by:
-
Magnus Karlsson authored
Introduce validation functions that can be optionally called by the Rx and Tx threads. These are then used to replace the Rx and Tx stats dispatchers. This so that we in the next commit can make the stats tests proper normal tests and not be some special case, as today. Signed-off-by:
-
Magnus Karlsson authored
Remove the veth pair when the tests are aborted by pressing ctrl-c. Currently in this situation, the veth pair is left on the system polluting the netdev space. Signed-off-by:
-
Magnus Karlsson authored
Add a timeout to the tests so that if all packets have not been received within 3 seconds, fail the ongoing test. Hinders a test from dead-locking if there is something wrong. Signed-off-by:
-
Magnus Karlsson authored
Fix the reporting of failed tests as it was broken in several ways. First, a failed test was reported as both failed and passed messing up the count. Second, tests were not aborted after a failure and could generate more "failures" messing up the count even more. Third, the failure reporting from the application to the shell script was wrong. It always reported pass. And finally, the handling of the failures in the launch script was not correct. Correct all this by propagating the failure up through the function calls to a calling function that can abort the test. A receiver or sender thread will mark the new variable in the test spec called fail, if a test has failed. This is then picked up by the main thread when everyone else has exited and this is then marked and propagated up to the calling script. Also add a summary function in the calling script so that a user does not have to go through the sub tests to see if something has failed. Signed-off-by:
-
Magnus Karlsson authored
Execute all xsk selftests for busy-poll mode too. Currently they were only run for the standard interrupt driven softirq mode. Replace the unused option queue-id with the new option busy-poll. Signed-off-by:
-
Magnus Karlsson authored
Do not try to send packets of zero length since they are dropped by veth after commit 726e2c59 ("veth: Ensure eth header is in skb's linear part"). Replace these two packets with packets of length 60 so that they are not dropped. Also clean up the confusing naming. MIN_PKT_SIZE was really MIN_ETH_PKT_SIZE and PKT_SIZE was both MIN_ETH_SIZE and the default packet size called just PKT_SIZE. Make it consistent by using the right define in the right place. Signed-off-by:
-
Magnus Karlsson authored
Remove the spec-file that is not used any longer from the shell scripts. Also remove an unused option. Signed-off-by:
-
Jiri Olsa authored
Adding bpf_program__set_insns that allows to set new instructions for a BPF program. This is a very advanced libbpf API and users need to know what they are doing. This should be used from prog_prepare_load_fn callback only. We can have changed instructions after calling prog_prepare_load_fn callback, reloading them. One of the users of this new API will be perf's internal BPF prologue generation. Signed-off-by:
Jiri Olsa <jolsa@kernel.org> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Acked-by:
Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220510074659.2557731-2-jolsa@kernel.org
-
Andrii Nakryiko authored
Drop unused iteration variable, move overflow prevention check into the for loop. Fixes: 0087a681 ("libbpf: Automatically fix up BPF_MAP_TYPE_RINGBUF size, if necessary") Reported-by:
Nathan Chancellor <nathan@kernel.org> Signed-off-by:
-
Andrii Nakryiko authored
Kui-Feng Lee says: ==================== Allow users to attach a 64-bits cookie to a bpf_link of fentry, fexit, or fmod_ret. This patchset includes several major changes. - Define struct bpf_tramp_links to replace bpf_tramp_prog. struct bpf_tramp_links collects bpf_links of a trampoline - Generate a trampoline to call bpf_progs of given bpf_links. - Trampolines always set/reset bpf_run_ctx before/after calling/leaving a tracing program. - Attach a cookie to a bpf_link of fentry/fexit/fmod_ret/lsm. The value will be available when running the associated bpf_prog. Th major differences from v6: - bpf_link_create() can create links of BPF_LSM_MAC attach type. - Add a test for lsm. - Add function proto of bpf_get_attach_cookie() for lsm. - Check BPF_LSM_MAC in bpf_prog_has_trampoline(). - Adapt to the changes of LINK_CREATE made by Andrii. The major differences from v7: - Change stack_size instead of pushing/popping run_ctx. - Move cookie to bpf_tramp_link from bpf_tracing_link.. v1: https://lore.kernel.org/all/20220126214809.3868787-1-kuifeng@fb.com/ v2: https://lore.kernel.org/bpf/20220316004231.1103318-1-kuifeng@fb.com/ v3: https://lore.kernel.org/bpf/20220407192552.2343076-1-kuifeng@fb.com/ v4: https://lore.kernel.org/bpf/20220411173429.4139609-1-kuifeng@fb.com/ v5: https://lore.kernel.org/bpf/20220412165555.4146407-1-kuifeng@fb.com/ v6: https://lore.kernel.org/bpf/20220416042940.656344-1-kuifeng@fb.com/ v7: https://lore.kernel.org/bpf/20220508032117.2783209-1-kuifeng@fb.com/ ==================== Signed-off-by:
-
Kui-Feng Lee authored
Make sure BPF cookies are correct for fentry/fexit/fmod_ret/lsm. Signed-off-by:
Kui-Feng Lee <kuifeng@fb.com> Signed-off-by:
-
Kui-Feng Lee authored
Add a cookie field to the attributes of bpf_link_create(). Add bpf_program__attach_trace_opts() to attach a cookie to a link. Signed-off-by:
-
Kui-Feng Lee authored
Pass a cookie along with BPF_LINK_CREATE requests. Add a bpf_cookie field to struct bpf_tracing_link to attach a cookie. The cookie of a bpf_tracing_link is available by calling bpf_get_attach_cookie when running the BPF program of the attached link. The value of a cookie will be set at bpf_tramp_run_ctx by the trampoline of the link. Signed-off-by:
-
Kui-Feng Lee authored
BPF trampolines will create a bpf_tramp_run_ctx, a bpf_run_ctx, on stacks and set/reset the current bpf_run_ctx before/after calling a bpf_prog. Signed-off-by:
-
Kui-Feng Lee authored
Replace struct bpf_tramp_progs with struct bpf_tramp_links to collect struct bpf_tramp_link(s) for a trampoline. struct bpf_tramp_link extends bpf_link to act as a linked list node. arch_prepare_bpf_trampoline() accepts a struct bpf_tramp_links to collects all bpf_tramp_link(s) that a trampoline should call. Change BPF trampoline and bpf_struct_ops to pass bpf_tramp_links instead of bpf_tramp_progs. Signed-off-by:
-
- 10 May, 2022 12 commits
-
-
Alexei Starovoitov authored
Jiri Olsa says: ==================== hi, sending additional fix for symbol resolving in kprobe multi link requested by Alexei and Andrii [1]. This speeds up bpftrace kprobe attachment, when using pure symbols (3344 symbols) to attach: Before: # perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }' ... 6.5681 +- 0.0225 seconds time elapsed ( +- 0.34% ) After: # perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }' ... 0.5661 +- 0.0275 seconds time elapsed ( +- 4.85% ) v6 changes: - rewrote patch 1 changelog and fixed the line length [Christoph] v5 changes: - added acks [Masami] - workaround in selftest for RCU warning by filtering out several functions to attach v4 changes: - fix compile issue [kernel test robot] - added acks [Andrii] v3 changes: - renamed kallsyms_lookup_names to ftrace_lookup_symbols and moved it to ftrace.c [Masami] - added ack [Andrii] - couple small test fixes [Andrii] v2 changes (first version [2]): - removed the 2 seconds check [Alexei] - moving/forcing symbols sorting out of kallsyms_lookup_names function [Alexei] - skipping one array allocation and copy_from_user [Andrii] - several small fixes [Masami,Andrii] - build fix [kernel test robot] thanks, jirka [1] https://lore.kernel.org/bpf/CAEf4BzZtQaiUxQ-sm_hH2qKPRaqGHyOfEsW96DxtBHRaKLoL3Q@mail.gmail.com/ [2] https://lore.kernel.org/bpf/20220407125224.310255-1-jolsa@kernel.org/ ==================== Signed-off-by: -
Jiri Olsa authored
Adding test that reads all functions from ftrace available_filter_functions file and attach them all through kprobe_multi API. It also prints stats info with -v option, like on my setup: test_bench_attach: found 48712 functions test_bench_attach: attached in 1.069s test_bench_attach: detached in 0.373s Acked-by:
-
Jiri Olsa authored
Using kallsyms_lookup_names function to speed up symbols lookup in kprobe multi link attachment and replacing with it the current kprobe_multi_resolve_syms function. This speeds up bpftrace kprobe attachment: # perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }' ... 6.5681 +- 0.0225 seconds time elapsed ( +- 0.34% ) After: # perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }' ... 0.5661 +- 0.0275 seconds time elapsed ( +- 4.85% ) Acked-by: -
Jiri Olsa authored
Using ftrace_lookup_symbols to speed up symbols lookup in register_fprobe_syms API. Acked-by:
Masami Hiramatsu <mhiramat@kernel.org> Signed-off-by:
-
Jiri Olsa authored
Adding ftrace_lookup_symbols function that resolves array of symbols with single pass over kallsyms. The user provides array of string pointers with count and pointer to allocated array for resolved values. int ftrace_lookup_symbols(const char **sorted_syms, size_t cnt, unsigned long *addrs) It iterates all kallsyms symbols and tries to loop up each in provided symbols array with bsearch. The symbols array needs to be sorted by name for this reason. We also check each symbol to pass ftrace_location, because this API will be used for fprobe symbols resolving. Suggested-by: -
Jiri Olsa authored
Making kallsyms_on_each_symbol generally available, so it can be used outside CONFIG_LIVEPATCH option in following changes. Rather than adding another ifdef option let's make the function generally available (when CONFIG_KALLSYMS option is defined). Cc: Christoph Hellwig <hch@lst.de> Reviewed-by:
-
Alexei Starovoitov authored
Dmitrii Dolgov says: ==================== Bpf links seem to be one of the important structures for which no iterator is provided. Such iterator could be useful in those cases when generic 'task/file' is not suitable or better performance is needed. The implementation is mostly copied from prog iterator. This time tests were executed, although I still had to exclude test_bpf_nf (failed to find BTF info for global/extern symbol 'bpf_skb_ct_lookup') -- since it's unrelated, I hope it's a minor issue. Per suggestion from the previous discussion, there is a new patch for converting CHECK to corresponding ASSERT_* macro. Such replacement is done only if the final result would be the same, e.g. CHECK with important-looking custom formatting strings are still in place -- from what I understand ASSERT_* doesn't allow to specify such format. The third small patch fixes what looks like a copy-paste error in the condition checking. ==================== Signed-off-by: -
Dmitrii Dolgov authored
Add a simple test for bpf link iterator Signed-off-by:
Dmitrii Dolgov <9erthalion6@gmail.com> Link: https://lore.kernel.org/r/20220510155233.9815-5-9erthalion6@gmail.comSigned-off-by:
-
Dmitrii Dolgov authored
Replace usage of CHECK with a corresponding ASSERT_* macro for bpf_iter tests. Only done if the final result is equivalent, no changes when replacement means loosing some information, e.g. from formatting string. Signed-off-by:
-
Dmitrii Dolgov authored
The original condition looks like a typo, verify the skeleton loading result instead. Signed-off-by:
-
Dmitrii Dolgov authored
Implement bpf_link iterator to traverse links via bpf_seq_file operations. The changeset is mostly shamelessly copied from commit a228a64f ("bpf: Add bpf_prog iterator") Signed-off-by:
-
Alexei Starovoitov authored
Kaixi Fan says: ==================== From: Kaixi Fan <fankaixi.li@bytedance.com> Now bpf code could not set tunnel source ip address of ip tunnel. So it could not support flow based tunnel mode completely. Because flow based tunnel mode could set tunnel source, destination ip address and tunnel key simultaneously. Flow based tunnel is useful for overlay networks. And by configuring tunnel source ip address, user could make their networks more elastic. For example, tunnel source ip could be used to select different egress nic interface for different flows with same tunnel destination ip. Another example, user could choose one of multiple ip address of the egress nic interface as the packet's tunnel source ip. Add tunnel and tunnel source testcases in test_progs. Other types of tunnel testcases would be moved to test_progs step by step in the future. v6: - use libbpf api to attach tc progs and remove some shell commands to reduce test runtime based on Alexei Starovoitov's suggestion v5: - fix some code format errors - use bpf kernel code at namespace at_ns0 to set tunnel metadata v4: - fix subject error of first patch v3: - move vxlan tunnel testcases to test_progs - replace bpf_trace_printk with bpf_printk - rename bpf kernel prog section name to tic v2: - merge vxlan tunnel and tunnel source ip testcases in test_tunnel.sh ==================== Signed-off-by:
-
