- 21 Apr, 2023 30 commits
-
-
Eduard Zingerman authored
Test verifier/runtime_jit automatically converted to use inline assembly. Signed-off-by:
Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20230421174234.2391278-18-eddyz87@gmail.comSigned-off-by:
Alexei Starovoitov <ast@kernel.org>
-
Eduard Zingerman authored
Test verifier/regalloc automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/ref_tracking automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/map_ptr_mixing automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/map_in_map automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/lwt automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/loops1 automatically converted to use inline assembly. There are a few modifications for the converted tests. "tracepoint" programs do not support test execution, change program type to "xdp" (which supports test execution) for the following tests that have __retval tags: - bounded loop, count to 4 - bonded loop containing forward jump Also, remove the __retval tag for test: - bounded loop, count from positive unknown to 4 As it's return value is a random number. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/jeq_infer_not_null automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/direct_packet_access automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/d_path automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/ctx automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/btf_ctx_access automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/bpf_get_stack automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
Test verifier/bounds automatically converted to use inline assembly. Signed-off-by:
-
Eduard Zingerman authored
In order to express test cases that use bpf_tail_call() intrinsic it is necessary to have several programs to be loaded at a time. This commit adds __auxiliary annotation to the set of annotations supported by test_loader.c. Programs marked as auxiliary are always loaded but are not treated as a separate test. For example: void dummy_prog1(void); struct { __uint(type, BPF_MAP_TYPE_PROG_ARRAY); __uint(max_entries, 4); __uint(key_size, sizeof(int)); __array(values, void (void)); } prog_map SEC(".maps") = { .values = { [0] = (void *) &dummy_prog1, }, }; SEC("tc") __auxiliary __naked void dummy_prog1(void) { asm volatile ("r0 = 42; exit;"); } SEC("tc") __description("reference tracking: check reference or tail call") __success __retval(0) __naked void check_reference_or_tail_call(void) { asm volatile ( "r2 = %[prog_map] ll;" "r3 = 0;" "call %[bpf_tail_call];" "r0 = 0;" "exit;" :: __imm(bpf_tail_call), : __clobber_all); } Signed-off-by: -
Alexei Starovoitov authored
Florian Westphal says: ==================== Changes since last version: - rework test case in last patch wrt. ctx->skb dereference etc (Alexei) - pacify bpf ci tests, netfilter program type missed string translation in libbpf helper. This still uses runtime btf walk rather than extending the btf trace array as Alexei suggested, I would do this later (or someone else can). v1 cover letter: Add minimal support to hook bpf programs to netfilter hooks, e.g. PREROUTING or FORWARD. For this the most relevant parts for registering a netfilter hook via the in-kernel api are exposed to userspace via bpf_link. The new program type is 'tracing style', i.e. there is no context access rewrite done by verifier, the function argument (struct bpf_nf_ctx) isn't stable. There is no support for direct packet access, dynptr api should be used instead. With this its possible to build a small test program such as: #include "vmlinux.h" extern int bpf_dynptr_from_skb(struct __sk_buff *skb, __u64 flags, struct bpf_dynptr *ptr__uninit) __ksym; extern void *bpf_dynptr_slice(const struct bpf_dynptr *ptr, uint32_t offset, void *buffer, uint32_t buffer__sz) __ksym; SEC("netfilter") int nf_test(struct bpf_nf_ctx *ctx) { struct nf_hook_state *state = ctx->state; struct sk_buff *skb = ctx->skb; const struct iphdr *iph, _iph; const struct tcphdr *th, _th; struct bpf_dynptr ptr; if (bpf_dynptr_from_skb(skb, 0, &ptr)) return NF_DROP; iph = bpf_dynptr_slice(&ptr, 0, &_iph, sizeof(_iph)); if (!iph) return NF_DROP; th = bpf_dynptr_slice(&ptr, iph->ihl << 2, &_th, sizeof(_th)); if (!th) return NF_DROP; bpf_printk("accept %x:%d->%x:%d, hook %d ifin %d\n", iph->saddr, bpf_ntohs(th->source), iph->daddr, bpf_ntohs(th->dest), state->hook, state->in->ifindex); return NF_ACCEPT; } Then, tail /sys/kernel/tracing/trace_pipe. Changes since v3: - uapi: remove 'reserved' struct member, s/prio/priority (Alexei) - add ctx access test cases (Alexei, see last patch) - some arm32 can only handle cmpxchg on u32 (build bot) - Fix kdoc annotations (Simon Horman) - bpftool: prefer p_err, not fprintf (Quentin) - add test cases in separate patch Changes since v2: 1. don't WARN when user calls 'bpftool loink detach' twice restrict attachment to ip+ip6 families, lets relax this later in case arp/bridge/netdev are needed too. 2. show netfilter links in 'bpftool net' output as well. Changes since v1: 1. Don't fail to link when CONFIG_NETFILTER=n (build bot) 2. Use test_progs instead of test_verifier (Alexei) Changes since last RFC version: 1. extend 'bpftool link show' to print prio/hooknum etc 2. extend 'nft list hooks' so it can print the bpf program id 3. Add an extra patch to artificially restrict bpf progs with same priority. Its fine from a technical pov but it will cause ordering issues (most recent one comes first). Can be removed later. 4. Add test_run support for netfilter prog type and a small extension to verifier tests to make sure we can't return verdicts like NF_STOLEN. 5. Alter the netfilter part of the bpf_link uapi struct: - add flags/reserved members. Not used here except returning errors when they are nonzero. Plan is to allow the bpf_link users to enable netfilter defrag or conntrack engine by setting feature flags at link create time in the future. ==================== Signed-off-by: -
Florian Westphal authored
Extend prog_tests with two test cases: # ./test_progs --allow=verifier_netfilter_retcode #278/1 verifier_netfilter_retcode/bpf_exit with invalid return code. test1:OK #278/2 verifier_netfilter_retcode/bpf_exit with valid return code. test2:OK #278/3 verifier_netfilter_retcode/bpf_exit with valid return code. test3:OK #278/4 verifier_netfilter_retcode/bpf_exit with invalid return code. test4:OK #278 verifier_netfilter_retcode:OK This checks that only accept and drop (0,1) are permitted. NF_QUEUE could be implemented later if we can guarantee that attachment of such programs can be rejected if they get attached to a pf/hook that doesn't support async reinjection. NF_STOLEN could be implemented via trusted helpers that can guarantee that the skb will eventually be free'd. v4: test case for bpf_nf_ctx access checks, requested by Alexei Starovoitov. v5: also check ctx->{state,skb} can be dereferenced (Alexei). # ./test_progs --allow=verifier_netfilter_ctx #281/1 verifier_netfilter_ctx/netfilter invalid context access, size too short:OK #281/2 verifier_netfilter_ctx/netfilter invalid context access, size too short:OK #281/3 verifier_netfilter_ctx/netfilter invalid context access, past end of ctx:OK #281/4 verifier_netfilter_ctx/netfilter invalid context, write:OK #281/5 verifier_netfilter_ctx/netfilter valid context read and invalid write:OK #281/6 verifier_netfilter_ctx/netfilter test prog with skb and state read access:OK #281/7 verifier_netfilter_ctx/netfilter test prog with skb and state read access @unpriv:OK #281 verifier_netfilter_ctx:OK Summary: 1/7 PASSED, 0 SKIPPED, 0 FAILED This checks: 1/2: partial reads of ctx->{skb,state} are rejected 3. read access past sizeof(ctx) is rejected 4. write to ctx content, e.g. 'ctx->skb = NULL;' is rejected 5. ctx->state content cannot be altered 6. ctx->state and ctx->skb can be dereferenced 7. ... same program fails for unpriv (CAP_NET_ADMIN needed). Link: https://lore.kernel.org/bpf/20230419021152.sjq4gttphzzy6b5f@dhcp-172-26-102-232.dhcp.thefacebook.com/ Link: https://lore.kernel.org/bpf/20230420201655.77kkgi3dh7fesoll@MacBook-Pro-6.local/Signed-off-by:Florian Westphal <fw@strlen.de> Link: https://lore.kernel.org/r/20230421170300.24115-8-fw@strlen.deSigned-off-by:
-
Florian Westphal authored
add glue code so a bpf program can be run using userspace-provided netfilter state and packet/skb. Default is to use ipv4:output hook point, but this can be overridden by userspace. Userspace provided netfilter state is restricted, only hook and protocol families can be overridden and only to ipv4/ipv6. Signed-off-by:
-
Florian Westphal authored
Dump protocol family, hook and priority value: $ bpftool link 2: netfilter prog 14 ip input prio -128 pids install(3264) 5: netfilter prog 14 ip6 forward prio 21 pids a.out(3387) 9: netfilter prog 14 ip prerouting prio 123 pids a.out(5700) 10: netfilter prog 14 ip input prio 21 pids test2(5701) v2: Quentin Monnet suggested to also add 'bpftool net' support: $ bpftool net xdp: tc: flow_dissector: netfilter: ip prerouting prio 21 prog_id 14 ip input prio -128 prog_id 14 ip input prio 21 prog_id 14 ip forward prio 21 prog_id 14 ip output prio 21 prog_id 14 ip postrouting prio 21 prog_id 14 'bpftool net' only dumps netfilter link type, links are sorted by protocol family, hook and priority. v5: fix bpf ci failure: libbpf needs small update to prog_type_name[] and probe_prog_load helper. v4: don't fail with -EOPNOTSUPP in libbpf probe_prog_load, update prog_type_name[] with "netfilter" entry (bpf ci) v3: fix bpf.h copy, 'reserved' member was removed (Alexei) use p_err, not fprintf (Quentin) Suggested-by:Quentin Monnet <quentin@isovalent.com> Link: https://lore.kernel.org/bpf/eeeaac99-9053-90c2-aa33-cc1ecb1ae9ca@isovalent.com/Reviewed-by:
-
Florian Westphal authored
This is just to avoid ordering issues between multiple bpf programs, this could be removed later in case it turns out to be too cautious. bpf prog could still be shared with non-bpf hook, otherwise we'd have to make conntrack hook registration fail just because a bpf program has same priority. Signed-off-by:
-
Florian Westphal authored
This allows userspace ("nft list hooks") to show which bpf program is attached to which hook. Without this, user only knows bpf prog is attached at prio x, y, z at INPUT and FORWARD, but can't tell which program is where. v4: kdoc fixups (Simon Horman) Link: https://lore.kernel.org/bpf/ZEELzpNCnYJuZyod@corigine.com/Signed-off-by: -
Florian Westphal authored
This adds minimal support for BPF_PROG_TYPE_NETFILTER bpf programs that will be invoked via the NF_HOOK() points in the ip stack. Invocation incurs an indirect call. This is not a necessity: Its possible to add 'DEFINE_BPF_DISPATCHER(nf_progs)' and handle the program invocation with the same method already done for xdp progs. This isn't done here to keep the size of this chunk down. Verifier restricts verdicts to either DROP or ACCEPT. Signed-off-by:
-
Florian Westphal authored
Add bpf_link support skeleton. To keep this reviewable, no bpf program can be invoked yet, if a program is attached only a c-stub is called and not the actual bpf program. Defaults to 'y' if both netfilter and bpf syscall are enabled in kconfig. Uapi example usage: union bpf_attr attr = { }; attr.link_create.prog_fd = progfd; attr.link_create.attach_type = 0; /* unused */ attr.link_create.netfilter.pf = PF_INET; attr.link_create.netfilter.hooknum = NF_INET_LOCAL_IN; attr.link_create.netfilter.priority = -128; err = bpf(BPF_LINK_CREATE, &attr, sizeof(attr)); ... this would attach progfd to ipv4:input hook. Such hook gets removed automatically if the calling program exits. BPF_NETFILTER program invocation is added in followup change. NF_HOOK_OP_BPF enum will eventually be read from nfnetlink_hook, it allows to tell userspace which program is attached at the given hook when user runs 'nft hook list' command rather than just the priority and not-very-helpful 'this hook runs a bpf prog but I can't tell which one'. Will also be used to disallow registration of two bpf programs with same priority in a followup patch. v4: arm32 cmpxchg only supports 32bit operand s/prio/priority/ v3: restrict prog attachment to ip/ip6 for now, lets lift restrictions if more use cases pop up (arptables, ebtables, netdev ingress/egress etc). Signed-off-by: -
Kui-Feng Lee authored
The "struct_ops register" subcommand now allows for an optional *LINK_DIR* to be included. This specifies the directory path where bpftool will pin struct_ops links with the same name as their corresponding map names. Signed-off-by:
Kui-Feng Lee <kuifeng@meta.com> Reviewed-by:
-
Kui-Feng Lee authored
You can include an optional path after specifying the object name for the 'struct_ops register' subcommand. Since the commit 226bc6ae ("Merge branch 'Transit between BPF TCP congestion controls.'") has been accepted, it is now possible to create a link for a struct_ops. This can be done by defining a struct_ops in SEC(".struct_ops.link") to make libbpf returns a real link. If we don't pin the links before leaving bpftool, they will disappear. To instruct bpftool to pin the links in a directory with the names of the maps, we need to provide the path of that directory. Signed-off-by:
-
Stanislav Fomichev authored
Make sure we get optlen exported instead of getting EFAULT. Signed-off-by:
Stanislav Fomichev <sdf@google.com> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20230418225343.553806-3-sdf@google.com
-
Stanislav Fomichev authored
Some socket options do getsockopt with optval=NULL to estimate the size of the final buffer (which is returned via optlen). This breaks BPF getsockopt assumptions about permitted optval buffer size. Let's enforce these assumptions only when non-NULL optval is provided. Fixes: 0d01da6a ("bpf: implement getsockopt and setsockopt hooks") Reported-by:
Martin KaFai Lau <martin.lau@kernel.org> Signed-off-by:
-
Magnus Karlsson authored
Put the flag MAP_HUGE_2MB in the correct flags argument instead of the wrong offset argument. Fixes: 2ddade32 ("selftests/xsk: Fix munmap for hugepage allocated umem") Reported-by:
Kal Cutter Conley <kal.conley@dectris.com> Signed-off-by:
Magnus Karlsson <magnus.karlsson@intel.com> Signed-off-by:
-
Dave Marchevsky authored
When calculating the address of the refcount_t struct within a local kptr, bpf_refcount_acquire_impl should add refcount_off bytes to the address of the local kptr. Due to some missing parens, the function is incorrectly adding sizeof(refcount_t) * refcount_off bytes. This patch fixes the calculation. Due to the incorrect calculation, bpf_refcount_acquire_impl was trying to refcount_inc some memory well past the end of local kptrs, resulting in kasan and refcount complaints, as reported in [0]. In that thread, Florian and Eduard discovered that bpf selftests written in the new style - with __success and an expected __retval, specifically - were not actually being run. As a result, selftests added in bpf_refcount series weren't really exercising this behavior, and thus didn't unearth the bug. With this fixed behavior it's safe to revert commit 7c4b96c0 ("selftests/bpf: disable program test run for progs/refcounted_kptr.c"), this patch does so. [0] https://lore.kernel.org/bpf/ZEEp+j22imoN6rn9@strlen.de/ Fixes: 7c50b1cb ("bpf: Add bpf_refcount_acquire kfunc") Reported-by:
Dave Marchevsky <davemarchevsky@fb.com> Signed-off-by:
-
Alexei Starovoitov authored
Florian and Eduard reported hard dead lock: [ 58.433327] _raw_spin_lock_irqsave+0x40/0x50 [ 58.433334] btf_put+0x43/0x90 [ 58.433338] bpf_find_btf_id+0x157/0x240 [ 58.433353] btf_parse_fields+0x921/0x11c0 This happens since btf->refcount can be 1 at the time of btf_put() and btf_put() will call btf_free_id() which will try to grab btf_idr_lock and will dead lock. Avoid the issue by doing btf_put() without locking. Fixes: 3d78417b ("bpf: Add bpf_btf_find_by_name_kind() helper.") Fixes: 1e89106d ("bpf: Add bpf_core_add_cands() and wire it into bpf_core_apply_relo_insn().") Reported-by:
-
- 20 Apr, 2023 9 commits
-
-
Alexei Starovoitov authored
Eduard Zingerman says: ==================== Florian Westphal found a bug in test_loader.c processing of __retval tag. Because of this bug the function test_loader.c:do_prog_test_run() never executed and all __retval test tags were ignored. See [1]. Fix for this bug uncovers two additional bugs: - During test_verifier tests migration to inline assembly (see [2]) I missed the fact that some tests require maps to contain mock values; - Some issue with a new refcounted_kptr test, which causes kernel to produce dead lock and refcount saturation warnings when subject to libbpf's bpf_test_run_opts(). This series fixes the bug in __retval() processing, and address the issue with test maps not being populated. The issue in refcounted_kptr is not addressed, __retval tags in those tests are commented out. I found that the following tests depend on test maps being populated: - progs/verifier_array_access.c - verifier/value_ptr_arith.c (planned for migration to inline assembly) Given the small amount of these tests I decided to opt for simple non-generic solution (see patch #4). [1] https://lore.kernel.org/bpf/f4c4aee644425842ee6aa8edf1da68f0a8260e7c.camel@gmail.com/T/ [2] https://lore.kernel.org/bpf/20230325025524.144043-1-eddyz87@gmail.com/ ==================== Signed-off-by:
-
Eduard Zingerman authored
Two test cases: - "valid read map access into a read-only array 1" and - "valid read map access into a read-only array 2" Expect that map_array_ro map is filled with mock data. This logic was not taken into acount during initial test conversion. This commit modifies prog_tests/verifier.c entry point for this test to fill the map. Fixes: a3c830ae ("selftests/bpf: verifier/array_access.c converted to inline assembly") Signed-off-by:
-
Eduard Zingerman authored
When a test case is annotated with __retval tag the test_loader engine would use libbpf's bpf_prog_test_run_opts() to do a test run of the program and compare retvals. This commit allows to perform arbitrary actions on bpf object right before test loader invokes bpf_prog_test_run_opts(). This could be used to setup some state for program execution, e.g. fill some maps. Signed-off-by:
-
Eduard Zingerman authored
Florian Westphal found a bug in and suggested a fix for test_loader.c processing of __retval tag. Because of this bug the function test_loader.c:do_prog_test_run() never executed and all __retval test tags were ignored. If this bug is fixed a number of test cases from progs/verifier_array_access.c fail with retval not matching the expected value. This test was recently converted to use test_loader.c and inline assembly in [1]. When doing the conversion I missed the important detail of test_verifier.c operation: when it creates fixup_map_array_ro, fixup_map_array_wo and fixup_map_array_small it populates these maps with a dummy record. Disabling the __retval checks for the affected verifier_array_access in this commit to avoid false-postivies in any potential bisects. The issue is addressed in the next patch. I verified that the __retval tags are now respected by changing expected return values for all tests annotated with __retval, and checking that these tests started to fail. [1] https://lore.kernel.org/bpf/20230325025524.144043-1-eddyz87@gmail.com/ Fixes: 19a8e06f ("selftests/bpf: Tests execution support for test_loader.c") Reported-by:
-
Eduard Zingerman authored
Florian Westphal found a bug in test_loader.c processing of __retval tag. Because of this bug the function test_loader.c:do_prog_test_run() never executed and all __retval test tags were ignored. This hid an issue with progs/refcounted_kptr.c tests. When __retval tag bug is fixed and refcounted_kptr.c tests are run kernel reports various issues and eventually hangs. Shortest reproducer is the following command run a few times: $ for i in $(seq 1 4); do (./test_progs --allow=refcounted_kptr &); done Commenting out __retval tags for these tests until this issue is resolved. Reported-by:
-
Quentin Monnet authored
The recent support for inline annotations in control flow graphs generated by bpftool introduced the usage of the "__fallthrough" macro in a switch/case block in btf_dumper.c. This change went through the bpf-next tree, but resulted in a merge conflict in linux-next, because this macro has been renamed "fallthrough" (no underscores) in the meantime. To address the conflict, we temporarily switch to a simple comment instead of a macro. Related: commit f7a858bf ("tools: Rename __fallthrough to fallthrough") Fixes: 9fd49684 ("bpftool: Support inline annotations when dumping the CFG of a program") Reported-by:
Sven Schnelle <svens@linux.ibm.com> Reported-by:
Thomas Richter <tmricht@linux.ibm.com> Suggested-by:
Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/all/yt9dttxlwal7.fsf@linux.ibm.com/ Link: https://lore.kernel.org/bpf/20230412123636.2358949-1-tmricht@linux.ibm.com/ Link: https://lore.kernel.org/bpf/20230420003333.90901-1-quentin@isovalent.com
-
Alexei Starovoitov authored
Feng zhou says: ==================== From: Feng Zhou <zhoufeng.zf@bytedance.com> Add support for integer type of accessing variable length array. Add a selftest to check it. ==================== Signed-off-by: -
Feng Zhou authored
Add prog test for accessing integer type of variable array in tracing program. In addition, hook load_balance function to access sd->span[0], only to confirm whether the load is successful. Because there is no direct way to trigger load_balance call. Co-developed-by:
Chengming Zhou <zhouchengming@bytedance.com> Signed-off-by:
Feng Zhou <zhoufeng.zf@bytedance.com> Link: https://lore.kernel.org/r/20230420032735.27760-3-zhoufeng.zf@bytedance.comSigned-off-by:
-
Feng Zhou authored
After this commit: bpf: Support variable length array in tracing programs (9c5f8a10) Trace programs can access variable length array, but for structure type. This patch adds support for integer type. Example: Hook load_balance struct sched_domain { ... unsigned long span[]; } The access: sd->span[0]. Co-developed-by:
-
- 19 Apr, 2023 1 commit
-
-
Magnus Karlsson authored
Fix the unmapping of hugepage allocated umems so that they are properly unmapped. The new test referred to in the fixes label, introduced a test that allocated a umem that is not a multiple of a 2M hugepage size. This is fine for mmap() that rounds the size up the nearest multiple of 2M. But munmap() requires the size to be a multiple of the hugepage size in order for it to unmap the region. The current behaviour of not properly unmapping the umem, was discovered when further additions of tests that require hugepages (unaligned mode tests only) started failing as the system was running out of hugepages. Fixes: c0801598 ("selftests: xsk: Add test UNALIGNED_INV_DESC_4K1_FRAME_SIZE") Signed-off-by:
-
