- 07 Aug, 2018 1 commit
-
-
Roman Gushchin authored
__cgroup_bpf_attach() and __cgroup_bpf_detach() functions have a good amount of duplicated code, which is possible to eliminate by introducing the update_effective_progs() helper function. The update_effective_progs() calls compute_effective_progs() and then in case of success it calls activate_effective_progs() for each descendant cgroup. In case of failure (OOM), it releases allocated prog arrays and return the error code. Signed-off-by:
Roman Gushchin <guro@fb.com> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Daniel Borkmann <daniel@iogearbox.net> Acked-by:
Song Liu <songliubraving@fb.com> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net>
-
- 04 Aug, 2018 1 commit
-
-
Jakub Kicinski authored
Add support for adjust_tail. There are no FW changes needed but add a FW capability just in case there would be any issue with previously released FW, or we will have to change the ABI in the future. The helper is trivial and shouldn't be used too often so just inline the body of the function. We add the delta to locally maintained packet length register and check for overflow, since add of negative value must overflow if result is positive. Note that if delta of 0 would be allowed in the kernel this trick stops working and we need one more instruction to compare lengths before and after the change. Signed-off-by:
Jakub Kicinski <jakub.kicinski@netronome.com> Reviewed-by:
Quentin Monnet <quentin.monnet@netronome.com> Signed-off-by:
-
- 03 Aug, 2018 1 commit
-
-
Roman Gushchin authored
Commit fbeb1603 ("bpf: verifier: MOV64 don't mark dst reg unbounded") revealed a typo in commit fb30d4b7 ("bpf: Add tests for map-in-map"): BPF_MOV64_REG(BPF_REG_0, 0) was used instead of BPF_MOV64_IMM(BPF_REG_0, 0). I've noticed the problem by running bpf kselftests. Fixes: fb30d4b7 ("bpf: Add tests for map-in-map") Signed-off-by:
Martin KaFai Lau <kafai@fb.com> Signed-off-by:
-
- 02 Aug, 2018 15 commits
-
-
Daniel Borkmann authored
Roman Gushchin says: ==================== This patchset implements cgroup local storage for bpf programs. The main idea is to provide a fast accessible memory for storing various per-cgroup data, e.g. number of transmitted packets. Cgroup local storage looks as a special type of map for userspace, and is accessible using generic bpf maps API for reading and updating of the data. The (cgroup inode id, attachment type) pair is used as a map key. A user can't create new entries or destroy existing entries; it happens automatically when a user attaches/detaches a bpf program to a cgroup. From a bpf program's point of view, cgroup storage is accessible without lookup using the special get_local_storage() helper function. It takes a map fd as an argument. It always returns a valid pointer to the corresponding memory area. To implement such a lookup-free access a pointer to the cgroup storage is saved for an attachment of a bpf program to a cgroup, if required by the program. Before running the program, it's saved in a special global per-cpu variable, which is accessible from the get_local_storage() helper. This patchset implement only cgroup local storage, however the API is intentionally made extensible to support other local storage types further: e.g. thread local storage, socket local storage, etc. v7->v6: - fixed a use-after-free bug, caused by not clearing prog->aux->cgroup_storage pointer after releasing the map v6->v5: - fixed an error with returning -EINVAL instead of a pointer v5->v4: - fixed an issue in verifier (test that flags == 0 properly) - added a corresponding test - added a note about synchronization, sync docs to tools/uapi/... - switched the cgroup test to use XADD - added a check for attr->max_entries to be 0, and atter->max_flags to be sane - use bpf_uncharge_memlock() in bpf_uncharge_memlock() - rebased to bpf-next v4->v3: - fixed a leak in cgroup attachment code (discovered by Daniel) - cgroup storage map will be released if the corresponding bpf program failed to load by any reason - introduced bpf_uncharge_memlock() helper v3->v2: - fixed more build and sparse issues - rebased to bpf-next v2->v1: - fixed build issues - removed explicit rlimit calls in patch 14 - rebased to bpf-next ==================== Signed-off-by: -
Roman Gushchin authored
The test_cgrp2_attach test covers bpf cgroup attachment code well, so let's re-use it for testing allocation/releasing of cgroup storage. The extension is pretty straightforward: the bpf program will use the cgroup storage to save the number of transmitted bytes. Expected output: $ ./test_cgrp2_attach2 Attached DROP prog. This ping in cgroup /foo should fail... ping: sendmsg: Operation not permitted Attached DROP prog. This ping in cgroup /foo/bar should fail... ping: sendmsg: Operation not permitted Attached PASS prog. This ping in cgroup /foo/bar should pass... Detached PASS from /foo/bar while DROP is attached to /foo. This ping in cgroup /foo/bar should fail... ping: sendmsg: Operation not permitted Attached PASS from /foo/bar and detached DROP from /foo. This ping in cgroup /foo/bar should pass... ### override:PASS ### multi:PASS Signed-off-by:
-
Roman Gushchin authored
Implement a test to cover the cgroup storage functionality. The test implements a bpf program which drops every second packet by using the cgroup storage as a persistent storage. The test also use the userspace API to check the data in the cgroup storage, alter it, and check that the loaded and attached bpf program sees the update. Expected output: $ ./test_cgroup_storage test_cgroup_storage:PASS Signed-off-by:
-
Roman Gushchin authored
Add the following verifier tests to cover the cgroup storage functionality: 1) valid access to the cgroup storage 2) invalid access: use regular hashmap instead of cgroup storage map 3) invalid access: use invalid map fd 4) invalid access: try access memory after the cgroup storage 5) invalid access: try access memory before the cgroup storage 6) invalid access: call get_local_storage() with non-zero flags For tests 2)-6) check returned error strings. Expected output: $ ./test_verifier #0/u add+sub+mul OK #0/p add+sub+mul OK #1/u DIV32 by 0, zero check 1 OK ... #280/p valid cgroup storage access OK #281/p invalid cgroup storage access 1 OK #282/p invalid cgroup storage access 2 OK #283/p invalid per-cgroup storage access 3 OK #284/p invalid cgroup storage access 4 OK #285/p invalid cgroup storage access 5 OK ... #649/p pass modified ctx pointer to helper, 2 OK #650/p pass modified ctx pointer to helper, 3 OK Summary: 901 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by:
-
Roman Gushchin authored
Allocate a temporary cgroup storage to use for bpf program test runs. Because the test program is not actually attached to a cgroup, the storage is allocated manually just for the execution of the bpf program. If the program is executed multiple times, the storage is not zeroed on each run, emulating multiple runs of the program, attached to a real cgroup. Signed-off-by:
-
Roman Gushchin authored
Add BPF_MAP_TYPE_CGROUP_STORAGE maps to the list of maps types which bpftool recognizes. Signed-off-by:
-
Roman Gushchin authored
Sync cgroup storage related changes: 1) new BPF_MAP_TYPE_CGROUP_STORAGE map type 2) struct bpf_cgroup_sotrage_key definition 3) get_local_storage() helper Signed-off-by:
-
Roman Gushchin authored
The bpf_get_local_storage() helper function is used to get a pointer to the bpf local storage from a bpf program. It takes a pointer to a storage map and flags as arguments. Right now it accepts only cgroup storage maps, and flags argument has to be 0. Further it can be extended to support other types of local storage: e.g. thread local storage etc. Signed-off-by:
-
Roman Gushchin authored
As there is one-to-one relation between a bpf program and cgroup local storage map, there is no sense in creating a map of cgroup local storage maps. Forbid it explicitly to avoid possible side effects. Signed-off-by:
-
Roman Gushchin authored
BPF_MAP_TYPE_CGROUP_STORAGE maps are special in a way that the access from the bpf program side is lookup-free. That means the result is guaranteed to be a valid pointer to the cgroup storage; no NULL-check is required. This patch introduces BPF_PTR_TO_MAP_VALUE return type, which is required to cause the verifier accept programs, which are not checking the map value pointer for being NULL. Signed-off-by:
-
Roman Gushchin authored
This patch converts bpf_prog_array from an array of prog pointers to the array of struct bpf_prog_array_item elements. This allows to save a cgroup storage pointer for each bpf program efficiently attached to a cgroup. Signed-off-by:
-
Roman Gushchin authored
If a bpf program is using cgroup local storage, allocate a bpf_cgroup_storage structure automatically on attaching the program to a cgroup and save the pointer into the corresponding bpf_prog_list entry. Analogically, release the cgroup local storage on detaching of the bpf program. Signed-off-by:
-
Roman Gushchin authored
This commit introduces the bpf_cgroup_storage_set() helper, which will be used to pass a pointer to a cgroup storage to the bpf helper. Signed-off-by:
-
Roman Gushchin authored
This commit introduces BPF_MAP_TYPE_CGROUP_STORAGE maps: a special type of maps which are implementing the cgroup storage. >From the userspace point of view it's almost a generic hash map with the (cgroup inode id, attachment type) pair used as a key. The only difference is that some operations are restricted: 1) a user can't create new entries, 2) a user can't remove existing entries. The lookup from userspace is o(log(n)). Signed-off-by:
-
Roman Gushchin authored
This commits extends existing bpf maps memory charging API to support dynamic charging/uncharging. This is required to account memory used by maps, if all entries are created dynamically after the map initialization. Signed-off-by:
-
- 31 Jul, 2018 9 commits
-
-
Arthur Fabre authored
When check_alu_op() handles a BPF_MOV64 between two registers, it calls check_reg_arg(DST_OP) on the dst register, marking it as unbounded. If the src and dst register are the same, this marks the src as unbounded, which can lead to unexpected errors for further checks that rely on bounds info. For example: BPF_MOV64_IMM(BPF_REG_2, 0), BPF_MOV64_REG(BPF_REG_2, BPF_REG_2), BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2), BPF_MOV64_IMM(BPF_REG_0, 0), BPF_EXIT_INSN(), Results in: "math between ctx pointer and register with unbounded min value is not allowed" check_alu_op() now uses check_reg_arg(DST_OP_NO_MARK), and MOVs that need to mark the dst register (MOVIMM, MOV32) do so. Added a test case for MOV64 dst == src, and dst != src. Signed-off-by:
Arthur Fabre <afabre@cloudflare.com> Acked-by:
Edward Cree <ecree@solarflare.com> Signed-off-by:
-
Andrey Ignatov authored
Add test to use get_socket_cookie() from BPF programs of types BPF_PROG_TYPE_SOCK_OPS and BPF_PROG_TYPE_CGROUP_SOCK_ADDR. The test attaches two programs to cgroup, runs TCP server and client in the cgroup and checks that two operations are done properly on client socket when user calls connect(2): 1. In BPF_CGROUP_INET6_CONNECT socket cookie is used as the key to write new value in a map for client socket. 2. In BPF_CGROUP_SOCK_OPS (BPF_SOCK_OPS_TCP_CONNECT_CB callback) the value written in "1." is found by socket cookie, since it's the same socket, and updated. Finally the test verifies the value in the map. Signed-off-by:
Andrey Ignatov <rdna@fb.com> Acked-by:
Yonghong Song <yhs@fb.com> Signed-off-by:
-
Andrey Ignatov authored
Add missing helper to bpf_helpers.h that is used in tests and samples. Signed-off-by:
-
Andrey Ignatov authored
Sync bpf_get_socket_cookie() related bpf UAPI changes to tools/. Signed-off-by:
-
Andrey Ignatov authored
bpf_get_socket_cookie() helper can be used to identify skb that correspond to the same socket. Though socket cookie can be useful in many other use-cases where socket is available in program context. Specifically BPF_PROG_TYPE_CGROUP_SOCK_ADDR and BPF_PROG_TYPE_SOCK_OPS programs can benefit from it so that one of them can augment a value in a map prepared earlier by other program for the same socket. The patch adds support to call bpf_get_socket_cookie() from BPF_PROG_TYPE_CGROUP_SOCK_ADDR and BPF_PROG_TYPE_SOCK_OPS. It doesn't introduce new helpers. Instead it reuses same helper name bpf_get_socket_cookie() but adds support to this helper to accept `struct bpf_sock_addr` and `struct bpf_sock_ops`. Documentation in bpf.h is changed in a way that should not break automatic generation of markdown. Signed-off-by:
-
Jakub Kicinski authored
'app' is dereferenced before used for the devlink trace point. In case FW is buggy and sends a control message to a VF queue we should make sure app is not NULL. Reported-by:
Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by:
Dirk van der Merwe <dirk.vandermerwe@netronome.com> Signed-off-by:
-
Taehee Yoo authored
run_lwt_bpf is called by bpf_{input/output/xmit}. These functions are already protected by rcu_read_lock. because lwtunnel_{input/output/xmit} holds rcu_read_lock and then calls bpf_{input/output/xmit}. So that rcu_read_lock in the run_lwt_bpf is unnecessary. Signed-off-by:Taehee Yoo <ap420073@gmail.com> Acked-by:
-
Mathieu Xhonneux authored
The seg6local LWT provides the End.DT6 action, which allows to decapsulate an outer IPv6 header containing a Segment Routing Header (SRH), full specification is available here: https://tools.ietf.org/html/draft-filsfils-spring-srv6-network-programming-05 This patch adds this action now to the seg6local BPF interface. Since it is not mandatory that the inner IPv6 header also contains a SRH, seg6_bpf_srh_state has been extended with a pointer to a possible SRH of the outermost IPv6 header. This helps assessing if the validation must be triggered or not, and avoids some calls to ipv6_find_hdr. v3: s/1/true, s/0/false for boolean values v2: - changed true/false -> 1/0 - preempt_enable no longer called in first conditional block Signed-off-by:
Mathieu Xhonneux <m.xhonneux@gmail.com> Signed-off-by:
-
Thomas Richter authored
Commit 531b014e ("tools: bpf: make use of reallocarray") causes a compiler error when building the perf tool in the linux-next tree. Compile file tools/lib/bpf/libbpf.c on a FEDORA 28 installation with gcc compiler version: gcc (GCC) 8.0.1 20180324 (Red Hat 8.0.1-0.20) shows this error message: [root@p23lp27] # make V=1 EXTRA_CFLAGS="-Wp,-D_FORTIFY_SOURCE=2 -O2" [...] make -f /home6/tmricht/linux-next/tools/build/Makefile.build dir=./util/scripting-engines obj=libperf libbpf.c: In function ‘bpf_object__elf_collect’: libbpf.c:811:15: error: ignoring return value of ‘strerror_r’, declared with attribute warn_unused_result [-Werror=unused-result] strerror_r(-err, errmsg, sizeof(errmsg)); ^ cc1: all warnings being treated as errors mv: cannot stat './.libbpf.o.tmp': No such file or directory /home6/tmricht/linux-next/tools/build/Makefile.build:96: recipe for target 'libbpf.o' failed Replace all occurrences of strerror() by calls to strerror_r(). To keep the compiler quiet also use the return value from strerror_r() otherwise a 'variable set but not use' warning which is treated as error terminates the compile. Fixes: 531b014e ("tools: bpf: make use of reallocarray") Suggested-by:
Thomas Richter <tmricht@linux.ibm.com> Reviewed-by:
Hendrik Brueckner <brueckner@linux.ibm.com> Reviewed-by:
-
- 27 Jul, 2018 13 commits
-
-
Daniel Borkmann authored
Tobin C. Harding says: ==================== Recently BPF docs were converted to RST format. A couple of things were missed: - Use 'index.rst' instead of 'README.rst'. Although README.rst will work just fine it is more typical to keep the subdirectory indices in a file called 'index.rst'. - Integrate files Documentation/bpf/*.rst into build system using toctree in Documentation/bpf/index.rst - Include bpf/index in top level toctree so bpf is indexed in the main kernel docs. - Make anal change to heading format (inline with rest of Documentation/). ==================== Signed-off-by: -
Tobin C. Harding authored
The majority of files in the kernel documentation index use capitalisation for all words, especially the shorter ones. BPF docs better fit in with the rest of the documentation if the heading is all capitalised. Capitalise document heading. Signed-off-by:
Tobin C. Harding <me@tobin.cc> Signed-off-by:
-
Tobin C. Harding authored
Recently bpf docs were converted to RST format. The new files were not added to the top level toctree. This causes build system to emit a warning of type WARNING: document isn't included in any toctree Add bpf/index.rst to Documentation/index.rst Signed-off-by:
-
Tobin C. Harding authored
Recently bpf/ docs were converted to us RST format. bp/index.rst was created out of README but toctree was not added to include files within Documentation/bpf/ Add toctree to Documentation/bpf/index.rst Signed-off-by:
-
Tobin C. Harding authored
Recently bpf/ docs were converted to use RST format. 'README.rst' was created but in order to fit in with the Sphinx build system this file should be named 'index.rst'. Rename file, fixes to integrate into Sphinx build system in following patches. docs: Rename Documentation/bpf/README.rst to Documentation/bpf/index.rst Signed-off-by:
-
Daniel Borkmann authored
Jakub Kicinski says: ==================== This set converts xdpsock_user.c and xdp_fwd_user.c to use libbpf instead of bpf_load.o. First two patches are minor improvements to libbpf to make the conversion (and use of libbpf in general) nicer. ==================== Acked-by:
-
Jakub Kicinski authored
Convert xdpsock_user.c to use libbpf instead of bpf_load.o. Signed-off-by:
-
Jakub Kicinski authored
Convert xdp_fwd_user.c to use libbpf instead of bpf_load.o. Signed-off-by:
-
Jakub Kicinski authored
Allow users to find programs by section names. Signed-off-by:
-
Jakub Kicinski authored
bpf_map__fd() handles NULL map gracefully and returns -EINVAL. bpf_program__fd() and bpf_program__nth_fd() crash in this case. Make the behaviour more consistent by validating prog pointer as well. Signed-off-by:
-
Daniel Borkmann authored
Jakub Kicinski says: ==================== This set is focused on improving the performance of perf events reported from BPF offload. Perf events can now be received on packet data queues, which significantly improves the performance (from total of 0.5 Msps to 5Msps per core). To get to this performance we need a fast path for control messages which will operate on raw buffers and recycle them immediately. Patch 5 replaces the map pointers for perf maps with map IDs. We look the pointers up in a hashtable, anyway, to validate they are correct, so there is no performance difference. Map IDs have the advantage of being easier to understand for users in case of errors (we no longer print raw pointers to the logs). Last patch improves info messages about map offload. ==================== Signed-off-by: -
Jakub Kicinski authored
FW can put constraints on map element size to maximize resource use and efficiency. When user attempts offload of a map which does not fit into those constraints an informational message is printed to kernel logs to inform user about the reason offload failed. Map offload does not have access to any advanced error reporting like verifier log or extack. There is also currently no way for us to nicely expose the FW capabilities to user space. Given all those constraints we should make sure log messages are as informative as possible. Improve them. Signed-off-by:
-
Jakub Kicinski authored
Record perf maps by map ID, not raw kernel pointer. This helps with debug messages, because printing pointers to logs is frowned upon, and makes debug easier for the users, as map ID is something they should be more familiar with. Note that perf maps are offload neutral, therefore IDs won't be orphaned. While at it use a rate limited print helper for the error message. Reported-by:
Kees Cook <keescook@chromium.org> Signed-off-by:
-
