- 10 Apr, 2022 11 commits
-
-
Jakub Kicinski authored
The current invese logic is harder to follow (and adds extra tests to the fast path). We have to enumerate all cases which need to keep the skb before consuming it. It's simpler to jump out of the full record flow as we detect those cases. This makes it clear that partial consumption and peek can only reach end of the function thru the !zc case so move the code up there. Signed-off-by:
Jakub Kicinski <kuba@kernel.org> Signed-off-by:
David S. Miller <davem@davemloft.net>
-
Jakub Kicinski authored
Whatever we do in the loop the skb should not remain on as ctx->recv_pkt afterwards. We can clear that pointer and restart strparser earlier. This adds overhead of extra linking and unlinking to rx_list but that's not large (upcoming change will switch to unlocked skb list operations). Signed-off-by:
-
Jakub Kicinski authored
tls_sw_advance_skb() always consumes the skb at the end of the loop. To fall here the following must be true: !async && !is_peek && !retain_skb retain_skb => !zc && rxm->full_len > len # but non-full record implies !zc, so above can be simplified as retain_skb => rxm->full_len > len !async && !is_peek && !(rxm->full_len > len) !async && !is_peek && rxm->full_len <= len tls_sw_advance_skb() returns false if len < rxm->full_len which can't be true given conditions above. Signed-off-by: -
Jakub Kicinski authored
Most of the conditions deciding if zero-copy can be used do not change throughout the iterations, so pre-calculate them. Signed-off-by:
-
Jakub Kicinski authored
We track both if the last record was handled by async crypto and how many records were async. This is not necessary. We implicitly assume once crypto goes async it will stay that way, otherwise we'd reorder records. So just track if we're in async mode, the exact number of records is not necessary. This change also forces us into "async" mode more consistently in case crypto ever decided to interleave async and sync. Signed-off-by:
-
Jakub Kicinski authored
tls_sw_advance_skb() caters to the async case when skb argument is NULL. In that case it simply unpauses the strparser. These are surprising semantics to a person reading the code, and result in higher LoC, so inline the __strp_unpause and only call tls_sw_advance_skb() when we actually move past an skb. Signed-off-by:
-
Jakub Kicinski authored
cmsg can be filled in during rx_list processing or normal receive. Consolidate the code. We don't need to keep the boolean to track if the cmsg was created. 0 is an invalid content type. Signed-off-by:
-
Jakub Kicinski authored
Since we are protected from async completions by decrypt_compl_lock we can drop the async_notify and reinit the completion before we start waiting. Signed-off-by:
-
Jakub Kicinski authored
We pass zc as a pointer to bool a few functions down as an in/out argument. This is error prone since C will happily evalue a pointer as a boolean (IOW forgetting *zc and writing zc leads to loss of developer time..). Wrap the arguments into a structure. Signed-off-by:
-
Jakub Kicinski authored
We plumb pointer to chunk all the way to the decryption method. It's set to the length of the text when decrypt_skb_update() returns. I think the code is written this way because original TLS implementation passed &chunk to zerocopy_from_iter() and this was carried forward as the code gotten more complex, without any refactoring. The fix for peek() introduced a new variable - to_decrypt which for all practical purposes is what chunk is going to get set to. Spare ourselves the pointer passing, use to_decrypt. Use this opportunity to clean things up a little further. Note that chunk / to_decrypt was mostly needed for the async path, since the sync path would access rxm->full_len (decryption transforms full_len from record size to text size). Use the right source of truth more explicitly. We have three cases: - async - it's TLS 1.2 only, so chunk == to_decrypt, but we need the min() because to_decrypt is a whole record and we don't want to underflow len. Note that we can't handle partial record by falling back to sync as it would introduce reordering against records in flight. - zc - again, TLS 1.2 only for now, so chunk == to_decrypt, we don't do zc if len < to_decrypt, no need to check again. - normal - it already handles chunk > len, we can factor out the assignment to rxm->full_len and share it with zc. Signed-off-by: -
Jakub Kicinski authored
sk is unused, remove it to make it clear the function doesn't poke at the socket. size_used is always 0 on input and @length on success. Signed-off-by:
-
- 09 Apr, 2022 2 commits
-
-
Colin Foster authored
Create a local device *dev in order to not dereference the platform_device several times throughout the probe function. Signed-off-by:
Colin Foster <colin.foster@in-advantage.com> Reviewed-by:
Vladimir Oltean <vladimir.oltean@nxp.com> Signed-off-by:
-
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-nextJakub Kicinski authored
Daniel Borkmann says: ==================== pull-request: bpf-next 2022-04-09 We've added 63 non-merge commits during the last 9 day(s) which contain a total of 68 files changed, 4852 insertions(+), 619 deletions(-). The main changes are: 1) Add libbpf support for USDT (User Statically-Defined Tracing) probes. USDTs are an abstraction built on top of uprobes, critical for tracing and BPF, and widely used in production applications, from Andrii Nakryiko. 2) While Andrii was adding support for x86{-64}-specific logic of parsing USDT argument specification, Ilya followed-up with USDT support for s390 architecture, from Ilya Leoshkevich. 3) Support name-based attaching for uprobe BPF programs in libbpf. The format supported is `u[ret]probe/binary_path:[raw_offset|function[+offset]]`, e.g. attaching to libc malloc can be done in BPF via SEC("uprobe/libc.so.6:malloc") now, from Alan Maguire. 4) Various load/store optimizations for the arm64 JIT to shrink the image size by using arm64 str/ldr immediate instructions. Also enable pointer authentication to verify return address for JITed code, from Xu Kuohai. 5) BPF verifier fixes for write access checks to helper functions, e.g. rd-only memory from bpf_*_cpu_ptr() must not be passed to helpers that write into passed buffers, from Kumar Kartikeya Dwivedi. 6) Fix overly excessive stack map allocation for its base map structure and buckets which slipped-in from cleanups during the rlimit accounting removal back then, from Yuntao Wang. 7) Extend the unstable CT lookup helpers for XDP and tc/BPF to report netfilter connection tracking tuple direction, from Lorenzo Bianconi. 8) Improve bpftool dump to show BPF program/link type names, Milan Landaverde. 9) Minor cleanups all over the place from various others. * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (63 commits) bpf: Fix excessive memory allocation in stack_map_alloc() selftests/bpf: Fix return value checks in perf_event_stackmap test selftests/bpf: Add CO-RE relos into linked_funcs selftests libbpf: Use weak hidden modifier for USDT BPF-side API functions libbpf: Don't error out on CO-RE relos for overriden weak subprogs samples, bpf: Move routes monitor in xdp_router_ipv4 in a dedicated thread libbpf: Allow WEAK and GLOBAL bindings during BTF fixup libbpf: Use strlcpy() in path resolution fallback logic libbpf: Add s390-specific USDT arg spec parsing logic libbpf: Make BPF-side of USDT support work on big-endian machines libbpf: Minor style improvements in USDT code libbpf: Fix use #ifdef instead of #if to avoid compiler warning libbpf: Potential NULL dereference in usdt_manager_attach_usdt() selftests/bpf: Uprobe tests should verify param/return values libbpf: Improve string parsing for uprobe auto-attach libbpf: Improve library identification for uprobe binary path resolution selftests/bpf: Test for writes to map key from BPF helpers selftests/bpf: Test passing rdonly mem to global func bpf: Reject writes for PTR_TO_MAP_KEY in check_helper_mem_access bpf: Check PTR_TO_MEM | MEM_RDONLY in check_helper_mem_access ... ==================== Link: https://lore.kernel.org/r/20220408231741.19116-1-daniel@iogearbox.netSigned-off-by:
-
- 08 Apr, 2022 27 commits
-
-
Yuntao Wang authored
The 'n_buckets * (value_size + sizeof(struct stack_map_bucket))' part of the allocated memory for 'smap' is never used after the memlock accounting was removed, thus get rid of it. [ Note, Daniel: Commit b936ca64 ("bpf: rework memlock-based memory accounting for maps") moved `cost += n_buckets * (value_size + sizeof(struct stack_map_bucket))` up and therefore before the bpf_map_area_alloc() allocation, sigh. In a later step commit c85d6913 ("bpf: move memory size checks to bpf_map_charge_init()"), and the overflow checks of `cost >= U32_MAX - PAGE_SIZE` moved into bpf_map_charge_init(). And then 37086810 ("bpf: Eliminate rlimit-based memory accounting for stackmap maps") finally removed the bpf_map_charge_init(). Anyway, the original code did the allocation same way as /after/ this fix. ] Fixes: b936ca64 ("bpf: rework memlock-based memory accounting for maps") Signed-off-by:
Yuntao Wang <ytcoode@gmail.com> Signed-off-by:
Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20220407130423.798386-1-ytcoode@gmail.com
-
Bert Kenward authored
The 8000 series and newer NICs all get hardware timestamps from the MAC and can provide timestamps on a normal TX queue, rather than via a slow path through the MC. As such we can use this path for any packet where a hardware timestamp is requested. This also enables support for PTP over transports other than IPv4+UDP. Signed-off-by:
Bert Kenward <bkenward@solarflare.com> Signed-off-by:
Edward Cree <ecree@xilinx.com> Link: https://lore.kernel.org/r/510652dc-54b4-0e11-657e-e37ee3ca26a9@gmail.comSigned-off-by:
-
Marek Vasut authored
Add cable test support for Micrel KSZ9x31 PHYs. Tested on i.MX8M Mini with KSZ9131RNX in 100/Full mode with pairs shuffled before magnetics: (note: Cable test started/completed messages are omitted) mx8mm-ksz9131-a-d-connected$ ethtool --cable-test eth0 Pair A code OK Pair B code Short within Pair Pair B, fault length: 0.80m Pair C code Short within Pair Pair C, fault length: 0.80m Pair D code OK mx8mm-ksz9131-a-b-connected$ ethtool --cable-test eth0 Pair A code OK Pair B code OK Pair C code Short within Pair Pair C, fault length: 0.00m Pair D code Short within Pair Pair D, fault length: 0.00m Tested on R8A77951 Salvator-XS with KSZ9031RNX and all four pairs connected: (note: Cable test started/completed messages are omitted) r8a7795-ksz9031-all-connected$ ethtool --cable-test eth0 Pair A code OK Pair B code OK Pair C code OK Pair D code OK The CTRL1000 CTL1000_ENABLE_MASTER and CTL1000_AS_MASTER bits are not restored by calling phy_init_hw(), they must be manually cached in ksz9x31_cable_test_start() and restored at the end of ksz9x31_cable_test_get_status(). Signed-off-by:
Marek Vasut <marex@denx.de> Cc: Heiner Kallweit <hkallweit1@gmail.com> Cc: Oleksij Rempel <linux@rempel-privat.de> Cc: Paolo Abeni <pabeni@redhat.com> Reviewed-by:
Andrew Lunn <andrew@lunn.ch> Tested-by:
Claudiu Beznea <claudiu.beznea@microchip.com> Link: https://lore.kernel.org/r/20220407105534.85833-1-marex@denx.deSigned-off-by:
-
Yuntao Wang authored
The bpf_get_stackid() function may also return 0 on success as per UAPI BPF helper documentation. Therefore, correct checks from 'val > 0' to 'val >= 0' to ensure that they cover all possible success return values. Signed-off-by:
-
Andrii Nakryiko authored
Add CO-RE relocations into __weak subprogs for multi-file linked_funcs selftest to make sure libbpf handles such combination well. Signed-off-by:
Andrii Nakryiko <andrii@kernel.org> Signed-off-by:
-
Andrii Nakryiko authored
Use __weak __hidden for bpf_usdt_xxx() APIs instead of much more confusing `static inline __noinline`. This was previously impossible due to libbpf erroring out on CO-RE relocations pointing to eliminated weak subprogs. Now that previous patch fixed this issue, switch back to __weak __hidden as it's a more direct way of specifying the desired behavior. Signed-off-by:
-
Andrii Nakryiko authored
During BPF static linking, all the ELF relocations and .BTF.ext information (including CO-RE relocations) are preserved for __weak subprograms that were logically overriden by either previous weak subprogram instance or by corresponding "strong" (non-weak) subprogram. This is just how native user-space linkers work, nothing new. But libbpf is over-zealous when processing CO-RE relocation to error out when CO-RE relocation belonging to such eliminated weak subprogram is encountered. Instead of erroring out on this expected situation, log debug-level message and skip the relocation. Fixes: db2b8b06 ("libbpf: Support CO-RE relocations for multi-prog sections") Signed-off-by:
-
Lorenzo Bianconi authored
In order to not miss any netlink message from the kernel, move routes monitor to a dedicated thread. Fixes: 85bf1f51 ("samples: bpf: Convert xdp_router_ipv4 to XDP samples helper") Signed-off-by:
Lorenzo Bianconi <lorenzo@kernel.org> Signed-off-by:
-
Andrii Nakryiko authored
During BTF fix up for global variables, global variable can be global weak and will have STB_WEAK binding in ELF. Support such global variables in addition to non-weak ones. This is not the problem when using BPF static linking, as BPF static linker "fixes up" BTF during generation so that libbpf doesn't have to do it anymore during bpf_object__open(), which led to this not being noticed for a while, along with a pretty rare (currently) use of __weak variables and maps. Reported-by:
Hengqi Chen <hengqi.chen@gmail.com> Signed-off-by:
Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20220407230446.3980075-2-andrii@kernel.org
-
Andrii Nakryiko authored
Coverity static analyzer complains that strcpy() can cause buffer overflow. Use libbpf_strlcpy() instead to be 100% sure this doesn't happen. Signed-off-by:
-
Andrii Nakryiko authored
Ilya Leoshkevich says: ==================== This series adds USDT support for s390, making the "usdt" test pass there. Patch 1 is a collection of minor cleanups, patch 2 adds BPF-side support, patch 3 adds userspace-side support. ==================== Signed-off-by: -
Ilya Leoshkevich authored
The logic is superficially similar to that of x86, but the small differences (no need for register table and dynamic allocation of register names, no $ sign before constants) make maintaining a common implementation too burdensome. Therefore simply add a s390x-specific version of parse_usdt_arg(). Note that while bcc supports index registers, this patch does not. This should not be a problem in most cases, since s390 uses a default value "nor" for STAP_SDT_ARG_CONSTRAINT. Signed-off-by:
Ilya Leoshkevich <iii@linux.ibm.com> Signed-off-by:
-
David S. Miller authored
Ido Schimmel says: ==================== net/sched: Better error reporting for offload failures This patchset improves error reporting to user space when offload fails during the flow action setup phase. That is, when failures occur in the actions themselves, even before calling device drivers. Requested / reported in [1]. This is done by passing extack to the offload_act_setup() callback and making use of it in the various actions. Patches #1-#2 change matchall and flower to log error messages to user space in accordance with the verbose flag. Patch #3 passes extack to the offload_act_setup() callback from the various call sites, including matchall and flower. Patches #4-#11 make use of extack in the various actions to report offload failures. Patch #12 adds an error message when the action does not support offload at all. Patches #13-#14 change matchall and flower to stop overwriting more specific error messages. [1] https://lore.kernel.org/netdev/20220317185249.5mff5u2x624pjewv@skbuf/ ==================== Signed-off-by:
-
Ido Schimmel authored
The various error paths of tc_setup_offload_action() now report specific error messages. Remove the generic messages to avoid overwriting the more specific ones. Before: # tc filter add dev dummy0 ingress pref 1 proto ip flower skip_sw dst_ip 198.51.100.1 action police rate 100Mbit burst 10000 Error: cls_flower: Failed to setup flow action. We have an error talking to the kernel After: # tc filter add dev dummy0 ingress pref 1 proto ip flower skip_sw dst_ip 198.51.100.1 action police rate 100Mbit burst 10000 Error: act_police: Offload not supported when conform/exceed action is "reclassify". We have an error talking to the kernel Signed-off-by:
Ido Schimmel <idosch@nvidia.com> Reviewed-by:
Petr Machata <petrm@nvidia.com> Signed-off-by:
-
Ido Schimmel authored
The various error paths of tc_setup_offload_action() now report specific error messages. Remove the generic messages to avoid overwriting the more specific ones. Before: # tc filter add dev dummy0 ingress pref 1 proto all matchall skip_sw action police rate 100Mbit burst 10000 Error: cls_matchall: Failed to setup flow action. We have an error talking to the kernel After: # tc filter add dev dummy0 ingress pref 1 proto all matchall skip_sw action police rate 100Mbit burst 10000 Error: act_police: Offload not supported when conform/exceed action is "reclassify". We have an error talking to the kernel Signed-off-by:
-
Ido Schimmel authored
For better error reporting to user space, add an extack message when the requested action does not support offload. Example: # echo 1 > /sys/kernel/tracing/events/netlink/netlink_extack/enable # tc filter add dev dummy0 ingress pref 1 proto all matchall skip_sw action nat ingress 192.0.2.1 198.51.100.1 Error: cls_matchall: Failed to setup flow action. We have an error talking to the kernel # cat /sys/kernel/tracing/trace_pipe tc-181 [000] b..1. 88.406093: netlink_extack: msg=Action does not support offload tc-181 [000] ..... 88.406108: netlink_extack: msg=cls_matchall: Failed to setup flow action Signed-off-by: -
Ido Schimmel authored
For better error reporting to user space, add an extack message when vlan action offload fails. Currently, the failure cannot be triggered, but add a message in case the action is extended in the future to support more than the current set of modes. Signed-off-by:
-
Ido Schimmel authored
For better error reporting to user space, add an extack message when tunnel_key action offload fails. Currently, the failure cannot be triggered, but add a message in case the action is extended in the future to support more than set/release modes. Signed-off-by:
-
Ido Schimmel authored
For better error reporting to user space, add extack messages when skbedit action offload fails. Example: # echo 1 > /sys/kernel/tracing/events/netlink/netlink_extack/enable # tc filter add dev dummy0 ingress pref 1 proto all matchall skip_sw action skbedit queue_mapping 1234 Error: cls_matchall: Failed to setup flow action. We have an error talking to the kernel # cat /sys/kernel/tracing/trace_pipe tc-185 [002] b..1. 31.802414: netlink_extack: msg=act_skbedit: Offload not supported when "queue_mapping" option is used tc-185 [002] ..... 31.802418: netlink_extack: msg=cls_matchall: Failed to setup flow action # tc filter add dev dummy0 ingress pref 1 proto all matchall skip_sw action skbedit inheritdsfield Error: cls_matchall: Failed to setup flow action. We have an error talking to the kernel # cat /sys/kernel/tracing/trace_pipe tc-187 [002] b..1. 45.985145: netlink_extack: msg=act_skbedit: Offload not supported when "inheritdsfield" option is used tc-187 [002] ..... 45.985160: netlink_extack: msg=cls_matchall: Failed to setup flow action Signed-off-by: -
Ido Schimmel authored
For better error reporting to user space, add extack messages when police action offload fails. Example: # echo 1 > /sys/kernel/tracing/events/netlink/netlink_extack/enable # tc filter add dev dummy0 ingress pref 1 proto all matchall skip_sw action police rate 100Mbit burst 10000 Error: cls_matchall: Failed to setup flow action. We have an error talking to the kernel # cat /sys/kernel/tracing/trace_pipe tc-182 [000] b..1. 21.592969: netlink_extack: msg=act_police: Offload not supported when conform/exceed action is "reclassify" tc-182 [000] ..... 21.592982: netlink_extack: msg=cls_matchall: Failed to setup flow action # tc filter add dev dummy0 ingress pref 1 proto all matchall skip_sw action police rate 100Mbit burst 10000 conform-exceed drop/continue Error: cls_matchall: Failed to setup flow action. We have an error talking to the kernel # cat /sys/kernel/tracing/trace_pipe tc-184 [000] b..1. 38.882579: netlink_extack: msg=act_police: Offload not supported when conform/exceed action is "continue" tc-184 [000] ..... 38.882593: netlink_extack: msg=cls_matchall: Failed to setup flow action Signed-off-by: -
Ido Schimmel authored
For better error reporting to user space, add an extack message when pedit action offload fails. Currently, the failure cannot be triggered, but add a message in case the action is extended in the future to support more than set/add commands. Signed-off-by:
-
Ido Schimmel authored
For better error reporting to user space, add extack messages when mpls action offload fails. Example: # echo 1 > /sys/kernel/tracing/events/netlink/netlink_extack/enable # tc filter add dev dummy0 ingress pref 1 proto all matchall skip_sw action mpls dec_ttl Error: cls_matchall: Failed to setup flow action. We have an error talking to the kernel # cat /sys/kernel/tracing/trace_pipe tc-182 [000] b..1. 18.693915: netlink_extack: msg=act_mpls: Offload not supported when "dec_ttl" option is used tc-182 [000] ..... 18.693921: netlink_extack: msg=cls_matchall: Failed to setup flow action Signed-off-by: -
Ido Schimmel authored
For better error reporting to user space, add an extack message when mirred action offload fails. Currently, the failure cannot be triggered, but add a message in case the action is extended in the future to support more than ingress/egress mirror/redirect. Signed-off-by:
-
Ido Schimmel authored
For better error reporting to user space, add extack messages when gact action offload fails. Example: # echo 1 > /sys/kernel/tracing/events/netlink/netlink_extack/enable # tc filter add dev dummy0 ingress pref 1 proto all matchall skip_sw action continue Error: cls_matchall: Failed to setup flow action. We have an error talking to the kernel # cat /sys/kernel/tracing/trace_pipe tc-181 [002] b..1. 105.493450: netlink_extack: msg=act_gact: Offload of "continue" action is not supported tc-181 [002] ..... 105.493466: netlink_extack: msg=cls_matchall: Failed to setup flow action # tc filter add dev dummy0 ingress pref 1 proto all matchall skip_sw action reclassify Error: cls_matchall: Failed to setup flow action. We have an error talking to the kernel # cat /sys/kernel/tracing/trace_pipe tc-183 [002] b..1. 124.126477: netlink_extack: msg=act_gact: Offload of "reclassify" action is not supported tc-183 [002] ..... 124.126489: netlink_extack: msg=cls_matchall: Failed to setup flow action # tc filter add dev dummy0 ingress pref 1 proto all matchall skip_sw action pipe action drop Error: cls_matchall: Failed to setup flow action. We have an error talking to the kernel # cat /sys/kernel/tracing/trace_pipe tc-185 [002] b..1. 137.097791: netlink_extack: msg=act_gact: Offload of "pipe" action is not supported tc-185 [002] ..... 137.097804: netlink_extack: msg=cls_matchall: Failed to setup flow action Signed-off-by: -
Ido Schimmel authored
The callback is used by various actions to populate the flow action structure prior to offload. Pass extack to this callback so that the various actions will be able to report accurate error messages to user space. Signed-off-by:
-
Ido Schimmel authored
The verbose flag was added in commit 81c7288b ("sched: cls: enable verbose logging") to avoid suppressing logging of error messages that occur "when the rule is not to be exclusively executed by the hardware". However, such error messages are currently suppressed when setup of flow action fails. Take the verbose flag into account to avoid suppressing error messages. This is done by using the extack pointer initialized by tc_cls_common_offload_init(), which performs the necessary checks. Before: # tc filter add dev dummy0 ingress pref 1 proto ip flower dst_ip 198.51.100.1 action police rate 100Mbit burst 10000 # tc filter add dev dummy0 ingress pref 2 proto ip flower verbose dst_ip 198.51.100.1 action police rate 100Mbit burst 10000 After: # tc filter add dev dummy0 ingress pref 1 proto ip flower dst_ip 198.51.100.1 action police rate 100Mbit burst 10000 # tc filter add dev dummy0 ingress pref 2 proto ip flower verbose dst_ip 198.51.100.1 action police rate 100Mbit burst 10000 Warning: cls_flower: Failed to setup flow action. Signed-off-by:
-
Ido Schimmel authored
The verbose flag was added in commit 81c7288b ("sched: cls: enable verbose logging") to avoid suppressing logging of error messages that occur "when the rule is not to be exclusively executed by the hardware". However, such error messages are currently suppressed when setup of flow action fails. Take the verbose flag into account to avoid suppressing error messages. This is done by using the extack pointer initialized by tc_cls_common_offload_init(), which performs the necessary checks. Signed-off-by:
-
