- 31 Jan, 2022 35 commits
-
-
Alexander Lobakin authored
For now, if the XDP prog returns XDP_PASS on XSK, the metadata will be lost as it doesn't get copied to the skb. Copy it along with the frame headers. Account its size on skb allocation, and when copying just treat it as a part of the frame and do a pull after to "move" it to the "reserved" zone. net_prefetch() xdp->data_meta and align the copy size to speed-up memcpy() a little and better match ixgbe_construct_skb(). Fixes: d0bcacd0 ("ixgbe: add AF_XDP zero-copy Rx support") Suggested-by:
Jesper Dangaard Brouer <brouer@redhat.com> Suggested-by:
Maciej Fijalkowski <maciej.fijalkowski@intel.com> Signed-off-by:
Alexander Lobakin <alexandr.lobakin@intel.com> Reviewed-by:
Michal Swiatkowski <michal.swiatkowski@linux.intel.com> Tested-by:
Sandeep Penigalapati <sandeep.penigalapati@intel.com> Signed-off-by:
Tony Nguyen <anthony.l.nguyen@intel.com>
-
Alexander Lobakin authored
{__,}napi_alloc_skb() allocates and reserves additional NET_SKB_PAD + NET_IP_ALIGN for any skb. OTOH, ixgbe_construct_skb_zc() currently allocates and reserves additional `xdp->data - xdp->data_hard_start`, which is XDP_PACKET_HEADROOM for XSK frames. There's no need for that at all as the frame is post-XDP and will go only to the networking stack core. Pass the size of the actual data only to __napi_alloc_skb() and don't reserve anything. This will give enough headroom for stack processing. Fixes: d0bcacd0 ("ixgbe: add AF_XDP zero-copy Rx support") Signed-off-by: -
Alexander Lobakin authored
To not dereference bi->xdp each time in ixgbe_construct_skb_zc(), pass bi->xdp as an argument instead of bi. We can also call xsk_buff_free() outside of the function as well as assign bi->xdp to NULL, which seems to make it closer to its name. Suggested-by:
-
Alexander Lobakin authored
{__,}napi_alloc_skb() allocates and reserves additional NET_SKB_PAD + NET_IP_ALIGN for any skb. OTOH, igc_construct_skb_zc() currently allocates and reserves additional `xdp->data_meta - xdp->data_hard_start`, which is about XDP_PACKET_HEADROOM for XSK frames. There's no need for that at all as the frame is post-XDP and will go only to the networking stack core. Pass the size of the actual data only (+ meta) to __napi_alloc_skb() and don't reserve anything. This will give enough headroom for stack processing. Also, net_prefetch() xdp->data_meta and align the copy size to speed-up memcpy() a little and better match igc_construct_skb(). Fixes: fc9df2a0 ("igc: Enable RX via AF_XDP zero-copy") Signed-off-by:Nechama Kraus <nechamax.kraus@linux.intel.com> Signed-off-by:
-
Alexander Lobakin authored
For now, if the XDP prog returns XDP_PASS on XSK, the metadata will be lost as it doesn't get copied to the skb. Copy it along with the frame headers. Account its size on skb allocation, and when copying just treat it as a part of the frame and do a pull after to "move" it to the "reserved" zone. net_prefetch() xdp->data_meta and align the copy size to speed-up memcpy() a little and better match ice_construct_skb(). Fixes: 2d4238f5 ("ice: Add support for AF_XDP") Suggested-by:
Kiran Bhandare <kiranx.bhandare@intel.com> Signed-off-by:
-
Alexander Lobakin authored
{__,}napi_alloc_skb() allocates and reserves additional NET_SKB_PAD + NET_IP_ALIGN for any skb. OTOH, ice_construct_skb_zc() currently allocates and reserves additional `xdp->data - xdp->data_hard_start`, which is XDP_PACKET_HEADROOM for XSK frames. There's no need for that at all as the frame is post-XDP and will go only to the networking stack core. Pass the size of the actual data only to __napi_alloc_skb() and don't reserve anything. This will give enough headroom for stack processing. Fixes: 2d4238f5 ("ice: Add support for AF_XDP") Signed-off-by: -
Alexander Lobakin authored
In "legacy-rx" mode represented by ice_construct_skb(), we can still use XDP (and XDP metadata), but after XDP_PASS the metadata will be lost as it doesn't get copied to the skb. Copy it along with the frame headers. Account its size on skb allocation, and when copying just treat it as a part of the frame and do a pull after to "move" it to the "reserved" zone. Point net_prefetch() to xdp->data_meta instead of data. This won't change anything when the meta is not here, but will save some cache misses otherwise. Suggested-by:
-
Alexander Lobakin authored
For now, if the XDP prog returns XDP_PASS on XSK, the metadata will be lost as it doesn't get copied to the skb. Copy it along with the frame headers. Account its size on skb allocation, and when copying just treat it as a part of the frame and do a pull after to "move" it to the "reserved" zone. net_prefetch() xdp->data_meta and align the copy size to speed-up memcpy() a little and better match i40e_construct_skb(). Fixes: 0a714186 ("i40e: add AF_XDP zero-copy Rx support") Suggested-by:
-
Alexander Lobakin authored
{__,}napi_alloc_skb() allocates and reserves additional NET_SKB_PAD + NET_IP_ALIGN for any skb. OTOH, i40e_construct_skb_zc() currently allocates and reserves additional `xdp->data - xdp->data_hard_start`, which is XDP_PACKET_HEADROOM for XSK frames. There's no need for that at all as the frame is post-XDP and will go only to the networking stack core. Pass the size of the actual data only to __napi_alloc_skb() and don't reserve anything. This will give enough headroom for stack processing. Fixes: 0a714186 ("i40e: add AF_XDP zero-copy Rx support") Signed-off-by: -
David S. Miller authored
Haiyang Zhang says: ==================== net: mana: Add XDP counters, reuse dropped pages Add drop, tx counters for XDP. Reuse dropped pages ==================== Signed-off-by:David S. Miller <davem@davemloft.net>
-
Haiyang Zhang authored
Reuse the dropped page in RX path to save page allocation overhead. Signed-off-by:
Haiyang Zhang <haiyangz@microsoft.com> Signed-off-by:
-
Haiyang Zhang authored
This counter will show up in ethtool stat. It is the number of packets received and forwarded by XDP program. Signed-off-by:
-
Haiyang Zhang authored
This counter will show up in ethtool stat data. Signed-off-by:
-
David S. Miller authored
Tony Lu says: ==================== net/smc: Improvements for TCP_CORK and sendfile() Currently, SMC use default implement for syscall sendfile() [1], which is wildly used in nginx and big data sences. Usually, applications use sendfile() with TCP_CORK: fstat(20, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0 setsockopt(19, SOL_TCP, TCP_CORK, [1], 4) = 0 writev(19, [{iov_base="HTTP/1.1 200 OK\r\nServer: nginx/1"..., iov_len=240}], 1) = 240 sendfile(19, 20, [0] => [4096], 4096) = 4096 close(20) = 0 setsockopt(19, SOL_TCP, TCP_CORK, [0], 4) = 0 The above is an example of Nginx, when sendfile() on, Nginx first enables TCP_CORK, write headers, the data will not be sent. Then call sendfile(), it reads file and write to sndbuf. When TCP_CORK is cleared, all pending data is sent out. The performance of the default implement of sendfile is lower than when it is off. After investigation, it shows two parts to improve: - unnecessary lock contention of delayed work - less data per send than when sendfile off Patch #1 tries to reduce lock_sock() contention in smc_tx_work(). Patch #2 removes timed work for corking, and let applications control it. See TCP_CORK [2] MSG_MORE [3]. Patch #3 adds MSG_SENDPAGE_NOTLAST for corking more data when sendfile(). Test environments: - CPU Intel Xeon Platinum 8 core, mem 32 GiB, nic Mellanox CX4 - socket sndbuf / rcvbuf: 16384 / 131072 bytes - server: smc_run nginx - client: smc_run ./wrk -c 100 -t 2 -d 30 http://192.168.100.1:8080/4k.html - payload: 4KB local disk file Items QPS sendfile off 272477.10 sendfile on (orig) 223622.79 sendfile on (this) 395847.21 This benchmark shows +45.28% improvement compared with sendfile off, and +77.02% compared with original sendfile implement. [1] https://man7.org/linux/man-pages/man2/sendfile.2.html [2] https://linux.die.net/man/7/tcp [3] https://man7.org/linux/man-pages/man2/send.2.html ==================== Signed-off-by: -
Tony Lu authored
This introduces a new corked flag, MSG_SENDPAGE_NOTLAST, which is involved in syscall sendfile() [1], it indicates this is not the last page. So we can cork the data until the page is not specify this flag. It has the same effect as MSG_MORE, but existed in sendfile() only. This patch adds a option MSG_SENDPAGE_NOTLAST for corking data, try to cork more data before sending when using sendfile(), which acts like TCP's behaviour. Also, this reimplements the default sendpage to inform that it is supported to some extent. [1] https://man7.org/linux/man-pages/man2/sendfile.2.htmlSigned-off-by:
Tony Lu <tonylu@linux.alibaba.com> Signed-off-by:
-
Tony Lu authored
Based on the manual of TCP_CORK [1] and MSG_MORE [2], these two options have the same effect. Applications can set these options and informs the kernel to pend the data, and send them out only when the socket or syscall does not specify this flag. In other words, there's no need to send data out by a delayed work, which will queue a lot of work. This removes corked delayed work with SMC_TX_CORK_DELAY (250ms), and the applications control how/when to send them out. It improves the performance for sendfile and throughput, and remove unnecessary race of lock_sock(). This also unlocks the limitation of sndbuf, and try to fill it up before sending. [1] https://linux.die.net/man/7/tcp [2] https://man7.org/linux/man-pages/man2/send.2.htmlSigned-off-by:
-
Tony Lu authored
According to the man page of TCP_CORK [1], if set, don't send out partial frames. All queued partial frames are sent when option is cleared again. When applications call setsockopt to disable TCP_CORK, this call is protected by lock_sock(), and tries to mod_delayed_work() to 0, in order to send pending data right now. However, the delayed work smc_tx_work is also protected by lock_sock(). There introduces lock contention for sending data. To fix it, send pending data directly which acts like TCP, without lock_sock() protected in the context of setsockopt (already lock_sock()ed), and cancel unnecessary dealyed work, which is protected by lock. [1] https://linux.die.net/man/7/tcpSigned-off-by:
-
David S. Miller authored
Akhmat Karakotov says: ==================== Make hash rethink configurable As it was shown in the report by Alexander Azimov, hash rethink at the client-side may lead to connection timeout toward stateful anycast services. Tom Herbert created a patchset to address this issue by applying hash rethink only after a negative routing event (3RTOs) [1]. This change also affects server-side behavior, which we found undesirable. This patchset changes defaults in a way to make them safe: hash rethink at the client-side is disabled and enabled at the server-side upon each RTO event or in case of duplicate acknowledgments. This patchset provides two options to change default behaviour. The hash rethink may be disabled at the server-side by the new sysctl option. Changes in the sysctl option don't affect default behavior at the client-side. Hash rethink can also be enabled/disabled with socket option or bpf syscalls which ovewrite both default and sysctl settings. This socket option is available on both client and server-side. This should provide mechanics to enable hash rethink inside administrative domain, such as DC, where hash rethink at the client-side can be desirable. [1] https://lore.kernel.org/netdev/20210809185314.38187-1-tom@herbertland.com/ v2: - Changed sysctl default to ENABLED in all patches. Reduced sysctl and socket option size to u8. Fixed netns bug reported by kernel test robot. v3: - Fixed bug with bad u8 comparison. Moved sk_txrehash to use less bytes in struct. Added WRITE_ONCE() in setsockopt in and READ_ONCE() in tcp_rtx_synack. v4: - Rebase and add documentation for sysctl option. v5: - Move sk_txrehash out of busy poll ifdef. ==================== Signed-off-by:
-
Akhmat Karakotov authored
Disabling rehash behavior did not affect SYN ACK retransmits because hash was forcefully changed bypassing the sk_rethink_hash function. This patch adds a condition which checks for rehash mode before resetting hash. Signed-off-by:
Akhmat Karakotov <hmukos@yandex-team.ru> Reviewed-by:
Eric Dumazet <edumazet@google.com> Signed-off-by:
-
Akhmat Karakotov authored
Add bpf socket option to override rehash behaviour from userspace or from bpf. Signed-off-by:
-
Akhmat Karakotov authored
Update Documentation/admin-guide/sysctl/net.rst with txrehash usage description. Signed-off-by:
-
Akhmat Karakotov authored
Add the SO_TXREHASH socket option to control hash rethink behavior per socket. When default mode is set, sockets disable rehash at initialization and use sysctl option when entering listen state. setsockopt() overrides default behavior. Signed-off-by:
-
Akhmat Karakotov authored
Add a per ns sysctl that controls the txhash rethink behavior: net.core.txrehash. When enabled, the same behavior is retained, when disabled, rethink is not performed. Sysctl is enabled by default. Signed-off-by:
-
Gerhard Engleder authored
timestamping checks socket options during initialisation. For the field bind_phc of the socket option SO_TIMESTAMPING it expects the value -1 if PHC is not bound. Actually the value of bind_phc is 0 if PHC is not bound. This results in the following output: SIOCSHWTSTAMP: tx_type 0 requested, got 0; rx_filter 0 requested, got 0 SO_TIMESTAMP 0 SO_TIMESTAMPNS 0 SO_TIMESTAMPING flags 0, bind phc 0 not expected, flags 0, bind phc -1 This is fixed by setting default value and expected value of bind_phc to 0. Fixes: 2214d703 ("selftests/net: timestamping: support binding PHC") Signed-off-by:
Gerhard Engleder <gerhard@engleder-embedded.com> Signed-off-by:
-
David S. Miller authored
Sergey Shtylyov says: ==================== Remove some dead code in the Renesas Ethernet drivers Here are 2 patches against DaveM's 'net-next.git' repo. The Renesas drivers call their ndo_stop() methods directly and they always return 0, making the result checks pointless, hence remove them... ==================== Signed-off-by: -
Sergey Shtylyov authored
sh_eth_close() always returns 0, hence the check in sh_eth_wol_restore() is pointless (however we cannot change the prototype of sh_eth_close() as it implements the driver's ndo_stop() method). Found by Linux Verification Center (linuxtesting.org) with the SVACE static analysis tool. Signed-off-by:
Sergey Shtylyov <s.shtylyov@omp.ru> Signed-off-by:
-
Sergey Shtylyov authored
ravb_close() always returns 0, hence the check in ravb_wol_restore() is pointless (however, we cannot change the prototype of ravb_close() as it implements the driver's ndo_stop() method). Found by Linux Verification Center (linuxtesting.org) with the SVACE static analysis tool. Signed-off-by:
-
Wei Yongjun authored
In case of error, the function devm_ioremap() returns NULL pointer not ERR_PTR(). The IS_ERR() test in the return value check should be replaced with NULL test. Fixes: 1d14eb15 ("net/fsl: xgmac_mdio: Use managed device resources") Reported-by:
Hulk Robot <hulkci@huawei.com> Signed-off-by:
Wei Yongjun <weiyongjun1@huawei.com> Reviewed-by:
Tobias Waldekranz <tobias@waldekranz.com> Signed-off-by:
-
David Ahern authored
ip_idents_reserve is only used in net/ipv4/route.c. Make it static and remove the export. Signed-off-by:
David Ahern <dsahern@kernel.org> Cc: Eric Dumazet <edumazet@google.com> Reviewed-by:
-
Heiner Kallweit authored
Add rtl_disable_exit_l1() for ensuring that the chip doesn't inadvertently exit ASPM L1 when being in a low-power mode. The new function is called from rtl_prepare_power_down() which has to be moved in the code to avoid a forward declaration. According to Realtek OCP register 0xc0ac shadows ERI register 0xd4 on RTL8168 versions from RTL8168g. This allows to simplify the code a little. v2: - call rtl_disable_exit_l1() also if DASH or WoL are enabled Suggested-by:
Chun-Hao Lin <hau@realtek.com> Signed-off-by:
Heiner Kallweit <hkallweit1@gmail.com> Signed-off-by:
-
Sergey Shtylyov authored
After following the call tree of phy_set_max_speed(), it became clear that this function never returns anything but 0, so we can change its result type to *void* and drop the result checks from the three drivers that actually bothered to do it... Found by Linux Verification Center (linuxtesting.org) with the SVACE static analysis tool. Signed-off-by:
Andrew Lunn <andrew@lunn.ch> Signed-off-by:
-
David S. Miller authored
Tobias Waldekranz says: ==================== net: dsa: mv88e6xxx: Improve indirect addressing performance The individual patches have all the details. This work was triggered by recent work on a platform that took 16s (sic) to load the mv88e6xxx module. The first patch gets rid of most of that time by replacing a very long delay with a tighter poll loop to wait for the busy bit to clear. The second patch shaves off some more time by avoiding redundant busy-bit-checks, saving 1 out of 4 MDIO operations for every register read/write in the optimal case. v1 -> v2: - Make sure that we always poll the busy bit at least twice, in the unlikely event that the first one is quick to query the hardware, but is then scheduled out for a long time before the timeout is checked. v2 -> v3: - Fallback to the longer sleeps after the initial two poll attempts. ==================== Signed-off-by: -
Tobias Waldekranz authored
Before this change, both the read and write callback would start out by asserting that the chip's busy flag was cleared. However, both callbacks also made sure to wait for the clearing of the busy bit before returning - making the initial check superfluous. The only time that would ever have an effect was if the busy bit was initially set for some reason. With that in mind, make sure to perform an initial check of the busy bit, after which both read and write can rely the previous operation to have waited for the bit to clear. This cuts the number of operations on the underlying MDIO bus by 25% Signed-off-by:
-
Tobias Waldekranz authored
Avoid a long delay when a busy bit is still set and has to be polled again. Measurements on a system with 2 Opals (6097F) and one Agate (6352) show that even with this much tighter loop, we have about a 50% chance of the bit being cleared on the first poll, all other accesses see the bit being cleared on the second poll. On a standard MDIO bus running MDC at 2.5MHz, a single access with 32 bits of preamble plus 32 bits of data takes 64*(1/2.5MHz) = 25.6us. This means that mv88e6xxx_smi_direct_wait took 26us + CPU overhead in the fast scenario, but 26us + 1500us + 26us + CPU overhead in the slow case - bringing the average close to 1ms. With this change in place, the slow case is closer to 2*26us + CPU overhead, with the average well below 100us - a 10x improvement. This translates to real-world winnings. On a 3-chip 20-port system, the modprobe time drops by 88%: Before: root@coronet:~# time modprobe mv88e6xxx real 0m 15.99s user 0m 0.00s sys 0m 1.52s After: root@coronet:~# time modprobe mv88e6xxx real 0m 2.21s user 0m 0.00s sys 0m 1.54s Signed-off-by:
-
Sun Shouxin authored
Since ipv6 neighbor solicitation and advertisement messages isn't handled gracefully in bond6 driver, we can see packet drop due to inconsistency between mac address in the option message and source MAC . Another examples is ipv6 neighbor solicitation and advertisement messages from VM via tap attached to host bridge, the src mac might be changed through balance-alb mode, but it is not synced with Link-layer address in the option message. The patch implements bond6's tx handle for ipv6 neighbor solicitation and advertisement messages. Suggested-by:
Hu Yadi <huyd12@chinatelecom.cn> Acked-by:
Jay Vosburgh <jay.vosburgh@canonical.com> Signed-off-by:
Sun Shouxin <sunshouxin@chinatelecom.cn> Signed-off-by:
-
- 29 Jan, 2022 5 commits
-
-
Jakub Kicinski authored
Since v2.5.44 and addition of ip_options_fragment() ip_options_build() does not render headers for fragments directly. @is_frag is always 0. Signed-off-by:
Jakub Kicinski <kuba@kernel.org> Signed-off-by:
-
David S. Miller authored
Robert Hancock says: ==================== Cadence MACB/GEM support for ZynqMP SGMII Changes to allow SGMII mode to work properly in the GEM driver on the Xilinx ZynqMP platform. Changes since v3: -more code formatting and error handling fixes Changes since v2: -fixed missing includes in DT binding example -fixed phy_init and phy_power_on error handling/cleanup, moved phy_power_on to open rather than probe Changes since v1: -changed order of controller reset and PHY init as per suggestion -switched device reset to be optional -updated bindings doc patch for switch to YAML ==================== Signed-off-by: -
Robert Hancock authored
The Cadence GEM/MACB driver now utilizes the platform-level reset on the ZynqMP platform. Add reset definitions to the ZynqMP platform device tree to allow this to be used. Signed-off-by:
Robert Hancock <robert.hancock@calian.com> Signed-off-by:
-
Robert Hancock authored
The GEM controllers on ZynqMP were missing some initialization steps which are required in some cases when using SGMII mode, which uses the PS-GTR transceivers managed by the phy-zynqmp driver. The GEM core appears to need a hardware-level reset in order to work properly in SGMII mode in cases where the GT reference clock was not present at initial power-on. This can be done using a reset mapped to the zynqmp-reset driver in the device tree. Also, when in SGMII mode, the GEM driver needs to ensure the PHY is initialized and powered on. Signed-off-by:
Claudiu Beznea <claudiu.beznea@microchip.com> Signed-off-by:
-
Robert Hancock authored
Updated macb DT binding documentation to reflect the phy-names, phys, resets, reset-names properties which are now used with ZynqMP GEM devices, and added a ZynqMP-specific DT example. Signed-off-by:
-
