Skip to content

L
linux
Commit 1f12177b authored by David S. Miller's avatar David S. Miller
Browse files
Options

Merge branch 'cpsw-switchdev' 

Grygorii Strashko says:

====================
net: ethernet: ti: introduce new cpsw switchdev based driver

Thank you All for review of v6.

There are no significant changes in this version, just fixed comments to v6.

--- v6
The major change in this version is DT bindings conversation to json-schema, and
fixed other comments to v5. Also added patch to clean up ALE on init and netif
restart.

--- v5
The major part of work done in this iteration is rebasing on top of net-next
with XDP series from Ivan Khoronzhuk [3], and enable XDP support in the new
CPSW switchdev driver (it was little bit painful ;(). There are mostly no
functional changes in new CPSW driver, just few fixes, sync with old driver
and cleanups/optimizations. So, I've kept rest of cover letter unchanged.

---
This series originally based on work [1][2] done by
Ilias Apalodimas <ilias.apalodimas@linaro.org>.

This the RFC v5 which introduces new CPSW switchdev based driver which is
operating in dual-emac mode by default, thus working as 2 individual
network interfaces. The Switch mode can be enabled by configuring devlink driver
parameter "switch_mode" to 1/true:
	devlink dev param set platform/48484000.switch \
	name switch_mode value 1 cmode runtime
This can be done regardless of the state of Port's netdev devices - UP/DOWN, but
Port's netdev devices have to be in UP before joining the bridge to avoid
overwriting of bridge configuration as CPSW switch driver completely reloads its
configuration when first Port changes its state to UP.
When the both interfaces joined the bridge - CPSW switch driver will start
marking packets with offload_fwd_mark flag unless "ale_bypass=0".
All configuration is implemented via switchdev API.

The previous solution of tracking both Ports joined the bridge
(from netdevice_notifier) proved to be not correct as changing CPSW switch
driver mode required cleanup of ALE table and CPSW settings which happens
while second Port is joined bridge and as result configuration loaded
by bridge for the first Port became corrupted.

The introduction of the new CPSW switchdev based driver (cpsw_new.c) is split
on two parts: Part 1 - basic dual-emac driver; Part 2 switchdev support.
Such approach has simplified code development and testing alot. And, I hope,
it will help with better review.

patches #1 - 5: preparation patches which also moves common code to cpsw_priv.c
patches #6 - 9: Introduce TI CPSW switch driver based on switchdev and new
 DT bindings
patch #10: new CPSW switchdev driver documentation
patch #11: adds DT nodes for new CPSW switchdev driver added for DRA7 SoC
patch #12: adds DT nodes for new cpsw switchdev driver for am571x-idk board
patch #13: enables build of TI CPSW driver

Most of the contents of the previous cover-letter have been added in
new driver documentation, so please refer to that for configuration,
testing and future work.

These patches can be found at (branch contains some additional patches required
for testing on top of net-next):
 https://github.com/grygoriyS/linux.git
 branch: lkml-5.4-switch-tbd-v7

changes in v7:
 - patch 2: added check for devm_kmalloc_array() return value
 - patch 6: fixed comments

changes in v6: https://lkml.org/lkml/2019/11/9/108
 - DT bindings converted to json-schema
 - netdev initialization is split on creation and registration.
   The netdevs registration happens now at the end of the pobe.
 - reworked cpsw_set_pauseparam() to use PHYlib APIs.
 - other comments for v5 fixed

v5: https://patchwork.kernel.org/cover/11208785/
 - rebase on top of net-next with XDP series from Ivan Khoronzhuk [3],
   and enable XDP support in the new CPSW switchdev driver
   cpsw driver (tested XDP_DROP only)
 - sync with old cpsw driver
 - implement comments from  Ivan Khoronzhuk and Rob Herring
 - fixed "NETDEV WATCHDOG: .." warning after interface after interface UP/DOWN,
   missed TX wake in cpsw_adjust_link()

v4: https://patchwork.kernel.org/cover/11010523/
 - finished split of common CPSW code
 - added devlink support
 - changed CPSW mode configuration approach: from netdevice_notifier to devlink
   parameter
 - refactor and clean up ALE changes which allows to modify VLANs/MDBs entries
 - added missed support for port QDISC_CBS and QDISC_MQPRIO
 - the CPSW is split on two parts: basic dual_mac driver and switchdev support
 - added missed callback .ndo_get_port_parent_id()
 - reworked ingress frames marking in switch mode (offload_fwd_mark)
 - applied comments from Andrew Lunn

v3: https://lwn.net/Articles/786677/
Changes in v3:
- alot of work done to split properly common code between legacy and switchdev
  CPSW drivers and clean up code
- CPSW switchdev interface updated to the current LKML switchdev interface
- actually new CPSW switchdev based driver introduced
- optimized dual_mac mode in new driver. Main change is that in promiscuous
mode P0_UNI_FLOOD (both ports) is enabled in addition to ALLMULTI (current
port) instead of ALE_BYPASS.  So, port in non promiscuous mode will keep
possibility of mcast and vlan filtering.
- changed bridge join sequnce: now switch mode will be enabled only when
both ports joined the bridge. CPSW will be switched to dual_mac mode if any
port leave bridge. ALE table is completly cleared and then refiled while
switching to switch mode - this simplidies code a lot, but introduces some
limitation to bridge setup sequence:
 ip link add name br0 type bridge
 ip link set dev br0 type bridge ageing_time 1000
 ip link set dev br0 type bridge vlan_filtering 0 <- disable
 echo 0 > /sys/class/net/br0/bridge/default_vlan

 ip link set dev sw0p1 up <- add ports
 ip link set dev sw0p2 up
 ip link set dev sw0p1 master br0
 ip link set dev sw0p2 master br0

 echo 1 > /sys/class/net/br0/bridge/default_vlan <- enable
 ip link set dev br0 type bridge vlan_filtering 1
 bridge vlan add dev br0 vid 1 pvid untagged self
- STP tested with vlan_filtering 1/0. To make STP work I've had to set
  NO_SA_UPDATE for all slave ports (see comment in code). It also required to
  statically register STP mcast address {0x01, 0x80, 0xc2, 0x0, 0x0, 0x0};
- allowed build both TI_CPSW and TI_CPSW_SWITCHDEV drivers
- PTP can be enabled on both ports in dual_mac mode

[1] https://patchwork.ozlabs.org/cover/929367/
[2] https://patches.linaro.org/cover/136709/
[3] https://patchwork.kernel.org/cover/11035813/
====================
Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parents b9242da6 3727d259
Hide whitespace changes
Inline Side-by-side
Showing with 4751 additions and 1340 deletions
Documentation/devicetree/bindings/net/ti,cpsw-switch.yaml 0 → 100644
View file @ 1f12177b
# SPDX-License-Identifier: GPL-2.0
%YAML 1.2
---
$id: http://devicetree.org/schemas/net/ti,cpsw-switch.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: TI SoC Ethernet Switch Controller (CPSW) Device Tree Bindings
maintainers:
- Grygorii Strashko <grygorii.strashko@ti.com>
- Sekhar Nori <nsekhar@ti.com>
description:
The 3-port switch gigabit ethernet subsystem provides ethernet packet
communication and can be configured as an ethernet switch. It provides the
gigabit media independent interface (GMII),reduced gigabit media
independent interface (RGMII), reduced media independent interface (RMII),
the management data input output (MDIO) for physical layer device (PHY)
management.
properties:
compatible:
oneOf:
- const: ti,cpsw-switch
- items:
- const: ti,am335x-cpsw-switch
- const: ti,cpsw-switch
- items:
- const: ti,am4372-cpsw-switch
- const: ti,cpsw-switch
- items:
- const: ti,dra7-cpsw-switch
- const: ti,cpsw-switch
reg:
maxItems: 1
description:
The physical base address and size of full the CPSW module IO range
ranges: true
clocks:
maxItems: 1
description: CPSW functional clock
clock-names:
maxItems: 1
items:
- const: fck
interrupts:
items:
- description: RX_THRESH interrupt
- description: RX interrupt
- description: TX interrupt
- description: MISC interrupt
interrupt-names:
items:
- const: "rx_thresh"
- const: "rx"
- const: "tx"
- const: "misc"
pinctrl-names: true
syscon:
$ref: /schemas/types.yaml#definitions/phandle
description:
Phandle to the system control device node which provides access to
efuse IO range with MAC addresses
ethernet-ports:
type: object
properties:
'#address-cells':
const: 1
'#size-cells':
const: 0
patternProperties:
"^port@[0-9]+$":
type: object
minItems: 1
maxItems: 2
description: CPSW external ports
allOf:
- $ref: ethernet-controller.yaml#
properties:
reg:
maxItems: 1
enum: [1, 2]
description: CPSW port number
phys:
$ref: /schemas/types.yaml#definitions/phandle-array
maxItems: 1
description: phandle on phy-gmii-sel PHY
label:
$ref: /schemas/types.yaml#/definitions/string-array
maxItems: 1
description: label associated with this port
ti,dual-emac-pvid:
$ref: /schemas/types.yaml#/definitions/uint32
maxItems: 1
minimum: 1
maximum: 1024
description:
Specifies default PORT VID to be used to segregate
ports. Default value - CPSW port number.
required:
- reg
- phys
mdio:
type: object
allOf:
- $ref: "ti,davinci-mdio.yaml#"
description:
CPSW MDIO bus.
cpts:
type: object
description:
The Common Platform Time Sync (CPTS) module
properties:
clocks:
maxItems: 1
description: CPTS reference clock
clock-names:
maxItems: 1
items:
- const: cpts
cpts_clock_mult:
$ref: /schemas/types.yaml#/definitions/uint32
description:
Numerator to convert input clock ticks into ns
cpts_clock_shift:
$ref: /schemas/types.yaml#/definitions/uint32
description:
Denominator to convert input clock ticks into ns.
Mult and shift will be calculated basing on CPTS rftclk frequency if
both cpts_clock_shift and cpts_clock_mult properties are not provided.
required:
- clocks
- clock-names
required:
- compatible
- reg
- ranges
- clocks
- clock-names
- interrupts
- interrupt-names
- '#address-cells'
- '#size-cells'
examples:
- |
#include <dt-bindings/interrupt-controller/irq.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/clock/dra7.h>
mac_sw: switch@0 {
compatible = "ti,dra7-cpsw-switch","ti,cpsw-switch";
reg = <0x0 0x4000>;
ranges = <0 0 0x4000>;
clocks = <&gmac_main_clk>;
clock-names = "fck";
#address-cells = <1>;
#size-cells = <1>;
syscon = <&scm_conf>;
inctrl-names = "default", "sleep";
interrupts = <GIC_SPI 334 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 335 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 336 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 337 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "rx_thresh", "rx", "tx", "misc";
ethernet-ports {
#address-cells = <1>;
#size-cells = <0>;
cpsw_port1: port@1 {
reg = <1>;
label = "port1";
mac-address = [ 00 00 00 00 00 00 ];
phys = <&phy_gmii_sel 1>;
phy-handle = <&ethphy0_sw>;
phy-mode = "rgmii";
ti,dual_emac_pvid = <1>;
};
cpsw_port2: port@2 {
reg = <2>;
label = "wan";
mac-address = [ 00 00 00 00 00 00 ];
phys = <&phy_gmii_sel 2>;
phy-handle = <&ethphy1_sw>;
phy-mode = "rgmii";
ti,dual_emac_pvid = <2>;
};
};
davinci_mdio_sw: mdio@1000 {
compatible = "ti,cpsw-mdio","ti,davinci_mdio";
reg = <0x1000 0x100>;
clocks = <&gmac_clkctrl DRA7_GMAC_GMAC_CLKCTRL 0>;
clock-names = "fck";
#address-cells = <1>;
#size-cells = <0>;
bus_freq = <1000000>;
ethphy0_sw: ethernet-phy@0 {
reg = <0>;
};
ethphy1_sw: ethernet-phy@1 {
reg = <1>;
};
};
cpts {
clocks = <&gmac_clkctrl DRA7_GMAC_GMAC_CLKCTRL 25>;
clock-names = "cpts";
};
};
Documentation/networking/device_drivers/ti/cpsw_switchdev.txt 0 → 100644
View file @ 1f12177b
* Texas Instruments CPSW switchdev based ethernet driver 2.0
- Port renaming
On older udev versions renaming of ethX to swXpY will not be automatically
supported
In order to rename via udev:
ip -d link show dev sw0p1 | grep switchid
SUBSYSTEM=="net", ACTION=="add", ATTR{phys_switch_id}==<switchid>, \
ATTR{phys_port_name}!="", NAME="sw0$attr{phys_port_name}"
====================
# Dual mac mode
====================
- The new (cpsw_new.c) driver is operating in dual-emac mode by default, thus
working as 2 individual network interfaces. Main differences from legacy CPSW
driver are:
- optimized promiscuous mode: The P0_UNI_FLOOD (both ports) is enabled in
addition to ALLMULTI (current port) instead of ALE_BYPASS.
So, Ports in promiscuous mode will keep possibility of mcast and vlan filtering,
which is provides significant benefits when ports are joined to the same bridge,
but without enabling "switch" mode, or to different bridges.
- learning disabled on ports as it make not too much sense for
segregated ports - no forwarding in HW.
- enabled basic support for devlink.
devlink dev show
platform/48484000.switch
devlink dev param show
platform/48484000.switch:
name switch_mode type driver-specific
values:
cmode runtime value false
name ale_bypass type driver-specific
values:
cmode runtime value false
Devlink configuration parameters
====================
See Documentation/networking/devlink-params-ti-cpsw-switch.txt
====================
# Bridging in dual mac mode
====================
The dual_mac mode requires two vids to be reserved for internal purposes,
which, by default, equal CPSW Port numbers. As result, bridge has to be
configured in vlan unaware mode or default_pvid has to be adjusted.
ip link add name br0 type bridge
ip link set dev br0 type bridge vlan_filtering 0
echo 0 > /sys/class/net/br0/bridge/default_pvid
ip link set dev sw0p1 master br0
ip link set dev sw0p2 master br0
- or -
ip link add name br0 type bridge
ip link set dev br0 type bridge vlan_filtering 0
echo 100 > /sys/class/net/br0/bridge/default_pvid
ip link set dev br0 type bridge vlan_filtering 1
ip link set dev sw0p1 master br0
ip link set dev sw0p2 master br0
====================
# Enabling "switch"
====================
The Switch mode can be enabled by configuring devlink driver parameter
"switch_mode" to 1/true:
devlink dev param set platform/48484000.switch \
name switch_mode value 1 cmode runtime
This can be done regardless of the state of Port's netdev devices - UP/DOWN, but
Port's netdev devices have to be in UP before joining to the bridge to avoid
overwriting of bridge configuration as CPSW switch driver copletly reloads its
configuration when first Port changes its state to UP.
When the both interfaces joined the bridge - CPSW switch driver will enable
marking packets with offload_fwd_mark flag unless "ale_bypass=0"
All configuration is implemented via switchdev API.
====================
# Bridge setup
====================
devlink dev param set platform/48484000.switch \
name switch_mode value 1 cmode runtime
ip link add name br0 type bridge
ip link set dev br0 type bridge ageing_time 1000
ip link set dev sw0p1 up
ip link set dev sw0p2 up
ip link set dev sw0p1 master br0
ip link set dev sw0p2 master br0
[*] bridge vlan add dev br0 vid 1 pvid untagged self
[*] if vlan_filtering=1. where default_pvid=1
=================
# On/off STP
=================
ip link set dev BRDEV type bridge stp_state 1/0
Note. Steps [*] are mandatory.
====================
# VLAN configuration
====================
bridge vlan add dev br0 vid 1 pvid untagged self <---- add cpu port to VLAN 1
Note. This step is mandatory for bridge/default_pvid.
=================
# Add extra VLANs
=================
1. untagged:
bridge vlan add dev sw0p1 vid 100 pvid untagged master
bridge vlan add dev sw0p2 vid 100 pvid untagged master
bridge vlan add dev br0 vid 100 pvid untagged self <---- Add cpu port to VLAN100
2. tagged:
bridge vlan add dev sw0p1 vid 100 master
bridge vlan add dev sw0p2 vid 100 master
bridge vlan add dev br0 vid 100 pvid tagged self <---- Add cpu port to VLAN100
====
FDBs
====
FDBs are automatically added on the appropriate switch port upon detection
Manually adding FDBs:
bridge fdb add aa:bb:cc:dd:ee:ff dev sw0p1 master vlan 100
bridge fdb add aa:bb:cc:dd:ee:fe dev sw0p2 master <---- Add on all VLANs
====
MDBs
====
MDBs are automatically added on the appropriate switch port upon detection
Manually adding MDBs:
bridge mdb add dev br0 port sw0p1 grp 239.1.1.1 permanent vid 100
bridge mdb add dev br0 port sw0p1 grp 239.1.1.1 permanent <---- Add on all VLANs
==================
Multicast flooding
==================
CPU port mcast_flooding is always on
Turning flooding on/off on swithch ports:
bridge link set dev sw0p1 mcast_flood on/off
==================
Access and Trunk port
==================
bridge vlan add dev sw0p1 vid 100 pvid untagged master
bridge vlan add dev sw0p2 vid 100 master
bridge vlan add dev br0 vid 100 self
ip link add link br0 name br0.100 type vlan id 100
Note. Setting PVID on Bridge device itself working only for
default VLAN (default_pvid).
=====================
NFS
=====================
The only way for NFS to work is by chrooting to a minimal environment when
switch configuration that will affect connectivity is needed.
Assuming you are booting NFS with eth1 interface(the script is hacky and
it's just there to prove NFS is doable).
setup.sh:
#!/bin/sh
mkdir proc
mount -t proc none /proc
ifconfig br0 > /dev/null
if [ $? -ne 0 ]; then
echo "Setting up bridge"
ip link add name br0 type bridge
ip link set dev br0 type bridge ageing_time 1000
ip link set dev br0 type bridge vlan_filtering 1
ip link set eth1 down
ip link set eth1 name sw0p1
ip link set dev sw0p1 up
ip link set dev sw0p2 up
ip link set dev sw0p2 master br0
ip link set dev sw0p1 master br0
bridge vlan add dev br0 vid 1 pvid untagged self
ifconfig sw0p1 0.0.0.0
udhchc -i br0
fi
umount /proc
run_nfs.sh:
#!/bin/sh
mkdir /tmp/root/bin -p
mkdir /tmp/root/lib -p
cp -r /lib/ /tmp/root/
cp -r /bin/ /tmp/root/
cp /sbin/ip /tmp/root/bin
cp /sbin/bridge /tmp/root/bin
cp /sbin/ifconfig /tmp/root/bin
cp /sbin/udhcpc /tmp/root/bin
cp /path/to/setup.sh /tmp/root/bin
chroot /tmp/root/ busybox sh /bin/setup.sh
run ./run_nfs.sh
Documentation/networking/devlink-params-ti-cpsw-switch.txt 0 → 100644
View file @ 1f12177b
ale_bypass [DEVICE, DRIVER-SPECIFIC]
Allows to enable ALE_CONTROL(4).BYPASS mode for debug purposes.
All packets will be sent to the Host port only if enabled.
Type: bool
Configuration mode: runtime
switch_mode [DEVICE, DRIVER-SPECIFIC]
Enable switch mode
Type: bool
Configuration mode: runtime
arch/arm/boot/dts/am571x-idk.dts
View file @ 1f12177b
......@@ -186,3 +186,30 @@ &mmc2 {
pinctrl-1 = <&mmc2_pins_hs>;
pinctrl-2 = <&mmc2_pins_ddr_rev20 &mmc2_iodelay_ddr_conf>;
};
&mac_sw {
pinctrl-names = "default", "sleep";
status = "okay";
};
&cpsw_port1 {
phy-handle = <&ethphy0_sw>;
phy-mode = "rgmii";
ti,dual-emac-pvid = <1>;
};
&cpsw_port2 {
phy-handle = <&ethphy1_sw>;
phy-mode = "rgmii";
ti,dual-emac-pvid = <2>;
};
&davinci_mdio_sw {
ethphy0_sw: ethernet-phy@0 {
reg = <0>;
};
ethphy1_sw: ethernet-phy@1 {
reg = <1>;
};
};
arch/arm/boot/dts/am572x-idk.dts
View file @ 1f12177b
......@@ -27,3 +27,8 @@ &mmc2 {
pinctrl-1 = <&mmc2_pins_hs>;
pinctrl-2 = <&mmc2_pins_ddr_rev20>;
};
&mac {
status = "okay";
dual_emac;
};
arch/arm/boot/dts/am574x-idk.dts
View file @ 1f12177b
......@@ -35,3 +35,8 @@ &mmc2 {
pinctrl-1 = <&mmc2_pins_default>;
pinctrl-2 = <&mmc2_pins_default>;
};
&mac {
status = "okay";
dual_emac;
};
arch/arm/boot/dts/am57xx-idk-common.dtsi
View file @ 1f12177b
......@@ -363,11 +363,6 @@ &rtc {
ext-clk-src;
};
&mac {
status = "okay";
dual_emac;
};
&cpsw_emac0 {
phy-handle = <&ethphy0>;
phy-mode = "rgmii";
......
arch/arm/boot/dts/dra7-l4.dtsi
View file @ 1f12177b
......@@ -3079,6 +3079,58 @@ cpsw_emac1: slave@300 {
phys = <&phy_gmii_sel 2>;
};
};
mac_sw: switch@0 {
compatible = "ti,dra7-cpsw-switch","ti,cpsw-switch";
reg = <0x0 0x4000>;
ranges = <0 0 0x4000>;
clocks = <&gmac_main_clk>;
clock-names = "fck";
#address-cells = <1>;
#size-cells = <1>;
syscon = <&scm_conf>;
status = "disabled";
interrupts = <GIC_SPI 334 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 335 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 336 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 337 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "rx_thresh", "rx", "tx", "misc";
ethernet-ports {
#address-cells = <1>;
#size-cells = <0>;
cpsw_port1: port@1 {
reg = <1>;
label = "port1";
mac-address = [ 00 00 00 00 00 00 ];
phys = <&phy_gmii_sel 1>;
};
cpsw_port2: port@2 {
reg = <2>;
label = "port2";
mac-address = [ 00 00 00 00 00 00 ];
phys = <&phy_gmii_sel 2>;
};
};
davinci_mdio_sw: mdio@1000 {
compatible = "ti,cpsw-mdio","ti,davinci_mdio";
clocks = <&gmac_main_clk>;
clock-names = "fck";
#address-cells = <1>;
#size-cells = <0>;
bus_freq = <1000000>;
reg = <0x1000 0x100>;
};
cpts {
clocks = <&gmac_clkctrl DRA7_GMAC_GMAC_CLKCTRL 25>;
clock-names = "cpts";
};
};
};
};
};
......
arch/arm/configs/omap2plus_defconfig
View file @ 1f12177b
......@@ -554,3 +554,4 @@ CONFIG_DEBUG_INFO_DWARF4=y
CONFIG_MAGIC_SYSRQ=y
CONFIG_SCHEDSTATS=y
# CONFIG_DEBUG_BUGVERBOSE is not set
CONFIG_TI_CPSW_SWITCHDEV=y
drivers/net/ethernet/ti/Kconfig
View file @ 1f12177b
......@@ -59,9 +59,24 @@ config TI_CPSW
To compile this driver as a module, choose M here: the module
will be called cpsw.
config TI_CPSW_SWITCHDEV
tristate "TI CPSW Switch Support with switchdev"
depends on ARCH_DAVINCI || ARCH_OMAP2PLUS || COMPILE_TEST
select NET_SWITCHDEV
select TI_DAVINCI_MDIO
select MFD_SYSCON
select REGMAP
select NET_DEVLINK
imply PHY_TI_GMII_SEL
help
This driver supports TI's CPSW Ethernet Switch.
To compile this driver as a module, choose M here: the module
will be called cpsw_new.
config TI_CPTS
bool "TI Common Platform Time Sync (CPTS) Support"
depends on TI_CPSW || TI_KEYSTONE_NETCP || COMPILE_TEST
depends on TI_CPSW || TI_KEYSTONE_NETCP || TI_CPSW_SWITCHDEV || COMPILE_TEST
depends on COMMON_CLK
depends on POSIX_TIMERS
---help---
......@@ -73,7 +88,7 @@ config TI_CPTS
config TI_CPTS_MOD
tristate
depends on TI_CPTS
default y if TI_CPSW=y || TI_KEYSTONE_NETCP=y
default y if TI_CPSW=y || TI_KEYSTONE_NETCP=y || TI_CPSW_SWITCHDEV=y
select NET_PTP_CLASSIFY
imply PTP_1588_CLOCK
default m
......
drivers/net/ethernet/ti/Makefile
View file @ 1f12177b
......@@ -15,6 +15,8 @@ obj-$(CONFIG_TI_CPSW_PHY_SEL) += cpsw-phy-sel.o
obj-$(CONFIG_TI_CPTS_MOD) += cpts.o
obj-$(CONFIG_TI_CPSW) += ti_cpsw.o
ti_cpsw-y := cpsw.o davinci_cpdma.o cpsw_ale.o cpsw_priv.o cpsw_sl.o cpsw_ethtool.o
obj-$(CONFIG_TI_CPSW_SWITCHDEV) += ti_cpsw_new.o
ti_cpsw_new-y := cpsw_switchdev.o cpsw_new.o davinci_cpdma.o cpsw_ale.o cpsw_sl.o cpsw_priv.o cpsw_ethtool.o
obj-$(CONFIG_TI_KEYSTONE_NETCP) += keystone_netcp.o
keystone_netcp-y := netcp_core.o cpsw_ale.o
......
drivers/net/ethernet/ti/cpsw.c
View file @ 1f12177b
......@@ -34,7 +34,6 @@
#include <net/page_pool.h>
#include <linux/bpf.h>
#include <linux/bpf_trace.h>
#include <linux/filter.h>
#include <linux/pinctrl/consumer.h>
#include <net/pkt_cls.h>
......@@ -64,10 +63,6 @@ static int descs_pool_size = CPSW_CPDMA_DESCS_POOL_SIZE_DEFAULT;
module_param(descs_pool_size, int, 0444);
MODULE_PARM_DESC(descs_pool_size, "Number of CPDMA CPPI descriptors in pool");
/* The buf includes headroom compatible with both skb and xdpf */
#define CPSW_HEADROOM_NA (max(XDP_PACKET_HEADROOM, NET_SKB_PAD) + NET_IP_ALIGN)
#define CPSW_HEADROOM ALIGN(CPSW_HEADROOM_NA, sizeof(long))
#define for_each_slave(priv, func, arg...) \
do { \
struct cpsw_slave *slave; \
......@@ -82,10 +77,16 @@ MODULE_PARM_DESC(descs_pool_size, "Number of CPDMA CPPI descriptors in pool");
(func)(slave++, ##arg); \
} while (0)
#define CPSW_XMETA_OFFSET ALIGN(sizeof(struct xdp_frame), sizeof(long))
static int cpsw_slave_index_priv(struct cpsw_common *cpsw,
struct cpsw_priv *priv)
{
return cpsw->data.dual_emac ? priv->emac_port : cpsw->data.active_slave;
}
#define CPSW_XDP_CONSUMED 1
#define CPSW_XDP_PASS 0
static int cpsw_get_slave_port(u32 slave_num)
{
return slave_num + 1;
}
static int cpsw_ndo_vlan_rx_add_vid(struct net_device *ndev,
__be16 proto, u16 vid);
......@@ -332,218 +333,6 @@ static void cpsw_ndo_set_rx_mode(struct net_device *ndev)
cpsw_del_mc_addr);
}
void cpsw_intr_enable(struct cpsw_common *cpsw)
{
writel_relaxed(0xFF, &cpsw->wr_regs->tx_en);
writel_relaxed(0xFF, &cpsw->wr_regs->rx_en);
cpdma_ctlr_int_ctrl(cpsw->dma, true);
return;
}
void cpsw_intr_disable(struct cpsw_common *cpsw)
{
writel_relaxed(0, &cpsw->wr_regs->tx_en);
writel_relaxed(0, &cpsw->wr_regs->rx_en);
cpdma_ctlr_int_ctrl(cpsw->dma, false);
return;
}
static int cpsw_is_xdpf_handle(void *handle)
{
return (unsigned long)handle & BIT(0);
}
static void *cpsw_xdpf_to_handle(struct xdp_frame *xdpf)
{
return (void *)((unsigned long)xdpf | BIT(0));
}
static struct xdp_frame *cpsw_handle_to_xdpf(void *handle)
{
return (struct xdp_frame *)((unsigned long)handle & ~BIT(0));
}
struct __aligned(sizeof(long)) cpsw_meta_xdp {
struct net_device *ndev;
int ch;
};
void cpsw_tx_handler(void *token, int len, int status)
{
struct cpsw_meta_xdp *xmeta;
struct xdp_frame *xdpf;
struct net_device *ndev;
struct netdev_queue *txq;
struct sk_buff *skb;
int ch;
if (cpsw_is_xdpf_handle(token)) {
xdpf = cpsw_handle_to_xdpf(token);
xmeta = (void *)xdpf + CPSW_XMETA_OFFSET;
ndev = xmeta->ndev;
ch = xmeta->ch;
xdp_return_frame(xdpf);
} else {
skb = token;
ndev = skb->dev;
ch = skb_get_queue_mapping(skb);
cpts_tx_timestamp(ndev_to_cpsw(ndev)->cpts, skb);
dev_kfree_skb_any(skb);
}
/* Check whether the queue is stopped due to stalled tx dma, if the
* queue is stopped then start the queue as we have free desc for tx
*/
txq = netdev_get_tx_queue(ndev, ch);
if (unlikely(netif_tx_queue_stopped(txq)))
netif_tx_wake_queue(txq);
ndev->stats.tx_packets++;
ndev->stats.tx_bytes += len;
}
static void cpsw_rx_vlan_encap(struct sk_buff *skb)
{
struct cpsw_priv *priv = netdev_priv(skb->dev);
struct cpsw_common *cpsw = priv->cpsw;
u32 rx_vlan_encap_hdr = *((u32 *)skb->data);
u16 vtag, vid, prio, pkt_type;
/* Remove VLAN header encapsulation word */
skb_pull(skb, CPSW_RX_VLAN_ENCAP_HDR_SIZE);
pkt_type = (rx_vlan_encap_hdr >>
CPSW_RX_VLAN_ENCAP_HDR_PKT_TYPE_SHIFT) &
CPSW_RX_VLAN_ENCAP_HDR_PKT_TYPE_MSK;
/* Ignore unknown & Priority-tagged packets*/
if (pkt_type == CPSW_RX_VLAN_ENCAP_HDR_PKT_RESERV ||
pkt_type == CPSW_RX_VLAN_ENCAP_HDR_PKT_PRIO_TAG)
return;
vid = (rx_vlan_encap_hdr >>
CPSW_RX_VLAN_ENCAP_HDR_VID_SHIFT) &
VLAN_VID_MASK;
/* Ignore vid 0 and pass packet as is */
if (!vid)
return;
/* Ignore default vlans in dual mac mode */
if (cpsw->data.dual_emac &&
vid == cpsw->slaves[priv->emac_port].port_vlan)
return;
prio = (rx_vlan_encap_hdr >>
CPSW_RX_VLAN_ENCAP_HDR_PRIO_SHIFT) &
CPSW_RX_VLAN_ENCAP_HDR_PRIO_MSK;
vtag = (prio << VLAN_PRIO_SHIFT) | vid;
__vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q), vtag);
/* strip vlan tag for VLAN-tagged packet */
if (pkt_type == CPSW_RX_VLAN_ENCAP_HDR_PKT_VLAN_TAG) {
memmove(skb->data + VLAN_HLEN, skb->data, 2 * ETH_ALEN);
skb_pull(skb, VLAN_HLEN);
}
}
static int cpsw_xdp_tx_frame(struct cpsw_priv *priv, struct xdp_frame *xdpf,
struct page *page)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_meta_xdp *xmeta;
struct cpdma_chan *txch;
dma_addr_t dma;
int ret, port;
xmeta = (void *)xdpf + CPSW_XMETA_OFFSET;
xmeta->ndev = priv->ndev;
xmeta->ch = 0;
txch = cpsw->txv[0].ch;
port = priv->emac_port + cpsw->data.dual_emac;
if (page) {
dma = page_pool_get_dma_addr(page);
dma += xdpf->headroom + sizeof(struct xdp_frame);
ret = cpdma_chan_submit_mapped(txch, cpsw_xdpf_to_handle(xdpf),
dma, xdpf->len, port);
} else {
if (sizeof(*xmeta) > xdpf->headroom) {
xdp_return_frame_rx_napi(xdpf);
return -EINVAL;
}
ret = cpdma_chan_submit(txch, cpsw_xdpf_to_handle(xdpf),
xdpf->data, xdpf->len, port);
}
if (ret) {
priv->ndev->stats.tx_dropped++;
xdp_return_frame_rx_napi(xdpf);
}
return ret;
}
static int cpsw_run_xdp(struct cpsw_priv *priv, int ch, struct xdp_buff *xdp,
struct page *page)
{
struct cpsw_common *cpsw = priv->cpsw;
struct net_device *ndev = priv->ndev;
int ret = CPSW_XDP_CONSUMED;
struct xdp_frame *xdpf;
struct bpf_prog *prog;
u32 act;
rcu_read_lock();
prog = READ_ONCE(priv->xdp_prog);
if (!prog) {
ret = CPSW_XDP_PASS;
goto out;
}
act = bpf_prog_run_xdp(prog, xdp);
switch (act) {
case XDP_PASS:
ret = CPSW_XDP_PASS;
break;
case XDP_TX:
xdpf = convert_to_xdp_frame(xdp);
if (unlikely(!xdpf))
goto drop;
cpsw_xdp_tx_frame(priv, xdpf, page);
break;
case XDP_REDIRECT:
if (xdp_do_redirect(ndev, xdp, prog))
goto drop;
/* Have to flush here, per packet, instead of doing it in bulk
* at the end of the napi handler. The RX devices on this
* particular hardware is sharing a common queue, so the
* incoming device might change per packet.
*/
xdp_do_flush_map();
break;
default:
bpf_warn_invalid_xdp_action(act);
/* fall through */
case XDP_ABORTED:
trace_xdp_exception(ndev, prog, act);
/* fall through -- handle aborts by dropping packet */
case XDP_DROP:
goto drop;
}
out:
rcu_read_unlock();
return ret;
drop:
rcu_read_unlock();
page_pool_recycle_direct(cpsw->page_pool[ch], page);
return ret;
}
static unsigned int cpsw_rxbuf_total_len(unsigned int len)
{
len += CPSW_HEADROOM;
......@@ -552,123 +341,6 @@ static unsigned int cpsw_rxbuf_total_len(unsigned int len)
return SKB_DATA_ALIGN(len);
}
static struct page_pool *cpsw_create_page_pool(struct cpsw_common *cpsw,
int size)
{
struct page_pool_params pp_params;
struct page_pool *pool;
pp_params.order = 0;
pp_params.flags = PP_FLAG_DMA_MAP;
pp_params.pool_size = size;
pp_params.nid = NUMA_NO_NODE;
pp_params.dma_dir = DMA_BIDIRECTIONAL;
pp_params.dev = cpsw->dev;
pool = page_pool_create(&pp_params);
if (IS_ERR(pool))
dev_err(cpsw->dev, "cannot create rx page pool\n");
return pool;
}
static int cpsw_ndev_create_xdp_rxq(struct cpsw_priv *priv, int ch)
{
struct cpsw_common *cpsw = priv->cpsw;
struct xdp_rxq_info *rxq;
struct page_pool *pool;
int ret;
pool = cpsw->page_pool[ch];
rxq = &priv->xdp_rxq[ch];
ret = xdp_rxq_info_reg(rxq, priv->ndev, ch);
if (ret)
return ret;
ret = xdp_rxq_info_reg_mem_model(rxq, MEM_TYPE_PAGE_POOL, pool);
if (ret)
xdp_rxq_info_unreg(rxq);
return ret;
}
static void cpsw_ndev_destroy_xdp_rxq(struct cpsw_priv *priv, int ch)
{
struct xdp_rxq_info *rxq = &priv->xdp_rxq[ch];
if (!xdp_rxq_info_is_reg(rxq))
return;
xdp_rxq_info_unreg(rxq);
}
static int cpsw_create_rx_pool(struct cpsw_common *cpsw, int ch)
{
struct page_pool *pool;
int ret = 0, pool_size;
pool_size = cpdma_chan_get_rx_buf_num(cpsw->rxv[ch].ch);
pool = cpsw_create_page_pool(cpsw, pool_size);
if (IS_ERR(pool))
ret = PTR_ERR(pool);
else
cpsw->page_pool[ch] = pool;
return ret;
}
void cpsw_destroy_xdp_rxqs(struct cpsw_common *cpsw)
{
struct net_device *ndev;
int i, ch;
for (ch = 0; ch < cpsw->rx_ch_num; ch++) {
for (i = 0; i < cpsw->data.slaves; i++) {
ndev = cpsw->slaves[i].ndev;
if (!ndev)
continue;
cpsw_ndev_destroy_xdp_rxq(netdev_priv(ndev), ch);
}
page_pool_destroy(cpsw->page_pool[ch]);
cpsw->page_pool[ch] = NULL;
}
}
int cpsw_create_xdp_rxqs(struct cpsw_common *cpsw)
{
struct net_device *ndev;
int i, ch, ret;
for (ch = 0; ch < cpsw->rx_ch_num; ch++) {
ret = cpsw_create_rx_pool(cpsw, ch);
if (ret)
goto err_cleanup;
/* using same page pool is allowed as no running rx handlers
* simultaneously for both ndevs
*/
for (i = 0; i < cpsw->data.slaves; i++) {
ndev = cpsw->slaves[i].ndev;
if (!ndev)
continue;
ret = cpsw_ndev_create_xdp_rxq(netdev_priv(ndev), ch);
if (ret)
goto err_cleanup;
}
}
return 0;
err_cleanup:
cpsw_destroy_xdp_rxqs(cpsw);
return ret;
}
static void cpsw_rx_handler(void *token, int len, int status)
{
struct page *new_page, *page = token;
......@@ -735,7 +407,8 @@ static void cpsw_rx_handler(void *token, int len, int status)
xdp.data_hard_start = pa;
xdp.rxq = &priv->xdp_rxq[ch];
ret = cpsw_run_xdp(priv, ch, &xdp, page);
port = priv->emac_port + cpsw->data.dual_emac;
ret = cpsw_run_xdp(priv, ch, &xdp, page, port);
if (ret != CPSW_XDP_PASS)
goto requeue;
......@@ -785,274 +458,6 @@ static void cpsw_rx_handler(void *token, int len, int status)
}
}
void cpsw_split_res(struct cpsw_common *cpsw)
{
u32 consumed_rate = 0, bigest_rate = 0;
struct cpsw_vector *txv = cpsw->txv;
int i, ch_weight, rlim_ch_num = 0;
int budget, bigest_rate_ch = 0;
u32 ch_rate, max_rate;
int ch_budget = 0;
for (i = 0; i < cpsw->tx_ch_num; i++) {
ch_rate = cpdma_chan_get_rate(txv[i].ch);
if (!ch_rate)
continue;
rlim_ch_num++;
consumed_rate += ch_rate;
}
if (cpsw->tx_ch_num == rlim_ch_num) {
max_rate = consumed_rate;
} else if (!rlim_ch_num) {
ch_budget = CPSW_POLL_WEIGHT / cpsw->tx_ch_num;
bigest_rate = 0;
max_rate = consumed_rate;
} else {
max_rate = cpsw->speed * 1000;
/* if max_rate is less then expected due to reduced link speed,
* split proportionally according next potential max speed
*/
if (max_rate < consumed_rate)
max_rate *= 10;
if (max_rate < consumed_rate)
max_rate *= 10;
ch_budget = (consumed_rate * CPSW_POLL_WEIGHT) / max_rate;
ch_budget = (CPSW_POLL_WEIGHT - ch_budget) /
(cpsw->tx_ch_num - rlim_ch_num);
bigest_rate = (max_rate - consumed_rate) /
(cpsw->tx_ch_num - rlim_ch_num);
}
/* split tx weight/budget */
budget = CPSW_POLL_WEIGHT;
for (i = 0; i < cpsw->tx_ch_num; i++) {
ch_rate = cpdma_chan_get_rate(txv[i].ch);
if (ch_rate) {
txv[i].budget = (ch_rate * CPSW_POLL_WEIGHT) / max_rate;
if (!txv[i].budget)
txv[i].budget++;
if (ch_rate > bigest_rate) {
bigest_rate_ch = i;
bigest_rate = ch_rate;
}
ch_weight = (ch_rate * 100) / max_rate;
if (!ch_weight)
ch_weight++;
cpdma_chan_set_weight(cpsw->txv[i].ch, ch_weight);
} else {
txv[i].budget = ch_budget;
if (!bigest_rate_ch)
bigest_rate_ch = i;
cpdma_chan_set_weight(cpsw->txv[i].ch, 0);
}
budget -= txv[i].budget;
}
if (budget)
txv[bigest_rate_ch].budget += budget;
/* split rx budget */
budget = CPSW_POLL_WEIGHT;
ch_budget = budget / cpsw->rx_ch_num;
for (i = 0; i < cpsw->rx_ch_num; i++) {
cpsw->rxv[i].budget = ch_budget;
budget -= ch_budget;
}
if (budget)
cpsw->rxv[0].budget += budget;
}
static irqreturn_t cpsw_tx_interrupt(int irq, void *dev_id)
{
struct cpsw_common *cpsw = dev_id;
writel(0, &cpsw->wr_regs->tx_en);
cpdma_ctlr_eoi(cpsw->dma, CPDMA_EOI_TX);
if (cpsw->quirk_irq) {
disable_irq_nosync(cpsw->irqs_table[1]);
cpsw->tx_irq_disabled = true;
}
napi_schedule(&cpsw->napi_tx);
return IRQ_HANDLED;
}
static irqreturn_t cpsw_rx_interrupt(int irq, void *dev_id)
{
struct cpsw_common *cpsw = dev_id;
cpdma_ctlr_eoi(cpsw->dma, CPDMA_EOI_RX);
writel(0, &cpsw->wr_regs->rx_en);
if (cpsw->quirk_irq) {
disable_irq_nosync(cpsw->irqs_table[0]);
cpsw->rx_irq_disabled = true;
}
napi_schedule(&cpsw->napi_rx);
return IRQ_HANDLED;
}
static int cpsw_tx_mq_poll(struct napi_struct *napi_tx, int budget)
{
u32 ch_map;
int num_tx, cur_budget, ch;
struct cpsw_common *cpsw = napi_to_cpsw(napi_tx);
struct cpsw_vector *txv;
/* process every unprocessed channel */
ch_map = cpdma_ctrl_txchs_state(cpsw->dma);
for (ch = 0, num_tx = 0; ch_map & 0xff; ch_map <<= 1, ch++) {
if (!(ch_map & 0x80))
continue;
txv = &cpsw->txv[ch];
if (unlikely(txv->budget > budget - num_tx))
cur_budget = budget - num_tx;
else
cur_budget = txv->budget;
num_tx += cpdma_chan_process(txv->ch, cur_budget);
if (num_tx >= budget)
break;
}
if (num_tx < budget) {
napi_complete(napi_tx);
writel(0xff, &cpsw->wr_regs->tx_en);
}
return num_tx;
}
static int cpsw_tx_poll(struct napi_struct *napi_tx, int budget)
{
struct cpsw_common *cpsw = napi_to_cpsw(napi_tx);
int num_tx;
num_tx = cpdma_chan_process(cpsw->txv[0].ch, budget);
if (num_tx < budget) {
napi_complete(napi_tx);
writel(0xff, &cpsw->wr_regs->tx_en);
if (cpsw->tx_irq_disabled) {
cpsw->tx_irq_disabled = false;
enable_irq(cpsw->irqs_table[1]);
}
}
return num_tx;
}
static int cpsw_rx_mq_poll(struct napi_struct *napi_rx, int budget)
{
u32 ch_map;
int num_rx, cur_budget, ch;
struct cpsw_common *cpsw = napi_to_cpsw(napi_rx);
struct cpsw_vector *rxv;
/* process every unprocessed channel */
ch_map = cpdma_ctrl_rxchs_state(cpsw->dma);
for (ch = 0, num_rx = 0; ch_map; ch_map >>= 1, ch++) {
if (!(ch_map & 0x01))
continue;
rxv = &cpsw->rxv[ch];
if (unlikely(rxv->budget > budget - num_rx))
cur_budget = budget - num_rx;
else
cur_budget = rxv->budget;
num_rx += cpdma_chan_process(rxv->ch, cur_budget);
if (num_rx >= budget)
break;
}
if (num_rx < budget) {
napi_complete_done(napi_rx, num_rx);
writel(0xff, &cpsw->wr_regs->rx_en);
}
return num_rx;
}
static int cpsw_rx_poll(struct napi_struct *napi_rx, int budget)
{
struct cpsw_common *cpsw = napi_to_cpsw(napi_rx);
int num_rx;
num_rx = cpdma_chan_process(cpsw->rxv[0].ch, budget);
if (num_rx < budget) {
napi_complete_done(napi_rx, num_rx);
writel(0xff, &cpsw->wr_regs->rx_en);
if (cpsw->rx_irq_disabled) {
cpsw->rx_irq_disabled = false;
enable_irq(cpsw->irqs_table[0]);
}
}
return num_rx;
}
static inline void soft_reset(const char *module, void __iomem *reg)
{
unsigned long timeout = jiffies + HZ;
writel_relaxed(1, reg);
do {
cpu_relax();
} while ((readl_relaxed(reg) & 1) && time_after(timeout, jiffies));
WARN(readl_relaxed(reg) & 1, "failed to soft-reset %s\n", module);
}
static void cpsw_set_slave_mac(struct cpsw_slave *slave,
struct cpsw_priv *priv)
{
slave_write(slave, mac_hi(priv->mac_addr), SA_HI);
slave_write(slave, mac_lo(priv->mac_addr), SA_LO);
}
static bool cpsw_shp_is_off(struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
u32 shift, mask, val;
val = readl_relaxed(&cpsw->regs->ptype);
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
shift = CPSW_FIFO_SHAPE_EN_SHIFT + 3 * slave->slave_num;
mask = 7 << shift;
val = val & mask;
return !val;
}
static void cpsw_fifo_shp_on(struct cpsw_priv *priv, int fifo, int on)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
u32 shift, mask, val;
val = readl_relaxed(&cpsw->regs->ptype);
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
shift = CPSW_FIFO_SHAPE_EN_SHIFT + 3 * slave->slave_num;
mask = (1 << --fifo) << shift;
val = on ? val | mask : val & ~mask;
writel_relaxed(val, &cpsw->regs->ptype);
}
static void _cpsw_adjust_link(struct cpsw_slave *slave,
struct cpsw_priv *priv, bool *link)
{
......@@ -1118,44 +523,6 @@ static void _cpsw_adjust_link(struct cpsw_slave *slave,
slave->mac_control = mac_control;
}
static int cpsw_get_common_speed(struct cpsw_common *cpsw)
{
int i, speed;
for (i = 0, speed = 0; i < cpsw->data.slaves; i++)
if (cpsw->slaves[i].phy && cpsw->slaves[i].phy->link)
speed += cpsw->slaves[i].phy->speed;
return speed;
}
static int cpsw_need_resplit(struct cpsw_common *cpsw)
{
int i, rlim_ch_num;
int speed, ch_rate;
/* re-split resources only in case speed was changed */
speed = cpsw_get_common_speed(cpsw);
if (speed == cpsw->speed || !speed)
return 0;
cpsw->speed = speed;
for (i = 0, rlim_ch_num = 0; i < cpsw->tx_ch_num; i++) {
ch_rate = cpdma_chan_get_rate(cpsw->txv[i].ch);
if (!ch_rate)
break;
rlim_ch_num++;
}
/* cases not dependent on speed */
if (!rlim_ch_num || rlim_ch_num == cpsw->tx_ch_num)
return 0;
return 1;
}
static void cpsw_adjust_link(struct net_device *ndev)
{
struct cpsw_priv *priv = netdev_priv(ndev);
......@@ -1295,334 +662,74 @@ static inline void cpsw_add_default_vlan(struct cpsw_priv *priv)
int unreg_mcast_mask;
reg = (cpsw->version == CPSW_VERSION_1) ? CPSW1_PORT_VLAN :
CPSW2_PORT_VLAN;
writel(vlan, &cpsw->host_port_regs->port_vlan);
for (i = 0; i < cpsw->data.slaves; i++)
slave_write(cpsw->slaves + i, vlan, reg);
if (priv->ndev->flags & IFF_ALLMULTI)
unreg_mcast_mask = ALE_ALL_PORTS;
else
unreg_mcast_mask = ALE_PORT_1 | ALE_PORT_2;
cpsw_ale_add_vlan(cpsw->ale, vlan, ALE_ALL_PORTS,
ALE_ALL_PORTS, ALE_ALL_PORTS,
unreg_mcast_mask);
}
static void cpsw_init_host_port(struct cpsw_priv *priv)
{
u32 fifo_mode;
u32 control_reg;
struct cpsw_common *cpsw = priv->cpsw;
/* soft reset the controller and initialize ale */
soft_reset("cpsw", &cpsw->regs->soft_reset);
cpsw_ale_start(cpsw->ale);
/* switch to vlan unaware mode */
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM, ALE_VLAN_AWARE,
CPSW_ALE_VLAN_AWARE);
control_reg = readl(&cpsw->regs->control);
control_reg |= CPSW_VLAN_AWARE | CPSW_RX_VLAN_ENCAP;
writel(control_reg, &cpsw->regs->control);
fifo_mode = (cpsw->data.dual_emac) ? CPSW_FIFO_DUAL_MAC_MODE :
CPSW_FIFO_NORMAL_MODE;
writel(fifo_mode, &cpsw->host_port_regs->tx_in_ctl);
/* setup host port priority mapping */
writel_relaxed(CPDMA_TX_PRIORITY_MAP,
&cpsw->host_port_regs->cpdma_tx_pri_map);
writel_relaxed(0, &cpsw->host_port_regs->cpdma_rx_chan_map);
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM,
ALE_PORT_STATE, ALE_PORT_STATE_FORWARD);
if (!cpsw->data.dual_emac) {
cpsw_ale_add_ucast(cpsw->ale, priv->mac_addr, HOST_PORT_NUM,
0, 0);
cpsw_ale_add_mcast(cpsw->ale, priv->ndev->broadcast,
ALE_PORT_HOST, 0, 0, ALE_MCAST_FWD_2);
}
}
int cpsw_fill_rx_channels(struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_meta_xdp *xmeta;
struct page_pool *pool;
struct page *page;
int ch_buf_num;
int ch, i, ret;
dma_addr_t dma;
for (ch = 0; ch < cpsw->rx_ch_num; ch++) {
pool = cpsw->page_pool[ch];
ch_buf_num = cpdma_chan_get_rx_buf_num(cpsw->rxv[ch].ch);
for (i = 0; i < ch_buf_num; i++) {
page = page_pool_dev_alloc_pages(pool);
if (!page) {
cpsw_err(priv, ifup, "allocate rx page err\n");
return -ENOMEM;
}
xmeta = page_address(page) + CPSW_XMETA_OFFSET;
xmeta->ndev = priv->ndev;
xmeta->ch = ch;
dma = page_pool_get_dma_addr(page) + CPSW_HEADROOM;
ret = cpdma_chan_idle_submit_mapped(cpsw->rxv[ch].ch,
page, dma,
cpsw->rx_packet_max,
0);
if (ret < 0) {
cpsw_err(priv, ifup,
"cannot submit page to channel %d rx, error %d\n",
ch, ret);
page_pool_recycle_direct(pool, page);
return ret;
}
}
cpsw_info(priv, ifup, "ch %d rx, submitted %d descriptors\n",
ch, ch_buf_num);
}
return 0;
}
static void cpsw_slave_stop(struct cpsw_slave *slave, struct cpsw_common *cpsw)
{
u32 slave_port;
slave_port = cpsw_get_slave_port(slave->slave_num);
if (!slave->phy)
return;
phy_stop(slave->phy);
phy_disconnect(slave->phy);
slave->phy = NULL;
cpsw_ale_control_set(cpsw->ale, slave_port,
ALE_PORT_STATE, ALE_PORT_STATE_DISABLE);
cpsw_sl_reset(slave->mac_sl, 100);
cpsw_sl_ctl_reset(slave->mac_sl);
}
static int cpsw_tc_to_fifo(int tc, int num_tc)
{
if (tc == num_tc - 1)
return 0;
return CPSW_FIFO_SHAPERS_NUM - tc;
}
static int cpsw_set_fifo_bw(struct cpsw_priv *priv, int fifo, int bw)
{
struct cpsw_common *cpsw = priv->cpsw;
u32 val = 0, send_pct, shift;
struct cpsw_slave *slave;
int pct = 0, i;
if (bw > priv->shp_cfg_speed * 1000)
goto err;
/* shaping has to stay enabled for highest fifos linearly
* and fifo bw no more then interface can allow
*/
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
send_pct = slave_read(slave, SEND_PERCENT);
for (i = CPSW_FIFO_SHAPERS_NUM; i > 0; i--) {
if (!bw) {
if (i >= fifo || !priv->fifo_bw[i])
continue;
dev_warn(priv->dev, "Prev FIFO%d is shaped", i);
continue;
}
if (!priv->fifo_bw[i] && i > fifo) {
dev_err(priv->dev, "Upper FIFO%d is not shaped", i);
return -EINVAL;
}
shift = (i - 1) * 8;
if (i == fifo) {
send_pct &= ~(CPSW_PCT_MASK << shift);
val = DIV_ROUND_UP(bw, priv->shp_cfg_speed * 10);
if (!val)
val = 1;
send_pct |= val << shift;
pct += val;
continue;
}
if (priv->fifo_bw[i])
pct += (send_pct >> shift) & CPSW_PCT_MASK;
}
if (pct >= 100)
goto err;
slave_write(slave, send_pct, SEND_PERCENT);
priv->fifo_bw[fifo] = bw;
dev_warn(priv->dev, "set FIFO%d bw = %d\n", fifo,
DIV_ROUND_CLOSEST(val * priv->shp_cfg_speed, 100));
return 0;
err:
dev_err(priv->dev, "Bandwidth doesn't fit in tc configuration");
return -EINVAL;
}
static int cpsw_set_fifo_rlimit(struct cpsw_priv *priv, int fifo, int bw)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
u32 tx_in_ctl_rg, val;
int ret;
ret = cpsw_set_fifo_bw(priv, fifo, bw);
if (ret)
return ret;
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
tx_in_ctl_rg = cpsw->version == CPSW_VERSION_1 ?
CPSW1_TX_IN_CTL : CPSW2_TX_IN_CTL;
if (!bw)
cpsw_fifo_shp_on(priv, fifo, bw);
val = slave_read(slave, tx_in_ctl_rg);
if (cpsw_shp_is_off(priv)) {
/* disable FIFOs rate limited queues */
val &= ~(0xf << CPSW_FIFO_RATE_EN_SHIFT);
CPSW2_PORT_VLAN;
/* set type of FIFO queues to normal priority mode */
val &= ~(3 << CPSW_FIFO_QUEUE_TYPE_SHIFT);
writel(vlan, &cpsw->host_port_regs->port_vlan);
/* set type of FIFO queues to be rate limited */
if (bw)
val |= 2 << CPSW_FIFO_QUEUE_TYPE_SHIFT;
else
priv->shp_cfg_speed = 0;
}
for (i = 0; i < cpsw->data.slaves; i++)
slave_write(cpsw->slaves + i, vlan, reg);
/* toggle a FIFO rate limited queue */
if (bw)
val |= BIT(fifo + CPSW_FIFO_RATE_EN_SHIFT);
if (priv->ndev->flags & IFF_ALLMULTI)
unreg_mcast_mask = ALE_ALL_PORTS;
else
val &= ~BIT(fifo + CPSW_FIFO_RATE_EN_SHIFT);
slave_write(slave, val, tx_in_ctl_rg);
unreg_mcast_mask = ALE_PORT_1 | ALE_PORT_2;
/* FIFO transmit shape enable */
cpsw_fifo_shp_on(priv, fifo, bw);
return 0;
cpsw_ale_add_vlan(cpsw->ale, vlan, ALE_ALL_PORTS,
ALE_ALL_PORTS, ALE_ALL_PORTS,
unreg_mcast_mask);
}
/* Defaults:
* class A - prio 3
* class B - prio 2
* shaping for class A should be set first
*/
static int cpsw_set_cbs(struct net_device *ndev,
struct tc_cbs_qopt_offload *qopt)
static void cpsw_init_host_port(struct cpsw_priv *priv)
{
struct cpsw_priv *priv = netdev_priv(ndev);
u32 fifo_mode;
u32 control_reg;
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
int prev_speed = 0;
int tc, ret, fifo;
u32 bw = 0;
tc = netdev_txq_to_tc(priv->ndev, qopt->queue);
/* enable channels in backward order, as highest FIFOs must be rate
* limited first and for compliance with CPDMA rate limited channels
* that also used in bacward order. FIFO0 cannot be rate limited.
*/
fifo = cpsw_tc_to_fifo(tc, ndev->num_tc);
if (!fifo) {
dev_err(priv->dev, "Last tc%d can't be rate limited", tc);
return -EINVAL;
}
/* do nothing, it's disabled anyway */
if (!qopt->enable && !priv->fifo_bw[fifo])
return 0;
/* shapers can be set if link speed is known */
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
if (slave->phy && slave->phy->link) {
if (priv->shp_cfg_speed &&
priv->shp_cfg_speed != slave->phy->speed)
prev_speed = priv->shp_cfg_speed;
/* soft reset the controller and initialize ale */
soft_reset("cpsw", &cpsw->regs->soft_reset);
cpsw_ale_start(cpsw->ale);
priv->shp_cfg_speed = slave->phy->speed;
}
/* switch to vlan unaware mode */
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM, ALE_VLAN_AWARE,
CPSW_ALE_VLAN_AWARE);
control_reg = readl(&cpsw->regs->control);
control_reg |= CPSW_VLAN_AWARE | CPSW_RX_VLAN_ENCAP;
writel(control_reg, &cpsw->regs->control);
fifo_mode = (cpsw->data.dual_emac) ? CPSW_FIFO_DUAL_MAC_MODE :
CPSW_FIFO_NORMAL_MODE;
writel(fifo_mode, &cpsw->host_port_regs->tx_in_ctl);
if (!priv->shp_cfg_speed) {
dev_err(priv->dev, "Link speed is not known");
return -1;
}
/* setup host port priority mapping */
writel_relaxed(CPDMA_TX_PRIORITY_MAP,
&cpsw->host_port_regs->cpdma_tx_pri_map);
writel_relaxed(0, &cpsw->host_port_regs->cpdma_rx_chan_map);
ret = pm_runtime_get_sync(cpsw->dev);
if (ret < 0) {
pm_runtime_put_noidle(cpsw->dev);
return ret;
}
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM,
ALE_PORT_STATE, ALE_PORT_STATE_FORWARD);
bw = qopt->enable ? qopt->idleslope : 0;
ret = cpsw_set_fifo_rlimit(priv, fifo, bw);
if (ret) {
priv->shp_cfg_speed = prev_speed;
prev_speed = 0;
if (!cpsw->data.dual_emac) {
cpsw_ale_add_ucast(cpsw->ale, priv->mac_addr, HOST_PORT_NUM,
0, 0);
cpsw_ale_add_mcast(cpsw->ale, priv->ndev->broadcast,
ALE_PORT_HOST, 0, 0, ALE_MCAST_FWD_2);
}
if (bw && prev_speed)
dev_warn(priv->dev,
"Speed was changed, CBS shaper speeds are changed!");
pm_runtime_put_sync(cpsw->dev);
return ret;
}
static void cpsw_cbs_resume(struct cpsw_slave *slave, struct cpsw_priv *priv)
static void cpsw_slave_stop(struct cpsw_slave *slave, struct cpsw_common *cpsw)
{
int fifo, bw;
for (fifo = CPSW_FIFO_SHAPERS_NUM; fifo > 0; fifo--) {
bw = priv->fifo_bw[fifo];
if (!bw)
continue;
cpsw_set_fifo_rlimit(priv, fifo, bw);
}
}
u32 slave_port;
static void cpsw_mqprio_resume(struct cpsw_slave *slave, struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
u32 tx_prio_map = 0;
int i, tc, fifo;
u32 tx_prio_rg;
slave_port = cpsw_get_slave_port(slave->slave_num);
if (!priv->mqprio_hw)
if (!slave->phy)
return;
for (i = 0; i < 8; i++) {
tc = netdev_get_prio_tc_map(priv->ndev, i);
fifo = CPSW_FIFO_SHAPERS_NUM - tc;
tx_prio_map |= fifo << (4 * i);
}
tx_prio_rg = cpsw->version == CPSW_VERSION_1 ?
CPSW1_TX_PRI_MAP : CPSW2_TX_PRI_MAP;
slave_write(slave, tx_prio_map, tx_prio_rg);
phy_stop(slave->phy);
phy_disconnect(slave->phy);
slave->phy = NULL;
cpsw_ale_control_set(cpsw->ale, slave_port,
ALE_PORT_STATE, ALE_PORT_STATE_DISABLE);
cpsw_sl_reset(slave->mac_sl, 100);
cpsw_sl_ctl_reset(slave->mac_sl);
}
static int cpsw_restore_vlans(struct net_device *vdev, int vid, void *arg)
......@@ -1853,207 +960,6 @@ static netdev_tx_t cpsw_ndo_start_xmit(struct sk_buff *skb,
return NETDEV_TX_BUSY;
}
#if IS_ENABLED(CONFIG_TI_CPTS)
static void cpsw_hwtstamp_v1(struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave = &cpsw->slaves[cpsw->data.active_slave];
u32 ts_en, seq_id;
if (!priv->tx_ts_enabled && !priv->rx_ts_enabled) {
slave_write(slave, 0, CPSW1_TS_CTL);
return;
}
seq_id = (30 << CPSW_V1_SEQ_ID_OFS_SHIFT) | ETH_P_1588;
ts_en = EVENT_MSG_BITS << CPSW_V1_MSG_TYPE_OFS;
if (priv->tx_ts_enabled)
ts_en |= CPSW_V1_TS_TX_EN;
if (priv->rx_ts_enabled)
ts_en |= CPSW_V1_TS_RX_EN;
slave_write(slave, ts_en, CPSW1_TS_CTL);
slave_write(slave, seq_id, CPSW1_TS_SEQ_LTYPE);
}
static void cpsw_hwtstamp_v2(struct cpsw_priv *priv)
{
struct cpsw_slave *slave;
struct cpsw_common *cpsw = priv->cpsw;
u32 ctrl, mtype;
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
ctrl = slave_read(slave, CPSW2_CONTROL);
switch (cpsw->version) {
case CPSW_VERSION_2:
ctrl &= ~CTRL_V2_ALL_TS_MASK;
if (priv->tx_ts_enabled)
ctrl |= CTRL_V2_TX_TS_BITS;
if (priv->rx_ts_enabled)
ctrl |= CTRL_V2_RX_TS_BITS;
break;
case CPSW_VERSION_3:
default:
ctrl &= ~CTRL_V3_ALL_TS_MASK;
if (priv->tx_ts_enabled)
ctrl |= CTRL_V3_TX_TS_BITS;
if (priv->rx_ts_enabled)
ctrl |= CTRL_V3_RX_TS_BITS;
break;
}
mtype = (30 << TS_SEQ_ID_OFFSET_SHIFT) | EVENT_MSG_BITS;
slave_write(slave, mtype, CPSW2_TS_SEQ_MTYPE);
slave_write(slave, ctrl, CPSW2_CONTROL);
writel_relaxed(ETH_P_1588, &cpsw->regs->ts_ltype);
writel_relaxed(ETH_P_8021Q, &cpsw->regs->vlan_ltype);
}
static int cpsw_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
{
struct cpsw_priv *priv = netdev_priv(dev);
struct hwtstamp_config cfg;
struct cpsw_common *cpsw = priv->cpsw;
if (cpsw->version != CPSW_VERSION_1 &&
cpsw->version != CPSW_VERSION_2 &&
cpsw->version != CPSW_VERSION_3)
return -EOPNOTSUPP;
if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
return -EFAULT;
/* reserved for future extensions */
if (cfg.flags)
return -EINVAL;
if (cfg.tx_type != HWTSTAMP_TX_OFF && cfg.tx_type != HWTSTAMP_TX_ON)
return -ERANGE;
switch (cfg.rx_filter) {
case HWTSTAMP_FILTER_NONE:
priv->rx_ts_enabled = 0;
break;
case HWTSTAMP_FILTER_ALL:
case HWTSTAMP_FILTER_NTP_ALL:
return -ERANGE;
case HWTSTAMP_FILTER_PTP_V1_L4_EVENT:
case HWTSTAMP_FILTER_PTP_V1_L4_SYNC:
case HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ:
priv->rx_ts_enabled = HWTSTAMP_FILTER_PTP_V1_L4_EVENT;
cfg.rx_filter = HWTSTAMP_FILTER_PTP_V1_L4_EVENT;
break;
case HWTSTAMP_FILTER_PTP_V2_L4_EVENT:
case HWTSTAMP_FILTER_PTP_V2_L4_SYNC:
case HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ:
case HWTSTAMP_FILTER_PTP_V2_L2_EVENT:
case HWTSTAMP_FILTER_PTP_V2_L2_SYNC:
case HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ:
case HWTSTAMP_FILTER_PTP_V2_EVENT:
case HWTSTAMP_FILTER_PTP_V2_SYNC:
case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ:
priv->rx_ts_enabled = HWTSTAMP_FILTER_PTP_V2_EVENT;
cfg.rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT;
break;
default:
return -ERANGE;
}
priv->tx_ts_enabled = cfg.tx_type == HWTSTAMP_TX_ON;
switch (cpsw->version) {
case CPSW_VERSION_1:
cpsw_hwtstamp_v1(priv);
break;
case CPSW_VERSION_2:
case CPSW_VERSION_3:
cpsw_hwtstamp_v2(priv);
break;
default:
WARN_ON(1);
}
return copy_to_user(ifr->ifr_data, &cfg, sizeof(cfg)) ? -EFAULT : 0;
}
static int cpsw_hwtstamp_get(struct net_device *dev, struct ifreq *ifr)
{
struct cpsw_common *cpsw = ndev_to_cpsw(dev);
struct cpsw_priv *priv = netdev_priv(dev);
struct hwtstamp_config cfg;
if (cpsw->version != CPSW_VERSION_1 &&
cpsw->version != CPSW_VERSION_2 &&
cpsw->version != CPSW_VERSION_3)
return -EOPNOTSUPP;
cfg.flags = 0;
cfg.tx_type = priv->tx_ts_enabled ? HWTSTAMP_TX_ON : HWTSTAMP_TX_OFF;
cfg.rx_filter = priv->rx_ts_enabled;
return copy_to_user(ifr->ifr_data, &cfg, sizeof(cfg)) ? -EFAULT : 0;
}
#else
static int cpsw_hwtstamp_get(struct net_device *dev, struct ifreq *ifr)
{
return -EOPNOTSUPP;
}
static int cpsw_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
{
return -EOPNOTSUPP;
}
#endif /*CONFIG_TI_CPTS*/
static int cpsw_ndo_ioctl(struct net_device *dev, struct ifreq *req, int cmd)
{
struct cpsw_priv *priv = netdev_priv(dev);
struct cpsw_common *cpsw = priv->cpsw;
int slave_no = cpsw_slave_index(cpsw, priv);
if (!netif_running(dev))
return -EINVAL;
switch (cmd) {
case SIOCSHWTSTAMP:
return cpsw_hwtstamp_set(dev, req);
case SIOCGHWTSTAMP:
return cpsw_hwtstamp_get(dev, req);
}
if (!cpsw->slaves[slave_no].phy)
return -EOPNOTSUPP;
return phy_mii_ioctl(cpsw->slaves[slave_no].phy, req, cmd);
}
static void cpsw_ndo_tx_timeout(struct net_device *ndev)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
int ch;
cpsw_err(priv, tx_err, "transmit timeout, restarting dma\n");
ndev->stats.tx_errors++;
cpsw_intr_disable(cpsw);
for (ch = 0; ch < cpsw->tx_ch_num; ch++) {
cpdma_chan_stop(cpsw->txv[ch].ch);
cpdma_chan_start(cpsw->txv[ch].ch);
}
cpsw_intr_enable(cpsw);
netif_trans_update(ndev);
netif_tx_wake_all_queues(ndev);
}
static int cpsw_ndo_set_mac_address(struct net_device *ndev, void *p)
{
struct cpsw_priv *priv = netdev_priv(ndev);
......@@ -2215,168 +1121,13 @@ static int cpsw_ndo_vlan_rx_kill_vid(struct net_device *ndev,
return ret;
}
static int cpsw_ndo_set_tx_maxrate(struct net_device *ndev, int queue, u32 rate)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
u32 min_rate;
u32 ch_rate;
int i, ret;
ch_rate = netdev_get_tx_queue(ndev, queue)->tx_maxrate;
if (ch_rate == rate)
return 0;
ch_rate = rate * 1000;
min_rate = cpdma_chan_get_min_rate(cpsw->dma);
if ((ch_rate < min_rate && ch_rate)) {
dev_err(priv->dev, "The channel rate cannot be less than %dMbps",
min_rate);
return -EINVAL;
}
if (rate > cpsw->speed) {
dev_err(priv->dev, "The channel rate cannot be more than 2Gbps");
return -EINVAL;
}
ret = pm_runtime_get_sync(cpsw->dev);
if (ret < 0) {
pm_runtime_put_noidle(cpsw->dev);
return ret;
}
ret = cpdma_chan_set_rate(cpsw->txv[queue].ch, ch_rate);
pm_runtime_put(cpsw->dev);
if (ret)
return ret;
/* update rates for slaves tx queues */
for (i = 0; i < cpsw->data.slaves; i++) {
slave = &cpsw->slaves[i];
if (!slave->ndev)
continue;
netdev_get_tx_queue(slave->ndev, queue)->tx_maxrate = rate;
}
cpsw_split_res(cpsw);
return ret;
}
static int cpsw_set_mqprio(struct net_device *ndev, void *type_data)
{
struct tc_mqprio_qopt_offload *mqprio = type_data;
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
int fifo, num_tc, count, offset;
struct cpsw_slave *slave;
u32 tx_prio_map = 0;
int i, tc, ret;
num_tc = mqprio->qopt.num_tc;
if (num_tc > CPSW_TC_NUM)
return -EINVAL;
if (mqprio->mode != TC_MQPRIO_MODE_DCB)
return -EINVAL;
ret = pm_runtime_get_sync(cpsw->dev);
if (ret < 0) {
pm_runtime_put_noidle(cpsw->dev);
return ret;
}
if (num_tc) {
for (i = 0; i < 8; i++) {
tc = mqprio->qopt.prio_tc_map[i];
fifo = cpsw_tc_to_fifo(tc, num_tc);
tx_prio_map |= fifo << (4 * i);
}
netdev_set_num_tc(ndev, num_tc);
for (i = 0; i < num_tc; i++) {
count = mqprio->qopt.count[i];
offset = mqprio->qopt.offset[i];
netdev_set_tc_queue(ndev, i, count, offset);
}
}
if (!mqprio->qopt.hw) {
/* restore default configuration */
netdev_reset_tc(ndev);
tx_prio_map = TX_PRIORITY_MAPPING;
}
priv->mqprio_hw = mqprio->qopt.hw;
offset = cpsw->version == CPSW_VERSION_1 ?
CPSW1_TX_PRI_MAP : CPSW2_TX_PRI_MAP;
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
slave_write(slave, tx_prio_map, offset);
pm_runtime_put_sync(cpsw->dev);
return 0;
}
static int cpsw_ndo_setup_tc(struct net_device *ndev, enum tc_setup_type type,
void *type_data)
{
switch (type) {
case TC_SETUP_QDISC_CBS:
return cpsw_set_cbs(ndev, type_data);
case TC_SETUP_QDISC_MQPRIO:
return cpsw_set_mqprio(ndev, type_data);
default:
return -EOPNOTSUPP;
}
}
static int cpsw_xdp_prog_setup(struct cpsw_priv *priv, struct netdev_bpf *bpf)
{
struct bpf_prog *prog = bpf->prog;
if (!priv->xdpi.prog && !prog)
return 0;
if (!xdp_attachment_flags_ok(&priv->xdpi, bpf))
return -EBUSY;
WRITE_ONCE(priv->xdp_prog, prog);
xdp_attachment_setup(&priv->xdpi, bpf);
return 0;
}
static int cpsw_ndo_bpf(struct net_device *ndev, struct netdev_bpf *bpf)
{
struct cpsw_priv *priv = netdev_priv(ndev);
switch (bpf->command) {
case XDP_SETUP_PROG:
return cpsw_xdp_prog_setup(priv, bpf);
case XDP_QUERY_PROG:
return xdp_attachment_query(&priv->xdpi, bpf);
default:
return -EINVAL;
}
}
static int cpsw_ndo_xdp_xmit(struct net_device *ndev, int n,
struct xdp_frame **frames, u32 flags)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
struct xdp_frame *xdpf;
int i, drops = 0;
int i, drops = 0, port;
if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK))
return -EINVAL;
......@@ -2389,7 +1140,8 @@ static int cpsw_ndo_xdp_xmit(struct net_device *ndev, int n,
continue;
}
if (cpsw_xdp_tx_frame(priv, xdpf, NULL))
port = priv->emac_port + cpsw->data.dual_emac;
if (cpsw_xdp_tx_frame(priv, xdpf, NULL, port))
drops++;
}
......@@ -2775,6 +1527,8 @@ static int cpsw_probe(struct platform_device *pdev)
return -ENOMEM;
platform_set_drvdata(pdev, cpsw);
cpsw_slave_index = cpsw_slave_index_priv;
cpsw->dev = dev;
mode = devm_gpiod_get_array_optional(dev, "mode", GPIOD_OUT_LOW);
......
drivers/net/ethernet/ti/cpsw_ale.c
View file @ 1f12177b
......@@ -5,6 +5,8 @@
* Copyright (C) 2012 Texas Instruments
*
*/
#include <linux/bitmap.h>
#include <linux/if_vlan.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
......@@ -382,6 +384,7 @@ int cpsw_ale_del_mcast(struct cpsw_ale *ale, const u8 *addr, int port_mask,
int flags, u16 vid)
{
u32 ale_entry[ALE_ENTRY_WORDS] = {0, 0, 0};
int mcast_members;
int idx;
idx = cpsw_ale_match_addr(ale, addr, (flags & ALE_VLAN) ? vid : 0);
......@@ -390,11 +393,15 @@ int cpsw_ale_del_mcast(struct cpsw_ale *ale, const u8 *addr, int port_mask,
cpsw_ale_read(ale, idx, ale_entry);
if (port_mask)
cpsw_ale_set_port_mask(ale_entry, port_mask,
if (port_mask) {
mcast_members = cpsw_ale_get_port_mask(ale_entry,
ale->port_mask_bits);
mcast_members &= ~port_mask;
cpsw_ale_set_port_mask(ale_entry, mcast_members,
ale->port_mask_bits);
else
} else {
cpsw_ale_set_entry_type(ale_entry, ALE_TYPE_FREE);
}
cpsw_ale_write(ale, idx, ale_entry);
return 0;
......@@ -415,7 +422,18 @@ static void cpsw_ale_set_vlan_mcast(struct cpsw_ale *ale, u32 *ale_entry,
writel(unreg_mcast, ale->params.ale_regs + ALE_VLAN_MASK_MUX(idx));
}
int cpsw_ale_add_vlan(struct cpsw_ale *ale, u16 vid, int port, int untag,
static void cpsw_ale_set_vlan_untag(struct cpsw_ale *ale, u32 *ale_entry,
u16 vid, int untag_mask)
{
cpsw_ale_set_vlan_untag_force(ale_entry,
untag_mask, ale->vlan_field_bits);
if (untag_mask & ALE_PORT_HOST)
bitmap_set(ale->p0_untag_vid_mask, vid, 1);
else
bitmap_clear(ale->p0_untag_vid_mask, vid, 1);
}
int cpsw_ale_add_vlan(struct cpsw_ale *ale, u16 vid, int port_mask, int untag,
int reg_mcast, int unreg_mcast)
{
u32 ale_entry[ALE_ENTRY_WORDS] = {0, 0, 0};
......@@ -427,8 +445,8 @@ int cpsw_ale_add_vlan(struct cpsw_ale *ale, u16 vid, int port, int untag,
cpsw_ale_set_entry_type(ale_entry, ALE_TYPE_VLAN);
cpsw_ale_set_vlan_id(ale_entry, vid);
cpsw_ale_set_vlan_untag(ale, ale_entry, vid, untag);
cpsw_ale_set_vlan_untag_force(ale_entry, untag, ale->vlan_field_bits);
if (!ale->params.nu_switch_ale) {
cpsw_ale_set_vlan_reg_mcast(ale_entry, reg_mcast,
ale->vlan_field_bits);
......@@ -437,7 +455,8 @@ int cpsw_ale_add_vlan(struct cpsw_ale *ale, u16 vid, int port, int untag,
} else {
cpsw_ale_set_vlan_mcast(ale, ale_entry, reg_mcast, unreg_mcast);
}
cpsw_ale_set_vlan_member_list(ale_entry, port, ale->vlan_field_bits);
cpsw_ale_set_vlan_member_list(ale_entry, port_mask,
ale->vlan_field_bits);
if (idx < 0)
idx = cpsw_ale_match_free(ale);
......@@ -450,6 +469,41 @@ int cpsw_ale_add_vlan(struct cpsw_ale *ale, u16 vid, int port, int untag,
return 0;
}
static void cpsw_ale_del_vlan_modify(struct cpsw_ale *ale, u32 *ale_entry,
u16 vid, int port_mask)
{
int reg_mcast, unreg_mcast;
int members, untag;
members = cpsw_ale_get_vlan_member_list(ale_entry,
ale->vlan_field_bits);
members &= ~port_mask;
untag = cpsw_ale_get_vlan_untag_force(ale_entry,
ale->vlan_field_bits);
reg_mcast = cpsw_ale_get_vlan_reg_mcast(ale_entry,
ale->vlan_field_bits);
unreg_mcast = cpsw_ale_get_vlan_unreg_mcast(ale_entry,
ale->vlan_field_bits);
untag &= members;
reg_mcast &= members;
unreg_mcast &= members;
cpsw_ale_set_vlan_untag(ale, ale_entry, vid, untag);
if (!ale->params.nu_switch_ale) {
cpsw_ale_set_vlan_reg_mcast(ale_entry, reg_mcast,
ale->vlan_field_bits);
cpsw_ale_set_vlan_unreg_mcast(ale_entry, unreg_mcast,
ale->vlan_field_bits);
} else {
cpsw_ale_set_vlan_mcast(ale, ale_entry, reg_mcast,
unreg_mcast);
}
cpsw_ale_set_vlan_member_list(ale_entry, members,
ale->vlan_field_bits);
}
int cpsw_ale_del_vlan(struct cpsw_ale *ale, u16 vid, int port_mask)
{
u32 ale_entry[ALE_ENTRY_WORDS] = {0, 0, 0};
......@@ -461,16 +515,83 @@ int cpsw_ale_del_vlan(struct cpsw_ale *ale, u16 vid, int port_mask)
cpsw_ale_read(ale, idx, ale_entry);
if (port_mask)
cpsw_ale_set_vlan_member_list(ale_entry, port_mask,
ale->vlan_field_bits);
else
if (port_mask) {
cpsw_ale_del_vlan_modify(ale, ale_entry, vid, port_mask);
} else {
cpsw_ale_set_vlan_untag(ale, ale_entry, vid, 0);
cpsw_ale_set_entry_type(ale_entry, ALE_TYPE_FREE);
}
cpsw_ale_write(ale, idx, ale_entry);
return 0;
}
int cpsw_ale_vlan_add_modify(struct cpsw_ale *ale, u16 vid, int port_mask,
int untag_mask, int reg_mask, int unreg_mask)
{
u32 ale_entry[ALE_ENTRY_WORDS] = {0, 0, 0};
int reg_mcast_members, unreg_mcast_members;
int vlan_members, untag_members;
int idx, ret = 0;
idx = cpsw_ale_match_vlan(ale, vid);
if (idx >= 0)
cpsw_ale_read(ale, idx, ale_entry);
vlan_members = cpsw_ale_get_vlan_member_list(ale_entry,
ale->vlan_field_bits);
reg_mcast_members = cpsw_ale_get_vlan_reg_mcast(ale_entry,
ale->vlan_field_bits);
unreg_mcast_members =
cpsw_ale_get_vlan_unreg_mcast(ale_entry,
ale->vlan_field_bits);
untag_members = cpsw_ale_get_vlan_untag_force(ale_entry,
ale->vlan_field_bits);
vlan_members |= port_mask;
untag_members = (untag_members & ~port_mask) | untag_mask;
reg_mcast_members = (reg_mcast_members & ~port_mask) | reg_mask;
unreg_mcast_members = (unreg_mcast_members & ~port_mask) | unreg_mask;
ret = cpsw_ale_add_vlan(ale, vid, vlan_members, untag_members,
reg_mcast_members, unreg_mcast_members);
if (ret) {
dev_err(ale->params.dev, "Unable to add vlan\n");
return ret;
}
dev_dbg(ale->params.dev, "port mask 0x%x untag 0x%x\n", vlan_members,
untag_mask);
return ret;
}
void cpsw_ale_set_unreg_mcast(struct cpsw_ale *ale, int unreg_mcast_mask,
bool add)
{
u32 ale_entry[ALE_ENTRY_WORDS];
int unreg_members = 0;
int type, idx;
for (idx = 0; idx < ale->params.ale_entries; idx++) {
cpsw_ale_read(ale, idx, ale_entry);
type = cpsw_ale_get_entry_type(ale_entry);
if (type != ALE_TYPE_VLAN)
continue;
unreg_members =
cpsw_ale_get_vlan_unreg_mcast(ale_entry,
ale->vlan_field_bits);
if (add)
unreg_members |= unreg_mcast_mask;
else
unreg_members &= ~unreg_mcast_mask;
cpsw_ale_set_vlan_unreg_mcast(ale_entry, unreg_members,
ale->vlan_field_bits);
cpsw_ale_write(ale, idx, ale_entry);
}
}
void cpsw_ale_set_allmulti(struct cpsw_ale *ale, int allmulti, int port)
{
u32 ale_entry[ALE_ENTRY_WORDS];
......@@ -779,6 +900,7 @@ void cpsw_ale_start(struct cpsw_ale *ale)
void cpsw_ale_stop(struct cpsw_ale *ale)
{
del_timer_sync(&ale->timer);
cpsw_ale_control_set(ale, 0, ALE_CLEAR, 1);
cpsw_ale_control_set(ale, 0, ALE_ENABLE, 0);
}
......@@ -791,6 +913,13 @@ struct cpsw_ale *cpsw_ale_create(struct cpsw_ale_params *params)
if (!ale)
return NULL;
ale->p0_untag_vid_mask =
devm_kmalloc_array(params->dev, BITS_TO_LONGS(VLAN_N_VID),
sizeof(unsigned long),
GFP_KERNEL);
if (!ale->p0_untag_vid_mask)
return ERR_PTR(-ENOMEM);
ale->params = *params;
ale->ageout = ale->params.ale_ageout * HZ;
......@@ -862,6 +991,7 @@ struct cpsw_ale *cpsw_ale_create(struct cpsw_ale_params *params)
ALE_UNKNOWNVLAN_FORCE_UNTAG_EGRESS;
}
cpsw_ale_control_set(ale, 0, ALE_CLEAR, 1);
return ale;
}
......
drivers/net/ethernet/ti/cpsw_ale.h
View file @ 1f12177b
......@@ -35,6 +35,7 @@ struct cpsw_ale {
u32 port_mask_bits;
u32 port_num_bits;
u32 vlan_field_bits;
unsigned long *p0_untag_vid_mask;
};
enum cpsw_ale_control {
......@@ -115,4 +116,14 @@ int cpsw_ale_control_set(struct cpsw_ale *ale, int port,
int control, int value);
void cpsw_ale_dump(struct cpsw_ale *ale, u32 *data);
static inline int cpsw_ale_get_vlan_p0_untag(struct cpsw_ale *ale, u16 vid)
{
return test_bit(vid, ale->p0_untag_vid_mask);
}
int cpsw_ale_vlan_add_modify(struct cpsw_ale *ale, u16 vid, int port_mask,
int untag_mask, int reg_mcast, int unreg_mcast);
void cpsw_ale_set_unreg_mcast(struct cpsw_ale *ale, int unreg_mcast_mask,
bool add);
#endif
drivers/net/ethernet/ti/cpsw_new.c 0 → 100644
View file @ 1f12177b
// SPDX-License-Identifier: GPL-2.0
/*
* Texas Instruments Ethernet Switch Driver
*
* Copyright (C) 2019 Texas Instruments
*/
#include <linux/io.h>
#include <linux/clk.h>
#include <linux/timer.h>
#include <linux/module.h>
#include <linux/irqreturn.h>
#include <linux/interrupt.h>
#include <linux/if_ether.h>
#include <linux/etherdevice.h>
#include <linux/net_tstamp.h>
#include <linux/phy.h>
#include <linux/phy/phy.h>
#include <linux/delay.h>
#include <linux/pm_runtime.h>
#include <linux/gpio/consumer.h>
#include <linux/of.h>
#include <linux/of_mdio.h>
#include <linux/of_net.h>
#include <linux/of_device.h>
#include <linux/if_vlan.h>
#include <linux/kmemleak.h>
#include <linux/sys_soc.h>
#include <net/page_pool.h>
#include <net/pkt_cls.h>
#include <net/devlink.h>
#include "cpsw.h"
#include "cpsw_ale.h"
#include "cpsw_priv.h"
#include "cpsw_sl.h"
#include "cpsw_switchdev.h"
#include "cpts.h"
#include "davinci_cpdma.h"
#include <net/pkt_sched.h>
static int debug_level;
static int ale_ageout = CPSW_ALE_AGEOUT_DEFAULT;
static int rx_packet_max = CPSW_MAX_PACKET_SIZE;
static int descs_pool_size = CPSW_CPDMA_DESCS_POOL_SIZE_DEFAULT;
struct cpsw_devlink {
struct cpsw_common *cpsw;
};
enum cpsw_devlink_param_id {
CPSW_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX,
CPSW_DL_PARAM_SWITCH_MODE,
CPSW_DL_PARAM_ALE_BYPASS,
};
/* struct cpsw_common is not needed, kept here for compatibility
* reasons witrh the old driver
*/
static int cpsw_slave_index_priv(struct cpsw_common *cpsw,
struct cpsw_priv *priv)
{
if (priv->emac_port == HOST_PORT_NUM)
return -1;
return priv->emac_port - 1;
}
static bool cpsw_is_switch_en(struct cpsw_common *cpsw)
{
return !cpsw->data.dual_emac;
}
static void cpsw_set_promiscious(struct net_device *ndev, bool enable)
{
struct cpsw_common *cpsw = ndev_to_cpsw(ndev);
bool enable_uni = false;
int i;
if (cpsw_is_switch_en(cpsw))
return;
/* Enabling promiscuous mode for one interface will be
* common for both the interface as the interface shares
* the same hardware resource.
*/
for (i = 0; i < cpsw->data.slaves; i++)
if (cpsw->slaves[i].ndev &&
(cpsw->slaves[i].ndev->flags & IFF_PROMISC))
enable_uni = true;
if (!enable && enable_uni) {
enable = enable_uni;
dev_dbg(cpsw->dev, "promiscuity not disabled as the other interface is still in promiscuity mode\n");
}
if (enable) {
/* Enable unknown unicast, reg/unreg mcast */
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM,
ALE_P0_UNI_FLOOD, 1);
dev_dbg(cpsw->dev, "promiscuity enabled\n");
} else {
/* Disable unknown unicast */
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM,
ALE_P0_UNI_FLOOD, 0);
dev_dbg(cpsw->dev, "promiscuity disabled\n");
}
}
/**
* cpsw_set_mc - adds multicast entry to the table if it's not added or deletes
* if it's not deleted
* @ndev: device to sync
* @addr: address to be added or deleted
* @vid: vlan id, if vid < 0 set/unset address for real device
* @add: add address if the flag is set or remove otherwise
*/
static int cpsw_set_mc(struct net_device *ndev, const u8 *addr,
int vid, int add)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
int mask, flags, ret, slave_no;
slave_no = cpsw_slave_index(cpsw, priv);
if (vid < 0)
vid = cpsw->slaves[slave_no].port_vlan;
mask = ALE_PORT_HOST;
flags = vid ? ALE_VLAN : 0;
if (add)
ret = cpsw_ale_add_mcast(cpsw->ale, addr, mask, flags, vid, 0);
else
ret = cpsw_ale_del_mcast(cpsw->ale, addr, 0, flags, vid);
return ret;
}
static int cpsw_update_vlan_mc(struct net_device *vdev, int vid, void *ctx)
{
struct addr_sync_ctx *sync_ctx = ctx;
struct netdev_hw_addr *ha;
int found = 0, ret = 0;
if (!vdev || !(vdev->flags & IFF_UP))
return 0;
/* vlan address is relevant if its sync_cnt != 0 */
netdev_for_each_mc_addr(ha, vdev) {
if (ether_addr_equal(ha->addr, sync_ctx->addr)) {
found = ha->sync_cnt;
break;
}
}
if (found)
sync_ctx->consumed++;
if (sync_ctx->flush) {
if (!found)
cpsw_set_mc(sync_ctx->ndev, sync_ctx->addr, vid, 0);
return 0;
}
if (found)
ret = cpsw_set_mc(sync_ctx->ndev, sync_ctx->addr, vid, 1);
return ret;
}
static int cpsw_add_mc_addr(struct net_device *ndev, const u8 *addr, int num)
{
struct addr_sync_ctx sync_ctx;
int ret;
sync_ctx.consumed = 0;
sync_ctx.addr = addr;
sync_ctx.ndev = ndev;
sync_ctx.flush = 0;
ret = vlan_for_each(ndev, cpsw_update_vlan_mc, &sync_ctx);
if (sync_ctx.consumed < num && !ret)
ret = cpsw_set_mc(ndev, addr, -1, 1);
return ret;
}
static int cpsw_del_mc_addr(struct net_device *ndev, const u8 *addr, int num)
{
struct addr_sync_ctx sync_ctx;
sync_ctx.consumed = 0;
sync_ctx.addr = addr;
sync_ctx.ndev = ndev;
sync_ctx.flush = 1;
vlan_for_each(ndev, cpsw_update_vlan_mc, &sync_ctx);
if (sync_ctx.consumed == num)
cpsw_set_mc(ndev, addr, -1, 0);
return 0;
}
static int cpsw_purge_vlan_mc(struct net_device *vdev, int vid, void *ctx)
{
struct addr_sync_ctx *sync_ctx = ctx;
struct netdev_hw_addr *ha;
int found = 0;
if (!vdev || !(vdev->flags & IFF_UP))
return 0;
/* vlan address is relevant if its sync_cnt != 0 */
netdev_for_each_mc_addr(ha, vdev) {
if (ether_addr_equal(ha->addr, sync_ctx->addr)) {
found = ha->sync_cnt;
break;
}
}
if (!found)
return 0;
sync_ctx->consumed++;
cpsw_set_mc(sync_ctx->ndev, sync_ctx->addr, vid, 0);
return 0;
}
static int cpsw_purge_all_mc(struct net_device *ndev, const u8 *addr, int num)
{
struct addr_sync_ctx sync_ctx;
sync_ctx.addr = addr;
sync_ctx.ndev = ndev;
sync_ctx.consumed = 0;
vlan_for_each(ndev, cpsw_purge_vlan_mc, &sync_ctx);
if (sync_ctx.consumed < num)
cpsw_set_mc(ndev, addr, -1, 0);
return 0;
}
static void cpsw_ndo_set_rx_mode(struct net_device *ndev)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
if (ndev->flags & IFF_PROMISC) {
/* Enable promiscuous mode */
cpsw_set_promiscious(ndev, true);
cpsw_ale_set_allmulti(cpsw->ale, IFF_ALLMULTI, priv->emac_port);
return;
}
/* Disable promiscuous mode */
cpsw_set_promiscious(ndev, false);
/* Restore allmulti on vlans if necessary */
cpsw_ale_set_allmulti(cpsw->ale,
ndev->flags & IFF_ALLMULTI, priv->emac_port);
/* add/remove mcast address either for real netdev or for vlan */
__hw_addr_ref_sync_dev(&ndev->mc, ndev, cpsw_add_mc_addr,
cpsw_del_mc_addr);
}
static unsigned int cpsw_rxbuf_total_len(unsigned int len)
{
len += CPSW_HEADROOM;
len += SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
return SKB_DATA_ALIGN(len);
}
static void cpsw_rx_handler(void *token, int len, int status)
{
struct page *new_page, *page = token;
void *pa = page_address(page);
int headroom = CPSW_HEADROOM;
struct cpsw_meta_xdp *xmeta;
struct cpsw_common *cpsw;
struct net_device *ndev;
int port, ch, pkt_size;
struct cpsw_priv *priv;
struct page_pool *pool;
struct sk_buff *skb;
struct xdp_buff xdp;
int ret = 0;
dma_addr_t dma;
xmeta = pa + CPSW_XMETA_OFFSET;
cpsw = ndev_to_cpsw(xmeta->ndev);
ndev = xmeta->ndev;
pkt_size = cpsw->rx_packet_max;
ch = xmeta->ch;
if (status >= 0) {
port = CPDMA_RX_SOURCE_PORT(status);
if (port)
ndev = cpsw->slaves[--port].ndev;
}
priv = netdev_priv(ndev);
pool = cpsw->page_pool[ch];
if (unlikely(status < 0) || unlikely(!netif_running(ndev))) {
/* In dual emac mode check for all interfaces */
if (cpsw->usage_count && status >= 0) {
/* The packet received is for the interface which
* is already down and the other interface is up
* and running, instead of freeing which results
* in reducing of the number of rx descriptor in
* DMA engine, requeue page back to cpdma.
*/
new_page = page;
goto requeue;
}
/* the interface is going down, pages are purged */
page_pool_recycle_direct(pool, page);
return;
}
new_page = page_pool_dev_alloc_pages(pool);
if (unlikely(!new_page)) {
new_page = page;
ndev->stats.rx_dropped++;
goto requeue;
}
if (priv->xdp_prog) {
if (status & CPDMA_RX_VLAN_ENCAP) {
xdp.data = pa + CPSW_HEADROOM +
CPSW_RX_VLAN_ENCAP_HDR_SIZE;
xdp.data_end = xdp.data + len -
CPSW_RX_VLAN_ENCAP_HDR_SIZE;
} else {
xdp.data = pa + CPSW_HEADROOM;
xdp.data_end = xdp.data + len;
}
xdp_set_data_meta_invalid(&xdp);
xdp.data_hard_start = pa;
xdp.rxq = &priv->xdp_rxq[ch];
ret = cpsw_run_xdp(priv, ch, &xdp, page, priv->emac_port);
if (ret != CPSW_XDP_PASS)
goto requeue;
/* XDP prog might have changed packet data and boundaries */
len = xdp.data_end - xdp.data;
headroom = xdp.data - xdp.data_hard_start;
/* XDP prog can modify vlan tag, so can't use encap header */
status &= ~CPDMA_RX_VLAN_ENCAP;
}
/* pass skb to netstack if no XDP prog or returned XDP_PASS */
skb = build_skb(pa, cpsw_rxbuf_total_len(pkt_size));
if (!skb) {
ndev->stats.rx_dropped++;
page_pool_recycle_direct(pool, page);
goto requeue;
}
skb->offload_fwd_mark = priv->offload_fwd_mark;
skb_reserve(skb, headroom);
skb_put(skb, len);
skb->dev = ndev;
if (status & CPDMA_RX_VLAN_ENCAP)
cpsw_rx_vlan_encap(skb);
if (priv->rx_ts_enabled)
cpts_rx_timestamp(cpsw->cpts, skb);
skb->protocol = eth_type_trans(skb, ndev);
/* unmap page as no netstack skb page recycling */
page_pool_release_page(pool, page);
netif_receive_skb(skb);
ndev->stats.rx_bytes += len;
ndev->stats.rx_packets++;
requeue:
xmeta = page_address(new_page) + CPSW_XMETA_OFFSET;
xmeta->ndev = ndev;
xmeta->ch = ch;
dma = page_pool_get_dma_addr(new_page) + CPSW_HEADROOM;
ret = cpdma_chan_submit_mapped(cpsw->rxv[ch].ch, new_page, dma,
pkt_size, 0);
if (ret < 0) {
WARN_ON(ret == -ENOMEM);
page_pool_recycle_direct(pool, new_page);
}
}
static int cpsw_add_vlan_ale_entry(struct cpsw_priv *priv,
unsigned short vid)
{
struct cpsw_common *cpsw = priv->cpsw;
int unreg_mcast_mask = 0;
int mcast_mask;
u32 port_mask;
int ret;
port_mask = (1 << priv->emac_port) | ALE_PORT_HOST;
mcast_mask = ALE_PORT_HOST;
if (priv->ndev->flags & IFF_ALLMULTI)
unreg_mcast_mask = mcast_mask;
ret = cpsw_ale_add_vlan(cpsw->ale, vid, port_mask, 0, port_mask,
unreg_mcast_mask);
if (ret != 0)
return ret;
ret = cpsw_ale_add_ucast(cpsw->ale, priv->mac_addr,
HOST_PORT_NUM, ALE_VLAN, vid);
if (ret != 0)
goto clean_vid;
ret = cpsw_ale_add_mcast(cpsw->ale, priv->ndev->broadcast,
mcast_mask, ALE_VLAN, vid, 0);
if (ret != 0)
goto clean_vlan_ucast;
return 0;
clean_vlan_ucast:
cpsw_ale_del_ucast(cpsw->ale, priv->mac_addr,
HOST_PORT_NUM, ALE_VLAN, vid);
clean_vid:
cpsw_ale_del_vlan(cpsw->ale, vid, 0);
return ret;
}
static int cpsw_ndo_vlan_rx_add_vid(struct net_device *ndev,
__be16 proto, u16 vid)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
int ret, i;
if (cpsw_is_switch_en(cpsw)) {
dev_dbg(cpsw->dev, ".ndo_vlan_rx_add_vid called in switch mode\n");
return 0;
}
if (vid == cpsw->data.default_vlan)
return 0;
ret = pm_runtime_get_sync(cpsw->dev);
if (ret < 0) {
pm_runtime_put_noidle(cpsw->dev);
return ret;
}
/* In dual EMAC, reserved VLAN id should not be used for
* creating VLAN interfaces as this can break the dual
* EMAC port separation
*/
for (i = 0; i < cpsw->data.slaves; i++) {
if (cpsw->slaves[i].ndev &&
vid == cpsw->slaves[i].port_vlan) {
ret = -EINVAL;
goto err;
}
}
dev_dbg(priv->dev, "Adding vlanid %d to vlan filter\n", vid);
ret = cpsw_add_vlan_ale_entry(priv, vid);
err:
pm_runtime_put(cpsw->dev);
return ret;
}
static int cpsw_restore_vlans(struct net_device *vdev, int vid, void *arg)
{
struct cpsw_priv *priv = arg;
if (!vdev || !vid)
return 0;
cpsw_ndo_vlan_rx_add_vid(priv->ndev, 0, vid);
return 0;
}
/* restore resources after port reset */
static void cpsw_restore(struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
/* restore vlan configurations */
vlan_for_each(priv->ndev, cpsw_restore_vlans, priv);
/* restore MQPRIO offload */
cpsw_mqprio_resume(&cpsw->slaves[priv->emac_port - 1], priv);
/* restore CBS offload */
cpsw_cbs_resume(&cpsw->slaves[priv->emac_port - 1], priv);
}
static void cpsw_init_stp_ale_entry(struct cpsw_common *cpsw)
{
char stpa[] = {0x01, 0x80, 0xc2, 0x0, 0x0, 0x0};
cpsw_ale_add_mcast(cpsw->ale, stpa,
ALE_PORT_HOST, ALE_SUPER, 0,
ALE_MCAST_BLOCK_LEARN_FWD);
}
static void cpsw_init_host_port_switch(struct cpsw_common *cpsw)
{
int vlan = cpsw->data.default_vlan;
writel(CPSW_FIFO_NORMAL_MODE, &cpsw->host_port_regs->tx_in_ctl);
writel(vlan, &cpsw->host_port_regs->port_vlan);
cpsw_ale_add_vlan(cpsw->ale, vlan, ALE_ALL_PORTS,
ALE_ALL_PORTS, ALE_ALL_PORTS,
ALE_PORT_1 | ALE_PORT_2);
cpsw_init_stp_ale_entry(cpsw);
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM, ALE_P0_UNI_FLOOD, 1);
dev_dbg(cpsw->dev, "Set P0_UNI_FLOOD\n");
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM, ALE_PORT_NOLEARN, 0);
}
static void cpsw_init_host_port_dual_mac(struct cpsw_common *cpsw)
{
int vlan = cpsw->data.default_vlan;
writel(CPSW_FIFO_DUAL_MAC_MODE, &cpsw->host_port_regs->tx_in_ctl);
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM, ALE_P0_UNI_FLOOD, 0);
dev_dbg(cpsw->dev, "unset P0_UNI_FLOOD\n");
writel(vlan, &cpsw->host_port_regs->port_vlan);
cpsw_ale_add_vlan(cpsw->ale, vlan, ALE_ALL_PORTS, ALE_ALL_PORTS, 0, 0);
/* learning make no sense in dual_mac mode */
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM, ALE_PORT_NOLEARN, 1);
}
static void cpsw_init_host_port(struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
u32 control_reg;
/* soft reset the controller and initialize ale */
soft_reset("cpsw", &cpsw->regs->soft_reset);
cpsw_ale_start(cpsw->ale);
/* switch to vlan unaware mode */
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM, ALE_VLAN_AWARE,
CPSW_ALE_VLAN_AWARE);
control_reg = readl(&cpsw->regs->control);
control_reg |= CPSW_VLAN_AWARE | CPSW_RX_VLAN_ENCAP;
writel(control_reg, &cpsw->regs->control);
/* setup host port priority mapping */
writel_relaxed(CPDMA_TX_PRIORITY_MAP,
&cpsw->host_port_regs->cpdma_tx_pri_map);
writel_relaxed(0, &cpsw->host_port_regs->cpdma_rx_chan_map);
/* disable priority elevation */
writel_relaxed(0, &cpsw->regs->ptype);
/* enable statistics collection only on all ports */
writel_relaxed(0x7, &cpsw->regs->stat_port_en);
/* Enable internal fifo flow control */
writel(0x7, &cpsw->regs->flow_control);
if (cpsw_is_switch_en(cpsw))
cpsw_init_host_port_switch(cpsw);
else
cpsw_init_host_port_dual_mac(cpsw);
cpsw_ale_control_set(cpsw->ale, HOST_PORT_NUM,
ALE_PORT_STATE, ALE_PORT_STATE_FORWARD);
}
static void cpsw_port_add_dual_emac_def_ale_entries(struct cpsw_priv *priv,
struct cpsw_slave *slave)
{
u32 port_mask = 1 << priv->emac_port | ALE_PORT_HOST;
struct cpsw_common *cpsw = priv->cpsw;
u32 reg;
reg = (cpsw->version == CPSW_VERSION_1) ? CPSW1_PORT_VLAN :
CPSW2_PORT_VLAN;
slave_write(slave, slave->port_vlan, reg);
cpsw_ale_add_vlan(cpsw->ale, slave->port_vlan, port_mask,
port_mask, port_mask, 0);
cpsw_ale_add_mcast(cpsw->ale, priv->ndev->broadcast,
ALE_PORT_HOST, ALE_VLAN, slave->port_vlan,
ALE_MCAST_FWD);
cpsw_ale_add_ucast(cpsw->ale, priv->mac_addr,
HOST_PORT_NUM, ALE_VLAN |
ALE_SECURE, slave->port_vlan);
cpsw_ale_control_set(cpsw->ale, priv->emac_port,
ALE_PORT_DROP_UNKNOWN_VLAN, 1);
/* learning make no sense in dual_mac mode */
cpsw_ale_control_set(cpsw->ale, priv->emac_port,
ALE_PORT_NOLEARN, 1);
}
static void cpsw_port_add_switch_def_ale_entries(struct cpsw_priv *priv,
struct cpsw_slave *slave)
{
u32 port_mask = 1 << priv->emac_port | ALE_PORT_HOST;
struct cpsw_common *cpsw = priv->cpsw;
u32 reg;
cpsw_ale_control_set(cpsw->ale, priv->emac_port,
ALE_PORT_DROP_UNKNOWN_VLAN, 0);
cpsw_ale_control_set(cpsw->ale, priv->emac_port,
ALE_PORT_NOLEARN, 0);
/* disabling SA_UPDATE required to make stp work, without this setting
* Host MAC addresses will jump between ports.
* As per TRM MAC address can be defined as unicast supervisory (super)
* by setting both (ALE_BLOCKED | ALE_SECURE) which should prevent
* SA_UPDATE, but HW seems works incorrectly and setting ALE_SECURE
* causes STP packets to be dropped due to ingress filter
* if (source address found) and (secure) and
* (receive port number != port_number))
* then discard the packet
*/
cpsw_ale_control_set(cpsw->ale, priv->emac_port,
ALE_PORT_NO_SA_UPDATE, 1);
cpsw_ale_add_mcast(cpsw->ale, priv->ndev->broadcast,
port_mask, ALE_VLAN, slave->port_vlan,
ALE_MCAST_FWD_2);
cpsw_ale_add_ucast(cpsw->ale, priv->mac_addr,
HOST_PORT_NUM, ALE_VLAN, slave->port_vlan);
reg = (cpsw->version == CPSW_VERSION_1) ? CPSW1_PORT_VLAN :
CPSW2_PORT_VLAN;
slave_write(slave, slave->port_vlan, reg);
}
static void cpsw_adjust_link(struct net_device *ndev)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
struct phy_device *phy;
u32 mac_control = 0;
slave = &cpsw->slaves[priv->emac_port - 1];
phy = slave->phy;
if (!phy)
return;
if (phy->link) {
mac_control = CPSW_SL_CTL_GMII_EN;
if (phy->speed == 1000)
mac_control |= CPSW_SL_CTL_GIG;
if (phy->duplex)
mac_control |= CPSW_SL_CTL_FULLDUPLEX;
/* set speed_in input in case RMII mode is used in 100Mbps */
if (phy->speed == 100)
mac_control |= CPSW_SL_CTL_IFCTL_A;
/* in band mode only works in 10Mbps RGMII mode */
else if ((phy->speed == 10) && phy_interface_is_rgmii(phy))
mac_control |= CPSW_SL_CTL_EXT_EN; /* In Band mode */
if (priv->rx_pause)
mac_control |= CPSW_SL_CTL_RX_FLOW_EN;
if (priv->tx_pause)
mac_control |= CPSW_SL_CTL_TX_FLOW_EN;
if (mac_control != slave->mac_control)
cpsw_sl_ctl_set(slave->mac_sl, mac_control);
/* enable forwarding */
cpsw_ale_control_set(cpsw->ale, priv->emac_port,
ALE_PORT_STATE, ALE_PORT_STATE_FORWARD);
netif_tx_wake_all_queues(ndev);
if (priv->shp_cfg_speed &&
priv->shp_cfg_speed != slave->phy->speed &&
!cpsw_shp_is_off(priv))
dev_warn(priv->dev, "Speed was changed, CBS shaper speeds are changed!");
} else {
netif_tx_stop_all_queues(ndev);
mac_control = 0;
/* disable forwarding */
cpsw_ale_control_set(cpsw->ale, priv->emac_port,
ALE_PORT_STATE, ALE_PORT_STATE_DISABLE);
cpsw_sl_wait_for_idle(slave->mac_sl, 100);
cpsw_sl_ctl_reset(slave->mac_sl);
}
if (mac_control != slave->mac_control)
phy_print_status(phy);
slave->mac_control = mac_control;
if (phy->link && cpsw_need_resplit(cpsw))
cpsw_split_res(cpsw);
}
static void cpsw_slave_open(struct cpsw_slave *slave, struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
struct phy_device *phy;
cpsw_sl_reset(slave->mac_sl, 100);
cpsw_sl_ctl_reset(slave->mac_sl);
/* setup priority mapping */
cpsw_sl_reg_write(slave->mac_sl, CPSW_SL_RX_PRI_MAP,
RX_PRIORITY_MAPPING);
switch (cpsw->version) {
case CPSW_VERSION_1:
slave_write(slave, TX_PRIORITY_MAPPING, CPSW1_TX_PRI_MAP);
/* Increase RX FIFO size to 5 for supporting fullduplex
* flow control mode
*/
slave_write(slave,
(CPSW_MAX_BLKS_TX << CPSW_MAX_BLKS_TX_SHIFT) |
CPSW_MAX_BLKS_RX, CPSW1_MAX_BLKS);
break;
case CPSW_VERSION_2:
case CPSW_VERSION_3:
case CPSW_VERSION_4:
slave_write(slave, TX_PRIORITY_MAPPING, CPSW2_TX_PRI_MAP);
/* Increase RX FIFO size to 5 for supporting fullduplex
* flow control mode
*/
slave_write(slave,
(CPSW_MAX_BLKS_TX << CPSW_MAX_BLKS_TX_SHIFT) |
CPSW_MAX_BLKS_RX, CPSW2_MAX_BLKS);
break;
}
/* setup max packet size, and mac address */
cpsw_sl_reg_write(slave->mac_sl, CPSW_SL_RX_MAXLEN,
cpsw->rx_packet_max);
cpsw_set_slave_mac(slave, priv);
slave->mac_control = 0; /* no link yet */
if (cpsw_is_switch_en(cpsw))
cpsw_port_add_switch_def_ale_entries(priv, slave);
else
cpsw_port_add_dual_emac_def_ale_entries(priv, slave);
if (!slave->data->phy_node)
dev_err(priv->dev, "no phy found on slave %d\n",
slave->slave_num);
phy = of_phy_connect(priv->ndev, slave->data->phy_node,
&cpsw_adjust_link, 0, slave->data->phy_if);
if (!phy) {
dev_err(priv->dev, "phy \"%pOF\" not found on slave %d\n",
slave->data->phy_node,
slave->slave_num);
return;
}
slave->phy = phy;
phy_attached_info(slave->phy);
phy_start(slave->phy);
/* Configure GMII_SEL register */
phy_set_mode_ext(slave->data->ifphy, PHY_MODE_ETHERNET,
slave->data->phy_if);
}
static int cpsw_ndo_stop(struct net_device *ndev)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
cpsw_info(priv, ifdown, "shutting down ndev\n");
slave = &cpsw->slaves[priv->emac_port - 1];
if (slave->phy)
phy_stop(slave->phy);
netif_tx_stop_all_queues(priv->ndev);
if (slave->phy) {
phy_disconnect(slave->phy);
slave->phy = NULL;
}
__hw_addr_ref_unsync_dev(&ndev->mc, ndev, cpsw_purge_all_mc);
if (cpsw->usage_count <= 1) {
napi_disable(&cpsw->napi_rx);
napi_disable(&cpsw->napi_tx);
cpts_unregister(cpsw->cpts);
cpsw_intr_disable(cpsw);
cpdma_ctlr_stop(cpsw->dma);
cpsw_ale_stop(cpsw->ale);
cpsw_destroy_xdp_rxqs(cpsw);
}
if (cpsw_need_resplit(cpsw))
cpsw_split_res(cpsw);
cpsw->usage_count--;
pm_runtime_put_sync(cpsw->dev);
return 0;
}
static int cpsw_ndo_open(struct net_device *ndev)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
int ret;
dev_info(priv->dev, "starting ndev. mode: %s\n",
cpsw_is_switch_en(cpsw) ? "switch" : "dual_mac");
ret = pm_runtime_get_sync(cpsw->dev);
if (ret < 0) {
pm_runtime_put_noidle(cpsw->dev);
return ret;
}
/* Notify the stack of the actual queue counts. */
ret = netif_set_real_num_tx_queues(ndev, cpsw->tx_ch_num);
if (ret) {
dev_err(priv->dev, "cannot set real number of tx queues\n");
goto pm_cleanup;
}
ret = netif_set_real_num_rx_queues(ndev, cpsw->rx_ch_num);
if (ret) {
dev_err(priv->dev, "cannot set real number of rx queues\n");
goto pm_cleanup;
}
/* Initialize host and slave ports */
if (!cpsw->usage_count)
cpsw_init_host_port(priv);
cpsw_slave_open(&cpsw->slaves[priv->emac_port - 1], priv);
/* initialize shared resources for every ndev */
if (!cpsw->usage_count) {
/* create rxqs for both infs in dual mac as they use same pool
* and must be destroyed together when no users.
*/
ret = cpsw_create_xdp_rxqs(cpsw);
if (ret < 0)
goto err_cleanup;
ret = cpsw_fill_rx_channels(priv);
if (ret < 0)
goto err_cleanup;
if (cpts_register(cpsw->cpts))
dev_err(priv->dev, "error registering cpts device\n");
napi_enable(&cpsw->napi_rx);
napi_enable(&cpsw->napi_tx);
if (cpsw->tx_irq_disabled) {
cpsw->tx_irq_disabled = false;
enable_irq(cpsw->irqs_table[1]);
}
if (cpsw->rx_irq_disabled) {
cpsw->rx_irq_disabled = false;
enable_irq(cpsw->irqs_table[0]);
}
}
cpsw_restore(priv);
/* Enable Interrupt pacing if configured */
if (cpsw->coal_intvl != 0) {
struct ethtool_coalesce coal;
coal.rx_coalesce_usecs = cpsw->coal_intvl;
cpsw_set_coalesce(ndev, &coal);
}
cpdma_ctlr_start(cpsw->dma);
cpsw_intr_enable(cpsw);
cpsw->usage_count++;
return 0;
err_cleanup:
cpsw_ndo_stop(ndev);
pm_cleanup:
pm_runtime_put_sync(cpsw->dev);
return ret;
}
static netdev_tx_t cpsw_ndo_start_xmit(struct sk_buff *skb,
struct net_device *ndev)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
struct cpts *cpts = cpsw->cpts;
struct netdev_queue *txq;
struct cpdma_chan *txch;
int ret, q_idx;
if (skb_padto(skb, CPSW_MIN_PACKET_SIZE)) {
cpsw_err(priv, tx_err, "packet pad failed\n");
ndev->stats.tx_dropped++;
return NET_XMIT_DROP;
}
if (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP &&
priv->tx_ts_enabled && cpts_can_timestamp(cpts, skb))
skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
q_idx = skb_get_queue_mapping(skb);
if (q_idx >= cpsw->tx_ch_num)
q_idx = q_idx % cpsw->tx_ch_num;
txch = cpsw->txv[q_idx].ch;
txq = netdev_get_tx_queue(ndev, q_idx);
skb_tx_timestamp(skb);
ret = cpdma_chan_submit(txch, skb, skb->data, skb->len,
priv->emac_port);
if (unlikely(ret != 0)) {
cpsw_err(priv, tx_err, "desc submit failed\n");
goto fail;
}
/* If there is no more tx desc left free then we need to
* tell the kernel to stop sending us tx frames.
*/
if (unlikely(!cpdma_check_free_tx_desc(txch))) {
netif_tx_stop_queue(txq);
/* Barrier, so that stop_queue visible to other cpus */
smp_mb__after_atomic();
if (cpdma_check_free_tx_desc(txch))
netif_tx_wake_queue(txq);
}
return NETDEV_TX_OK;
fail:
ndev->stats.tx_dropped++;
netif_tx_stop_queue(txq);
/* Barrier, so that stop_queue visible to other cpus */
smp_mb__after_atomic();
if (cpdma_check_free_tx_desc(txch))
netif_tx_wake_queue(txq);
return NETDEV_TX_BUSY;
}
static int cpsw_ndo_set_mac_address(struct net_device *ndev, void *p)
{
struct sockaddr *addr = (struct sockaddr *)p;
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
int ret, slave_no;
int flags = 0;
u16 vid = 0;
slave_no = cpsw_slave_index(cpsw, priv);
if (!is_valid_ether_addr(addr->sa_data))
return -EADDRNOTAVAIL;
ret = pm_runtime_get_sync(cpsw->dev);
if (ret < 0) {
pm_runtime_put_noidle(cpsw->dev);
return ret;
}
vid = cpsw->slaves[slave_no].port_vlan;
flags = ALE_VLAN | ALE_SECURE;
cpsw_ale_del_ucast(cpsw->ale, priv->mac_addr, HOST_PORT_NUM,
flags, vid);
cpsw_ale_add_ucast(cpsw->ale, addr->sa_data, HOST_PORT_NUM,
flags, vid);
ether_addr_copy(priv->mac_addr, addr->sa_data);
ether_addr_copy(ndev->dev_addr, priv->mac_addr);
cpsw_set_slave_mac(&cpsw->slaves[slave_no], priv);
pm_runtime_put(cpsw->dev);
return 0;
}
static int cpsw_ndo_vlan_rx_kill_vid(struct net_device *ndev,
__be16 proto, u16 vid)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
int ret;
int i;
if (cpsw_is_switch_en(cpsw)) {
dev_dbg(cpsw->dev, "ndo del vlan is called in switch mode\n");
return 0;
}
if (vid == cpsw->data.default_vlan)
return 0;
ret = pm_runtime_get_sync(cpsw->dev);
if (ret < 0) {
pm_runtime_put_noidle(cpsw->dev);
return ret;
}
for (i = 0; i < cpsw->data.slaves; i++) {
if (cpsw->slaves[i].ndev &&
vid == cpsw->slaves[i].port_vlan)
goto err;
}
dev_dbg(priv->dev, "removing vlanid %d from vlan filter\n", vid);
cpsw_ale_del_vlan(cpsw->ale, vid, 0);
cpsw_ale_del_ucast(cpsw->ale, priv->mac_addr,
HOST_PORT_NUM, ALE_VLAN, vid);
cpsw_ale_del_mcast(cpsw->ale, priv->ndev->broadcast,
0, ALE_VLAN, vid);
cpsw_ale_flush_multicast(cpsw->ale, 0, vid);
err:
pm_runtime_put(cpsw->dev);
return ret;
}
static int cpsw_ndo_get_phys_port_name(struct net_device *ndev, char *name,
size_t len)
{
struct cpsw_priv *priv = netdev_priv(ndev);
int err;
err = snprintf(name, len, "p%d", priv->emac_port);
if (err >= len)
return -EINVAL;
return 0;
}
#ifdef CONFIG_NET_POLL_CONTROLLER
static void cpsw_ndo_poll_controller(struct net_device *ndev)
{
struct cpsw_common *cpsw = ndev_to_cpsw(ndev);
cpsw_intr_disable(cpsw);
cpsw_rx_interrupt(cpsw->irqs_table[0], cpsw);
cpsw_tx_interrupt(cpsw->irqs_table[1], cpsw);
cpsw_intr_enable(cpsw);
}
#endif
static int cpsw_ndo_xdp_xmit(struct net_device *ndev, int n,
struct xdp_frame **frames, u32 flags)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct xdp_frame *xdpf;
int i, drops = 0;
if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK))
return -EINVAL;
for (i = 0; i < n; i++) {
xdpf = frames[i];
if (xdpf->len < CPSW_MIN_PACKET_SIZE) {
xdp_return_frame_rx_napi(xdpf);
drops++;
continue;
}
if (cpsw_xdp_tx_frame(priv, xdpf, NULL, priv->emac_port))
drops++;
}
return n - drops;
}
static int cpsw_get_port_parent_id(struct net_device *ndev,
struct netdev_phys_item_id *ppid)
{
struct cpsw_common *cpsw = ndev_to_cpsw(ndev);
ppid->id_len = sizeof(cpsw->base_mac);
memcpy(&ppid->id, &cpsw->base_mac, ppid->id_len);
return 0;
}
static const struct net_device_ops cpsw_netdev_ops = {
.ndo_open = cpsw_ndo_open,
.ndo_stop = cpsw_ndo_stop,
.ndo_start_xmit = cpsw_ndo_start_xmit,
.ndo_set_mac_address = cpsw_ndo_set_mac_address,
.ndo_do_ioctl = cpsw_ndo_ioctl,
.ndo_validate_addr = eth_validate_addr,
.ndo_tx_timeout = cpsw_ndo_tx_timeout,
.ndo_set_rx_mode = cpsw_ndo_set_rx_mode,
.ndo_set_tx_maxrate = cpsw_ndo_set_tx_maxrate,
#ifdef CONFIG_NET_POLL_CONTROLLER
.ndo_poll_controller = cpsw_ndo_poll_controller,
#endif
.ndo_vlan_rx_add_vid = cpsw_ndo_vlan_rx_add_vid,
.ndo_vlan_rx_kill_vid = cpsw_ndo_vlan_rx_kill_vid,
.ndo_setup_tc = cpsw_ndo_setup_tc,
.ndo_get_phys_port_name = cpsw_ndo_get_phys_port_name,
.ndo_bpf = cpsw_ndo_bpf,
.ndo_xdp_xmit = cpsw_ndo_xdp_xmit,
.ndo_get_port_parent_id = cpsw_get_port_parent_id,
};
static void cpsw_get_drvinfo(struct net_device *ndev,
struct ethtool_drvinfo *info)
{
struct cpsw_common *cpsw = ndev_to_cpsw(ndev);
struct platform_device *pdev;
pdev = to_platform_device(cpsw->dev);
strlcpy(info->driver, "cpsw-switch", sizeof(info->driver));
strlcpy(info->version, "2.0", sizeof(info->version));
strlcpy(info->bus_info, pdev->name, sizeof(info->bus_info));
}
static int cpsw_set_pauseparam(struct net_device *ndev,
struct ethtool_pauseparam *pause)
{
struct cpsw_common *cpsw = ndev_to_cpsw(ndev);
struct cpsw_priv *priv = netdev_priv(ndev);
int slave_no;
slave_no = cpsw_slave_index(cpsw, priv);
if (!cpsw->slaves[slave_no].phy)
return -EINVAL;
if (!phy_validate_pause(cpsw->slaves[slave_no].phy, pause))
return -EINVAL;
priv->rx_pause = pause->rx_pause ? true : false;
priv->tx_pause = pause->tx_pause ? true : false;
phy_set_asym_pause(cpsw->slaves[slave_no].phy,
priv->rx_pause, priv->tx_pause);
return 0;
}
static int cpsw_set_channels(struct net_device *ndev,
struct ethtool_channels *chs)
{
return cpsw_set_channels_common(ndev, chs, cpsw_rx_handler);
}
static const struct ethtool_ops cpsw_ethtool_ops = {
.get_drvinfo = cpsw_get_drvinfo,
.get_msglevel = cpsw_get_msglevel,
.set_msglevel = cpsw_set_msglevel,
.get_link = ethtool_op_get_link,
.get_ts_info = cpsw_get_ts_info,
.get_coalesce = cpsw_get_coalesce,
.set_coalesce = cpsw_set_coalesce,
.get_sset_count = cpsw_get_sset_count,
.get_strings = cpsw_get_strings,
.get_ethtool_stats = cpsw_get_ethtool_stats,
.get_pauseparam = cpsw_get_pauseparam,
.set_pauseparam = cpsw_set_pauseparam,
.get_wol = cpsw_get_wol,
.set_wol = cpsw_set_wol,
.get_regs_len = cpsw_get_regs_len,
.get_regs = cpsw_get_regs,
.begin = cpsw_ethtool_op_begin,
.complete = cpsw_ethtool_op_complete,
.get_channels = cpsw_get_channels,
.set_channels = cpsw_set_channels,
.get_link_ksettings = cpsw_get_link_ksettings,
.set_link_ksettings = cpsw_set_link_ksettings,
.get_eee = cpsw_get_eee,
.set_eee = cpsw_set_eee,
.nway_reset = cpsw_nway_reset,
.get_ringparam = cpsw_get_ringparam,
.set_ringparam = cpsw_set_ringparam,
};
static int cpsw_probe_dt(struct cpsw_common *cpsw)
{
struct device_node *node = cpsw->dev->of_node, *tmp_node, *port_np;
struct cpsw_platform_data *data = &cpsw->data;
struct device *dev = cpsw->dev;
int ret;
u32 prop;
if (!node)
return -EINVAL;
tmp_node = of_get_child_by_name(node, "ethernet-ports");
if (!tmp_node)
return -ENOENT;
data->slaves = of_get_child_count(tmp_node);
if (data->slaves != CPSW_SLAVE_PORTS_NUM) {
of_node_put(tmp_node);
return -ENOENT;
}
data->active_slave = 0;
data->channels = CPSW_MAX_QUEUES;
data->ale_entries = CPSW_ALE_NUM_ENTRIES;
data->dual_emac = 1;
data->bd_ram_size = CPSW_BD_RAM_SIZE;
data->mac_control = 0;
data->slave_data = devm_kcalloc(dev, CPSW_SLAVE_PORTS_NUM,
sizeof(struct cpsw_slave_data),
GFP_KERNEL);
if (!data->slave_data)
return -ENOMEM;
/* Populate all the child nodes here...
*/
ret = devm_of_platform_populate(dev);
/* We do not want to force this, as in some cases may not have child */
if (ret)
dev_warn(dev, "Doesn't have any child node\n");
for_each_child_of_node(tmp_node, port_np) {
struct cpsw_slave_data *slave_data;
const void *mac_addr;
u32 port_id;
ret = of_property_read_u32(port_np, "reg", &port_id);
if (ret < 0) {
dev_err(dev, "%pOF error reading port_id %d\n",
port_np, ret);
goto err_node_put;
}
if (!port_id || port_id > CPSW_SLAVE_PORTS_NUM) {
dev_err(dev, "%pOF has invalid port_id %u\n",
port_np, port_id);
ret = -EINVAL;
goto err_node_put;
}
slave_data = &data->slave_data[port_id - 1];
slave_data->disabled = !of_device_is_available(port_np);
if (slave_data->disabled)
continue;
slave_data->slave_node = port_np;
slave_data->ifphy = devm_of_phy_get(dev, port_np, NULL);
if (IS_ERR(slave_data->ifphy)) {
ret = PTR_ERR(slave_data->ifphy);
dev_err(dev, "%pOF: Error retrieving port phy: %d\n",
port_np, ret);
goto err_node_put;
}
if (of_phy_is_fixed_link(port_np)) {
ret = of_phy_register_fixed_link(port_np);
if (ret) {
if (ret != -EPROBE_DEFER)
dev_err(dev, "%pOF failed to register fixed-link phy: %d\n",
port_np, ret);
goto err_node_put;
}
slave_data->phy_node = of_node_get(port_np);
} else {
slave_data->phy_node =
of_parse_phandle(port_np, "phy-handle", 0);
}
if (!slave_data->phy_node) {
dev_err(dev, "%pOF no phy found\n", port_np);
ret = -ENODEV;
goto err_node_put;
}
ret = of_get_phy_mode(port_np, &slave_data->phy_if);
if (ret) {
dev_err(dev, "%pOF read phy-mode err %d\n",
port_np, ret);
goto err_node_put;
}
mac_addr = of_get_mac_address(port_np);
if (!IS_ERR(mac_addr)) {
ether_addr_copy(slave_data->mac_addr, mac_addr);
} else {
ret = ti_cm_get_macid(dev, port_id - 1,
slave_data->mac_addr);
if (ret)
goto err_node_put;
}
if (of_property_read_u32(port_np, "ti,dual-emac-pvid",
&prop)) {
dev_err(dev, "%pOF Missing dual_emac_res_vlan in DT.\n",
port_np);
slave_data->dual_emac_res_vlan = port_id;
dev_err(dev, "%pOF Using %d as Reserved VLAN\n",
port_np, slave_data->dual_emac_res_vlan);
} else {
slave_data->dual_emac_res_vlan = prop;
}
}
of_node_put(tmp_node);
return 0;
err_node_put:
of_node_put(port_np);
return ret;
}
static void cpsw_remove_dt(struct cpsw_common *cpsw)
{
struct cpsw_platform_data *data = &cpsw->data;
int i = 0;
for (i = 0; i < cpsw->data.slaves; i++) {
struct cpsw_slave_data *slave_data = &data->slave_data[i];
struct device_node *port_np = slave_data->phy_node;
if (port_np) {
if (of_phy_is_fixed_link(port_np))
of_phy_deregister_fixed_link(port_np);
of_node_put(port_np);
}
}
}
static int cpsw_create_ports(struct cpsw_common *cpsw)
{
struct cpsw_platform_data *data = &cpsw->data;
struct net_device *ndev, *napi_ndev = NULL;
struct device *dev = cpsw->dev;
struct cpsw_priv *priv;
int ret = 0, i = 0;
for (i = 0; i < cpsw->data.slaves; i++) {
struct cpsw_slave_data *slave_data = &data->slave_data[i];
if (slave_data->disabled)
continue;
ndev = devm_alloc_etherdev_mqs(dev, sizeof(struct cpsw_priv),
CPSW_MAX_QUEUES,
CPSW_MAX_QUEUES);
if (!ndev) {
dev_err(dev, "error allocating net_device\n");
return -ENOMEM;
}
priv = netdev_priv(ndev);
priv->cpsw = cpsw;
priv->ndev = ndev;
priv->dev = dev;
priv->msg_enable = netif_msg_init(debug_level, CPSW_DEBUG);
priv->emac_port = i + 1;
if (is_valid_ether_addr(slave_data->mac_addr)) {
ether_addr_copy(priv->mac_addr, slave_data->mac_addr);
dev_info(cpsw->dev, "Detected MACID = %pM\n",
priv->mac_addr);
} else {
eth_random_addr(slave_data->mac_addr);
dev_info(cpsw->dev, "Random MACID = %pM\n",
priv->mac_addr);
}
ether_addr_copy(ndev->dev_addr, slave_data->mac_addr);
ether_addr_copy(priv->mac_addr, slave_data->mac_addr);
cpsw->slaves[i].ndev = ndev;
ndev->features |= NETIF_F_HW_VLAN_CTAG_FILTER |
NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_NETNS_LOCAL;
ndev->netdev_ops = &cpsw_netdev_ops;
ndev->ethtool_ops = &cpsw_ethtool_ops;
SET_NETDEV_DEV(ndev, dev);
if (!napi_ndev) {
/* CPSW Host port CPDMA interface is shared between
* ports and there is only one TX and one RX IRQs
* available for all possible TX and RX channels
* accordingly.
*/
netif_napi_add(ndev, &cpsw->napi_rx,
cpsw->quirk_irq ?
cpsw_rx_poll : cpsw_rx_mq_poll,
CPSW_POLL_WEIGHT);
netif_tx_napi_add(ndev, &cpsw->napi_tx,
cpsw->quirk_irq ?
cpsw_tx_poll : cpsw_tx_mq_poll,
CPSW_POLL_WEIGHT);
}
napi_ndev = ndev;
}
return ret;
}
static void cpsw_unregister_ports(struct cpsw_common *cpsw)
{
int i = 0;
for (i = 0; i < cpsw->data.slaves; i++) {
if (!cpsw->slaves[i].ndev)
continue;
unregister_netdev(cpsw->slaves[i].ndev);
}
}
static int cpsw_register_ports(struct cpsw_common *cpsw)
{
int ret = 0, i = 0;
for (i = 0; i < cpsw->data.slaves; i++) {
if (!cpsw->slaves[i].ndev)
continue;
/* register the network device */
ret = register_netdev(cpsw->slaves[i].ndev);
if (ret) {
dev_err(cpsw->dev,
"cpsw: err registering net device%d\n", i);
cpsw->slaves[i].ndev = NULL;
break;
}
}
if (ret)
cpsw_unregister_ports(cpsw);
return ret;
}
bool cpsw_port_dev_check(const struct net_device *ndev)
{
if (ndev->netdev_ops == &cpsw_netdev_ops) {
struct cpsw_common *cpsw = ndev_to_cpsw(ndev);
return !cpsw->data.dual_emac;
}
return false;
}
static void cpsw_port_offload_fwd_mark_update(struct cpsw_common *cpsw)
{
int set_val = 0;
int i;
if (!cpsw->ale_bypass &&
(cpsw->br_members == (ALE_PORT_1 | ALE_PORT_2)))
set_val = 1;
dev_dbg(cpsw->dev, "set offload_fwd_mark %d\n", set_val);
for (i = 0; i < cpsw->data.slaves; i++) {
struct net_device *sl_ndev = cpsw->slaves[i].ndev;
struct cpsw_priv *priv = netdev_priv(sl_ndev);
priv->offload_fwd_mark = set_val;
}
}
static int cpsw_netdevice_port_link(struct net_device *ndev,
struct net_device *br_ndev)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
if (!cpsw->br_members) {
cpsw->hw_bridge_dev = br_ndev;
} else {
/* This is adding the port to a second bridge, this is
* unsupported
*/
if (cpsw->hw_bridge_dev != br_ndev)
return -EOPNOTSUPP;
}
cpsw->br_members |= BIT(priv->emac_port);
cpsw_port_offload_fwd_mark_update(cpsw);
return NOTIFY_DONE;
}
static void cpsw_netdevice_port_unlink(struct net_device *ndev)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
cpsw->br_members &= ~BIT(priv->emac_port);
cpsw_port_offload_fwd_mark_update(cpsw);
if (!cpsw->br_members)
cpsw->hw_bridge_dev = NULL;
}
/* netdev notifier */
static int cpsw_netdevice_event(struct notifier_block *unused,
unsigned long event, void *ptr)
{
struct net_device *ndev = netdev_notifier_info_to_dev(ptr);
struct netdev_notifier_changeupper_info *info;
int ret = NOTIFY_DONE;
if (!cpsw_port_dev_check(ndev))
return NOTIFY_DONE;
switch (event) {
case NETDEV_CHANGEUPPER:
info = ptr;
if (netif_is_bridge_master(info->upper_dev)) {
if (info->linking)
ret = cpsw_netdevice_port_link(ndev,
info->upper_dev);
else
cpsw_netdevice_port_unlink(ndev);
}
break;
default:
return NOTIFY_DONE;
}
return notifier_from_errno(ret);
}
static struct notifier_block cpsw_netdevice_nb __read_mostly = {
.notifier_call = cpsw_netdevice_event,
};
static int cpsw_register_notifiers(struct cpsw_common *cpsw)
{
int ret = 0;
ret = register_netdevice_notifier(&cpsw_netdevice_nb);
if (ret) {
dev_err(cpsw->dev, "can't register netdevice notifier\n");
return ret;
}
ret = cpsw_switchdev_register_notifiers(cpsw);
if (ret)
unregister_netdevice_notifier(&cpsw_netdevice_nb);
return ret;
}
static void cpsw_unregister_notifiers(struct cpsw_common *cpsw)
{
cpsw_switchdev_unregister_notifiers(cpsw);
unregister_netdevice_notifier(&cpsw_netdevice_nb);
}
static const struct devlink_ops cpsw_devlink_ops = {
};
static int cpsw_dl_switch_mode_get(struct devlink *dl, u32 id,
struct devlink_param_gset_ctx *ctx)
{
struct cpsw_devlink *dl_priv = devlink_priv(dl);
struct cpsw_common *cpsw = dl_priv->cpsw;
dev_dbg(cpsw->dev, "%s id:%u\n", __func__, id);
if (id != CPSW_DL_PARAM_SWITCH_MODE)
return -EOPNOTSUPP;
ctx->val.vbool = !cpsw->data.dual_emac;
return 0;
}
static int cpsw_dl_switch_mode_set(struct devlink *dl, u32 id,
struct devlink_param_gset_ctx *ctx)
{
struct cpsw_devlink *dl_priv = devlink_priv(dl);
struct cpsw_common *cpsw = dl_priv->cpsw;
int vlan = cpsw->data.default_vlan;
bool switch_en = ctx->val.vbool;
bool if_running = false;
int i;
dev_dbg(cpsw->dev, "%s id:%u\n", __func__, id);
if (id != CPSW_DL_PARAM_SWITCH_MODE)
return -EOPNOTSUPP;
if (switch_en == !cpsw->data.dual_emac)
return 0;
if (!switch_en && cpsw->br_members) {
dev_err(cpsw->dev, "Remove ports from BR before disabling switch mode\n");
return -EINVAL;
}
rtnl_lock();
for (i = 0; i < cpsw->data.slaves; i++) {
struct cpsw_slave *slave = &cpsw->slaves[i];
struct net_device *sl_ndev = slave->ndev;
if (!sl_ndev || !netif_running(sl_ndev))
continue;
if_running = true;
}
if (!if_running) {
/* all ndevs are down */
cpsw->data.dual_emac = !switch_en;
for (i = 0; i < cpsw->data.slaves; i++) {
struct cpsw_slave *slave = &cpsw->slaves[i];
struct net_device *sl_ndev = slave->ndev;
struct cpsw_priv *priv;
if (!sl_ndev)
continue;
priv = netdev_priv(sl_ndev);
if (switch_en)
vlan = cpsw->data.default_vlan;
else
vlan = slave->data->dual_emac_res_vlan;
slave->port_vlan = vlan;
}
goto exit;
}
if (switch_en) {
dev_info(cpsw->dev, "Enable switch mode\n");
/* enable bypass - no forwarding; all traffic goes to Host */
cpsw_ale_control_set(cpsw->ale, 0, ALE_BYPASS, 1);
/* clean up ALE table */
cpsw_ale_control_set(cpsw->ale, 0, ALE_CLEAR, 1);
cpsw_ale_control_get(cpsw->ale, 0, ALE_AGEOUT);
cpsw_init_host_port_switch(cpsw);
for (i = 0; i < cpsw->data.slaves; i++) {
struct cpsw_slave *slave = &cpsw->slaves[i];
struct net_device *sl_ndev = slave->ndev;
struct cpsw_priv *priv;
if (!sl_ndev)
continue;
priv = netdev_priv(sl_ndev);
slave->port_vlan = vlan;
if (netif_running(sl_ndev))
cpsw_port_add_switch_def_ale_entries(priv,
slave);
}
cpsw_ale_control_set(cpsw->ale, 0, ALE_BYPASS, 0);
cpsw->data.dual_emac = false;
} else {
dev_info(cpsw->dev, "Disable switch mode\n");
/* enable bypass - no forwarding; all traffic goes to Host */
cpsw_ale_control_set(cpsw->ale, 0, ALE_BYPASS, 1);
cpsw_ale_control_set(cpsw->ale, 0, ALE_CLEAR, 1);
cpsw_ale_control_get(cpsw->ale, 0, ALE_AGEOUT);
cpsw_init_host_port_dual_mac(cpsw);
for (i = 0; i < cpsw->data.slaves; i++) {
struct cpsw_slave *slave = &cpsw->slaves[i];
struct net_device *sl_ndev = slave->ndev;
struct cpsw_priv *priv;
if (!sl_ndev)
continue;
priv = netdev_priv(slave->ndev);
slave->port_vlan = slave->data->dual_emac_res_vlan;
cpsw_port_add_dual_emac_def_ale_entries(priv, slave);
}
cpsw_ale_control_set(cpsw->ale, 0, ALE_BYPASS, 0);
cpsw->data.dual_emac = true;
}
exit:
rtnl_unlock();
return 0;
}
static int cpsw_dl_ale_ctrl_get(struct devlink *dl, u32 id,
struct devlink_param_gset_ctx *ctx)
{
struct cpsw_devlink *dl_priv = devlink_priv(dl);
struct cpsw_common *cpsw = dl_priv->cpsw;
dev_dbg(cpsw->dev, "%s id:%u\n", __func__, id);
switch (id) {
case CPSW_DL_PARAM_ALE_BYPASS:
ctx->val.vbool = cpsw_ale_control_get(cpsw->ale, 0, ALE_BYPASS);
break;
default:
return -EOPNOTSUPP;
}
return 0;
}
static int cpsw_dl_ale_ctrl_set(struct devlink *dl, u32 id,
struct devlink_param_gset_ctx *ctx)
{
struct cpsw_devlink *dl_priv = devlink_priv(dl);
struct cpsw_common *cpsw = dl_priv->cpsw;
int ret = -EOPNOTSUPP;
dev_dbg(cpsw->dev, "%s id:%u\n", __func__, id);
switch (id) {
case CPSW_DL_PARAM_ALE_BYPASS:
ret = cpsw_ale_control_set(cpsw->ale, 0, ALE_BYPASS,
ctx->val.vbool);
if (!ret) {
cpsw->ale_bypass = ctx->val.vbool;
cpsw_port_offload_fwd_mark_update(cpsw);
}
break;
default:
return -EOPNOTSUPP;
}
return 0;
}
static const struct devlink_param cpsw_devlink_params[] = {
DEVLINK_PARAM_DRIVER(CPSW_DL_PARAM_SWITCH_MODE,
"switch_mode", DEVLINK_PARAM_TYPE_BOOL,
BIT(DEVLINK_PARAM_CMODE_RUNTIME),
cpsw_dl_switch_mode_get, cpsw_dl_switch_mode_set,
NULL),
DEVLINK_PARAM_DRIVER(CPSW_DL_PARAM_ALE_BYPASS,
"ale_bypass", DEVLINK_PARAM_TYPE_BOOL,
BIT(DEVLINK_PARAM_CMODE_RUNTIME),
cpsw_dl_ale_ctrl_get, cpsw_dl_ale_ctrl_set, NULL),
};
static int cpsw_register_devlink(struct cpsw_common *cpsw)
{
struct device *dev = cpsw->dev;
struct cpsw_devlink *dl_priv;
int ret = 0;
cpsw->devlink = devlink_alloc(&cpsw_devlink_ops, sizeof(*dl_priv));
if (!cpsw->devlink)
return -ENOMEM;
dl_priv = devlink_priv(cpsw->devlink);
dl_priv->cpsw = cpsw;
ret = devlink_register(cpsw->devlink, dev);
if (ret) {
dev_err(dev, "DL reg fail ret:%d\n", ret);
goto dl_free;
}
ret = devlink_params_register(cpsw->devlink, cpsw_devlink_params,
ARRAY_SIZE(cpsw_devlink_params));
if (ret) {
dev_err(dev, "DL params reg fail ret:%d\n", ret);
goto dl_unreg;
}
devlink_params_publish(cpsw->devlink);
return ret;
dl_unreg:
devlink_unregister(cpsw->devlink);
dl_free:
devlink_free(cpsw->devlink);
return ret;
}
static void cpsw_unregister_devlink(struct cpsw_common *cpsw)
{
devlink_params_unpublish(cpsw->devlink);
devlink_params_unregister(cpsw->devlink, cpsw_devlink_params,
ARRAY_SIZE(cpsw_devlink_params));
devlink_unregister(cpsw->devlink);
devlink_free(cpsw->devlink);
}
static const struct of_device_id cpsw_of_mtable[] = {
{ .compatible = "ti,cpsw-switch"},
{ .compatible = "ti,am335x-cpsw-switch"},
{ .compatible = "ti,am4372-cpsw-switch"},
{ .compatible = "ti,dra7-cpsw-switch"},
{ /* sentinel */ },
};
MODULE_DEVICE_TABLE(of, cpsw_of_mtable);
static const struct soc_device_attribute cpsw_soc_devices[] = {
{ .family = "AM33xx", .revision = "ES1.0"},
{ /* sentinel */ }
};
static int cpsw_probe(struct platform_device *pdev)
{
const struct soc_device_attribute *soc;
struct device *dev = &pdev->dev;
struct cpsw_common *cpsw;
struct resource *ss_res;
struct gpio_descs *mode;
void __iomem *ss_regs;
int ret = 0, ch;
struct clk *clk;
int irq;
cpsw = devm_kzalloc(dev, sizeof(struct cpsw_common), GFP_KERNEL);
if (!cpsw)
return -ENOMEM;
cpsw_slave_index = cpsw_slave_index_priv;
cpsw->dev = dev;
cpsw->slaves = devm_kcalloc(dev,
CPSW_SLAVE_PORTS_NUM,
sizeof(struct cpsw_slave),
GFP_KERNEL);
if (!cpsw->slaves)
return -ENOMEM;
mode = devm_gpiod_get_array_optional(dev, "mode", GPIOD_OUT_LOW);
if (IS_ERR(mode)) {
ret = PTR_ERR(mode);
dev_err(dev, "gpio request failed, ret %d\n", ret);
return ret;
}
clk = devm_clk_get(dev, "fck");
if (IS_ERR(clk)) {
ret = PTR_ERR(clk);
dev_err(dev, "fck is not found %d\n", ret);
return ret;
}
cpsw->bus_freq_mhz = clk_get_rate(clk) / 1000000;
ss_res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
ss_regs = devm_ioremap_resource(dev, ss_res);
if (IS_ERR(ss_regs)) {
ret = PTR_ERR(ss_regs);
return ret;
}
cpsw->regs = ss_regs;
irq = platform_get_irq_byname(pdev, "rx");
if (irq < 0)
return irq;
cpsw->irqs_table[0] = irq;
irq = platform_get_irq_byname(pdev, "tx");
if (irq < 0)
return irq;
cpsw->irqs_table[1] = irq;
platform_set_drvdata(pdev, cpsw);
/* This may be required here for child devices. */
pm_runtime_enable(dev);
/* Need to enable clocks with runtime PM api to access module
* registers
*/
ret = pm_runtime_get_sync(dev);
if (ret < 0) {
pm_runtime_put_noidle(dev);
pm_runtime_disable(dev);
return ret;
}
ret = cpsw_probe_dt(cpsw);
if (ret)
goto clean_dt_ret;
soc = soc_device_match(cpsw_soc_devices);
if (soc)
cpsw->quirk_irq = 1;
cpsw->rx_packet_max = rx_packet_max;
cpsw->descs_pool_size = descs_pool_size;
eth_random_addr(cpsw->base_mac);
ret = cpsw_init_common(cpsw, ss_regs, ale_ageout,
(u32 __force)ss_res->start + CPSW2_BD_OFFSET,
descs_pool_size);
if (ret)
goto clean_dt_ret;
cpsw->wr_regs = cpsw->version == CPSW_VERSION_1 ?
ss_regs + CPSW1_WR_OFFSET :
ss_regs + CPSW2_WR_OFFSET;
ch = cpsw->quirk_irq ? 0 : 7;
cpsw->txv[0].ch = cpdma_chan_create(cpsw->dma, ch, cpsw_tx_handler, 0);
if (IS_ERR(cpsw->txv[0].ch)) {
dev_err(dev, "error initializing tx dma channel\n");
ret = PTR_ERR(cpsw->txv[0].ch);
goto clean_cpts;
}
cpsw->rxv[0].ch = cpdma_chan_create(cpsw->dma, 0, cpsw_rx_handler, 1);
if (IS_ERR(cpsw->rxv[0].ch)) {
dev_err(dev, "error initializing rx dma channel\n");
ret = PTR_ERR(cpsw->rxv[0].ch);
goto clean_cpts;
}
cpsw_split_res(cpsw);
/* setup netdevs */
ret = cpsw_create_ports(cpsw);
if (ret)
goto clean_unregister_netdev;
/* Grab RX and TX IRQs. Note that we also have RX_THRESHOLD and
* MISC IRQs which are always kept disabled with this driver so
* we will not request them.
*
* If anyone wants to implement support for those, make sure to
* first request and append them to irqs_table array.
*/
ret = devm_request_irq(dev, cpsw->irqs_table[0], cpsw_rx_interrupt,
0, dev_name(dev), cpsw);
if (ret < 0) {
dev_err(dev, "error attaching irq (%d)\n", ret);
goto clean_unregister_netdev;
}
ret = devm_request_irq(dev, cpsw->irqs_table[1], cpsw_tx_interrupt,
0, dev_name(dev), cpsw);
if (ret < 0) {
dev_err(dev, "error attaching irq (%d)\n", ret);
goto clean_unregister_netdev;
}
ret = cpsw_register_notifiers(cpsw);
if (ret)
goto clean_unregister_netdev;
ret = cpsw_register_devlink(cpsw);
if (ret)
goto clean_unregister_notifiers;
ret = cpsw_register_ports(cpsw);
if (ret)
goto clean_unregister_notifiers;
dev_notice(dev, "initialized (regs %pa, pool size %d) hw_ver:%08X %d.%d (%d)\n",
&ss_res->start, descs_pool_size,
cpsw->version, CPSW_MAJOR_VERSION(cpsw->version),
CPSW_MINOR_VERSION(cpsw->version),
CPSW_RTL_VERSION(cpsw->version));
pm_runtime_put(dev);
return 0;
clean_unregister_notifiers:
cpsw_unregister_notifiers(cpsw);
clean_unregister_netdev:
cpsw_unregister_ports(cpsw);
clean_cpts:
cpts_release(cpsw->cpts);
cpdma_ctlr_destroy(cpsw->dma);
clean_dt_ret:
cpsw_remove_dt(cpsw);
pm_runtime_put_sync(dev);
pm_runtime_disable(dev);
return ret;
}
static int cpsw_remove(struct platform_device *pdev)
{
struct cpsw_common *cpsw = platform_get_drvdata(pdev);
int ret;
ret = pm_runtime_get_sync(&pdev->dev);
if (ret < 0) {
pm_runtime_put_noidle(&pdev->dev);
return ret;
}
cpsw_unregister_notifiers(cpsw);
cpsw_unregister_devlink(cpsw);
cpsw_unregister_ports(cpsw);
cpts_release(cpsw->cpts);
cpdma_ctlr_destroy(cpsw->dma);
cpsw_remove_dt(cpsw);
pm_runtime_put_sync(&pdev->dev);
pm_runtime_disable(&pdev->dev);
return 0;
}
static struct platform_driver cpsw_driver = {
.driver = {
.name = "cpsw-switch",
.of_match_table = cpsw_of_mtable,
},
.probe = cpsw_probe,
.remove = cpsw_remove,
};
module_platform_driver(cpsw_driver);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("TI CPSW switchdev Ethernet driver");
drivers/net/ethernet/ti/cpsw_priv.c
View file @ 1f12177b
......@@ -5,20 +5,415 @@
* Copyright (C) 2019 Texas Instruments
*/
#include <linux/bpf.h>
#include <linux/bpf_trace.h>
#include <linux/if_ether.h>
#include <linux/if_vlan.h>
#include <linux/kmemleak.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/net_tstamp.h>
#include <linux/of.h>
#include <linux/phy.h>
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
#include <linux/skbuff.h>
#include <net/page_pool.h>
#include <net/pkt_cls.h>
#include "cpsw.h"
#include "cpts.h"
#include "cpsw_ale.h"
#include "cpsw_priv.h"
#include "cpsw_sl.h"
#include "davinci_cpdma.h"
int (*cpsw_slave_index)(struct cpsw_common *cpsw, struct cpsw_priv *priv);
void cpsw_intr_enable(struct cpsw_common *cpsw)
{
writel_relaxed(0xFF, &cpsw->wr_regs->tx_en);
writel_relaxed(0xFF, &cpsw->wr_regs->rx_en);
cpdma_ctlr_int_ctrl(cpsw->dma, true);
}
void cpsw_intr_disable(struct cpsw_common *cpsw)
{
writel_relaxed(0, &cpsw->wr_regs->tx_en);
writel_relaxed(0, &cpsw->wr_regs->rx_en);
cpdma_ctlr_int_ctrl(cpsw->dma, false);
}
void cpsw_tx_handler(void *token, int len, int status)
{
struct cpsw_meta_xdp *xmeta;
struct xdp_frame *xdpf;
struct net_device *ndev;
struct netdev_queue *txq;
struct sk_buff *skb;
int ch;
if (cpsw_is_xdpf_handle(token)) {
xdpf = cpsw_handle_to_xdpf(token);
xmeta = (void *)xdpf + CPSW_XMETA_OFFSET;
ndev = xmeta->ndev;
ch = xmeta->ch;
xdp_return_frame(xdpf);
} else {
skb = token;
ndev = skb->dev;
ch = skb_get_queue_mapping(skb);
cpts_tx_timestamp(ndev_to_cpsw(ndev)->cpts, skb);
dev_kfree_skb_any(skb);
}
/* Check whether the queue is stopped due to stalled tx dma, if the
* queue is stopped then start the queue as we have free desc for tx
*/
txq = netdev_get_tx_queue(ndev, ch);
if (unlikely(netif_tx_queue_stopped(txq)))
netif_tx_wake_queue(txq);
ndev->stats.tx_packets++;
ndev->stats.tx_bytes += len;
}
irqreturn_t cpsw_tx_interrupt(int irq, void *dev_id)
{
struct cpsw_common *cpsw = dev_id;
writel(0, &cpsw->wr_regs->tx_en);
cpdma_ctlr_eoi(cpsw->dma, CPDMA_EOI_TX);
if (cpsw->quirk_irq) {
disable_irq_nosync(cpsw->irqs_table[1]);
cpsw->tx_irq_disabled = true;
}
napi_schedule(&cpsw->napi_tx);
return IRQ_HANDLED;
}
irqreturn_t cpsw_rx_interrupt(int irq, void *dev_id)
{
struct cpsw_common *cpsw = dev_id;
cpdma_ctlr_eoi(cpsw->dma, CPDMA_EOI_RX);
writel(0, &cpsw->wr_regs->rx_en);
if (cpsw->quirk_irq) {
disable_irq_nosync(cpsw->irqs_table[0]);
cpsw->rx_irq_disabled = true;
}
napi_schedule(&cpsw->napi_rx);
return IRQ_HANDLED;
}
int cpsw_tx_mq_poll(struct napi_struct *napi_tx, int budget)
{
struct cpsw_common *cpsw = napi_to_cpsw(napi_tx);
int num_tx, cur_budget, ch;
u32 ch_map;
struct cpsw_vector *txv;
/* process every unprocessed channel */
ch_map = cpdma_ctrl_txchs_state(cpsw->dma);
for (ch = 0, num_tx = 0; ch_map & 0xff; ch_map <<= 1, ch++) {
if (!(ch_map & 0x80))
continue;
txv = &cpsw->txv[ch];
if (unlikely(txv->budget > budget - num_tx))
cur_budget = budget - num_tx;
else
cur_budget = txv->budget;
num_tx += cpdma_chan_process(txv->ch, cur_budget);
if (num_tx >= budget)
break;
}
if (num_tx < budget) {
napi_complete(napi_tx);
writel(0xff, &cpsw->wr_regs->tx_en);
}
return num_tx;
}
int cpsw_tx_poll(struct napi_struct *napi_tx, int budget)
{
struct cpsw_common *cpsw = napi_to_cpsw(napi_tx);
int num_tx;
num_tx = cpdma_chan_process(cpsw->txv[0].ch, budget);
if (num_tx < budget) {
napi_complete(napi_tx);
writel(0xff, &cpsw->wr_regs->tx_en);
if (cpsw->tx_irq_disabled) {
cpsw->tx_irq_disabled = false;
enable_irq(cpsw->irqs_table[1]);
}
}
return num_tx;
}
int cpsw_rx_mq_poll(struct napi_struct *napi_rx, int budget)
{
struct cpsw_common *cpsw = napi_to_cpsw(napi_rx);
int num_rx, cur_budget, ch;
u32 ch_map;
struct cpsw_vector *rxv;
/* process every unprocessed channel */
ch_map = cpdma_ctrl_rxchs_state(cpsw->dma);
for (ch = 0, num_rx = 0; ch_map; ch_map >>= 1, ch++) {
if (!(ch_map & 0x01))
continue;
rxv = &cpsw->rxv[ch];
if (unlikely(rxv->budget > budget - num_rx))
cur_budget = budget - num_rx;
else
cur_budget = rxv->budget;
num_rx += cpdma_chan_process(rxv->ch, cur_budget);
if (num_rx >= budget)
break;
}
if (num_rx < budget) {
napi_complete_done(napi_rx, num_rx);
writel(0xff, &cpsw->wr_regs->rx_en);
}
return num_rx;
}
int cpsw_rx_poll(struct napi_struct *napi_rx, int budget)
{
struct cpsw_common *cpsw = napi_to_cpsw(napi_rx);
int num_rx;
num_rx = cpdma_chan_process(cpsw->rxv[0].ch, budget);
if (num_rx < budget) {
napi_complete_done(napi_rx, num_rx);
writel(0xff, &cpsw->wr_regs->rx_en);
if (cpsw->rx_irq_disabled) {
cpsw->rx_irq_disabled = false;
enable_irq(cpsw->irqs_table[0]);
}
}
return num_rx;
}
void cpsw_rx_vlan_encap(struct sk_buff *skb)
{
struct cpsw_priv *priv = netdev_priv(skb->dev);
u32 rx_vlan_encap_hdr = *((u32 *)skb->data);
struct cpsw_common *cpsw = priv->cpsw;
u16 vtag, vid, prio, pkt_type;
/* Remove VLAN header encapsulation word */
skb_pull(skb, CPSW_RX_VLAN_ENCAP_HDR_SIZE);
pkt_type = (rx_vlan_encap_hdr >>
CPSW_RX_VLAN_ENCAP_HDR_PKT_TYPE_SHIFT) &
CPSW_RX_VLAN_ENCAP_HDR_PKT_TYPE_MSK;
/* Ignore unknown & Priority-tagged packets*/
if (pkt_type == CPSW_RX_VLAN_ENCAP_HDR_PKT_RESERV ||
pkt_type == CPSW_RX_VLAN_ENCAP_HDR_PKT_PRIO_TAG)
return;
vid = (rx_vlan_encap_hdr >>
CPSW_RX_VLAN_ENCAP_HDR_VID_SHIFT) &
VLAN_VID_MASK;
/* Ignore vid 0 and pass packet as is */
if (!vid)
return;
/* Untag P0 packets if set for vlan */
if (!cpsw_ale_get_vlan_p0_untag(cpsw->ale, vid)) {
prio = (rx_vlan_encap_hdr >>
CPSW_RX_VLAN_ENCAP_HDR_PRIO_SHIFT) &
CPSW_RX_VLAN_ENCAP_HDR_PRIO_MSK;
vtag = (prio << VLAN_PRIO_SHIFT) | vid;
__vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q), vtag);
}
/* strip vlan tag for VLAN-tagged packet */
if (pkt_type == CPSW_RX_VLAN_ENCAP_HDR_PKT_VLAN_TAG) {
memmove(skb->data + VLAN_HLEN, skb->data, 2 * ETH_ALEN);
skb_pull(skb, VLAN_HLEN);
}
}
void cpsw_set_slave_mac(struct cpsw_slave *slave, struct cpsw_priv *priv)
{
slave_write(slave, mac_hi(priv->mac_addr), SA_HI);
slave_write(slave, mac_lo(priv->mac_addr), SA_LO);
}
void soft_reset(const char *module, void __iomem *reg)
{
unsigned long timeout = jiffies + HZ;
writel_relaxed(1, reg);
do {
cpu_relax();
} while ((readl_relaxed(reg) & 1) && time_after(timeout, jiffies));
WARN(readl_relaxed(reg) & 1, "failed to soft-reset %s\n", module);
}
void cpsw_ndo_tx_timeout(struct net_device *ndev)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
int ch;
cpsw_err(priv, tx_err, "transmit timeout, restarting dma\n");
ndev->stats.tx_errors++;
cpsw_intr_disable(cpsw);
for (ch = 0; ch < cpsw->tx_ch_num; ch++) {
cpdma_chan_stop(cpsw->txv[ch].ch);
cpdma_chan_start(cpsw->txv[ch].ch);
}
cpsw_intr_enable(cpsw);
netif_trans_update(ndev);
netif_tx_wake_all_queues(ndev);
}
static int cpsw_get_common_speed(struct cpsw_common *cpsw)
{
int i, speed;
for (i = 0, speed = 0; i < cpsw->data.slaves; i++)
if (cpsw->slaves[i].phy && cpsw->slaves[i].phy->link)
speed += cpsw->slaves[i].phy->speed;
return speed;
}
int cpsw_need_resplit(struct cpsw_common *cpsw)
{
int i, rlim_ch_num;
int speed, ch_rate;
/* re-split resources only in case speed was changed */
speed = cpsw_get_common_speed(cpsw);
if (speed == cpsw->speed || !speed)
return 0;
cpsw->speed = speed;
for (i = 0, rlim_ch_num = 0; i < cpsw->tx_ch_num; i++) {
ch_rate = cpdma_chan_get_rate(cpsw->txv[i].ch);
if (!ch_rate)
break;
rlim_ch_num++;
}
/* cases not dependent on speed */
if (!rlim_ch_num || rlim_ch_num == cpsw->tx_ch_num)
return 0;
return 1;
}
void cpsw_split_res(struct cpsw_common *cpsw)
{
u32 consumed_rate = 0, bigest_rate = 0;
struct cpsw_vector *txv = cpsw->txv;
int i, ch_weight, rlim_ch_num = 0;
int budget, bigest_rate_ch = 0;
u32 ch_rate, max_rate;
int ch_budget = 0;
for (i = 0; i < cpsw->tx_ch_num; i++) {
ch_rate = cpdma_chan_get_rate(txv[i].ch);
if (!ch_rate)
continue;
rlim_ch_num++;
consumed_rate += ch_rate;
}
if (cpsw->tx_ch_num == rlim_ch_num) {
max_rate = consumed_rate;
} else if (!rlim_ch_num) {
ch_budget = CPSW_POLL_WEIGHT / cpsw->tx_ch_num;
bigest_rate = 0;
max_rate = consumed_rate;
} else {
max_rate = cpsw->speed * 1000;
/* if max_rate is less then expected due to reduced link speed,
* split proportionally according next potential max speed
*/
if (max_rate < consumed_rate)
max_rate *= 10;
if (max_rate < consumed_rate)
max_rate *= 10;
ch_budget = (consumed_rate * CPSW_POLL_WEIGHT) / max_rate;
ch_budget = (CPSW_POLL_WEIGHT - ch_budget) /
(cpsw->tx_ch_num - rlim_ch_num);
bigest_rate = (max_rate - consumed_rate) /
(cpsw->tx_ch_num - rlim_ch_num);
}
/* split tx weight/budget */
budget = CPSW_POLL_WEIGHT;
for (i = 0; i < cpsw->tx_ch_num; i++) {
ch_rate = cpdma_chan_get_rate(txv[i].ch);
if (ch_rate) {
txv[i].budget = (ch_rate * CPSW_POLL_WEIGHT) / max_rate;
if (!txv[i].budget)
txv[i].budget++;
if (ch_rate > bigest_rate) {
bigest_rate_ch = i;
bigest_rate = ch_rate;
}
ch_weight = (ch_rate * 100) / max_rate;
if (!ch_weight)
ch_weight++;
cpdma_chan_set_weight(cpsw->txv[i].ch, ch_weight);
} else {
txv[i].budget = ch_budget;
if (!bigest_rate_ch)
bigest_rate_ch = i;
cpdma_chan_set_weight(cpsw->txv[i].ch, 0);
}
budget -= txv[i].budget;
}
if (budget)
txv[bigest_rate_ch].budget += budget;
/* split rx budget */
budget = CPSW_POLL_WEIGHT;
ch_budget = budget / cpsw->rx_ch_num;
for (i = 0; i < cpsw->rx_ch_num; i++) {
cpsw->rxv[i].budget = ch_budget;
budget -= ch_budget;
}
if (budget)
cpsw->rxv[0].budget += budget;
}
int cpsw_init_common(struct cpsw_common *cpsw, void __iomem *ss_regs,
int ale_ageout, phys_addr_t desc_mem_phys,
int descs_pool_size)
......@@ -28,6 +423,7 @@ int cpsw_init_common(struct cpsw_common *cpsw, void __iomem *ss_regs,
struct cpsw_platform_data *data;
struct cpdma_params dma_params;
struct device *dev = cpsw->dev;
struct device_node *cpts_node;
void __iomem *cpts_regs;
int ret = 0, i;
......@@ -122,11 +518,859 @@ int cpsw_init_common(struct cpsw_common *cpsw, void __iomem *ss_regs,
return -ENOMEM;
}
cpsw->cpts = cpts_create(cpsw->dev, cpts_regs, cpsw->dev->of_node);
cpts_node = of_get_child_by_name(cpsw->dev->of_node, "cpts");
if (!cpts_node)
cpts_node = cpsw->dev->of_node;
cpsw->cpts = cpts_create(cpsw->dev, cpts_regs, cpts_node);
if (IS_ERR(cpsw->cpts)) {
ret = PTR_ERR(cpsw->cpts);
cpdma_ctlr_destroy(cpsw->dma);
}
of_node_put(cpts_node);
return ret;
}
#if IS_ENABLED(CONFIG_TI_CPTS)
static void cpsw_hwtstamp_v1(struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
u32 ts_en, seq_id;
if (!priv->tx_ts_enabled && !priv->rx_ts_enabled) {
slave_write(slave, 0, CPSW1_TS_CTL);
return;
}
seq_id = (30 << CPSW_V1_SEQ_ID_OFS_SHIFT) | ETH_P_1588;
ts_en = EVENT_MSG_BITS << CPSW_V1_MSG_TYPE_OFS;
if (priv->tx_ts_enabled)
ts_en |= CPSW_V1_TS_TX_EN;
if (priv->rx_ts_enabled)
ts_en |= CPSW_V1_TS_RX_EN;
slave_write(slave, ts_en, CPSW1_TS_CTL);
slave_write(slave, seq_id, CPSW1_TS_SEQ_LTYPE);
}
static void cpsw_hwtstamp_v2(struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
u32 ctrl, mtype;
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
ctrl = slave_read(slave, CPSW2_CONTROL);
switch (cpsw->version) {
case CPSW_VERSION_2:
ctrl &= ~CTRL_V2_ALL_TS_MASK;
if (priv->tx_ts_enabled)
ctrl |= CTRL_V2_TX_TS_BITS;
if (priv->rx_ts_enabled)
ctrl |= CTRL_V2_RX_TS_BITS;
break;
case CPSW_VERSION_3:
default:
ctrl &= ~CTRL_V3_ALL_TS_MASK;
if (priv->tx_ts_enabled)
ctrl |= CTRL_V3_TX_TS_BITS;
if (priv->rx_ts_enabled)
ctrl |= CTRL_V3_RX_TS_BITS;
break;
}
mtype = (30 << TS_SEQ_ID_OFFSET_SHIFT) | EVENT_MSG_BITS;
slave_write(slave, mtype, CPSW2_TS_SEQ_MTYPE);
slave_write(slave, ctrl, CPSW2_CONTROL);
writel_relaxed(ETH_P_1588, &cpsw->regs->ts_ltype);
writel_relaxed(ETH_P_8021Q, &cpsw->regs->vlan_ltype);
}
static int cpsw_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
{
struct cpsw_priv *priv = netdev_priv(dev);
struct cpsw_common *cpsw = priv->cpsw;
struct hwtstamp_config cfg;
if (cpsw->version != CPSW_VERSION_1 &&
cpsw->version != CPSW_VERSION_2 &&
cpsw->version != CPSW_VERSION_3)
return -EOPNOTSUPP;
if (copy_from_user(&cfg, ifr->ifr_data, sizeof(cfg)))
return -EFAULT;
/* reserved for future extensions */
if (cfg.flags)
return -EINVAL;
if (cfg.tx_type != HWTSTAMP_TX_OFF && cfg.tx_type != HWTSTAMP_TX_ON)
return -ERANGE;
switch (cfg.rx_filter) {
case HWTSTAMP_FILTER_NONE:
priv->rx_ts_enabled = 0;
break;
case HWTSTAMP_FILTER_ALL:
case HWTSTAMP_FILTER_NTP_ALL:
return -ERANGE;
case HWTSTAMP_FILTER_PTP_V1_L4_EVENT:
case HWTSTAMP_FILTER_PTP_V1_L4_SYNC:
case HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ:
priv->rx_ts_enabled = HWTSTAMP_FILTER_PTP_V1_L4_EVENT;
cfg.rx_filter = HWTSTAMP_FILTER_PTP_V1_L4_EVENT;
break;
case HWTSTAMP_FILTER_PTP_V2_L4_EVENT:
case HWTSTAMP_FILTER_PTP_V2_L4_SYNC:
case HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ:
case HWTSTAMP_FILTER_PTP_V2_L2_EVENT:
case HWTSTAMP_FILTER_PTP_V2_L2_SYNC:
case HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ:
case HWTSTAMP_FILTER_PTP_V2_EVENT:
case HWTSTAMP_FILTER_PTP_V2_SYNC:
case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ:
priv->rx_ts_enabled = HWTSTAMP_FILTER_PTP_V2_EVENT;
cfg.rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT;
break;
default:
return -ERANGE;
}
priv->tx_ts_enabled = cfg.tx_type == HWTSTAMP_TX_ON;
switch (cpsw->version) {
case CPSW_VERSION_1:
cpsw_hwtstamp_v1(priv);
break;
case CPSW_VERSION_2:
case CPSW_VERSION_3:
cpsw_hwtstamp_v2(priv);
break;
default:
WARN_ON(1);
}
return copy_to_user(ifr->ifr_data, &cfg, sizeof(cfg)) ? -EFAULT : 0;
}
static int cpsw_hwtstamp_get(struct net_device *dev, struct ifreq *ifr)
{
struct cpsw_common *cpsw = ndev_to_cpsw(dev);
struct cpsw_priv *priv = netdev_priv(dev);
struct hwtstamp_config cfg;
if (cpsw->version != CPSW_VERSION_1 &&
cpsw->version != CPSW_VERSION_2 &&
cpsw->version != CPSW_VERSION_3)
return -EOPNOTSUPP;
cfg.flags = 0;
cfg.tx_type = priv->tx_ts_enabled ? HWTSTAMP_TX_ON : HWTSTAMP_TX_OFF;
cfg.rx_filter = priv->rx_ts_enabled;
return copy_to_user(ifr->ifr_data, &cfg, sizeof(cfg)) ? -EFAULT : 0;
}
#else
static int cpsw_hwtstamp_get(struct net_device *dev, struct ifreq *ifr)
{
return -EOPNOTSUPP;
}
static int cpsw_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
{
return -EOPNOTSUPP;
}
#endif /*CONFIG_TI_CPTS*/
int cpsw_ndo_ioctl(struct net_device *dev, struct ifreq *req, int cmd)
{
struct cpsw_priv *priv = netdev_priv(dev);
struct cpsw_common *cpsw = priv->cpsw;
int slave_no = cpsw_slave_index(cpsw, priv);
if (!netif_running(dev))
return -EINVAL;
switch (cmd) {
case SIOCSHWTSTAMP:
return cpsw_hwtstamp_set(dev, req);
case SIOCGHWTSTAMP:
return cpsw_hwtstamp_get(dev, req);
}
if (!cpsw->slaves[slave_no].phy)
return -EOPNOTSUPP;
return phy_mii_ioctl(cpsw->slaves[slave_no].phy, req, cmd);
}
int cpsw_ndo_set_tx_maxrate(struct net_device *ndev, int queue, u32 rate)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
u32 min_rate;
u32 ch_rate;
int i, ret;
ch_rate = netdev_get_tx_queue(ndev, queue)->tx_maxrate;
if (ch_rate == rate)
return 0;
ch_rate = rate * 1000;
min_rate = cpdma_chan_get_min_rate(cpsw->dma);
if ((ch_rate < min_rate && ch_rate)) {
dev_err(priv->dev, "The channel rate cannot be less than %dMbps",
min_rate);
return -EINVAL;
}
if (rate > cpsw->speed) {
dev_err(priv->dev, "The channel rate cannot be more than 2Gbps");
return -EINVAL;
}
ret = pm_runtime_get_sync(cpsw->dev);
if (ret < 0) {
pm_runtime_put_noidle(cpsw->dev);
return ret;
}
ret = cpdma_chan_set_rate(cpsw->txv[queue].ch, ch_rate);
pm_runtime_put(cpsw->dev);
if (ret)
return ret;
/* update rates for slaves tx queues */
for (i = 0; i < cpsw->data.slaves; i++) {
slave = &cpsw->slaves[i];
if (!slave->ndev)
continue;
netdev_get_tx_queue(slave->ndev, queue)->tx_maxrate = rate;
}
cpsw_split_res(cpsw);
return ret;
}
static int cpsw_tc_to_fifo(int tc, int num_tc)
{
if (tc == num_tc - 1)
return 0;
return CPSW_FIFO_SHAPERS_NUM - tc;
}
bool cpsw_shp_is_off(struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
u32 shift, mask, val;
val = readl_relaxed(&cpsw->regs->ptype);
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
shift = CPSW_FIFO_SHAPE_EN_SHIFT + 3 * slave->slave_num;
mask = 7 << shift;
val = val & mask;
return !val;
}
static void cpsw_fifo_shp_on(struct cpsw_priv *priv, int fifo, int on)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
u32 shift, mask, val;
val = readl_relaxed(&cpsw->regs->ptype);
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
shift = CPSW_FIFO_SHAPE_EN_SHIFT + 3 * slave->slave_num;
mask = (1 << --fifo) << shift;
val = on ? val | mask : val & ~mask;
writel_relaxed(val, &cpsw->regs->ptype);
}
static int cpsw_set_fifo_bw(struct cpsw_priv *priv, int fifo, int bw)
{
struct cpsw_common *cpsw = priv->cpsw;
u32 val = 0, send_pct, shift;
struct cpsw_slave *slave;
int pct = 0, i;
if (bw > priv->shp_cfg_speed * 1000)
goto err;
/* shaping has to stay enabled for highest fifos linearly
* and fifo bw no more then interface can allow
*/
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
send_pct = slave_read(slave, SEND_PERCENT);
for (i = CPSW_FIFO_SHAPERS_NUM; i > 0; i--) {
if (!bw) {
if (i >= fifo || !priv->fifo_bw[i])
continue;
dev_warn(priv->dev, "Prev FIFO%d is shaped", i);
continue;
}
if (!priv->fifo_bw[i] && i > fifo) {
dev_err(priv->dev, "Upper FIFO%d is not shaped", i);
return -EINVAL;
}
shift = (i - 1) * 8;
if (i == fifo) {
send_pct &= ~(CPSW_PCT_MASK << shift);
val = DIV_ROUND_UP(bw, priv->shp_cfg_speed * 10);
if (!val)
val = 1;
send_pct |= val << shift;
pct += val;
continue;
}
if (priv->fifo_bw[i])
pct += (send_pct >> shift) & CPSW_PCT_MASK;
}
if (pct >= 100)
goto err;
slave_write(slave, send_pct, SEND_PERCENT);
priv->fifo_bw[fifo] = bw;
dev_warn(priv->dev, "set FIFO%d bw = %d\n", fifo,
DIV_ROUND_CLOSEST(val * priv->shp_cfg_speed, 100));
return 0;
err:
dev_err(priv->dev, "Bandwidth doesn't fit in tc configuration");
return -EINVAL;
}
static int cpsw_set_fifo_rlimit(struct cpsw_priv *priv, int fifo, int bw)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
u32 tx_in_ctl_rg, val;
int ret;
ret = cpsw_set_fifo_bw(priv, fifo, bw);
if (ret)
return ret;
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
tx_in_ctl_rg = cpsw->version == CPSW_VERSION_1 ?
CPSW1_TX_IN_CTL : CPSW2_TX_IN_CTL;
if (!bw)
cpsw_fifo_shp_on(priv, fifo, bw);
val = slave_read(slave, tx_in_ctl_rg);
if (cpsw_shp_is_off(priv)) {
/* disable FIFOs rate limited queues */
val &= ~(0xf << CPSW_FIFO_RATE_EN_SHIFT);
/* set type of FIFO queues to normal priority mode */
val &= ~(3 << CPSW_FIFO_QUEUE_TYPE_SHIFT);
/* set type of FIFO queues to be rate limited */
if (bw)
val |= 2 << CPSW_FIFO_QUEUE_TYPE_SHIFT;
else
priv->shp_cfg_speed = 0;
}
/* toggle a FIFO rate limited queue */
if (bw)
val |= BIT(fifo + CPSW_FIFO_RATE_EN_SHIFT);
else
val &= ~BIT(fifo + CPSW_FIFO_RATE_EN_SHIFT);
slave_write(slave, val, tx_in_ctl_rg);
/* FIFO transmit shape enable */
cpsw_fifo_shp_on(priv, fifo, bw);
return 0;
}
/* Defaults:
* class A - prio 3
* class B - prio 2
* shaping for class A should be set first
*/
static int cpsw_set_cbs(struct net_device *ndev,
struct tc_cbs_qopt_offload *qopt)
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave;
int prev_speed = 0;
int tc, ret, fifo;
u32 bw = 0;
tc = netdev_txq_to_tc(priv->ndev, qopt->queue);
/* enable channels in backward order, as highest FIFOs must be rate
* limited first and for compliance with CPDMA rate limited channels
* that also used in bacward order. FIFO0 cannot be rate limited.
*/
fifo = cpsw_tc_to_fifo(tc, ndev->num_tc);
if (!fifo) {
dev_err(priv->dev, "Last tc%d can't be rate limited", tc);
return -EINVAL;
}
/* do nothing, it's disabled anyway */
if (!qopt->enable && !priv->fifo_bw[fifo])
return 0;
/* shapers can be set if link speed is known */
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
if (slave->phy && slave->phy->link) {
if (priv->shp_cfg_speed &&
priv->shp_cfg_speed != slave->phy->speed)
prev_speed = priv->shp_cfg_speed;
priv->shp_cfg_speed = slave->phy->speed;
}
if (!priv->shp_cfg_speed) {
dev_err(priv->dev, "Link speed is not known");
return -1;
}
ret = pm_runtime_get_sync(cpsw->dev);
if (ret < 0) {
pm_runtime_put_noidle(cpsw->dev);
return ret;
}
bw = qopt->enable ? qopt->idleslope : 0;
ret = cpsw_set_fifo_rlimit(priv, fifo, bw);
if (ret) {
priv->shp_cfg_speed = prev_speed;
prev_speed = 0;
}
if (bw && prev_speed)
dev_warn(priv->dev,
"Speed was changed, CBS shaper speeds are changed!");
pm_runtime_put_sync(cpsw->dev);
return ret;
}
static int cpsw_set_mqprio(struct net_device *ndev, void *type_data)
{
struct tc_mqprio_qopt_offload *mqprio = type_data;
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
int fifo, num_tc, count, offset;
struct cpsw_slave *slave;
u32 tx_prio_map = 0;
int i, tc, ret;
num_tc = mqprio->qopt.num_tc;
if (num_tc > CPSW_TC_NUM)
return -EINVAL;
if (mqprio->mode != TC_MQPRIO_MODE_DCB)
return -EINVAL;
ret = pm_runtime_get_sync(cpsw->dev);
if (ret < 0) {
pm_runtime_put_noidle(cpsw->dev);
return ret;
}
if (num_tc) {
for (i = 0; i < 8; i++) {
tc = mqprio->qopt.prio_tc_map[i];
fifo = cpsw_tc_to_fifo(tc, num_tc);
tx_prio_map |= fifo << (4 * i);
}
netdev_set_num_tc(ndev, num_tc);
for (i = 0; i < num_tc; i++) {
count = mqprio->qopt.count[i];
offset = mqprio->qopt.offset[i];
netdev_set_tc_queue(ndev, i, count, offset);
}
}
if (!mqprio->qopt.hw) {
/* restore default configuration */
netdev_reset_tc(ndev);
tx_prio_map = TX_PRIORITY_MAPPING;
}
priv->mqprio_hw = mqprio->qopt.hw;
offset = cpsw->version == CPSW_VERSION_1 ?
CPSW1_TX_PRI_MAP : CPSW2_TX_PRI_MAP;
slave = &cpsw->slaves[cpsw_slave_index(cpsw, priv)];
slave_write(slave, tx_prio_map, offset);
pm_runtime_put_sync(cpsw->dev);
return 0;
}
int cpsw_ndo_setup_tc(struct net_device *ndev, enum tc_setup_type type,
void *type_data)
{
switch (type) {
case TC_SETUP_QDISC_CBS:
return cpsw_set_cbs(ndev, type_data);
case TC_SETUP_QDISC_MQPRIO:
return cpsw_set_mqprio(ndev, type_data);
default:
return -EOPNOTSUPP;
}
}
void cpsw_cbs_resume(struct cpsw_slave *slave, struct cpsw_priv *priv)
{
int fifo, bw;
for (fifo = CPSW_FIFO_SHAPERS_NUM; fifo > 0; fifo--) {
bw = priv->fifo_bw[fifo];
if (!bw)
continue;
cpsw_set_fifo_rlimit(priv, fifo, bw);
}
}
void cpsw_mqprio_resume(struct cpsw_slave *slave, struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
u32 tx_prio_map = 0;
int i, tc, fifo;
u32 tx_prio_rg;
if (!priv->mqprio_hw)
return;
for (i = 0; i < 8; i++) {
tc = netdev_get_prio_tc_map(priv->ndev, i);
fifo = CPSW_FIFO_SHAPERS_NUM - tc;
tx_prio_map |= fifo << (4 * i);
}
tx_prio_rg = cpsw->version == CPSW_VERSION_1 ?
CPSW1_TX_PRI_MAP : CPSW2_TX_PRI_MAP;
slave_write(slave, tx_prio_map, tx_prio_rg);
}
int cpsw_fill_rx_channels(struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_meta_xdp *xmeta;
struct page_pool *pool;
struct page *page;
int ch_buf_num;
int ch, i, ret;
dma_addr_t dma;
for (ch = 0; ch < cpsw->rx_ch_num; ch++) {
pool = cpsw->page_pool[ch];
ch_buf_num = cpdma_chan_get_rx_buf_num(cpsw->rxv[ch].ch);
for (i = 0; i < ch_buf_num; i++) {
page = page_pool_dev_alloc_pages(pool);
if (!page) {
cpsw_err(priv, ifup, "allocate rx page err\n");
return -ENOMEM;
}
xmeta = page_address(page) + CPSW_XMETA_OFFSET;
xmeta->ndev = priv->ndev;
xmeta->ch = ch;
dma = page_pool_get_dma_addr(page) + CPSW_HEADROOM;
ret = cpdma_chan_idle_submit_mapped(cpsw->rxv[ch].ch,
page, dma,
cpsw->rx_packet_max,
0);
if (ret < 0) {
cpsw_err(priv, ifup,
"cannot submit page to channel %d rx, error %d\n",
ch, ret);
page_pool_recycle_direct(pool, page);
return ret;
}
}
cpsw_info(priv, ifup, "ch %d rx, submitted %d descriptors\n",
ch, ch_buf_num);
}
return 0;
}
static struct page_pool *cpsw_create_page_pool(struct cpsw_common *cpsw,
int size)
{
struct page_pool_params pp_params;
struct page_pool *pool;
pp_params.order = 0;
pp_params.flags = PP_FLAG_DMA_MAP;
pp_params.pool_size = size;
pp_params.nid = NUMA_NO_NODE;
pp_params.dma_dir = DMA_BIDIRECTIONAL;
pp_params.dev = cpsw->dev;
pool = page_pool_create(&pp_params);
if (IS_ERR(pool))
dev_err(cpsw->dev, "cannot create rx page pool\n");
return pool;
}
static int cpsw_create_rx_pool(struct cpsw_common *cpsw, int ch)
{
struct page_pool *pool;
int ret = 0, pool_size;
pool_size = cpdma_chan_get_rx_buf_num(cpsw->rxv[ch].ch);
pool = cpsw_create_page_pool(cpsw, pool_size);
if (IS_ERR(pool))
ret = PTR_ERR(pool);
else
cpsw->page_pool[ch] = pool;
return ret;
}
static int cpsw_ndev_create_xdp_rxq(struct cpsw_priv *priv, int ch)
{
struct cpsw_common *cpsw = priv->cpsw;
struct xdp_rxq_info *rxq;
struct page_pool *pool;
int ret;
pool = cpsw->page_pool[ch];
rxq = &priv->xdp_rxq[ch];
ret = xdp_rxq_info_reg(rxq, priv->ndev, ch);
if (ret)
return ret;
ret = xdp_rxq_info_reg_mem_model(rxq, MEM_TYPE_PAGE_POOL, pool);
if (ret)
xdp_rxq_info_unreg(rxq);
return ret;
}
static void cpsw_ndev_destroy_xdp_rxq(struct cpsw_priv *priv, int ch)
{
struct xdp_rxq_info *rxq = &priv->xdp_rxq[ch];
if (!xdp_rxq_info_is_reg(rxq))
return;
xdp_rxq_info_unreg(rxq);
}
void cpsw_destroy_xdp_rxqs(struct cpsw_common *cpsw)
{
struct net_device *ndev;
int i, ch;
for (ch = 0; ch < cpsw->rx_ch_num; ch++) {
for (i = 0; i < cpsw->data.slaves; i++) {
ndev = cpsw->slaves[i].ndev;
if (!ndev)
continue;
cpsw_ndev_destroy_xdp_rxq(netdev_priv(ndev), ch);
}
page_pool_destroy(cpsw->page_pool[ch]);
cpsw->page_pool[ch] = NULL;
}
}
int cpsw_create_xdp_rxqs(struct cpsw_common *cpsw)
{
struct net_device *ndev;
int i, ch, ret;
for (ch = 0; ch < cpsw->rx_ch_num; ch++) {
ret = cpsw_create_rx_pool(cpsw, ch);
if (ret)
goto err_cleanup;
/* using same page pool is allowed as no running rx handlers
* simultaneously for both ndevs
*/
for (i = 0; i < cpsw->data.slaves; i++) {
ndev = cpsw->slaves[i].ndev;
if (!ndev)
continue;
ret = cpsw_ndev_create_xdp_rxq(netdev_priv(ndev), ch);
if (ret)
goto err_cleanup;
}
}
return 0;
err_cleanup:
cpsw_destroy_xdp_rxqs(cpsw);
return ret;
}
static int cpsw_xdp_prog_setup(struct cpsw_priv *priv, struct netdev_bpf *bpf)
{
struct bpf_prog *prog = bpf->prog;
if (!priv->xdpi.prog && !prog)
return 0;
if (!xdp_attachment_flags_ok(&priv->xdpi, bpf))
return -EBUSY;
WRITE_ONCE(priv->xdp_prog, prog);
xdp_attachment_setup(&priv->xdpi, bpf);
return 0;
}
int cpsw_ndo_bpf(struct net_device *ndev, struct netdev_bpf *bpf)
{
struct cpsw_priv *priv = netdev_priv(ndev);
switch (bpf->command) {
case XDP_SETUP_PROG:
return cpsw_xdp_prog_setup(priv, bpf);
case XDP_QUERY_PROG:
return xdp_attachment_query(&priv->xdpi, bpf);
default:
return -EINVAL;
}
}
int cpsw_xdp_tx_frame(struct cpsw_priv *priv, struct xdp_frame *xdpf,
struct page *page, int port)
{
struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_meta_xdp *xmeta;
struct cpdma_chan *txch;
dma_addr_t dma;
int ret;
xmeta = (void *)xdpf + CPSW_XMETA_OFFSET;
xmeta->ndev = priv->ndev;
xmeta->ch = 0;
txch = cpsw->txv[0].ch;
if (page) {
dma = page_pool_get_dma_addr(page);
dma += xdpf->headroom + sizeof(struct xdp_frame);
ret = cpdma_chan_submit_mapped(txch, cpsw_xdpf_to_handle(xdpf),
dma, xdpf->len, port);
} else {
if (sizeof(*xmeta) > xdpf->headroom) {
xdp_return_frame_rx_napi(xdpf);
return -EINVAL;
}
ret = cpdma_chan_submit(txch, cpsw_xdpf_to_handle(xdpf),
xdpf->data, xdpf->len, port);
}
if (ret) {
priv->ndev->stats.tx_dropped++;
xdp_return_frame_rx_napi(xdpf);
}
return ret;
}
int cpsw_run_xdp(struct cpsw_priv *priv, int ch, struct xdp_buff *xdp,
struct page *page, int port)
{
struct cpsw_common *cpsw = priv->cpsw;
struct net_device *ndev = priv->ndev;
int ret = CPSW_XDP_CONSUMED;
struct xdp_frame *xdpf;
struct bpf_prog *prog;
u32 act;
rcu_read_lock();
prog = READ_ONCE(priv->xdp_prog);
if (!prog) {
ret = CPSW_XDP_PASS;
goto out;
}
act = bpf_prog_run_xdp(prog, xdp);
switch (act) {
case XDP_PASS:
ret = CPSW_XDP_PASS;
break;
case XDP_TX:
xdpf = convert_to_xdp_frame(xdp);
if (unlikely(!xdpf))
goto drop;
cpsw_xdp_tx_frame(priv, xdpf, page, port);
break;
case XDP_REDIRECT:
if (xdp_do_redirect(ndev, xdp, prog))
goto drop;
/* Have to flush here, per packet, instead of doing it in bulk
* at the end of the napi handler. The RX devices on this
* particular hardware is sharing a common queue, so the
* incoming device might change per packet.
*/
xdp_do_flush_map();
break;
default:
bpf_warn_invalid_xdp_action(act);
/* fall through */
case XDP_ABORTED:
trace_xdp_exception(ndev, prog, act);
/* fall through -- handle aborts by dropping packet */
case XDP_DROP:
goto drop;
}
out:
rcu_read_unlock();
return ret;
drop:
rcu_read_unlock();
page_pool_recycle_direct(cpsw->page_pool[ch], page);
return ret;
}
drivers/net/ethernet/ti/cpsw_priv.h
View file @ 1f12177b
......@@ -54,6 +54,7 @@ do { \
#define HOST_PORT_NUM 0
#define CPSW_ALE_PORTS_NUM 3
#define CPSW_SLAVE_PORTS_NUM 2
#define SLIVER_SIZE 0x40
#define CPSW1_HOST_PORT_OFFSET 0x028
......@@ -65,6 +66,7 @@ do { \
#define CPSW1_CPTS_OFFSET 0x500
#define CPSW1_ALE_OFFSET 0x600
#define CPSW1_SLIVER_OFFSET 0x700
#define CPSW1_WR_OFFSET 0x900
#define CPSW2_HOST_PORT_OFFSET 0x108
#define CPSW2_SLAVE_OFFSET 0x200
......@@ -76,6 +78,7 @@ do { \
#define CPSW2_ALE_OFFSET 0xd00
#define CPSW2_SLIVER_OFFSET 0xd80
#define CPSW2_BD_OFFSET 0x2000
#define CPSW2_WR_OFFSET 0x1200
#define CPDMA_RXTHRESH 0x0c0
#define CPDMA_RXFREE 0x0e0
......@@ -113,12 +116,15 @@ do { \
#define IRQ_NUM 2
#define CPSW_MAX_QUEUES 8
#define CPSW_CPDMA_DESCS_POOL_SIZE_DEFAULT 256
#define CPSW_ALE_AGEOUT_DEFAULT 10 /* sec */
#define CPSW_ALE_NUM_ENTRIES 1024
#define CPSW_FIFO_QUEUE_TYPE_SHIFT 16
#define CPSW_FIFO_SHAPE_EN_SHIFT 16
#define CPSW_FIFO_RATE_EN_SHIFT 20
#define CPSW_TC_NUM 4
#define CPSW_FIFO_SHAPERS_NUM (CPSW_TC_NUM - 1)
#define CPSW_PCT_MASK 0x7f
#define CPSW_BD_RAM_SIZE 0x2000
#define CPSW_RX_VLAN_ENCAP_HDR_PRIO_SHIFT 29
#define CPSW_RX_VLAN_ENCAP_HDR_PRIO_MSK GENMASK(2, 0)
......@@ -279,6 +285,7 @@ struct cpsw_slave_data {
u8 mac_addr[ETH_ALEN];
u16 dual_emac_res_vlan; /* Reserved VLAN for DualEMAC */
struct phy *ifphy;
bool disabled;
};
struct cpsw_platform_data {
......@@ -286,9 +293,9 @@ struct cpsw_platform_data {
u32 ss_reg_ofs; /* Subsystem control register offset */
u32 channels; /* number of cpdma channels (symmetric) */
u32 slaves; /* number of slave cpgmac ports */
u32 active_slave; /* time stamping, ethtool and SIOCGMIIPHY slave */
u32 active_slave;/* time stamping, ethtool and SIOCGMIIPHY slave */
u32 ale_entries; /* ale table size */
u32 bd_ram_size; /*buffer descriptor ram size */
u32 bd_ram_size; /*buffer descriptor ram size */
u32 mac_control; /* Mac control register */
u16 default_vlan; /* Def VLAN for ALE lookup in VLAN aware mode*/
bool dual_emac; /* Enable Dual EMAC mode */
......@@ -344,10 +351,15 @@ struct cpsw_common {
bool tx_irq_disabled;
u32 irqs_table[IRQ_NUM];
struct cpts *cpts;
struct devlink *devlink;
int rx_ch_num, tx_ch_num;
int speed;
int usage_count;
struct page_pool *page_pool[CPSW_MAX_QUEUES];
u8 br_members;
struct net_device *hw_bridge_dev;
bool ale_bypass;
u8 base_mac[ETH_ALEN];
};
struct cpsw_priv {
......@@ -368,19 +380,14 @@ struct cpsw_priv {
u32 emac_port;
struct cpsw_common *cpsw;
int offload_fwd_mark;
};
#define ndev_to_cpsw(ndev) (((struct cpsw_priv *)netdev_priv(ndev))->cpsw)
#define napi_to_cpsw(napi) container_of(napi, struct cpsw_common, napi)
#define cpsw_slave_index(cpsw, priv) \
((cpsw->data.dual_emac) ? priv->emac_port : \
cpsw->data.active_slave)
static inline int cpsw_get_slave_port(u32 slave_num)
{
return slave_num + 1;
}
extern int (*cpsw_slave_index)(struct cpsw_common *cpsw,
struct cpsw_priv *priv);
struct addr_sync_ctx {
struct net_device *ndev;
......@@ -389,6 +396,35 @@ struct addr_sync_ctx {
int flush; /* flush flag */
};
#define CPSW_XMETA_OFFSET ALIGN(sizeof(struct xdp_frame), sizeof(long))
#define CPSW_XDP_CONSUMED 1
#define CPSW_XDP_PASS 0
struct __aligned(sizeof(long)) cpsw_meta_xdp {
struct net_device *ndev;
int ch;
};
/* The buf includes headroom compatible with both skb and xdpf */
#define CPSW_HEADROOM_NA (max(XDP_PACKET_HEADROOM, NET_SKB_PAD) + NET_IP_ALIGN)
#define CPSW_HEADROOM ALIGN(CPSW_HEADROOM_NA, sizeof(long))
static inline int cpsw_is_xdpf_handle(void *handle)
{
return (unsigned long)handle & BIT(0);
}
static inline void *cpsw_xdpf_to_handle(struct xdp_frame *xdpf)
{
return (void *)((unsigned long)xdpf | BIT(0));
}
static inline struct xdp_frame *cpsw_handle_to_xdpf(void *handle)
{
return (struct xdp_frame *)((unsigned long)handle & ~BIT(0));
}
int cpsw_init_common(struct cpsw_common *cpsw, void __iomem *ss_regs,
int ale_ageout, phys_addr_t desc_mem_phys,
int descs_pool_size);
......@@ -399,6 +435,29 @@ void cpsw_intr_disable(struct cpsw_common *cpsw);
void cpsw_tx_handler(void *token, int len, int status);
int cpsw_create_xdp_rxqs(struct cpsw_common *cpsw);
void cpsw_destroy_xdp_rxqs(struct cpsw_common *cpsw);
int cpsw_ndo_bpf(struct net_device *ndev, struct netdev_bpf *bpf);
int cpsw_xdp_tx_frame(struct cpsw_priv *priv, struct xdp_frame *xdpf,
struct page *page, int port);
int cpsw_run_xdp(struct cpsw_priv *priv, int ch, struct xdp_buff *xdp,
struct page *page, int port);
irqreturn_t cpsw_tx_interrupt(int irq, void *dev_id);
irqreturn_t cpsw_rx_interrupt(int irq, void *dev_id);
int cpsw_tx_mq_poll(struct napi_struct *napi_tx, int budget);
int cpsw_tx_poll(struct napi_struct *napi_tx, int budget);
int cpsw_rx_mq_poll(struct napi_struct *napi_rx, int budget);
int cpsw_rx_poll(struct napi_struct *napi_rx, int budget);
void cpsw_rx_vlan_encap(struct sk_buff *skb);
void soft_reset(const char *module, void __iomem *reg);
void cpsw_set_slave_mac(struct cpsw_slave *slave, struct cpsw_priv *priv);
void cpsw_ndo_tx_timeout(struct net_device *ndev);
int cpsw_need_resplit(struct cpsw_common *cpsw);
int cpsw_ndo_ioctl(struct net_device *dev, struct ifreq *req, int cmd);
int cpsw_ndo_set_tx_maxrate(struct net_device *ndev, int queue, u32 rate);
int cpsw_ndo_setup_tc(struct net_device *ndev, enum tc_setup_type type,
void *type_data);
bool cpsw_shp_is_off(struct cpsw_priv *priv);
void cpsw_cbs_resume(struct cpsw_slave *slave, struct cpsw_priv *priv);
void cpsw_mqprio_resume(struct cpsw_slave *slave, struct cpsw_priv *priv);
/* ethtool */
u32 cpsw_get_msglevel(struct net_device *ndev);
......
drivers/net/ethernet/ti/cpsw_switchdev.c 0 → 100644
View file @ 1f12177b
// SPDX-License-Identifier: GPL-2.0
/*
* Texas Instruments switchdev Driver
*
* Copyright (C) 2019 Texas Instruments
*
*/
#include <linux/etherdevice.h>
#include <linux/if_bridge.h>
#include <linux/netdevice.h>
#include <linux/workqueue.h>
#include <net/switchdev.h>
#include "cpsw.h"
#include "cpsw_ale.h"
#include "cpsw_priv.h"
#include "cpsw_switchdev.h"
struct cpsw_switchdev_event_work {
struct work_struct work;
struct switchdev_notifier_fdb_info fdb_info;
struct cpsw_priv *priv;
unsigned long event;
};
static int cpsw_port_stp_state_set(struct cpsw_priv *priv,
struct switchdev_trans *trans, u8 state)
{
struct cpsw_common *cpsw = priv->cpsw;
u8 cpsw_state;
int ret = 0;
if (switchdev_trans_ph_prepare(trans))
return 0;
switch (state) {
case BR_STATE_FORWARDING:
cpsw_state = ALE_PORT_STATE_FORWARD;
break;
case BR_STATE_LEARNING:
cpsw_state = ALE_PORT_STATE_LEARN;
break;
case BR_STATE_DISABLED:
cpsw_state = ALE_PORT_STATE_DISABLE;
break;
case BR_STATE_LISTENING:
case BR_STATE_BLOCKING:
cpsw_state = ALE_PORT_STATE_BLOCK;
break;
default:
return -EOPNOTSUPP;
}
ret = cpsw_ale_control_set(cpsw->ale, priv->emac_port,
ALE_PORT_STATE, cpsw_state);
dev_dbg(priv->dev, "ale state: %u\n", cpsw_state);
return ret;
}
static int cpsw_port_attr_br_flags_set(struct cpsw_priv *priv,
struct switchdev_trans *trans,
struct net_device *orig_dev,
unsigned long brport_flags)
{
struct cpsw_common *cpsw = priv->cpsw;
bool unreg_mcast_add = false;
if (switchdev_trans_ph_prepare(trans))
return 0;
if (brport_flags & BR_MCAST_FLOOD)
unreg_mcast_add = true;
dev_dbg(priv->dev, "BR_MCAST_FLOOD: %d port %u\n",
unreg_mcast_add, priv->emac_port);
cpsw_ale_set_unreg_mcast(cpsw->ale, BIT(priv->emac_port),
unreg_mcast_add);
return 0;
}
static int cpsw_port_attr_br_flags_pre_set(struct net_device *netdev,
struct switchdev_trans *trans,
unsigned long flags)
{
if (flags & ~(BR_LEARNING | BR_MCAST_FLOOD))
return -EINVAL;
return 0;
}
static int cpsw_port_attr_set(struct net_device *ndev,
const struct switchdev_attr *attr,
struct switchdev_trans *trans)
{
struct cpsw_priv *priv = netdev_priv(ndev);
int ret;
dev_dbg(priv->dev, "attr: id %u port: %u\n", attr->id, priv->emac_port);
switch (attr->id) {
case SWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS:
ret = cpsw_port_attr_br_flags_pre_set(ndev, trans,
attr->u.brport_flags);
break;
case SWITCHDEV_ATTR_ID_PORT_STP_STATE:
ret = cpsw_port_stp_state_set(priv, trans, attr->u.stp_state);
dev_dbg(priv->dev, "stp state: %u\n", attr->u.stp_state);
break;
case SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS:
ret = cpsw_port_attr_br_flags_set(priv, trans, attr->orig_dev,
attr->u.brport_flags);
break;
default:
ret = -EOPNOTSUPP;
break;
}
return ret;
}
static u16 cpsw_get_pvid(struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
u32 __iomem *port_vlan_reg;
u32 pvid;
if (priv->emac_port) {
int reg = CPSW2_PORT_VLAN;
if (cpsw->version == CPSW_VERSION_1)
reg = CPSW1_PORT_VLAN;
pvid = slave_read(cpsw->slaves + (priv->emac_port - 1), reg);
} else {
port_vlan_reg = &cpsw->host_port_regs->port_vlan;
pvid = readl(port_vlan_reg);
}
pvid = pvid & 0xfff;
return pvid;
}
static void cpsw_set_pvid(struct cpsw_priv *priv, u16 vid, bool cfi, u32 cos)
{
struct cpsw_common *cpsw = priv->cpsw;
void __iomem *port_vlan_reg;
u32 pvid;
pvid = vid;
pvid |= cfi ? BIT(12) : 0;
pvid |= (cos & 0x7) << 13;
if (priv->emac_port) {
int reg = CPSW2_PORT_VLAN;
if (cpsw->version == CPSW_VERSION_1)
reg = CPSW1_PORT_VLAN;
/* no barrier */
slave_write(cpsw->slaves + (priv->emac_port - 1), pvid, reg);
} else {
/* CPU port */
port_vlan_reg = &cpsw->host_port_regs->port_vlan;
writel(pvid, port_vlan_reg);
}
}
static int cpsw_port_vlan_add(struct cpsw_priv *priv, bool untag, bool pvid,
u16 vid, struct net_device *orig_dev)
{
bool cpu_port = netif_is_bridge_master(orig_dev);
struct cpsw_common *cpsw = priv->cpsw;
int unreg_mcast_mask = 0;
int reg_mcast_mask = 0;
int untag_mask = 0;
int port_mask;
int ret = 0;
u32 flags;
if (cpu_port) {
port_mask = BIT(HOST_PORT_NUM);
flags = orig_dev->flags;
unreg_mcast_mask = port_mask;
} else {
port_mask = BIT(priv->emac_port);
flags = priv->ndev->flags;
}
if (flags & IFF_MULTICAST)
reg_mcast_mask = port_mask;
if (untag)
untag_mask = port_mask;
ret = cpsw_ale_vlan_add_modify(cpsw->ale, vid, port_mask, untag_mask,
reg_mcast_mask, unreg_mcast_mask);
if (ret) {
dev_err(priv->dev, "Unable to add vlan\n");
return ret;
}
if (cpu_port)
cpsw_ale_add_ucast(cpsw->ale, priv->mac_addr,
HOST_PORT_NUM, ALE_VLAN, vid);
if (!pvid)
return ret;
cpsw_set_pvid(priv, vid, 0, 0);
dev_dbg(priv->dev, "VID add: %s: vid:%u ports:%X\n",
priv->ndev->name, vid, port_mask);
return ret;
}
static int cpsw_port_vlan_del(struct cpsw_priv *priv, u16 vid,
struct net_device *orig_dev)
{
bool cpu_port = netif_is_bridge_master(orig_dev);
struct cpsw_common *cpsw = priv->cpsw;
int port_mask;
int ret = 0;
if (cpu_port)
port_mask = BIT(HOST_PORT_NUM);
else
port_mask = BIT(priv->emac_port);
ret = cpsw_ale_del_vlan(cpsw->ale, vid, port_mask);
if (ret != 0)
return ret;
/* We don't care for the return value here, error is returned only if
* the unicast entry is not present
*/
if (cpu_port)
cpsw_ale_del_ucast(cpsw->ale, priv->mac_addr,
HOST_PORT_NUM, ALE_VLAN, vid);
if (vid == cpsw_get_pvid(priv))
cpsw_set_pvid(priv, 0, 0, 0);
/* We don't care for the return value here, error is returned only if
* the multicast entry is not present
*/
cpsw_ale_del_mcast(cpsw->ale, priv->ndev->broadcast,
port_mask, ALE_VLAN, vid);
dev_dbg(priv->dev, "VID del: %s: vid:%u ports:%X\n",
priv->ndev->name, vid, port_mask);
return ret;
}
static int cpsw_port_vlans_add(struct cpsw_priv *priv,
const struct switchdev_obj_port_vlan *vlan,
struct switchdev_trans *trans)
{
bool untag = vlan->flags & BRIDGE_VLAN_INFO_UNTAGGED;
struct net_device *orig_dev = vlan->obj.orig_dev;
bool cpu_port = netif_is_bridge_master(orig_dev);
bool pvid = vlan->flags & BRIDGE_VLAN_INFO_PVID;
u16 vid;
dev_dbg(priv->dev, "VID add: %s: vid:%u flags:%X\n",
priv->ndev->name, vlan->vid_begin, vlan->flags);
if (cpu_port && !(vlan->flags & BRIDGE_VLAN_INFO_BRENTRY))
return 0;
if (switchdev_trans_ph_prepare(trans))
return 0;
for (vid = vlan->vid_begin; vid <= vlan->vid_end; vid++) {
int err;
err = cpsw_port_vlan_add(priv, untag, pvid, vid, orig_dev);
if (err)
return err;
}
return 0;
}
static int cpsw_port_vlans_del(struct cpsw_priv *priv,
const struct switchdev_obj_port_vlan *vlan)
{
struct net_device *orig_dev = vlan->obj.orig_dev;
u16 vid;
for (vid = vlan->vid_begin; vid <= vlan->vid_end; vid++) {
int err;
err = cpsw_port_vlan_del(priv, vid, orig_dev);
if (err)
return err;
}
return 0;
}
static int cpsw_port_mdb_add(struct cpsw_priv *priv,
struct switchdev_obj_port_mdb *mdb,
struct switchdev_trans *trans)
{
struct net_device *orig_dev = mdb->obj.orig_dev;
bool cpu_port = netif_is_bridge_master(orig_dev);
struct cpsw_common *cpsw = priv->cpsw;
int port_mask;
int err;
if (switchdev_trans_ph_prepare(trans))
return 0;
if (cpu_port)
port_mask = BIT(HOST_PORT_NUM);
else
port_mask = BIT(priv->emac_port);
err = cpsw_ale_add_mcast(cpsw->ale, mdb->addr, port_mask,
ALE_VLAN, mdb->vid, 0);
dev_dbg(priv->dev, "MDB add: %s: vid %u:%pM ports: %X\n",
priv->ndev->name, mdb->vid, mdb->addr, port_mask);
return err;
}
static int cpsw_port_mdb_del(struct cpsw_priv *priv,
struct switchdev_obj_port_mdb *mdb)
{
struct net_device *orig_dev = mdb->obj.orig_dev;
bool cpu_port = netif_is_bridge_master(orig_dev);
struct cpsw_common *cpsw = priv->cpsw;
int del_mask;
int err;
if (cpu_port)
del_mask = BIT(HOST_PORT_NUM);
else
del_mask = BIT(priv->emac_port);
err = cpsw_ale_del_mcast(cpsw->ale, mdb->addr, del_mask,
ALE_VLAN, mdb->vid);
dev_dbg(priv->dev, "MDB del: %s: vid %u:%pM ports: %X\n",
priv->ndev->name, mdb->vid, mdb->addr, del_mask);
return err;
}
static int cpsw_port_obj_add(struct net_device *ndev,
const struct switchdev_obj *obj,
struct switchdev_trans *trans,
struct netlink_ext_ack *extack)
{
struct switchdev_obj_port_vlan *vlan = SWITCHDEV_OBJ_PORT_VLAN(obj);
struct switchdev_obj_port_mdb *mdb = SWITCHDEV_OBJ_PORT_MDB(obj);
struct cpsw_priv *priv = netdev_priv(ndev);
int err = 0;
dev_dbg(priv->dev, "obj_add: id %u port: %u\n",
obj->id, priv->emac_port);
switch (obj->id) {
case SWITCHDEV_OBJ_ID_PORT_VLAN:
err = cpsw_port_vlans_add(priv, vlan, trans);
break;
case SWITCHDEV_OBJ_ID_PORT_MDB:
case SWITCHDEV_OBJ_ID_HOST_MDB:
err = cpsw_port_mdb_add(priv, mdb, trans);
break;
default:
err = -EOPNOTSUPP;
break;
}
return err;
}
static int cpsw_port_obj_del(struct net_device *ndev,
const struct switchdev_obj *obj)
{
struct switchdev_obj_port_vlan *vlan = SWITCHDEV_OBJ_PORT_VLAN(obj);
struct switchdev_obj_port_mdb *mdb = SWITCHDEV_OBJ_PORT_MDB(obj);
struct cpsw_priv *priv = netdev_priv(ndev);
int err = 0;
dev_dbg(priv->dev, "obj_del: id %u port: %u\n",
obj->id, priv->emac_port);
switch (obj->id) {
case SWITCHDEV_OBJ_ID_PORT_VLAN:
err = cpsw_port_vlans_del(priv, vlan);
break;
case SWITCHDEV_OBJ_ID_PORT_MDB:
case SWITCHDEV_OBJ_ID_HOST_MDB:
err = cpsw_port_mdb_del(priv, mdb);
break;
default:
err = -EOPNOTSUPP;
break;
}
return err;
}
static void cpsw_fdb_offload_notify(struct net_device *ndev,
struct switchdev_notifier_fdb_info *rcv)
{
struct switchdev_notifier_fdb_info info;
info.addr = rcv->addr;
info.vid = rcv->vid;
info.offloaded = true;
call_switchdev_notifiers(SWITCHDEV_FDB_OFFLOADED,
ndev, &info.info, NULL);
}
static void cpsw_switchdev_event_work(struct work_struct *work)
{
struct cpsw_switchdev_event_work *switchdev_work =
container_of(work, struct cpsw_switchdev_event_work, work);
struct cpsw_priv *priv = switchdev_work->priv;
struct switchdev_notifier_fdb_info *fdb;
struct cpsw_common *cpsw = priv->cpsw;
int port = priv->emac_port;
rtnl_lock();
switch (switchdev_work->event) {
case SWITCHDEV_FDB_ADD_TO_DEVICE:
fdb = &switchdev_work->fdb_info;
dev_dbg(cpsw->dev, "cpsw_fdb_add: MACID = %pM vid = %u flags = %u %u -- port %d\n",
fdb->addr, fdb->vid, fdb->added_by_user,
fdb->offloaded, port);
if (!fdb->added_by_user)
break;
if (memcmp(priv->mac_addr, (u8 *)fdb->addr, ETH_ALEN) == 0)
port = HOST_PORT_NUM;
cpsw_ale_add_ucast(cpsw->ale, (u8 *)fdb->addr, port,
fdb->vid ? ALE_VLAN : 0, fdb->vid);
cpsw_fdb_offload_notify(priv->ndev, fdb);
break;
case SWITCHDEV_FDB_DEL_TO_DEVICE:
fdb = &switchdev_work->fdb_info;
dev_dbg(cpsw->dev, "cpsw_fdb_del: MACID = %pM vid = %u flags = %u %u -- port %d\n",
fdb->addr, fdb->vid, fdb->added_by_user,
fdb->offloaded, port);
if (!fdb->added_by_user)
break;
if (memcmp(priv->mac_addr, (u8 *)fdb->addr, ETH_ALEN) == 0)
port = HOST_PORT_NUM;
cpsw_ale_del_ucast(cpsw->ale, (u8 *)fdb->addr, port,
fdb->vid ? ALE_VLAN : 0, fdb->vid);
break;
default:
break;
}
rtnl_unlock();
kfree(switchdev_work->fdb_info.addr);
kfree(switchdev_work);
dev_put(priv->ndev);
}
/* called under rcu_read_lock() */
static int cpsw_switchdev_event(struct notifier_block *unused,
unsigned long event, void *ptr)
{
struct net_device *ndev = switchdev_notifier_info_to_dev(ptr);
struct switchdev_notifier_fdb_info *fdb_info = ptr;
struct cpsw_switchdev_event_work *switchdev_work;
struct cpsw_priv *priv = netdev_priv(ndev);
int err;
if (event == SWITCHDEV_PORT_ATTR_SET) {
err = switchdev_handle_port_attr_set(ndev, ptr,
cpsw_port_dev_check,
cpsw_port_attr_set);
return notifier_from_errno(err);
}
if (!cpsw_port_dev_check(ndev))
return NOTIFY_DONE;
switchdev_work = kzalloc(sizeof(*switchdev_work), GFP_ATOMIC);
if (WARN_ON(!switchdev_work))
return NOTIFY_BAD;
INIT_WORK(&switchdev_work->work, cpsw_switchdev_event_work);
switchdev_work->priv = priv;
switchdev_work->event = event;
switch (event) {
case SWITCHDEV_FDB_ADD_TO_DEVICE:
case SWITCHDEV_FDB_DEL_TO_DEVICE:
memcpy(&switchdev_work->fdb_info, ptr,
sizeof(switchdev_work->fdb_info));
switchdev_work->fdb_info.addr = kzalloc(ETH_ALEN, GFP_ATOMIC);
if (!switchdev_work->fdb_info.addr)
goto err_addr_alloc;
ether_addr_copy((u8 *)switchdev_work->fdb_info.addr,
fdb_info->addr);
dev_hold(ndev);
break;
default:
kfree(switchdev_work);
return NOTIFY_DONE;
}
queue_work(system_long_wq, &switchdev_work->work);
return NOTIFY_DONE;
err_addr_alloc:
kfree(switchdev_work);
return NOTIFY_BAD;
}
static struct notifier_block cpsw_switchdev_notifier = {
.notifier_call = cpsw_switchdev_event,
};
static int cpsw_switchdev_blocking_event(struct notifier_block *unused,
unsigned long event, void *ptr)
{
struct net_device *dev = switchdev_notifier_info_to_dev(ptr);
int err;
switch (event) {
case SWITCHDEV_PORT_OBJ_ADD:
err = switchdev_handle_port_obj_add(dev, ptr,
cpsw_port_dev_check,
cpsw_port_obj_add);
return notifier_from_errno(err);
case SWITCHDEV_PORT_OBJ_DEL:
err = switchdev_handle_port_obj_del(dev, ptr,
cpsw_port_dev_check,
cpsw_port_obj_del);
return notifier_from_errno(err);
case SWITCHDEV_PORT_ATTR_SET:
err = switchdev_handle_port_attr_set(dev, ptr,
cpsw_port_dev_check,
cpsw_port_attr_set);
return notifier_from_errno(err);
default:
break;
}
return NOTIFY_DONE;
}
static struct notifier_block cpsw_switchdev_bl_notifier = {
.notifier_call = cpsw_switchdev_blocking_event,
};
int cpsw_switchdev_register_notifiers(struct cpsw_common *cpsw)
{
int ret = 0;
ret = register_switchdev_notifier(&cpsw_switchdev_notifier);
if (ret) {
dev_err(cpsw->dev, "register switchdev notifier fail ret:%d\n",
ret);
return ret;
}
ret = register_switchdev_blocking_notifier(&cpsw_switchdev_bl_notifier);
if (ret) {
dev_err(cpsw->dev, "register switchdev blocking notifier ret:%d\n",
ret);
unregister_switchdev_notifier(&cpsw_switchdev_notifier);
}
return ret;
}
void cpsw_switchdev_unregister_notifiers(struct cpsw_common *cpsw)
{
unregister_switchdev_blocking_notifier(&cpsw_switchdev_bl_notifier);
unregister_switchdev_notifier(&cpsw_switchdev_notifier);
}
drivers/net/ethernet/ti/cpsw_switchdev.h 0 → 100644
View file @ 1f12177b
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Texas Instruments Ethernet Switch Driver
*/
#ifndef DRIVERS_NET_ETHERNET_TI_CPSW_SWITCHDEV_H_
#define DRIVERS_NET_ETHERNET_TI_CPSW_SWITCHDEV_H_
#include <net/switchdev.h>
bool cpsw_port_dev_check(const struct net_device *dev);
int cpsw_switchdev_register_notifiers(struct cpsw_common *cpsw);
void cpsw_switchdev_unregister_notifiers(struct cpsw_common *cpsw);
#endif /* DRIVERS_NET_ETHERNET_TI_CPSW_SWITCHDEV_H_ */
drivers/phy/ti/Kconfig
View file @ 1f12177b
......@@ -90,8 +90,8 @@ config TWL4030_USB
config PHY_TI_GMII_SEL
tristate
default y if TI_CPSW=y
depends on TI_CPSW || COMPILE_TEST
default y if TI_CPSW=y || TI_CPSW_SWITCHDEV=y
depends on TI_CPSW || TI_CPSW_SWITCHDEV || COMPILE_TEST
select GENERIC_PHY
select REGMAP
default m
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7