Various hardware statistics that are available for Siena are
unavailable or meaningless for Falcon.  Huntington adds further to the
NIC-type-specific statistics, as it has different MAC blocks from
Falcon/Siena.

All NIC types still provide most statistics by DMA, and use
little-endian byte order.

Therefore:
1. Add some general utility functions for reporting hardware statistics,
   efx_nic_describe_stats() and efx_nic_update_stats().
2. Add an efx_nic_type::describe_stats operation to get the number and
   names of statistics, implemented using efx_nic_describe_stats()
3. Change efx_nic_type::update_stats to store the core statistics
   (struct rtnl_link_stats64) or full statistics (array of u64) in a
   caller-provided buffer.  Use efx_nic_update_stats() to aid in the
   implementation.
4. Rename struct efx_ethtool_stat to struct efx_sw_stat_desc and
   EFX_ETHTOOL_NUM_STATS to EFX_ETHTOOL_SW_STAT_COUNT.
5. Remove efx_nic::mac_stats and struct efx_mac_stats.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

It's not really helpful to pretend ethtool string arrays are
structured.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

We only ever used the XMAC (10G link speed) in production.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Merge the per-NIC-type MTD probe selection and struct efx_mtd_ops into
struct efx_nic_type.  Move the implementations into the appropriate
source files.

Several NVRAM functions are now only called from MTD operations which
are now implemented in the same file (falcon.c or mcdi.c).  There is no
need for them to be extern, or to be defined at all if CONFIG_SFC_MTD
is not enabled, so move them into the #ifdef CONFIG_SFC_MTD sections
in those files.

Most of the SPI-related definitions are also only used in falcon.c,
so move them there.  Put the remainder of spi.h into nic.h (which
previously included it).
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Currently we use struct efx_mtd to represent a physical NVRAM device
and struct efx_mtd_partition to represent a partition on that device.
But this only really makes sense for Falcon, as we don't know or care
whether MC-managed NVRAM partitions are on one or more physical
devices.  It complicates iteration and provides little benefit.
Therefore:

- Replace the pointer to efx_mtd in mtd_info::priv with a pointer to efx_nic
- Move the falcon_spi_device pointer into the union in struct efx_mtd_partition
- Move the device name to efx_mtd_partition::dev_type_name
- Move the efx_mtd_ops pointer to efx_nic::mtd_ops
- Make efx_nic::mtd_list a list of partitions
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

On Falcon we implement MAC filtering requested by the stack using the
MAC wrapper's single unicast filter and multicast hash filter.  Siena
is very similar, though MAC configuration is mediated by the MC.

Since MCDI operations may sleep, reconfiguration is deferred from
ndo_set_rx_mode to a work item.  However, it still updates the private
variables describing the filter state synchronously.  Contrary to
comments, the later use of these variables is not protected using the
address lock, resulting in race conditions.

Move the state update to a new function
efx_farch_filter_sync_rx_mode() and make the Falcon-arch MAC
configuration functions call that, so that its use is consistently
serialised by the mac_lock.

Invert and rename the promiscuous flag to the more accurate
unicast_filter, and comment that both this and multicast_hash are
not used on EF10.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

There is only one user now, but we're about to add many more.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

MAC filters inserted on request from the stack (ndo_set_rx_mode)
should allow manual steering but not removal.  Currently we have a
special case for Siena's all-multicast and all-unicast MAC filters,
but on EF10 we need to allow for steering of precise MAC filters as
well.

The EFX_FILTER_FLAG_RX_STACK flag changes the behaviour of replacement
and removal requests:

- Replacement *of* a filter with this flag never clears the flag but
  does change steering and saved priority
- Replacement *by* a filter with this flag only sets the flag but does
  not change steering
- Removal with priority < EFX_FILTER_PRI_REQUIRED really resets RX
  steering and saved priority

This could support precise MAC filtering on Siena in future.

As a side-benefit, the default MAC filters are hidden from ethtool
until they are steered.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Move the special case for removal of default filters from
efx_farch_filter_table_clear_entry() into a wrapper function,
efx_farch_filter_table_remove().  Move the existence and priority
checks into the latter and use it where appropriate.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Aside from accelerated RFS, there is almost nothing that can be shared
between the filter table implementations for the Falcon architecture
and EF10.

Move the few shared functions into efx.c and rx.c and the rest into
farch.c.  Introduce efx_nic_type operations for the implementation and
inline wrapper functions that call these.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Currently every call to efx_farch_filter_table_clear_entry() is
shortly followed by a conditional reset of the table limits.  The new
limits (0) are not pushed to hardware until the next filter insertion.
Move both the reset and the hardware reconfiguration into
efx_farch_filter_table_clear_entry(), and add an explanatory comment.

Also, make consistent use of the term 'search limit' for the maximum
number of probes the NIC must make when searching for a filter of a
particular type.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Move the common state from struct efx_filter_state into struct efx_nic.
Rename struct efx_filter_state to efx_farch_filter_state and change
the type of efx_nic::filter_state to void *.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Replace type field with match_flags.  Add rss_context and match values
covering of most of what is now in the MCDI protocol.

Change some fields into bitfields so that the structure size doesn't grow
beyond 64 bytes.

Ditch the filter decoding functions as it is now easier to pick apart
the abstract structure.

Rewrite ethtool NFC rule functions to set/get filter match flags and
values directly.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

The filter table(s) on EF10 are managed by firmware and will need
almost entirely separate code.  Rename the types and functions used
within the existing implementation.  The current definition of struct
efx_filter_spec is really implementation-specific, so we need to keep
it.  For now, define a separate structure for the internal
representation but leave them identical.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

efx_fini_eventq() needs to be idempotent but EF10 firmware is
picky about queue states.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

The workarounds that currently use EFX_WORKAROUND_ALWAYS are in
Falcon-specific or Falcon-arch-specific code, so get rid of the
conditions altogether.  Add/move comments as appropriate.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

EF10 functions don't have a fixed BAR size, and the minimum is not
large enough for all the queues we might want to allocate.  We have to
find out the BAR size at run-time, and therefore phys_addr_channels
and mem_map_size cannot be defined per-NIC-type.

Change efx_nic_type::mem_map_size to a function pointer which is
called to find the wanted memory map size (before probe).

Replace efx_nic_type::phys_addr_channels with efx_nic::max_channels,
to be initialised by the probe function.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

When we poll for MCDI request completion, we don't hold the interface
lock while setting the response fields in struct efx_mcdi_iface.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

MCDI v2 adds a second header dword with wider command and length
fields.  It also defines extra error codes.

Change the fallback error number for unknown MCDI error codes from EIO
to EPROTO.  EIO is treated as indicating the MCDI transport has failed
and we need to reset the function, which is rather drastic.

v2 error codes and lengths don't fit into completion events, so for a
v2-capable transport, always read the response header rather then
using the event fields.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

EF10 controllers do not have shared memory for communication with the
MC; instead it reads requests and writes responses in host memory,
which allows for longer messages.  It is also responsible for all
datapath control operations and hardware resource allocation, which
requires a large number of new commands and adds more possible error
cases.  MCDI v2 extends the message header to support this.

Update the MCDI protocol definition header to include v2 lengths,
errors and messages, and a few definitions specific to the
SFC9100 family (codenames Farmingdale and Huntington) which is
the first generation of EF10.

Some messages have been extended, so adjust the code accordingly:
- The request for MC_CMD_DRV_ATTACH now includes a datapath firmware
  ID.  This is ignored by Siena but we should fill it in anyway,
  initially always specifying low-latency datapath.
- The response for MC_CMD_GET_LOOPBACK_MODES now includes a 40G
  field.  Accept shorter responses that don't include it.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Currently we only translate error codes in efx_mcdi_poll(), but we
also need to do so in efx_mcdi_ev_cpl().

The reason we didn't notice before is that the MC firmware error codes
are mostly taken from Unix/Linux and no translation is necessary on
most architectures.  Make sure we notice any future failure by
changing the sign of resprc (matching the kernel convention) and BUG
if it's ever positive at command completion.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Add efx_nic_type operations for the many efx_nic functions that need
to be implemented different on EF10.  For now, change most of the
existing efx_nic_*() functions into inline wrappers.  As a later step,
we may be able to improve branch prediction for operations used on the
fast path by copying the pointers into each queue/channel structure.

Move the Falcon/Siena implementations to new file farch.c and rename
the functions and static data to use a prefix of 'efx_farch_'.

Move efx_may_push_tx_desc() to nic.h, as the EF10 TX code will also
use it.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Currently efx_stop_datapath() will try to flush our DMA queues (if DMA
is enabled), then finalise software and hardware state for each queue.
However, for EF10 we must ask the MC to finalise each queue, which
implicitly starts flushing it, and then wait for the flush events.
We therefore need to delegate more of this to the NIC type.

Combine all the hardware operations into a new NIC-type operation
efx_nic_type::fini_dmaq, and call this before tearing down the
software state and buffers for all the DMA queues.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

efx_unregister_netdev() should not call efx_release_tx_buffers()
directly, as it is already done when closing the device:
efx_net_stop() -> efx_stop_all() -> efx_stop_datapath() ->
efx_fini_tx_queue() -> efx_release_tx_buffers().

(This was presumably a workaround for a race between efx_stop_all()
and the data path that has since been properly fixed.)
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

rx_queue::enabled guards refill, so rename it to reflect that.  Clear
it at the start of the queue teardown process rather than waiting for
the RX queue to be flushed.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

We unconditionally acknowledge legacy interrupts just before disabling
them.  This workaround is needed on Falcon A1 but probably not on
later chips where the legacy interrupt mechanism is different.  It was
also originally done after the IRQ handler was removed, not before.
Restore the original behaviour for Falcon A1 only by doing this
acknowledgement in the efx_nic_type::fini operation.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

There are many problems with the current efx_stop_interrupts() and
efx_start_interrupts():

1. On Siena, it is unsafe to disable the master IRQ enable bit
(DRV_INT_EN_KER) while any IRQ sources are enabled.

2. On EF10 there is no master IRQ enable bit, so we cannot expect to
defer IRQs without tearing down event queues.  (Though I don't think
we will need to keep any event queues around while the device is down,
as we do for VFDI on Siena.)

3. synchronize_irq() only waits for a running IRQ handler to finish,
not for any propagation through IRQ controllers.  Therefore an IRQ may
still be received and handled after efx_stop_interrupts() returns.
IRQ handlers can then race with channel reallocation.

To fix this:

a. Introduce a software IRQ enable flag.  So long as this is clear,
IRQ handlers will only acknowledge IRQs and not touch the channel
structures.

b. Define a new struct efx_msi_context as the context for MSIs.  This
is never reallocated and is sufficient to find the software enable
flag and the channel structure.  It also includes the channel/IRQ
name, which was previously separated out as it must also not be
reallocated.

c. Split efx_{start,stop}_interrupts() into
efx_{,soft_}_{enable,disable}_interrupts().  The 'soft' functions
don't touch the hardware master enable flag (if it exists) and don't
reinitialise or tear down channels with the keep_eventq flag set.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

efx_process_channel_now() is unneeded since self-tests can rely on
normal NAPI polling.  Remove it and all calls to it.

efx_channel::work_pending and efx_channel_processed() are also
unneeded (the latter being the same as efx_nic_eventq_read_ack()).
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

The EF10 architecture has a very different register layout from
previous controllers, so we'll use separate files for the two sets of
register definitions.  Use 'farch' as an abbreviation for
Falcon-architecture.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

On EF10, the firmware is in charge of allocating buffer table entries.
Change struct efx_special_buffer to use a struct efx_buffer member,
so that it can be used with efx_nic_{alloc,free}_buffer() in that
case.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Most call sites for efx_nic_alloc_buffer() are part of the probe or
reconfiguration paths and can allocate with GFP_KERNEL.  A few others
should use GFP_NOIO (I think).  Only one is in atomic context and
must use the current GFP_ATOMIC.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Move the lowest layer (transport) of the current MCDI code to
per-NIC-type operations.

Introduce a new structure and efx_nic member for MCDI-specific data.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

This should probably be done during MCDI initialisation for any NIC.
Change efx_mcdi_init() to return an error code.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

Collect together MCDI port functions from mcdi.c, mcdi_mac.c,
mcdi_phy.c and siena.c.  Rename the 'siena' functions accordingly.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

EF10 does not include a multicast hash filter, so this function is
specific to Siena.
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>