Ocelot chips (VSC7511, VSC7512, VSC7513, VSC7514) don't support bulk read
operations over SPI.

Many SPI buses have hardware that can optimize consecutive reads.
Essentially an address is written to the chip, and if the SPI controller
continues to toggle the clock, subsequent register values are reported.
This can lead to significant optimizations, because the time between
"address is written to the chip" and "chip starts to report data" can often
take a fixed amount of time.

When support for Ocelot chips were added in commit f3e89362 ("mfd:
ocelot: Add support for the vsc7512 chip via spi") it was believed that
this optimization was supported. However it is not.

Most register transactions with the Ocelot chips are not done in bulk, so
this bug could go unnoticed. The one scenario where bulk register
operations _are_ performed is when polling port statistics counters, which
was added in commit d87b1c08 ("net: mscc: ocelot: use bulk reads for
stats").

Things get slightly more complicated here...

A bug was introduced in commit d4c36765 ("net: mscc: ocelot: keep
ocelot_stat_layout by reg address, not offset") that broke the optimization
of bulk reads. This means that when Ethernet support for the VSC7512 chip
was added in commit 3d7316ac ("net: dsa: ocelot: add external ocelot
switch control") things were actually working "as expected".

The bulk read opmtimization was discovered, and fixed in commit
6acc72a4 ("net: mscc: ocelot: fix stats region batching") and the
timing optimizations for SPI were noticed. A bulk read went from ~14ms to
~2ms. But this timing improvement came at the cost of every register
reading zero due the fact that bulk reads don't work.

The read timings increase back to 13-14ms, but that's a price worth paying
in order to receive valid data. This is verified in a DSA setup (cpsw-new
switch tied to port 0 on the VSC7512, after having been running overnight)

     Rx Octets: 16222055 # Counters from CPSW switch
     Tx Octets: 12034702
     Net Octets: 28256757
     p00_rx_octets: 12034702 # Counters from Ocelot switch
     p00_rx_frames_below_65_octets: 0
     p00_rx_frames_65_to_127_octets: 88188
     p00_rx_frames_128_to_255_octets: 13
     p00_rx_frames_256_to_511_octets: 0
     p00_rx_frames_512_to_1023_octets: 0
     p00_rx_frames_over_1526_octets: 3306
     p00_tx_octets: 16222055

Fixes: f3e89362 ("mfd: ocelot: Add support for the vsc7512 chip via spi")
Signed-off-by: Colin Foster <colin.foster@in-advantage.com>
Signed-off-by: Lee Jones <lee@kernel.org>
Link: https://lore.kernel.org/r/20230322141130.2531256-1-colin.foster@in-advantage.com