Commit 50180074 authored by Jakub Kicinski's avatar Jakub Kicinski Committed by David S. Miller

net/tls: add kernel-driven resync mechanism for TX

TLS offload drivers keep track of TCP seq numbers to make sure
the packets are fed into the HW in order.

When packets get dropped on the way through the stack, the driver
will get out of sync and have to use fallback encryption, but unless
TCP seq number is resynced it will never match the packets correctly
(or even worse - use incorrect record sequence number after TCP seq
wraps).

Existing drivers (mlx5) feed the entire record on every out-of-order
event, allowing FW/HW to always be in sync.

This patch adds an alternative, more akin to the RX resync.  When
driver sees a frame which is past its expected sequence number the
stream must have gotten out of order (if the sequence number is
smaller than expected its likely a retransmission which doesn't
require resync).  Driver will ask the stack to perform TX sync
before it submits the next full record, and fall back to software
crypto until stack has performed the sync.
Signed-off-by: default avatarJakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: default avatarDirk van der Merwe <dirk.vandermerwe@netronome.com>
Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parent eeb2efaf
...@@ -206,7 +206,11 @@ TX ...@@ -206,7 +206,11 @@ TX
Segments transmitted from an offloaded socket can get out of sync Segments transmitted from an offloaded socket can get out of sync
in similar ways to the receive side-retransmissions - local drops in similar ways to the receive side-retransmissions - local drops
are possible, though network reorders are not. are possible, though network reorders are not. There are currently
two mechanisms for dealing with out of order segments.
Crypto state rebuilding
~~~~~~~~~~~~~~~~~~~~~~~
Whenever an out of order segment is transmitted the driver provides Whenever an out of order segment is transmitted the driver provides
the device with enough information to perform cryptographic operations. the device with enough information to perform cryptographic operations.
...@@ -225,6 +229,35 @@ was just a retransmission. The former is simpler, and does not require ...@@ -225,6 +229,35 @@ was just a retransmission. The former is simpler, and does not require
retransmission detection therefore it is the recommended method until retransmission detection therefore it is the recommended method until
such time it is proven inefficient. such time it is proven inefficient.
Next record sync
~~~~~~~~~~~~~~~~
Whenever an out of order segment is detected the driver requests
that the ``ktls`` software fallback code encrypt it. If the segment's
sequence number is lower than expected the driver assumes retransmission
and doesn't change device state. If the segment is in the future, it
may imply a local drop, the driver asks the stack to sync the device
to the next record state and falls back to software.
Resync request is indicated with:
.. code-block:: c
void tls_offload_tx_resync_request(struct sock *sk, u32 got_seq, u32 exp_seq)
Until resync is complete driver should not access its expected TCP
sequence number (as it will be updated from a different context).
Following helper should be used to test if resync is complete:
.. code-block:: c
bool tls_offload_tx_resync_pending(struct sock *sk)
Next time ``ktls`` pushes a record it will first send its TCP sequence number
and TLS record number to the driver. Stack will also make sure that
the new record will start on a segment boundary (like it does when
the connection is initially added).
RX RX
-- --
......
...@@ -212,6 +212,11 @@ struct tls_offload_context_tx { ...@@ -212,6 +212,11 @@ struct tls_offload_context_tx {
enum tls_context_flags { enum tls_context_flags {
TLS_RX_SYNC_RUNNING = 0, TLS_RX_SYNC_RUNNING = 0,
/* Unlike RX where resync is driven entirely by the core in TX only
* the driver knows when things went out of sync, so we need the flag
* to be atomic.
*/
TLS_TX_SYNC_SCHED = 1,
}; };
struct cipher_context { struct cipher_context {
...@@ -619,6 +624,24 @@ tls_offload_rx_resync_set_type(struct sock *sk, enum tls_offload_sync_type type) ...@@ -619,6 +624,24 @@ tls_offload_rx_resync_set_type(struct sock *sk, enum tls_offload_sync_type type)
tls_offload_ctx_rx(tls_ctx)->resync_type = type; tls_offload_ctx_rx(tls_ctx)->resync_type = type;
} }
static inline void tls_offload_tx_resync_request(struct sock *sk)
{
struct tls_context *tls_ctx = tls_get_ctx(sk);
WARN_ON(test_and_set_bit(TLS_TX_SYNC_SCHED, &tls_ctx->flags));
}
/* Driver's seq tracking has to be disabled until resync succeeded */
static inline bool tls_offload_tx_resync_pending(struct sock *sk)
{
struct tls_context *tls_ctx = tls_get_ctx(sk);
bool ret;
ret = test_bit(TLS_TX_SYNC_SCHED, &tls_ctx->flags);
smp_mb__after_atomic();
return ret;
}
int tls_proccess_cmsg(struct sock *sk, struct msghdr *msg, int tls_proccess_cmsg(struct sock *sk, struct msghdr *msg,
unsigned char *record_type); unsigned char *record_type);
void tls_register_device(struct tls_device *device); void tls_register_device(struct tls_device *device);
......
...@@ -209,6 +209,29 @@ void tls_device_free_resources_tx(struct sock *sk) ...@@ -209,6 +209,29 @@ void tls_device_free_resources_tx(struct sock *sk)
tls_free_partial_record(sk, tls_ctx); tls_free_partial_record(sk, tls_ctx);
} }
static void tls_device_resync_tx(struct sock *sk, struct tls_context *tls_ctx,
u32 seq)
{
struct net_device *netdev;
struct sk_buff *skb;
u8 *rcd_sn;
skb = tcp_write_queue_tail(sk);
if (skb)
TCP_SKB_CB(skb)->eor = 1;
rcd_sn = tls_ctx->tx.rec_seq;
down_read(&device_offload_lock);
netdev = tls_ctx->netdev;
if (netdev)
netdev->tlsdev_ops->tls_dev_resync(netdev, sk, seq, rcd_sn,
TLS_OFFLOAD_CTX_DIR_TX);
up_read(&device_offload_lock);
clear_bit_unlock(TLS_TX_SYNC_SCHED, &tls_ctx->flags);
}
static void tls_append_frag(struct tls_record_info *record, static void tls_append_frag(struct tls_record_info *record,
struct page_frag *pfrag, struct page_frag *pfrag,
int size) int size)
...@@ -264,6 +287,10 @@ static int tls_push_record(struct sock *sk, ...@@ -264,6 +287,10 @@ static int tls_push_record(struct sock *sk,
list_add_tail(&record->list, &offload_ctx->records_list); list_add_tail(&record->list, &offload_ctx->records_list);
spin_unlock_irq(&offload_ctx->lock); spin_unlock_irq(&offload_ctx->lock);
offload_ctx->open_record = NULL; offload_ctx->open_record = NULL;
if (test_bit(TLS_TX_SYNC_SCHED, &ctx->flags))
tls_device_resync_tx(sk, ctx, tp->write_seq);
tls_advance_record_sn(sk, prot, &ctx->tx); tls_advance_record_sn(sk, prot, &ctx->tx);
for (i = 0; i < record->num_frags; i++) { for (i = 0; i < record->num_frags; i++) {
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment