• Toke Høiland-Jørgensen's avatar
    xdp: Move devmap bulk queue into struct net_device · 75ccae62
    Toke Høiland-Jørgensen authored
    Commit 96360004 ("xdp: Make devmap flush_list common for all map
    instances"), changed devmap flushing to be a global operation instead of a
    per-map operation. However, the queue structure used for bulking was still
    allocated as part of the containing map.
    
    This patch moves the devmap bulk queue into struct net_device. The
    motivation for this is reusing it for the non-map variant of XDP_REDIRECT,
    which will be changed in a subsequent commit.  To avoid other fields of
    struct net_device moving to different cache lines, we also move a couple of
    other members around.
    
    We defer the actual allocation of the bulk queue structure until the
    NETDEV_REGISTER notification devmap.c. This makes it possible to check for
    ndo_xdp_xmit support before allocating the structure, which is not possible
    at the time struct net_device is allocated. However, we keep the freeing in
    free_netdev() to avoid adding another RCU callback on NETDEV_UNREGISTER.
    
    Because of this change, we lose the reference back to the map that
    originated the redirect, so change the tracepoint to always return 0 as the
    map ID and index. Otherwise no functional change is intended with this
    patch.
    
    After this patch, the relevant part of struct net_device looks like this,
    according to pahole:
    
    	/* --- cacheline 14 boundary (896 bytes) --- */
    	struct netdev_queue *      _tx __attribute__((__aligned__(64))); /*   896     8 */
    	unsigned int               num_tx_queues;        /*   904     4 */
    	unsigned int               real_num_tx_queues;   /*   908     4 */
    	struct Qdisc *             qdisc;                /*   912     8 */
    	unsigned int               tx_queue_len;         /*   920     4 */
    	spinlock_t                 tx_global_lock;       /*   924     4 */
    	struct xdp_dev_bulk_queue * xdp_bulkq;           /*   928     8 */
    	struct xps_dev_maps *      xps_cpus_map;         /*   936     8 */
    	struct xps_dev_maps *      xps_rxqs_map;         /*   944     8 */
    	struct mini_Qdisc *        miniq_egress;         /*   952     8 */
    	/* --- cacheline 15 boundary (960 bytes) --- */
    	struct hlist_head  qdisc_hash[16];               /*   960   128 */
    	/* --- cacheline 17 boundary (1088 bytes) --- */
    	struct timer_list  watchdog_timer;               /*  1088    40 */
    
    	/* XXX last struct has 4 bytes of padding */
    
    	int                        watchdog_timeo;       /*  1128     4 */
    
    	/* XXX 4 bytes hole, try to pack */
    
    	struct list_head   todo_list;                    /*  1136    16 */
    	/* --- cacheline 18 boundary (1152 bytes) --- */
    Signed-off-by: default avatarToke Høiland-Jørgensen <toke@redhat.com>
    Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
    Acked-by: default avatarBjörn Töpel <bjorn.topel@intel.com>
    Acked-by: default avatarJohn Fastabend <john.fastabend@gmail.com>
    Link: https://lore.kernel.org/bpf/157918768397.1458396.12673224324627072349.stgit@toke.dk
    75ccae62
dev.c 260 KB