• Maciej Fijalkowski's avatar
    xsk: make xsk_buff_pool responsible for clearing xdp_buff::flags · f7f6aa8e
    Maciej Fijalkowski authored
    XDP multi-buffer support introduced XDP_FLAGS_HAS_FRAGS flag that is
    used by drivers to notify data path whether xdp_buff contains fragments
    or not. Data path looks up mentioned flag on first buffer that occupies
    the linear part of xdp_buff, so drivers only modify it there. This is
    sufficient for SKB and XDP_DRV modes as usually xdp_buff is allocated on
    stack or it resides within struct representing driver's queue and
    fragments are carried via skb_frag_t structs. IOW, we are dealing with
    only one xdp_buff.
    
    ZC mode though relies on list of xdp_buff structs that is carried via
    xsk_buff_pool::xskb_list, so ZC data path has to make sure that
    fragments do *not* have XDP_FLAGS_HAS_FRAGS set. Otherwise,
    xsk_buff_free() could misbehave if it would be executed against xdp_buff
    that carries a frag with XDP_FLAGS_HAS_FRAGS flag set. Such scenario can
    take place when within supplied XDP program bpf_xdp_adjust_tail() is
    used with negative offset that would in turn release the tail fragment
    from multi-buffer frame.
    
    Calling xsk_buff_free() on tail fragment with XDP_FLAGS_HAS_FRAGS would
    result in releasing all the nodes from xskb_list that were produced by
    driver before XDP program execution, which is not what is intended -
    only tail fragment should be deleted from xskb_list and then it should
    be put onto xsk_buff_pool::free_list. Such multi-buffer frame will never
    make it up to user space, so from AF_XDP application POV there would be
    no traffic running, however due to free_list getting constantly new
    nodes, driver will be able to feed HW Rx queue with recycled buffers.
    Bottom line is that instead of traffic being redirected to user space,
    it would be continuously dropped.
    
    To fix this, let us clear the mentioned flag on xsk_buff_pool side
    during xdp_buff initialization, which is what should have been done
    right from the start of XSK multi-buffer support.
    
    Fixes: 1bbc04de ("ice: xsk: add RX multi-buffer support")
    Fixes: 1c9ba9c1 ("i40e: xsk: add RX multi-buffer support")
    Fixes: 24ea5012 ("xsk: support mbuf on ZC RX")
    Signed-off-by: default avatarMaciej Fijalkowski <maciej.fijalkowski@intel.com>
    Link: https://lore.kernel.org/r/20240124191602.566724-3-maciej.fijalkowski@intel.comSigned-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
    f7f6aa8e
xdp_sock_drv.h 8.71 KB