• Netanel Belgazal's avatar
    net: ena: fix race condition between submit and completion admin command · 661d2b0c
    Netanel Belgazal authored
    Bug:
    "Completion context is occupied" error printout will be noticed in
    dmesg.
    This error will cause the admin command to fail, which will lead to
    an ena_probe() failure or a watchdog reset (depends on which admin
    command failed).
    
    Root cause:
    __ena_com_submit_admin_cmd() is the function that submits new entries to
    the admin queue.
    The function have a check that makes sure the queue is not full and the
    function does not override any outstanding command.
    It uses head and tail indexes for this check.
    The head is increased by ena_com_handle_admin_completion() which runs
    from interrupt context, and the tail index is increased by the submit
    function (the function is running under ->q_lock, so there is no risk
    of multithread increment).
    Each command is associated with a completion context. This context
    allocated before call to __ena_com_submit_admin_cmd() and freed by
    ena_com_wait_and_process_admin_cq_interrupts(), right after the command
    was completed.
    
    This can lead to a state where the head was increased, the check passed,
    but the completion context is still in use.
    
    Solution:
    Use the atomic variable ->outstanding_cmds instead of using the head and
    the tail indexes.
    This variable is safe for use since it is bumped in get_comp_ctx() in
    __ena_com_submit_admin_cmd() and is freed by comp_ctxt_release()
    
    Fixes: 1738cd3e ("Add a driver for Amazon Elastic Network Adapters (ENA)")
    Signed-off-by: default avatarNetanel Belgazal <netanel@amazon.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    661d2b0c
ena_com.c 72.3 KB