• Paolo Abeni's avatar
    sched: manipulate __QDISC_STATE_RUNNING in qdisc_run_* helpers · 32f7b44d
    Paolo Abeni authored
    Currently NOLOCK qdiscs pay a measurable overhead to atomically
    manipulate the __QDISC_STATE_RUNNING. Such bit is flipped twice per
    packet in the uncontended scenario with packet rate below the
    line rate: on packed dequeue and on the next, failing dequeue attempt.
    
    This changeset moves the bit manipulation into the qdisc_run_{begin,end}
    helpers, so that the bit is now flipped only once per packet, with
    measurable performance improvement in the uncontended scenario.
    
    This also allows simplifying the qdisc teardown code path - since
    qdisc_is_running() is now effective for each qdisc type - and avoid a
    possible race between qdisc_run() and dev_deactivate_many(), as now
    the some_qdisc_is_busy() can properly detect NOLOCK qdiscs being busy
    dequeuing packets.
    Signed-off-by: default avatarPaolo Abeni <pabeni@redhat.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    32f7b44d
dev.c 228 KB