• John Ogness's avatar
    printk: ringbuffer: Do not skip non-finalized records with prb_next_seq() · 5f72e52b
    John Ogness authored
    Commit f244b4dc ("printk: ringbuffer: Improve
    prb_next_seq() performance") introduced an optimization for
    prb_next_seq() by using best-effort to track recently finalized
    records. However, the order of finalization does not
    necessarily match the order of the records. The optimization
    changed prb_next_seq() to return inconsistent results, possibly
    yielding sequence numbers that are not available to readers
    because they are preceded by non-finalized records or they are
    not yet visible to the reader CPU.
    
    Rather than simply best-effort tracking recently finalized
    records, force the committing writer to read records and
    increment the last "contiguous block" of finalized records. In
    order to do this, the sequence number instead of ID must be
    stored because ID's cannot be directly compared.
    
    A new memory barrier pair is introduced to guarantee that a
    reader can always read the records up until the sequence number
    returned by prb_next_seq() (unless the records have since
    been overwritten in the ringbuffer).
    
    This restores the original functionality of prb_next_seq()
    while also keeping the optimization.
    
    For 32bit systems, only the lower 32 bits of the sequence
    number are stored. When reading the value, it is expanded to
    the full 64bit sequence number using the 32bit seq macros,
    which fold in the value returned by prb_first_seq().
    
    Fixes: f244b4dc ("printk: ringbuffer: Improve prb_next_seq() performance")
    Signed-off-by: default avatarJohn Ogness <john.ogness@linutronix.de>
    Reviewed-by: default avatarPetr Mladek <pmladek@suse.com>
    Link: https://lore.kernel.org/r/20240207134103.1357162-5-john.ogness@linutronix.deSigned-off-by: default avatarPetr Mladek <pmladek@suse.com>
    5f72e52b
printk_ringbuffer.c 70.2 KB