• Nadav Amit's avatar
    x86/mm/tlb: Ignore f->new_tlb_gen when zero · 8f1d56f6
    Nadav Amit authored
    Commit aa442849 ("x86/mm/tlb: Avoid reading mm_tlb_gen when
    possible") introduced an optimization to skip superfluous TLB
    flushes based on the generation provided in flush_tlb_info.
    
    However, arch_tlbbatch_flush() does not provide any generation in
    flush_tlb_info and populates the flush_tlb_info generation with
    0.  This 0 is causes the flush_tlb_info to be interpreted as a
    superfluous, old flush.  As a result, try_to_unmap_one() would
    not perform any TLB flushes.
    
    Fix it by checking whether f->new_tlb_gen is nonzero. Zero value
    is anyhow is an invalid generation value. To avoid future
    confusion, introduce TLB_GENERATION_INVALID constant and use it
    properly. Add warnings to ensure no partial flushes are done with
    TLB_GENERATION_INVALID or when f->mm is NULL, since this does not
    make any sense.
    
    In addition, add the missing unlikely().
    
    [ dhansen: change VM_BUG_ON() -> VM_WARN_ON(), clarify changelog ]
    
    Fixes: aa442849 ("x86/mm/tlb: Avoid reading mm_tlb_gen when possible")
    Reported-by: default avatarHugh Dickins <hughd@google.com>
    Signed-off-by: default avatarNadav Amit <namit@vmware.com>
    Signed-off-by: default avatarDave Hansen <dave.hansen@linux.intel.com>
    Tested-by: default avatarHugh Dickins <hughd@google.com>
    Link: https://lkml.kernel.org/r/20220710232837.3618-1-namit@vmware.com
    8f1d56f6
tlb.c 38.5 KB