• Nicholas Piggin's avatar
    powerpc/64s/radix: Fix mm_cpumask trimming race vs kthread_use_mm · a665eec0
    Nicholas Piggin authored
    Commit 0cef77c7 ("powerpc/64s/radix: flush remote CPUs out of
    single-threaded mm_cpumask") added a mechanism to trim the mm_cpumask of
    a process under certain conditions. One of the assumptions is that
    mm_users would not be incremented via a reference outside the process
    context with mmget_not_zero() then go on to kthread_use_mm() via that
    reference.
    
    That invariant was broken by io_uring code (see previous sparc64 fix),
    but I'll point Fixes: to the original powerpc commit because we are
    changing that assumption going forward, so this will make backports
    match up.
    
    Fix this by no longer relying on that assumption, but by having each CPU
    check the mm is not being used, and clearing their own bit from the mask
    only if it hasn't been switched-to by the time the IPI is processed.
    
    This relies on commit 38cf307c ("mm: fix kthread_use_mm() vs TLB
    invalidate") and ARCH_WANT_IRQS_OFF_ACTIVATE_MM to disable irqs over mm
    switch sequences.
    
    Fixes: 0cef77c7 ("powerpc/64s/radix: flush remote CPUs out of single-threaded mm_cpumask")
    Signed-off-by: default avatarNicholas Piggin <npiggin@gmail.com>
    Reviewed-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
    Depends-on: 38cf307c ("mm: fix kthread_use_mm() vs TLB invalidate")
    Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
    Link: https://lore.kernel.org/r/20200914045219.3736466-5-npiggin@gmail.com
    a665eec0
radix_tlb.c 33.7 KB