• Nick Piggin's avatar
    fix SMP data race in pagetable setup vs walking · 362a61ad
    Nick Piggin authored
    There is a possible data race in the page table walking code. After the split
    ptlock patches, it actually seems to have been introduced to the core code, but
    even before that I think it would have impacted some architectures (powerpc
    and sparc64, at least, walk the page tables without taking locks eg. see
    find_linux_pte()).
    
    The race is as follows:
    The pte page is allocated, zeroed, and its struct page gets its spinlock
    initialized. The mm-wide ptl is then taken, and then the pte page is inserted
    into the pagetables.
    
    At this point, the spinlock is not guaranteed to have ordered the previous
    stores to initialize the pte page with the subsequent store to put it in the
    page tables. So another Linux page table walker might be walking down (without
    any locks, because we have split-leaf-ptls), and find that new pte we've
    inserted. It might try to take the spinlock before the store from the other
    CPU initializes it. And subsequently it might read a pte_t out before stores
    from the other CPU have cleared the memory.
    
    There are also similar races in higher levels of the page tables. They
    obviously don't involve the spinlock, but could see uninitialized memory.
    
    Arch code and hardware pagetable walkers that walk the pagetables without
    locks could see similar uninitialized memory problems, regardless of whether
    split ptes are enabled or not.
    
    I prefer to put the barriers in core code, because that's where the higher
    level logic happens, but the page table accessors are per-arch, and open-coding
    them everywhere I don't think is an option. I'll put the read-side barriers
    in alpha arch code for now (other architectures perform data-dependent loads
    in order).
    Signed-off-by: default avatarNick Piggin <npiggin@suse.de>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    362a61ad
pgtable.h 13.6 KB