• Suren Baghdasaryan's avatar
    mm: add per-VMA lock and helper functions to control it · 5e31275c
    Suren Baghdasaryan authored
    Introduce per-VMA locking.  The lock implementation relies on a per-vma
    and per-mm sequence counters to note exclusive locking:
    
      - read lock - (implemented by vma_start_read) requires the vma
        (vm_lock_seq) and mm (mm_lock_seq) sequence counters to differ.
        If they match then there must be a vma exclusive lock held somewhere.
      - read unlock - (implemented by vma_end_read) is a trivial vma->lock
        unlock.
      - write lock - (vma_start_write) requires the mmap_lock to be held
        exclusively and the current mm counter is assigned to the vma counter.
        This will allow multiple vmas to be locked under a single mmap_lock
        write lock (e.g. during vma merging). The vma counter is modified
        under exclusive vma lock.
      - write unlock - (vma_end_write_all) is a batch release of all vma
        locks held. It doesn't pair with a specific vma_start_write! It is
        done before exclusive mmap_lock is released by incrementing mm
        sequence counter (mm_lock_seq).
      - write downgrade - if the mmap_lock is downgraded to the read lock, all
        vma write locks are released as well (effectivelly same as write
        unlock).
    
    Link: https://lkml.kernel.org/r/20230227173632.3292573-13-surenb@google.comSigned-off-by: default avatarSuren Baghdasaryan <surenb@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    5e31275c
fork.c 82.8 KB