• Mel Gorman's avatar
    mm: activate !PageLRU pages on mark_page_accessed if page is on local pagevec · 059285a2
    Mel Gorman authored
    If a page is on a pagevec then it is !PageLRU and mark_page_accessed()
    may fail to move a page to the active list as expected.  Now that the
    LRU is selected at LRU drain time, mark pages PageActive if they are on
    the local pagevec so it gets moved to the correct list at LRU drain
    time.  Using a debugging patch it was found that for a simple git
    checkout based workload that pages were never added to the active file
    list in practice but with this patch applied they are.
    
    				before   after
    LRU Add Active File                  0      750583
    LRU Add Active Anon            2640587     2702818
    LRU Add Inactive File          8833662     8068353
    LRU Add Inactive Anon              207         200
    
    Note that only pages on the local pagevec are considered on purpose.  A
    !PageLRU page could be in the process of being released, reclaimed,
    migrated or on a remote pagevec that is currently being drained.
    Marking it PageActive is vunerable to races where PageLRU and Active
    bits are checked at the wrong time.  Page reclaim will trigger
    VM_BUG_ONs but depending on when the race hits, it could also free a
    PageActive page to the page allocator and trigger a bad_page warning.
    Similarly a potential race exists between a per-cpu drain on a pagevec
    list and an activation on a remote CPU.
    
    				lru_add_drain_cpu
    				__pagevec_lru_add
    				  lru = page_lru(page);
    mark_page_accessed
      if (PageLRU(page))
        activate_page
      else
        SetPageActive
    				  SetPageLRU(page);
    				  add_page_to_lru_list(page, lruvec, lru);
    
    In this case a PageActive page is added to the inactivate list and later
    the inactive/active stats will get skewed.  While the PageActive checks
    in vmscan could be removed and potentially dealt with, a skew in the
    statistics would be very difficult to detect.  Hence this patch deals
    just with the common case where a page being marked accessed has just
    been added to the local pagevec.
    Signed-off-by: default avatarMel Gorman <mgorman@suse.de>
    Cc: Jan Kara <jack@suse.cz>
    Cc: Rik van Riel <riel@redhat.com>
    Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
    Cc: Alexey Lyahkov <alexey.lyashkov@gmail.com>
    Cc: Andrew Perepechko <anserper@ya.ru>
    Cc: Robin Dong <sanbai@taobao.com>
    Cc: Theodore Tso <tytso@mit.edu>
    Cc: Hugh Dickins <hughd@google.com>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Bernd Schubert <bernd.schubert@fastmail.fm>
    Cc: David Howells <dhowells@redhat.com>
    Cc: Trond Myklebust <Trond.Myklebust@netapp.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    059285a2
swap.c 24.6 KB