• Mike Kravetz's avatar
    hugetlbfs: fix races and page leaks during migration · cb6acd01
    Mike Kravetz authored
    hugetlb pages should only be migrated if they are 'active'.  The
    routines set/clear_page_huge_active() modify the active state of hugetlb
    pages.
    
    When a new hugetlb page is allocated at fault time, set_page_huge_active
    is called before the page is locked.  Therefore, another thread could
    race and migrate the page while it is being added to page table by the
    fault code.  This race is somewhat hard to trigger, but can be seen by
    strategically adding udelay to simulate worst case scheduling behavior.
    Depending on 'how' the code races, various BUG()s could be triggered.
    
    To address this issue, simply delay the set_page_huge_active call until
    after the page is successfully added to the page table.
    
    Hugetlb pages can also be leaked at migration time if the pages are
    associated with a file in an explicitly mounted hugetlbfs filesystem.
    For example, consider a two node system with 4GB worth of huge pages
    available.  A program mmaps a 2G file in a hugetlbfs filesystem.  It
    then migrates the pages associated with the file from one node to
    another.  When the program exits, huge page counts are as follows:
    
      node0
      1024    free_hugepages
      1024    nr_hugepages
    
      node1
      0       free_hugepages
      1024    nr_hugepages
    
      Filesystem                         Size  Used Avail Use% Mounted on
      nodev                              4.0G  2.0G  2.0G  50% /var/opt/hugepool
    
    That is as expected.  2G of huge pages are taken from the free_hugepages
    counts, and 2G is the size of the file in the explicitly mounted
    filesystem.  If the file is then removed, the counts become:
    
      node0
      1024    free_hugepages
      1024    nr_hugepages
    
      node1
      1024    free_hugepages
      1024    nr_hugepages
    
      Filesystem                         Size  Used Avail Use% Mounted on
      nodev                              4.0G  2.0G  2.0G  50% /var/opt/hugepool
    
    Note that the filesystem still shows 2G of pages used, while there
    actually are no huge pages in use.  The only way to 'fix' the filesystem
    accounting is to unmount the filesystem
    
    If a hugetlb page is associated with an explicitly mounted filesystem,
    this information in contained in the page_private field.  At migration
    time, this information is not preserved.  To fix, simply transfer
    page_private from old to new page at migration time if necessary.
    
    There is a related race with removing a huge page from a file and
    migration.  When a huge page is removed from the pagecache, the
    page_mapping() field is cleared, yet page_private remains set until the
    page is actually freed by free_huge_page().  A page could be migrated
    while in this state.  However, since page_mapping() is not set the
    hugetlbfs specific routine to transfer page_private is not called and we
    leak the page count in the filesystem.
    
    To fix that, check for this condition before migrating a huge page.  If
    the condition is detected, return EBUSY for the page.
    
    Link: http://lkml.kernel.org/r/74510272-7319-7372-9ea6-ec914734c179@oracle.com
    Link: http://lkml.kernel.org/r/20190212221400.3512-1-mike.kravetz@oracle.com
    Fixes: bcc54222 ("mm: hugetlb: introduce page_huge_active")
    Signed-off-by: default avatarMike Kravetz <mike.kravetz@oracle.com>
    Reviewed-by: default avatarNaoya Horiguchi <n-horiguchi@ah.jp.nec.com>
    Cc: Michal Hocko <mhocko@kernel.org>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>
    Cc: Mel Gorman <mgorman@techsingularity.net>
    Cc: Davidlohr Bueso <dave@stgolabs.net>
    Cc: <stable@vger.kernel.org>
    [mike.kravetz@oracle.com: v2]
      Link: http://lkml.kernel.org/r/7534d322-d782-8ac6-1c8d-a8dc380eb3ab@oracle.com
    [mike.kravetz@oracle.com: update comment and changelog]
      Link: http://lkml.kernel.org/r/420bcfd6-158b-38e4-98da-26d0cd85bd01@oracle.comSigned-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    cb6acd01
hugetlb.c 133 KB