Commit 1f25fe20 authored by Kirill A. Shutemov's avatar Kirill A. Shutemov Committed by Linus Torvalds

mm, thp: adjust conditions when we can reuse the page on WP fault

With new refcounting we will be able map the same compound page with
PTEs and PMDs.  It requires adjustment to conditions when we can reuse
the page on write-protection fault.

For PTE fault we can't reuse the page if it's part of huge page.

For PMD we can only reuse the page if nobody else maps the huge page or
it's part.  We can do it by checking page_mapcount() on each sub-page,
but it's expensive.

The cheaper way is to check page_count() to be equal 1: every mapcount
takes page reference, so this way we can guarantee, that the PMD is the
only mapping.

This approach can give false negative if somebody pinned the page, but
that doesn't affect correctness.
Signed-off-by: default avatarKirill A. Shutemov <kirill.shutemov@linux.intel.com>
Tested-by: default avatarSasha Levin <sasha.levin@oracle.com>
Tested-by: default avatarAneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Acked-by: default avatarJerome Marchand <jmarchan@redhat.com>
Acked-by: default avatarVlastimil Babka <vbabka@suse.cz>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Rik van Riel <riel@redhat.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Steve Capper <steve.capper@linaro.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Christoph Lameter <cl@linux.com>
Cc: David Rientjes <rientjes@google.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
parent f627c2f5
...@@ -538,7 +538,8 @@ static inline int swp_swapcount(swp_entry_t entry) ...@@ -538,7 +538,8 @@ static inline int swp_swapcount(swp_entry_t entry)
return 0; return 0;
} }
#define reuse_swap_page(page) (page_mapcount(page) == 1) #define reuse_swap_page(page) \
(!PageTransCompound(page) && page_mapcount(page) == 1)
static inline int try_to_free_swap(struct page *page) static inline int try_to_free_swap(struct page *page)
{ {
......
...@@ -1199,7 +1199,17 @@ int do_huge_pmd_wp_page(struct mm_struct *mm, struct vm_area_struct *vma, ...@@ -1199,7 +1199,17 @@ int do_huge_pmd_wp_page(struct mm_struct *mm, struct vm_area_struct *vma,
page = pmd_page(orig_pmd); page = pmd_page(orig_pmd);
VM_BUG_ON_PAGE(!PageCompound(page) || !PageHead(page), page); VM_BUG_ON_PAGE(!PageCompound(page) || !PageHead(page), page);
if (page_mapcount(page) == 1) { /*
* We can only reuse the page if nobody else maps the huge page or it's
* part. We can do it by checking page_mapcount() on each sub-page, but
* it's expensive.
* The cheaper way is to check page_count() to be equal 1: every
* mapcount takes page reference reference, so this way we can
* guarantee, that the PMD is the only mapping.
* This can give false negative if somebody pinned the page, but that's
* fine.
*/
if (page_mapcount(page) == 1 && page_count(page) == 1) {
pmd_t entry; pmd_t entry;
entry = pmd_mkyoung(orig_pmd); entry = pmd_mkyoung(orig_pmd);
entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
......
...@@ -926,6 +926,9 @@ int reuse_swap_page(struct page *page) ...@@ -926,6 +926,9 @@ int reuse_swap_page(struct page *page)
VM_BUG_ON_PAGE(!PageLocked(page), page); VM_BUG_ON_PAGE(!PageLocked(page), page);
if (unlikely(PageKsm(page))) if (unlikely(PageKsm(page)))
return 0; return 0;
/* The page is part of THP and cannot be reused */
if (PageTransCompound(page))
return 0;
count = page_mapcount(page); count = page_mapcount(page);
if (count <= 1 && PageSwapCache(page)) { if (count <= 1 && PageSwapCache(page)) {
count += page_swapcount(page); count += page_swapcount(page);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment