Commit eefb9b27 authored by David Hildenbrand's avatar David Hildenbrand Committed by Andrew Morton

mm: improve folio_likely_mapped_shared() using the mapcount of large folios

We can now read the mapcount of large folios very efficiently.  Use it to
improve our handling of partially-mappable folios, falling back to making
a guess only in case the folio is not "obviously mapped shared".

We can now better detect partially-mappable folios where the first page is
not mapped as "mapped shared", reducing "false negatives"; but false
negatives are still possible.

While at it, fixup a wrong comment (false positive vs.  false negative)
for KSM folios.

Link: https://lkml.kernel.org/r/20240409192301.907377-6-david@redhat.comSigned-off-by: default avatarDavid Hildenbrand <david@redhat.com>
Reviewed-by: default avatarYin Fengwei <fengwei.yin@intel.com>
Cc: Chris Zankel <chris@zankel.net>
Cc: Hugh Dickins <hughd@google.com>
Cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: Miaohe Lin <linmiaohe@huawei.com>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Naoya Horiguchi <nao.horiguchi@gmail.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Richard Chang <richardycc@google.com>
Cc: Rich Felker <dalias@libc.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Yang Shi <shy828301@gmail.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
parent 05c5323b
...@@ -2184,7 +2184,7 @@ static inline size_t folio_size(struct folio *folio) ...@@ -2184,7 +2184,7 @@ static inline size_t folio_size(struct folio *folio)
* indicate "mapped shared" (false positive) when two VMAs in the same MM * indicate "mapped shared" (false positive) when two VMAs in the same MM
* cover the same file range. * cover the same file range.
* #. For (small) KSM folios, the return value can wrongly indicate "mapped * #. For (small) KSM folios, the return value can wrongly indicate "mapped
* shared" (false negative), when the folio is mapped multiple times into * shared" (false positive), when the folio is mapped multiple times into
* the same MM. * the same MM.
* *
* Further, this function only considers current page table mappings that * Further, this function only considers current page table mappings that
...@@ -2201,7 +2201,22 @@ static inline size_t folio_size(struct folio *folio) ...@@ -2201,7 +2201,22 @@ static inline size_t folio_size(struct folio *folio)
*/ */
static inline bool folio_likely_mapped_shared(struct folio *folio) static inline bool folio_likely_mapped_shared(struct folio *folio)
{ {
return page_mapcount(folio_page(folio, 0)) > 1; int mapcount = folio_mapcount(folio);
/* Only partially-mappable folios require more care. */
if (!folio_test_large(folio) || unlikely(folio_test_hugetlb(folio)))
return mapcount > 1;
/* A single mapping implies "mapped exclusively". */
if (mapcount <= 1)
return false;
/* If any page is mapped more than once we treat it "mapped shared". */
if (folio_entire_mapcount(folio) || mapcount > folio_nr_pages(folio))
return true;
/* Let's guess based on the first subpage. */
return atomic_read(&folio->_mapcount) > 0;
} }
#ifndef HAVE_ARCH_MAKE_PAGE_ACCESSIBLE #ifndef HAVE_ARCH_MAKE_PAGE_ACCESSIBLE
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment