• Lance Yang's avatar
    mm/rmap: remove duplicated exit code in pagewalk loop · 26d21b18
    Lance Yang authored
    Patch series "Reclaim lazyfree THP without splitting", v8.
    
    This series adds support for reclaiming PMD-mapped THP marked as lazyfree
    without needing to first split the large folio via
    split_huge_pmd_address().
    
    When the user no longer requires the pages, they would use
    madvise(MADV_FREE) to mark the pages as lazy free.  Subsequently, they
    typically would not re-write to that memory again.
    
    During memory reclaim, if we detect that the large folio and its PMD are
    both still marked as clean and there are no unexpected references(such as
    GUP), so we can just discard the memory lazily, improving the efficiency
    of memory reclamation in this case.
    
    Performance Testing
    ===================
    
    On an Intel i5 CPU, reclaiming 1GiB of lazyfree THPs using
    mem_cgroup_force_empty() results in the following runtimes in seconds
    (shorter is better):
    
    --------------------------------------------
    |     Old       |      New       |  Change  |
    --------------------------------------------
    |   0.683426    |    0.049197    |  -92.80% |
    --------------------------------------------
    
    
    This patch (of 8):
    
    Introduce the labels walk_done and walk_abort as exit points to eliminate
    duplicated exit code in the pagewalk loop.
    
    Link: https://lkml.kernel.org/r/20240614015138.31461-1-ioworker0@gmail.com
    Link: https://lkml.kernel.org/r/20240614015138.31461-2-ioworker0@gmail.comSigned-off-by: default avatarLance Yang <ioworker0@gmail.com>
    Reviewed-by: default avatarZi Yan <ziy@nvidia.com>
    Reviewed-by: default avatarBaolin Wang <baolin.wang@linux.alibaba.com>
    Reviewed-by: default avatarDavid Hildenbrand <david@redhat.com>
    Reviewed-by: default avatarBarry Song <baohua@kernel.org>
    Cc: Bang Li <libang.li@antgroup.com>
    Cc: Fangrui Song <maskray@google.com>
    Cc: Jeff Xie <xiehuan09@gmail.com>
    Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
    Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
    Cc: Michal Hocko <mhocko@suse.com>
    Cc: Minchan Kim <minchan@kernel.org>
    Cc: Muchun Song <songmuchun@bytedance.com>
    Cc: Peter Xu <peterx@redhat.com>
    Cc: Ryan Roberts <ryan.roberts@arm.com>
    Cc: SeongJae Park <sj@kernel.org>
    Cc: Yang Shi <shy828301@gmail.com>
    Cc: Yin Fengwei <fengwei.yin@intel.com>
    Cc: Zach O'Keefe <zokeefe@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    26d21b18
rmap.c 78.5 KB