• Hugh Dickins's avatar
    s390: add pte_free_defer() for pgtables sharing page · 8211dad6
    Hugh Dickins authored
    Add s390-specific pte_free_defer(), to free table page via call_rcu(). 
    pte_free_defer() will be called inside khugepaged's retract_page_tables()
    loop, where allocating extra memory cannot be relied upon.  This precedes
    the generic version to avoid build breakage from incompatible pgtable_t.
    
    This version is more complicated than others: because s390 fits two 2K
    page tables into one 4K page (so page->rcu_head must be shared between
    both halves), and already uses page->lru (which page->rcu_head overlays)
    to list any free halves; with clever management by page->_refcount bits.
    
    Build upon the existing management, adjusted to follow a new rule: that a
    page is never on the free list if pte_free_defer() was used on either half
    (marked by PageActive).  And for simplicity, delay calling RCU until both
    halves are freed.
    
    Not adding back unallocated fragments to the list in pte_free_defer() can
    result in wasting some amount of memory for pagetables, depending on how
    long the allocated fragment will stay in use.  In practice, this effect is
    expected to be insignificant, and not justify a far more complex approach,
    which might allow to add the fragments back later in __tlb_remove_table(),
    where we might not have a stable mm any more.
    
    [hughd@google.com: Claudio finds warning on mm_has_pgste() more useful than on mm_alloc_pgste()]
      Link: https://lkml.kernel.org/r/3bc095ba-a180-ce3b-82b1-2bfc64612f3@google.com
    Link: https://lkml.kernel.org/r/94eccf5f-264c-8abe-4567-e77f4b4e14a@google.comSigned-off-by: default avatarHugh Dickins <hughd@google.com>
    Reviewed-by: default avatarGerald Schaefer <gerald.schaefer@linux.ibm.com>
    Tested-by: default avatarAlexander Gordeev <agordeev@linux.ibm.com>
    Acked-by: default avatarAlexander Gordeev <agordeev@linux.ibm.com>
    Cc: Alistair Popple <apopple@nvidia.com>
    Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
    Cc: Anshuman Khandual <anshuman.khandual@arm.com>
    Cc: Axel Rasmussen <axelrasmussen@google.com>
    Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
    Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
    Cc: Christoph Hellwig <hch@infradead.org>
    Cc: Claudio Imbrenda <imbrenda@linux.ibm.com>
    Cc: David Hildenbrand <david@redhat.com>
    Cc: "David S. Miller" <davem@davemloft.net>
    Cc: Heiko Carstens <hca@linux.ibm.com>
    Cc: Huang, Ying <ying.huang@intel.com>
    Cc: Ira Weiny <ira.weiny@intel.com>
    Cc: Jann Horn <jannh@google.com>
    Cc: Jason Gunthorpe <jgg@ziepe.ca>
    Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
    Cc: Lorenzo Stoakes <lstoakes@gmail.com>
    Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
    Cc: Mel Gorman <mgorman@techsingularity.net>
    Cc: Miaohe Lin <linmiaohe@huawei.com>
    Cc: Michael Ellerman <mpe@ellerman.id.au>
    Cc: Mike Kravetz <mike.kravetz@oracle.com>
    Cc: Mike Rapoport (IBM) <rppt@kernel.org>
    Cc: Minchan Kim <minchan@kernel.org>
    Cc: Naoya Horiguchi <naoya.horiguchi@nec.com>
    Cc: Pavel Tatashin <pasha.tatashin@soleen.com>
    Cc: Peter Xu <peterx@redhat.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Qi Zheng <zhengqi.arch@bytedance.com>
    Cc: Ralph Campbell <rcampbell@nvidia.com>
    Cc: Russell King <linux@armlinux.org.uk>
    Cc: SeongJae Park <sj@kernel.org>
    Cc: Song Liu <song@kernel.org>
    Cc: Steven Price <steven.price@arm.com>
    Cc: Suren Baghdasaryan <surenb@google.com>
    Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
    Cc: Vasily Gorbik <gor@linux.ibm.com>
    Cc: Vishal Moola (Oracle) <vishal.moola@gmail.com>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Cc: Will Deacon <will@kernel.org>
    Cc: Yang Shi <shy828301@gmail.com>
    Cc: Yu Zhao <yuzhao@google.com>
    Cc: Zack Rusin <zackr@vmware.com>
    Cc: Zi Yan <ziy@nvidia.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    8211dad6
pgalloc.c 21.3 KB