• Jianyu Zhan's avatar
    mm: introdule compound_head_by_tail() · d2ee40ea
    Jianyu Zhan authored
    Currently, in put_compound_page(), we have
    
    ======
    if (likely(!PageTail(page))) {                  <------  (1)
            if (put_page_testzero(page)) {
                     /*
                     ¦* By the time all refcounts have been released
                     ¦* split_huge_page cannot run anymore from under us.
                     ¦*/
                     if (PageHead(page))
                             __put_compound_page(page);
                     else
                             __put_single_page(page);
             }
             return;
    }
    
    /* __split_huge_page_refcount can run under us */
    page_head = compound_head(page);        <------------ (2)
    ======
    
    if at (1) ,  we fail the check, this means page is *likely* a tail page.
    
    Then at (2), as compoud_head(page) is inlined, it is :
    
    ======
    static inline struct page *compound_head(struct page *page)
    {
              if (unlikely(PageTail(page))) {           <----------- (3)
                  struct page *head = page->first_page;
    
                    smp_rmb();
                    if (likely(PageTail(page)))
                            return head;
            }
            return page;
    }
    ======
    
    here, the (3) unlikely in the case is a negative hint, because it is
    *likely* a tail page.  So the check (3) in this case is not good, so I
    introduce a helper for this case.
    
    So this patch introduces compound_head_by_tail() which deals with a
    possible tail page(though it could be spilt by a racy thread), and make
    compound_head() a wrapper on it.
    
    This patch has no functional change, and it reduces the object
    size slightly:
       text    data     bss     dec     hex  filename
      11003    1328      16   12347    303b  mm/swap.o.orig
      10971    1328      16   12315    301b  mm/swap.o.patched
    
    I've ran "perf top -e branch-miss" to observe branch-miss in this case.
    As Michael points out, it's a slow path, so only very few times this case
    happens.  But I grep'ed the code base, and found there still are some
    other call sites could be benifited from this helper.  And given that it
    only bloating up the source by only 5 lines, but with a reduced object
    size.  I still believe this helper deserves to exsit.
    Signed-off-by: default avatarJianyu Zhan <nasa4836@gmail.com>
    Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Jiang Liu <liuj97@gmail.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Sasha Levin <sasha.levin@oracle.com>
    Cc: Wanpeng Li <liwanp@linux.vnet.ibm.com>
    Cc: Hugh Dickins <hughd@google.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    d2ee40ea
swap.c 30.5 KB