• Steve Capper's avatar
    arm64: hugetlb: Avoid unnecessary clearing in huge_ptep_set_access_flags · 031e6e6b
    Steve Capper authored
    For contiguous hugetlb, huge_ptep_set_access_flags performs a
    get_clear_flush (which then flushes the TLBs) even when no change of ptes
    is necessary.
    
    Unfortunately, this behaviour can lead to back-to-back page faults being
    generated when running with multiple threads that access the same
    contiguous huge page.
    
    Thread 1                     |  Thread 2
    -----------------------------+------------------------------
    hugetlb_fault                |
    huge_ptep_set_access_flags   |
      -> invalidate pte range    | hugetlb_fault
    continue processing          | wait for hugetlb_fault_mutex
    release mutex and return     | huge_ptep_set_access_flags
                                 |   -> invalidate pte range
    hugetlb_fault
    ...
    
    This patch changes huge_ptep_set_access_flags s.t. we first read the
    contiguous range of ptes (whilst preserving dirty information); the pte
    range is only then invalidated where necessary and this prevents further
    spurious page faults.
    
    Fixes: d8bdcff2 ("arm64: hugetlb: Add break-before-make logic for contiguous entries")
    Reported-by: default avatarLei Zhang <zhang.lei@jp.fujitsu.com>
    Signed-off-by: default avatarSteve Capper <steve.capper@arm.com>
    Signed-off-by: default avatarWill Deacon <will.deacon@arm.com>
    031e6e6b
hugetlbpage.c 10.8 KB