• Vlastimil Babka's avatar
    cpuset: use static key better and convert to new API · 002f2906
    Vlastimil Babka authored
    An important function for cpusets is cpuset_node_allowed(), which
    optimizes on the fact if there's a single root CPU set, it must be
    trivially allowed.  But the check "nr_cpusets() <= 1" doesn't use the
    cpusets_enabled_key static key the right way where static keys eliminate
    branching overhead with jump labels.
    
    This patch converts it so that static key is used properly.  It's also
    switched to the new static key API and the checking functions are
    converted to return bool instead of int.  We also provide a new variant
    __cpuset_zone_allowed() which expects that the static key check was
    already done and they key was enabled.  This is needed for
    get_page_from_freelist() where we want to also avoid the relatively
    slower check when ALLOC_CPUSET is not set in alloc_flags.
    
    The impact on the page allocator microbenchmark is less than expected
    but the cleanup in itself is worthwhile.
    
                                                 4.6.0-rc2                  4.6.0-rc2
                                           multcheck-v1r20               cpuset-v1r20
      Min      alloc-odr0-1               348.00 (  0.00%)           348.00 (  0.00%)
      Min      alloc-odr0-2               254.00 (  0.00%)           254.00 (  0.00%)
      Min      alloc-odr0-4               213.00 (  0.00%)           213.00 (  0.00%)
      Min      alloc-odr0-8               186.00 (  0.00%)           183.00 (  1.61%)
      Min      alloc-odr0-16              173.00 (  0.00%)           171.00 (  1.16%)
      Min      alloc-odr0-32              166.00 (  0.00%)           163.00 (  1.81%)
      Min      alloc-odr0-64              162.00 (  0.00%)           159.00 (  1.85%)
      Min      alloc-odr0-128             160.00 (  0.00%)           157.00 (  1.88%)
      Min      alloc-odr0-256             169.00 (  0.00%)           166.00 (  1.78%)
      Min      alloc-odr0-512             180.00 (  0.00%)           180.00 (  0.00%)
      Min      alloc-odr0-1024            188.00 (  0.00%)           187.00 (  0.53%)
      Min      alloc-odr0-2048            194.00 (  0.00%)           193.00 (  0.52%)
      Min      alloc-odr0-4096            199.00 (  0.00%)           198.00 (  0.50%)
      Min      alloc-odr0-8192            202.00 (  0.00%)           201.00 (  0.50%)
      Min      alloc-odr0-16384           203.00 (  0.00%)           202.00 (  0.49%)
    Signed-off-by: default avatarVlastimil Babka <vbabka@suse.cz>
    Signed-off-by: default avatarMel Gorman <mgorman@techsingularity.net>
    Acked-by: default avatarZefan Li <lizefan@huawei.com>
    Cc: Jesper Dangaard Brouer <brouer@redhat.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    002f2906
page_alloc.c 201 KB