• Linus Torvalds's avatar
    cpu masks: optimize and clean up cpumask_of_cpu() · e56b3bc7
    Linus Torvalds authored
    Clean up and optimize cpumask_of_cpu(), by sharing all the zero words.
    
    Instead of stupidly generating all possible i=0...NR_CPUS 2^i patterns
    creating a huge array of constant bitmasks, realize that the zero words
    can be shared.
    
    In other words, on a 64-bit architecture, we only ever need 64 of these
    arrays - with a different bit set in one single world (with enough zero
    words around it so that we can create any bitmask by just offsetting in
    that big array). And then we just put enough zeroes around it that we
    can point every single cpumask to be one of those things.
    
    So when we have 4k CPU's, instead of having 4k arrays (of 4k bits each,
    with one bit set in each array - 2MB memory total), we have exactly 64
    arrays instead, each 8k bits in size (64kB total).
    
    And then we just point cpumask(n) to the right position (which we can
    calculate dynamically). Once we have the right arrays, getting
    "cpumask(n)" ends up being:
    
      static inline const cpumask_t *get_cpu_mask(unsigned int cpu)
      {
              const unsigned long *p = cpu_bit_bitmap[1 + cpu % BITS_PER_LONG];
              p -= cpu / BITS_PER_LONG;
              return (const cpumask_t *)p;
      }
    
    This brings other advantages and simplifications as well:
    
     - we are not wasting memory that is just filled with a single bit in
       various different places
    
     - we don't need all those games to re-create the arrays in some dense
       format, because they're already going to be dense enough.
    
    if we compile a kernel for up to 4k CPU's, "wasting" that 64kB of memory
    is a non-issue (especially since by doing this "overlapping" trick we
    probably get better cache behaviour anyway).
    
    [ mingo@elte.hu:
    
      Converted Linus's mails into a commit. See:
    
         http://lkml.org/lkml/2008/7/27/156
         http://lkml.org/lkml/2008/7/28/320
    
      Also applied a family filter - which also has the side-effect of leaving
      out the bits where Linus calls me an idio... Oh, never mind ;-)
    ]
    Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
    Cc: Rusty Russell <rusty@rustcorp.com.au>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Al Viro <viro@ZenIV.linux.org.uk>
    Cc: Mike Travis <travis@sgi.com>
    Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
    e56b3bc7
setup_percpu.c 8.98 KB