1. 16 Feb, 2011 31 commits
    • Tejun Heo's avatar
      x86-64, NUMA: Build and use direct emulated nid -> phys nid mapping · 9d073cae
      Tejun Heo authored
      NUMA emulation copied physical NUMA configuration into physnodes[] and
      used it to reverse-map emulated nodes to physical nodes, which is
      unnecessarily convoluted.  Build emu_nid_to_phys[] array to map
      emulated nids directly to the matching physical nids and use it in
      numa_add_cpu().
      
      physnodes[] will be removed with further patches.
      
      - v2: Build failure when CONFIG_DEBUG_PER_CPU_MAPS due to missing
        local variable definition fixed.  Reported by Ingo.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      9d073cae
    • Tejun Heo's avatar
      x86-64, NUMA: Trivial changes to prepare for emulation updates · d9c515ea
      Tejun Heo authored
      * Separate out numa_add_memblk_to() from numa_add_memblk() so that
        different numa_meminfo can be used.
      
      * Rename cmdline to emu_cmdline.
      
      * Drop @start/last_pfn from numa_emulation() and use max_pfn directly.
      
      This patch doesn't introduce any behavior change.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      d9c515ea
    • Tejun Heo's avatar
      x86-64, NUMA: Implement generic node distance handling · ac7136b6
      Tejun Heo authored
      Node distance either used direct node comparison, ACPI PXM comparison
      or ACPI SLIT table lookup.  This patch implements generic node
      distance handling.  NUMA init methods can call numa_set_distance() to
      set distance between nodes and the common __node_distance()
      implementation will report the set distance.
      
      Due to the way NUMA emulation is implemented, the generic node
      distance handling is used only when emulation is not used.  Later
      patches will update NUMA emulation to use the generic distance
      mechanism.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      ac7136b6
    • Tejun Heo's avatar
      x86-64, NUMA: Kill mem_nodes_parsed · 4697bdcc
      Tejun Heo authored
      With all memory configuration information now carried in numa_meminfo,
      there's no need to keep mem_nodes_parsed separate.  Drop it and use
      numa_nodes_parsed for CPU / memory-less nodes.
      
      A new helper numa_nodemask_from_meminfo() is added to calculate
      memnode mask on the fly which is currently used to set
      node_possible_map.
      
      This simplifies NUMA init methods a bit and removes a source of
      possible inconsistencies.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      4697bdcc
    • Tejun Heo's avatar
      x86-64, NUMA: Rename cpu_nodes_parsed to numa_nodes_parsed · 92d4a437
      Tejun Heo authored
      It's no longer necessary to keep both cpu_nodes_parsed and
      mem_nodes_parsed.  In preparation for merge, rename cpu_nodes_parsed
      to numa_nodes_parsed.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      92d4a437
    • Tejun Heo's avatar
      x86-64, NUMA: Kill numa_nodes[] · 91556237
      Tejun Heo authored
      numa_nodes[] doesn't carry any information which isn't present in
      numa_meminfo.  Each entry is simply min/max range of all the memblks
      for the node.  This is not only redundant but also inaccurate when
      memblks for different nodes interleave - for example,
      find_node_by_addr() can return the wrong nodeid.
      
      Kill numa_nodes[] and always use numa_meminfo instead.
      
      * nodes_cover_memory() is renamed to numa_meminfo_cover_memory() and
        now operations on numa_meminfo and returns bool.
      
      * setup_node_bootmem() needs min/max range.  Compute the range on the
        fly.  setup_node_bootmem() invocation is restructured to use outer
        loop instead of hardcoding the double invocations.
      
      * find_node_by_addr() now operates on numa_meminfo.
      
      * setup_physnodes() builds physnodes[] from memblks.  This will go
        away when emulation code is updated to use struct numa_meminfo.
      
      This patch also makes the following misc changes.
      
      * Clearing of nodes_add[] clearing is converted to memset().
      
      * numa_add_memblk() in amd_numa_init() is moved down a bit for
        consistency.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      91556237
    • Tejun Heo's avatar
      x86-64, NUMA: Add common find_node_by_addr() · a844ef46
      Tejun Heo authored
      srat_64.c and amdtopology_64.c had their own versions of
      find_node_by_addr() which were basically the same.  Add common one in
      numa_64.c and remove the duplicates.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      a844ef46
    • Tejun Heo's avatar
      x86-64, NUMA: consolidate and improve memblk sanity checks · 56e827fb
      Tejun Heo authored
      memblk sanity check was scattered around and incomplete.  Consolidate
      and improve.
      
      * Confliction detection and cutoff_node() logic are moved to
        numa_cleanup_meminfo().
      
      * numa_cleanup_meminfo() clears the unused memblks before returning.
      
      * Check and warn about invalid input parameters in numa_add_memblk().
      
      * Check the maximum number of memblk isn't exceeded in
        numa_add_memblk().
      
      * numa_cleanup_meminfo() is now called before numa_emulation() so that
        the emulation code also uses the cleaned up version.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      56e827fb
    • Tejun Heo's avatar
      x86-64, NUMA: make numa_cleanup_meminfo() prettier · 2e756be4
      Tejun Heo authored
      * Factor out numa_remove_memblk_from().
      
      * Hole detection doesn't need separate start/end.  Calculate start/end
        once.
      
      * Relocate comment.
      
      * Define iterators at the top and remove unnecessary prefix
        increments.
      
      This prepares for further improvements to the function.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      2e756be4
    • Tejun Heo's avatar
      x86-64, NUMA: Separate out numa_cleanup_meminfo() · f9c60251
      Tejun Heo authored
      Separate out numa_cleanup_meminfo() from numa_register_memblks().
      node_possible_map initialization is moved to the top of the split
      numa_register_memblks().
      
      This patch doesn't cause behavior change.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      f9c60251
    • Tejun Heo's avatar
      x86-64, NUMA: Introduce struct numa_meminfo · 97e7b78d
      Tejun Heo authored
      Arrays for memblks and nodeids and their length lived in separate
      variables making things unnecessarily cumbersome.  Introduce struct
      numa_meminfo which contains all memory configuration info.  This patch
      doesn't cause any behavior change.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      97e7b78d
    • Tejun Heo's avatar
      x86-64, NUMA: Remove %NULL @nodeids handling from compute_hash_shift() · 8968dab8
      Tejun Heo authored
      numa_emulation() called compute_hash_shift() with %NULL @nodeids which
      meant identity mapping between index and nodeid.  Make
      numa_emulation() build identity array and drop %NULL @nodeids handling
      from populate_memnodemap() and thus from compute_hash_shift().  This
      is to prepare for transition to using memblks instead.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      8968dab8
    • Tejun Heo's avatar
      x86-64, NUMA: Kill {acpi|amd|dummy}_scan_nodes() · 5d371b08
      Tejun Heo authored
      They are empty now.  Kill them.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      5d371b08
    • Tejun Heo's avatar
      x86-64, NUMA: Unify the rest of memblk registration · fd0435d8
      Tejun Heo authored
      Move the remaining memblk registration logic from acpi_scan_nodes() to
      numa_register_memblks() and initmem_init().
      
      This applies nodes_cover_memory() sanity check, memory node sorting
      and node_online() checking, which were only applied to acpi, to all
      init methods.
      
      As all memblk registration is moved to common code, active range
      clearing is moved to initmem_init() too and removed from bad_srat().
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      fd0435d8
    • Tejun Heo's avatar
      x86-64, NUMA: Unify use of memblk in all init methods · 43a662f0
      Tejun Heo authored
      Make both amd and dummy use numa_add_memblk() to describe the detected
      memory blocks.  This allows initmem_init() to call
      numa_register_memblk() regardless of init method in use.  Drop custom
      memory registration codes from amd and dummy.
      
      After this change, memblk merge/cleanup in numa_register_memblks() is
      applied to all init methods.
      
      As this makes compute_hash_shift() and numa_register_memblks() used
      only inside numa_64.c, make them static.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      43a662f0
    • Tejun Heo's avatar
      x86-64, NUMA: Factor out memblk handling into numa_{add|register}_memblk() · ef396ec9
      Tejun Heo authored
      Factor out memblk handling from srat_64.c into two functions in
      numa_64.c.  This patch doesn't introduce any behavior change.  The
      next patch will make all init methods use these functions.
      
      - v2: Fixed build failure on 32bit due to misplaced NR_NODE_MEMBLKS.
            Reported by Ingo.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      ef396ec9
    • Tejun Heo's avatar
      x86-64, NUMA: Kill {acpi|amd}_get_nodes() · 19095548
      Tejun Heo authored
      With common numa_nodes[], common code in numa_64.c can access it
      directly.  Copy directly and kill {acpi|amd}_get_nodes().
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      19095548
    • Tejun Heo's avatar
      x86-64, NUMA: Use common numa_nodes[] · 206e4208
      Tejun Heo authored
      ACPI and amd are using separate nodes[] array.  Add numa_nodes[] and
      use them in all NUMA init methods.  cutoff_node() cleanup is moved
      from srat_64.c to numa_64.c and applied in initmem_init() regardless
      of init methods.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      206e4208
    • Tejun Heo's avatar
      x86-64, NUMA: Move apicid to numa mapping initialization from amd_scan_nodes() to amd_numa_init() · 45fe6c78
      Tejun Heo authored
      This brings amd initialization behavior closer to that of acpi.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      45fe6c78
    • Tejun Heo's avatar
      x86-64, NUMA: Remove local variable found from amd_numa_init() · 99df738c
      Tejun Heo authored
      Use weight count on mem_nodes_parsed instead.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      99df738c
    • Tejun Heo's avatar
      x86-64, NUMA: Use common {cpu|mem}_nodes_parsed · ec8cf29b
      Tejun Heo authored
      ACPI and amd are using separate nodes_parsed masks.  Add
      {cpu|mem}_nodes_parsed and use them in all NUMA init methods.
      Initialization of the masks and building node_possible_map are now
      handled commonly by initmem_init().
      
      dummy_numa_init() is updated to set node 0 on both masks.  While at
      it, move the info messages from scan to init.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      ec8cf29b
    • Tejun Heo's avatar
      x86-64, NUMA: Restructure initmem_init() · ffe77a46
      Tejun Heo authored
      Reorganize initmem_init() such that,
      
      * Different NUMA init methods are iterated in a consistent way.
      
      * Each iteration re-initializes all the parameters and different
        method can be tried after a failure.
      
      * Dummy init is handled the same as other methods.
      
      Apart from how retry after failure, this patch doesn't change the
      behavior.  The call sequences are kept equivalent across the
      conversion.
      
      After the change, bad_srat() doesn't need to clear apic to node
      mapping or worry about numa_off.  Simplified accordingly.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      ffe77a46
    • Tejun Heo's avatar
      x86, NUMA: Move *_numa_init() invocations into initmem_init() · d8fc3afc
      Tejun Heo authored
      There's no reason for these to live in setup_arch().  Move them inside
      initmem_init().
      
      - v2: x86-32 initmem_init() weren't updated breaking 32bit builds.
        Fixed.  Found by Ankita.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Ankita Garg <ankita@in.ibm.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      d8fc3afc
    • Tejun Heo's avatar
      x86-64, NUMA: Wrap acpi_numa_init() so that failure can be indicated by return value · a9aec56a
      Tejun Heo authored
      Because of the way ACPI tables are parsed, the generic
      acpi_numa_init() couldn't return failure when error was detected by
      arch hooks.  Instead, the failure state was recorded and later arch
      dependent init hook - acpi_scan_nodes() - would fail.
      
      Wrap acpi_numa_init() with x86_acpi_numa_init() so that failure can be
      indicated as return value immediately.  This is in preparation for
      further NUMA init cleanups.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      a9aec56a
    • Tejun Heo's avatar
      x86-64, NUMA: Unify {acpi|amd}_{numa_init|scan_nodes}() arguments and return values · 940fed2e
      Tejun Heo authored
      The functions used during NUMA initialization - *_numa_init() and
      *_scan_nodes() - have different arguments and return values.  Unify
      them such that they all take no argument and return 0 on success and
      -errno on failure.  This is in preparation for further NUMA init
      cleanups.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      940fed2e
    • Tejun Heo's avatar
      x86, NUMA: Drop @start/last_pfn from initmem_init() · 86ef4dbf
      Tejun Heo authored
      initmem_init() extensively accesses and modifies global data
      structures and the parameters aren't even followed depending on which
      path is being used.  Drop @start/last_pfn and let it deal with
      @max_pfn directly.  This is in preparation for further NUMA init
      cleanups.
      
      - v2: x86-32 initmem_init() weren't updated breaking 32bit builds.
        Fixed.  Found by Yinghai.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      86ef4dbf
    • Tejun Heo's avatar
      x86-64, NUMA: Simplify hotplug node handling in acpi_numa_memory_affinity_init() · 13081df5
      Tejun Heo authored
      Hotplug node handling in acpi_numa_memory_affinity_init() was
      unnecessarily complicated with storing the original nodes[] entry and
      restoring it afterwards.  Simplify it by not modifying the nodes[]
      entry for hotplug nodes from the beginning.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      13081df5
    • Tejun Heo's avatar
      x86-64, NUMA: Make dummy node initialization path similar to non-dummy ones · 7d36b7bc
      Tejun Heo authored
      Dummy node initialization in initmem_init() didn't initialize apicid
      to node mapping and set cpu to node mapping directly by caling
      numa_set_node(), which is different from non-dummy init paths.
      
      Update it such that they behave similarly.  Initialize apicid to node
      mapping and call numa_init_array().  The actual cpu to node mapping is
      handled by init_cpu_to_node() later.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarYinghai Lu <yinghai@kernel.org>
      Cc: Brian Gerst <brgerst@gmail.com>
      Cc: Cyrill Gorcunov <gorcunov@gmail.com>
      Cc: Shaohui Zheng <shaohui.zheng@intel.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: H. Peter Anvin <hpa@linux.intel.com>
      7d36b7bc
    • Ingo Molnar's avatar
      Merge branch 'x86/amd-nb' into x86/mm · 275a88d3
      Ingo Molnar authored
      Merge reason: consolidate it into the more generic x86/mm tree to prevent conflicts
                    with ongoing NUMA work.
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      275a88d3
    • Ingo Molnar's avatar
      Merge branch 'x86/numa' into x86/mm · 52b8b8d7
      Ingo Molnar authored
      Merge reason: consolidate it into the more generic x86/mm tree to prevent conflicts.
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      52b8b8d7
    • Ingo Molnar's avatar
      Merge branch 'x86/bootmem' into x86/mm · 02ac81a8
      Ingo Molnar authored
      Merge reason: the topic is ready - consolidate it into the more generic x86/mm tree
                    and prevent conflicts.
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      02ac81a8
  2. 15 Feb, 2011 1 commit
    • Borislav Petkov's avatar
      x86, amd: Initialize variable properly · 9e81509e
      Borislav Petkov authored
      Commit d518573d ("x86, amd: Normalize compute unit IDs on
      multi-node processors") introduced compute unit normalization
      but causes a compiler warning:
      
       arch/x86/kernel/cpu/amd.c: In function 'amd_detect_cmp':
       arch/x86/kernel/cpu/amd.c:268: warning: 'cores_per_cu' may be used uninitialized in this function
       arch/x86/kernel/cpu/amd.c:268: note: 'cores_per_cu' was declared here
      
      The compiler is right - initialize it with a proper value.
      
      Also, fix up a comment while at it.
      Reported-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarBorislav Petkov <borislav.petkov@amd.com>
      Cc: Andreas Herrmann <andreas.herrmann3@amd.com>
      LKML-Reference: <20110214171451.GB10076@kryptos.osrc.amd.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      9e81509e
  3. 14 Feb, 2011 8 commits
    • David Rientjes's avatar
      x86, numa: Add error handling for bad cpu-to-node mappings · 14392fd3
      David Rientjes authored
      CONFIG_DEBUG_PER_CPU_MAPS may return NUMA_NO_NODE when an
      early_cpu_to_node() mapping hasn't been initialized.  In such a
      case, it emits a warning and continues without an issue but
      callers may try to use the return value to index into an array.
      
      We can catch those errors and fail silently since a warning has
      already been emitted.  No current user of numa_add_cpu()
      requires this error checking to avoid a crash, but it's better
      to be proactive in case a future user happens to have a bug and
      a user tries to diagnose it with CONFIG_DEBUG_PER_CPU_MAPS.
      Reported-by: default avatarJesper Juhl <jj@chaosbits.net>
      Signed-off-by: default avatarDavid Rientjes <rientjes@google.com>
      Cc: Tejun Heo <tj@kernel.org>
      LKML-Reference: <alpine.DEB.2.00.1102071407250.7812@chino.kir.corp.google.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      14392fd3
    • Ingo Molnar's avatar
      Merge commit 'v2.6.38-rc4' into x86/numa · b366801c
      Ingo Molnar authored
      Merge reason: Merge latest fixes before applying new patch.
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      b366801c
    • Kamal Mostafa's avatar
      x86: Emit "mem=nopentium ignored" warning when not supported · 9a6d44b9
      Kamal Mostafa authored
      Emit warning when "mem=nopentium" is specified on any arch other
      than x86_32 (the only that arch supports it).
      Signed-off-by: default avatarKamal Mostafa <kamal@canonical.com>
      BugLink: http://bugs.launchpad.net/bugs/553464
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Len Brown <len.brown@intel.com>
      Cc: Rafael J. Wysocki <rjw@sisk.pl>
      LKML-Reference: <1296783486-23033-2-git-send-email-kamal@canonical.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      Cc: <stable@kernel.org>
      9a6d44b9
    • Kamal Mostafa's avatar
      x86: Fix panic when handling "mem={invalid}" param · 77eed821
      Kamal Mostafa authored
      Avoid removing all of memory and panicing when "mem={invalid}"
      is specified, e.g. mem=blahblah, mem=0, or mem=nopentium (on
      platforms other than x86_32).
      Signed-off-by: default avatarKamal Mostafa <kamal@canonical.com>
      BugLink: http://bugs.launchpad.net/bugs/553464
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Len Brown <len.brown@intel.com>
      Cc: Rafael J. Wysocki <rjw@sisk.pl>
      Cc: <stable@kernel.org> # .3x: as far back as it applies
      LKML-Reference: <1296783486-23033-1-git-send-email-kamal@canonical.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      77eed821
    • Shaohua Li's avatar
      x86: Avoid tlbstate lock if not enough cpus · 7064d865
      Shaohua Li authored
      This one isn't related to previous patch. If online cpus are
      below NUM_INVALIDATE_TLB_VECTORS, we don't need the lock. The
      comments in the code declares we don't need the check, but a hot
      lock still needs an atomic operation and expensive, so add the
      check here.
      
      Uses nr_cpu_ids here as suggested by Eric Dumazet.
      Signed-off-by: default avatarShaohua Li <shaohua.li@intel.com>
      Acked-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Cc: Andi Kleen <andi@firstfloor.org>
      LKML-Reference: <1295232730.1949.710.camel@sli10-conroe>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      7064d865
    • Shaohua Li's avatar
      x86: Scale up the number of TLB invalidate vectors with NR_CPUs, up to 32 · 70e4a369
      Shaohua Li authored
      Make the maxium TLB invalidate vectors depend on NR_CPUS linearly,
      with a maximum of 32 vectors.
      
      We currently only have 8 vectors for TLB invalidation and that is clearly
      inadequate. If we have a lot of CPUs, the CPUs need share the 8 vectors and
      tlbstate_lock is used to protect them. flush_tlb_page() is
      heavily used in page reclaim, which will cause a lot of lock
      contention for tlbstate_lock.
      
      Andi Kleen suggested increasing the vectors number to 32, which should be
      good for current typical systems to reduce the tlbstate_lock contention.
      
      My test system has 4 sockets and 64G memory, and 64 CPUs. My
      workload creates 64 processes. Each process mmap reads a big
      empty sparse file. The total size of the files are 2*total_mem,
      so this will cause a lot of page reclaim.
      
      Below is the result I get from perf call-graph profiling:
      
       without the patch:
       ------------------
      
          24.25%           usemem  [kernel]                                   [k] _raw_spin_lock
                           |
                           --- _raw_spin_lock
                              |
                              |--42.15%-- native_flush_tlb_others
      
       with the patch:
       ------------------
      
          14.96%           usemem  [kernel]                                   [k] _raw_spin_lock
                           |
                           --- _raw_spin_lock
                              |--13.89%-- native_flush_tlb_others
      
      So this heavily reduces the tlbstate_lock contention.
      Suggested-by: default avatarAndi Kleen <andi@firstfloor.org>
      Signed-off-by: default avatarShaohua Li <shaohua.li@intel.com>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      LKML-Reference: <1295232727.1949.709.camel@sli10-conroe>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      70e4a369
    • Shaohua Li's avatar
      x86: Allocate 32 tlb_invalidate_interrupt handler stubs · 3a09fb45
      Shaohua Li authored
      Add up to 32 invalidate_interrupt handlers. How many handlers are
      added depends on NUM_INVALIDATE_TLB_VECTORS. So if
      NUM_INVALIDATE_TLB_VECTORS is smaller than 32, we reduce code
      size.
      Signed-off-by: default avatarShaohua Li <shaohua.li@intel.com>
      Cc: Andi Kleen <andi@firstfloor.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      LKML-Reference: <1295232725.1949.708.camel@sli10-conroe>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      3a09fb45
    • Shaohua Li's avatar
      x86: Cleanup vector usage · 60f6e65d
      Shaohua Li authored
      Cleanup the vector usage and make them continuous if possible.
      Signed-off-by: default avatarShaohua Li <shaohua.li@intel.com>
      Cc: Andi Kleen <andi@firstfloor.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      LKML-Reference: <1295232722.1949.707.camel@sli10-conroe>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      60f6e65d