1. 29 Apr, 2013 40 commits
    • majianpeng's avatar
      fs/buffer.c: remove unnecessary init operation after allocating buffer_head. · e7600409
      majianpeng authored
      bh allocation uses kmem_cache_zalloc() so we needn't call
      'init_buffer(bh, NULL, NULL)' and perform other set-zero-operations.
      Signed-off-by: default avatarJianpeng Ma <majianpeng@gmail.com>
      Cc: Jan Kara <jack@suse.cz>
      Cc: Theodore Ts'o <tytso@mit.edu>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      e7600409
    • Yasuaki Ishimatsu's avatar
      numa, cpu hotplug: change links of CPU and node when changing node number by onlining CPU · 34640468
      Yasuaki Ishimatsu authored
      When booting x86 system contains memoryless node, node numbers of CPUs
      on memoryless node were changed to nearest online node number by
      init_cpu_to_node() because the node is not online.
      
      In my system, node numbers of cpu#30-44 and 75-89 were changed from 2 to
      0 as follows:
      
        $ numactl --hardware
        available: 2 nodes (0-1)
        node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 30 31 32 33 34 35 36 37 38 39 40
        41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 75 76 77 78 79 80 81 82
        83 84 85 86 87 88 89
        node 0 size: 32394 MB
        node 0 free: 27898 MB
        node 1 cpus: 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 60 61 62 63 64 65 66
        67 68 69 70 71 72 73 74
        node 1 size: 32768 MB
        node 1 free: 30335 MB
      
      If we hot add memory to memoryless node and offine/online all CPUs on
      the node, node numbers of these CPUs are changed to correct node numbers
      by srat_detect_node() because the node become online.
      
      In this case, node numbers of cpu#30-44 and 75-89 were changed from 0 to
      2 in my system as follows:
      
        $ numactl --hardware
        available: 3 nodes (0-2)
        node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 45 46 47 48 49 50 51 52 53 54 55
        56 57 58 59
        node 0 size: 32394 MB
        node 0 free: 27218 MB
        node 1 cpus: 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 60 61 62 63 64 65 66
        67 68 69 70 71 72 73 74
        node 1 size: 32768 MB
        node 1 free: 30014 MB
        node 2 cpus: 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 75 76 77 78 79 80 81
        82 83 84 85 86 87 88 89
        node 2 size: 16384 MB
        node 2 free: 16384 MB
      
      But "cpu to node" and "node to cpu" links were not changed as follows:
      
        $ ls /sys/devices/system/cpu/cpu30/|grep node
        node0
        $ ls /sys/devices/system/node/node0/|grep cpu30
        cpu30
      
      "numactl --hardware" shows that cpu30 belongs to node 2.  But sysfs
      links does not change.
      
      This patch changes "cpu to node" and "node to cpu" links when node
      number changed by onlining CPU.
      Signed-off-by: default avatarYasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
      Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      Cc: "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>
      Cc: Greg KH <greg@kroah.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      34640468
    • Randy Dunlap's avatar
      mm: fix memory_hotplug.c printk format warning · 349daa0f
      Randy Dunlap authored
      PFN_PHYS() is a phys_addr_t, which can be u32 or u64.
      Fix the build warning when phys_addr_t is u32.
      
        mm/memory_hotplug.c: warning: format '%llx' expects argument of type 'long long unsigned int', but argument 2 has type 'unsigned int' [-Wformat]:  => 1685:3
        mm/memory_hotplug.c: warning: format '%llx' expects argument of type 'long long unsigned int', but argument 3 has type 'unsigned int' [-Wformat]:  => 1685:3
      Signed-off-by: default avatarRandy Dunlap <rdunlap@infradead.org>
      Reported-by: default avatarGeert Uytterhoeven <geert@linux-m68k.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      349daa0f
    • Mel Gorman's avatar
      mm: swap: mark swap pages writeback before queueing for direct IO · 0cdc444a
      Mel Gorman authored
      As pointed out by Andrew Morton, the swap-over-NFS writeback is not
      setting PageWriteback before it is queued for direct IO.  While swap
      pages do not participate in BDI or process dirty accounting and the IO
      is synchronous, the writeback bit is still required and not setting it
      in this case was an oversight.  swapoff depends on the page writeback to
      synchronoise all pending writes on a swap page before it is reused.
      Swapcache freeing and reuse depend on checking the PageWriteback under
      lock to ensure the page is safe to reuse.
      
      Direct IO handlers and the direct IO handler for NFS do not deal with
      PageWriteback as they are synchronous writes.  In the case of NFS, it
      schedules pages (or a page in the case of swap) for IO and then waits
      synchronously for IO to complete in nfs_direct_write().  It is
      recognised that this is a slowdown from normal swap handling which is
      asynchronous and uses a completion handler.  Shoving PageWriteback
      handling down into direct IO handlers looks like a bad fit to handle the
      swap case although it may have to be dealt with some day if swap is
      converted to use direct IO in general and bmap is finally done away
      with.  At that point it will be necessary to refit asynchronous direct
      IO with completion handlers onto the swap subsystem.
      
      As swapcache currently depends on PageWriteback to protect against
      races, this patch sets PageWriteback under the page lock before queueing
      it for direct IO.  It is cleared when the direct IO handler returns.  IO
      errors are treated similarly to the direct-to-bio case except PageError
      is not set as in the case of swap-over-NFS, it is likely to be a
      transient error.
      
      It was asked what prevents such a page being reclaimed in parallel.
      With this patch applied, such a page will now be skipped (most of the
      time) or blocked until the writeback completes.  Reclaim checks
      PageWriteback under the page lock before calling try_to_free_swap and
      the page lock should prevent the page being requeued for IO before it is
      freed.
      
      This and Jerome's related patch should considered for -stable as far
      back as 3.6 when swap-over-NFS was introduced.
      
      [akpm@linux-foundation.org: use pr_err_ratelimited()]
      [akpm@linux-foundation.org: remove hopefully-unneeded cast in printk]
      Signed-off-by: default avatarMel Gorman <mgorman@suse.de>
      Cc: Jerome Marchand <jmarchan@redhat.com>
      Cc: Hugh Dickins <hughd@google.com>
      Cc: <stable@vger.kernel.org>	[3.6+]
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      0cdc444a
    • Jerome Marchand's avatar
      swap: redirty page if page write fails on swap file · 2d30d31e
      Jerome Marchand authored
      Since commit 62c230bc ("mm: add support for a filesystem to activate
      swap files and use direct_IO for writing swap pages"), swap_writepage()
      calls direct_IO on swap files.  However, in that case the page isn't
      redirtied if I/O fails, and is therefore handled afterwards as if it has
      been successfully written to the swap file, leading to memory corruption
      when the page is eventually swapped back in.
      
      This patch sets the page dirty when direct_IO() fails.  It fixes a
      memory corruption that happened while using swap-over-NFS.
      Signed-off-by: default avatarJerome Marchand <jmarchan@redhat.com>
      Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Acked-by: default avatarMel Gorman <mgorman@suse.de>
      Cc: Hugh Dickins <hughd@google.com>
      Cc: <stable@vger.kernel.org>	[3.6+]
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      2d30d31e
    • David Rientjes's avatar
      mm, memcg: give exiting processes access to memory reserves · 465adcf1
      David Rientjes authored
      A memcg may livelock when oom if the process that grabs the hierarchy's
      oom lock is never the first process with PF_EXITING set in the memcg's
      task iteration.
      
      The oom killer, both global and memcg, will defer if it finds an
      eligible process that is in the process of exiting and it is not being
      ptraced.  The idea is to allow it to exit without using memory reserves
      before needlessly killing another process.
      
      This normally works fine except in the memcg case with a large number of
      threads attached to the oom memcg.  In this case, the memcg oom killer
      only gets called for the process that grabs the hierarchy's oom lock;
      all others end up blocked on the memcg's oom waitqueue.  Thus, if the
      process that grabs the hierarchy's oom lock is never the first
      PF_EXITING process in the memcg's task iteration, the oom killer is
      constantly deferred without anything making progress.
      
      The fix is to give PF_EXITING processes access to memory reserves so
      that we've marked them as oom killed without any iteration.  This allows
      __mem_cgroup_try_charge() to succeed so that the process may exit.  This
      makes the memcg oom killer exemption for TIF_MEMDIE tasks, now
      immediately granted for processes with pending SIGKILLs and those in the
      exit path, to be equivalent to what is done for the global oom killer.
      Signed-off-by: default avatarDavid Rientjes <rientjes@google.com>
      Acked-by: default avatarMichal Hocko <mhocko@suse.cz>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      465adcf1
    • Kirill A. Shutemov's avatar
      thp: fix huge zero page logic for page with pfn == 0 · 5918d10a
      Kirill A. Shutemov authored
      Current implementation of huge zero page uses pfn value 0 to indicate
      that the page hasn't allocated yet.  It assumes that buddy page
      allocator can't return page with pfn == 0.
      
      Let's rework the code to store 'struct page *' of huge zero page, not
      its pfn.  This way we can avoid the weak assumption.
      
      [akpm@linux-foundation.org: fix sparse warning]
      Signed-off-by: default avatarKirill A. Shutemov <kirill.shutemov@linux.intel.com>
      Reported-by: default avatarMinchan Kim <minchan@kernel.org>
      Acked-by: default avatarMinchan Kim <minchan@kernel.org>
      Reviewed-by: default avatarAndrea Arcangeli <aarcange@redhat.com>
      Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Cc: Wu Fengguang <fengguang.wu@intel.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      5918d10a
    • Li Zefan's avatar
      memcg: avoid accessing memcg after releasing reference · fd0ccaf2
      Li Zefan authored
      This might cause a use-after-free bug.
      Signed-off-by: default avatarLi Zefan <lizefan@huawei.com>
      Cc: Glauber Costa <glommer@parallels.com>
      Acked-by: default avatarMichal Hocko <mhocko@suse.cz>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      fd0ccaf2
    • Dmitry Monakhov's avatar
      fs: fix fsync() error reporting · 865ffef3
      Dmitry Monakhov authored
      There are two convenient ways to report errors to userspace
      
      1) retun error to original syscall for example write(2)
      2) mark mapping with error flag and return it on later fsync(2)
      
      Second one is broken if (mapping->nrpages == 0) This is real-life
      situation because after error pages are likey to be truncated or
      invalidated.
      
      We have to return an error regardless to number of pages in the mapping.
      
      #Original testcase: git@github.com:dmonakhov/xfstests.git
      MOUNT_OPTIONS="-b1024"
      ./check shared/305
      Signed-off-by: default avatarDmitry Monakhov <dmonakhov@openvz.org>
      Reviewed-by: default avatarJan Kara <jack@suse.cz>
      Cc: Al Viro <viro@zeniv.linux.org.uk>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      865ffef3
    • Tang Chen's avatar
      memblock: fix missing comment of memblock_insert_region() · 209ff86d
      Tang Chen authored
      There is no comment for parameter nid of memblock_insert_region().
      This patch adds comment for it.
      Signed-off-by: default avatarTang Chen <tangchen@cn.fujitsu.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      209ff86d
    • Tang Chen's avatar
      mm: Remove unused parameter of pages_correctly_reserved() · 6056d619
      Tang Chen authored
      nr_pages is not used in pages_correctly_reserved().
      So remove it.
      Signed-off-by: default avatarTang Chen <tangchen@cn.fujitsu.com>
      Reviewed-by: default avatarWang Shilong <wangsl-fnst@cn.fujitsu.com>
      Reviewed-by: default avatarWen Congyang <wency@cn.fujitsu.com>
      Acked-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      6056d619
    • Yasuaki Ishimatsu's avatar
      firmware, memmap: fix firmware_map_entry leak · 7a6f93b0
      Yasuaki Ishimatsu authored
      When hot removing memory, a firmware_map_entry which has memory range of
      the memory is released by release_firmware_map_entry().  If the entry is
      allocated by bootmem, release_firmware_map_entry() adds the entry to
      map_entires_bootmem list when firmware_map_find_entry() finds the entry
      from map_entries list.  But firmware_map_find_entry never find the entry
      sicne map_entires list does not have the entry.  So the entry just
      leaks.
      
      Here are steps of leaking firmware_map_entry:
      firmware_map_remove()
      -> firmware_map_find_entry()
         Find released entry from map_entries list
      -> firmware_map_remove_entry()
         Delete the entry from map_entries list
      -> remove_sysfs_fw_map_entry()
         ...
         -> release_firmware_map_entry()
            -> firmware_map_find_entry()
               Find the entry from map_entries list but the entry has been
               deleted from map_entries list. So the entry is not added
               to map_entries_bootmem. Thus the entry leaks
      
      release_firmware_map_entry() should not call firmware_map_find_entry()
      since releaed entry has been deleted from map_entries list.  So the
      patch delete firmware_map_find_entry() from releae_firmware_map_entry()
      Signed-off-by: default avatarYasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
      Reviewed-by: default avatarWanpeng Li <liwanp@linux.vnet.ibm.com>
      Reviewed-by: default avatarTang Chen <tangchen@cn.fujitsu.com>
      Acked-by: default avatarToshi Kani <toshi.kani@hp.com>
      Cc: Wen Congyang <wency@cn.fujitsu.com>
      Cc: Greg KH <greg@kroah.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      7a6f93b0
    • Cody P Schafer's avatar
    • Shaohua Li's avatar
      mm: thp: add split tail pages to shrink page list in page reclaim · 5bc7b8ac
      Shaohua Li authored
      In page reclaim, huge page is split.  split_huge_page() adds tail pages
      to LRU list.  Since we are reclaiming a huge page, it's better we
      reclaim all subpages of the huge page instead of just the head page.
      This patch adds split tail pages to shrink page list so the tail pages
      can be reclaimed soon.
      
      Before this patch, run a swap workload:
        thp_fault_alloc 3492
        thp_fault_fallback 608
        thp_collapse_alloc 6
        thp_collapse_alloc_failed 0
        thp_split 916
      
      With this patch:
        thp_fault_alloc 4085
        thp_fault_fallback 16
        thp_collapse_alloc 90
        thp_collapse_alloc_failed 0
        thp_split 1272
      
      fallback allocation is reduced a lot.
      
      [akpm@linux-foundation.org: fix CONFIG_SWAP=n build]
      Signed-off-by: default avatarShaohua Li <shli@fusionio.com>
      Acked-by: default avatarRik van Riel <riel@redhat.com>
      Acked-by: default avatarMinchan Kim <minchan@kernel.org>
      Acked-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
      Reviewed-by: default avatarWanpeng Li <liwanp@linux.vnet.ibm.com>
      Cc: Andrea Arcangeli <aarcange@redhat.com>
      Cc: Hugh Dickins <hughd@google.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      5bc7b8ac
    • Seth Jennings's avatar
      mm: allow for outstanding swap writeback accounting · 1eec6702
      Seth Jennings authored
      To prevent flooding the swap device with writebacks, frontswap backends
      need to count and limit the number of outstanding writebacks.  The
      incrementing of the counter can be done before the call to
      __swap_writepage().  However, the caller must receive a notification
      when the writeback completes in order to decrement the counter.
      
      To achieve this functionality, this patch modifies __swap_writepage() to
      take the bio completion callback function as an argument.
      
      end_swap_bio_write(), the normal bio completion function, is also made
      non-static so that code doing the accounting can call it after the
      accounting is done.
      
      There should be no behavioural change to existing code.
      Signed-off-by: default avatarSeth Jennings <sjenning@linux.vnet.ibm.com>
      Signed-off-by: default avatarBob Liu <bob.liu@oracle.com>
      Acked-by: default avatarMinchan Kim <minchan@kernel.org>
      Reviewed-by: default avatarDan Magenheimer <dan.magenheimer@oracle.com>
      Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      1eec6702
    • Seth Jennings's avatar
      mm: break up swap_writepage() for frontswap backends · 2f772e6c
      Seth Jennings authored
      swap_writepage() is currently where frontswap hooks into the swap write
      path to capture pages with the frontswap_store() function.  However, if
      a frontswap backend wants to "resume" the writeback of a page to the
      swap device, it can't call swap_writepage() as the page will simply
      reenter the backend.
      
      This patch separates swap_writepage() into a top and bottom half, the
      bottom half named __swap_writepage() to allow a frontswap backend, like
      zswap, to resume writeback beyond the frontswap_store() hook.
      
      __add_to_swap_cache() is also made non-static so that the page for which
      writeback is to be resumed can be added to the swap cache.
      Signed-off-by: default avatarSeth Jennings <sjenning@linux.vnet.ibm.com>
      Signed-off-by: default avatarBob Liu <bob.liu@oracle.com>
      Acked-by: default avatarMinchan Kim <minchan@kernel.org>
      Reviewed-by: default avatarDan Magenheimer <dan.magenheimer@oracle.com>
      Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      2f772e6c
    • Cyril Hrubis's avatar
      mm/mmap: check for RLIMIT_AS before unmapping · e8420a8e
      Cyril Hrubis authored
      Fix a corner case for MAP_FIXED when requested mapping length is larger
      than rlimit for virtual memory.  In such case any overlapping mappings
      are unmapped before we check for the limit and return ENOMEM.
      
      The check is moved before the loop that unmaps overlapping parts of
      existing mappings.  When we are about to hit the limit (currently mapped
      pages + len > limit) we scan for overlapping pages and check again
      accounting for them.
      
      This fixes situation when userspace program expects that the previous
      mappings are preserved after the mmap() syscall has returned with error.
      (POSIX clearly states that successfull mapping shall replace any
      previous mappings.)
      
      This corner case was found and can be tested with LTP testcase:
      
      testcases/open_posix_testsuite/conformance/interfaces/mmap/24-2.c
      
      In this case the mmap, which is clearly over current limit, unmaps
      dynamic libraries and the testcase segfaults right after returning into
      userspace.
      
      I've also looked at the second instance of the unmapping loop in the
      do_brk().  The do_brk() is called from brk() syscall and from vm_brk().
      The brk() syscall checks for overlapping mappings and bails out when
      there are any (so it can't be triggered from the brk syscall).  The
      vm_brk() is called only from binmft handlers so it shouldn't be
      triggered unless binmft handler created overlapping mappings.
      Signed-off-by: default avatarCyril Hrubis <chrubis@suse.cz>
      Reviewed-by: default avatarMel Gorman <mgorman@suse.de>
      Reviewed-by: default avatarWanpeng Li <liwanp@linux.vnet.ibm.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      e8420a8e
    • Anton Vorontsov's avatar
      memcg: add memory.pressure_level events · 70ddf637
      Anton Vorontsov authored
      With this patch userland applications that want to maintain the
      interactivity/memory allocation cost can use the pressure level
      notifications.  The levels are defined like this:
      
      The "low" level means that the system is reclaiming memory for new
      allocations.  Monitoring this reclaiming activity might be useful for
      maintaining cache level.  Upon notification, the program (typically
      "Activity Manager") might analyze vmstat and act in advance (i.e.
      prematurely shutdown unimportant services).
      
      The "medium" level means that the system is experiencing medium memory
      pressure, the system might be making swap, paging out active file
      caches, etc.  Upon this event applications may decide to further analyze
      vmstat/zoneinfo/memcg or internal memory usage statistics and free any
      resources that can be easily reconstructed or re-read from a disk.
      
      The "critical" level means that the system is actively thrashing, it is
      about to out of memory (OOM) or even the in-kernel OOM killer is on its
      way to trigger.  Applications should do whatever they can to help the
      system.  It might be too late to consult with vmstat or any other
      statistics, so it's advisable to take an immediate action.
      
      The events are propagated upward until the event is handled, i.e.  the
      events are not pass-through.  Here is what this means: for example you
      have three cgroups: A->B->C.  Now you set up an event listener on
      cgroups A, B and C, and suppose group C experiences some pressure.  In
      this situation, only group C will receive the notification, i.e.  groups
      A and B will not receive it.  This is done to avoid excessive
      "broadcasting" of messages, which disturbs the system and which is
      especially bad if we are low on memory or thrashing.  So, organize the
      cgroups wisely, or propagate the events manually (or, ask us to
      implement the pass-through events, explaining why would you need them.)
      
      Performance wise, the memory pressure notifications feature itself is
      lightweight and does not require much of bookkeeping, in contrast to the
      rest of memcg features.  Unfortunately, as of current memcg
      implementation, pages accounting is an inseparable part and cannot be
      turned off.  The good news is that there are some efforts[1] to improve
      the situation; plus, implementing the same, fully API-compatible[2]
      interface for CONFIG_MEMCG=n case (e.g.  embedded) is also a viable
      option, so it will not require any changes on the userland side.
      
      [1] http://permalink.gmane.org/gmane.linux.kernel.cgroups/6291
      [2] http://lkml.org/lkml/2013/2/21/454
      
      [akpm@linux-foundation.org: coding-style fixes]
      [akpm@linux-foundation.org: fix CONFIG_CGROPUPS=n warnings]
      Signed-off-by: default avatarAnton Vorontsov <anton.vorontsov@linaro.org>
      Acked-by: default avatarKirill A. Shutemov <kirill@shutemov.name>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Tejun Heo <tj@kernel.org>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Pekka Enberg <penberg@kernel.org>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Glauber Costa <glommer@parallels.com>
      Cc: Michal Hocko <mhocko@suse.cz>
      Cc: Luiz Capitulino <lcapitulino@redhat.com>
      Cc: Greg Thelen <gthelen@google.com>
      Cc: Leonid Moiseichuk <leonid.moiseichuk@nokia.com>
      Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
      Cc: Minchan Kim <minchan@kernel.org>
      Cc: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
      Cc: John Stultz <john.stultz@linaro.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      70ddf637
    • Rasmus Villemoes's avatar
      mm: madvise: complete input validation before taking lock · 84d96d89
      Rasmus Villemoes authored
      In madvise(), there doesn't seem to be any reason for taking the
      &current->mm->mmap_sem before start and len_in have been validated.
      Incidentally, this removes the need for the out: label.
      
      [akpm@linux-foundation.org: s/out_plug/out/, per David]
      Signed-off-by: default avatarRasmus Villemoes <linux@rasmusvillemoes.dk>
      Acked-by: default avatarKOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Acked-by: default avatarDavid Rientjes <rientjes@google.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      84d96d89
    • David Rientjes's avatar
      mm, hotplug: avoid compiling memory hotremove functions when disabled · 4edd7cef
      David Rientjes authored
      __remove_pages() is only necessary for CONFIG_MEMORY_HOTREMOVE.  PowerPC
      pseries will return -EOPNOTSUPP if unsupported.
      
      Adding an #ifdef causes several other functions it depends on to also
      become unnecessary, which saves in .text when disabled (it's disabled in
      most defconfigs besides powerpc, including x86).  remove_memory_block()
      becomes static since it is not referenced outside of
      drivers/base/memory.c.
      
      Build tested on x86 and powerpc with CONFIG_MEMORY_HOTREMOVE both enabled
      and disabled.
      Signed-off-by: default avatarDavid Rientjes <rientjes@google.com>
      Acked-by: default avatarToshi Kani <toshi.kani@hp.com>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
      Cc: Wen Congyang <wency@cn.fujitsu.com>
      Cc: Tang Chen <tangchen@cn.fujitsu.com>
      Cc: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      4edd7cef
    • Toshi Kani's avatar
      mm: change __remove_pages() to call release_mem_region_adjustable() · fe74ebb1
      Toshi Kani authored
      Change __remove_pages() to call release_mem_region_adjustable().  This
      allows a requested memory range to be released from the iomem_resource
      table even if it does not match exactly to an resource entry but still
      fits into.  The resource entries initialized at bootup usually cover the
      whole contiguous memory ranges and may not necessarily match with the
      size of memory hot-delete requests.
      
      If release_mem_region_adjustable() failed, __remove_pages() emits a
      warning message and continues to proceed as it was the case with
      release_mem_region().  release_mem_region(), which is defined to
      __release_region(), emits a warning message and returns no error since a
      void function.
      Signed-off-by: default avatarToshi Kani <toshi.kani@hp.com>
      Reviewed-by : Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
      Acked-by: default avatarDavid Rientjes <rientjes@google.com>
      Cc: Ram Pai <linuxram@us.ibm.com>
      Cc: T Makphaibulchoke <tmac@hp.com>
      Cc: Wen Congyang <wency@cn.fujitsu.com>
      Cc: Tang Chen <tangchen@cn.fujitsu.com>
      Cc: Jiang Liu <jiang.liu@huawei.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      fe74ebb1
    • Toshi Kani's avatar
      resource: add release_mem_region_adjustable() · 825f787b
      Toshi Kani authored
      Add release_mem_region_adjustable(), which releases a requested region
      from a currently busy memory resource.  This interface adjusts the
      matched memory resource accordingly even if the requested region does
      not match exactly but still fits into.
      
      This new interface is intended for memory hot-delete.  During bootup,
      memory resources are inserted from the boot descriptor table, such as
      EFI Memory Table and e820.  Each memory resource entry usually covers
      the whole contigous memory range.  Memory hot-delete request, on the
      other hand, may target to a particular range of memory resource, and its
      size can be much smaller than the whole contiguous memory.  Since the
      existing release interfaces like __release_region() require a requested
      region to be exactly matched to a resource entry, they do not allow a
      partial resource to be released.
      
      This new interface is restrictive (i.e.  release under certain
      conditions), which is consistent with other release interfaces,
      __release_region() and __release_resource().  Additional release
      conditions, such as an overlapping region to a resource entry, can be
      supported after they are confirmed as valid cases.
      
      There is no change to the existing interfaces since their restriction is
      valid for I/O resources.
      
      [akpm@linux-foundation.org: use GFP_ATOMIC under write_lock()]
      [akpm@linux-foundation.org: switch back to GFP_KERNEL, less buggily]
      [akpm@linux-foundation.org: remove unneeded and wrong kfree(), per Toshi]
      Signed-off-by: default avatarToshi Kani <toshi.kani@hp.com>
      Reviewed-by : Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
      Cc: David Rientjes <rientjes@google.com>
      Reviewed-by: default avatarRam Pai <linuxram@us.ibm.com>
      Cc: T Makphaibulchoke <tmac@hp.com>
      Cc: Wen Congyang <wency@cn.fujitsu.com>
      Cc: Tang Chen <tangchen@cn.fujitsu.com>
      Cc: Jiang Liu <jiang.liu@huawei.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      825f787b
    • Toshi Kani's avatar
      resource: add __adjust_resource() for internal use · ae8e3a91
      Toshi Kani authored
      Add __adjust_resource(), which is called by adjust_resource() internally
      after the resource_lock is held.  There is no interface change to
      adjust_resource().  This change allows other functions to call
      __adjust_resource() internally while the resource_lock is held.
      Signed-off-by: default avatarToshi Kani <toshi.kani@hp.com>
      Reviewed-by: default avatarYasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
      Acked-by: default avatarDavid Rientjes <rientjes@google.com>
      Cc: Ram Pai <linuxram@us.ibm.com>
      Cc: T Makphaibulchoke <tmac@hp.com>
      Cc: Wen Congyang <wency@cn.fujitsu.com>
      Cc: Tang Chen <tangchen@cn.fujitsu.com>
      Cc: Jiang Liu <jiang.liu@huawei.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      ae8e3a91
    • Srivatsa S. Bhat's avatar
      mm: rewrite the comment over migrate_pages() more comprehensibly · c73e5c9c
      Srivatsa S. Bhat authored
      The comment over migrate_pages() looks quite weird, and makes it hard to
      grasp what it is trying to say.  Rewrite it more comprehensibly.
      Signed-off-by: default avatarSrivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
      Acked-by: default avatarChristoph Lameter <cl@linux.com>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Hugh Dickins <hughd@google.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      c73e5c9c
    • Minchan Kim's avatar
      THP: fix comment about memory barrier · 52f37629
      Minchan Kim authored
      Currently the memory barrier in __do_huge_pmd_anonymous_page doesn't
      work.  Because lru_cache_add_lru uses pagevec so it could miss spinlock
      easily so above rule was broken so user might see inconsistent data.
      
      I was not first person who pointed out the problem.  Mel and Peter
      pointed out a few months ago and Peter pointed out further that even
      spin_lock/unlock can't make sure of it:
      
        http://marc.info/?t=134333512700004
      
      	In particular:
      
              	*A = a;
              	LOCK
              	UNLOCK
              	*B = b;
      
      	may occur as:
      
              	LOCK, STORE *B, STORE *A, UNLOCK
      
      At last, Hugh pointed out that even we don't need memory barrier in
      there because __SetPageUpdate already have done it from Nick's commit
      0ed361de ("mm: fix PageUptodate data race") explicitly.
      
      So this patch fixes comment on THP and adds same comment for
      do_anonymous_page, too because everybody except Hugh was missing that.
      It means we need a comment about that.
      Signed-off-by: default avatarMinchan Kim <minchan@kernel.org>
      Acked-by: default avatarAndrea Arcangeli <aarcange@redhat.com>
      Acked-by: default avatarDavid Rientjes <rientjes@google.com>
      Acked-by: default avatarKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Hugh Dickins <hughd@google.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      52f37629
    • Yijing Wang's avatar
      mm: remove CONFIG_HOTPLUG ifdefs · f1cb0879
      Yijing Wang authored
      CONFIG_HOTPLUG is going away as an option, cleanup CONFIG_HOTPLUG
      ifdefs in mm files.
      Signed-off-by: default avatarYijing Wang <wangyijing@huawei.com>
      Acked-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      Acked-by: default avatarRik van Riel <riel@redhat.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      f1cb0879
    • Michel Lespinasse's avatar
      mm/memcontrol.c: remove unnecessary ; · 573b400d
      Michel Lespinasse authored
      Just a trivial issue I stumbled on while doing something else...
      Signed-off-by: default avatarMichel Lespinasse <walken@google.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      573b400d
    • Andrew Shewmaker's avatar
      mm: reinititalise user and admin reserves if memory is added or removed · 1640879a
      Andrew Shewmaker authored
      Alter the admin and user reserves of the previous patches in this series
      when memory is added or removed.
      
      If memory is added and the reserves have been eliminated or increased
      above the default max, then we'll trust the admin.
      
      If memory is removed and there isn't enough free memory, then we need to
      reset the reserves.
      
      Otherwise keep the reserve set by the admin.
      
      The reserve reset code is the same as the reserve initialization code.
      
      I tested hot addition and removal by triggering it via sysfs.  The
      reserves shrunk when they were set high and memory was removed.  They
      were reset higher when memory was added again.
      
      [akpm@linux-foundation.org: use register_hotmemory_notifier()]
      [akpm@linux-foundation.org: init_user_reserve() and init_admin_reserve can no longer be __meminit]
      [fengguang.wu@intel.com: make init_reserve_notifier() static]
      [akpm@linux-foundation.org: coding-style fixes]
      Signed-off-by: default avatarAndrew Shewmaker <agshew@gmail.com>
      Signed-off-by: default avatarFengguang Wu <fengguang.wu@intel.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      1640879a
    • Andrew Shewmaker's avatar
      mm: replace hardcoded 3% with admin_reserve_pages knob · 4eeab4f5
      Andrew Shewmaker authored
      Add an admin_reserve_kbytes knob to allow admins to change the hardcoded
      memory reserve to something other than 3%, which may be multiple
      gigabytes on large memory systems.  Only about 8MB is necessary to
      enable recovery in the default mode, and only a few hundred MB are
      required even when overcommit is disabled.
      
      This affects OVERCOMMIT_GUESS and OVERCOMMIT_NEVER.
      
      admin_reserve_kbytes is initialized to min(3% free pages, 8MB)
      
      I arrived at 8MB by summing the RSS of sshd or login, bash, and top.
      
      Please see first patch in this series for full background, motivation,
      testing, and full changelog.
      
      [akpm@linux-foundation.org: coding-style fixes]
      [akpm@linux-foundation.org: make init_admin_reserve() static]
      Signed-off-by: default avatarAndrew Shewmaker <agshew@gmail.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      4eeab4f5
    • Andrew Shewmaker's avatar
      mm: limit growth of 3% hardcoded other user reserve · c9b1d098
      Andrew Shewmaker authored
      Add user_reserve_kbytes knob.
      
      Limit the growth of the memory reserved for other user processes to
      min(3% current process size, user_reserve_pages).  Only about 8MB is
      necessary to enable recovery in the default mode, and only a few hundred
      MB are required even when overcommit is disabled.
      
      user_reserve_pages defaults to min(3% free pages, 128MB)
      
      I arrived at 128MB by taking the max VSZ of sshd, login, bash, and top ...
      then adding the RSS of each.
      
      This only affects OVERCOMMIT_NEVER mode.
      
      Background
      
      1. user reserve
      
      __vm_enough_memory reserves a hardcoded 3% of the current process size for
      other applications when overcommit is disabled.  This was done so that a
      user could recover if they launched a memory hogging process.  Without the
      reserve, a user would easily run into a message such as:
      
      bash: fork: Cannot allocate memory
      
      2. admin reserve
      
      Additionally, a hardcoded 3% of free memory is reserved for root in both
      overcommit 'guess' and 'never' modes.  This was intended to prevent a
      scenario where root-cant-log-in and perform recovery operations.
      
      Note that this reserve shrinks, and doesn't guarantee a useful reserve.
      
      Motivation
      
      The two hardcoded memory reserves should be updated to account for current
      memory sizes.
      
      Also, the admin reserve would be more useful if it didn't shrink too much.
      
      When the current code was originally written, 1GB was considered
      "enterprise".  Now the 3% reserve can grow to multiple GB on large memory
      systems, and it only needs to be a few hundred MB at most to enable a user
      or admin to recover a system with an unwanted memory hogging process.
      
      I've found that reducing these reserves is especially beneficial for a
      specific type of application load:
      
       * single application system
       * one or few processes (e.g. one per core)
       * allocating all available memory
       * not initializing every page immediately
       * long running
      
      I've run scientific clusters with this sort of load.  A long running job
      sometimes failed many hours (weeks of CPU time) into a calculation.  They
      weren't initializing all of their memory immediately, and they weren't
      using calloc, so I put systems into overcommit 'never' mode.  These
      clusters run diskless and have no swap.
      
      However, with the current reserves, a user wishing to allocate as much
      memory as possible to one process may be prevented from using, for
      example, almost 2GB out of 32GB.
      
      The effect is less, but still significant when a user starts a job with
      one process per core.  I have repeatedly seen a set of processes
      requesting the same amount of memory fail because one of them could not
      allocate the amount of memory a user would expect to be able to allocate.
      For example, Message Passing Interfce (MPI) processes, one per core.  And
      it is similar for other parallel programming frameworks.
      
      Changing this reserve code will make the overcommit never mode more useful
      by allowing applications to allocate nearly all of the available memory.
      
      Also, the new admin_reserve_kbytes will be safer than the current behavior
      since the hardcoded 3% of available memory reserve can shrink to something
      useless in the case where applications have grabbed all available memory.
      
      Risks
      
      * "bash: fork: Cannot allocate memory"
      
        The downside of the first patch-- which creates a tunable user reserve
        that is only used in overcommit 'never' mode--is that an admin can set
        it so low that a user may not be able to kill their process, even if
        they already have a shell prompt.
      
        Of course, a user can get in the same predicament with the current 3%
        reserve--they just have to launch processes until 3% becomes negligible.
      
      * root-cant-log-in problem
      
        The second patch, adding the tunable rootuser_reserve_pages, allows
        the admin to shoot themselves in the foot by setting it too small.  They
        can easily get the system into a state where root-can't-log-in.
      
        However, the new admin_reserve_kbytes will be safer than the current
        behavior since the hardcoded 3% of available memory reserve can shrink
        to something useless in the case where applications have grabbed all
        available memory.
      
      Alternatives
      
       * Memory cgroups provide a more flexible way to limit application memory.
      
         Not everyone wants to set up cgroups or deal with their overhead.
      
       * We could create a fourth overcommit mode which provides smaller reserves.
      
         The size of useful reserves may be drastically different depending
         on the whether the system is embedded or enterprise.
      
       * Force users to initialize all of their memory or use calloc.
      
         Some users don't want/expect the system to overcommit when they malloc.
         Overcommit 'never' mode is for this scenario, and it should work well.
      
      The new user and admin reserve tunables are simple to use, with low
      overhead compared to cgroups.  The patches preserve current behavior where
      3% of memory is less than 128MB, except that the admin reserve doesn't
      shrink to an unusable size under pressure.  The code allows admins to tune
      for embedded and enterprise usage.
      
      FAQ
      
       * How is the root-cant-login problem addressed?
         What happens if admin_reserve_pages is set to 0?
      
         Root is free to shoot themselves in the foot by setting
         admin_reserve_kbytes too low.
      
         On x86_64, the minimum useful reserve is:
           8MB for overcommit 'guess'
         128MB for overcommit 'never'
      
         admin_reserve_pages defaults to min(3% free memory, 8MB)
      
         So, anyone switching to 'never' mode needs to adjust
         admin_reserve_pages.
      
       * How do you calculate a minimum useful reserve?
      
         A user or the admin needs enough memory to login and perform
         recovery operations, which includes, at a minimum:
      
         sshd or login + bash (or some other shell) + top (or ps, kill, etc.)
      
         For overcommit 'guess', we can sum resident set sizes (RSS)
         because we only need enough memory to handle what the recovery
         programs will typically use. On x86_64 this is about 8MB.
      
         For overcommit 'never', we can take the max of their virtual sizes (VSZ)
         and add the sum of their RSS. We use VSZ instead of RSS because mode
         forces us to ensure we can fulfill all of the requested memory allocations--
         even if the programs only use a fraction of what they ask for.
         On x86_64 this is about 128MB.
      
         When swap is enabled, reserves are useful even when they are as
         small as 10MB, regardless of overcommit mode.
      
         When both swap and overcommit are disabled, then the admin should
         tune the reserves higher to be absolutley safe. Over 230MB each
         was safest in my testing.
      
       * What happens if user_reserve_pages is set to 0?
      
         Note, this only affects overcomitt 'never' mode.
      
         Then a user will be able to allocate all available memory minus
         admin_reserve_kbytes.
      
         However, they will easily see a message such as:
      
         "bash: fork: Cannot allocate memory"
      
         And they won't be able to recover/kill their application.
         The admin should be able to recover the system if
         admin_reserve_kbytes is set appropriately.
      
       * What's the difference between overcommit 'guess' and 'never'?
      
         "Guess" allows an allocation if there are enough free + reclaimable
         pages. It has a hardcoded 3% of free pages reserved for root.
      
         "Never" allows an allocation if there is enough swap + a configurable
         percentage (default is 50) of physical RAM. It has a hardcoded 3% of
         free pages reserved for root, like "Guess" mode. It also has a
         hardcoded 3% of the current process size reserved for additional
         applications.
      
       * Why is overcommit 'guess' not suitable even when an app eventually
         writes to every page? It takes free pages, file pages, available
         swap pages, reclaimable slab pages into consideration. In other words,
         these are all pages available, then why isn't overcommit suitable?
      
         Because it only looks at the present state of the system. It
         does not take into account the memory that other applications have
         malloced, but haven't initialized yet. It overcommits the system.
      
      Test Summary
      
      There was little change in behavior in the default overcommit 'guess'
      mode with swap enabled before and after the patch. This was expected.
      
      Systems run most predictably (i.e. no oom kills) in overcommit 'never'
      mode with swap enabled. This also allowed the most memory to be allocated
      to a user application.
      
      Overcommit 'guess' mode without swap is a bad idea. It is easy to
      crash the system. None of the other tested combinations crashed.
      This matches my experience on the Roadrunner supercomputer.
      
      Without the tunable user reserve, a system in overcommit 'never' mode
      and without swap does not allow the admin to recover, although the
      admin can.
      
      With the new tunable reserves, a system in overcommit 'never' mode
      and without swap can be configured to:
      
      1. maximize user-allocatable memory, running close to the edge of
      recoverability
      
      2. maximize recoverability, sacrificing allocatable memory to
      ensure that a user cannot take down a system
      
      Test Description
      
      Fedora 18 VM - 4 x86_64 cores, 5725MB RAM, 4GB Swap
      
      System is booted into multiuser console mode, with unnecessary services
      turned off. Caches were dropped before each test.
      
      Hogs are user memtester processes that attempt to allocate all free memory
      as reported by /proc/meminfo
      
      In overcommit 'never' mode, memory_ratio=100
      
      Test Results
      
      3.9.0-rc1-mm1
      
      Overcommit | Swap | Hogs | MB Got/Wanted | OOMs | User Recovery | Admin Recovery
      ----------   ----   ----   -------------   ----   -------------   --------------
      guess        yes    1      5432/5432       no     yes             yes
      guess        yes    4      5444/5444       1      yes             yes
      guess        no     1      5302/5449       no     yes             yes
      guess        no     4      -               crash  no              no
      
      never        yes    1      5460/5460       1      yes             yes
      never        yes    4      5460/5460       1      yes             yes
      never        no     1      5218/5432       no     no              yes
      never        no     4      5203/5448       no     no              yes
      
      3.9.0-rc1-mm1-tunablereserves
      
      User and Admin Recovery show their respective reserves, if applicable.
      
      Overcommit | Swap | Hogs | MB Got/Wanted | OOMs | User Recovery | Admin Recovery
      ----------   ----   ----   -------------   ----   -------------   --------------
      guess        yes    1      5419/5419       no     - yes           8MB yes
      guess        yes    4      5436/5436       1      - yes           8MB yes
      guess        no     1      5440/5440       *      - yes           8MB yes
      guess        no     4      -               crash  - no            8MB no
      
      * process would successfully mlock, then the oom killer would pick it
      
      never        yes    1      5446/5446       no     10MB yes        20MB yes
      never        yes    4      5456/5456       no     10MB yes        20MB yes
      never        no     1      5387/5429       no     128MB no        8MB barely
      never        no     1      5323/5428       no     226MB barely    8MB barely
      never        no     1      5323/5428       no     226MB barely    8MB barely
      
      never        no     1      5359/5448       no     10MB no         10MB barely
      
      never        no     1      5323/5428       no     0MB no          10MB barely
      never        no     1      5332/5428       no     0MB no          50MB yes
      never        no     1      5293/5429       no     0MB no          90MB yes
      
      never        no     1      5001/5427       no     230MB yes       338MB yes
      never        no     4*     4998/5424       no     230MB yes       338MB yes
      
      * more memtesters were launched, able to allocate approximately another 100MB
      
      Future Work
      
       - Test larger memory systems.
      
       - Test an embedded image.
      
       - Test other architectures.
      
       - Time malloc microbenchmarks.
      
       - Would it be useful to be able to set overcommit policy for
         each memory cgroup?
      
       - Some lines are slightly above 80 chars.
         Perhaps define a macro to convert between pages and kb?
         Other places in the kernel do this.
      
      [akpm@linux-foundation.org: coding-style fixes]
      [akpm@linux-foundation.org: make init_user_reserve() static]
      Signed-off-by: default avatarAndrew Shewmaker <agshew@gmail.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      c9b1d098
    • Andrew Morton's avatar
      kernel/cpuset.c: use register_hotmemory_notifier() · d8f10cb3
      Andrew Morton authored
      Use the new interface, remove one ifdef.  No code size changes.
      
      We could/should have been using __meminit/__meminitdata here but there's
      now no point in doing that because all this code is elided at compile time.
      
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Tejun Heo <tj@kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      d8f10cb3
    • Andrew Morton's avatar
      fs/proc/kcore.c: use register_hotmemory_notifier() · 3c743a7f
      Andrew Morton authored
      Saves an ifdef, no code size changes
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      3c743a7f
    • Andrew Morton's avatar
      drivers/base/node.c: switch to register_hotmemory_notifier() · 6e259e7d
      Andrew Morton authored
      Squishes a warning which my change to hotplug_memory_notifier() added.
      
      I want to keep that warning, because it is punishment for failnig to check
      the hotplug_memory_notifier() return value.
      
      Cc: Greg KH <greg@kroah.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      6e259e7d
    • Andrew Morton's avatar
      mm/slub.c: use register_hotmemory_notifier() · 3ac38faa
      Andrew Morton authored
      Squishes a statement-with-no-effect warning, removes some ifdefs and
      shrinks .text by 2 bytes.
      
      Note that this code fails to check for blocking_notifier_chain_register()
      failures.
      
      Cc: Pekka Enberg <penberg@kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      3ac38faa
    • Andrew Morton's avatar
      ipc/util.c: use register_hotmemory_notifier() · 8f68fa2d
      Andrew Morton authored
      Squishes a statement-with-no-effect warning, removes some ifdefs and
      shrinks .text by one byte!
      
      Note that this code fails to check for blocking_notifier_chain_register()
      failures.
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      8f68fa2d
    • Andrew Morton's avatar
      include/linux/memory.h: implement register_hotmemory_notifier() · f02c6968
      Andrew Morton authored
      When CONFIG_MEMORY_HOTPLUG=n, we don't want the memory-hotplug notifier
      handlers to be included in the .o files, for space reasons.
      
      The existing hotplug_memory_notifier() tries to handle this but testing
      with gcc-4.4.4 shows that it doesn't work - the hotplug functions are
      still present in the .o files.
      
      So implement a new register_hotmemory_notifier() which is a copy of
      register_hotcpu_notifier(), and which actually works as desired.
      hotplug_memory_notifier() and register_memory_notifier() callsites
      should be converted to use this new register_hotmemory_notifier().
      
      While we're there, let's repair the existing hotplug_memory_notifier():
      it simply stomps on the register_memory_notifier() return value, so
      well-behaved code cannot check for errors.  Apparently non of the
      existing callers were well-behaved :(
      
      Cc: Andrew Shewmaker <agshew@gmail.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      f02c6968
    • Cody P Schafer's avatar
      powerpc/mm/numa: use setup_nr_node_ids() instead of opencoding. · f9d531b8
      Cody P Schafer authored
      [sfr@canb.auug.org.au: add missing semicolon]
      Signed-off-by: default avatarCody P Schafer <cody@linux.vnet.ibm.com>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Signed-off-by: default avatarStephen Rothwell <sfr@canb.auug.org.au>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      f9d531b8
    • Cody P Schafer's avatar
      x86/mm/numa: use setup_nr_node_ids() instead of opencoding. · d2ad351e
      Cody P Schafer authored
      Signed-off-by: default avatarCody P Schafer <cody@linux.vnet.ibm.com>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Acked-by: default avatarYinghai Lu <yinghai@kernel.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      d2ad351e
    • Cody P Schafer's avatar
      page_alloc: make setup_nr_node_ids() usable for arch init code · f9872caf
      Cody P Schafer authored
      powerpc and x86 were opencoding copies of setup_nr_node_ids(), which
      page_alloc provides but makes static.  Make it avaliable to the archs in
      linux/mm.h.
      Signed-off-by: default avatarCody P Schafer <cody@linux.vnet.ibm.com>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      f9872caf
    • Russ Anderson's avatar
      mm: speedup in __early_pfn_to_nid · 7c243c71
      Russ Anderson authored
      When booting on a large memory system, the kernel spends considerable
      time in memmap_init_zone() setting up memory zones.  Analysis shows
      significant time spent in __early_pfn_to_nid().
      
      The routine memmap_init_zone() checks each PFN to verify the nid is
      valid.  __early_pfn_to_nid() sequentially scans the list of pfn ranges
      to find the right range and returns the nid.  This does not scale well.
      On a 4 TB (single rack) system there are 308 memory ranges to scan.  The
      higher the PFN the more time spent sequentially spinning through memory
      ranges.
      
      Since memmap_init_zone() increments pfn, it will almost always be
      looking for the same range as the previous pfn, so check that range
      first.  If it is in the same range, return that nid.  If not, scan the
      list as before.
      
      A 4 TB (single rack) UV1 system takes 512 seconds to get through the
      zone code.  This performance optimization reduces the time by 189
      seconds, a 36% improvement.
      
      A 2 TB (single rack) UV2 system goes from 212.7 seconds to 99.8 seconds,
      a 112.9 second (53%) reduction.
      
      [akpm@linux-foundation.org: make the statics __meminitdata]
      [akpm@linux-foundation.org: fix comment formatting]
      [akpm@linux-foundation.org: fix ia64, per yinghai]
      [akpm@linux-foundation.org: add missing semicolon, per Tony]
      Signed-off-by: default avatarRuss Anderson <rja@sgi.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      Tested-by: default avatar"Luck, Tony" <tony.luck@intel.com>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Lin Feng <linfeng@cn.fujitsu.com>
      Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      7c243c71