• Oleg Nesterov's avatar
    oom: introduce find_lock_task_mm() to fix !mm false positives · dd8e8f40
    Oleg Nesterov authored
    Almost all ->mm == NULL checks in oom_kill.c are wrong.
    
    The current code assumes that the task without ->mm has already released
    its memory and ignores the process.  However this is not necessarily true
    when this process is multithreaded, other live sub-threads can use this
    ->mm.
    
    - Remove the "if (!p->mm)" check in select_bad_process(), it is
      just wrong.
    
    - Add the new helper, find_lock_task_mm(), which finds the live
      thread which uses the memory and takes task_lock() to pin ->mm
    
    - change oom_badness() to use this helper instead of just checking
      ->mm != NULL.
    
    - As David pointed out, select_bad_process() must never choose the
      task without ->mm, but no matter what oom_badness() returns the
      task can be chosen if nothing else has been found yet.
    
      Change oom_badness() to return int, change it to return -1 if
      find_lock_task_mm() fails, and change select_bad_process() to
      check points >= 0.
    
    Note! This patch is not enough, we need more changes.
    
    	- oom_badness() was fixed, but oom_kill_task() still ignores
    	  the task without ->mm
    
    	- oom_forkbomb_penalty() should use find_lock_task_mm() too,
    	  and it also needs other changes to actually find the first
    	  first-descendant children
    
    This will be addressed later.
    
    [kosaki.motohiro@jp.fujitsu.com: use in badness(), __oom_kill_task()]
    Signed-off-by: default avatarOleg Nesterov <oleg@redhat.com>
    Signed-off-by: default avatarDavid Rientjes <rientjes@google.com>
    Signed-off-by: default avatarKOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    dd8e8f40
oom_kill.c 17.9 KB