From: Rusty Russell <rusty@rustcorp.com.au>

1) Create an in_sched_functions() function in sched.c and make the
   archs use it.  (Two archs have wchan #if 0'd out: left them alone).

2) Move __sched from linux/init.h to linux/sched.h and add comment.

3) Rename __scheduling_functions_start_here/end_here to __sched_text_start/end.

Thanks to wli and Sam Ravnborg for clue donation.

With strange workloads which do a lot of quick truncation on small filesystems
it is possible to get into a situation where there are free blocks on the
disk, but they are not allocatable at this time due to their having been freed
up in the current JBD transaction.  Applications get unexpected ENOSPC errors.

We can fix that with this patch, originally by Andreas Dilger which forces a
single commit+retry when an ENOSPC is encountered.

From: Mikael Pettersson <mikpe@csd.uu.se>

The real problem is that SMP with nmi_watchdog=2 initialises the lapic NMI
watchdog but doesn't check it and therefore doesn't reduce nmi_hz.  This is
an SMP bug.

The patch changes smpboot.c to do a check_nmi_watchdog() at the appropriate
place, which fixes the high NMI frequency problem w/o changing anything
else.  I've verified that it solves the problem on my MP-capable UP box.

From: Philippe Elie <phil.el@wanadoo.fr>

With nmi_watchdog=2 and a P4 ht box the nmi is occurring only on logical
processor 0, it's better to get it on both.

With this patch, on x86 SMP and nmi_watchdog=2, nmi interupts occur at 1000
hz (if the cpu is loaded) not at the intended 1 hz rate but that's a distinct
problem.

From: Raghavan <raghav@in.ibm.com>,
      me

I am submitting a patch that fixes 2 race conditions in the 32 bit ioctl
emulation code.(fs/compat.c) Since the search is not locked; when a
ioctl_trans structure is deleted, corruption can occur.

The following scenarios discuss the race conditions:

1) When the search is hapenning, if any ioctl_trans structure gets
   deleted; then rather than searching the hash table, the code will start
   searching the free list.

while (t && t->cmd != cmd)
        -

From: Ralf Baechle <ralf@linux-mips.org>

Get to work under 2.6 sorting out the giant mess this has been.  Further
cleanups would require a full crapectomy of wd33c93.c itself ...

From: Ralf Baechle <ralf@linux-mips.org>

This driver has been obsoleted by drivers/serial/dz.c.

From: Ralf Baechle <ralf@linux-mips.org>

From: Ralf Baechle <ralf@linux-mips.org>

Add missing definition PORT_IP22ZILOG which is need by ip22zilog driver.

From: Ralf Baechle <ralf@linux-mips.org>

This patch adds the GBE video driver for the video system in SGI IP32
aka O2 and it's i386-based equivalent the Visual Workstation.
                                                                              
This driver obsoletes sgivwfb.c; but I'd prefer to play safe and remove
it after some additional time, just in case.

From: Ralf Baechle <ralf@linux-mips.org>

The RM200's onboard video really is a plain old boring Cyrix PCI card.

From: Ralf Baechle <ralf@linux-mips.org>

Make the driver for Newport aka XL work in 2.6.

From: Ralf Baechle <ralf@linux-mips.org>

CONFIG_MIPS is always defined, for 32-bit and 64-bit.

From: Ralf Baechle <ralf@linux-mips.org>

From: Ralf Baechle <ralf@linux-mips.org>

 - Kconfig cleanups:
    - enable DMA_NONCOHERENT, DMA_COHERENT or DMA_IP27 via reverse dependencies
    - untangle VRC4171 / VRC4173 selection
    - R10000 support enables PREFETCH
    - SEAD needs IRQ_CPU
 - Update defconfig against latest Kconfig files.
 - Fix computation of return address if syscall number was out of range
 - Add power managment hooks in signal code.
 - Don't try to handle signals when previous context was not in user mode.
 - Fix serial interface setup for VR41xx systems.
 - Build fixes after CLEAR_BITMAP changed name.
 - Removes bogus comment from <asm/checksum.h>
 - <asm/hdreg.h> is dead.
 - Start collecting common definitions for PMON firmware in <asm/pmon.h>
 - Define ARCH_MIN_TASKALIGN to 8; we have 64-bit members even on 32-bit
   kernels if we're running on MIPS II or better.

From: Jan Kara <jack@ucw.cz>

Attached patch should fix reported deadlock in journalled quota code.
quotactl() call was violating the locking rules and didn't start transaction
when it should.

From: <raven@themaw.net>

  Found a couple of symbols not exported that were needed by the ext3.ko
  module.

From: Srivatsa Vaddagiri <vatsa@in.ibm.com>

Noticed that migration_thread can examine "kthread_should_stop()?" without
setting its state to TASK_INTERRUPTIBLE first.  This can cause kthread_stop
on that thread to block forever ...

P.S 	- I assumed that having the task state set to TASK_INTERRUTIBLE
	  while it is doing active_load_balance is fine. It seemed to be
	  the case earlier also.

From: Srivatsa Vaddagiri <vatsa@in.ibm.com>

Fix the race in sys_sched_getaffinity.  Patch below takes cpu_hotplug lock
before reading cpus_allowed mask of a task.

From: Srivatsa Vaddagiri <vatsa@in.ibm.com>

migrate_all_tasks is currently run with rest of the machine stopped.
It iterates thr' the complete task table, turning off cpu affinity of any task
that it finds affine to the dying cpu. Depending on the task table
size this can take considerable time. All this time machine is stopped, doing
nothing.

Stopping the machine for such extended periods can be avoided if we do
task migration in CPU_DEAD notification and that's precisely what this patch
does.

The patch puts idle task to the _front_ of the dying CPU's runqueue at the 
highest priority possible. This cause idle thread to run _immediately_ after
kstopmachine thread yields. Idle thread notices that its cpu is offline and
dies quickly. Task migration can then be done at leisure in CPU_DEAD
notification, when rest of the CPUs are running.

Some advantages with this approach are:

	- More scalable. Predicatable amout of time that machine is stopped.
	- No changes to hot path/core code. We are just exploiting scheduler
	  rules which runs the next high-priority task on the runqueue. Also
	  since I put idle task to the _front_ of the runqueue, there
	  are no races when a equally high priority task is woken up
	  and added to the runqueue. It gets in at the back of the runqueue,
	  _after_ idle task!
	- cpu_is_offline check that is presenty required in try_to_wake_up,
	  idle_balance and rebalance_tick can be removed, thus speeding them
	  up a bit

From: Srivatsa Vaddagiri <vatsa@in.ibm.com>

  Rusty mentioned that the unlikely hints against cpu_is_offline is
  redundant since the macro already has that hint.  Patch below removes those
  redundant hints I added.

From: Nick Piggin <nickpiggin@yahoo.com.au>

The SMT wake_idle code really wants to look at a non-local CPU's domain in
order to check for idle siblings.

So change the domain attachment code a little bit so we continue to hold a
runqueue's lock while attaching a new domain.  This means the locking rules
have changed to: you may access your own domain without any lock, you must
hold a remote runqueue's lock in order to view its domain.

From: Nick Piggin <nickpiggin@yahoo.com.au>

This actually does produce better code, especially under the locked
section.

Turns a conditional + unconditional jump under the lock in the unlikely
case into a cmov outside the lock.

From: Nick Piggin <nickpiggin@yahoo.com.au>

It makes NEWLY_IDLE balances cause find_busiest_group return the busiest
available group even if there isn't an imbalance.  Basically - try a bit
harder to prevent schedule emptying the runqueue.

It is quite aggressive, but that isn't so bad because we don't (by default)
do NEWLY_IDLE balancing across NUMA nodes, and NEWLY_IDLE balancing is always
restricted to cache_hot tasks.

It picked up a little bit of idle time that dbt2-pgsql was seeing...

From: Ingo Molnar <mingo@elte.hu>

Implement balancing during clone().  It does the following things:

- introduces SD_BALANCE_CLONE that can serve as a tool for an
  architecture to limit the search-idlest-CPU scope on clone().
  E.g. the 512-CPU systems should rather not enable this.

- uses the highest sd for the imbalance_pct, not this_rq (which didnt
  make sense).

- unifies balance-on-exec and balance-on-clone via the find_idlest_cpu()
  function. Gets rid of sched_best_cpu() which was still a bit
  inconsistent IMO, it used 'min_load < load' as a condition for
  balancing - while a more correct approach would be to use half of the
  imbalance_pct, like passive balancing does.

- the patch also reintroduces the possibility to do SD_BALANCE_EXEC on
  SMP systems, and activates it - to get testing.

- NOTE: there's one thing in this patch that is slightly unclean: i
  introduced wake_up_forked_thread. I did this to make it easier to get
  rid of this patch later (wake_up_forked_process() has lots of
  dependencies in various architectures). If this capability remains in
  the kernel then i'll clean it up and introduce one function for
  wake_up_forked_process/thread.

- NOTE2: i added the SD_BALANCE_CLONE flag to the NUMA CPU template too.
  Some NUMA architectures probably want to disable this.

From: Ingo Molnar <mingo@elte.hu>

This does the source/target cleanup.  This is a no-functionality patch which
also adds more comments to explain these functions.

From: Nick Piggin <nickpiggin@yahoo.com.au>

This patch starts to balance woken processes when half the relevant domain's
imbalance_pct is reached.  Previously balancing would start after a small,
constant difference in waker/wakee runqueue loads was reached, which would
cause too much process movement when there are lots of processes running.

It also turns wake balancing into a domain flag while previously it was always
on.  Now sched domains can "soft partition" an SMP system without using
processor affinities.

From: Ingo Molnar <mingo@elte.hu>

This re-adds cleanups which were lost in splitups of an earlier patch.

From: Nick Piggin <nickpiggin@yahoo.com.au>

The attached patch is required to work correctly with the CPU hotplug
framework.  John Hawkes reports successful booting with this.

From: Ingo Molnar <mingo@elte.hu>

The attached patch extends sync wakeups to the process sys_exit() path too:
the chldwait wakeup can be done sync, since we know that the process is
going to exit (and thus deschedule).

The most visible effect of this change is strace's behavior on SMP systems:
it now stays on a single CPU, together with the traced child.  (previously
it would run in parallel to the child, bouncing around madly.)

From: Ingo Molnar <mingo@elte.hu>

Helper function for later patches

From: Ingo Molnar <mingo@elte.hu>

Uninline things

From: Nick Piggin <nickpiggin@yahoo.com.au>

Minor cleanups from Ingo's patch including task_hot (do it right in
try_to_wake_up too).

From: Nick Piggin <nickpiggin@yahoo.com.au>

De-racify the sched domain setup code.  This involves creating a dummy
"init" domain during sched_init (which is called early).

When topology information becomes available, the sched domains are then
built and attached.  The attach mechanism is asynchronous and uses the
migration threads, which perform the switch with interrupts off.  This is a
quiescent state, so domains can still be lockless on the read side.  It
also allows us to change the domains at runtime without much more work. 
This is something SGI is interested in to elegantly do soft partitioning of
their systems without having to use hard cpu affinities (which cause
balancing problems of their own).

The current setup code also has a race somewhere because it is unable to
boot on a 384 CPU system.



From: Anton Blanchard <anton@samba.org>

   This is basically a mindless ppc64 merge of the x86 changes to sched
   domain init code.

   Actually if I produce a sibling_map[] then the x86 code and the ppc64
   will be identical.  Maybe we can merge it.

From: Anton Blanchard <anton@samba.org>

It seems someone has been making trivial changes without using grep.

From: Anton Blanchard <anton@samba.org>

Below are the diffs between the current ppc64 sched init stuff and x86.

- Ignore the POWER5 specific stuff, I dont set up a sibling map yet.
- What should I set cache_hot_time to?

large cpumask typechecking requirements (perhaps useful on x86 as well):
- cpu->cpumask = CPU_MASK_NONE -> cpus_clear(cpu->cpumask);
- cpus_and(nodemask, node_to_cpumask(i), cpu_possible_map) doesnt work,
  need to use a temporary

From: Nick Piggin <nickpiggin@yahoo.com.au>

After the for_each_domain change, the warn here won't trigger, instead it
will oops in the if statement.  Also, make sure we don't pass an empty
cpumask to for_each_cpu.

From: Nick Piggin <nickpiggin@yahoo.com.au>

From: John Hawkes

The following brings up performance on a 64-way Altix.  This system being on
the smaller end of the scale should also be applicable to other NUMA systems.

From: Nick Piggin <nickpiggin@yahoo.com.au>

Imbalance calculations were not right.  This would cause unneeded migration.

From: Nick Piggin <nickpiggin@yahoo.com.au>

Make affine wakes and "passive load balancing" more conservative.  Aggressive
affine wakeups were causing huge regressions in dbt3-pgsql on 8-way non NUMA
systems at OSDL's STP.

From: Rusty Russell <rusty@rustcorp.com.au>

From: Srivatsa Vaddagiri <vatsa@in.ibm.com>
From: Andrew Morton <akpm@osdl.org>
From: Rusty Russell <rusty@rustcorp.com.au>

We want to get rid of lock_cpu_hotplug() in sched_migrate_task.  Found
that lockless migration of execing task is _extremely_ racy.  The
races I hit are described below, alongwith probable solutions.

Task migration done elsewhere should be safe (?) since they either
hold the lock (sys_sched_setaffinity) or are done entirely with preemption 
disabled (load_balance).

   sched_balance_exec does:

	a. disables preemption
	b. finds new_cpu for current
	c. enables preemption
	d. calls sched_migrate_task to migrate current to new_cpu

   and sched_migrate_task does:

	e. task_rq_lock(p)
	f. migrate_task(p, dest_cpu ..)
		(if we have to wait for migration thread)
		g. task_rq_unlock()
		h. wake_up_process(rq->migration_thread)
		i. wait_for_completion()

   Several things can happen here:

	1. new_cpu can go down after h and before migration thread has
	   got around to handle the request

	   ==> we need to add a cpu_is_offline check in __migrate_task

	2. new_cpu can go down between c and d or before f.

	   ===> Even though this case is automatically handled by the above 
	        change (migrate_task being called on a running task, current,
		will delegate migration to migration thread), would it be 
	 	good practice to avoid calling migrate_task in the first place
		itself when dest_cpu is offline. This means adding another
	 	cpu_is_offline check after e in sched_migrate_task

	3. The 'current' task can get preempted _immediately_ after
	   g and when it comes back, task_cpu(p) can be dead. In
	   which case, it is invalid to do wake_up on a non-existent migration 
	   thread.  (rq->migration_thread can be NULL).

	   ===> We should disable preemption thr' g and h

	4. Before migration thread gets around to handle the request, its cpu
	   goes dead. This will leave unhandled migration requests in the dead 
	   cpu. 

	   ===> We need to wakeup sleeping requestors (if any) in CPU_DEAD
	        notification.

I really wonder if we can get rid of these issues by avoiding balancing at 
exec time and instead have it balanced during load_balance ..Alternately
if this is valuable and we want to retain it, I think we still need to
consider a read/write sem, with sched_migrate_task doing down_read_trylock.
This may eliminate the deadlock I hit between cpu_up and CPU_UP_PREPARE 
notification, which had forced me away from r/w sem.

Anyway patch below addresses the above races. Its against 2.6.6-rc2-mm1
and has been tested on a 4way Intel Pentium SMP m/c.


Rusty sez:

Two other changes:
1) I grabbed a reference to the thread, rather than using
preempt_disable().  It's the more obvious way I think.

2) Why the wait_to_die code?  It might be needed if we move tasks after
stop_machine, but for nowI don't see the problem with the migration
thread running on the wrong CPU for a bit: nothing is on this runqueue
so active_load_balance is safe, and __migrate task will be a noop (due
to cpu_is_offline() check).  If there is a problem, your fix is racy,
because we could be preempted immediately afterwards.

So I just stop the kthread then wakeup any remaining...

From: Ingo Molnar <mingo@elte.hu>

The trivial fixes.

- added recent trivial bits from Nick's and my patches.
- hotplug CPU fix
- early init cleanup