- 19 Apr, 2011 2 commits
-
-
Venkatesh Pallipadi authored
When a task in a taskgroup sleeps, pick_next_task starts all the way back at the root and picks the task/taskgroup with the min vruntime across all runnable tasks. But when there are many frequently sleeping tasks across different taskgroups, it makes better sense to stay with same taskgroup for its slice period (or until all tasks in the taskgroup sleeps) instead of switching cross taskgroup on each sleep after a short runtime. This helps specifically where taskgroups corresponds to a process with multiple threads. The change reduces the number of CR3 switches in this case. Example: Two taskgroups with 2 threads each which are running for 2ms and sleeping for 1ms. Looking at sched:sched_switch shows: BEFORE: taskgroup_1 threads [5004, 5005], taskgroup_2 threads [5016, 5017] cpu-soaker-5004 [003] 3683.391089 cpu-soaker-5016 [003] 3683.393106 cpu-soaker-5005 [003] 3683.395119 cpu-soaker-5017 [003] 3683.397130 cpu-soaker-5004 [003] 3683.399143 cpu-soaker-5016 [003] 3683.401155 cpu-soaker-5005 [003] 3683.403168 cpu-soaker-5017 [003] 3683.405170 AFTER: taskgroup_1 threads [21890, 21891], taskgroup_2 threads [21934, 21935] cpu-soaker-21890 [003] 865.895494 cpu-soaker-21935 [003] 865.897506 cpu-soaker-21934 [003] 865.899520 cpu-soaker-21935 [003] 865.901532 cpu-soaker-21934 [003] 865.903543 cpu-soaker-21935 [003] 865.905546 cpu-soaker-21891 [003] 865.907548 cpu-soaker-21890 [003] 865.909560 cpu-soaker-21891 [003] 865.911571 cpu-soaker-21890 [003] 865.913582 cpu-soaker-21891 [003] 865.915594 cpu-soaker-21934 [003] 865.917606 Similar problem is there when there are multiple taskgroups and say a task A preempts currently running task B of taskgroup_1. On schedule, pick_next_task can pick an unrelated task on taskgroup_2. Here it would be better to give some preference to task B on pick_next_task. A simple (may be extreme case) benchmark I tried was tbench with 2 tbench client processes with 2 threads each running on a single CPU. Avg throughput across 5 50 sec runs was: BEFORE: 105.84 MB/sec AFTER: 112.42 MB/sec Signed-off-by:Venkatesh Pallipadi <venki@google.com> Acked-by:
Rik van Riel <riel@redhat.com> Signed-off-by:
Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1302802253-25760-1-git-send-email-venki@google.comSigned-off-by:
Ingo Molnar <mingo@elte.hu>
-
Venkatesh Pallipadi authored
Make set_*_buddy() work on non-task sched_entity, to facilitate the use of next_buddy to cache a group entity in cases where one of the tasks within that entity sleeps or gets preempted. set_skip_buddy() was incorrectly comparing the policy of task that is yielding to be not equal to SCHED_IDLE. Yielding should happen even when task yielding is SCHED_IDLE. This change removes the policy check on the yielding task. Signed-off-by:
-
- 18 Apr, 2011 1 commit
-
-
Ingo Molnar authored
Merge reason: the rq locking changes are stable, propagate them into the .40 queue. Signed-off-by:
-
- 14 Apr, 2011 23 commits
-
-
Peter Zijlstra authored
Oleg noticed that need_migrate_task() doesn't need the ->on_cpu check now that ttwu() doesn't do remote enqueues for !->on_rq && ->on_cpu, so remove the helper and replace the single instance with a direct ->on_rq test. Suggested-by:
Oleg Nesterov <oleg@redhat.com> Reviewed-by:
Frank Rowand <frank.rowand@am.sony.com> Signed-off-by:
-
Peter Zijlstra authored
Now that we've removed the rq->lock requirement from the first part of ttwu() and can compute placement without holding any rq->lock, ensure we execute the second half of ttwu() on the actual cpu we want the task to run on. This avoids having to take rq->lock and doing the task enqueue remotely, saving lots on cacheline transfers. As measured using: http://oss.oracle.com/~mason/sembench.c $ for i in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor ; do echo performance > $i; done $ echo 4096 32000 64 128 > /proc/sys/kernel/sem $ ./sembench -t 2048 -w 1900 -o 0 unpatched: run time 30 seconds 647278 worker burns per second patched: run time 30 seconds 816715 worker burns per second Reviewed-by:
-
Peter Zijlstra authored
Factor our helper functions to make the inner workings of try_to_wake_up() more obvious, this also allows for adding remote queues. Reviewed-by:
-
Peter Zijlstra authored
The ttwu_post_activation() code does the core wakeup, it sets TASK_RUNNING and performs wakeup-preemption, so give is a more descriptive name. Reviewed-by:
-
Peter Zijlstra authored
In order to call ttwu_stat() without holding rq->lock we must remove its rq argument. Since we need to change rq stats, account to the local rq instead of the task rq, this is safe since we have IRQs disabled. Reviewed-by:
-
Peter Zijlstra authored
Currently ttwu() does two rq->lock acquisitions, once on the task's old rq, holding it over the p->state fiddling and load-balance pass. Then it drops the old rq->lock to acquire the new rq->lock. By having serialized ttwu(), p->sched_class, p->cpus_allowed with p->pi_lock, we can now drop the whole first rq->lock acquisition. The p->pi_lock serializing concurrent ttwu() calls protects p->state, which we will set to TASK_WAKING to bridge possible p->pi_lock to rq->lock gaps and serialize set_task_cpu() calls against task_rq_lock(). The p->pi_lock serialization of p->sched_class allows us to call scheduling class methods without holding the rq->lock, and the serialization of p->cpus_allowed allows us to do the load-balancing bits without races. Reviewed-by:
-
Peter Zijlstra authored
Since we can now call select_task_rq() and set_task_cpu() with only p->pi_lock held, and sched_exec() load-balancing has always been optimistic, drop all rq->lock usage. Oleg also noted that need_migrate_task() will always be true for current, so don't bother calling that at all. Reviewed-by:
-
Peter Zijlstra authored
Since p->pi_lock now protects all things needed to call select_task_rq() avoid the double remote rq->lock acquisition and rely on p->pi_lock. Reviewed-by:
-
Peter Zijlstra authored
In order to be able to call set_task_cpu() while either holding p->pi_lock or task_rq(p)->lock we need to hold both locks in order to stabilize task_rq(). This makes task_rq_lock() acquire both locks, and have __task_rq_lock() validate that p->pi_lock is held. This increases the locking overhead for most scheduler syscalls but allows reduction of rq->lock contention for some scheduler hot paths (ttwu). Reviewed-by:
-
Peter Zijlstra authored
Since we now serialize ttwu() using p->pi_lock, we also need to serialize ttwu_local() using that, otherwise, once we drop the rq->lock from ttwu() it can race with ttwu_local(). Reviewed-by:
-
Peter Zijlstra authored
In prepratation of having to call task_contributes_to_load() without holding rq->lock, we need to store the result until we do and can update the rq accounting accordingly. Reviewed-by:
-
Peter Zijlstra authored
In order to avoid reading partial updated min_vruntime values on 32bit implement a seqcount like solution. Reviewed-by:
-
Peter Zijlstra authored
In preparation of calling this without rq->lock held, remove the dependency on the rq argument. Reviewed-by:
-
Peter Zijlstra authored
In preparation of calling select_task_rq() without rq->lock held, drop the dependency on the rq argument. Reviewed-by:
-
Peter Zijlstra authored
Currently p->pi_lock already serializes p->sched_class, also put p->cpus_allowed and try_to_wake_up() under it, this prepares the way to do the first part of ttwu() without holding rq->lock. By having p->sched_class and p->cpus_allowed serialized by p->pi_lock, we prepare the way to call select_task_rq() without holding rq->lock. Reviewed-by:
-
Peter Zijlstra authored
Provide a generic p->on_rq because the p->se.on_rq semantics are unfavourable for lockless wakeups but needed for sched_fair. In particular, p->on_rq is only cleared when we actually dequeue the task in schedule() and not on any random dequeue as done by things like __migrate_task() and __sched_setscheduler(). This also allows us to remove p->se usage from !sched_fair code. Reviewed-by:
-
Peter Zijlstra authored
Collect all ttwu() stat code into a single function and ensure its always called for an actual wakeup (changing p->state to TASK_RUNNING). Reviewed-by:
-
Peter Zijlstra authored
try_to_wake_up() would only return a success when it would have to place a task on a rq, change that to every time we change p->state to TASK_RUNNING, because that's the real measure of wakeups. This results in that success is always true for the tracepoints. Reviewed-by:
-
Peter Zijlstra authored
wq_worker_waking_up() needs to match wq_worker_sleeping(), since the latter is only called on deactivate, move the former near activate. Signed-off-by:
-
Peter Zijlstra authored
Since we now have p->on_cpu unconditionally available, use it to re-implement mutex_spin_on_owner. Requested-by:
Thomas Gleixner <tglx@linutronix.de> Reviewed-by:
-
Peter Zijlstra authored
Always provide p->on_cpu so that we can determine if its on a cpu without having to lock the rq. Reviewed-by:
-
Peter Zijlstra authored
For future rework of try_to_wake_up() we'd like to push part of that function onto the CPU the task is actually going to run on. In order to do so we need a generic callback from the existing scheduler IPI. This patch introduces such a generic callback: scheduler_ipi() and implements it as a NOP. BenH notes: PowerPC might use this IPI on offline CPUs under rare conditions! Acked-by:
Russell King <rmk+kernel@arm.linux.org.uk> Acked-by:
Martin Schwidefsky <schwidefsky@de.ibm.com> Acked-by:
Chris Metcalf <cmetcalf@tilera.com> Acked-by:
Jesper Nilsson <jesper.nilsson@axis.com> Acked-by:
Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by:
Ralf Baechle <ralf@linux-mips.org> Reviewed-by:
-
Ingo Molnar authored
Merge reason: Pick up this upstream commit: 6631e635: block: don't flush plugged IO on forced preemtion scheduling As it modifies the scheduler and we'll queue up dependent patches. Signed-off-by:
-
- 13 Apr, 2011 14 commits
-
-
Linus Torvalds authored
Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mjg59/platform-drivers-x86 * 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mjg59/platform-drivers-x86: x86 platform drivers: Build fix for intel_pmic_gpio
-
git://git.kernel.org/pub/scm/linux/kernel/git/egtvedt/avr32-2.6Linus Torvalds authored
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/egtvedt/avr32-2.6: avr32: add ATAG_BOARDINFO don't check platform_get_irq's return value against zero avr32: init cannot ignore signals sent by force_sig_info() avr32: fix deadlock when reading clock list in debugfs avr32: Fix .size directive for cpu_enter_idle avr32: At32ap: pio fix typo "))" on gpio_irq_unmask prototype fix the wrong argument of the functions definition
-
git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6Linus Torvalds authored
* 'drm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (22 commits) Revert "i915: restore only the mode of this driver on lastclose" Revert "ttm: Utilize the DMA API for pages that have TTM_PAGE_FLAG_DMA32 set." i915: select VIDEO_OUTPUT_CONTROL for ACPI_VIDEO drm/radeon/kms: properly program vddci on evergreen+ drm/radeon/kms: add voltage type to atom set voltage function drm/radeon/kms: fix pcie_p callbacks on btc and cayman drm/radeon/kms: fix suspend on rv530 asics drm/radeon/kms: clean up gart dummy page handling drm/radeon/kms: make radeon i2c put/get bytes less noisy drm/radeon/kms: pll tweaks for rv6xx drm/radeon: Fix KMS legacy backlight support if CONFIG_BACKLIGHT_CLASS_DEVICE=m. radeon: Fix KMS CP writeback on big endian machines. i915: restore only the mode of this driver on lastclose drm/nvc0: improve vm flush function drm/nv50-nvc0: remove some code that doesn't belong here drm/nv50: use "nv86" tlb flush method on everything except 0x50/0xac drm/nouveau: quirk for XFX GT-240X-YA drm/nv50-nvc0: work around an evo channel hang that some people see drm/nouveau: implement init table opcode 0x5c drm/nouveau: fix oops on unload with disabled LVDS panel ...
-
Matthew Garrett authored
Fix an incorrect function name so the driver builds. Signed-off-by:Matthew Garrett <mjg@redhat.com>
-
Linus Torvalds authored
We really only want to unplug the pending IO when the process actually goes to sleep. So move the test for flushing the plug up to the place where we actually deactivate the task - where we have properly checked for preemption and for the process really sleeping. Acked-by:
Jens Axboe <jaxboe@fusionio.com> Acked-by:
Linus Torvalds <torvalds@linux-foundation.org>
-
Linus Torvalds authored
Commit 982134ba ("mm: avoid wrapping vm_pgoff in mremap()") fixed the case of a expanding mapping causing vm_pgoff wrapping when you used mremap. But there was another case where we expand mappings hiding in plain sight: the automatic stack expansion. This fixes that case too. This one also found by Robert Święcki, using his nasty system call fuzzer tool. Good job. Reported-and-tested-by:
Robert Święcki <robert@swiecki.net> Cc: stable@kernel.org Signed-off-by:
-
Andreas Bießmann authored
The ATAG_BOARDINFO is intended to hand over the information bd->bi_board_number from u-boot to the kernel. This piece of information can be used to implement some kind of board identification while booting the kernel. Therefore it is placed in .initdata section and can be accessed via the new symbol board_number only while initializing the kernel. Signed-off-by:
Andreas Bießmann <biessmann@corscience.de> Signed-off-by:
Hans-Christian Egtvedt <hans-christian.egtvedt@atmel.com>
-
Uwe Kleine-König authored
platform_get_irq returns -ENXIO on failure, so !int_irq was probably always true. Better use (int)int_irq <= 0. Note that a return value of zero is still handled as error even though this could mean irq0. This is a followup to 305b3228 that changed the return value of platform_get_irq from 0 to -ENXIO on error. Acked-by:
Uwe Kleine-König <u.kleine-koenig@pengutronix.de> Signed-off-by:
-
Matt Fleming authored
We can delete the code that checks to see if we're sending an ignored signal to init because force_sig_info() already handles this case. force_sig_info() will kill init even if the signal handler is SIG_DFL and the scenario described in the comment where init might "generate the same exception over and over again" cannot occur (force_sig_info() clears SIGNAL_UNKILLABLE to ensure that init will die). Also, the use of is_global_init() is not correct in the multhreaded case, as Oleg Nesterov explains, "is_global_init() is not right in theory, /sbin/init can be multithreaded. And, this doesn't cover the sub-namespace inits... I'd suggest to check SIGNAL_UNKILLABLE, but looking closer I think you can simply remove this code." It seems this code was copied from arch/powerpc in March 2007 in commit 623b0355 "[AVR32] Clean up exception handling code" but the code was deleted from arch/powerpc in November 2009 in commit a0592d42 "powerpc: kill the obsolete code under is_global_init()" So catch up with powerpc and delete the bogus code. Signed-off-by:
Matt Fleming <matt.fleming@linux.intel.com> Signed-off-by:
-
Ole Henrik Jahren authored
When writing out /sys/kernel/debug/at32ap_clk, clock list lock is being held while clk_get() is called. clk_get() attempts to take the same lock, which results in deadlock. Introduce and call lock free version, __clk_get(), instead. Signed-off-by:
Ole Henrik Jahren <olehenja@alumni.ntnu.no> Cc: Hans-Christian Egtvedt <hans-christian.egtvedt@atmel.com> Signed-off-by:
-
Ben Hutchings authored
gas used to accept (and ignore?) .size directives which referred to undefined symbols, as this does. In binutils 2.21 these are treated as errors. Signed-off-by:
Ben Hutchings <ben@decadent.org.uk> Signed-off-by:
-
Jean-Christophe PLAGNIOL-VILLARD authored
introduce in commit d75f1bfdSigned-off-by:
Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Hans-Christian Egtvedt <hans-christian.egtvedt@atmel.com> Cc: Nicolas Ferre <nicolas.ferre@atmel.com> Cc: Patrice Vilchez <patrice.vilchez@atmel.com> Signed-off-by:
-
Wanlong Gao authored
The functions of eic_chip's memebers use the wrong argument . Signed-off-by:
Wanlong Gao <wanlong.gao@gmail.com> Signed-off-by:
-
Geert Uytterhoeven authored
Signed-off-by:
Geert Uytterhoeven <geert@linux-m68k.org> Acked-by:
Greg Ungerer <gerg@uclinux.org> Signed-off-by:
-
