Mike Kravetz pointed out that the set_cpus_allowed optimization he
suggested has a small but possible race condition wherein the system
could still be operating in the context of the task but it is not
running.  On top of this, the runqueue lock can be dropped in
load_balance and thus we can race and set task->cpu at a very unpleasant
time.

My solution is to just remove the optimization.