Commit 27de3482 authored by Rafael J. Wysocki's avatar Rafael J. Wysocki

cpufreq: governor: Fix race in dbs_update_util_handler()

There is a scenario that may lead to undesired results in
dbs_update_util_handler().  Namely, if two CPUs sharing a policy
enter the funtion at the same time, pass the sample delay check
and then one of them is stalled until dbs_work_handler() (queued
up by the other CPU) clears the work counter, it may update the
work counter and queue up another work item prematurely.

To prevent that from happening, use the observation that the CPU
queuing up a work item in dbs_update_util_handler() updates the
last sample time.  This means that if another CPU was stalling after
passing the sample delay check and now successfully updated the work
counter as a result of the race described above, it will see the new
value of the last sample time which is different from what it used in
the sample delay check before.  If that happens, the sample delay
check passed previously is not valid any more, so the CPU should not
continue.

Fixes: f17cbb53783c (cpufreq: governor: Avoid atomic operations in hot paths)
Signed-off-by: default avatarRafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: default avatarViresh Kumar <viresh.kumar@linaro.org>
parent 94ab5e03
...@@ -340,7 +340,7 @@ static void dbs_update_util_handler(struct update_util_data *data, u64 time, ...@@ -340,7 +340,7 @@ static void dbs_update_util_handler(struct update_util_data *data, u64 time,
{ {
struct cpu_dbs_info *cdbs = container_of(data, struct cpu_dbs_info, update_util); struct cpu_dbs_info *cdbs = container_of(data, struct cpu_dbs_info, update_util);
struct policy_dbs_info *policy_dbs = cdbs->policy_dbs; struct policy_dbs_info *policy_dbs = cdbs->policy_dbs;
u64 delta_ns; u64 delta_ns, lst;
/* /*
* The work may not be allowed to be queued up right now. * The work may not be allowed to be queued up right now.
...@@ -356,7 +356,8 @@ static void dbs_update_util_handler(struct update_util_data *data, u64 time, ...@@ -356,7 +356,8 @@ static void dbs_update_util_handler(struct update_util_data *data, u64 time,
* of sample_delay_ns used in the computation may be stale. * of sample_delay_ns used in the computation may be stale.
*/ */
smp_rmb(); smp_rmb();
delta_ns = time - policy_dbs->last_sample_time; lst = READ_ONCE(policy_dbs->last_sample_time);
delta_ns = time - lst;
if ((s64)delta_ns < policy_dbs->sample_delay_ns) if ((s64)delta_ns < policy_dbs->sample_delay_ns)
return; return;
...@@ -365,9 +366,19 @@ static void dbs_update_util_handler(struct update_util_data *data, u64 time, ...@@ -365,9 +366,19 @@ static void dbs_update_util_handler(struct update_util_data *data, u64 time,
* at this point. Otherwise, we need to ensure that only one of the * at this point. Otherwise, we need to ensure that only one of the
* CPUs sharing the policy will do that. * CPUs sharing the policy will do that.
*/ */
if (policy_dbs->is_shared && if (policy_dbs->is_shared) {
!atomic_add_unless(&policy_dbs->work_count, 1, 1)) if (!atomic_add_unless(&policy_dbs->work_count, 1, 1))
return; return;
/*
* If another CPU updated last_sample_time in the meantime, we
* shouldn't be here, so clear the work counter and bail out.
*/
if (unlikely(lst != READ_ONCE(policy_dbs->last_sample_time))) {
atomic_set(&policy_dbs->work_count, 0);
return;
}
}
policy_dbs->last_sample_time = time; policy_dbs->last_sample_time = time;
policy_dbs->work_in_progress = true; policy_dbs->work_in_progress = true;
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment