Rather than referencing the cache directly read back the values we are
going to restore, supporting refactoring to use regmap.
Signed-off-by: Mark Brown <broonie@linaro.org>