The cache is based on the full register map so confuses things if used
for a partial map.
Reported-by: Bard Liao <bardliao@realtek.com>
Signed-off-by: Mark Brown <broonie@opensource.wolfsonmicro.com>