Avoid extra special casing by setting the cache_bypass flag when we're
not caching.
Signed-off-by: Mark Brown <broonie@opensource.wolfsonmicro.com>