When the hardware is in output mode, reading the value from the
hardware is not giving the correct value back. Instead read the
value from the cache so we get the right value.
Suggested-by: Russell King <linux@arm.linux.org.uk>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>