In poweroff, we set the reset bit and the power down bit, but only
managed to unset the reset bit for poweron.  This meant that if HDMI
did -EPROBE_DEFER after it had grabbed its clocks, we'd power down the
PLLH (that had been on at boot time) and never recover.
Signed-off-by: Eric Anholt <eric@anholt.net>
Fixes: 41691b88 ("clk: bcm2835: Add support for programming the audio domain clocks")
Cc: stable@vger.kernel.org
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>