Allow users to keep on specifying their output frequency when disabling
the reference clock.
Reported-by: Kyung Kwee Ryu <Kyung-Kwee.Ryu@wolfsonmicro.com>
Signed-off-by: Mark Brown <broonie@opensource.wolfsonmicro.com>