We don't need to many clocks enable after startup, to reduce some
power consumption.
Signed-off-by: Xing Zheng <zhengxing@rock-chips.com>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>