clk_prepare_enable can fail and if it does the controller must not be
considered enabled. So check for errors, properly unwind and give the
error code back to the caller.

While touching the clock code also enable the clocks in the same
direction and disable in reverse order.
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ti.com>