Remove some legacy code and replace it by the clksrc-of code.

Do some cleanup and code consolidation.
Signed-off-by: Yoshinori Sato <ysato@users.sourceforge.jp>
Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>