This patch moves clock related operations to the remoteproc prepare()
and unprepare() callbacks so that the PM runtime framework doesn't
have to be involved needlessly.  This provides a simpler approach and
requires less code.

Based on the work from Paul Cercueil published here:
https://lore.kernel.org/linux-remoteproc/20191116170846.67220-4-paul@crapouillou.net/Reviewed-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Link: https://lore.kernel.org/r/20200630163118.3830422-2-mathieu.poirier@linaro.orgSigned-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>