We don't need to fetch the parent index for clocks if they only
have one parent. Doing this also avoid an unnecessary allocation
for the parent cache.
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Michael Turquette <mturquette@linaro.org>