Currently, there is some logic for the driver to work without devfreq.
However, the driver actually fails to probe if !CONFIG_PM_DEVFREQ.

Fix this by selecting devfreq, and drop the additional checks
for devfreq.
Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20190517150042.776-1-ezequiel@collabora.com