When deferred probe happens driver will try to ioremap multiple times
and will fail. Memory resource.start variable is a global variable,
modifications in this field will be accumulated on every probe.
Fix this by moving the above operations after driver hold all
required PHY's.
Signed-off-by: Ivan T. Ivanov <iivanov@mm-sol.com>
Signed-off-by: Felipe Balbi <balbi@ti.com>