This reverts commit 5b725054.

The rtc block on i.MX53 is a completely different hardware than the
one found on i.MX25.

Cc: <stable@vger.kernel.org> #4.14
Reported-by: Noel Vellemans <Noel.Vellemans@visionbms.com>
Suggested-by: Juergen Borleis <jbe@pengutronix.de>
Signed-off-by: Fabio Estevam <fabio.estevam@nxp.com>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>