Following mach-imx we break out the l2x0 handling into its
own file, avoiding some ifdefs. Also remove unnecessary creation
of local pointers when there is already one file-local readily
available.

Cc: Srinidhi Kasagar <srinidhi.kasagar@stericsson.com>
Cc: Rabin Vincent <rabin.vincent@stericsson.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>