Remove duplicated configurations and move specific details into
each corresponding dtsi file for the href versions.

Cc: Lee Jones <lee.jones@linaro.org>
Cc: devicetree@vger.kernel.org
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>