The board has an external pull-up on the card-detect signal, so there's no
need to add another one.

This also removes a DTC warning.
Acked-by: Chen-Yu Tsai <wens@csie.org>
Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>