Most of the memory controller drivers do not depend on architecture
specific code so can be compile tested to increase build coverage.

When compile tested, do not enable them by default.
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Link: https://lore.kernel.org/r/20200724074038.5597-7-krzk@kernel.org