The common case is building a kernel for one CPU type, and we are
able to allow GCC to optimise any the PMD entry cache handling
assembly which will never be used.