According to the ARM ARMv7, explicit barriers are necessary when using
synchronisation primitives such as SWP{B}. The use of these
instructions does not automatically imply a barrier and any ordering
requirements by the software must be explicitly expressed with the use
of suitable barriers.

Based on this, remove the barriers from SWP{B} emulation.
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>