As Russell pointed out, original patch will break some serial configurations
because of the dependency of the <asm/serial.h> header file.

Revert it first and try to find out other solution later

Cc: Javier Herrero <jherrero@hvsistemas.es>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Cc: Russell King <rmk+lkml@arm.linux.org.uk>
Signed-off-by: Bryan Wu <cooloney@kernel.org>