Don't use uart_change_speed() when trying to restore the serial
console settings - the port may not have a tty associated with
it.
Signed-off-by: Russell King <rmk@arm.linux.org.uk>