This reverts commit 6d01bb9d.

The exact same code was added in commit 3239fd31 (serial: of-serial: fetch
line number from DT) a few lined above. Doing this once should be enough.

Cc: Rob Herring <robh@kernel.org>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>