This reverts commit 39e0786d.

On request from microblaze developers, they are going to handle
this differently.
Signed-off-by: David S. Miller <davem@davemloft.net>