This reverts commit d3ed7402.

Further upstream discussion between Thomas and Marek decided this needed
more work and driver specifics. So revert before it goes upstream.
Signed-off-by: Dave Airlie <airlied@redhat.com>