This reverts commit 49c458e5. It
seems to have some side effects in the non-kms cases.
Signed-off-by: Dave Airlie <airlied@redhat.com>