Usually we wait for the host to complete the unref request, then cleanup
the guest-side state of the object in the completion callback.  When
submitting the unref command failed the completion callback will not be
called though, so cleanup right away.

Fixes a WARN on stale mm entries on driver shutdown.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
Link: http://patchwork.freedesktop.org/patch/msgid/20200908070723.6394-4-kraxel@redhat.com