Properly handle resource cleanup on unplug/exit.

Spotted by Jonathan Cameron
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>