Gadgetfs driver called usb_gadget_unregister_driver unconditionally, even
if it didn't register it earlier due to other failures. This patch fixes
this.
Reported-by: Vegard Nossum <vegard.nossum@oracle.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Tested-by: Vegard Nossum <vegard.nossum@oracle.com>
Signed-off-by: Felipe Balbi <balbi@kernel.org>