If we try to modprobe a second gadget driver before
rmmoding the first one, the reference for the first
gadget driver would get NULLed avoiding usb to change
gadget drivers later.

Cc: David Brownell <david-b@pacbell.net>
Cc: Tony Lindgren <tony@atomide.com>
Signed-off-by: Felipe Balbi <felipe.balbi@nokia.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>