current renesas_usbhs is using new style udc_start/stop from
af1d7056
(usb: gadget: renesas: convert to new style).

But current renesas_usbhs driver didn't care about gadget.dev.driver
when udc_stop. it cause rmmod oops.
This patch care it.
Signed-off-by: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: Felipe Balbi <balbi@ti.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>