This solves the module unload problem for some usb-serial drivers
(like visor.c and ftdi_sio.c), and makes usb drivers much simpler.