This fixes the issue where the Generic driver would bind to all usb-serial
devices, so the disconnect would not properly go to the real driver that
controlled the device.  This was very bad when unloading the module with
the device still connected.
Signed-off-by: Greg Kroah-Hartman <greg@kroah.com>