The problem was that when the refcount hit 0, it was unconditionally
assuming that it had been added, which is wrong.  The patch below
corrects that, and fixes the Oops when loading the floppy driver.