This fixes a potential oops (depending on how the HCD handles it)
from a recent patch of mine.  Evidently I didn't test this bit a
device that'd show the problem, sigh.  Luckily Martin Diehl was
testing and found this.