This seems to be the symptom of a few neat bugs, hence be more
obnoxious when this fails.
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Signed-Off-by: Daniel Vetter <daniel.vetter@ffwll.ch>