The case where test runner is not enabled got broken in
2a457867

This time, we had test to prevent regression.

/reviewed-on nexedi/slapos!440