• Daniel Latypov's avatar
    kunit: tool: print summary of failed tests if a few failed out of a lot · f19dd011
    Daniel Latypov authored
    E.g. all the hw_breakpoint tests are failing right now.
    So if I run `kunit.py run --altests --arch=x86_64`, then I see
    > Testing complete. Ran 408 tests: passed: 392, failed: 9, skipped: 7
    
    Seeing which 9 tests failed out of the hundreds is annoying.
    If my terminal doesn't have scrollback support, I have to resort to
    looking at `.kunit/test.log` for the `not ok` lines.
    
    Teach kunit.py to print a summarized list of failures if the # of tests
    reachs an arbitrary threshold (>=100 tests).
    
    To try and keep the output from being too long/noisy, this new logic
    a) just reports "parent_test failed" if every child test failed
    b) won't print anything if there are >10 failures (also arbitrary).
    
    With this patch, we get an extra line of output showing:
    > Testing complete. Ran 408 tests: passed: 392, failed: 9, skipped: 7
    > Failures: hw_breakpoint
    
    This also works with parameterized tests, e.g. if I add a fake failure
    > Failures: kcsan.test_atomic_builtins_missing_barrier.threads=6
    
    Note: we didn't have enough tests for this to be a problem before.
    But with commit 980ac3ad ("kunit: tool: rename all_test_uml.config,
    use it for --alltests"), --alltests works and thus running >100 tests
    will probably become more common.
    Signed-off-by: default avatarDaniel Latypov <dlatypov@google.com>
    Reviewed-by: default avatarDavid Gow <davidgow@google.com>
    Signed-off-by: default avatarShuah Khan <skhan@linuxfoundation.org>
    f19dd011
kunit_parser.py 22.9 KB