• Kirill Smelkov's avatar
    py.bench: Automatically discover benchmarks in test files · ffb40903
    Kirill Smelkov authored
    Since the beginning (9bf03d9c "py.bench: New command to benchmark python
    code similarly to `go test -bench`") py.bench was automatically
    discovering benchmarks in bench_*.py files only. This was inherited from
    wendelin.core which keeps its benchmarks in those files.
    
    However in pygolang, following Go convention(*), we already have several
    benchmarks that reside together with tests in same *_test.py files. And
    currently just running py.bench does not discover them.
    
    -> Let's fix this and teach py.bench to automatically discover
    benchmarks in the test files by default as well.
    
    Pytest's default is to look for tests in test_*.py and *_test.py (+).
    Add those patterns and also keep bench_*.py for backward compatibility.
    
    Before this patch running py.bench inside pygolang repository does not
    run any benchmark at all. After the patch py.bench runs all the
    benchmarks by default:
    
        (z-dev) kirr@deca:~/src/tools/go/pygolang$ py.bench
        ========================= test session starts ==========================
        platform linux2 -- Python 2.7.18, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
        rootdir: /home/kirr/src/tools/go/pygolang
        plugins: timeout-1.4.2, profiling-1.7.0, mock-2.0.0
        collected 18 items
    
        pymod: golang/golang_str_test.py
        Benchmarkstddecode              2000000 0.756 µs/op
        Benchmarkudecode                20000   74.359 µs/op
        Benchmarkstdencode              3000000 0.327 µs/op
        Benchmarkbencode                40000   32.613 µs/op
    
        pymod: golang/golang_test.py
        Benchmarkpyx_select_nogil       500000  2.051 µs/op
        Benchmarkpyx_go_nogil           90000   12.177 µs/op
        Benchmarkpyx_chan_nogil         600000  1.826 µs/op
        Benchmarkgo                     80000   13.267 µs/op
        Benchmarkchan                   500000  2.076 µs/op
        Benchmarkselect                 300000  3.835 µs/op
        Benchmarkdef                    30000000        0.035 µs/op
        Benchmarkfunc_def               40000   29.387 µs/op
        Benchmarkcall                   30000000        0.043 µs/op
        Benchmarkfunc_call              2000000 0.819 µs/op
        Benchmarktry_finally            20000000        0.096 µs/op
        Benchmarkdefer                  600000  1.755 µs/op
    
        pymod: golang/sync_test.py
        Benchmarkworkgroup_empty        40000   25.807 µs/op
        Benchmarkworkgroup_raise        40000   31.637 µs/op                     [100%]
    
        =========================== warnings summary ===========================
    
    (*) see https://pkg.go.dev/cmd/go#hdr-Test_packages
    (+) see https://docs.pytest.org/en/7.1.x/reference/reference.html#confval-python_files
    
    /reviewed-by @jerome
    /reviewed-on !20
    ffb40903
pybench.py 9.52 KB