Commit 2e294072 authored by Kirill Smelkov's avatar Kirill Smelkov

benchstat: skip empty rows when just dumping data

When comparing 2 files, benchstat skips rows that are not present in
both benchmarks. However currently when benchstat'ing just 1 file or 3
or more, benchstat dumps every row for every unit, even if row results
for the unit are completely empty.

When several units are in use, this creates a lot of noise of empty
benchmark lines output. For example for the following benchmark

```
BenchmarkTwoHourMarathon 1 7200000000000 ns/op 14400000000000 user-ns/op 5 ns/GC 12 quick-bytes
BenchmarkTwoHourMarathon 1 7200000000000 ns/op 14500000000000 user-ns/op 5 ns/GC 16 quick-bytes
BenchmarkTwoHourMarathon 1 7200000000000 ns/op 14600000000000 user-ns/op 5 ns/GC 12 quick-bytes
BenchmarkTwoHourMarathon 1 7200000000000 ns/op 14200000000000 user-ns/op 5 ns/GC 16 quick-bytes
BenchmarkTwoHourMarathon 1 7200000000000 ns/op 14300000000000 user-ns/op 5 ns/GC 12 quick-bytes

BenchmarkTimens 1 31 ns/op 21 user-ns/op
BenchmarkTimeμs 1 32 μs/op 22 user-μs/op	# mu = U+03bc
BenchmarkTimeµs 1 33 µs/op 23 user-µs/op	# mu = U+b5
BenchmarkTimeus 1 34 us/op 24 user-us/op
BenchmarkTimems 1 35 ms/op 25 user-ms/op
BenchmarkTimes  1 36  s/op 26 user-s/op

BenchmarkTimeObjectns 1 41 ns/object
BenchmarkTimeObjectµs 1 42 µs/object
BenchmarkTimeObjectus 1 43 us/object
BenchmarkTimeObjectms 1 44 ms/object
BenchmarkTimeObjects  1 45 s/object
```

current output is

```
name             time/op
TwoHourMarathon    7200s ± 0%
Timens            31.0ns ± 0%
Timeμs            32.0µs ± 0%
Timeµs            33.0µs ± 0%
Timeus            34.0µs ± 0%
Timems            35.0ms ± 0%
Times              36.0s ± 0%
TimeObjectns
TimeObjectµs
TimeObjectus
TimeObjectms
TimeObjects

name             user-time/op
TwoHourMarathon   14400s ± 1%
Timens            21.0ns ± 0%
Timeμs            22.0µs ± 0%
Timeµs            23.0µs ± 0%
Timeus            24.0µs ± 0%
Timems            25.0ms ± 0%
Times              26.0s ± 0%
TimeObjectns
TimeObjectµs
TimeObjectus
TimeObjectms
TimeObjects

name             time/GC
TwoHourMarathon   5.00ns ± 0%
Timens
Timeμs
Timeµs
Timeus
Timems
Times
TimeObjectns
TimeObjectµs
TimeObjectus
TimeObjectms
TimeObjects

name             quick-bytes
TwoHourMarathon    13.6B ±18%
Timens
Timeμs
Timeµs
Timeus
Timems
Times
TimeObjectns
TimeObjectµs
TimeObjectus
TimeObjectms
TimeObjects

name             time/object
TwoHourMarathon
Timens
Timeμs
Timeµs
Timeus
Timems
Times
TimeObjectns      41.0ns ± 0%
TimeObjectµs      42.0µs ± 0%
TimeObjectus      43.0µs ± 0%
TimeObjectms      44.0ms ± 0%
TimeObjects        45.0s ± 0%
```

and it becomes

```
name             time/op
TwoHourMarathon    7200s ± 0%
Timens            31.0ns ± 0%
Timeμs            32.0µs ± 0%
Timeµs            33.0µs ± 0%
Timeus            34.0µs ± 0%
Timems            35.0ms ± 0%
Times              36.0s ± 0%

name             user-time/op
TwoHourMarathon   14400s ± 1%
Timens            21.0ns ± 0%
Timeμs            22.0µs ± 0%
Timeµs            23.0µs ± 0%
Timeus            24.0µs ± 0%
Timems            25.0ms ± 0%
Times              26.0s ± 0%

name             time/GC
TwoHourMarathon   5.00ns ± 0%

name             quick-bytes
TwoHourMarathon    13.6B ±18%

name             time/object
TimeObjectns      41.0ns ± 0%
TimeObjectµs      42.0µs ± 0%
TimeObjectus      43.0µs ± 0%
TimeObjectms      44.0ms ± 0%
TimeObjects        45.0s ± 0%
```

after this patch.

Change-Id: Ic260e39913af7b2c3aa1ff850aed65002d553d6c
parent 99f0dffb
......@@ -75,6 +75,20 @@ func (c *Collection) Tables() []*Table {
}
}
// if data for this unit is all missing - omit row entirely.
missing := true
for _, cfg := range c.Configs {
k := key
k.Config = cfg
if c.Metrics[k] != nil {
missing = false
break
}
}
if missing {
continue
}
// If there are only two configs being compared, add stats.
if table.OldNewDelta {
k0 := key
......
......@@ -39,6 +39,8 @@ func TestGolden(t *testing.T) {
check(t, "packagesold", "packagesold.txt")
check(t, "packages", "packagesold.txt", "packagesnew.txt")
check(t, "units", "units-old.txt", "units-new.txt")
check(t, "units1", "units-old.txt")
check(t, "units3", "units-old.txt", "units-new.txt", "units-new.txt")
check(t, "zero", "-delta-test=none", "zero-old.txt", "zero-new.txt")
}
......
name time/op
TwoHourMarathon 7200s ± 0%
Timens 31.0ns ± 0%
Timeμs 32.0µs ± 0%
Timeµs 33.0µs ± 0%
Timeus 34.0µs ± 0%
Timems 35.0ms ± 0%
Times 36.0s ± 0%
name user-time/op
TwoHourMarathon 14400s ± 1%
Timens 21.0ns ± 0%
Timeμs 22.0µs ± 0%
Timeµs 23.0µs ± 0%
Timeus 24.0µs ± 0%
Timems 25.0ms ± 0%
Times 26.0s ± 0%
name time/GC
TwoHourMarathon 5.00ns ± 0%
name quick-bytes
TwoHourMarathon 13.6B ±18%
name time/object
TimeObjectns 41.0ns ± 0%
TimeObjectµs 42.0µs ± 0%
TimeObjectus 43.0µs ± 0%
TimeObjectms 44.0ms ± 0%
TimeObjects 45.0s ± 0%
name \ time/op units-old.txt units-new.txt units-new.txt
TwoHourMarathon 7200s ± 0% 7200s ± 0% 7200s ± 0%
Timens 31.0ns ± 0% 31.0ns ± 0% 31.0ns ± 0%
Timeμs 32.0µs ± 0% 32.0µs ± 0% 32.0µs ± 0%
Timeµs 33.0µs ± 0% 33.0µs ± 0% 33.0µs ± 0%
Timeus 34.0µs ± 0% 34.0µs ± 0% 34.0µs ± 0%
Timems 35.0ms ± 0% 35.0ms ± 0% 35.0ms ± 0%
Times 36.0s ± 0% 36.0s ± 0% 36.0s ± 0%
name \ user-time/op units-old.txt units-new.txt units-new.txt
TwoHourMarathon 14400s ± 1% 14380s ± 3% 14380s ± 3%
Timens 21.0ns ± 0% 21.0ns ± 0% 21.0ns ± 0%
Timeμs 22.0µs ± 0% 22.0µs ± 0% 22.0µs ± 0%
Timeµs 23.0µs ± 0% 23.0µs ± 0% 23.0µs ± 0%
Timeus 24.0µs ± 0% 24.0µs ± 0% 24.0µs ± 0%
Timems 25.0ms ± 0% 25.0ms ± 0% 25.0ms ± 0%
Times 26.0s ± 0% 26.0s ± 0% 26.0s ± 0%
name \ time/GC units-old.txt units-new.txt units-new.txt
TwoHourMarathon 5.00ns ± 0% 5.00ns ± 0% 5.00ns ± 0%
name \ quick-bytes units-old.txt units-new.txt units-new.txt
TwoHourMarathon 13.6B ±18% 13.6B ±18% 13.6B ±18%
name \ time/object units-old.txt units-new.txt units-new.txt
TimeObjectns 41.0ns ± 0% 41.0ns ± 0% 41.0ns ± 0%
TimeObjectµs 42.0µs ± 0% 42.0µs ± 0% 42.0µs ± 0%
TimeObjectus 43.0µs ± 0% 43.0µs ± 0% 43.0µs ± 0%
TimeObjectms 44.0ms ± 0% 44.0ms ± 0% 44.0ms ± 0%
TimeObjects 45.0s ± 0% 45.0s ± 0% 45.0s ± 0%
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment