• Andrii Nakryiko's avatar
    selftests/bpf: Add veristat tool for mass-verifying BPF object files · c8bc5e05
    Andrii Nakryiko authored
    Add a small tool, veristat, that allows mass-verification of
    a set of *libbpf-compatible* BPF ELF object files. For each such object
    file, veristat will attempt to verify each BPF program *individually*.
    Regardless of success or failure, it parses BPF verifier stats and
    outputs them in human-readable table format. In the future we can also
    add CSV and JSON output for more scriptable post-processing, if necessary.
    
    veristat allows to specify a set of stats that should be output and
    ordering between multiple objects and files (e.g., so that one can
    easily order by total instructions processed, instead of default file
    name, prog name, verdict, total instructions order).
    
    This tool should be useful for validating various BPF verifier changes
    or even validating different kernel versions for regressions.
    
    Here's an example for some of the heaviest selftests/bpf BPF object
    files:
    
      $ sudo ./veristat -s insns,file,prog {pyperf,loop,test_verif_scale,strobemeta,test_cls_redirect,profiler}*.linked3.o
      File                                  Program                               Verdict  Duration, us  Total insns  Total states  Peak states
      ------------------------------------  ------------------------------------  -------  ------------  -----------  ------------  -----------
      loop3.linked3.o                       while_true                            failure        350990      1000001          9663         9663
      test_verif_scale3.linked3.o           balancer_ingress                      success        115244       845499          8636         2141
      test_verif_scale2.linked3.o           balancer_ingress                      success         77688       773445          3048          788
      pyperf600.linked3.o                   on_event                              success       2079872       624585         30335        30241
      pyperf600_nounroll.linked3.o          on_event                              success        353972       568128         37101         2115
      strobemeta.linked3.o                  on_event                              success        455230       557149         15915        13537
      test_verif_scale1.linked3.o           balancer_ingress                      success         89880       554754          8636         2141
      strobemeta_nounroll2.linked3.o        on_event                              success        433906       501725         17087         1912
      loop6.linked3.o                       trace_virtqueue_add_sgs               success        282205       398057          8717          919
      loop1.linked3.o                       nested_loops                          success        125630       361349          5504         5504
      pyperf180.linked3.o                   on_event                              success       2511740       160398         11470        11446
      pyperf100.linked3.o                   on_event                              success        744329        87681          6213         6191
      test_cls_redirect.linked3.o           cls_redirect                          success         54087        78925          4782          903
      strobemeta_subprogs.linked3.o         on_event                              success         57898        65420          1954          403
      test_cls_redirect_subprogs.linked3.o  cls_redirect                          success         54522        64965          4619          958
      strobemeta_nounroll1.linked3.o        on_event                              success         43313        57240          1757          382
      pyperf50.linked3.o                    on_event                              success        194355        46378          3263         3241
      profiler2.linked3.o                   tracepoint__syscalls__sys_enter_kill  success         23869        43372          1423          542
      pyperf_subprogs.linked3.o             on_event                              success         29179        36358          2499         2499
      profiler1.linked3.o                   tracepoint__syscalls__sys_enter_kill  success         13052        27036          1946          936
      profiler3.linked3.o                   tracepoint__syscalls__sys_enter_kill  success         21023        26016          2186          915
      profiler2.linked3.o                   kprobe__vfs_link                      success          5255        13896           303          271
      profiler1.linked3.o                   kprobe__vfs_link                      success          7792        12687          1042         1041
      profiler3.linked3.o                   kprobe__vfs_link                      success          7332        10601           865          865
      profiler2.linked3.o                   kprobe_ret__do_filp_open              success          3417         8900           216          199
      profiler2.linked3.o                   kprobe__vfs_symlink                   success          3548         8775           203          186
      pyperf_global.linked3.o               on_event                              success         10007         7563           520          520
      profiler3.linked3.o                   kprobe_ret__do_filp_open              success          4708         6464           532          532
      profiler1.linked3.o                   kprobe_ret__do_filp_open              success          3090         6445           508          508
      profiler3.linked3.o                   kprobe__vfs_symlink                   success          4477         6358           521          521
      profiler1.linked3.o                   kprobe__vfs_symlink                   success          3381         6347           507          507
      profiler2.linked3.o                   raw_tracepoint__sched_process_exec    success          2464         5874           292          189
      profiler3.linked3.o                   raw_tracepoint__sched_process_exec    success          2677         4363           397          283
      profiler2.linked3.o                   kprobe__proc_sys_write                success          1800         4355           143          138
      profiler1.linked3.o                   raw_tracepoint__sched_process_exec    success          1649         4019           333          240
      pyperf600_bpf_loop.linked3.o          on_event                              success          2711         3966           306          306
      profiler2.linked3.o                   raw_tracepoint__sched_process_exit    success          1234         3138            83           66
      profiler3.linked3.o                   kprobe__proc_sys_write                success          1755         2623           223          223
      profiler1.linked3.o                   kprobe__proc_sys_write                success          1222         2456           193          193
      loop2.linked3.o                       while_true                            success           608         1783            57           30
      profiler3.linked3.o                   raw_tracepoint__sched_process_exit    success           789         1680           146          146
      profiler1.linked3.o                   raw_tracepoint__sched_process_exit    success           592         1526           133          133
      strobemeta_bpf_loop.linked3.o         on_event                              success          1015         1512           106          106
      loop4.linked3.o                       combinations                          success           165          524            18           17
      profiler3.linked3.o                   raw_tracepoint__sched_process_fork    success           196          299            25           25
      profiler1.linked3.o                   raw_tracepoint__sched_process_fork    success           109          265            19           19
      profiler2.linked3.o                   raw_tracepoint__sched_process_fork    success           111          265            19           19
      loop5.linked3.o                       while_true                            success            47           84             9            9
      ------------------------------------  ------------------------------------  -------  ------------  -----------  ------------  -----------
    Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
    Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    Link: https://lore.kernel.org/bpf/20220909193053.577111-4-andrii@kernel.org
    c8bc5e05
Makefile 23.6 KB