cgen,pref: add -coverage support + vcover tool

[felipensp]

Building with coverage support

v -cov ./dir-output file.v
or
v -cov ./dir-output run file.v

Every run of ./file will generate a cover.<build hash>.<time hash>.json file.

VCOVDIR env variable can be used to change the output dir.
VCOVDIR=/tmp/cover ./file

Reporting to stdout

$ v cover -file vcover.01HYNV1D0CSFFXKKJT5QB7T624.2040586979 
  17.44 |   15 |   86 | /home/felipe/github/v/vlib/strings/builder.c.v
   9.21 |   14 |  152 | /home/felipe/github/v/vlib/math/bits/bits.v
  65.66 |  130 |  198 | /home/felipe/github/v/vlib/strconv/f64_str.c.v
  32.10 |   52 |  162 | /home/felipe/github/v/vlib/strconv/utilities.c.v
  28.57 |   28 |   98 | /home/felipe/github/v/vlib/strconv/utilities.v
  44.44 |    4 |    9 | /home/felipe/github/v/vlib/builtin/array.v
  26.42 |   28 |  106 | /home/felipe/github/v/vlib/builtin/array_d_gcboehm_opt.v
  28.57 |    6 |   21 | /home/felipe/github/v/vlib/builtin/backtraces_nix.c.v
  34.45 |   72 |  209 | /home/felipe/github/v/vlib/builtin/builtin.c.v
   3.57 |    1 |   28 | /home/felipe/github/v/vlib/builtin/builtin.v
  14.29 |    1 |    7 | /home/felipe/github/v/vlib/builtin/builtin_d_gcboehm.c.v
  33.33 |    2 |    6 | /home/felipe/github/v/vlib/builtin/builtin_nix.c.v
  46.67 |    7 |   15 | /home/felipe/github/v/vlib/builtin/cfns_wrapper.c.v
   7.25 |    5 |   69 | /home/felipe/github/v/vlib/builtin/float.c.v
  35.44 |   73 |  206 | /home/felipe/github/v/vlib/builtin/int.v
  20.00 |    3 |   15 | /home/felipe/github/v/vlib/builtin/result.v
   2.33 |   22 |  946 | /home/felipe/github/v/vlib/builtin/string.v
  16.53 |   60 |  363 | /home/felipe/github/v/vlib/builtin/string_interpolation.v
  15.94 |   11 |   69 | /home/felipe/github/v/vlib/builtin/utf8.v
  11.43 |    8 |   70 | /home/felipe/github/v/vlib/json/json_primitives.c.v
 100.00 |    2 |    2 | bug.v
Total coverage: 21 files, 19.18% coverage

Merging and reporting to stdout

v cov -dir ./coverage-dir

Test coverage

v -cov /tmp/cover test vlib/json/
v cover -dir /tmp/cover

Filtering results

v cover -dir /tmp/cover -filter vlib/json,vlib/toml

[spytheman]

Excellent work.

[spytheman]

@felipensp, I've tested running this:

if true {
	println('branch 1')
	if true {
		println('branch 1.1')
		if true {
			println('branch 1.1.1')
		} else {
			println('branch 1.1.2')
		}
	} else {
		println('branch 1.2')
		if true {
			println('branch 1.2.1')
		} else {
			println('branch 1.2.2')
		}
	}
} else {
	println('branch 2')
	if true {
		println('branch 2.1')
		if true {
			println('branch 2.1.1')
		} else {
			println('branch 2.1.2')
		}
	} else {
		println('branch 2.2')
		if true {
			println('branch 2.2.1')
		} else {
			println('branch 2.2.2')
		}
	}
}

with: v -coverage - run branches.v, and I got:

#0 13:07:08 ^ v_coverage /v/oo>v -coverage - run branches.v
branch 1
branch 1.1
branch 1.1.1
V coverage
--------------------------------------------------
[   0.00%]    0 /   98 | /space/v/oo/vlib/strings/builder.c.v
[   0.00%]    0 /   43 | /space/v/oo/vlib/strings/similarity.v
[   0.00%]    0 /   18 | /space/v/oo/vlib/strings/strings.c.v
[   0.00%]    0 /   59 | /space/v/oo/vlib/strings/strings.v
[   0.00%]    0 /  152 | /space/v/oo/vlib/math/bits/bits.v
[   0.00%]    0 /    0 | /space/v/oo/vlib/math/bits/bits_tables.v
[   0.00%]    0 /    8 | /space/v/oo/vlib/math/bits/unsafe_bits.v
[   0.00%]    0 /  191 | /space/v/oo/vlib/strconv/atof.c.v
[   0.00%]    0 /   60 | /space/v/oo/vlib/strconv/atofq.c.v
[   0.00%]    0 /  104 | /space/v/oo/vlib/strconv/atoi.v
[   0.00%]    0 /  176 | /space/v/oo/vlib/strconv/f32_str.c.v
[   0.00%]    0 /  198 | /space/v/oo/vlib/strconv/f64_str.c.v
[   0.00%]    0 /    0 | /space/v/oo/vlib/strconv/f64_str.v
[   0.00%]    0 /   13 | /space/v/oo/vlib/strconv/format.v
[   0.00%]    0 /  269 | /space/v/oo/vlib/strconv/format_mem.c.v
[   0.00%]    0 /    4 | /space/v/oo/vlib/strconv/ftoa.c.v
[   0.00%]    0 /   38 | /space/v/oo/vlib/strconv/number_to_base.c.v
[   0.00%]    0 /    0 | /space/v/oo/vlib/strconv/tables.v
[   0.00%]    0 /  166 | /space/v/oo/vlib/strconv/utilities.c.v
[   0.00%]    0 /   98 | /space/v/oo/vlib/strconv/utilities.v
[   0.00%]    0 /  433 | /space/v/oo/vlib/strconv/vprintf.c.v
[   0.00%]    0 /    9 | /space/v/oo/vlib/builtin/array.v
[   3.36%]    4 /  119 | /space/v/oo/vlib/builtin/array_d_gcboehm_opt.v
[   0.00%]    0 /    4 | /space/v/oo/vlib/builtin/backtraces.c.v
[   0.00%]    0 /   21 | /space/v/oo/vlib/builtin/backtraces_nix.c.v
[  18.48%]   39 /  211 | /space/v/oo/vlib/builtin/builtin.c.v
[   0.00%]    0 /   28 | /space/v/oo/vlib/builtin/builtin.v
[  14.29%]    1 /    7 | /space/v/oo/vlib/builtin/builtin_d_gcboehm.c.v
[  33.33%]    2 /    6 | /space/v/oo/vlib/builtin/builtin_nix.c.v
[  20.00%]    3 /   15 | /space/v/oo/vlib/builtin/cfns_wrapper.c.v
[   0.00%]    0 /    2 | /space/v/oo/vlib/builtin/chan.v
[   0.00%]    0 /   11 | /space/v/oo/vlib/builtin/character_inout.c.v
[   0.00%]    0 /   69 | /space/v/oo/vlib/builtin/float.c.v
[   0.00%]    0 /  207 | /space/v/oo/vlib/builtin/int.v
[   0.00%]    0 /   34 | /space/v/oo/vlib/builtin/map.c.v
[   0.00%]    0 /  306 | /space/v/oo/vlib/builtin/map.v
[   0.00%]    0 /   38 | /space/v/oo/vlib/builtin/map_d_gcboehm_opt.v
[   0.00%]    0 /    3 | /space/v/oo/vlib/builtin/option.c.v
[   0.00%]    0 /    8 | /space/v/oo/vlib/builtin/option.v
[   0.00%]    0 /   15 | /space/v/oo/vlib/builtin/result.v
[   0.00%]    0 /   24 | /space/v/oo/vlib/builtin/rune.v
[   0.00%]    0 /  206 | /space/v/oo/vlib/builtin/sorted_map.v
[   0.00%]    0 /  951 | /space/v/oo/vlib/builtin/string.v
[   0.00%]    0 /    9 | /space/v/oo/vlib/builtin/string_charptr_byteptr_helpers.v
[   0.00%]    0 /  364 | /space/v/oo/vlib/builtin/string_interpolation.v
[   0.00%]    0 /   18 | /space/v/oo/vlib/builtin/utf8.c.v
[   0.00%]    0 /   69 | /space/v/oo/vlib/builtin/utf8.v
[  54.55%]    6 /   11 | branches.v
--------------------------------------------------
Total coverage: 48 files, 1.12% coverage
#0 13:07:41 ^ v_coverage /v/oo>

Why does it print all the .v files for all modules (used in the program), given that for most programs, most of their functionality, will not be used?

I think that skipping all the ones, for which there is 0 coverage will be more useful and more compact.

[felipensp]

I am planning to do:

• Add env var for dir for output files
• Generate the counter file on each run
• Generate the separated metadata file
• cmd tool (vcover) to merging/reporting

[JalonSolov]

The Windows gcc failure appears to be completely unrelated... and very, very weird:

  FAIL  [ 252/1864] C:  3547.7 ms, R:    39.108 ms vlib/net/unix/use_net_and_net_unix_together_test.v
         retry: 1
      comp_cmd: "D:\a\v\v\v.exe" -skip-running   -o "C:\Users\runneradmin\AppData\Local\Temp\v_0\tsession_1368_01HYNCRVAR27QE8T5X47RA2JE4\251_2\use_net_and_net_unix_together_test.exe" "D:\a\v\v\vlib\net\unix\use_net_and_net_unix_together_test.v"
       run_cmd: "C:\Users\runneradmin\AppData\Local\Temp\v_0\tsession_1368_01HYNCRVAR27QE8T5X47RA2JE4\251_2\use_net_and_net_unix_together_test.exe"
failure code: 1; foutput.len: 437; failure output:
---------------------------------------------------------------------------------------------------- retry: 0 ; max_retry: 0 ; r.exit_code: 1 ; trimmed_output.len: 269
V panic: net: resolve_addrs Unix socket address is too long; could not resolve path C:\Users\runneradmin\AppData\Local\Temp\v_0\tsession_1368_01HYNCRVAR27QE8T5X47RA2JE4\net_unix_test\domain_socket

v hash: c46c1e3

print_backtrace_skipping_top_frames is not implemented

[spytheman]

The Windows gcc failure appears to be completely unrelated... and very, very weird:

It is my fault. It is fixed on master now.