===== GCOV (code coverage) support =====

==== Introduction ====

Code coverage testing is a powerful development aid, predominantly to verify the completeness of test sets. A program that has been compiled with code coverage support will keep track of how many times each of its basic blocks is invoked. This information can be extracted at run time, and mapped back to the program's source code in order to the programmer an idea how many times each line of source code has been triggered at run time.

MINIX3 offers support for code coverage testing of its system services. The original code coverage infrastructure aimed at supporting GCOV for GCC, but MINIX3 has since largely moved to using LLVM, and the GCC GCOV facilities likely no longer work. However, as of MINIX 3.4.0 (git commit 3ac5849), it is possible to obtaining coverage data for MINIX3 system services compiled with LLVM. This new facility extends the previous GCOV infrastructure.

The code coverage infrastructure currently supports obtaining coverage data for the source code modules of MINIX3 system services, including all servers and drivers. System service libraries are not yet included for practical reasons. Code coverage is not yet supported for the kernel either. Userland programs are not supported by this infrastructure at all.

==== How to use ====

Coverage support has to be compiled in. This can be done by setting the MKCOVERAGE build variable. MKCOVERAGE is supported for native compilation of MINIX3 on MINIX3:

> MKCOVERAGE=yes make build

It is also supported for crosscompilation:

> BUILDVARS="-V MKCOVERAGE=yes" ./releasetools/x86_hdboot.sh

The compilation process will generate .gcno files that contain static information on how to map back coverage data to the original source code.

After booting a system built with MKCOVERAGE=yes, one can then obtain the dynamic coverage information using the gcov-pull(8) command:

> gcov-pull <label>

The <label> parameter is the label of a running system service. For example:

> gcov-pull vfs

This will generate a set of .gcda files in the current directory, one for each source code module. For native systems, it is typically most convenient to invoke gcov-pull(8) from the corresponding system service's source code directory (e.g., in /usr/src/minix/servers/vfs), so that the source, .gcno, and .gcda files for each module are in one place.

With crosscompilation, one will typically want to copy the .gcda files back to the crosscompilation environment, and analyze the results from there. Note that in this case, the .gcno files will also not be located in the same directory as the source code, but rather in its object directory.

The LLVM llvm-cov(1) utility can be used to produce meaningful output from the combination of the source, .gcno, and .gcda files. The llvm-cov(1) utility itself has been modified heavily in recent times, which means that different syntax is needed for different LLVM versions.

On systems with LLVM 3.4 (as of writing, the LLVM version on native MINIX3), one can get a view of the source code of, say, module "foo.c" with per-line coverage information using the following command, dumped to stdout:

> llvm-cov -gcno=foo.gcno -gcda=foo.gcda

With later LLVM versions, the following syntax can be used instead:

> llvm-cov gcov [-o path/to/gcno/files] foo.c

This will generate a file "foo.c.gcov" with the same per-line coverage output.

The output prefixes each source code line with a counter that shows the number of times the corresponding basic block has been invoked, or "#####" if the basic block has not been invoked yet. The newer llvm-cov has other features that may be interesting as well. For even fancier things such as generating webpages from the results, see various webpages on how to use LLVM GCOV.

==== Implementation ====

The system service part of the GCOV implementation is in libsys and hooked into the System Event Framework, which practically means that system services need not support code coverage support explicitly.

VFS is used as gateway to obtain the coverage information: gcov-pull(8) calls into VFS requesting that coverage data is to be obtained from a particular system service, into a buffer provided by gcov-pull. Unless VFS itself is the target, VFS relays the request to the system service identified by the label given to gcov-pull. The system service's SEF routines intercept the request and force the compiler-provided GCOV support routines to flush the coverage data. These compiler-provided GCOV support routines call particular hook functions that would typically write the resulting coverage data to files directly. These hook functions are implemented in libsys as well, and instead copy back the data to the buffer provided by gcov-pull. Once done, gcov-pull then produces the actual coverage data files.

The hook functions differ between GCC and LLVM. In our current implementation, libsys/llvm_gcov.c implements the LLVM hook functions by performing a translation to the GCC hook functions in libsys/gcov.c.

There are inherent dangers associated with VFS's relay function in this story: VFS's relay calls are blocking (ipc_sendrec) calls, which means that if the target system service does not respond to the request, VFS and with that the entire system may deadlock. This should be improved in the future, but for now the GCOV infrastructure should be considered as a somewhat dangerous, debugging-only facility. The corresponding system call is not part of the MINIX3 ABI and should never be used outside gcov-pull(8).