LCOV Documentation (2.6)

LCOV is a graphical tool which collects and aggregates coverage data from multiple sources then generates HTML reports to visualize the data. It supports line, function, branch and MC/DC coverage. LCOV was originally written to display coverage data GCC’s coverage testing tool gcov - but has been enhanced to support multiple tools and languages - including C/C++, Perl, Python, Java and SystemVerilog.

Manual Pages

Callback Scripts

LCOV provides callback scripts to customize version control system integration, coverage criteria enforcement and various other purposes:

  • Annotate scripts:

    • --annotate-script option

    • extract file author/date data (examples: gitblame.pm, p4annotate.pm)

  • Version scripts:

    • --version-script option

    • extract and compare file versions (examples: gitversion.pm, batchGitVersion.pm, P4version.pm, get_signature)

  • Diff scripts:

    • used by --diff-file option

    • generate unified source text diffs (examples: gitdiff, p4udiff)

  • Criteria scripts:``

    • –criteria-script`` option

    • check and enforce coverage thresholds (examples: criteria.pm, threshold.pm)

  • Subset/code review:

    • --select-script option

    • generate HTML report showing only particular subset of sources (example: select.pm)

  • Unreachable code:

    • –unreachable-script` option

    • tag unreachable expressions so they are not counted/do not appear in the coverage report (example: unreach.pm)

  • Modify code appearance:

    • --simplify-script option

    • shorten very long C++ template names (example: simplify.pm)

  • Find corresponding source file (in non-trivial build environment:

    • --resolve-script option

The callback scripts shipped with the LCOV release are primarily intended only as examples of possible callback implementations. The expectation is that users will want or need to customize the callbacks in order to support their specific environment and requirements.

For details, see the *-script option section in the individual tool man pages (genhtml, llvm2lcov, etc.) Note that not all tools support all options. For example, --diff-file and --annotate-script are supported by genhtml only.

Getting Started

  1. Point your environment to your installation of LCOV - or install LCOV using make install.

    • Note that sphinx-build is required in order to build documentation, but you can skip documentation building by passing passing the make LCOV_NO_DOC=1 flag to your make command.

    • Similarly, you can skip building the LCOV XS extension by passing make LCOV_NO_XS=1 ... to your make command.

    • Note that, if you pass COVERAGE=1 to your make command, then the XS implementation will be instrumented for coverage data collection. See .../tsts/Makefile for more information.

    • The XS extension is C++ and requires g++ 8 or later (14 or later recommended - and required for COVERAGE=1 to collect MC/DC data). If the first g++ on your PATH is older than that, select the compiler explicitly:

      • make LCOV_CXX=/path/to/g++ ... names the compiler directly (CXX is honored too; LCOV_CXX wins).

      If no usable compiler is found, the build will fail. LCOV loads the extension when present and silently uses its pure-Perl implementation when it is not, so a failed extension build does not corrupt results - it only makes execution slower, with nothing to indicate why. Check which implementation is in use with:

      perl -I$LCOV_HOME/lib -e 'require lcovutil; print $lcovutil::XS_LOADED ? "XS\n" : "pure Perl\n"'

      Setting LCOV_PURE_PERL=1 forces the pure-Perl implementation at run time even when the extension is available.

  2. Prepare your executables:

    • C/C++: compile and link with coverage flags: --coverage or -fprofile-arcs -ftest-coverage.

    • Perl, Python, etc. - see the other tools in this release.

  3. Run your tests

  4. Capture coverage

    • C/C++: lcov --capture --directory . --output-file coverage.info

    • Other languages: see other tools in this release and/or consult your toolchain documentation.

  5. Generate HTML report: genhtml coverage.info --output-directory out

  6. Read the man pages and/or the HTML documentation to discover other capabilities and options.

Windows path names

A path can be written in Windows style - with a drive letter, and with either forward or backslash separator, or a mixture of the two - even when the tool is run by a Perl or a Python (e.g., on Cygwin, MSYS, or git-bash) which only understands Unix-style paths (forward slash, no drive). Such a perl cannot open D:\my\local\tools\lcov\bin\genhtml at all - but such a path may be written on the command line by a Windows caller, or come from an environment variable - so LCOV has to handle them all.

The drives are mounted in such an installation, so the same file has a name that perl does understand, and the tools translate Windows names to that format: forward slashes throughout, and the mount point of the drive in front. D: is /d under MSYS and git-bash and /cygdrive/d under Cygwin, each configurable in the installation’s own fstab. So, with LCOV installed in D:\my\local\tools\lcov and JaCoCo on the Z: drive -

Written as

Used as

D:\my\local\tools\lcov\bin\genhtml

/d/my/local/tools/lcov/bin/genhtml

D:/my/local/tools/lcov/bin/genhtml

the same

D:\my\local/tools\lcov/bin\genhtml

the same

Z:\jacoco\lib\jacococli.jar

/z/jacoco/lib/jacococli.jar

lib\jacococli.jar

lib/jacococli.jar

\\build01\share\jacoco\jacococli.jar

//build01/share/jacoco/jacococli.jar

Q:\jacoco\lib\jacococli.jar

Q:/jacoco/lib/jacococli.jar

The last three rows are the paths which name no mounted drive. A backslash is the Windows separator wherever it appears, so turning the separators around is all a relative path needs, and it is what makes a UNC name usable as well - that is //host/share/... on these perls. Q: in the last row is a drive which this installation has not mounted: there is no name to translate it to, so it is left as it stands and the drive you named is what the complaint about it names.

Note that only the two usual mount points are checked, so a drive which your installation’s fstab mounts somewhere else - /mnt/d, say - has to be named the way that perl names it. And a drive-relative name, D:jacococli.jar, is left alone: there is no per-drive current directory to resolve it against.

A native Windows perl - one whose $^O is MSWin32 - understands a Windows path itself: it opens one, it makes one absolute, and it splits one into a directory and a file name. Nothing is translated for such a perl, in either direction, and a Windows path is what it is handed and what it hands on. Nothing is translated on a Unix host either, where a drive letter names nothing: what you write there is what is used.

py2lcov and xml2lcov are Python rather than Perl, and do the same thing, for the same reason: the os.path of a Cygwin or MSYS python is posixpath, which does not know a drive letter either. The same table above describes what they do with a name, and a native Windows python - one whose sys.platform is win32 - is left alone in the same way as a native Windows perl.

Example

The LCOV source distribution includes a complete working example in directory $LCOV_HOME/share/lcov/example (or in the example subdirectory, if you are using an lcov source version).

The example demonstrates:

  • Compiling C/C++ code with coverage instrumentation

  • Running tests and capturing coverage data

  • Generating HTML coverage reports

  • Using differential coverage analysis

  • Using coverage and part of your code review process

To see some examples of LCOV generated HTML coverage reports:

$ cp -r $|TOOL_NAME|_HOME/share/lcov/example .
$ cd example
$ make

Review the ‘make’ log and the generated data, and then point a web browser into the resulting reports.

The example builds with GCC by default. You will need to make a few changes if you want to use LLVM instead.

  • Default view:

    • Point your browser to output/index.html

  • Hierarchical view:

    • Point your browser to hierarchical/index.html

    • Note that that the coverage data is the same - only the report format is different:

      • Follows directory structure, similar to MS file viewer (--hierarchical flag)

      • Additional navigation links also enabled (--show-navigation flag)

  • Differential coverage:

    • Point your browser to exampleRepo/differential/index.html

    • This example is slightly complicated because it emulates a moderately realistic project in that it pretends to see project changes:

      • updates to two project source files example.c and iterate.c

      • change to the test suite: only one test of updated code rather than 3 of the original code

      The Makefile simulates this by checking code into a git repo, building an executable and then updating a few source files, rebuilding, and running some tests.

      There is one repo, exampleRepo, and both this example and the Java coverage with JaCoCo example below use it - the way a project written in more than one language keeps one repo rather than one per language. It is built by whichever of those examples runs first, with the sources of both of them checked in as its baseline revision, and is removed by make clean. This example modifies some of those sources and commits them as a second revision, so it starts by putting the repo back at baseline - which does nothing at all the first time it is run.

  • Code review:

    • point your browser to exampleRepo/review/index.html

    • This example builds on the Differential coverage example, above to emulate a possible code review methodology in which adds code coverage to the review criteria. The intent is to generate a reduced report which shows only the code changes which negatively affect code coverage - while removing other details which only distract from the review.

      • Use the genhtml --select-script ... feature to show only new source code which was negatively affected by the change under review (uncovered and/or lost code). You might want to modify the select criteria to include positive change (e.g., GNC, GBC, and GIC categories).

      • Real use cases are likely to use more sophisticated select-script callbacks (e.g., to select from a range of changelists).

      • The example uses caching and profile history to improve runtime performance - see the man pages for a more detailed description of the features. There is no effect with a tiny example - but a real project may see benefit. The spreadsheet.py application script can be used to convert JSON profile files into more readable excel spreadsheets. This can be useful to see the effect (if any) of the caching and/or history features, and can show where time is spent for your example. This can be helpful, to suggest opportunities to optimize the LCOV implementation.

  • Create diff data from previous HTML coverage report and current source code (i.e., when revision control has not been updated or is not available).

    • see make example_html2lcov and/or point your browser to repo2/differential2/index.html and repo2/review/index.html to see reports generated using this data.

  • Java coverage with JaCoCo:

    • point your browser to exampleRepo/jacoco_report/index.html

    • make example_java compiles HelloWorld.java with debug information, runs it with the JaCoCo agent attached, translates the data JaCoCo collected with the jacoco2lcov tool, and generates a report using the same author/date and version callbacks as the Differential coverage example above. The source is checked in for the same reason: that is where the callbacks read the annotations and versions from. It is checked into the same exampleRepo as the C sources of that example - see there.

    • JaCoCo reports line, branch and function (method) coverage. There is no MC/DC data in a JaCoCo report.

    • This example needs a JDK and a JaCoCo installation. The ./java_avail.sh script tries to find them - and complains if it can’t.

      • java and javac have to be on PATH - or JAVA_HOME has to name the JDK, and they are then used from $JAVA_HOME/bin. JAVA_HOME wins when both are true. It has to be a JDK rather than a JRE, because the example compiles its own source.

      • JACOCO_HOME has to be set, and to name a directory with jacocoagent.jar in it (either at the top level or under lib, which is where a JaCoCo release puts it).

      The Makefile runs java_avail.sh before running the java example - and skips the example if something is missing. make still runs to completion on a machine which has no Java.

      How java and JaCoCo come to be in your environment is up to you - they may be installed on the system, unpacked anywhere and named with JAVA_HOME and JACOCO_HOME, or provided by whatever environment or package manager your site uses. The example only checks whether they exist - then uses them if they do.

Feel free to edit the Makefile or to run the lcov utilities directly, to see the effect of other options that you find in the lcov man pages.

AUTHOR

Peter Oberparleiter <Peter.Oberparleiter@de.ibm.com>

Original LCOV implementation

Henry Cox <henry.cox@mediatek.com>

Differential coverage, age/author binning, etc..

LCOV Community

Ideas, suggestions, fixes…lots of help

License

LCOV is licensed under the GNU General Public License. See the LICENSE file for details.