Visual Studio has code coverage built in, but only if you run VS Premium or better. Sometimes this isn’t available, which is what sent me looking for a free alternative.
OpenCover fits this goal perfectly. The output isn’t as pretty as VS, but no worries. ReportGenerator has integrations with the OpenCover output and creates some really nice reports. Below is the output and the simple steps that worked for me.
Summary page, with clickable links to details for each class.
Class level summary, including cyclomatic complexity and per-method details.
Source code lines with highlighting for quick visual inspection.
Getting the Tools
As with most things these days, Nuget is the answer. There are packages for both of these tools that pull the bits down into your project’s “packages” folder.
PM> install-package OpenCover PM> install-package ReportGenerator
This creates some version specific folders in your .packages folder. In my case, “OpenCover.4.5.1604” and “ReportGenerator.188.8.131.52”. Awesome! Couple that with a solution that uses NuGet package restore and you don’t even have to check them in, which makes Git happy!
Generating the Reports
To get the report, you basically need to do 3 steps:
- Build the solution (and restore nuget packages)
- Run unit tests via OpenCover
- Run ReportGenerator on the output
Let’s break that down… We’ll use my “Monitored Undo Framework” project as a real world example. I added the Nuget packages and enabled Nuget Package Restore on the solution.
For step 2, I crafted a command line that would tell OpenCover to run my MSTest unit tests, saving the output to a “reports” directory. (Note: The carrot “^” simply allows the statement to exist over multiple lines, rather than one long single line statement.)
REM Run unit tests through OpenCover REM This allows OpenCover to gather code coverage results .\packages\OpenCover.4.5.1604\OpenCover.Console.exe^ -register:user^ -target:MSTest.exe^ -targetargs:"/noresults /noisolation /testcontainer:\..\releases\Latest\NET40\MonitoredUndoTests.dll"^ -filter:+[*]*^ -output:.reportoutput.xml
The above statement passes the arguments needed by OpenCover and MSTest to make it all happen. In this case, I have a single unit test project/dll and I run all tests, so it’s simple. If you have NUnit unit tests, you simply substitute the alternate “-target” and “-targetargs” parameters to OpenCover.
For step 3, I pass the output of OpenCover into ReportGenerator:
REM Generate the report .\packages\ReportGenerator.184.108.40.206\ReportGenerator.exe^ -reports:.reportoutput.xml^ -targetdir:.report^ -reporttypes:Html,HtmlSummary^ -filters:-MonitoredUndoTests*
The above uses the “output.xml” file, plus some command line parameters to create a pretty nice looking code coverage report. ReportGenerator does some nice things to make these reports extra awesome.
- It reviews the PDB to locate the relavant source code.
- Pulls the source into the report, with green / red highlights to show what’s covered
- Calculates coverage percentages and cyclomatic complexity.
- Presents the info in a nice, hierarchical table with bar charts showing coverage percentage for each class.
Putting it All Together
Getting this to work via a single (ok double) click looks like the following batch file, which I put in the same directory as my solution. In my case, I happen to have a different output directory than usual, but you can adjust to your tastes.
REM Bring dev tools into the PATH. call "C:Program Files (x86)Microsoft Visual Studio 11.0Common7ToolsVsDevCmd.bat" mkdir .report REM Restore packages msbuild ..nugetNuGet.targets /target:RestorePackages REM Ensure build is up to date msbuild "..\src\Monitored Undo Framework.sln"^ /target:Rebuild^ /property:Configuration=Release;OutDir=..\..\releases\Latest\NET40 REM Run unit tests through OpenCover REM This allows OpenCover to gather code coverage results .\packages\OpenCover.4.5.1604\OpenCover.Console.exe^ -register:user^ -target:MSTest.exe^ -targetargs:"/noresults /noisolation /testcontainer:..\releases\Latest\NET40\MonitoredUndoTests.dll"^ -filter:+[*]*^ -output:.\reportoutput.xml REM Generate the report .\packages\ReportGenerator.220.127.116.11\ReportGenerator.exe^ -reports:.reportoutput.xml^ -targetdir:.report^ -reporttypes:Html,HtmlSummary^ -filters:-MonitoredUndoTests* REM Open the report start .\reportindex.htm pause
If you’re ambitious, you could set this up to run with each build by simply adding it as a build step in your projects. And with the files under your full control, you could do whatever other processing / checks you’d like to do as part of a CI build too.
For larger projects, you’ll find the other command line parameters of good use. You can easily:
- Narrow down what coverage results are included in the report. This could be helpful to exclude 3rd party code, unit tests, etc.
- Filter the list of tests that are actually executed. MSTest lets you filter by “category” or “trait” using the “/category:” parameter. You can even use logical operators to include / exclude traits as needed. The trick is getting the command line to properly handle the embedded quotes. Using two double quotes next to each other seems to do the trick: “MSTest.exe […] /category:””!Performance&!Web Service Integration”””
Until next time…