Code Coverage via OpenCover and ReportGenerator

Visual Studio has code coverage built in, but only if you run VS Premium or better. Sometimes this isn’t available, which is what sent me looking for a free alternative.

OpenCover fits this goal perfectly. The output isn’t as pretty as VS, but no worries. ReportGenerator has integrations with the OpenCover output and creates some really nice reports. Below is the output and the simple steps that worked for me.

Summary page, with clickable links to details for each class.
Code coverage summary of Monitored Undo Framework

Class level summary, including cyclomatic complexity and per-method details.
Code coverage summary of a class.

 

Source code lines with highlighting for quick visual inspection.
Code coverage source code highlighting.

Getting the Tools

As with most things these days, Nuget is the answer. There are packages for both of these tools that pull the bits down into your project’s “packages” folder.

PM> install-package OpenCover
PM> install-package ReportGenerator

This creates some version specific folders in your .packages folder. In my case, “OpenCover.4.5.1604” and “ReportGenerator.1.9.0.0”. Awesome! Couple that with a solution that uses NuGet package restore and you don’t even have to check them in, which makes Git happy!

Generating the Reports

To get the report, you basically need to do 3 steps:

  1. Build the solution (and restore nuget packages)
  2. Run unit tests via OpenCover
  3. Run ReportGenerator on the output

Let’s break that down… We’ll use my “Monitored Undo Framework” project as a real world example. I added the Nuget packages and enabled Nuget Package Restore on the solution.

For step 2, I crafted a command line that would tell OpenCover to run my MSTest unit tests, saving the output to a “reports” directory. (Note: The carrot “^” simply allows the statement to exist over multiple lines, rather than one long single line statement.)

REM Run unit tests through OpenCover REM This allows OpenCover to gather code coverage results .packagesOpenCover.4.5.1604OpenCover.Console.exe^  -register:user^  -target:MSTest.exe^  -targetargs:"/noresults /noisolation /testcontainer:..releasesLatestNET40MonitoredUndoTests.dll"^  -filter:+[*]*^  -output:.reportoutput.xml   

The above statement passes the arguments needed by OpenCover and MSTest to make it all happen. In this case, I have a single unit test project/dll and I run all tests, so it’s simple. If you have NUnit unit tests, you simply substitute the alternate “-target” and “-targetargs” parameters to OpenCover.

For step 3, I pass the output of OpenCover into ReportGenerator:

REM Generate the report .packagesReportGenerator.1.9.0.0ReportGenerator.exe^  -reports:.reportoutput.xml^  -targetdir:.report^  -reporttypes:Html,HtmlSummary^  -filters:-MonitoredUndoTests*   

The above uses the “output.xml” file, plus some command line parameters to create a pretty nice looking code coverage report. ReportGenerator does some nice things to make these reports extra awesome.

  • It reviews the PDB to locate the relavant source code.
  • Pulls the source into the report, with green / red highlights to show what’s covered
  • Calculates coverage percentages and cyclomatic complexity. 
  • Presents the info in a nice, hierarchical table with bar charts showing coverage percentage for each class.

Putting it All Together

Getting this to work via a single (ok double) click looks like the following batch file, which I put in the same directory as my solution. In my case, I happen to have a different output directory than usual, but you can adjust to your tastes.

REM Bring dev tools into the PATH. call "C:Program Files (x86)Microsoft Visual Studio 11.0Common7ToolsVsDevCmd.bat"  mkdir .report  REM Restore packages msbuild ..nugetNuGet.targets /target:RestorePackages  REM Ensure build is up to date msbuild "..srcMonitored Undo Framework.sln"^  /target:Rebuild^  /property:Configuration=Release;OutDir=....releasesLatestNET40  REM Run unit tests through OpenCover REM This allows OpenCover to gather code coverage results .packagesOpenCover.4.5.1604OpenCover.Console.exe^  -register:user^  -target:MSTest.exe^  -targetargs:"/noresults /noisolation /testcontainer:..releasesLatestNET40MonitoredUndoTests.dll"^  -filter:+[*]*^  -output:.reportoutput.xml  REM Generate the report .packagesReportGenerator.1.9.0.0ReportGenerator.exe^  -reports:.reportoutput.xml^  -targetdir:.report^  -reporttypes:Html,HtmlSummary^  -filters:-MonitoredUndoTests*  REM Open the report start .reportindex.htm  pause 

Next Steps

If you’re ambitious, you could set this up to run with each build by simply adding it as a build step in your projects. And with the files under your full control, you could do whatever other processing / checks you’d like to do as part of a CI build too.

For larger projects, you’ll find the other command line parameters of good use. You can easily:

  • Narrow down what coverage results are included in the report. This could be helpful to exclude 3rd party code, unit tests, etc.
  • Filter the list of tests that are actually executed. MSTest lets you filter by “category” or “trait” using the “/category:” parameter. You can even use logical operators to include / exclude traits as needed. The trick is getting the command line to properly handle the embedded quotes. Using two double quotes next to each other seems to do the trick: “MSTest.exe […] /category:””!Performance&!Web Service Integration”””

Until next time…

Cheers!

One thought on “Code Coverage via OpenCover and ReportGenerator

Add yours

  1. Hi @Surjeet,You should be able to run whatever tests you want and capture the coverage, so long as OpenCover is attached to the code that you want to measure. That usually means having OpenCover launch the app or site for you.What kind of functional tests do you have? – Nathan

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a website or blog at WordPress.com

Up ↑

%d bloggers like this: