Code Coverage - Testing with OpenCover and PartCover

 

Just some days ago, a new code coverage tool was released for the first time. It's called OpenCover. Shaun Wilde created this tool, since PartCover 4 has some issues that are difficult to resolve with the current code base. He describes some of them on his blog.
In this post I will do a comparison between OpenCover and PartCover and show how these tools can be used to get the coverage of unit tests.

Comparison

OpenCover runs on 32/64 Bit Windows and in contrast to PartCover you can capture the coverage of 64 Bit assemblies.
PartCover does not support 64 Bit assemblies, and does not work on 64 Bit systems out of the box. You can get it working by using this workaround.
The following table shows which combinations will work:

OpenCoverPartCover
NUnit32/64 Bit32 Bit
MSTest32 Bit (MSTest does not support 64 Bit)32 Bit

Advantages of OpenCover

OpenCover is currently released as a beta version, but it works pretty fine already. In my opinion it has the following advantages:

  • Supports 64 Bit assemblies
  • No manual DLL registration is required
  • Lower memory usage

Usage

The provided sample solution contains a simple class and a unit test which can be executed with MSTest and NUnit. The solution is already compiled in 32 Bit, 64 Bit and AnyCPU mode.
Furthermore the download contains several batch files which enable you to execute the the unit test with OpenCover and PartCover.

Please note: The batch files for MSTest will only work on 64 Bit systems. If you use a 32 Bit OS please adjust the path to MSTest.exe.

OpenCover can be executed in the following way:

OpenCover.Console.exe -register:user -target:nunit-console-x86.exe -targetargs:"/noshadow Test.dll" -filter:+[*]* -output:coverage.xml

PartCover can be executed in the following way:

PartCover.Registration.exe PartCover.dll
PartCover.exe --target "nunit-console-x86.exe" --target-args "Test.dll /noshadow" --include [*]* --output coverage.xml
PartCover.Registration.exe

After running the unit tests, a XML file containing the coverage information is generated. If you like, you can use ReportGenerator to generate a nice HTML report out of it.

In a previous post I described how you could integrate the code coverage in your build process by using MSBuild.

One final hint: If you want to execute/debug NUnit tests within Visual Studio 2010 you can use this NUnitForVS. This plugin lets you execute the test with the regular test tools:

TestToolbar

Downloads

Feedly Feedly Tweet


Related posts


Comments


Daniel

Daniel

1/20/2016

@Ganesh: Please make sure that PDB files are present and use the correct command line parameters for OpenCover. See https://github.com/OpenCover/opencover/wiki/Usage The correct parameters depend on the test framework you are using.


Ganesh

Ganesh

1/20/2016

Hi, I am trying to generate code coverage for a .net application. It's a 64 bit windows server 2012 machine and decided to use open cover and report generator tool. Whenever I try to generate XML file using open cover, from command prompt, I keep getting an error which says invalid switch "d" and pdb files are missing Please tell me what am I missing here !!! Thanks a lot in advance !!!


Mewanbanjop Mawroh

Mewanbanjop Mawroh

3/27/2014

@Daniel: I found the problem. I replaced "-filter:+[MyData.*]*" with "-filter:+[MyData*]*". The 'dot' made a difference. Thanks for your speedy reply.


Daniel

Daniel

3/27/2014

@Mewanbanjop: Hard to tell what's wrong without the code. But have you tried -filter:+[*]* ? If this works, you could start to add filters again and figure out which one causes the classes from MyData.dll to disappear from the report.


Mewanbanjop Mawroh

Mewanbanjop Mawroh

3/27/2014

@Daniel, Sorry for late reply. Yes the PDBs are in the same bin directory and I pass the following filter to open cover "-filter:+[MyData.*]*" since there are other dlls in the bin folder.


Daniel

Daniel

3/25/2014

@Mewanbanjop: Are the PDBs in the same directory? I think OpenCover needs them. Do you pass any filters to OpenCover? Please have a look at my sample project. Is there something different to your approach?


Mewanbanjop Mawroh

Mewanbanjop Mawroh

3/25/2014

Hi Daniel, I have a C# project say MyData and a corresponding nunit test project for it, MyDataTest. I build both the projects and run opencover on MyDataTest.dll. I then run Reportgenerator against the xml generated by open-cover. The html result displays only the MyDatatest.dll and shows the test classes. However the Data.dll and its corresponding classes are not shown in the result. It is wierd as I would expect the opposite or atleast both. i.e MyData.dll and its classes be shown as the output. Am I missing something??


Damien

Damien

3/6/2012

Thanks man, that was really helpful.


Marco Medrano

Marco Medrano

12/13/2011

Good article and the tool helps so much. Thanks a lot, specially by the SampleApplication_VS2010.zip


Daniel

Daniel

9/13/2011

@Malcolm: Thanks for your comment. I will take a look at the problem.


Malcolm

Malcolm

9/13/2011

I tried out ReportGenerator with OpenCover and they seem to work well. One issue I have found. It doesn't show the coverage for nest/inner classes. Is there config for this? Or would it be easy to implement?


Deedee123

Deedee123

8/9/2011

OK, yes that is right. Actually my application checks to see if a method has a coverage of more than 90%, if not, it will flag it as an error on the server build. In that case, if there are 8-9 methods in a class and someone is writing tests to cover them, it is helpful to know at a glance which methods still do not meet that criteria (instead of having to scroll down and look for red highlights every time, and calculate the method coverage manually). This is a common feature in NCoverExplorer, PartCover browser..and so I thought maybe ReportGenerator has it too. Thanks for you reply.


Daniel

Daniel

8/9/2011

@Deedee123: Currently ReportGenerator only supports coverage by line. I'm not sure if a method based coverage is really helpful. If a method has a coverage less that 100%, you have to look at the line coverage to figure out what is missing. And classes with many many lines of code should be avoided anyway :-)


Deedee123

Deedee123

8/9/2011

Thanks for the article! However, does the ReportGenerator display the coverage down to the method level too? I can see only the class coverage %, and clicking on it shows the lines that are covered/not covered. It will be great to be able to find out which methods are uncovered/less covered at a glance, rather than having to always scroll down to see the lines (esp when classes are huge) Thanks,


Alexandre Victoor

Alexandre Victoor

6/30/2011

Tanks a lot for this article ! No reason anymore not to try opencover :-)


Daniel

Daniel

6/22/2011

@Shaun: Thanks for your comment. I updated my post accordingly.


Shaun Wilde

Shaun Wilde

6/22/2011
http://scubamunki.blogspot.com

PartCover also comes with a registration tool that does the per user registration for those users running under restrictive accounts. Thanks for your effort in supporting OpenCover