Java 7 Code Coverage with Gradle and Jacoco

Thanks

Steven Dicks’ post on Jacoco and Gradle is a great start to integrating Jacoco and Gradle, this is a small iteration on top of that work.

Java 7 Code Coverage

The state of Code Coverage took a serious turn for the worst when Java 7 came out. The byte-code changes in Java 7 effectively made Emma and Cobertura defunct. They will not work with Java 7 constructs. Fortunately there is a new player in town called JaCoCo (for Java Code Coverage). JaCoCo is the successor to Emma which is being built on the knowledge gained over the years by the Eclipse and Emma teams on how best to do code coverage. And works with Java 7 out-of-the-box.

The advantage of using established tools is that they generally are well supported across your toolchain. JaCoCo is fairly new and so support in Gradle isn’t so smooth. Fortunately Steven’s post got me started down the right path. The one thing that I wanted to improve right away was to use transitive dependency declarations as opposed to having local jar files in my source repository. JaCoCo is now available in the Maven repos so we can do that. One thing to note is that the default files build in the Maven repo are Eclipse plugins, so we need to reference the “runtime” classifier in our dependency

The Gradle Script


configurations {
codeCoverage
codeCoverageAnt
}
dependencies {
codeCoverage 'org.jacoco:org.jacoco.agent:0.5.10.201208310627:runtime@jar'
codeCoverageAnt 'org.jacoco:org.jacoco.ant:0.5.10.201208310627'
}
test {
systemProperties = System.properties
jvmArgs "-javaagent:${configurations.codeCoverage.asPath}=destfile=${project.buildDir.path}/coverage-results/jacoco.exec,sessionid=HSServ,append=false",
'-Djacoco=true',
'-Xms128m',
'-Xmx512m',
'-XX:MaxPermSize=128m'
}
task generateCoverageReport << { ant { taskdef(name:'jacocoreport', classname: 'org.jacoco.ant.ReportTask', classpath: configurations.codeCoverageAnt.asPath) mkdir dir: "build/reports/coverage" jacocoreport { executiondata { fileset(dir: "build/coverage-results") { include name: 'jacoco.exec' } } structure(name: project.name) { classfiles { fileset(dir: "build/classes/main") { exclude name: 'org/ifxforum/**/*' exclude name: 'org/gca/euronet/generated**/*' } } sourcefiles(encoding: 'CP1252') { fileset dir: "src/main/java" } } xml destfile: "build/reports/coverage/jacoco.xml" html destdir: "build/reports/coverage" } } }

A Few Details

The magic is in the jvmArgs of the test block. JaCoCo is run as a Java Agent which uses the runtime instrumentation feature added in Java 6 to be able to inspect the running code. Extra arguments can be added to JaCoCo there including things like excludes to exclude specific classes from coverage. The available parameters are the same as the maven JaCoCo parameters.

The generateCoverageReport task converts the jacoco.exec binary into html files for human consumption. If you're just integrating with a CI tool, like Jenkins, then you probably don't need this, but it's handy for local use and to dig into the details of what's covered.

Loose Ends

One problem that I ran into was referencing project paths like the project.buildDir from within an Ant task. Hopefully someone will come along and let me know how that's done.

MSBuild Task for PartCover

I continue to lament the dearth of option for Test Coverage in the .NET world.

In the Java world you have open source tools like Emma and Cobertura that are widely used and supported (and many more) as well as proprietary tools like Clover available.

.NET we have an open source NCover SF that requires you to do odd code instrumentation and is essentially dead it seams, another NCover which is proprietary and costs money and PartCover which is open source, but doesn’t seem real active.

Don’t get me wrong, NCover.org is a good option if you are willing to spend the money for it. But with a team of 30+ and a CI server, I’m not sure if I want to drop $10k on it. (NCover up until version 1.5.8 was Free Software (GPL) before it was closed. Who has the source and why haven’t you forked it yet?)

]]>

If you’re not willing to pay that basically leaves PartCover. But of course you want to integrate your code coverage with your automated build. There was no support for MSBuild out of the box, so I decided to build it.

Creating an MSBuild Task

I need to do 2 things to run PartCover:

  1. Register the native PartCover.CorDriver.dll
  2. Execute the PartCover.exe with the proper options

Register Native DLL

To see the details on how to register a native DLL using .NET code so my earlier post Register and Unregister COM DLL from .NET Code.

Execute PartCover

The MSBuild framework provides a ToolTask base class whose whole purpose is for executing external command line tools. I used this as the base of the task.

1. ToolName

First you override the ToolName property to return the name of the EXE to run. Nothing special here, it’s just the executable name.


protected override string ToolName
{
get { return "PartCover.exe"; }
}

2. Properties

Next to start build the task you go about defining all of the settings that a user will need to set to execute the task. You then create those as Properties on the class and they will be set by MSBuild. Start with the simple things that someone will need to pass to get the tool to execute properly. You can build from there for other properties. If possible give the properties sane defaults so that people don’t have to override them in their build file.


// ...

///

/// The application to execute to get the coverage results.
/// Generally this will be your unit testing exe.
///

public string Target
{
get { return _target; }
set { _target = value; }
}

///

/// The arguments to pass to the executable
///

public string TargetArgs
{
get { return _targetArgs; }
set { _targetArgs = value; }
}

public string WorkingDirectory
{
get { return _workingDirectory; }
set { _workingDirectory = value; }
}

// ...

3. Command Arguments

Then you need to override string GenerateCommandLineCommands() method. The whole purpose of this method is to construct any command line parameters that need to be passed to the ToolName command using the Properties defined in the task.


protected override string GenerateCommandLineCommands()
{
StringBuilder builder = new StringBuilder();
AppendIfPresent(builder, "--target", Target);
AppendIfPresent(builder, "--target-work-dir", WorkingDirectory);
AppendIfPresent(builder, "--target-args", QuoteIfNeeded(TargetArgs));
AppendIfPresent(builder, "--output", Output);

AppendMultipleItemsTo(builder, "--include", Include);
AppendMultipleItemsTo(builder, "--exclude", Exclude);

Log.LogCommandLine(builder.ToString());

return builder.ToString();
}

5. Execute

Finally, if you have anything special to do, you can override the Execute(). In this case, I wanted to handle the registering and de-registering of the Core.dll. Make sure that you call the base.Execute() method so that the TaskTarget can do the work that it needs to do.


public override bool Execute()
{
string corDriverPath = Path.Combine(ToolPath, CorDriverName);
Log.LogMessage("CoreDriver: {0}", corDriverPath);
using (Registrar registrar = new Registrar(corDriverPath))
{
registrar.RegisterComDLL();
return base.Execute();
}
}

To see the whole thing, download the files at the bottom of this post.

How to Use PartCover with MSBuild

Now that you have a Custom task you need to create a Target in your MSBuild file to execute the task.









Download the code:
PartCover MSbuild.zip

Good luck and I hope someone else finds this useful.