Code Coverage Tools for Scala

Unit Testing Scala Continuous Integration Hudson Code Coverage

Unit Testing Problem Overview

What are the available code coverage tools for Scala?

I have Scala spec tests and a Hudson continuous integration set-up. Is there something I can hook-in to this setup to measure and track code coverage?

Unit Testing Solutions

Solution 1 - Unit Testing

SCCT is a compiler plugin which instruments the classes to gather coverage data:

http://mtkopone.github.com/scct/

Solution 2 - Unit Testing

I use Cobertura. However, any Java coverage tool should work just fine. The only catch is that you will end up with a large number of auto-generated classes in your coverage list. This is because while Scala compiles down into very natural JVM bytecode, it is forced to produce an unnaturally large number of classes to accommodate common functional features like lazy evaluation.

Solution 3 - Unit Testing

Undercover is little better.

Solution 4 - Unit Testing

One problem with non-mainstream languages (such as Scala) is that tools are hard to find, because they are hard to build.

This technical paper http://www.semdesigns.com/Company/Publications/TestCoverage.pdf">Branch Coverage for Arbitrary Languages Made Easy (I'm the author) describes how to build test coverage tools for langauges in systematic way to help get around this problem, using a generic tool-building infrastructure.

We've implemented http://www.semanticdesigns.com/Products/TestCoverage">test coverage tools for Java, C#, COBOL, C, C++, PL/SQL, ... this way, including instrumenters, data collection and test coverage display and reporting. It would be straightforward to implement Scala like this.

The solutions posed by other answers produces confusing information from the implementation of Scala ("auto genreated classes"). What developers want to see is coverage data in terms of their code. The approach we use instruments the source code, so the results are stated entirely and only in terms of the source code; even the test coverage viewer shows the source code covered with coverage information.

Solution 5 - Unit Testing

I use jacoco. It does not require compile- or runtime- dependencies, instruments classes on the fly w/o special instrumentation phase.

Also it integrated with Sonar and published on Maven Central.

Here is example: https://github.com/Godin/sonar-experiments/tree/master/jacoco-examples/scala-example

I would like to add better reporting: more detailed branch coverage makrup, excluding of generated classes/methods, and to be handy like ScalaDoc (see SCCT reports for example)

Solution 6 - Unit Testing

I've put together a SBT plugin called xsbt-coveralls-plugin that uses scct under the hood, but publishes the results to http://coveralls.io.

Disclaimer: I've only just built this plugin yesterday (10th March 2013) so don't expect it to be perfect yet, but do send bugs and feature requests to the github page

Still, it's good if you want to code coverage reports to be publicly visible. Check out an example of the results here

Content Type	Original Author	Original Content on Stackoverflow
Question	arnab	View Question on Stackoverflow
Solution 1 - Unit Testing	Joni	View Answer on Stackoverflow
Solution 2 - Unit Testing	Daniel Spiewak	View Answer on Stackoverflow
Solution 3 - Unit Testing	user152392	View Answer on Stackoverflow
Solution 4 - Unit Testing	Ira Baxter	View Answer on Stackoverflow
Solution 5 - Unit Testing	Andriy Plokhotnyuk	View Answer on Stackoverflow
Solution 6 - Unit Testing	theon	View Answer on Stackoverflow