Junit: splitting integration test and Unit tests

I've inherited a load of Junit test, but these tests (apart from most not working) are a mixture of actual unit test and integration tests (requiring external systems, db etc).

So I'm trying to think of a way to actually separate them out, so that I can run the unit test nice and quickly and the integration tests after that.

The options are..

1. Split them into separate directories.

2. Move to Junit4 (from v3) and annotate the classes to separate them.

3. Use a file naming convention to tell what a class is , i.e. AdapterATest and AdapterAIntergrationTest.

3 has the issue that Eclipse has the option to "Run all tests in the selected project/package or folder". So it would make it very hard to just run the integration tests.

2: runs the risk that developers might start writing integration tests in unit test classes and it just gets messy.

1: Seems like the neatest solution, but my gut says there must be a better solution out there.

So that is my question, how do you lot break apart integration tests and proper unit tests?

You can split them very easily using JUnit categories and Maven.

This is shown very, very briefly below by splitting unit and integration tests.

Define A Marker Interface

The first step in grouping a test using categories is to create a marker interface.

This interface will be used to mark all of the tests that you want to be run as integration tests.

public interface IntegrationTest {}

Mark your test classes

Add the category annotation to the top of your test class. It takes the name of your new interface.

import org.junit.experimental.categories.Category;
@Category(IntegrationTest.class)
public class ExampleIntegrationTest{
  @Test
  public void longRunningServiceTest() throws Exception {
  }
}

Configure Maven Unit Tests

The beauty of this solution is that nothing really changes for the unit test side of things.

We simply add some configuration to the maven surefire plugin to make it to ignore any integration tests.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.11</version>
  <dependencies>
   <dependency>
     <groupId>org.apache.maven.surefire</groupId>
     <artifactId>surefire-junit47</artifactId>
     <version>2.12</version>
   </dependency>
  </dependencies>
  <configuration>
    <includes>
      <include>**/*.class</include>
    </includes>
    <excludedGroups>com.test.annotation.type.IntegrationTest</excludedGroups>
  </configuration>
</plugin>

When you do a mvn clean test only your unmarked unit tests will run.

Configure Maven Integration Tests

Again the configuration for this is very simple.

To run only the integration tests, use this:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.11</version>
  <dependencies>
   <dependency>
     <groupId>org.apache.maven.surefire</groupId>
     <artifactId>surefire-junit47</artifactId>
     <version>2.12</version>
   </dependency>
  </dependencies>
  <configuration>
    <groups>com.test.annotation.type.IntegrationTest</groups>
  </configuration>
</plugin>

If you wrap this in a profile with id IT, you can run only the fast tests using mvn clean install. To run just the integration/slow tests, use mvn clean install -P IT.

But most often, you will want to run the fast tests by default and all tests with -P IT. If that's the case, then you have to use a trick:

<profiles>
    <profile>
        <id>IT</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <excludedGroups>java.io.Serializable</excludedGroups> <!-- An empty element doesn't overwrite, so I'm using an interface here which no one will ever use -->
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

As you can see, I'm excluding tests that are annotated with java.io.Serializable. This is necessary because the profile will inherit the default config of the Surefire plugin, so even if you say <excludedGroups/> or <excludedGroups></excludedGroups>, the value com.test.annotation.type.IntegrationTest will be used.

You also can't use none since it has to be an interface on the classpath (Maven will check this).

Notes:

• The dependency to surefire-junit47 is only necessary when Maven doesn't switch to the JUnit 4 runner automatically. Using the groups or excludedGroups element should trigger the switch. See here.

• Most of the code above was taken from the documentation for the Maven Failsafe plugin. See the section "Using JUnit Categories" on this page.

• During my tests, I found that this even works when you use @RunWith() annotations to run suites or Spring-based tests.

We use Maven Surefire Plugin to run unit tests and Maven Failsafe Plugin to run integration tests. Unit tests follow the **/Test*.java **/*Test.java **/*TestCase.java naming conventions, integration tests - **/IT*.java **/*IT.java **/*ITCase.java. So it's actually your option number three.

In a couple of projects we use TestNG and define different test groups for integration/unit tests, but this is probably not suitable for you.

I would move up to Junit4 just for having it :)

You could separate them into different test suites. I don't know how they are organised in Junit3 but it should be easy in Junit4 just to build up test suites and put all the real unit tests in one of them and then use a second suite for the integration tests.

Now define a run configuration for both suites in eclipse and you can easily run a single suite. These suites also could be launched from an automated process allowing you to run the unit tests every time the source changes and maybe the integration tests (if they are really large) only once a day or once an hour.

I currently use separate directories due to organisational policy (and Junit 3 legacy) but I'm looking to transition to annotations myself now I'm on Junit 4.

I wouldn't be overly concerned about developers putting integration tests in your unit test classes - add a rule in your coding standards if necessary.

I'm interested to know what sort of other solutions there might be apart from annotations or physically separating the classes..

Using IfProfileValue spring annotation makes it possible to achieve this without a maven plugin or configuration required.

Annotate the integration test classes or methods using the IfProfileValue

import org.springframework.test.annotation.IfProfileValue;

@IfProfileValue(name="test-groups", value="integration")
public class ExampleIntegrationTest{
    @Test
    public void longRunningServiceTest() throws Exception {
    }
} 

To run using unit tests only:

mvn clean test

To run using integration test and unit tests:

mvn clean test -Dtest-groups=integration

Also, "Run all test" in an IDE would run only unit test. Add -Dtest-groups=integration to VM arguments to run both integration and unit tests.

There's not one right answer. As you've explained, there are several ways to do it which will work. I've done both the file naming scheme and splitting things into different directories.

It sounds like splitting thing up into different directories might work better for you, and that seems a little clearer to me, so I'd lean towards that.

I don't think I would try annotations because that seems more fine-grained to me. Do you really want these two types of tests mixed together in the same file? I wouldn't.