Continuing test execution in junit4 even when one of the asserts fails

I have my existing framework built up using Jfunc which provides a facility to continue exection even when one of the asserts in the test case fails. Jfunc uses junit 3.x framework. But now we are migrating to junit4 so I can't use Jfunc anymore and have replaced it with junit 4.10 jar.
Now the problem is since we have extensively used jfunc in our framework, and with junit 4 we want to make our code continue the execution even when one of the asserts fails in a test case.
Does anyone has any suggestion/idea for this, i know in junit the tests needs to be more atomic i.e. one assert per test case but we can't do that in our framework for some reason.

You can do this using an ErrorCollector rule.

To use it, first add the rule as a field in your test class:

public class MyTest {
    @Rule
    public ErrorCollector collector = new ErrorCollector();

    //...tests...
}

Then replace your asserts with calls to collector.checkThat(...).

e.g.

@Test
public void myTest() {
    collector.checkThat("a", equalTo("b"));
    collector.checkThat(1, equalTo(2));
}

I use the ErrorCollector too but also use assertThat and place them in a try catch block.

import static org.junit.Assert.*;
import static org.hamcrest.Matchers.*;

@Rule
public ErrorCollector collector = new ErrorCollector();

@Test
public void calculatedValueShouldEqualExpected() {
	try {
		assertThat(calculatedValue(), is(expected));
	} catch (Throwable t) {
		collector.addError(t);
		// do something
	}
}

You can also use assertj - soft assertion

@Test
public void testCollectErrors(){
   SoftAssertions softly = new SoftAssertions();
   softly.assertThat(true).isFalse();
   softly.assertThat(false).isTrue();
   // Don't forget to call SoftAssertions global verification !
   softly.assertAll();
}

Also exist other way to use it without manually invoke softly.assertAll();

1. with rule
2. with autoclosable
3. Using the static assertSoftly method

Use try/finally blocks. This worked in my case:

...
try {
	assert(...)
} finally {
	// code to be executed after assert
}
...

Try - catch, in "try" use the assertion, in "catch" add the possible error to collection. Then throw the exception at the end of test, in tearDown(). So if there will be fail/error in assert, it will be catched and test will continue. (The collection in example is static, you can also make new instance in setUp() for each @Test)

    public static List<String> errors = new ArrayList<>();


    try {
        //some assert...
    }
    catch (AssertionError error) {
        errors.add(error.toString());
    }



    @After
    public void tearDown() {

        try {
            if (!errors.isEmpty()) {
                throw new AssertionError(errors);
            }
        }
        finally {
            //empty list because it's static, alternatively make instance for each test in setUp()
            errors.clear();
        }
    }

I created my own simple assertions class. Easy to extend with your use-cases:

public class MyEquals {

public static void checkTestSummary(MyTestSummary myTestSummary) {
    final List<MyTestResult> conditions = myTestSummary.getTestResults();
    final int total = conditions.size();

    final boolean isSuccessful = myTestSummary.isSuccessful();
    if (isSuccessful) {
        System.out.println(format("All [%s] conditions are successful!", total));
    } else {
        final List<MyTestResult> failedConditions = conditions.stream().filter(MyTestResult::isTestResult).collect(Collectors.toList());
        System.out.println(format("\nNot yet.. [%s out of %s] conditions are failed", failedConditions.size(), total));
    }

    if (!isSuccessful) {
        for (int i = 0; i < total; i++) {
            final MyTestResult myTestResult = conditions.get(i);
            if (myTestResult.isTestResult()) {
                System.out.println(format("  Success [%s of %s] => Expected %s Actual %s Good!", i + 1, total, myTestResult.getExpected(), myTestResult.getActual()));
            } else {
                System.out.println(format("!! Failed [%s of %s] => Expected %s Actual %s", i + 1, total, myTestResult.getExpected(), myTestResult.getActual()));
            }
        }
    }

    assertTrue(isSuccessful);
}

public static void myAssertEquals(MyTestSummary myTestSummary, Object expected, Object actual) {
    if (checkEquals(expected, actual)) {
        assertEquals(expected, actual);
        myTestSummary.addSuccessfulResult(expected, actual);
    } else {
        myTestSummary.addFailedResult(expected, actual);
        myTestSummary.setSuccessful(false);
    }
}

public static boolean checkEquals(Object value1, Object value2) {
    if (value1 == null && value2 == null) {
        return true;
    } else if (value1 != null && value2 == null) {
        return false;
    } else if (value1 == null && value2 != null) {
        return false;
    } else if (value1 != null && value2 != null) {
        return value1.equals(value2);
    }
    return false;
}

}

@Builder
@Value
public class MyTestResult {

    String expected;

    String actual;

    boolean testResult;

 }



@Data
public class MyTestSummary {

private boolean successful = true;

private List<MyTestResult> testResults = new ArrayList<>();

public MyTestSummary() {
}

public void addSuccessfulResult(Object expected, Object actual) {
    getTestResults().add(MyTestResult.builder()
        .expected(String.valueOf(expected))
        .actual(String.valueOf(actual))
        .testResult(true)
        .build()
    );
}

public void addFailedResult(Object expected, Object actual) {
    getTestResults().add(MyTestResult.builder()
        .expected(String.valueOf(expected))
        .actual(String.valueOf(actual))
        .testResult(false)
        .build()
    );
}
}

Usage in the junit test

@Test
public void testThat() {
    MyTestSummary myTestSummary = new MyTestSummary();
    myAssertEquals(myTestSummary, 10, 5 + 5);
    myAssertEquals(myTestSummary, "xxx", "x" + "x");
    checkTestSummary(myTestSummary);
}

Output:

Not yet.. [1 out of 2] conditions are failed
Success [1 of 2] => Expected 10 Actual 10 Good!
!! Failed [2 of 2] => Expected xxx Actual xx

 org.opentest4j.AssertionFailedError: expected: <true> but was: <false>
 Expected :true
 Actual   :false

Another option is the observable pattern in conjunction with lambda expressions. You can use something like the above.

public class MyTestClass {

    private final List<Consumer<MyTestClass>> AFTER_EVENT = new ArrayList<>();

    @After
    public void tearDown() {
        AFTER_EVENT.stream().forEach(c -> c.accept(this));
    }

    @Test
    public void testCase() {
        //=> Arrange
        AFTER_EVENT.add((o) -> {
            // do something after an assertion fail.
        }));

        //=> Act

        //=> Assert
        Assert.assertTrue(false);
    }
}