Should the JUnit message state the condition of success or failure?

I can write an assertion message one of two ways. Stating success:

assertEquals( "objects should be identical", expected, actual );

Or stating the condition of being broken:

assertEquals( "objects aren't identical", expected, actual );

Is there a standard for this in JUnit specifically? If not, what are the arguments for each side?

P.S. I've seen articles on the web demonstrating both of these without explanation, so just saying "search Google" is not an answer!

[UPDATE]

Everyone is getting hung up on the fact that I used assertEquals and therefore the message is probably useless. But of course that's just because I wanted to illustrate the question simply.

So imagine instead it's:

assertTrue( ... big long multi-line expression ... );

Where a message is useful.

I rarely even bother with a message, at least for assertEquals. Any sensible test runner will explain that you were using assertEquals and the two things which were meant to be equal. Neither of your messages give more information than that.

I usually find that unit test failures are transient things - I'll rapidly find out what's wrong and fix it. The "finding out what's wrong" usually involves enough detail that a single message isn't going to make much difference. Consider "time saved by having a message" vs "time spent thinking of messages" :)

EDIT: Okay, one case where I might use a message: when there's a compact description in text which isn't obvious from the string representation of the object.

For example: "Expected date to be December 1st" when comparing dates stored as milliseconds.

I wouldn't worry about how you express it exactly though: just make sure it's obvious from the message which way you mean. Either "should be" or "wasn't" is fine - just "December 1st" wouldn't be obvious.

According to the junit API the message is the "the identifying message for the AssertionError" so its not a message describing the condition that should be met but a message describing what's wrong if the condition isn't met. So in your example "objects aren't identical" seems to be more conformant.

Unlike many others I feel that using a message is extremely helpful for many reasons:

1. The person looking at the logs of a failed test may not be the person who wrote the test. It can take time to read through the code and understand what case the assertion is meant to address. A helpful message will save time.

2. Even in the event it is the developer of the test who is looking at the logs it may have been days or months since the test was written and, again, a message can save time.

My advice would be to write the message with a statement of the expected behavior. For example:

assertEquals("The method should be invoked 3 times", 3, invocationCount);

I don't think it matters at all - You already know that a failure happened, and therefore it doesn't matter if the message states what should have happened, or what shouldn't happen.

The goal of the message is to help you when it can, not to obtain some completeness.

Obviously, in the case of assertEquals this is less important, but the message is important in the case of general asserts. The message should help you obtain enough context to understand right away what exactly failed.

However, the amount of needed context (and thus the details in the message) should depend on how you get the report. For example, if you get it in Eclipse, you can easily go and interact and see what happened, so the message is less imporrtant. However, if you get your reports emailed to you (e.g., from a continuous build server) then you want the message to provide enough information so that you will have an idea of what is going on before you even go to the corresponding source code.

I would like to answer the question without considering, if a message in generel is useful.

If a test fails, something is wrong. I know this. I want to know why it is broken. That's very easy to find out because I just have to open the test case and the SUT. Like Jon said, it's very easy to fix it (hopefully ;-) ).

But what about the message? The message is for me an advice, what could be done to turn it into a green test case. So I would appreciate if there's an advice given in the message text, how to fix this problem or where to search for the problem.

Another interesting aspect would be the usage of positive expressions. It's worth a consideration to use positive text messages. In your example, I would use Objects should be identical. But that's a small reason.

I see this question from two perspectives,

First and the most common perspective, which is already being discussed by most of us here: From the perspective of someone who is seeing the logs and trying to fix the error: I believe that both the messages provides equal information.

Second perspective is that of someone who is reading/ maintaining/ reviewing the code: As we have been talking since ages about the readability and simplicity of code. So, it is also equally important as well.

We have been made to believe that my code should be simple and self explanatory so that no explicit comments are needed and I strongly agree with it.

From this perspective:

These messages make it a lot easier to read and go through the code as they serve the dual purpose of documentation as well as error reporting:

assertEquals( "objects should be identical", expected, actual );
assertTrue( "flag should have been set", flag );
assertNotNull( "object must not be null", object );

These messages are not so reader friendly as they talk about the unexpected condition:

assertEquals( "objects aren't identical", expected, actual );
assertTrue( "flag is not set", flag );
assertNotNull( "object is null", object );

According to specs, the message is to describe the error when it occurs. And is useful when you build your application in a CI environment, like Jenkins, and using plugins to analyze error results.

http://junit.sourceforge.net/javadoc/org/junit/Assert.html#assertTrue(java.lang.String,%20boolean)

> message - the identifying message for the AssertionError (null okay)

Vote me down too (like Jon), but the only time I've ever use a message like this (on assert equals) is when building a single test with a matrix of values and one of the test elements fails: I use the message to indicate which test case failed. Otherwise, the text is totally redundant.

From the javadocs of JUnit:

> Asserts that two objects are equal. If > they are not an AssertionFailedError > is thrown with the given message.

According to the API, the message can be whatever you want. I would argue that the two options you have are both the same and both superfluous. The success or failure of the assert already provides all the information you are providing in the message.

It follows for me that you should have either nothing (there is an assert that doesn't take a string on purpose) OR include a message with meaning beyond what is already there.

So I guess this is a reiteration of Jon's answer, but too verbose to be a comment.

I don't put a message for the case you cite, unless I'm running a test where I have an array of similar test values that I'm running in a loop and I want to pinpoint exactly which one failed. Then I add a message to tell me which one.

I agree that providing a message is helpful and I always provide one.

To me, the useful thing to include is a clear statement of what went wrong - usually involving the words 'should' or 'should not'.

E.g., "objects are equal" is ambiguous - does it mean the objects are equal and that's why the test failed? Or that objects should be equal but they aren't? But if you say "Objects should be equal" or "Objects should not be equal" it's obvious why the assertion failed.

I particularly like how the Spock test framework encourages tests that read like a story and have come to structure tests under different frameworks similarly. I'm not particularly concerned about the individual error message making a lot of sense, I aim for quickly wrapping my head around the entire test once I open it:

assertEquals("Cloned and persisted items", mockedTiCount, clonedTis.size());
assertTrue("Belong to new transaction", clonedTis.stream().allMatch(ti -> ti.getTransaction().getId().equals(cloned.getId())));
assertNotEquals("Which has a different ID", t.getId(), cloned.getId());
assertEquals("While the originals are left intact", mockedTiCount, transactionItemRepository.findByTransactionId(t.getId()).size());

Opting for many small tests instead of few large ones helps here as well, as does a neatly structured, hopefully reusable, test setup code.

Those messages are particularly important when a test fails. And when it fails, you don't only want to know that the objects should be equal, you want to know why they should be equal.

So instead of

assertEquals( "objects should be identical", expected, actual );

use

assertEquals( "Status code should be 403 (forbidden), because a user "     
+ "with role customer must not be allowed to access the admin console.", 
403, statusCode );

And yes, messages are allowed to be very long, as long as they help the developer to identify the issue fast and fix it the right way.