Comparison between Mockito vs JMockit - why is Mockito voted better than JMockit?

I'm investigating which mocking framework to use for my project and have narrowed it down to JMockit and Mockito.

I notice that Mockito was voted "the best mock framework for Java" on Stackoverflow.
In comparing features on JMockit's "Mocking Tool Comparision Matrix" it appears that JMockit has multiple different features.

Does anyone have any specific information (not opinions) on what Mockito can do which can't be achieved with JMockit and vice versa?

Update Sep 2019: The only mocking framework supported (by default) by Spring Boot is Mockito. If you use Spring, the answer is quite obvious.

---

I'd say the competition is between JMockit and PowerMock, then Mockito.

I'd leave "plain" jMock and EasyMock because they use only proxy & CGLIB and do not use Java 5 instrumentation like the newer frameworks.

jMock also didn't have a stable release for over 4 years. jMock 2.6.0 required 2 years to go from RC1 to RC2, and then another 2 years before it actually got released.

Regarding Proxy & CGLIB vs instrumentation:

> (EasyMock and jMock) are based on java.lang.reflect.Proxy, > which requires an interface to be > implemented. Additionally, they > support the creation of mock objects > for classes through CGLIB subclass > generation. Because of that, said > classes cannot be final and only > overridable instance methods can be > mocked. Most importantly, however, > when using these tools the > dependencies of code under test (that > is, the objects of other classes on > which a given class under test > depends) must be controlled by the > tests, so that mock instances can be > passed to the clients of those > dependencies. Therefore, dependencies > can't simply be instantiated with the > new operator in a client class for > which we want to write unit tests. > > Ultimately, the technical limitations > of conventional mocking tools impose > the following design restrictions on > production code: > > 1. Each class which may need to be mocked in a test must either implement > a separate interface or not be final. > 2. The dependencies of each class to be tested must either be obtained > through configurable instance creation > methods (factories or a Service > Locator), or be exposed for dependency > injection. Otherwise, unit tests won't > be able to pass mock implementations > of dependencies to the unit under > test. > 3. Since only instance methods can be mocked, classes to be unit tested > cannot call any static methods on > their dependencies, nor instantiate > them using any of the constructors.

The above is copied from http://jmockit.org/about.html . Further, it compares between itself (JMockit), PowerMock, and Mockito in several ways:

> There are now other mocking tools for > Java which also overcome the > limitations of the conventional ones, > between them PowerMock, jEasyTest, and > MockInject. The one that comes closest > to the feature set of JMockit is > PowerMock, so I will briefly evaluate > it here (besides, the other two are > more limited and don't seem to be > actively developed anymore). > > ### JMockit vs PowerMock > > * First of all, PowerMock does not provide a complete API for mocking, > but instead works as an extension to > another tool, which currently can be > EasyMock or Mockito. This is obviously > an advantage for existing users of > those tools. > * JMockit, on the other hand, provides entirely new APIs, although > its main API (Expectations) is similar > to both EasyMock and jMock. While this > creates a longer learning curve, it > also allows JMockit to provide a > simpler, more consistent, and easier > to use API. > * Compared to the JMockit Expectations API, the PowerMock API is > more "low-level", forcing users to > figure out and specify which classes > need to be prepared for testing (with > the @PrepareForTest({ClassA.class, > ...}) annotation) and requiring > specific API calls to deal with > various kinds of language constructs > that may be present in the production > code: static methods > (mockStatic(ClassA.class)), > constructors > (suppress(constructor(ClassXyz.class))), > constructor invocations > (expectNew(AClass.class)), partial > mocks (createPartialMock(ClassX.class, > "methodToMock")), etc. > * With JMockit Expectations, all kinds of methods and constructors are > mocked in a purely declarative way, > with partial mocking specified through > regular expressions in the @Mocked > annotation or by simply "un-mocking" > the members with no recorded > expectations; that is, the developer > simply declares some shared "mock > fields" for the test class, or some > "local mock fields" and/or "mock > parameters" for individual test > methods (and in this last case the > @Mocked annotation often won't be > needed). > * Some capabilities available in JMockit, such as support for mocking > equals and hashCode, overridden > methods, and others, are currently not > supported in PowerMock. Also, there is > no equivalent to JMockit's ability to > capture instances and mock > implementations of specified base > types as the test executes, without > the test code itself having any > knowledge of the actual implementation > classes. > * PowerMock uses custom class loaders (usually one per test class) > in order to generate modified versions > of the mocked classes. Such heavy use > of custom class loaders can lead to > conflicts with third-party libraries, > hence the need to sometimes use the > @PowerMockIgnore("package.to.be.ignored") > annotation on test classes. > * The mechanism used by JMockit (runtime instrumentation through a > "Java agent") is simpler and safer, > although it does require passing a > "-javaagent" parameter to the JVM when > developing on JDK 1.5; on JDK 1.6+ > (which can always be used for > development, even if deploying on an > older version) there is no such > requirement, since JMockit can > transparently load the Java agent on > demand by using the Attach API. > > Another recent mocking tool is > Mockito. Although it does not attempt > to overcome the limitations of older > tools (jMock, EasyMock), it does > introduce a new style of behavior > testing with mocks. JMockit also > supports this alternative style, > through the Verifications API. > > ### JMockit vs Mockito > > * Mockito relies on explicit calls to its API in order to separate code > between the record (when(...)) and > verify (verify(...)) phases. This > means that any invocation to a mock > object in test code will also require > a call to the mocking API. > Additionally, this will often lead to > repetitive when(...) and > verify(mock)... calls. > * With JMockit, no similar calls exist. Sure, we have the new > NonStrictExpectations() and new > Verifications() constructor calls, but > they occur only once per test > (typically), and are completely > separate from the invocations to > mocked methods and constructors. > * The Mockito API contains several inconsistencies in the syntax used for > invocations to mocked methods. In the > record phase, we have calls like > when(mock.mockedMethod(args))... while > in the verify phase this same call > will be written as > verify(mock).mockedMethod(args). > Notice that in the first case the > invocation to mockedMethod is made > directly on the mock object, while in > the second case it is made on the > object returned by verify(mock). > * JMockit has no such inconsistencies because invocations to > mocked methods are always made > directly on the mocked instances > themselves. (With one exception only: > to match invocations on the same > mocked instance, an onInstance(mock) > call is used, resulting in code like > onInstance(mock).mockedMethod(args); > most tests won't need to use this, > though.) > * Just like other mocking tools which rely on method > chaining/wrapping, Mockito also runs > into inconsistent syntax when stubbing > void methods. For example, you write > when(mockedList.get(1)).thenThrow(new > RuntimeException()); for a non-void > method, and doThrow(new > RuntimeException()).when(mockedList).clear(); > for a void one. With JMockit, it's > always the same syntax: > mockedList.clear(); result = new > RuntimeException();. > * Yet another inconsistency occurs in the use of Mockito spies: "mocks" > that allow the real methods to be > executed on the spied instance. For > example, if spy refers to an empty > List, then instead of writing > when(spy.get(0)).thenReturn("foo") you > will need to write > doReturn("foo").when(spy).get(0). With > JMockit, the dynamic mocking feature > provides similar functionality to > spies, but without this issue since > real methods only get executed during > the replay phase. > * In EasyMock and jMock, the first mocking APIs for Java, the focus was > entirely on the recording of expected > invocations of mocked methods, for > mock objects that (by default) do not > allow unexpected invocations. Those > APIs also provide the recording of > allowed invocations for mock objects > that do allow unexpected invocations, > but this was treated as a second-class > feature. Additionally, with these > tools there is no way to explicitly > verify invocations to mocks after the > code under test is exercised. All such > verifications are performed implicitly > and automatically. > * In Mockito (and also in Unitils Mock), the opposite viewpoint is > taken. All invocations to mock objects > that may happen during the test, > whether recorded or not, are allowed, > never expected. Verification is > performed explicitly after the code > under test is exercised, never > automatically. > * Both approaches are too extreme, and consequently less than optimal. > JMockit Expectations & Verifications > is the only API that allows the > developer to seamlessly choose the > best combination of strict (expected > by default) and non-strict (allowed by > default) mock invocations for each > test. > * To be more clear, the Mockito API has the following shortcoming. If you > need to verify that an invocation to a > non-void mocked method happened during > the test, but the test requires a > return value from that method that is > different from the default for the > return type, then the Mockito test > will have duplicate code: a > when(mock.someMethod()).thenReturn(xyz) > call in the record phase, and a > verify(mock).someMethod() in the > verify phase. With JMockit, a strict > expectation can always be recorded, > which won't have to be explicitly > verified. Alternatively, an invocation > count constraint (times = 1) can be > specified for any recorded non-strict > expectation (with Mockito such > constraints can only be specified in a > verify(mock, constraint) call). > * Mockito has poor syntax for verifications in order, and for full > verifications (that is, checking that > all invocations to mock objects are > explicitly verified). In the first > case, an extra object needs to be > created, and calls to verify made on > it: InOrder inOrder = inOrder(mock1, > mock2, ...). In the second case, calls > like verifyNoMoreInteractions(mock) or > verifyZeroInteractions(mock1, mock2) > need to be made. > * With JMockit, you simply write new VerificationsInOrder() or new > FullVerifications() instead of new > Verifications() (or new > FullVerificationsInOrder() to combine > both requirements). No need to specify > which mock objects are involved. No > extra mocking API calls. And as a > bonus, by calling > unverifiedInvocations() inside an > ordered verification block, you can > perform order-related verifications > that are simply impossible in Mockito. > > Finally, the JMockit Testing Toolkit > has a wider scope and more ambitious > goals than other mocking toolkits, in > order to provide a complete and > sophisticated developer testing > solution. A good API for mocking, even > without artificial limitations, is not > enough for productive creation of > tests. An IDE-agnostic, easy to use, > and well integrated Code Coverage tool > is also essential, and that's what > JMockit Coverage aims to provide. > Another piece of the developer testing > toolset which will become more useful > as the test suite grows in size is the > ability to incrementally rerun tests > after a localized change to production > code; this is also included in the > Coverage tool.

(granted, the source may be biased, but well...)

I'd say go with JMockit. It's the easiest to use, flexible, and works for pretty much all cases even difficult ones and scenarios when you can't control the class to be tested (or you can't break it due to compatibility reasons etc.).

My experiences with JMockit have been very positive.

I worked with both Mockito and JMockit, and my experience with them is:

• Mockito:

  • implicit mocking (-> better usability, but has the danger of failing to detect not-allowed method calls on mocks)
  • explicit verification

• EasyMock:

  • explict mocking
  • implicit verification

• JMockit:

  • supports both

• Besides, other benefits of JMockit:

  • if you're mocking static methods/constructors etc (such as extending a very old legacy code base without UT), you'll have two choices: 1) Mockito/EasyMock with Powermock extension or 2) Jmockit
  • built-in coverage report

I personally prefer JMockit, which I think is more feature rich and flexible, but requires a little bit steeper learning curve. There're usually multiple ways to achieve the same mocking effect, and requires more care when designing the mocks.

I use jMockit only because of it's reflection libraries in Deencapsultation.class. I actually love Mockito's style, but I refuse to change my code and muddy up my API just so a limited testing framework can get at it. And I'm a fan of testing all my code, so a framework that can't easily test private methods is not what I want to be using.

I was swayed by this article

After a (admittedly large) learning curve, jMockit is now my main unit testing framework for mocks.

For easy testing of our legacy codebase (with lots of static method calls, etc.), JMockit has been invaluable. [Shameless plug for an article on my blog]

I personally prefer EasyMock.
The ability to divert between nice, normal and strict mocking controls is one on my favorite feature.