mockito-vs-easymock-vs-jmockit
Mockito vs EasyMock vs JMockit
1. Introduction
In this post, we’re going to talk about mocking: what it is, why to use it and several examples of how to mock the same test case using some of the most used mocking libraries for Java.
We’ll start with some formal/semi-formal definitions of mocking concepts; then we’ll present the case under test, follow up with examples for each library and end up with some conclusions. The chosen libraries are Mockito, EasyMock, and JMockit.
If you feel that you already know the basics of mocking, maybe you can skip to Point 2 without reading the next three points.
1.2. Reasons to Use Mocks
We’ll start assuming that you already code following some driven development methodology centered on tests (TDD, ATDD or BDD). Or simply that you want to create a test for an existing class that relies on dependencies to achieve its functionality.
In any case, when unit-testing a class, we want to test only its functionality and not that of its dependencies (either because we trust their implementation or because we’ll test it ourselves).
To achieve this, we need to provide to the object-under-test, a replacement that we can control for that dependency. This way we can force extreme return values, exception throwing or simply reduce time-consuming methods to a fixed return value.
This controlled replacement is the mock, and it will help you to simplify test coding and to reduce test execution time.
1.3. Mock Concepts and Definition
Let’s see four definitions from an article written by Martin Fowler that sums up the basics everyone should know about mocks:
-
Dummy objects are passed around but never actually used. Usually, they are just used to fill parameter lists.
-
Fake objects have working implementations, but usually, take some shortcut which makes them not suitable for production (an in memory database is a good example).
-
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what’s programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it ‘sent’, or maybe only how many messages it ‘sent’.
-
Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
1.4 To Mock or Not to Mock: That Is the Question
Not everything must be mocked. Sometimes it’s better to do an integration test as mocking that method/feature would be just working for little actual benefit. In our test case (that will be shown in the next point) that would be testing the LoginDao.
The LoginDao would use some third party library for DB access, and mocking it would only consist on assuring that parameters had been prepared for the call, but we still would need to test that the call returns the data we wanted.
For that reason, it won’t be included in this example (although we could write both the unit test with mock calls for the third party library calls AND an integration test with DBUnit for testing the actual performance of the third party library).
2. Test Case
With everything in the previous section in mind, let’s propose a quite typical test case and how we’ll test it using mocks (when it makes sense to use mocks). This will help us to have a common scenario for later on been able to compare the different mocking libraries.
2.1 Proposed Case
The login request will be handled by a controller, that uses a service, which uses a DAO (that looks for user credentials on a DB). We won’t deepen too much into each layer’s implementation and will focus more on the interactions between the components of each layer.
This way, we’ll have a LoginController, a LoginService and a LoginDAO. Let’s see a diagram for clarification:
2.2 Implementation
We’ll follow now with the implementation used for the test case, so we can understand what’s happening (or what should happen) on the tests.
We’ll start with the model used for all operations, UserForm, that will only hold the user’s name and password (we’re using public access modifiers to simplify) and a getter method for the username field to allow mocking for that property:
public class UserForm {
public String password;
public String username;
public String getUsername(){
return username;
}
}Let’s follow with LoginDAO, that will be void of functionality as we only want its methods to be there so we can mock them when needed:
public class LoginDao {
public int login(UserForm userForm){
return 0;
}
}LoginDao will be used by LoginService in its login method. LoginService will also have a setCurrentUser method that returns void to test that mocking.
public class LoginService {
private LoginDao loginDao;
private String currentUser;
public boolean login(UserForm userForm) {
assert null != userForm;
int loginResults = loginDao.login(userForm);
switch (loginResults){
case 1:
return true;
default:
return false;
}
}
public void setCurrentUser(String username) {
if(null != username){
this.currentUser = username;
}
}
}Finally, LoginController will use LoginService for its login method. This will include:
-
a case in which no calls to the mocked service will be done.
-
a case in which only one method will be called.
-
a case in which all methods will be called.
-
a case in which exception throwing will be tested.
public class LoginController {
public LoginService loginService;
public String login(UserForm userForm){
if(null == userForm){
return "ERROR";
}else{
boolean logged;
try {
logged = loginService.login(userForm);
} catch (Exception e) {
return "ERROR";
}
if(logged){
loginService.setCurrentUser(userForm.getUsername());
return "OK";
}else{
return "KO";
}
}
}
}Now that we’ve seen what is it that we’re trying to test let’s see how we’ll mock it with each library.
3. Test Setup
For Mockito we’ll be using version 2.8.9.
The easiest way of creating and using mocks is via the @Mock and @InjectMocks annotations. The first one will create a mock for the class used to define the field and the second one will try to inject said created mocks into the annotated mock.
There are more annotations such as @Spy that lets you create a partial mock (a mock that uses the normal implementation in non-mocked methods).
That being said, you need to call MockitoAnnotations.initMocks(this) before executing any tests that would use said mocks for all of this “magic” to work. This is usually done in a @Before annotated method. You can also use the MockitoJUnitRunner.
public class LoginControllerTest {
@Mock
private LoginDao loginDao;
@Spy
@InjectMocks
private LoginService spiedLoginService;
@Mock
private LoginService loginService;
@InjectMocks
private LoginController loginController;
@Before
public void setUp() {
loginController = new LoginController();
MockitoAnnotations.initMocks(this);
}
}3.2 EasyMock
For EasyMock, we’ll be using version 3.4 (Javadoc). Note that with EasyMock, for mocks to start “working”, you must call EasyMock.replay(mock) on every test method, or you will receive an exception.
Mocks and tested classes can also be defined via annotations, but in this case, instead of calling a static method for it to work, we’ll be using the EasyMockRunner for the test class.
Mocks are created with the @Mock annotation and the tested object with the @TestSubject one (which will get its dependencies injected from created mocks). The tested object must be created in-line.
@RunWith(EasyMockRunner.class)
public class LoginControllerTest {
@Mock
private LoginDao loginDao;
@Mock
private LoginService loginService;
@TestSubject
private LoginController loginController = new LoginController();
}3.3. JMockit
For JMockit we’ll be using version 1.24 (Javadoc) as version 1.25 hasn’t been released yet (at least while writing this).
Setup for JMockit is as easy as with Mockito, with the exception that there is no specific annotation for partial mocks (and really no need either) and that you must use JMockit as the test runner.
Mocks are defined using the @Injectable annotation (that will create only one mock instance) or with @Mocked annotation (that will create mocks for every instance of the class of the annotated field).
The tested instance gets created (and its mocked dependencies injected) using the @Tested annotation.
@RunWith(JMockit.class)
public class LoginControllerTest {
@Injectable
private LoginDao loginDao;
@Injectable
private LoginService loginService;
@Tested
private LoginController loginController;
}4. Verifying No Calls to Mock
For verifying that a mock received no calls in Mockito, you have the method verifyZeroInteractions() that accepts a mock.
@Test
public void assertThatNoMethodHasBeenCalled() {
loginController.login(null);
Mockito.verifyZeroInteractions(loginService);
}5. Defining Mocked Method Calls and Verifying Calls to Mocks
For mocking method calls, you can use Mockito.when(mock.method(args)).thenReturn(value). Here you can return different values for more than one call just adding them as more parameters: thenReturn(value1, value2, value-n, …).
Note that you can’t mock void returning methods with this syntax. In said cases, you’ll use a verification of said method (as shown on line 11).
For verifying calls to a mock you can use Mockito.verify(mock).method(args) and you can also verify that no more calls were done to a mock using verifyNoMoreInteractions(mock).
For verifying args, you can pass specific values or use predefined matchers like any(), anyString(), anyInt(). There are a lot more of that kind of matchers and even the possibility to define your matchers which we’ll see in following examples.
@Test
public void assertTwoMethodsHaveBeenCalled() {
UserForm userForm = new UserForm();
userForm.username = "foo";
Mockito.when(loginService.login(userForm)).thenReturn(true);
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
Mockito.verify(loginService).login(userForm);
Mockito.verify(loginService).setCurrentUser("foo");
}
@Test
public void assertOnlyOneMethodHasBeenCalled() {
UserForm userForm = new UserForm();
userForm.username = "foo";
Mockito.when(loginService.login(userForm)).thenReturn(false);
String login = loginController.login(userForm);
Assert.assertEquals("KO", login);
Mockito.verify(loginService).login(userForm);
Mockito.verifyNoMoreInteractions(loginService);
}5.2. EasyMock
For verifying calls to a mock, you can use EasyMock.verify(mock), but you must call it always after calling EasyMock.replay(mock).
For verifying args, you can pass specific values, or you have predefined matchers like isA(Class.class), anyString(), anyInt(), and a lot more of that kind of matchers and again the possibility to define your matchers.
@Test
public void assertTwoMethodsHaveBeenCalled() {
UserForm userForm = new UserForm();
userForm.username = "foo";
EasyMock.expect(loginService.login(userForm)).andReturn(true);
loginService.setCurrentUser("foo");
EasyMock.replay(loginService);
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
EasyMock.verify(loginService);
}
@Test
public void assertOnlyOneMethodHasBeenCalled() {
UserForm userForm = new UserForm();
userForm.username = "foo";
EasyMock.expect(loginService.login(userForm)).andReturn(false);
EasyMock.replay(loginService);
String login = loginController.login(userForm);
Assert.assertEquals("KO", login);
EasyMock.verify(loginService);
}5.3. JMockit
With JMockit, you have defined steps for testing: record, replay and verify.
Record is done in a new Expectations()\{\{}} block (into which you can define actions for several mocks), replay is done simply by invoking a method of the tested class (that should call some mocked object), and verification is done inside a new Verifications()\{\{}} block (into which you can define verifications for several mocks).
For mocking method calls, you can use mock.method(args); result = value; inside any Expectations block. Here you can return different values for more than one call just using returns(value1, value2, …, valuen); instead of result = value;.
For verifying calls to a mock you can use new Verifications()\{\{mock.call(value)}} or new Verifications(mock)\{\{}} to verify every expected call previously defined.
For verifying args, you can pass specific values, or you have predefined values like any, anyString, anyLong, and a lot more of that kind of special values and again the possibility to define your matchers (that must be Hamcrest matchers).
@Test
public void assertTwoMethodsHaveBeenCalled() {
UserForm userForm = new UserForm();
userForm.username = "foo";
new Expectations() {{
loginService.login(userForm); result = true;
loginService.setCurrentUser("foo");
}};
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
new FullVerifications(loginService) {};
}
@Test
public void assertOnlyOneMethodHasBeenCalled() {
UserForm userForm = new UserForm();
userForm.username = "foo";
new Expectations() {{
loginService.login(userForm); result = false;
// no expectation for setCurrentUser
}};
String login = loginController.login(userForm);
Assert.assertEquals("KO", login);
new FullVerifications(loginService) {};
}6. Mocking Exception Throwing
Exception throwing can be mocked using .thenThrow(ExceptionClass.class) after a Mockito.when(mock.method(args)).
@Test
public void mockExceptionThrowin() {
UserForm userForm = new UserForm();
Mockito.when(loginService.login(userForm)).thenThrow(IllegalArgumentException.class);
String login = loginController.login(userForm);
Assert.assertEquals("ERROR", login);
Mockito.verify(loginService).login(userForm);
Mockito.verifyZeroInteractions(loginService);
}6.2. EasyMock
Exception throwing can be mocked using .andThrow(new ExceptionClass()) after an EasyMock.expect(…) call.
@Test
public void mockExceptionThrowing() {
UserForm userForm = new UserForm();
EasyMock.expect(loginService.login(userForm)).andThrow(new IllegalArgumentException());
EasyMock.replay(loginService);
String login = loginController.login(userForm);
Assert.assertEquals("ERROR", login);
EasyMock.verify(loginService);
}6.3. JMockit
Mocking exception throwing with JMockito is especially easy. Just return an Exception as the result of a mocked method call instead of the “normal” return.
@Test
public void mockExceptionThrowing() {
UserForm userForm = new UserForm();
new Expectations() {{
loginService.login(userForm); result = new IllegalArgumentException();
// no expectation for setCurrentUser
}};
String login = loginController.login(userForm);
Assert.assertEquals("ERROR", login);
new FullVerifications(loginService) {};
}7. Mocking an Object to Pass Around
You can create a mock also to pass as an argument for a method call. With Mockito, you can do that with a one-liner.
@Test
public void mockAnObjectToPassAround() {
UserForm userForm = Mockito.when(Mockito.mock(UserForm.class).getUsername())
.thenReturn("foo").getMock();
Mockito.when(loginService.login(userForm)).thenReturn(true);
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
Mockito.verify(loginService).login(userForm);
Mockito.verify(loginService).setCurrentUser("foo");
}7.2. EasyMock
Mocks can be created in-line with EasyMock.mock(Class.class). Afterward, you can use EasyMock.expect(mock.method()) to prepare it for execution, always remembering to call EasyMock.replay(mock) before using it.
@Test
public void mockAnObjectToPassAround() {
UserForm userForm = EasyMock.mock(UserForm.class);
EasyMock.expect(userForm.getUsername()).andReturn("foo");
EasyMock.expect(loginService.login(userForm)).andReturn(true);
loginService.setCurrentUser("foo");
EasyMock.replay(userForm);
EasyMock.replay(loginService);
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
EasyMock.verify(userForm);
EasyMock.verify(loginService);
}7.3. JMockit
To mock an object for just one method, you can simply pass it mocked as a parameter to the test method. Then you can create expectations as with any other mock.
@Test
public void mockAnObjectToPassAround(@Mocked UserForm userForm) {
new Expectations() {{
userForm.getUsername(); result = "foo";
loginService.login(userForm); result = true;
loginService.setCurrentUser("foo");
}};
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
new FullVerifications(loginService) {};
new FullVerifications(userForm) {};
}8. Custom Argument Matching
Sometimes argument matching for mocked calls needs to be a little more complex than just a fixed value or anyString(). For that cases with Mockito has its matcher class that is used with argThat(ArgumentMatcher<>).
@Test
public void argumentMatching() {
UserForm userForm = new UserForm();
userForm.username = "foo";
// default matcher
Mockito.when(loginService.login(Mockito.any(UserForm.class))).thenReturn(true);
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
Mockito.verify(loginService).login(userForm);
// complex matcher
Mockito.verify(loginService).setCurrentUser(ArgumentMatchers.argThat(
new ArgumentMatcher<String>() {
@Override
public boolean matches(String argument) {
return argument.startsWith("foo");
}
}
));
}8.2. EasyMock
Custom argument matching is a little bit more complicated with EasyMock as you need to create a static method in which you create the actual matcher and then report it with EasyMock.reportMatcher(IArgumentMatcher).
Once this method is created, you use it on your mock expectation with a call to the method (like seen in the example in line ).
@Test
public void argumentMatching() {
UserForm userForm = new UserForm();
userForm.username = "foo";
// default matcher
EasyMock.expect(loginService.login(EasyMock.isA(UserForm.class))).andReturn(true);
// complex matcher
loginService.setCurrentUser(specificArgumentMatching("foo"));
EasyMock.replay(loginService);
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
EasyMock.verify(loginService);
}
private static String specificArgumentMatching(String expected) {
EasyMock.reportMatcher(new IArgumentMatcher() {
@Override
public boolean matches(Object argument) {
return argument instanceof String
&& ((String) argument).startsWith(expected);
}
@Override
public void appendTo(StringBuffer buffer) {
//NOOP
}
});
return null;
}8.3. JMockit
Custom argument matching with JMockit is done with the special withArgThat(Matcher) method (that receives Hamcrest‘s Matcher objects).
@Test
public void argumentMatching() {
UserForm userForm = new UserForm();
userForm.username = "foo";
// default matcher
new Expectations() {{
loginService.login((UserForm) any);
result = true;
// complex matcher
loginService.setCurrentUser(withArgThat(new BaseMatcher<String>() {
@Override
public boolean matches(Object item) {
return item instanceof String && ((String) item).startsWith("foo");
}
@Override
public void describeTo(Description description) {
//NOOP
}
}));
}};
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
new FullVerifications(loginService) {};
}9. Partial Mocking
Mockito allows partial mocking (a mock that uses the real implementation instead of mocked method calls in some of its methods) in two ways.
You can either use .thenCallRealMethod() in a normal mock method call definition, or you can create a spy instead of a mock in which case the default behavior for that will be to call the real implementation in all non-mocked methods.
@Test
public void partialMocking() {
// use partial mock
loginController.loginService = spiedLoginService;
UserForm userForm = new UserForm();
userForm.username = "foo";
// let service's login use implementation so let's mock DAO call
Mockito.when(loginDao.login(userForm)).thenReturn(1);
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
// verify mocked call
Mockito.verify(spiedLoginService).setCurrentUser("foo");
}9.2. EasyMock
Partial mocking also gets a little more complicated with EasyMock, as you need to define which methods will be mocked when creating the mock.
This is done with EasyMock.partialMockBuilder(Class.class).addMockedMethod(“methodName”).createMock(). Once this is done, you can use the mock as any other non-partial mock.
@Test
public void partialMocking() {
UserForm userForm = new UserForm();
userForm.username = "foo";
// use partial mock
LoginService loginServicePartial = EasyMock.partialMockBuilder(LoginService.class)
.addMockedMethod("setCurrentUser").createMock();
loginServicePartial.setCurrentUser("foo");
// let service's login use implementation so let's mock DAO call
EasyMock.expect(loginDao.login(userForm)).andReturn(1);
loginServicePartial.setLoginDao(loginDao);
loginController.loginService = loginServicePartial;
EasyMock.replay(loginDao);
EasyMock.replay(loginServicePartial);
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
// verify mocked call
EasyMock.verify(loginServicePartial);
EasyMock.verify(loginDao);
}9.3. JMockit
Partial mocking with JMockit is especially easy. Every method call for which no mocked behavior has been defined in an Expectations()\{\{}} uses the “real” implementation.
In this case as no expectation is given for LoginService.login(UserForm) the actual implementation (and the call to LoginDAO.login(UserForm)) is performed.
@Test
public void partialMocking() {
// use partial mock
LoginService partialLoginService = new LoginService();
partialLoginService.setLoginDao(loginDao);
loginController.loginService = partialLoginService;
UserForm userForm = new UserForm();
userForm.username = "foo";
// let service's login use implementation so let's mock DAO call
new Expectations() {{
loginDao.login(userForm); result = 1;
// no expectation for loginService.login
partialLoginService.setCurrentUser("foo");
}};
String login = loginController.login(userForm);
Assert.assertEquals("OK", login);
// verify mocked call
new FullVerifications(partialLoginService) {};
new FullVerifications(loginDao) {};
}10. Conclusion
In this post, we’ve been comparing three Java mock libraries, each one with its strong points and downsides.
-
All three of them are easily configured with annotations to help you define mocks and the object-under-test, with runners to make mock injection as painless as possible.
-
We’d say Mockito would win here as it has a special annotation for partial mocks, but JMockit doesn’t even need it, so let’s say that it’s a tie between those two.
-
-
All three of them follow more or less the record-replay-verify pattern, but in our opinion, the best one to do so is JMockit as it forces you to use those in blocks, so tests get more structured.
-
Easiness of use is important so you can work as less as possible to define your tests. JMockit will be the chosen option for its fixed-always-the-same structure.
-
Mockito is more or less THE most known so that the community will be bigger.
-
Having to call replay every time you want to use a mock is a clear no-go, so we’ll put a minus one for EasyMock.
-
Consistency/simplicity is also important for me. We loved the way of returning results of JMockit that is the same for “normal” results as for exceptions.
Will all this being said, we’re going to choose JMockit as a kind of a winner even though up till now we’ve been using Mockito as we’ve been captivated by its simplicity and fixed structure and will try and use it from now on.
The full implementation of this tutorial can be found on the GitHub project so feel free to download it and play with it.
