junit-failure-vs-error

Posted on: 2019-10-08 2019-11-19
Tags: JUnit

The Difference Between Failure and Error in JUnit

1. Introduction

In this tutorial, we’ll explore the differences between a failure and an error in JUnit tests.

In short, failures are unfulfilled assertions while errors are due to abnormal test execution.

2. Sample Code

Let’s consider a very simplistic example, namely a calculator class that has one method to divide two double values:

public static double divideNumbers(double dividend, double divisor) {
    if (divisor == 0) {
        throw new ArithmeticException("Division by zero!");
    }
    return dividend / divisor;
}

Note that Java doesn’t actually throw an ArithmeticException on its own for double division – it returns Infinity or NaN.

3. Example Failure

When writing unit tests with JUnit, there will likely be situations when tests fail. One possibility is that our code does not meet its test criteria. That means one or more test cases fail due to assertions not being fulfilled.

In the following example, the assertion will fail, because the result of the division is 2 and not 15. Our assertion and the actual result simply don’t match:

@Test
void whenDivideNumbers_thenExpectWrongResult() {
    double result = SimpleCalculator.divideNumbers(6, 3);
    assertEquals(15, result);
}

4. Example Error

Another possibility is that we have an unexpected situation during test execution, most likely due to an exception; for example, accessing a null reference will raise a RuntimeException.

Let’s see an example, where the test will abort with an error because we’re attempting to divide by zero which we explicitly guard against by throwing an exception in our calculator code:

@Test
void whenDivideByZero_thenThrowsException(){
    SimpleCalculator.divideNumbers(10, 0);
}

Now, we could fix this test by simply including the exception as one of our assertions.

@Test
void whenDivideByZero_thenAssertException(){
    assertThrows(ArithmeticException.class, () -> SimpleCalculator.divideNumbers(10, 0));
}

Then, if the exception is thrown, the test passes, but if not then that would be another failure.

5. Conclusion

Both failure and error in JUnit tests indicate an undesired situation, but their semantics are different. Failures notify of an invalid test result, errors indicate an unexpected test execution.

Also, please check out the example code at GitHub.

getdocs

3043