Published on

Using JUnit 5's Parameterized Tests to Easily Test All Permutations of a Branch

Authors

Today I had to add more tests to a piece of code I was working on. The function I was testing took in 2 parameters. These parameters were both of the same enum type. They were plugged in to formulas in different parts of the if statement. The else branch of this section used these 2 parameters together to get a result but excluded one of the enums defined in the class.

I wanted a way to put tests in place to ensure this last branch was properly covered. An obvious yet naive approach would be to write a test for each permutation. But this will quickly become unfeasible due to the number of possible combinations as you add more values to the enum.

Another approach is to write code that gets all permutations, iterates through them and runs tests on each iteration. This would definitely solve the issue of an explosion of tests but if one of those permutations breaks it is difficult to debug. Luckily in JUnit 5 there is a feature called @ParameterizedTests and specifically for my case @MethodSource.

So first what I needed to do was write a data provider to give me each of the permutations I needed, in Kotlin I wrote something like the below:

companion object {
    @JvmStatic
    fun allCombindationsOfCompassPointsExcludingNorth(): Stream<Pair<CompassPoint, CompassPoint>> {
        val compassPointsWithoutNorth = CompassPoint.values().filterNot { it == CompassPoint.NORTH }
        val permutations: MutableSet<Pair<CompassPoint, CompassPoint>> = mutableSetOf()
        compassPointsWithoutNorth.forEach { outer ->
            compassPointsWithoutNorth.forEach { inner ->
                permutations.add(Pair(outer, inner))
            }
        }
        return permutations.stream()
    }
}

In the above I had to put this in a companion object as I needed this method to be static as per the JUnit documentation and Kotlin only allows you to make something @JvmStatic if you put it in a companion object. What this code does is:

  • Exclude a point from my enum ,in this case NORTH, as it does not form part of the else code branch I am testing.
  • It iterates through each of the remaining enum values and again internally to build up a set of Pairs of possible permutations.
  • I used a set to not allow duplicate pairs
  • This set is then returned as a Stream so that JUnit can inject each element in this collection as the method parameter to the @Parameterized test method we will use this for.

I then use this in my test as follows:

@ParameterizedTest
@MethodSource("allCombindationsOfCompassPointsExcludingNorth")
fun `navigate for two points that are not NORTH goes in the correct direction`(inputAndOutputCompassPoints: Pair<CompassPoint, CompassPoint>) {
    val (inputCompassPoint, outputCompassPoint) = inputAndOutputCompassPoints
    //...
    // your test code here
}

JUnit 5 will now run this test for each element in the set I build up in my allCombindationsOfCompassPointsExcludingNorth passing each element as a method parameter. This differs from the approach of iterating through the set yourself since if one or more method permutations fail IDEs ,with support for JUnit 5, will give you a much better error message as if each permutation were its own test.

TestResults
        CompassTest
            navigate for two points that are not NORTH goes in the correct direction NORTH(Pair)
                [1] (NORTH, NORTH)
                [2] (NORTH, SOUTH)
                ...

In IntelliJ for example I can click each error to see exactly where it broke for that permutations. All other permutations that pass will not show up.

I took this even further by modifying the tests I wrote in a previous post that use @EnumSource but specify only some of the values in the enum. This @EnumSource solution did not sit well with me as if the enum needs to be changed I need to go and update the tests. I used what I learnt today to refactor these tests to use a method provider similar to the below:

companion object {
    @SuppressWarnings("unused")
    @JvmStatic
    fun allCompassPointsThatAreNotNorth(): Stream<CompassPoint> {
        return CompassPoint.values().filterNot { it == CompassPoint.NORTH }.stream()
    }
}

This then results in my test method that used to have to explicitly specify what points it needed as below:

@ParameterizedTest
@EnumSource(value = CompassPoint::class, names = arrayOf("SOUTH", "EAST", "WEST"))

Becoming:

@ParameterizedTest
@MethodSource("allCompassPointsThatAreNotNorth")

Which is much more maintainable.

In case you were wondering if you do not make this method provider static or in Kotlin's case @JvmStatic running the test will give you an error similar to the following:

org.junit.platform.commons.JUnitException: Could not find factory method [allCombindationsOfCompassPointsExcludingNorth] in class [my.company.CompassTest]