Selective Runner

Last semester, I was trying to replicate the results of “Empirical studies of test case prioritization in a JUnit testing environment ¹“ as a part of curricular project. On this paper, they have studied about various techniques of test case prioritization all based around code coverage in a JUnit testing environment.

For that project, I needed something that would allow me to run the test methods in any arbitrary order. The test methods can come from any test classes. For example: a test class A has test1A, test2A, and test3A as test methods, and test class B has test1B, test2B as test methods. In this project, the prioritization strategy would give the order of test execution as say, “test1B, test2A, test1A, test2B, test3A” and we need to execute the test in the given order. But JUnit allows test execution only at test class level. For this, I implemented a small wrapper class called Selective Runner on top of exisiting JUnit4’s runner, available in Github. With this custom runner, we can write our test case as:

@RunWith(SelectiveRunner.class)
@SelectiveRunner.SuiteMethods({
        "com.example.B#test1B",
        "com.example.A#test2A",
        "com.example.A#test1A",
        "com.example.B#test2B",
        "com.example.A#test3A",
})
public class SelectiveTest {
}