Repository navigation
Crash with JUnit 5.14 due to TestPlan internal API breaking change #390
Description
I see you've just updated to JUnit 5.14, but haven't really changed anything other than the dep.
I'm getting this crash on JUnit 5.13 to 5.14 update (keeping your lib 1.8):
java.lang.NoSuchMethodError: No direct method (ZLorg/junit/platform/engine/ConfigurationParameters;Lorg/junit/platform/engine/reporting/OutputDirectoryProvider;)V in class Lorg/junit/platform/launcher/TestPlan; or its super classes (declaration of 'org.junit.platform.launcher.TestPlan' appears in /data/app/net.twisterrob.colorfilters.android.base.test-12STHJQGEsmSwg_I_QSkWg==/base.apk!classes5.dex)
java.lang.NoSuchMethodError: No direct method <init>(ZLorg/junit/platform/engine/ConfigurationParameters;Lorg/junit/platform/engine/reporting/OutputDirectoryProvider;)V in class Lorg/junit/platform/launcher/TestPlan; or its super classes (declaration of 'org.junit.platform.launcher.TestPlan' appears in /data/app/net.twisterrob.colorfilters.android.base.test-12STHJQGEsmSwg_I_QSkWg==/base.apk!classes5.dex)
at de.mannodermaus.junit5.internal.runners.AndroidJUnitPlatformTestTree$ModifiedTestPlan.<init>(AndroidJUnitPlatformTestTree.kt:184)
at de.mannodermaus.junit5.internal.runners.AndroidJUnitPlatformTestTree.<init>(AndroidJUnitPlatformTestTree.kt:33)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5.generateTestTree(AndroidJUnit5.kt:74)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5.testTree_delegate$lambda$0(AndroidJUnit5.kt:30)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5.$r8$lambda$KSfagmfVkPVu2wrnBs3DKovzKK4(Unknown Source:0)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5$$ExternalSyntheticLambda0.invoke(D8$$SyntheticClass:0)
at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:86)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5.getTestTree(AndroidJUnit5.kt:30)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5.getDescription(AndroidJUnit5.kt:32)
at de.mannodermaus.junit5.internal.runners.JUnit5RunnerFactoryKt.hasExecutableTests(JUnit5RunnerFactory.kt:32)
at de.mannodermaus.junit5.internal.runners.JUnit5RunnerFactoryKt.tryCreateJUnit5Runner(JUnit5RunnerFactory.kt:28)
at de.mannodermaus.junit5.AndroidJUnit5Builder.runnerForClass(AndroidJUnit5Builder.kt:70)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at androidx.test.internal.runner.AndroidRunnerBuilder.runnerForClass(AndroidRunnerBuilder.java:146)
at androidx.test.internal.runner.ScanningTestLoader.doCreateRunner(ScanningTestLoader.java:50)
at androidx.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:64)
at androidx.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:842)
at androidx.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:672)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:453)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2189)
Version Catalog: https://github.com/TWiStErRob/net.twisterrob.colorfilters/blob/5bf271cc162aa69dc1ccf65a2d17acc23d0455fd/gradle/libs.versions.toml
Reference: https://github.com/TWiStErRob/net.twisterrob.colorfilters/blob/5bf271cc162aa69dc1ccf65a2d17acc23d0455fd/component/test-base-ui/build.gradle#L40
Failed CI: https://github.com/TWiStErRob/net.twisterrob.colorfilters/actions/runs/18140714173/job/51630654198?pr=474
Test report: https://github.com/TWiStErRob/net.twisterrob.colorfilters/pull/474/checks?check_run_id=51631156586
Build scan: https://scans.gradle.com/s/krkvqoazfjik2
I wonder if I'm missing something, or there's an actual breakage your CI didn't pick up?
Crash pointing to
Breakage caused by
junit-team/junit-framework@d1335cd#diff-9f26cdfdb6e74804dd2eddb0eb902ed4b0391e67b972c1dde3e630b8ff41e293R101 (TestPlan.java#101)
My guess is that a recompilation on your side will make android-junit5 1.9.0 incompatible with JUnit <5.14, because
-public interface OutputDirectoryProvider {
+public interface OutputDirectoryProvider extends OutputDirectoryCreator {means that the new bytecode will bind to the new constructor and a similar (but opposite) exception to the above will throw on earlier versions.
Activity
mannodermaus commentedon Oct 1, 2025
Hi, thanks for reporting. I'm surprised that CI did not catch this, and in fact I cannot seem to make it happen locally just yet either. I can see the deprecation of
OutputDirectoryProviderin favor ofOutputDirectoryCreatorand you are correct that this would cause a binary incompatibility. The best bet is probably to update the instrumentation to the new type and instruct people to double-check that they will use JUnit 5.14.x together with android-junit5 1.14.0.x.TWiStErRob commentedon Oct 1, 2025
To repro, I would imagine what's missing for you is to separate what version is used for compiling the lib and what is used for testing in androidTest.
mannodermaus commentedon Oct 2, 2025
Yeah, that probably makes sense. Honestly, I'm not sure I want to be compatible with whichever version of JUnit 5, though - that's why the plugin version scheme follows its cadence so closely. I will prioritize releasing a proper 1.14.0.0 with compatibility for JUnit 5.14.0 (accompanied by instrumentation 1.9.0 under the hood), but users of an older version would have to stick to the 1.13.x.y line. 🤔
TWiStErRob commentedon Oct 2, 2025
Makes sense, I (Renovate) would be happy to update two libs together.
At the same time it's a bit annoying not being able to use any version. Is there a particular reason for not using public APIs?
mannodermaus commentedon Oct 2, 2025
I'll need to review again, but when this was originally written, IIRC the only way to create a customized TestPlan was to go through this API. Unfortunately, we do need a few unorthodox hooks into the Jupiter system to connect the Android instrumentation correctly...
TWiStErRob commentedon Oct 5, 2025
I think all this needs then is a new release (compiler will take care of the rest) and the only supported version will be 5.14.x.
#391 is not even necessary for 5.14, but it is for 6.0 support.
mpkorstanje commentedon Oct 11, 2025
As an aside. I found the JUnit team to be quite responsive to feature requests. You'll have missed the boat on JUnit 5, but for JUnit 6 you could request feature that allow you to connect properly.
mannodermaus commentedon Oct 11, 2025
@TWiStErRob Plugin 1.14.0.0 is available now, please update and the problem should've disappeared.
@mpkorstanje When time allows, I will reevaluate the integration. Thing is, it was the Android side of things that necessitated certain modifications to the test plan on JUnit's side, so I'm not entirely sure if that would remedy the situation completely. Having stable and maintained APIs is still a benefit, though!
TWiStErRob commentedon Oct 12, 2025
All good on 5.13 and 5.14, nice one!