Skip to content

Run scripted tests on Scala 2.12, 2.13 and 3 - #1829

Draft
lrytz wants to merge 8 commits into
sbt:developfrom
lrytz:scala3-scripted
Draft

lrytz wants to merge 8 commits into
sbt:developfrom
lrytz:scala3-scripted

Conversation

@lrytz

@lrytz lrytz commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Scripted tests ran on Scala 2.12 unless a test pinned another version. So the bridges most users get were barely tested against zinc: 12 tests ran on Scala 3, and none on scala2-sbt-bridge, which sbt uses for Scala 2.13.12 and later. Zinc's 2.13 tests used zinc's own 2.13 bridge.

Now every scripted test runs on 2.12, 2.13 (scala2-sbt-bridge) and 3, unless its build.json pins a version. A marker file next to the script excludes one version and says why: pending-3 treats the test as pending on Scala 3, disabled-2.12 skips it on 2.12. scriptedScalaVersion (or -Dscripted.scalaVersion=3) restricts a run to one version. See contributing-docs/03_scripted_tests.md.

Harness fixes needed for Scala 3:

  • Scripted deleted a removed class's .class files but kept its .tasty, so Scala 3 still found the class and tests expecting an error passed.
  • run found main methods through the extracted API, where the Scala 3 bridge leaves out members inherited from Scala 2 classes, so object Main extends App had no main. It now uses the main classes the bridge reports, as sbt does.

Tests:

  • Tests that rely on Scala 2 behaviour (-Xfatal-warnings, warning texts, Scala 2 syntax, macros) get a Scala 3 copy or are disabled on Scala 3.
  • Tests that fail because of bridge bugs are pending on that version, with the reason in the marker file:
    • On 2.13, the tests of six fixes made in zinc's own bridge that scala2-sbt-bridge doesn't have yet (PRs 1316, 1324, 1507, 1782, 1803, 1807).
    • On Scala 3, four bridge bugs (under-compilation with structural types, over-compilation with companions and constructor defaults), the <empty>.B naming bug fixed in scala3 PR 27136, and a crash on a Java class named C$ with pipelining.
  • Two tests that were pending everywhere pass on Scala 3 and are now pending on Scala 2 only.
  • Tests pinned to Scala 3 or to zinc's 2.13 bridge that are plain Scala now run on all versions.

crossTestBridges now runs the bridge unit tests on 2.12 as well as 2.13, since zinc's own bridge matters mostly for 2.12 now.

The scripted suite takes about 3.5 minutes locally instead of 1.2.

lrytz and others added 8 commits September 24, 2026 20:56
When a class was removed, scripted deleted its .class files but kept its
.tasty, so Scala 3 still found the class and tests that expect an error
passed. Scripted now manages TASTy files as sbt does for Scala 3.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Scripted's `run` looked for main methods in the extracted API. The Scala 3
bridge leaves members inherited from Scala 2 classes out of the API, so an
`object Main extends App` had no main. Use the main classes the bridge
reports instead, as sbt does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Scripted tests ran on 2.12 unless a test pinned another version, so most of
them never exercised the Scala 3 bridge. Now every test that doesn't pin its
version runs on both, and a marker file such as pending-3 or disabled-2.12
excludes one version. scriptedScalaVersion, or -Dscripted.scalaVersion,
restricts a run to one version.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Tests that rely on Scala 2 behaviour get a Scala 3 copy or disabled-3, and
tests that fail because of Scala 3 bridge bugs get pending-3. Two tests that
were pending on all versions pass on Scala 3 and are now pending-2.12 only.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
These tests were pinned to Scala 3 because they were written for Scala 3
bugs, but they are plain Scala and pass on 2.12 as well. subproject-pipelining-3
was a copy of subproject-pipelining, which now runs on Scala 3 itself.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
sbt compiles Scala 2.13.12 and later with scala2-sbt-bridge from the Scala
distribution, but scripted only tested zinc's own 2.13 bridge. Scripted
tests now also run on 2.13 with scala2-sbt-bridge. It is tested with 2.13.18,
so that its version differs from the one zinc's own 2.13 bridge is built for;
the bridge provider tells them apart by version.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
These tests were pinned to zinc's own 2.13 bridge but pass on 2.12 too.
Macro tests are disabled on Scala 3. Tests for fixes in zinc's own bridge
that scala2-sbt-bridge doesn't have yet are pending on 2.13.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Zinc's own bridge matters mostly for 2.12, but crossTestBridges only tested
it on 2.13. It now runs the bridge tests on both.

2.12 records slightly different used names, and doesn't warn about
procedure syntax, so the test for code actions only runs on 2.13.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
retronym added a commit to retronym/zinc that referenced this pull request Oct 5, 2026
Scripted picks a bridge by Scala version, and compilerBridgeScala213Bin
used scala213, the version of the compiler-bridge sources. So "2.13.y"
silently ran on zinc's own bridge. Use scala213ForBridge for it, and fail
fast if the two versions are ever equal.

Bump scala213ForBridge from 2.13.16 to 2.13.18, the latest 2.13 release
(as sbt#1829 also does). sbt compiles 2.13.12 and later with the
scala2-sbt-bridge of the project's own Scala version, so "2.13.y" should
test the newest stock bridge, which is what current sbt users get. It also
gives the two 2.13 bridges different versions, which version-based lookup
needs here; the next commit selects bridges by label, after which that is
no longer required.

Add module-inheritance-extra-hash-213-bin, pending: scala2-sbt-bridge does
not report name kinds yet (sbt#1807's bridge side), so `object B
extends A` still counts as trait B inheriting A on 2.13 and D recompiles.
retronym added a commit to retronym/zinc that referenced this pull request Oct 5, 2026
Scripted picks a bridge by Scala version, and compilerBridgeScala213Bin
used scala213, the version of the compiler-bridge sources. So "2.13.y"
silently ran on zinc's own bridge. Use scala213ForBridge for it, and fail
fast if the two versions are ever equal.

Bump scala213ForBridge from 2.13.16 to 2.13.18, the latest 2.13 release
(as sbt#1829 also does). sbt compiles 2.13.12 and later with the
scala2-sbt-bridge of the project's own Scala version, so "2.13.y" should
test the newest stock bridge, which is what current sbt users get. It also
gives the two 2.13 bridges different versions, which version-based lookup
needs here; the next commit selects bridges by label, after which that is
no longer required.

Add module-inheritance-extra-hash-213-bin, pending: scala2-sbt-bridge does
not report name kinds yet (sbt#1807's bridge side), so `object B
extends A` still counts as trait B inheriting A on 2.13 and D recompiles.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant