Skip to content

Make local whole-suite coverage collection robust to test_core startup crashes #414

Description

@leehack

Problem

During regression QA on commit fda0eb71b55e872736c123c72e339bc5122f102f, local whole-suite coverage collection could crash inside Dart test_core before any tests started. The same checkout passed the normal full VM suite (1,556 tests) and full Chrome suite (864 tests), so this is a coverage-runner/tooling failure rather than evidence of a product regression.

Impact

The local coverage gate can become nondeterministic or unavailable, making it harder to distinguish genuine coverage regressions from test-runner startup failures.

Acceptance criteria

  • Capture and preserve the exact test_core startup exception and environment on reproduction.
  • Provide a deterministic repository command or wrapper for whole-suite coverage collection.
  • Fail with actionable diagnostics when the Dart coverage runner cannot initialize.
  • Demonstrate repeated successful coverage runs without weakening the 70% threshold or omitting VM/Chrome coverage.
  • Keep normal full VM and Chrome suites unchanged.

Discovered while validating issues #402, #406, and #407; track separately because it is unrelated to the specialized grammar fixes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority:P2Planned next: useful unblocked work or validation after P1 items

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions