Skip to content

fix: make workspace quality gates deterministic - #417

Merged
leehack merged 13 commits into
mainfrom
fix/deterministic-workspace-gates
Aug 23, 2026
Merged

fix: make workspace quality gates deterministic#417
leehack merged 13 commits into
mainfrom
fix/deterministic-workspace-gates

Conversation

@leehack

@leehack leehack commented Aug 22, 2026

Copy link
Copy Markdown
Owner

Closes #405.

Summary

  • add one fail-closed workspace manifest covering the root package, all five maintained examples, and both companion packages
  • resolve dependencies for the root and maintained examples before repository-wide format/analyze while preserving the companions' independent validation lanes
  • prune only actual generated trees, including Flutter ephemeral output, .symlinks, build output, and CocoaPods Pods
  • reject missing, duplicate, non-canonical, or newly unclassified package roots and stop at the first dependency-resolution failure
  • use the same bootstrap boundary in CI, contributor guidance, maintainer docs, the PR checklist, and the test-matrix command
  • define current Flutter stable, matching CI, as the canonical repository-wide formatter while retaining the older runtime-support minimum

This is contributor-tooling scope only. Runtime APIs and supported model, backend, and platform behavior are unchanged.

Before / after

On the original report baseline, running only the root dart pub get left nested examples unresolved: repository-wide format exited 1 with unresolved flutter_lints warnings and five toolchain-dependent rewrites, while analysis reported 9,224 cascading nested-package issues.

At exact head 2033e97c860bd2545b47011aaae1da077d3978bb, with the canonical current-stable toolchain:

  • dart run tool/prepare_workspace.dart classifies all eight maintained package roots and resolves the root plus five examples
  • repository-wide formatting checks 567 files with zero changes
  • root analysis completes with no issues

The two companion packages remain fail-closed manifest members but retain their existing dedicated dependency, analysis, test, SwiftPM, and publish-dry-run lanes. This avoids dirtying the root checkout with companion analyzer migrations while preserving their full validation.

Production completeness

  • Exact manifest contains one canonical root plus all five examples and both companions, with no duplicates
  • All examples prepare for root quality gates; companions remain independently prepared and validated
  • Missing and unclassified packages fail before dependency resolution
  • Exact SDK manager and command order are pinned; first failure stops later commands
  • Generated trees are pruned without over-pruning lookalike source directories
  • Windows path separators normalize to manifest-style /
  • Missing executables and invalid layouts return stable, named exit codes with actionable diagnostics and no stack trace
  • Contributor, maintainer, PR-template, CI, and test-matrix commands agree
  • No runtime API, native/Web/LiteRT asset, release, changelog, example behavior, or capability change

Validation

  • dart run tool/prepare_workspace.dart — 8 roots classified; root + 5 examples resolved
  • dart format --output=none --set-exit-if-changed . — 567 files, 0 changes
  • dart analyze — no issues
  • focused tooling tests — 21 passed
  • dart test -p vm -j 1 --exclude-tags local-only — 1,553 passed, 71 fixture-dependent skips
  • representative existing tiny-GGUF native/model integration in the full VM suite — passed
  • dart run tool/testing/check_platform_boundaries.dart
  • dart run tool/testing/verify_release_docs_versions.dart
  • ./tool/docs/build_site.sh
  • ./tool/docs/validate_links.sh
  • scoped analysis, formatting, workflow validation, and git diff --check
  • exact-head GitHub Actions run 32617906980 — all 11 checks passed, including Windows, macOS, Linux VM coverage, Chrome, Web Chat, docs, companions, analysis, prompt parity, and aggregate coverage
  • fresh independent blocking-only QA on exact head — PASS, no blocking finding

The changed behavior is workspace discovery/preparation, so a new model-specific smoke is N/A. The full VM and hosted Web Chat/native jobs retain representative real runtime/model coverage without claiming it validates tooling semantics.

Independent QA and review follow-through

Prior independent and Copilot reviews exposed and drove durable fixes for:

  • misleading optional-Flutter guidance and a semicolon-separated test-matrix command that could mask preparation failure
  • symlink traversal and incomplete generated-directory pruning, including real Flutter/ephemeral, .symlinks, and CocoaPods paths
  • uncaught SDK launch failures
  • platform-dependent Windows relative paths
  • manifest uniqueness/canonicality, exact package ownership, exact manager/order, missing packages, and first-failure behavior
  • lookalike source directory names that must not be pruned
  • the unnamed EX_USAGE layout-failure exit code

Fresh independent QA passed on exact head 2033e97c860bd2545b47011aaae1da077d3978bb with no blocking finding. Exact-head Copilot rereview recommends approval with zero new comments. All four review threads were answered and resolved.

Risk and sequencing

The canonical repository-wide gate now requires Flutter and validates the maintained package boundary before running. Runtime behavior is unchanged.

PR #420 overlaps CI/template/AGENTS/test-matrix files. To preserve both contracts cleanly, merge this workspace-bootstrap fix first; then integrate current main into #420 and revalidate its regression gate on the combined tree.

Copilot AI lite review requested due to automatic review settings August 22, 2026 20:04

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new workspace bootstrapper should prune generated directories during discovery and should catch Process.start failures to provide actionable errors instead of stack traces.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR makes the repository’s documented quality gates (format/analyze) deterministic by introducing a single workspace bootstrap entrypoint that resolves all maintained package roots before running the root gates, and then aligning CI + contributor/maintainer guidance to use that same entrypoint.

Changes:

  • Add tool/prepare_workspace.dart to deterministically pub get all maintained package roots (with lockfile enforcement for the chat app) and fail closed on unclassified/missing packages.
  • Update CI, the test-matrix command, PR template, and docs to run the workspace preparation step before format/analyze.
  • Apply formatter-only changes to the previously reported test files.
File summaries
File Description
website/docs/maintainers/release-workflow.md Adds workspace bootstrap to release checklist before format/analyze.
website/docs/maintainers/docs-site.md Documents the new canonical preparation boundary and exclusions.
tool/testing/test_matrix.dart Makes the “static-format-analyze” row fail-closed and prep-aware.
tool/prepare_workspace.dart New deterministic workspace bootstrapper and manifest validation.
test/unit/tooling/test_matrix_test.dart Asserts the test-matrix row keeps preparation ahead of format/analyze.
test/unit/tooling/prepare_workspace_test.dart Adds unit coverage for manifest validation, ordering, lockfile enforcement, and symlink behavior.
test/unit/tooling/macos_litert_lm_prepare_app_script_test.dart Formatter-only updates to match canonical formatting.
test/unit/core/engine/engine_test.dart Formatter-only updates to match canonical formatting.
test/integration/engine_reloading_test.dart Formatter-only updates to match canonical formatting.
test/integration/core/template/llama_cpp_template_detection_integration_test.dart Formatter-only updates to match canonical formatting.
test/e2e/backends/llama_cpp_chat_template_backend_e2e_test.dart Formatter-only updates to match canonical formatting.
README.md Adds workspace preparation to the documented “package changes” command sequence.
CONTRIBUTING.md Updates prerequisites and guidance to reflect Flutter requirement for repo-wide gates + new bootstrap step.
AGENTS.md Replaces root dart pub get with the canonical workspace bootstrap command.
.github/workflows/ci.yml Uses tool/prepare_workspace.dart as the CI dependency installation boundary.
.github/pull_request_template.md Adds the workspace preparation command to the PR test plan checklist.
Review details
  • Files reviewed: 16/16 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tool/prepare_workspace.dart Outdated
Comment thread tool/prepare_workspace.dart Outdated
Copilot AI review requested due to automatic review settings August 22, 2026 20:09

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The workspace discovery ignore logic is not robust to real Flutter-generated Flutter/ephemeral and iOS .symlinks paths, which can make subsequent bootstrap runs fail with unclassified package errors.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

test/unit/tooling/prepare_workspace_test.dart:70

  • The generated Flutter ephemeral tree path casing differs across platforms; in this repo it’s commonly .../Flutter/ephemeral/... (capital Flutter). Updating this fixture to use the real casing (and adding an iOS .symlinks example) will ensure the ignore logic is validated against the actual generated directory names.
    for (final path in <String>[
      'example/chat_app/.dart_tool/generated/pubspec.yaml',
      'example/chat_app/build/generated/pubspec.yaml',
      'example/chat_app/macos/flutter/ephemeral/plugin/pubspec.yaml',
    ]) {
  • Files reviewed: 16/16 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread tool/prepare_workspace.dart Outdated
Copilot AI review requested due to automatic review settings August 22, 2026 20:14

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The new canonical prepare_workspace entry point can terminate with an uncaught ProcessException (e.g., missing flutter), producing a stack trace instead of a stable exit code and actionable diagnostics.

Review details

Suppressed comments (1)

tool/prepare_workspace.dart:172

  • runWorkspaceCommand does not handle Process.start failures (e.g., flutter/dart not on PATH, permissions). In that case Dart will throw a ProcessException and the script will terminate with a stack trace instead of a clear, actionable error and stable exit code for CI/contributors.
  final process = await Process.start(
    executable,
    arguments,
    workingDirectory: workingDirectory,
    mode: ProcessStartMode.inheritStdio,
  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings August 22, 2026 20:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are cohesive, well-tested for the new workspace bootstrap behavior, and consistently applied across CI and documentation without introducing runtime behavior changes.

Review details
  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings August 22, 2026 20:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The new bootstrap logic is explicit, fail-closed, CI-integrated, and covered by targeted unit tests that validate ordering, pruning, and failure propagation.

Review details
  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings August 22, 2026 20:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The bootstrap logic, CI integration, and documentation updates are consistent and are backed by targeted regression tests for the previously non-deterministic workspace conditions.

Review details
  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

leehack commented Aug 22, 2026

Copy link
Copy Markdown
Owner Author

Final QA handoff: PR #417 remains draft/open/mergeable. Exact base 080fb1c08b78a143bf482b885ebeb16dfa7ca0cd; exact head 3dfd3989c4c1916f89d94b777b6b38debfe32c35. Hosted CI run 32600783616 is fully green (all 11 jobs). Local focused workspace tests, root analyzer, formatter, essential matrix, and diff checks pass; generated Pods pruning is covered. All 4 review threads are resolved. Separate Sol blocking-only QA is still the remaining readiness gate; do not mark ready or merge.

Copilot AI review requested due to automatic review settings August 22, 2026 21:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

There are verified documentation and tooling issues (CONTRIBUTING bootstrap path mismatch and generated-directory pruning gaps) that should be addressed before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

tool/prepare_workspace.dart:114

  • discoverWorkspacePubspecs currently prunes .dart_tool, build, .symlinks, and Flutter/ephemeral, but it will still recursively traverse CocoaPods output like ios/Pods/ / macos/Pods/ (these are gitignored in example/chat_app/{ios,macos}/.gitignore). That can make tool/prepare_workspace.dart unexpectedly slow on machines that have built the iOS/macOS runners. Consider pruning Pods the same way as other generated directories.
  final name = segments.last;
  return name == '.dart_tool' ||
      name == 'build' ||
      name == '.symlinks' ||
      _isFlutterEphemeral(segments);
  • Files reviewed: 11/11 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread CONTRIBUTING.md
Copilot AI review requested due to automatic review settings August 22, 2026 21:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The new deterministic workspace bootstrap is well-scoped, covered by targeted unit tests, and integrated consistently into CI and contributor documentation without introducing runtime behavior changes.

Review details
  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings August 22, 2026 21:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

A new unit test is platform-dependent (Windows path separators) and should be normalized to avoid failing on Windows.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

test/unit/tooling/prepare_workspace_test.dart:40

  • The test builds relative paths using substring, which preserves platform separators (e.g. example\chat_app on Windows) but later compares against workspacePackages entries that use /. This makes the assertion platform-dependent and can fail on Windows.
        commandRunner: (executable, arguments, workingDirectory) async {
          final relative = workingDirectory == root.path
              ? '.'
              : workingDirectory.substring(root.path.length + 1);
          commands.add('$executable ${arguments.join(' ')} @ $relative');
          return 0;
  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings August 23, 2026 04:17

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The bootstrap behavior is well-scoped, consistently integrated across CI/docs/templates, and includes targeted unit tests covering the key determinism and failure-mode requirements.

Review details
  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings August 23, 2026 04:20

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are tooling-only, well-covered by targeted unit tests, and consistently integrated across CI and documentation with no apparent runtime impact.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

tool/prepare_workspace.dart:161

  • prepareWorkspace returns 64 for manifest/layout failures, but the exit code is a magic number here. Naming it (and documenting it as EX_USAGE) makes the intent clearer and reduces the chance of accidental drift if the code later needs to reuse the same exit code in multiple places.
  final layoutErrors = validateWorkspaceManifest(repositoryRoot);
  if (layoutErrors.isNotEmpty) {
    for (final error in layoutErrors) {
      stderr.writeln(error);
    }
    return 64;
  }
  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings August 23, 2026 04:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The bootstrap logic is cohesive and well-tested, CI is updated to use it consistently, and the documentation changes align with the new canonical quality-gate workflow.

Review details
  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@leehack
leehack marked this pull request as ready for review August 23, 2026 06:49
@leehack
leehack merged commit c53ee8d into main Aug 23, 2026
12 checks passed
@leehack
leehack deleted the fix/deterministic-workspace-gates branch August 23, 2026 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make root format and analyze gates deterministic across nested examples

2 participants