Skip to content

[FLO-13.7] Add clean-room compatibility, CI tiers, and regression promotion #34

Description

@szmyty

Parent: #13
Depends on: #28, #29, #30, #31, #32, #33
Suite roadmap: #11
Downstream audit: #12

Outcome

Turn the completed scenario families into a maintainable release gate: a clean-room provider compatibility matrix, enforceable CI tiers, deterministic coverage reporting, and a safe path for promoting production or fuzz regressions.

Scope

  • Build the compatibility matrix for every Flow plan, run, checkpoint, and provider contract version current policy promises to consume.
  • Exercise independently built or immutable provider artifacts; never import sibling workspaces.
  • Define and enforce pull-request hermetic smoke/failure coverage, scheduled broader compatibility/resource coverage, and manual/release clean-room provider coverage.
  • Record exact Flow/provider revisions, artifact digests, scenario-manifest version, toolchain, timestamps, and completeness state.
  • Make unavailable optional dependencies explicit skips or unavailable evidence, not silent passes.
  • Add deterministic sharding, time/memory/output/artifact-size budgets, and retention rules.
  • Document how a bug or fuzz finding is minimized, assigned a stable scenario ID, given provenance, reviewed, and promoted.
  • Generate a machine-readable coverage report and human summary without claiming finite fixtures prove universal support.
  • Reconcile Build cross-tool orchestration and failure fixture suites #13 and [EPIC] Finish the flow suite and prove production-ready orchestration #11 with exact commits, workflow runs, provider artifacts, residual gaps, and intentional deferrals.

Acceptance criteria

  • The compatibility matrix covers every supported version edge and distinguishes supported, unsupported, unavailable, and untested combinations.
  • At least one clean-room run uses immutable provider artifacts rather than source-tree imports.
  • PR, scheduled, and release tiers have enforced budgets and honest skip semantics.
  • Canonical fixtures are drift-checked and never rewritten by CI.
  • Regression promotion preserves provenance, minimization, redistribution safety, and expected typed behavior.
  • Coverage reports expose gaps and do not overclaim.
  • Documentation, package contents, workflows, contract checks, and repository validation pass.
  • Parent Build cross-tool orchestration and failure fixture suites #13 is ready to close only after all acceptance evidence is linked and reconciled.

Non-goals

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions