From 6bd7553803b550f8827a44dcfc71de0ff91b5dfe Mon Sep 17 00:00:00 2001 From: Douwe de Vries Date: Wed, 28 Oct 2026 22:33:50 +0100 Subject: [PATCH] chore(release): prepare OpenAI-qualified 10.0.2 candidate --- .../22-flow10-guidance-release/README.md | 68 ++++++++++++++++ CHANGELOG.md | 12 ++- README.md | 6 +- docs/development.md | 2 +- docs/release-qualification.md | 29 ++++--- evals/README.md | 7 +- evals/release-policy.ts | 33 +++++++- package.json | 2 +- tests/documentation-contract.test.ts | 6 +- tests/eval-release-sampling.test.ts | 18 ++--- tests/qualification-cli.test.ts | 10 +-- tests/release-1000.test.ts | 12 +-- tests/release-1002.test.ts | 78 +++++++++++++++++++ 13 files changed, 231 insertions(+), 52 deletions(-) create mode 100644 .agents/plans/22-flow10-guidance-release/README.md create mode 100644 tests/release-1002.test.ts diff --git a/.agents/plans/22-flow10-guidance-release/README.md b/.agents/plans/22-flow10-guidance-release/README.md new file mode 100644 index 00000000..ef831866 --- /dev/null +++ b/.agents/plans/22-flow10-guidance-release/README.md @@ -0,0 +1,68 @@ +# Flow 10.0.2 qualification + +Ship PR #211's guidance consolidation and status-error reporting correction. +The package bytes changed, so the narrow offline patch path does not apply. +Qualify a new exact artifact; preserve 10.0.1's completed campaigns, canary, +publication records and consumed budgets without reuse or partial-result pooling. + +## Policy + +The operator selected OpenAI-only with review of an exact-version exception. +Only 10.0.2 gains that exception, pinned to direct `openai/gpt-6.1-sol` for manager +and reviewer. 10.0.0/10.0.1 retain their exact historical policies. Future versions +retain two distinct providers; aliases of the same provider do not create breadth. +No cross-provider reliability claim follows from a single-provider qualification. + +Keep the existing 17-case prefix and all its pass/sample thresholds, zero false +completions, zero unsubmitted reviews, canonical Linux/OpenCode 2.1.4 host, +HTTP transport, recovery off, and environment-only reserves. Append the three +status cases at three primary attempts each, 100% pass rate, one declared reserve +each. This gives 20 cases, 81 primary cells, 20 reserves; 99 primary steps and 26 +reserve steps, plus one model route probe: 100 planned or 126 maximum top-level +starts. Failed starts count; internal turns and tokens are not capped. + +The status fixtures are synthetic starting state, not paid review observations. +In particular the blocked fixture carries a seeded failed review. Retain it as +fixture context; do not describe it as a review generated by this campaign. + +## Gates and approvals + +1. Review and merge version, policy, regression tests, installation pins and + changelog through normal CI. Pass frozen install, root check, replay, audit and + local-provider packed native smoke without importing an account. +2. Freeze the exact merged source, artifact bytes, manifest, grader closure, + canonical toolchain and Linux preparation image. Verify source before/after, + all packaged outputs and byte-exact repack. Prepare a sealed proposal and + supervisor using maintained entry points. Never run a stale 10.0.1 supervisor. +3. Ask for 126 fresh starts bound to the frozen candidate/model and a concrete + expiry. Create no ledger or model call before approval. Use the existing + OpenCode 2 OpenAI subscription, with no API-key fallback or other vendor. +4. After grant, run zero-generation native readiness, then the complete new + matrix once. Retain original failures. Only declared retryable environment + reserves activate; product/evaluator failures do not. At expiry/cap stop new + starts and drain authorized work. Close only owned worker/source/tunnel. +5. Inspect actual report rates, samples, false completion and reviewer gates. + Only after PASS prepare the exact-artifact canary offline and request its + separate one-start approval. The matrix budget cannot fund that canary. +6. Seal/regrade complete evidence, land it without changing product bytes, + recheck main CI and exact artifact identity, and publish through the maintained + release workflow under release authority. Verify npm and GitHub bytes match. + +The earlier 224 qualification/canary starts stay consumed. Their unused balances +are not transferred or refunded. The protected native service is accessed only +through authorized APIs; no OS query or signal of the protected native account service. +No source guard removal, reauthentication or credential fallback is authorized. + +## Preparation evidence + +- Frozen install: no dependency changes. Audit: no vulnerabilities across 278 + packages. Replay: all 13 gated cassettes reproduce. +- Fifty focused policy, sampling and documentation tests pass. The first full + check retained two stale current-version sampling assertions; both are fixed. + Clean committed push preflight repeats the complete gate before submission. +- Packed local-provider native smoke passes two tests and 128 assertions without + importing an account or generating real-model work. +- Independent GPT-6.1 Sol review: two rounds, final APPROVE. Exact-version scope, + historical policy preservation and 100/126 dispatch arithmetic were checked. +- This document and the count draft authorize no paid starts. The final proposal + still requires exact merged-source/artifact/evaluator bindings and approval. diff --git a/CHANGELOG.md b/CHANGELOG.md index 63b48451..8e89fb62 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,13 +2,17 @@ One short entry per release, written for users deciding whether to upgrade. -## Unreleased +## [10.0.2] - Put ordinary Flow execution before recovery details, share goal-alignment and delivery instructions across the existing guides, and clarify read-only status - routing. Commands, tool inputs, Session v5 and runtime gates are unchanged. -- Add three development status evals and one offline guidance check. Offline - results do not claim model improvement or qualify a new release. + routing. Commands, tool inputs, Session v5 schema and runtime gates are unchanged. +- Add three status evals and one offline guidance check. Qualification retains + all 17 earlier cases and adds the three status cases at 100% over three attempts. + Exact 10.0.2 pins manager and reviewer to OpenAI GPT-6.1 Sol; future releases + keep two providers. No cross-provider reliability claim follows. + +Upgrade with `opencode2 plugin add opencode-plugin-flow@10.0.2`. ## [10.0.1] diff --git a/README.md b/README.md index 9a079565..327bfdf5 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,7 @@ Install OpenCode v2 and the exact Flow release: ```sh npm install -g @opencode/cli@2.1.4 opencode2 --version -opencode2 plugin add opencode-plugin-flow@10.0.1 +opencode2 plugin add opencode-plugin-flow@10.0.2 ``` The version command must report `opencode v2.1.4` or a later tested 2.x version. @@ -57,7 +57,7 @@ For project scope, add this entry to the project's `opencode.json`: ```json { - "plugins": ["opencode-plugin-flow@10.0.1"] + "plugins": ["opencode-plugin-flow@10.0.2"] } ``` @@ -74,7 +74,7 @@ Set plugin defaults: ```json { "plugins": [{ - "package": "opencode-plugin-flow@10.0.1", + "package": "opencode-plugin-flow@10.0.2", "options": { "reviewer": { "model": "provider/model", "steps": 80 }, "typesafeApiKeyEnv": "MY_TYPESAFE_KEY" diff --git a/docs/development.md b/docs/development.md index fc24269f..eacd8eb4 100644 --- a/docs/development.md +++ b/docs/development.md @@ -158,7 +158,7 @@ Follow the [frozen-candidate sequence](release-qualification.md#running-it): finish fixes and dependency updates, pass deterministic checks, approve paid evals. `bun run qualify -- --campaign-dir --canary ` seals the -policy grid, canary and grader evidence. 10.0.0/10.0.1 pin `openai/gpt-6.1-sol`. +policy grid, canary and grader evidence. 10.0.0/10.0.1/10.0.2 pin `openai/gpt-6.1-sol`. See [model policy](release-qualification.md#the-qualification-bar) for historical exceptions and later defaults. Commit the bundle before tagging; interrupted results cannot qualify. diff --git a/docs/release-qualification.md b/docs/release-qualification.md index 42b276e0..4380cb33 100644 --- a/docs/release-qualification.md +++ b/docs/release-qualification.md @@ -9,7 +9,7 @@ This page owns release thresholds, candidate freezing, and publication order. | Threshold | Value | Why | | --- | --- | --- | -| Distinct providers | 1 for 9.1.0, 9.2.0, 9.3.0, 9.4.0, 9.5.0, 9.6.0, and 10.0.0/10.0.1; otherwise 2 | 9.5.0, 9.6.0, and 10.0.0/10.0.1 pin `openai/gpt-6.1-sol`; earlier exceptions pin GPT-6 Sol. No cross-provider reliability claim. | +| Distinct providers | 1 for 9.1.0, 9.2.0, 9.3.0, 9.4.0, 9.5.0, 9.6.0, and 10.0.0/10.0.1/10.0.2; otherwise 2 | Explicit version-scoped model pins below; no cross-provider reliability claim. | | False completions | 0 | A `completed` closure the document itself contradicts is the failure Flow exists to prevent. | | Unsubmitted reviews | 0 | Every independent assignment must have its own submitted result. | | Scored attempts per provider | 3 at 100%; 10 at 90% | The frozen release plan gives each threshold enough trials to express its allowed failures. | @@ -26,6 +26,7 @@ This page owns release thresholds, candidate freezing, and publication order. | `auto-two-features-evidence` | 100% in 9.4.0 through 9.6.0 and the v2 profile | Two dependent features, native reviewer packet access and final gate. | | `auto-prerequisite-repair` | 100% in 9.4.0 through 9.6.0 and the v2 profile | In-target repair or accepted failed-gate amendment; immutable canonical gate. | | `auto-observe-with-required-pass` | 100% in 9.4.0 through 9.6.0 and the v2 profile | Honest nonzero audit plus separate required pass on the reviewed source. | +| Three status cases | 100% from 10.0.2 | Read-only compact/detail/error routes preserve state, supplied delivery and mutation disclosure. | | Five delivery cases | 100% in 9.6.0 and the v2 profile | Completion, deferral, nonzero observations, full detail and idle status preserve truthful native facts. | Ungated exploratory scenarios are listed in [evals](../evals/README.md#scenarios). @@ -50,16 +51,14 @@ failures never activate reserves. Evaluator failure is `NOT VERIFIED`; persistence failure stops without a finalized report. Repository code owns catalog order; persisted `catalog.json` must match. -Primary/reserve cells on GPT-6 Sol are 38/8 in 9.1.0 and 9.2.0, 48/9 in -9.3.0, and 57/12 in 9.4.0. The 9.4.0 profile adds three autonomous cases. -Version 9.5.0 keeps those twelve cases on GPT-6.1 Sol, with 57/12 cells. -Version 9.6.0 adds five delivery cases, collects them first, and uses 72/17 -cells. 10.0.0/10.0.1 retain all 17 cases on `openai/gpt-6.1-sol` with 72 primary -and 17 reserve cells. Other v2 releases retain two routes and 144/34 cells. -Manager and reviewer share the approved model per cell. Recovery is off. -Other historical generic profiles use 96/18. Cell counts are not dispatch limits. -Changed order requires a new freeze and approval. Existing campaigns stay immutable; narrowed or merged reports -cannot qualify. +Primary/reserve cells: 9.1.0/9.2.0 use 38/8; 9.3.0 uses 48/9; +9.4.0/9.5.0 use 57/12; 9.6.0 and 10.0.0/10.0.1 use 72/17. +10.0.2 appends three status cases to that 17-case prefix, each three attempts at +100%, giving 81/20 cells. Later v2 releases keep two providers and 20 cases, +162/40 cells. Other historical generic profiles use 96/18. +Manager and reviewer share the approved model per cell; recovery is off. +Cells are not dispatch limits. Changed order requires a new freeze and approval. +Existing campaigns stay immutable; narrowed or merged reports cannot qualify. Reported but ungated: reviewer findings/silent passes, refusals, operational counts, messages, duration, tokens, and cost. @@ -115,7 +114,7 @@ Keep that ledger across retries. Budget-stopped campaigns cannot qualify. For native accounts, follow the [source setup](../evals/README.md#native-account-authentication). Prepare with `--native-auth`, launch with `--native-auth-source`, and record with the matching `--launch-receipt`. Live connection data remains private. -For 9.1.0 through 9.4.0, pin GPT-6 Sol. Exact 9.5.0, 9.6.0, and 10.0.0/10.0.1 pin GPT-6.1 Sol. +For 9.1.0 through 9.4.0, pin GPT-6 Sol. Exact 9.5.0, 9.6.0, and 10.0.0/10.0.1/10.0.2 pin GPT-6.1 Sol. ```bash env -u TYPESAFE_API_KEY -u OPENCODE_FLOW_REVIEWER_MODEL \ @@ -133,9 +132,9 @@ The canary launcher needs its separately approved ledger in `FLOW_EVAL_AUTHORIZA and provider access through its environment. It prints the native export path, collects it before stopping the isolated host, and charges failed launches. -Current steps derive 90 primary dispatches, 23 reserves, and one route -probe, giving 91 planned or 114 maximum top-level dispatches. Historical 9.6.0 -has the same counts. Internal turns and reviewer generations are separate; +10.0.2 derives 99 primary dispatches, 26 reserves, and one route probe: 100 planned +or 126 maximum top-level starts. Historical 9.6.0 and 10.0.0/10.0.1 use 91/114. +The default two-provider 20-case profile uses 200 planned or 252 maximum starts. Internal turns and reviewer generations are separate; no dollar cap follows. Canary approval is separate. Refresh the draft at final freeze. Paid work stays HOLD until purpose, models, count, and expiry are approved. diff --git a/evals/README.md b/evals/README.md index 9febd911..385a4bbe 100644 --- a/evals/README.md +++ b/evals/README.md @@ -21,8 +21,8 @@ The new `guidance-status-ready`, `guidance-status-blocked`, and `guidance-status-error` cases measure compact-only status, one detail read for a blocked state, and stopping after a status error. Each seeds synthetic Session state and checks observed calls, unchanged persisted state/workspace, and the -actual returned status/error text. They are development report-only cases and do -not change the frozen 10.0.1 release matrix. +actual returned status/error text. They remain development report-only in ordinary runs. From 10.0.2 they are also +required release cases; the frozen 10.0.1 matrix stays unchanged. For model behavior, use the ordinary runner after obtaining a current bounded [dispatch authorization](../.agents/plans/05-release-simplification/README.md#authorize-paid-work): @@ -167,7 +167,8 @@ Release mode is strictly sequential (`--concurrency 1`). Versions 9.1.0 and 9.3.0 pins the same model with 48 primary targets and nine reserves. Version 9.4.0 uses 57 primary targets and 12 reserves on the same route. 10.0.0/10.0.1 pin `openai/gpt-6.1-sol` across all 17 cases with 72 primary cells and 17 reserves. -Other v2 releases retain two routes and 144/34 cells. These exceptions make +10.0.2 adds the three required status cases for 81/20 cells on the same route. +Later v2 releases require two providers and 162/40 cells. These exceptions make no cross-provider reliability claim. Cells are scheduling units; count scenario steps and probes separately for the dispatch authorization. Only retained retryable diff --git a/evals/release-policy.ts b/evals/release-policy.ts index f1855bfc..1e391242 100644 --- a/evals/release-policy.ts +++ b/evals/release-policy.ts @@ -165,6 +165,28 @@ const deliveryParsed = parseCaseCatalog([ if (!deliveryParsed.ok) throw new Error("Delivery release policy is invalid."); const DELIVERY_RELEASE_CATALOG = deliveryParsed.value; +// 10.0.2 adds status coverage after the unchanged historical 17-case prefix. +const guidanceParsed = parseCaseCatalog([ + ...DELIVERY_RELEASE_CATALOG, + ...[ + "guidance-status-ready", + "guidance-status-blocked", + "guidance-status-error", + ].map((caseId) => ({ + caseId, + caseVersion: 1, + evidenceClass: "conformance", + oracle: "durable-state", + release: "required", + minProviders: 1, + minScoredAttempts: 3, + minPassRate: 1, + reviewerPromotionRecordSha256: null, + })), +]); +if (!guidanceParsed.ok) throw new Error("Guidance release policy is invalid."); +const GUIDANCE_RELEASE_CATALOG = guidanceParsed.value; + export type ReleaseProfile = { readonly catalog: ValidatedCaseCatalog; readonly requiredModels: readonly ModelIdentity[] | null; @@ -197,18 +219,23 @@ export function targetsOpenCodeV2(packageVersion: string): boolean { } const V2_RELEASE: ReleaseProfile = { - catalog: DELIVERY_RELEASE_CATALOG.map((row) => ({ + catalog: GUIDANCE_RELEASE_CATALOG.map((row) => ({ ...row, minProviders: 2, })), requiredModels: null, }; -const OPENAI_ONLY_V2_RELEASES = new Set(["10.0.0", "10.0.1"]); +const OPENAI_ONLY_V2_RELEASES = new Set(["10.0.0", "10.0.1", "10.0.2"]); export function releaseProfile(packageVersion: string): ReleaseProfile { if (OPENAI_ONLY_V2_RELEASES.has(packageVersion)) - return openAiOnlyRelease(DELIVERY_RELEASE_CATALOG, "gpt-6.1-sol"); + return openAiOnlyRelease( + packageVersion === "10.0.2" + ? GUIDANCE_RELEASE_CATALOG + : DELIVERY_RELEASE_CATALOG, + "gpt-6.1-sol", + ); if (targetsOpenCodeV2(packageVersion)) return V2_RELEASE; if (packageVersion === "9.6.0") return openAiOnlyRelease(DELIVERY_RELEASE_CATALOG, "gpt-6.1-sol"); diff --git a/package.json b/package.json index 1d07c37d..ab838e75 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "opencode-plugin-flow", - "version": "10.0.1", + "version": "10.0.2", "description": "Durable planning, validation, and independent review for OpenCode v2", "type": "module", "repository": { diff --git a/tests/documentation-contract.test.ts b/tests/documentation-contract.test.ts index 0c8caf12..da2b2c9d 100644 --- a/tests/documentation-contract.test.ts +++ b/tests/documentation-contract.test.ts @@ -599,8 +599,10 @@ describe("Flow documentation contract", () => { changelog, `[${packageVersion}]${releaseDate ? ` - ${releaseDate}` : ""}`, ); - expect(currentReleaseNotes).toContain( - `opencode plugin add opencode-plugin-flow@${packageVersion}`, + expect(currentReleaseNotes).toMatch( + new RegExp( + `opencode2? plugin add opencode-plugin-flow@${packageVersion.replaceAll(".", "\\.")}`, + ), ); // Release notes must state the schema impact explicitly rather than the // uninformative "public surface is unchanged". diff --git a/tests/eval-release-sampling.test.ts b/tests/eval-release-sampling.test.ts index b5535bc5..808e3f28 100644 --- a/tests/eval-release-sampling.test.ts +++ b/tests/eval-release-sampling.test.ts @@ -381,24 +381,24 @@ describe("release eval sampling", () => { ).toThrow("openai/gpt-6.1-sol"); }); - test("schedules the current OpenAI-only seventeen-case grid", () => { + test("schedules the current OpenAI-only twenty-case grid", () => { const scenarios = releaseScenarios(packageJson.version); - expect(scenarios).toHaveLength(17); + expect(scenarios).toHaveLength(20); const plan = campaignPlanFor({ models: ["openai/gpt-6.1-sol"], scenarios, sampling: { kind: "release", packageVersion: packageJson.version }, opencodeVersion: "2.1.4", }); - expect(plan.stoppingRule.count).toBe(72); - expect(plan.budget.maxAttempts).toBe(89); - expect(plan.abortPolicy.maxReplacementBlocks).toBe(17); + expect(plan.stoppingRule.count).toBe(81); + expect(plan.budget.maxAttempts).toBe(101); + expect(plan.abortPolicy.maxReplacementBlocks).toBe(20); expect( plan.cells.filter((cell) => cell.schedule === "primary"), - ).toHaveLength(72); + ).toHaveLength(81); expect( plan.cells.filter((cell) => cell.schedule === "environment-reserve"), - ).toHaveLength(17); + ).toHaveLength(20); }); test("keeps ordinary scenario catalogs report-only", () => { @@ -603,8 +603,8 @@ for (const version of ["10.0.0", "10.0.1"]) { } }); } -test("10.0.2 retains the default two-provider requirement", () => { - expect(releaseCatalog("10.0.2").every((row) => row.minProviders === 2)).toBe( +test("10.0.3 retains the default two-provider requirement", () => { + expect(releaseCatalog("10.0.3").every((row) => row.minProviders === 2)).toBe( true, ); }); diff --git a/tests/qualification-cli.test.ts b/tests/qualification-cli.test.ts index def77d9e..073cdcd7 100644 --- a/tests/qualification-cli.test.ts +++ b/tests/qualification-cli.test.ts @@ -55,11 +55,11 @@ import { assertQualificationBundle } from "../scripts/release-metadata.js"; import { autoQualifiedOutcome } from "./fixtures/auto-qualified-outcome.js"; -test("current release defaults retain 17 cases on OpenAI and three autonomous attempts", () => { - expect(packageJson.version).toBe("10.0.1"); +test("current release defaults retain 20 cases on OpenAI and three autonomous attempts", () => { + expect(packageJson.version).toBe("10.0.2"); const scenarios = releaseScenarios(packageJson.version); const catalog = releaseCatalog(packageJson.version); - expect(scenarios).toHaveLength(17); + expect(scenarios).toHaveLength(20); expect([...new Set(catalog.map((row) => row.minProviders))]).toEqual([1]); const plan = campaignPlanFor({ models: ["openai/gpt-6.1-sol"], @@ -68,11 +68,11 @@ test("current release defaults retain 17 cases on OpenAI and three autonomous at opencodeVersion: releaseHostPolicy(packageJson.version).opencodeVersion, }); expect(plan.cells.filter((cell) => cell.schedule === "primary")).toHaveLength( - 72, + 81, ); expect( plan.cells.filter((cell) => cell.schedule === "environment-reserve"), - ).toHaveLength(17); + ).toHaveLength(20); expect( [ ...new Set(plan.cells.map((cell) => cell.managerModel?.routeProvider)), diff --git a/tests/release-1000.test.ts b/tests/release-1000.test.ts index c3dbaf93..d2f134f1 100644 --- a/tests/release-1000.test.ts +++ b/tests/release-1000.test.ts @@ -141,10 +141,10 @@ test("OpenAI-only 10.0.0 admits only the complete canonical direct identity", () ); }); -test("OpenAI-only exception leaves later release defaults unchanged", () => { - for (const version of ["10.0.2", "10.1.0", "11.0.0"]) { +test("OpenAI-only exceptions keep later releases on two providers", () => { + for (const version of ["10.0.3", "10.1.0", "11.0.0"]) { expect(releaseCatalog(version).map((row) => row.minProviders)).toEqual( - Array(17).fill(2), + Array(20).fill(2), ); expect(() => assertReleaseModels([manager], version)).toThrow("exactly 2"); expect(() => @@ -156,15 +156,15 @@ test("OpenAI-only exception leaves later release defaults unchanged", () => { assertReleaseModels([manager, alternate], version); const cells = releaseCellsFor([manager, alternate], version); expect(cells.filter((cell) => cell.schedule === "primary")).toHaveLength( - 144, + 162, ); expect( cells.filter((cell) => cell.schedule === "environment-reserve"), - ).toHaveLength(34); + ).toHaveLength(40); expect(releaseReviewerModel(version, alternate)).toEqual(alternate); expect(releaseReviewerModel(version)).toBeNull(); expect(releasePolicySha256(version)).toBe( - "sha256:86953cd5da1a0af6a541243abd1d8b92ec5d4c2e0aff26334a5be1d21187c3f6", + "sha256:64a9ab9be09cf3505746ffe0b9c1c1ed1d4746921a34f931784310cf9b9ae445", ); } }); diff --git a/tests/release-1002.test.ts b/tests/release-1002.test.ts new file mode 100644 index 00000000..e2deda28 --- /dev/null +++ b/tests/release-1002.test.ts @@ -0,0 +1,78 @@ +import { expect, test } from "bun:test"; +import { + assertReleaseModels, + releaseCatalog, + releaseCellsFor, + releasePolicySha256, + releaseReviewerModel, +} from "../evals/release-policy.js"; +import type { ModelIdentity } from "../evals/report.js"; +import { releaseScenarios } from "../evals/run.js"; + +const sol: ModelIdentity = { + routeProvider: "openai", + gateway: null, + family: "gpt-6.1-sol", + model: "gpt-6.1-sol", + revision: null, +}; + +test("10.0.2 preserves its historical prefix and requires three status cases", () => { + const catalog = releaseCatalog("10.0.2"); + expect(catalog.slice(0, 17)).toEqual([...releaseCatalog("10.0.1")]); + expect(catalog.slice(17).map((row) => row.caseId)).toEqual([ + "guidance-status-ready", + "guidance-status-blocked", + "guidance-status-error", + ]); + for (const row of catalog.slice(17)) { + expect(row.release).toBe("required"); + expect(row.minScoredAttempts).toBe(3); + expect(row.minPassRate).toBe(1); + expect(row.minProviders).toBe(1); + } + const scenarios = releaseScenarios("10.0.2"); + expect(scenarios.map((row) => row.id)).toEqual( + catalog.map((row) => row.caseId), + ); + const cells = releaseCellsFor([sol], "10.0.2"); + const primary = cells.filter((cell) => cell.schedule === "primary"); + const reserves = cells.filter( + (cell) => cell.schedule === "environment-reserve", + ); + expect(primary).toHaveLength(81); + expect(reserves).toHaveLength(20); + const steps = new Map(scenarios.map((row) => [row.id, row.steps.length])); + const countSteps = (rows: typeof cells) => + rows.reduce((total, row) => total + (steps.get(row.caseId) ?? 0), 0); + expect(countSteps(primary) + 1).toBe(100); + expect(countSteps(primary) + countSteps(reserves) + 1).toBe(126); + for (const cell of cells) expect(cell.reviewerModel).toEqual(sol); +}); + +test("10.0.2 accepts only the approved canonical OpenAI manager/reviewer identity", () => { + expect(() => assertReleaseModels([sol], "10.0.2")).not.toThrow(); + expect(releaseReviewerModel("10.0.2")).toEqual(sol); + for (const forged of [ + { ...sol, routeProvider: "other" }, + { ...sol, model: "other" }, + { ...sol, family: "other" }, + { ...sol, gateway: "proxy" }, + { ...sol, revision: "preview" }, + { ...sol, variant: "fast" }, + ]) + expect(() => assertReleaseModels([forged], "10.0.2")).toThrow(); + for (const models of [[], [sol, sol]]) + expect(() => assertReleaseModels(models, "10.0.2")).toThrow(); + expect(releaseCatalog("10.0.3").map((row) => row.minProviders)).toEqual( + Array(20).fill(2), + ); + expect(() => assertReleaseModels([sol], "10.0.3")).toThrow(); +}); + +test("10.0.2 does not rewrite the qualified 10.0.1 policy", () => { + expect(releaseCatalog("10.0.1")).toHaveLength(17); + expect(releasePolicySha256("10.0.1")).toBe( + "sha256:62eb9715b2d3188a0d944106a8d8c1beb890cfbd72f58bf056f0ea79706bebec", + ); +});