You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The develop and research workflow machines chain markdown-producing prompts: each machine's prompt tells its agent to write artifacts with specific section headings, and the next machine's prompt tells its agent to read/resolve those same headings. Today the section lists are hand-edited string literals in separate files. Nothing enforces that the producer's list and the consumer's list stay in sync, and two heading-based parsers silently fall back to UNKNOWN when a prompt renames a heading.
This issue proposes a single-source prompt contract registry that prompt builders import and interpolate, plus a small enforcement test that runs the hard parsers against synthetic artifacts built from the registry. Drift becomes impossible by construction.
Problem
Concrete drift today
src/machines/develop/plan-review.machine.js:91-101 is the producer:
src/machines/develop/implementation.machine.js:132-147 is the consumer:
## Step 1: Address Critique
Update ${paths.plan} to resolve every finding in the critique —
Critical Issues, Over-Engineering Concerns, Data Structure Review,
Concerns, and Questions.
These are two independent string literals. PR #321 caught (over five review rounds) that Data Structure Review was added as a required section in plan-review without updating Step 1 of implementation, so the agent skipped resolving that section entirely. Nothing in the test suite noticed.
Hard parser contracts
Two parsers key off heading text:
parsePlanVerdict (src/machines/develop/plan-review.machine.js:115) matches /^#{1,6}\s+(?:\d+\.\s+)?Verdict\b[^\n]*/gim against PLANREVIEW.md. Renaming Verdict in the prompt returns UNKNOWN, which downgrades to a fallback inline-verdict regex and then to failing the plan-review machine.
import{CONTRACTS,renderRequiredSections}from"../prompt-contracts.js";constprompt=`...Required sections (in order):${renderRequiredSections(CONTRACTS["PLANREVIEW.md"])}...`;
Consumers (implementation.machine.js, quality-review.machine.js where it reads ISSUE.md/PLAN.md, and any research machines that read upstream artifacts):
import{CONTRACTS,renderCritiqueSectionList}from"../prompt-contracts.js";constprompt=`...## Step 1: Address CritiqueUpdate ${paths.plan} to resolve every finding in the critique —${renderCritiqueSectionList(CONTRACTS["PLANREVIEW.md"])}....`;
For conditional fragments (the difficulty ≥ 3 Red/Green TDD subsection in ISSUE.md, the planExhausted branch in quality-review, active-branches in planning), keep the conditionals inline in the template literal. Only the static section list comes from the registry.
3. test/prompt-contracts.test.js — enforcement
The test is deliberately small because import-interpolation does most of the work:
Registry shape test. Every entry has producedBy, requiredSections, and either consumers or parsedBy (or both). Every producedBy and consumers[i].file resolves to an existing file on disk.
Parser roundtrip test. For each entry with parsedBy, build a synthetic markdown artifact containing the headingMatches heading, call the parser, assert it returns a non-UNKNOWN/non-null value. This catches parser drift against the registry.
Rendered-output sanity. Call renderRequiredSections(entry) and assert the output contains every heading from the registry (a smoke test that the render helpers handle edge cases like empty descriptions or missing suffixes).
No grep-the-file-string check is needed — because the prompt builders import from the registry, the same string literally cannot diverge.
How to add a new section to an existing artifact: update the registry entry → run npm test → renderRequiredSections regenerates the producer prompt, renderCritiqueSectionList regenerates the consumer prompt, the parser roundtrip test catches heading-name changes
How to add a new artifact: register it → wire producer and consumer to import the registry → add parser entry if applicable
A one-line link from DESIGN.md pointing here
Files
New:
src/machines/prompt-contracts.js — the registry + render helpers
test/prompt-contracts.test.js — the enforcement test
docs/prompt-contracts.md — the human-readable guide
Modified (prompt builders):
src/machines/develop/issue-draft.machine.js
src/machines/develop/planning.machine.js
src/machines/develop/plan-review.machine.js (both buildCritiqueRetryPrompt and basePromptBody)
src/machines/develop/implementation.machine.js
src/machines/develop/quality-review.machine.js (producer of REVIEW_FINDINGS.md; consumer of ISSUE.md/PLAN.md)
src/machines/research/issue-synthesis.machine.js
src/machines/research/spec-architect.machine.js
src/machines/research/issue-publish.machine.js (consumer of synthesis output)
Modified (docs):
DESIGN.md — add one-line link to docs/prompt-contracts.md
Scope
In scope:
src/machines/prompt-contracts.js with entries for develop artifacts (ISSUE.md, PLAN.md, PLANREVIEW.md, REVIEW_FINDINGS.md) and research artifacts (issue-backlog JSON, spec-architect JSON, issue-synthesis draft schema)
Render helpers (renderRequiredSections, renderCritiqueSectionList, and any others the prompt builders need) in the same module
Refactor of every prompt builder listed above to import from the registry
test/prompt-contracts.test.js with registry-shape, parser-roundtrip, and render-sanity assertions
docs/prompt-contracts.md
One-line reference from DESIGN.md
Out of scope:
Moving the PLAN.mdTesting Strategy or ISSUE.mdRequirementsprose into the registry. Only static section names + short descriptions move — multi-paragraph prose stays in the template.
Fancy schema validation (zod / JSON-schema) for the registry itself — plain objects are fine, the test enforces shape.
Refactoring parsePlanVerdict / parseReviewVerdict to read the registry at runtime (they still use hardcoded regexes; the registry documents what they parse, the test verifies the match still works).
Adding the registry to the MCP tool surface (no external caller needs it).
Requirements (EARS)
The src/machines/prompt-contracts.js module shall export CONTRACTS with one entry per cross-prompt artifact in the develop and research workflows.
Every prompt builder that today hand-writes a required-sections list shall import section names from the registry via a render helper, so the section list appears exactly once in the codebase.
test/prompt-contracts.test.jsshall fail when a registry entry names a non-existent producer/consumer file.
test/prompt-contracts.test.jsshall fail when parsePlanVerdict or parseReviewVerdict cannot extract a verdict from a synthetic artifact built from the registry headings.
WHEN a contributor adds a new section to CONTRACTS, the producer and consumer prompts shall automatically reflect it on the next build (no further edits required).
docs/prompt-contracts.mdshall describe the update workflow for adding sections and artifacts.
DESIGN.mdshall link to docs/prompt-contracts.md.
Verification
npm test passes (new tests added, existing 806 still green)
npm run lint passes with no new warnings
Manual drift check: temporarily rename Data Structure Review → Data Shapes Review in the registry, run npm test, observe the rendered prompt changes everywhere consistently, and the parser roundtrip still passes (it only tests the Verdict heading); then revert.
Every .machine.js file under src/machines/develop/ and src/machines/research/ that produced or consumed a required-sections list now imports from prompt-contracts.js.
No grep of the codebase outside prompt-contracts.js finds the string "Critical Issues", "Over-Engineering Concerns", "Data Structure Review", etc.
Out of Scope (follow-up candidates)
Migrating hard parsers (parsePlanVerdict, parseReviewVerdict) to derive their regex from the registry at runtime. The current issue keeps them hardcoded but test-verified.
Generating docs/prompt-contracts.md from the registry at build time (keep it hand-written for now).
Reference
The drift pattern was surfaced across 5 rounds of codex_loop_review on PR #321 (feat/pike-rules-in-prompts). Commits 25f0d86..913e883 show each round and the specific contract that drifted.
Summary
The develop and research workflow machines chain markdown-producing prompts: each machine's prompt tells its agent to write artifacts with specific section headings, and the next machine's prompt tells its agent to read/resolve those same headings. Today the section lists are hand-edited string literals in separate files. Nothing enforces that the producer's list and the consumer's list stay in sync, and two heading-based parsers silently fall back to
UNKNOWNwhen a prompt renames a heading.This issue proposes a single-source prompt contract registry that prompt builders import and interpolate, plus a small enforcement test that runs the hard parsers against synthetic artifacts built from the registry. Drift becomes impossible by construction.
Problem
Concrete drift today
src/machines/develop/plan-review.machine.js:91-101is the producer:src/machines/develop/implementation.machine.js:132-147is the consumer:These are two independent string literals. PR #321 caught (over five review rounds) that
Data Structure Reviewwas added as a required section in plan-review without updating Step 1 of implementation, so the agent skipped resolving that section entirely. Nothing in the test suite noticed.Hard parser contracts
Two parsers key off heading text:
parsePlanVerdict(src/machines/develop/plan-review.machine.js:115) matches/^#{1,6}\s+(?:\d+\.\s+)?Verdict\b[^\n]*/gimagainstPLANREVIEW.md. RenamingVerdictin the prompt returnsUNKNOWN, which downgrades to a fallback inline-verdict regex and then to failing the plan-review machine.parseReviewVerdict(src/machines/develop/quality-review.machine.js:35) matches/^##\s+VERDICT:\s+(APPROVED|REVISE)\s*$/gmagainstREVIEW_FINDINGS.md. RenamingVERDICTbreaks the workflow silently —parseReviewVerdictreturns{ verdict: null, findings: content }.Neither drift would be caught by the current test suite.
Research workflow has the same pattern
issue-synthesis.machine.jsproduces a JSON backlog thatissue-publish.machine.jsconsumes.spec-architect.machine.jsproducesdomains/decisions/phasesconsumed byspec-render.machine.js.Proposal
1.
src/machines/prompt-contracts.js— single source of truthNew module that exports one entry per artifact, with section names, descriptions, producer path, consumer paths, and parser references:
The registry is the only place the section names appear. Everything else derives from it.
2. Refactor prompt builders to import and interpolate
Touch every prompt builder that today has a hand-written required-sections list, and replace the literal with a call to the registry.
Producers (
planning.machine.js,plan-review.machine.js,quality-review.machine.js,issue-draft.machine.js,issue-synthesis.machine.js,spec-architect.machine.js,issue-publish.machine.js):Consumers (
implementation.machine.js,quality-review.machine.jswhere it readsISSUE.md/PLAN.md, and any research machines that read upstream artifacts):For conditional fragments (the difficulty ≥ 3 Red/Green TDD subsection in
ISSUE.md, theplanExhaustedbranch in quality-review, active-branches in planning), keep the conditionals inline in the template literal. Only the static section list comes from the registry.3.
test/prompt-contracts.test.js— enforcementThe test is deliberately small because import-interpolation does most of the work:
producedBy,requiredSections, and eitherconsumersorparsedBy(or both). EveryproducedByandconsumers[i].fileresolves to an existing file on disk.parsedBy, build a synthetic markdown artifact containing theheadingMatchesheading, call the parser, assert it returns a non-UNKNOWN/non-nullvalue. This catches parser drift against the registry.renderRequiredSections(entry)and assert the output contains every heading from the registry (a smoke test that the render helpers handle edge cases like empty descriptions or missing suffixes).No grep-the-file-string check is needed — because the prompt builders
importfrom the registry, the same string literally cannot diverge.4.
docs/prompt-contracts.md— human-readableShort doc (< 100 lines) explaining:
npm test→renderRequiredSectionsregenerates the producer prompt,renderCritiqueSectionListregenerates the consumer prompt, the parser roundtrip test catches heading-name changesDESIGN.mdpointing hereFiles
New:
src/machines/prompt-contracts.js— the registry + render helperstest/prompt-contracts.test.js— the enforcement testdocs/prompt-contracts.md— the human-readable guideModified (prompt builders):
src/machines/develop/issue-draft.machine.jssrc/machines/develop/planning.machine.jssrc/machines/develop/plan-review.machine.js(bothbuildCritiqueRetryPromptandbasePromptBody)src/machines/develop/implementation.machine.jssrc/machines/develop/quality-review.machine.js(producer ofREVIEW_FINDINGS.md; consumer ofISSUE.md/PLAN.md)src/machines/research/issue-synthesis.machine.jssrc/machines/research/spec-architect.machine.jssrc/machines/research/issue-publish.machine.js(consumer of synthesis output)Modified (docs):
DESIGN.md— add one-line link todocs/prompt-contracts.mdScope
In scope:
src/machines/prompt-contracts.jswith entries for develop artifacts (ISSUE.md,PLAN.md,PLANREVIEW.md,REVIEW_FINDINGS.md) and research artifacts (issue-backlog JSON, spec-architect JSON, issue-synthesis draft schema)renderRequiredSections,renderCritiqueSectionList, and any others the prompt builders need) in the same moduletest/prompt-contracts.test.jswith registry-shape, parser-roundtrip, and render-sanity assertionsdocs/prompt-contracts.mdDESIGN.mdOut of scope:
PLAN.mdTesting StrategyorISSUE.mdRequirementsprose into the registry. Only static section names + short descriptions move — multi-paragraph prose stays in the template.parsePlanVerdict/parseReviewVerdictto read the registry at runtime (they still use hardcoded regexes; the registry documents what they parse, the test verifies the match still works).Requirements (EARS)
src/machines/prompt-contracts.jsmodule shall exportCONTRACTSwith one entry per cross-prompt artifact in the develop and research workflows.test/prompt-contracts.test.jsshall fail when a registry entry names a non-existent producer/consumer file.test/prompt-contracts.test.jsshall fail whenparsePlanVerdictorparseReviewVerdictcannot extract a verdict from a synthetic artifact built from the registry headings.CONTRACTS, the producer and consumer prompts shall automatically reflect it on the next build (no further edits required).docs/prompt-contracts.mdshall describe the update workflow for adding sections and artifacts.DESIGN.mdshall link todocs/prompt-contracts.md.Verification
npm testpasses (new tests added, existing 806 still green)npm run lintpasses with no new warningsData Structure Review→Data Shapes Reviewin the registry, runnpm test, observe the rendered prompt changes everywhere consistently, and the parser roundtrip still passes (it only tests theVerdictheading); then revert..machine.jsfile undersrc/machines/develop/andsrc/machines/research/that produced or consumed a required-sections list now imports fromprompt-contracts.js.prompt-contracts.jsfinds the string"Critical Issues","Over-Engineering Concerns","Data Structure Review", etc.Out of Scope (follow-up candidates)
parsePlanVerdict,parseReviewVerdict) to derive their regex from the registry at runtime. The current issue keeps them hardcoded but test-verified.docs/prompt-contracts.mdfrom the registry at build time (keep it hand-written for now).Reference
The drift pattern was surfaced across 5 rounds of
codex_loop_reviewon PR #321 (feat/pike-rules-in-prompts). Commits25f0d86..913e883show each round and the specific contract that drifted.