Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,8 @@ coder run develop --repo ./api --input "$(coder artifacts get <research-run-id>)

Internally, the artifact JSON is validated against the downstream workflow's input schema before dispatch. Validation failure is a hard error with a clear message.

See also: [Cross-Prompt Contract Patterns](docs/prompt-contracts.md) for the section heading specifications used to validate artifact outputs.

---

### 6. Agent Skill
Expand Down
77 changes: 77 additions & 0 deletions docs/prompt-contracts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Cross-Prompt Contract Registry

`src/machines/prompt-contracts.js` is the single source of truth for artifact
section headings, producer/consumer file paths, and JSON payload field schemas
shared across develop and research machine prompts.

## Why

Prompt builders in different machines independently define the same section
headings. When a heading is renamed in the producer, consumers silently
reference a stale name, causing the agent to skip sections. The registry
eliminates this by centralizing section names with render helpers that surface
mismatches at test time.

## Access Patterns

Three tiers serve different prompt-assembly needs:

| Pattern | When to use | Example |
|---------|-------------|---------|
| `renderRequiredSections(key)` | Plain numbered list, no inline prose | `plan-review.machine.js` |
| `renderCritiqueSectionList(key, names)` | Prose references; **throws** if name not found | `implementation.machine.js` |
| `CONTRACTS[key].sections` iteration | Interleaved instructional prose between items | `planning.machine.js`, `issue-draft.machine.js` |

### `renderRequiredSections(contractKey)`

Returns a numbered list string:

```
1. Critical Issues (Must Fix)
2. Over-Engineering Concerns
...
```

### `renderCritiqueSectionList(contractKey, sectionNames)`

Returns a comma-joined string of the requested names. Throws an `Error`
(matching `/not found/`) if any name is absent from the registry.

### Direct `CONTRACTS[key].sections` iteration

For prompts that need to insert conditional or multi-line prose after specific
sections (e.g., EARS syntax after "Requirements", Red/Green TDD after "Testing
Strategy"), iterate the sections array and use **name-based checks**:

```js
CONTRACTS["ISSUE.md"].sections.map((s, i) => {
let line = `${i + 1}. **${s.name}**: ${s.description}`;
if (s.name === "Requirements") {
line += ":\n - Ubiquitous: ...";
}
return line;
}).join("\n")
```

## Adding a Section

1. Add the `{ name, description }` entry to the appropriate `CONTRACTS` key in
`src/machines/prompt-contracts.js`.
2. Update the producer machine to emit the new section heading.
3. Update consumer machines that reference sections (the throw-on-miss helper
catches stale references at test time).
4. Run `node ./scripts/run-tests.mjs test/prompt-contracts.test.js` to verify.

## Adding an Artifact

1. Add a new key to `CONTRACTS` with `description`, `producedBy`, `consumers`,
and either `sections` (markdown) or `fields` (JSON).
2. Update the test expectations in `test/prompt-contracts.test.js` (the
`expectedKeys` array).
3. Run the full suite: `npm test`.

## JSON Payload Contracts

Research machines use `fields` entries (e.g., `{ issues: "array" }`) with
`requirePayloadFields()` from `research/_shared.js`. Changing a field name in
the registry automatically updates the validation call site.
58 changes: 32 additions & 26 deletions src/helpers.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { createHash } from "node:crypto";
import { existsSync, readFileSync, writeFileSync } from "node:fs";
import path from "node:path";
import { jsonrepair } from "jsonrepair";
import { CONTRACTS } from "./machines/prompt-contracts.js";
import { DEFAULT_PASS_ENV } from "./pass-env.js";
import { runPpcommitBranch, runPpcommitNative } from "./ppcommit.js";
import { runShellSync } from "./systemd-run.js";
Expand Down Expand Up @@ -679,14 +680,28 @@ export function runPlanreview(repoDir, planPath, critiquePath) {
}

/**
* Run plan review using gemini CLI directly (with native search grounding).
* This alternative to the planreview tool better leverages Gemini's
* ability to search for and verify external API documentation.
* Build the Gemini plan-review prompt from CONTRACTS["PLANREVIEW.md"] sections.
* Exported for testing — ensures Gemini review headings stay in sync with the registry.
*/
export function runPlanreviewWithGemini(repoDir, planPath, critiquePath) {
const planContent = readFileSync(planPath, "utf8");
export function buildGeminiReviewPrompt(planContent) {
const critiqueSections = CONTRACTS["PLANREVIEW.md"].sections;
const sectionMarkdown = critiqueSections
.map((s) => {
if (s.name.startsWith("Verdict")) {
return `### ${s.name}
IMPORTANT: Your verdict line MUST be exactly one of these four options, with no other words:
- REJECT
- REVISE
- PROCEED WITH CAUTION
- APPROVED

Do NOT paraphrase (e.g. do not write "Needs Revision" or "Needs Rework"). Write the exact keyword.`;
}
return `### ${s.name}\n${s.description}.`;
})
.join("\n\n");

const reviewPrompt = `You are a rigorous, experienced senior principal engineer reviewing a technical plan.
return `You are a rigorous, experienced senior principal engineer reviewing a technical plan.

## CRITICAL: Verify External Dependencies

Expand Down Expand Up @@ -740,29 +755,20 @@ ${planContent}

After verifying external APIs via search, provide your critique:

### Critical Issues (Must Fix)
Issues that would cause the plan to fail or violate constraints.

### Over-Engineering Concerns
Unnecessary complexity, abstractions, or scope creep.

### Concerns (Should Address)
Problems that should be addressed but won't cause immediate failure.

### Questions (Need Clarification)
Ambiguities or assumptions that need to be verified.

### Verdict
IMPORTANT: Your verdict line MUST be exactly one of these four options, with no other words:
- REJECT
- REVISE
- PROCEED WITH CAUTION
- APPROVED

Do NOT paraphrase (e.g. do not write "Needs Revision" or "Needs Rework"). Write the exact keyword.
${sectionMarkdown}

Be specific. Reference what you found in your searches about the external APIs.
Reference specific sections in the plan when identifying over-engineering.`;
}

/**
* Run plan review using gemini CLI directly (with native search grounding).
* This alternative to the planreview tool better leverages Gemini's
* ability to search for and verify external API documentation.
*/
export function runPlanreviewWithGemini(repoDir, planPath, critiquePath) {
const planContent = readFileSync(planPath, "utf8");
const reviewPrompt = buildGeminiReviewPrompt(planContent);

// Use gemini CLI with yolo mode and text output
const cmd = heredocPipe(reviewPrompt, "gemini --yolo -o text");
Expand Down
3 changes: 2 additions & 1 deletion src/machines/develop/implementation.machine.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import {
saveState,
} from "../../state/workflow-state.js";
import { defineMachine } from "../_base.js";
import { renderCritiqueSectionList } from "../prompt-contracts.js";
import { makeClaudeSessionId } from "./_session.js";
import {
artifactPaths,
Expand Down Expand Up @@ -130,7 +131,7 @@ Build upon existing correct work. Do not duplicate or revert it.
const implPrompt = `${recoveryContext}Read ${paths.plan} and ${paths.critique}.

## Step 1: Address Critique
Update ${paths.plan} to address any Critical Issues or Over-Engineering Concerns from the critique.
Update ${paths.plan} to address any ${renderCritiqueSectionList("PLANREVIEW.md", ["Critical Issues (Must Fix)", "Over-Engineering Concerns"])} from the critique.
If critique says REJECT, revise the plan significantly before proceeding.
${
useRedGreen
Expand Down
25 changes: 16 additions & 9 deletions src/machines/develop/issue-draft.machine.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import { ScratchpadPersistence } from "../../state/persistence.js";
import { loadState, saveState } from "../../state/workflow-state.js";
import { buildIssueBranchName } from "../../worktrees.js";
import { defineMachine } from "../_base.js";
import { CONTRACTS } from "../prompt-contracts.js";
import {
artifactPaths,
checkArtifactCollisions,
Expand Down Expand Up @@ -359,16 +360,18 @@ Output ONLY markdown suitable for writing directly to ISSUE.md.
If you wrote ISSUE.md to disk via a tool, also output its full contents to stdout.

## Required Sections (in order)
1. **Metadata**: Source, Issue ID, Repo Root (relative path), Difficulty (1-5)
2. **Problem**: What's wrong or missing — reference specific files/functions
3. **Requirements**: Behavioral requirements using EARS Syntax Patterns:
${CONTRACTS["ISSUE.md"].sections
.map((s, i) => {
let line = `${i + 1}. **${s.name}**: ${s.description}`;
if (s.name === "Requirements") {
line += `:
- Ubiquitous: The <system> shall <behavior>.
- Event-driven: WHEN <trigger>, the <system> shall <behavior>.
- State-driven: WHILE <state>, the <system> shall <behavior>.
- Unwanted Behavior: IF <trigger>, THEN the <system> shall <behavior>.
- Optional Feature: WHERE <feature is present>, the <system> shall <behavior>.
4. **Changes**: Exactly which files need to change and how
5. **Testing Strategy**: Search the codebase for existing test files/patterns, then specify:
- Optional Feature: WHERE <feature is present>, the <system> shall <behavior>.`;
} else if (s.name === "Testing Strategy") {
line += `. Search the codebase for existing test files/patterns, then specify:
- **Existing tests**: Which test files cover related behavior (paths + what they test)
- **Test patterns**: The repo's test framework, conventions, assertion style
- **New test cases**: Concrete test cases to write — inputs, expected outputs, edge cases
Expand All @@ -379,9 +382,13 @@ If you wrote ISSUE.md to disk via a tool, also output its full contents to stdou
- Test name, assertion, and expected failure reason (missing function, wrong return value, etc.)
- These form the RED phase — they must fail for the right reasons before implementation begins`
: `- For this low-complexity issue, a lightweight test-after approach is acceptable if a failing-test-first approach isn't practical`
}
6. **Verification**: A concrete shell command or test to prove the fix works (e.g. \`npm test\`, \`node -e "..."\`, \`curl ...\`). This is critical — downstream agents use this to close the feedback loop.
7. **Out of Scope**: What this does NOT include
}`;
} else if (s.name === "Verification") {
line += ` (e.g. \`npm test\`, \`node -e "..."\`, \`curl ...\`). This is critical — downstream agents use this to close the feedback loop.`;
}
return line;
})
.join("\n")}
`;

const res = await agent.execute(issuePrompt, {
Expand Down
16 changes: 5 additions & 11 deletions src/machines/develop/plan-review.machine.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import {
} from "../../helpers.js";
import { loadState, saveState } from "../../state/workflow-state.js";
import { defineMachine } from "../_base.js";
import { renderRequiredSections } from "../prompt-contracts.js";
import { withSessionResume } from "./_session.js";
import {
artifactPaths,
Expand Down Expand Up @@ -78,7 +79,7 @@ function tryWriteCritiqueFromStdout(reviewRes, critiquePath) {
* Fresh-session retry: same task spec as the primary review prompt so the agent
* reads PLAN.md and applies round/constraints — not a generic template critique.
*/
function buildCritiqueRetryPrompt(planPath, critiquePath, round) {
export function buildCritiqueRetryPrompt(planPath, critiquePath, round) {
const roundNote =
round > 0
? `\n\nNote: This is revision round ${round + 1}. The prior plan was rejected. Focus on whether the issues from the prior critique have been addressed.`
Expand All @@ -89,11 +90,8 @@ function buildCritiqueRetryPrompt(planPath, critiquePath, round) {
`1. Read the implementation plan at **${planPath}** in full.\n` +
`2. Write a critical plan critique as markdown to **${critiquePath}**.${roundNote}\n\n` +
`Required sections (in order):\n` +
`1. Critical Issues (Must Fix)\n` +
`2. Over-Engineering Concerns\n` +
`3. Concerns (Should Address)\n` +
`4. Questions (Need Clarification)\n` +
`5. Verdict (REJECT | REVISE | PROCEED WITH CAUTION | APPROVED)\n\n` +
renderRequiredSections("PLANREVIEW.md") +
`\n\n` +
`Constraints:\n` +
`- Do not modify tracked files.\n` +
`- Keep critique concrete with file-level references when possible.\n` +
Expand Down Expand Up @@ -248,11 +246,7 @@ export default defineMachine({
const basePromptBody = `Review ${paths.plan} and write a critical plan critique to ${paths.critique}.${roundNote}

Required sections (in order):
1. Critical Issues (Must Fix)
2. Over-Engineering Concerns
3. Concerns (Should Address)
4. Questions (Need Clarification)
5. Verdict (REJECT | REVISE | PROCEED WITH CAUTION | APPROVED)
${renderRequiredSections("PLANREVIEW.md")}

Constraints:
- Do not modify tracked files.
Expand Down
20 changes: 11 additions & 9 deletions src/machines/develop/planning.machine.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import { z } from "zod";
import { stripAgentNoise } from "../../helpers.js";
import { loadState, saveState } from "../../state/workflow-state.js";
import { defineMachine } from "../_base.js";
import { CONTRACTS } from "../prompt-contracts.js";
import {
executeWithSessionAuthRetry,
makeClaudeSessionId,
Expand Down Expand Up @@ -229,21 +230,22 @@ Select the simplest approach that solves the problem.
## Phase 3: Write Plan to ${paths.plan}

Structure:
1. **Summary**: One paragraph describing what will change
2. **Approach**: Which approach and why (reference existing patterns)
3. **Files to Modify**: List each file with specific changes
4. **Files to Create**: Only if absolutely necessary (prefer modifying existing files)
5. **Dependencies**: Any new dependencies with version and justification
6. **Testing Strategy**:
- Reference the testing strategy from ISSUE.md if present
${CONTRACTS["PLAN.md"].sections
.map((s, i) => {
let line = `${i + 1}. **${s.name}**: ${s.description}`;
if (s.name === "Testing Strategy") {
line += `:\n - Reference the testing strategy from ISSUE.md if present
- List existing test files that validate related behavior
- Describe specific test cases to write (inputs, expected outputs, edge cases)
- Specify the test command to run
- **Red/Green TDD** (when ISSUE.md difficulty >= 3 or change is non-trivial):
- RED phase: List the exact test files to create/modify and the failing assertions to write BEFORE implementation. Each test should target one requirement from ISSUE.md. Describe the expected failure (e.g. "ReferenceError: parseConfig is not defined", "Expected 3 but got undefined").
- GREEN phase: Which files to implement and in what order to make tests pass incrementally.
- If ISSUE.md difficulty < 3 and the change is straightforward, note that test-after is acceptable.
7. **Out of Scope**: Explicitly list what this change does NOT include
- If ISSUE.md difficulty < 3 and the change is straightforward, note that test-after is acceptable.`;
}
return line;
})
.join("\n")}

## Complexity Budget
- Prefer modifying 1-3 files over touching many files
Expand Down
14 changes: 6 additions & 8 deletions src/machines/develop/quality-review.machine.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import {
} from "../../helpers.js";
import { loadState, saveState } from "../../state/workflow-state.js";
import { defineMachine } from "../_base.js";
import { CONTRACTS } from "../prompt-contracts.js";
import {
executeWithSessionAuthRetry,
makeClaudeSessionId,
Expand Down Expand Up @@ -44,20 +45,17 @@ export function parseReviewVerdict(filePath) {
}

export function buildSpecDeltaPrompt(issuePath, planPath) {
const deltaSections = CONTRACTS.SPEC_DELTA.sections
.map((s) => `### ${s.name}\n(${s.description})`)
.join("\n\n");

return `Compare ${issuePath} (original requirements) with ${planPath} (technical plan).

Write a concise "Spec Delta Summary" with these sections:

## Spec Delta Summary

### Additions
(New technical constraints or approaches introduced in the plan)

### Refinements
(Changes in approach from the original issue)

### Omissions
(Requirements deferred or removed in the plan)
${deltaSections}

Keep each section to 2-5 bullets max. Output only the summary, no other text.`;
}
Expand Down
Loading