Skip to content

Run agents on harness adapters, with pi on a run kernel - #73

Merged
jacobprall merged 1 commit into
render-examples:mainfrom
jacobprall:harness-adapters
Oct 28, 2026
Merged

jacobprall merged 1 commit into
render-examples:mainfrom
jacobprall:harness-adapters

Conversation

@jacobprall

@jacobprall jacobprall commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #67. Replaces #70 with one change.

The Cursor SDK (#68) and Pydantic AI (#69) adapters are no longer in this PR. They're kept on the harness-cursor and harness-pydantic branches, still on the earlier runtime interface. Each ships once it runs as a driver on the kernel below.

An agent now names a harness: a value an adapter in app/harness/ builds from its library's own configuration, instead of a model-tier string. The model is whatever you gave the library:

pi({ model: getBuiltinModel("anthropic", "claude-opus-5") }) // typed: a mistyped model fails typecheck

config.models, ModelTier, and modelFor are gone. The built-in choices live in app/agents/models.ts as piSonnet and piOpus, each named for its library and model, and the defaults are unchanged: Triage and PR Reviewer on Sonnet, the spec writer and implementer on Opus.

The run kernel

  • The kernel owns each run (app/runner/core/kernel.ts):
    • It emits started, then exactly one result or error, and picks the exit code.
    • It enforces max_turns, and timeout_ms minus its safety margin, and handles cancellation.
    • It builds the agent's factory tools and submit.
  • The first outcome is final. That outcome is an accepted submission, the last rejected submission, the turn limit, the timeout, or cancellation. The driver then gets 5 seconds to stop before the outcome is reported anyway.
  • A driver only translates its library (app/runner/core/driver.ts):
    • open(ctx) reports the library version and the exact model.
    • run() offers the model ctx.tools and ctx.submit, and calls ctx.requestModel() before every model request.
    • It reports what the library observed through ctx.report, and stops when ctx.signal aborts.
    • A driver never sees the emitter, the submission, or exit codes. The kernel drops a tool result that was never announced, and anything reported after the outcome.
  • max_turns counts model requests. pi's requests pass the kernel's check in its streamFn, so retries of a transient model error count, and a retry past the limit is never sent.
  • No hidden reminder turn. A model that stops without calling submit ends with no_submission.
  • Production code has no test mode. Tests drive the kernel with pi's fake provider and a scripted driver with no library at all.

Runner and Sandbox

  • Runner layout:
    • core/: the versioned contract (protocol 2), kernel, driver interface, emitter, submit checks, input parsing, and keys.
    • tools/: the shared factory tools, with a library-neutral view.
    • pi/: the pi driver and its tool bindings.
  • The bundle: agent definitions are bundled into the runner, which runs the agent named in its input on that agent's harness. runner.mjs --self-test proves the bundle and every agent definition load.
  • Payload: when a Sandbox opens and its runner bundle's hash differs from this deploy's, the bundle is swapped in, root-owned, before verify.sh. Harness, model, and agent changes need no snapshot refresh.

Also

  • Credentials: keys come from the harnesses. runAgent, doctor, and the software-factory-workflows env group derive them.
  • Telemetry: agent.finished records the harness, its version, and the model.
  • Emitter fix: createEmitter's terminated getter was copied as a value by Object.assign, so it always read false. It now reflects the stream.

Verification

  • npm run check: 167 tests pass, including:
    • 13 kernel unit tests on outcomes, limits, cancellation, the stop grace, tools, and reporting;
    • 5 pi driver tests on no reminder, retries counting as turns, a batch that submits, and an unknown model; and
    • the runner conformance suite on pi and the scripted driver.
  • The tests catch a real bug: loosening the kernel's turn limit by one fails both the kernel and pi tests.
  • On a test deployment, with this commit plus a demo-target config change:
    • One autopilot run on the kernel (jacobprall/software-factory-demo#17) ran all four agents on pi. Triage, the spec writer, the implementer, and PR Reviewer each ended ok, in 2, 2, 4, and 2 model requests, for about $0.13 in total. It opened draft PR #18 with exactly the requested file and posted a COMMENT review. The Workflows service logged no errors.
    • The run's first Sandbox swapped in the new runner bundle, with no snapshot refresh.
    • Timeouts, the turn limit, and rejected submissions are covered only by the tests above.
    • Earlier revisions on the same deployment covered pi-only Triage, a stale-payload swap, and a fresh-snapshot open.

Deploying

Deploy factory-workflows. Existing snapshots get the new runner bundle when each Sandbox opens.

@jacobprall jacobprall changed the title Run agents on harness adapters for pi, Pydantic AI, and the Cursor SDK Run agents on harness adapters, with pi on a run kernel Oct 28, 2026
An agent now names a harness: a value an adapter in app/harness/ builds
from its library's own configuration, instead of a model tier string.
The model is whatever you gave the library. config.models, ModelTier,
and modelFor are gone; the built-in choices live in app/agents/models.ts.

  pi({ model: getBuiltinModel("anthropic", "claude-opus-5") })

The runner's kernel (app/runner/core/kernel.ts) owns each run: it emits
started and exactly one result or error, picks the exit code, enforces
max_turns and timeout_ms, handles cancellation, builds the agent's tools
and submit, and keeps the first outcome reached. A harness supplies a
driver (app/runner/core/driver.ts) that only translates its library: it
binds the tools and submit, calls requestModel() before every model
request, reports what the library observed, and stops when the run ends.
A driver never sees the emitter, the submission, or exit codes.

pi is the only driver. Every provider request, retries included, passes
the kernel's turn check, so max_turns counts model requests. A model
that stops without submitting ends with no_submission; there is no
hidden reminder turn. Production code has no test mode: tests drive the
kernel with pi's fake provider and a scripted driver.

Runner: a library-neutral core (versioned contract, kernel, emitter,
submit checks, input parsing, keys), shared factory tools, and the pi
driver. Agent definitions are bundled into the runner, which runs the
agent named in its input on that agent's harness. runner.mjs --self-test
proves the bundle and every agent definition load.

Sandbox: each Sandbox swaps in this deploy's runner bundle, root-owned,
before verify.sh when its hash differs, so harness, model, and agent
changes need no snapshot refresh.

Keys come from the harnesses' credentials: runAgent, doctor, and the
workflows env group derive them. agent.finished records the harness, its
version, and the model the runner reports.

createEmitter's terminated getter was copied as a value by Object.assign
and always read false; it now reflects the stream.

The Cursor SDK and Pydantic AI adapters (render-examples#68, render-examples#69) are kept on the
harness-cursor and harness-pydantic branches until they move to drivers.

Fixes render-examples#67

Co-authored-by: Cursor <cursoragent@cursor.com>
@jacobprall
jacobprall merged commit b77f537 into render-examples:main Oct 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Refactor agents around imported harness adapters

1 participant