Repository navigation
Run agents on imported harness adapters - #70
Closed
jacobprall wants to merge 1 commit into
Closed
jacobprall wants to merge 1 commit into
jacobprall wants to merge 1 commit into
Conversation
This was referenced Oct 27, 2026
An agent now names a harness value built by an adapter in app/harness/ from its library's own configuration, instead of a model tier string. The pi adapter takes a typed model from Pi's catalog, so a mistyped model ID fails typecheck. config.models, ModelTier, and modelFor are gone; the built-in choices live in app/agents/models.ts. A harness is the runtime the runner calls in the Sandbox (run and selfTest, app/runner/core/harness.ts) plus what the host needs before a run (a label and the credential env var, app/harness/types.ts). What a harness must do beyond those signatures, such as giving the model only the factory's tools and submit, is pinned by the conformance suite. The runner is split into a library-neutral core (versioned contract, emitter, submit checks, input parsing, keys), shared factory tools with a library-neutral view, and the pi loop. Agent definitions are bundled into the runner, which runs the agent named in its input on that agent's harness. Each Sandbox replaces a snapshot's runner bundle with this deploy's when their hashes differ, before verify.sh, so harness and model changes need no snapshot refresh. Keys come from the harnesses: runAgent, doctor, and the workflows env group derive them from each harness's credential. agent.finished records the model the runner reports. Tests run a conformance suite over the runner contract against the pi harness and a scripted harness with no agent library, and check that the bundle imports no host-only code. Refs render-examples#67
jacobprall
force-pushed
the
harness-refactor
branch
from
October 27, 2026 23:10
c0d0b25 to
85d2bbb
Compare
Collaborator
Author
|
Consolidated into #73. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #67. First of three: the Pydantic AI (#69) and Cursor SDK (#68) adapter PRs build on this one.
An agent now names a harness, a value built by an adapter in
app/harness/from its library's own configuration, instead of a model-tier string. The adapter you import is the runtime, and the model is whatever you gave the library:getBuiltinModelis typed against pi's catalog, so a mistyped model ID failsnpm run typecheck.config.models,ModelTier, andmodelForare gone. Default behavior is unchanged: same models, thinking levels, tools, and outputs.Design
app/runner/now has three parts:core/: the versioned contract, the event emitter with redaction and one final event, thesubmitchecks, input parsing, and keys.tools/: the shared factory tools, plus aFactoryToolview with no library types.pi/: the pi loop.protocol_version;model_provider,model, andthinking_levelare gone.startedreportsharness,harness_version, andmodel, andagent.finishedrecords them.bashapplies the read-only policy. An agent whose harness can't give it its tools fails when it's defined.runner.mjshash with this deploy's bundle beforeverify.shruns, and installs this deploy's bundle, root-owned, when they differ. So harness and model changes need no snapshot refresh.runAgent,doctor, and thesoftware-factory-workflowsenv group derive the key list from the agents' harnesses.Two simplifications from the issue: the input keeps prompt, tools, schema, and limits (no prompt inlining needed), and the host compares file hashes itself instead of sending a
bundle_hash.Tests
submitrule;max_turns;--test-stub.--self-testreports every bundled harness.app/configimport.verify.sh; a matching one is left alone.Verification
npm run checkpasses: 153 tests.live-teston the fork). A Triage run on a real repository replaced the snapshot's stale runner bundle before verification, ran Triage on Sonnet and the spec writer on Opus, and recordedharness,harnessVersion, andmodelonagent.finished.Deploying
Deploy
factory-workflowsafter merging. Existing snapshots get the new bundle when each Sandbox opens.