Skip to content

Add Roder harness through its native app-server - #588

Open
PandelisZ wants to merge 2 commits into
HarnessRouter:mainfrom
PandelisZ:pz/add-roder
Open

PandelisZ wants to merge 2 commits into
HarnessRouter:mainfrom
PandelisZ:pz/add-roder

Conversation

@PandelisZ

@PandelisZ PandelisZ commented Oct 25, 2026 •

Copy link
Copy Markdown

Tracking issue: #587

Summary

Add Roder through its native roder app-server JSON-RPC interface, following Codex's app-server approach. Each turn starts a process, creates or reads a persistent thread, and passes an explicit provider and model.

The change registers Roder in the runner, gateway, console, installer, and custom-harness verification suite. The catalog offers 61 models across GPT, Claude, Gemini, Grok, DeepSeek, Kimi, Qwen, and other families, with gpt-6.1-sol as the default. Availability depends on the selected Responses connection, including OpenRouter, Vercel, and LLMTR.

The adapter maps typed text, reasoning, tool, usage, and terminal events to the canonical stream. It preserves history across restarts, switches models on follow-ups, reports missing history, and enforces disabled native tools. It also loads skills, passes the current AGENTS.md as developer context, and executes attached MCP tools.

Credentials stay behind the relay; checkpoints exclude native auth/config files. Turns use existing cancellation, timeout, and step-budget controls. The installer downloads Roder when missing and verifies it against the release's published SHA256SUMS, following the existing installer pattern.

Verification

  • Runner: 1,196 tests passed. Gateway: 1,388 passed and 2 skipped with the database MCP suite excluded; the local PostgreSQL instance lacks that suite's seeded postgres role. Qoder's existing runner tests ran with its optional SDK installed.
  • Console: TypeScript check, production build, and 97 suites containing 871 tests passed.
  • Released binary: 19 process tests passed on both macOS and Linux against a deterministic Responses endpoint, without model credentials or inference charges. Coverage includes current model IDs reaching the provider unchanged, streaming, follow-up history and model switching, missing sessions, tool disables, provider errors, artifacts, native skills/instructions, MCP calls/unavailable servers, merged-stderr completion handling, and the relay/result collector.
  • Container: built and started with HR_BACKENDS=roder; the installer verified SHA256, and the runner reported Roder available. An HTTP smoke test completed two turns on one thread with a model switch and history intact.

The container build used WITH_DOC_PREVIEW=0 WITH_MEDIA=0 WITH_BUILTIN_SKILLS=0. Upstream built-in-skills tooling exited 134 during the default build. Disabling that optional bundle allowed the build to complete; bundled starter kits remained enabled.

Run the adapter checks:

python -m pytest runner/tests/test_roder_backend.py
python -m pytest gateway/tests/test_roder_catalog.py gateway/tests/test_instruction_file_label.py
HR_RODER_TEST_BIN=/path/to/roder python -m pytest runner/tests/test_roder_process.py

Limits

Live provider-matrix columns, console artifact-rendering parity, the custom-harness scenario, family tour, and benchmarks still need testing. Documentation records these gaps; Codex results do not count as Roder verification.

Roder stores external tools at thread creation. Changing MCP attachments or schemas requires a new session; the adapter rejects changed declarations.

The installer uses Roder's latest release and its published checksums. It reuses existing installations.

@vercel

vercel Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
unified-harness-protocol Ready Ready Preview Oct 29, 2026 3:21am UTC

Request Review

@richard-epsilla richard-epsilla left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this, and for the care in the adapter: the event mapping, the step budget, process-group cleanup, the stderr drain, resume_lost, and keeping the key behind the relay (argv, config and env all checked) are all right. A few things need to change before a new base can ship, most of them about what every self-hosted container would download and offer.

Installer (docker/entrypoint.sh)

  1. It fetches releases/download/latest, and latest is rebuilt and re-uploaded on every push to master. The SHA256SUMS comes from the same release, so it proves the transfer, not which build arrived. Please pin a versioned release (roder/vX.Y.Z, overridable with HR_RODER_VERSION) and commit its expected SHA256 in this repo, the way the other pinned bases are.
  2. The guard is [ ! -x ], so a volume keeps its first download forever. Re-install when roder --version differs from the pin, as mimo, jcode and dcode do.
  3. Please leave roder out of the default HR_BACKENDS until its catalog is measured (below). Today every container that leaves the variable unset would download and run it.
  4. The installer prints Apache-2.0, but upstream is MIT. Please add a NOTICE entry, as dcode has one, and show the pinned version in the console instead of latest.

Catalog
5. A base here offers only the models measured on it, through artifacts, skills, tools, follow-ups and model switches with a real provider. The process tests prove that ids reach the wire unchanged, which is worth having, but no model was run. Please start with gpt-6.1-sol and whatever passes a live five-scenario column. I'm happy to run that column on our test instance once the binary is pinned.
6. The base sends everything through /responses, like dcode. dcode measured that TokenRouter's /responses refuses every Claude id and five Gemini ids, and that the HarnessRouter route answers 404 for eight Gemini ids. Roder needs the same _CHANNEL_REFUSES entries, or users pick those models and fail on the first turn.
7. Please revert the edits to gateway/tests/test_default_models.py and test_muse_wonder.py. Those tests guard what was measured, so a new base cannot join them until it has been.

Driver
8. thread/approvalRequested is a documented wait state. The driver fails fast only on user-input and plan-exit requests, so an approval request would hold the turn until the timeout. Please answer it or end the turn with a clear error.
9. One non-JSON stdout line, or one line over 8 MiB, fails the whole turn. Please skip a line that is not JSON, as the codex path does, and handle a long line without failing.
10. developerContext is not in the documented turn/start request. If Roder also reads AGENTS.md itself, the instructions arrive twice; if it ignores the field, they never arrive. Please confirm which happens and use the documented path.

Docs
11. docs/support-matrix-notes.md has an em dash in "Roder — adapter qualification". docs/roder.md says the installer follows the jcode pattern, but jcode is pinned. "No auto-update runs inside a turn" needs a source or should go.
12. scripts/support-matrix/plugins/run-matrix.py doesn't list roder, so the plugin dimension is never measured.

A new base ships in a minor release, not in the 0.37.x bugfix line now going out. Once 1 to 7 are in, I'll measure it and we can go from there.

This branch was successfully deployed

1 active deployment
Preview — b77776ed Deployed Oct 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants