[ai-mountain]

Why

B2 (approved by Peyton; unblocked 2026-09-15 once the rootless router candidate.router.source probe passed, #1919). As deployed today, the contract-v1 certification canary cannot run on the production validator:

• the Compose scorer URL was rejected;
• the pack wasn't in the scorer image;
• a rootless daemon is required, but only rootful DinD exists;
• the exchange URL host didn't match;
• the certify timeout was 30 s.

A post-claim failure would also strand the one-shot lease. This PR is the validator, scorer and infra side. The Platform side (lease expiry, allowlist, attempt cap) is a separate PR.

What

• Scorer origin: one shared scorer_control_origin rule for both the canary and the local shadow supervisor, which admits exactly the Compose origin http://sandbox-docker:8000. One private no-proxy scorer client is shared.
• Pack:
  • baked into the scorer image, read-only;
  • LoadPublicPack verifies grader files against the manifest's grader_files and the visible workspace against a pinned listing digest, and rejects extras and links;
  • .dockerignore and the loader skip exactly __pycache__, *.pyc and .DS_Store.
• Coding host isolation: if the coding host fails to build, it is logged as a refusal and every coding route returns 404. Ordinary scoring keeps working; no more log.Fatalf for the whole scorer.
• Rootless daemon required:
  • DITTOBENCH_CODING_DOCKER_HOST / validator_stack_coding_docker_host must be a unix:// socket;
  • tcp://, /var/run/docker.sock, /run/docker.sock and the scorer's own DOCKER_HOST are refused;
  • every coding Docker call uses that endpoint.
• Readiness before any irreversible step: GET /v1/coding/certifier/canary/readiness (same bearer, no side effects) re-verifies the pack, checks rootless and isolated labels, confirms the runtime image digest is present, and returns the pack digests. The validator refuses to issue unless it's explicitly ready, and aborts rather than claims if the lease's digests differ from the scorer's. The wire format is pinned by a shared contract vector.
• Strict validator targeting:
  • VALIDATOR_CODING_CANARY_AGENT_IDS (1–16 UUIDs) plus VALIDATOR_CODING_CANARY_VALIDATOR_HOTKEY, which must equal the local hotkey; empty refuses all;
  • offers for anything else are dropped; targets are rechecked before issue;
  • a lease for another agent or hotkey is aborted before claim.
• Runtime image repository: Docker repository grammar, identical in Go and Ansible.
• Deadline: the certify budget is the time left on the lease, enforced by asyncio.timeout with short connect/write/pool limits. Every deadline error says "deadline exceeded". Docs state 20-minute leases.
• Exchange URL: platform_coding_validator_api_base_url defaults to https://{{ platform_domain }}, the validator-facing origin. The prod host_vars pin is gone, and the assert requires exactly that origin.
• Rendering: validator_stack renders the default-off canary with two switches, and the role refuses either switch without the rootless socket and exact targets.

Tests

• Python: 353 passed across 12 files, plus 12 Platform exchange tests. ruff and mypy are clean.
• Go: codingexecutor, codingcanary, codinghost, sandbox, codingharness, codinghostedruntime, codinghostedinput, the executor-scorer and the cmd/dittobench-api coding and control-plane tests all pass. -race passes on codingcanary, codinghost and codingexecutor.
• Ansible 2.21.2: the check playbook gives ok=152, rescued=44, failed=0, with each of the 44 reject cases failing at its intended assertion. Syntax checks pass for gcp-validator-prod and gcp-platform-app.
• Pack image stage: built with junk files planted; the junk was excluded and the loader accepted the pinned manifest digest.
• Mutations: all caught (rootful or tcp endpoint allowed, readiness ignored or hard-wired, targets or hotkey dropped, pack binding removed, pre- and post-issue checks removed).

Default-off / not authorized

Both switches default off, and targets default empty (refuse all). No certification run, no production activation, no host change.

Still required before any certification attempt

• Dedicated isolated rootless coding daemon (separate PR):
  • separate from sandbox-docker, labelled io.heyditto.dittobench.isolated=true, with rootless in SecurityOptions;
  • its socket mounted into dittobench-api via DITTOBENCH_CODING_DOCKER_HOST, with egress network and proxy env set;
  • host gateway and port 11438 reachable;
  • runtime image preloaded by digest, and stale-resource cleanup;
  • readiness returns ready: true.
• Platform lease and allowlist PR: merged and verified. A receipt-margin rule from that PR may need a small follow-up here, e.g. the certify budget becomes the lease deadline minus a submission margin.

Questions for Peyton

1. Is non-fatal coding host failure also OK for the shadow gate?
2. The role requires 1–16 targets once the validator switch is on, so it can't be staged in refuse-all mode. Is that OK?
3. Should the validator also check the artifact identity (Platform's allowlist binds it)? Done; see the 2026-09-16 update.
4. after_score re-offers an already certified agent. Should the validator skip it?
5. Is the visible workspace digest acceptable as a Go constant?
6. Should CodingPublicationClient's separate local-origin check be unified?

Update (2026-09-15): certification-only runtime settings (settings leak fix)

Peyton's decision: give certification its own proxy, gateway and no-CA settings.

The leak. The coding host built its Docker client with sandbox.NewLocalDocker(), which reads the ordinary scorer's environment. On Compose that meant the coding harness would inherit:

• DITTOBENCH_SANDBOX_EGRESS_NETWORK=ditto-sandbox;
• the OpenRouter shim CA bundle (DITTOBENCH_OPENROUTER_SHIM_CA_BUNDLE_PATH, mounted and injected as SSL_CERT_FILE/REQUESTS_CA_BUNDLE/CURL_CA_BUNDLE/NODE_EXTRA_CA_CERTS);
• any DITTOBENCH_SANDBOX_EGRESS_PROXY, host gateway, memory/tmpfs/pids/start overrides, seccomp/AppArmor names, GITHUB_TOKEN_FILE and DITTOBENCH_ALLOW_PRIVATE_HARNESS.

Setting a proxy for the coding host would also have injected HTTPS_PROXY into every ordinary scoring container.

Fix.

• sandbox.NewCodingRuntimeDocker builds the coding client only from explicit settings:
  • DITTOBENCH_CODING_EGRESS_NETWORK;
  • DITTOBENCH_CODING_EGRESS_PROXY, exactly http://<private IPv4>:<port>;
  • DITTOBENCH_CODING_HOST_GATEWAY_IP, a private IPv4;
  • optional DITTOBENCH_CODING_SECCOMP_PROFILE and DITTOBENCH_CODING_APPARMOR_PROFILE (unconfined refused).
• It mounts no CA bundle, carries no GitHub token, keeps the private-harness guard on, and fixes the limits (3g memory, 512m tmpfs, 2 CPUs, 512 pids).
• Any missing value refuses the coding gate before side effects, with no fallback to DITTOBENCH_SANDBOX_*. A coding proxy or gateway equal to the sandbox value is also refused.
• Docker CLI calls to a dedicated endpoint (sandbox and codingexecutor) drop inherited DOCKER_* selectors (context, TLS, config, API version) and proxy variables.
• Compose passes the three new variables empty (refuse). validator_stack renders them only behind the scorer canary switch and validates them.

Tests.

• Go: a whole-struct comparison under a poisoned shared environment; harness docker run args contain no CA mount or TLS variables and only the coding proxy, network and gateway; 25 refusal shapes; CLI environment selectors; the cmd no-fallback, equal-to-shared and before-side-effects tests.
• Python: the compose defaults assertion, plus the earlier suites.
• Ansible 2.21.2: validator-stack-coding-canary.yml gives ok=215, rescued=65, failed=0 (21 new reject cases).
• Also resynced with main (the router ledger seam conflicted in the validator entrypoint, config and control routes).

Superseded by #1932 for the canary. #1932 moves certification to a host-level service reached over a Unix socket. Its readiness v2 requires a placement proof the Compose scorer cannot provide, so the Compose canary route here is never ready there.

Head: 746b8d310.

Review update (2026-09-15): independent adversarial review

An independent security and correctness review of the settings leak fix found no leak. Nothing falls back to DITTOBENCH_SANDBOX_*, the coding LocalDocker mounts no CA bundle, and no harness -e or TLS variable reaches the coding container. The proxy and gateway are validated, and a value equal to the sandbox value is refused.

Test defect (fixed). TestDedicatedDockerEndpointInheritsNoDockerOrProxySelector used the production inheritedDockerSelector as its own oracle. As a result, narrowing the stripped set (for example to DOCKER_HOST only, or dropping HTTPS_PROXY) still passed. The test now uses an independent DOCKER_* / *_PROXY oracle. The executor twin also poisons all_proxy and http_proxy, so each dropped variable is mutation-checked.

Head: 9e3c6a59e. CI is green.

Update (2026-09-16): targets bind the artifact and screened-image digests

The validator's targets now match #1923's four-field Platform allowlist: agent_id, artifact_sha256, screened_image_sha256 and validator_hotkey. Before, they bound only the agent and the hotkey, so the validator would have let through a lease for a rebuilt image or a different artifact.

What changed

• Setting. VALIDATOR_CODING_CANARY_TARGETS replaces VALIDATOR_CODING_CANARY_AGENT_IDS. It takes 1 to 16 unique <agent_id>:<artifact_sha256>:<screened_image_sha256> entries: a canonical non-nil UUID and two lowercase 64-hex digests, the same form as the lease authority. VALIDATOR_CODING_CANARY_VALIDATOR_HOTKEY is still the fourth field and must equal the local hotkey. Empty still refuses everything. Agent-only entries, sha256: prefixes, uppercase or short digests, duplicates and more than 16 entries are refused at config parse.
• Worker (CodingCanaryTargets).
  • An offer is queued only if its agent has a target.
  • After issue, binds() requires the lease's agent, hotkey, agent_artifact_sha256 and screened_image_sha256 to be exactly one listed tuple. Otherwise the still-issued lease is aborted before claim.
  • After claim, the claimed authority must keep the issued identity and still bind. It is checked before the harness launch, grant, exchange or certify call.
  • One agent may be listed with two exact image identities, as on Platform. A target for another image never matches.
• validator_stack. validator_stack_coding_canary_targets is a list of mappings with exactly agent_id, artifact_sha256 and screened_image_sha256. It renders as the setting above. Validation checks list shape, 1 to 16 entries, the exact key set, UUID and digest grammar, a non-nil agent and unique tuples.
• Compose and docs are renamed to match.

Tests

• Python: 1574 passed across ditto/tests/validator and the compose, runtime, scorer image, workflow and release-plan guards. ruff and mypy are clean. New cases cover:
  • binds() for each of the four fields, including swapped artifact and image digests;
  • an issued lease with a different artifact, image or both (aborted, never claimed);
  • a claimed lease with a different artifact or image (no launch, grant, exchange, revoke, certify or submit);
  • 18 malformed target values.
• Ansible 2.21.2: tests/validator-stack-coding-canary.yml gives ok=242, rescued=74, failed=0. All 17 target reject cases fail at their intended assertion (checked with -vv), and the complete profile renders the exact target string.

Head: 6eac25140.

Restack

Restacked on main e1a8fa2 (#1937 v13 integration). Clean merge.

🤖 Generated with Claude Code

Note

Make the certification canary runnable and fail closed before a lease

• Adds a readiness probe that checks pack integrity, executor daemon isolation, and runtime image eligibility before the canary issues a lease; the worker aborts any lease whose authority or scorer-pack identity does not match the configured exact target set.
• Routes all coding Docker CLI calls through a dedicated rootless Unix-socket endpoint, separate from the ordinary scorer daemon, with inherited Docker and proxy selectors stripped from the environment.
• Loads and re-verifies the certification pack against pinned visible-workspace and grader-tree digests, rejecting tampered, extra, linked, or oversized files.
• Wires operator inputs through Ansible validation, Compose environment defaults, and a read-only certification-pack Dockerfile stage; all canary and coding-runtime settings default off and empty.
• Risk: codingDockerHostFromEnvironment and sandbox.NewCodingRuntimeDocker reject endpoints shared with the ordinary scorer daemon; a coding-host construction failure now disables coding routes instead of terminating startup, so reviewers should confirm that no existing deployment relies on fatal startup behavior for misconfigured coding settings.

^{Macroscope summarized 2b7df44.}

[github-actions]

No dashboard preview for this commit. This PR resolved to preview profiles stack with no dashboard profile: it changes no apps/platform/dashboard/ path, so there is no dashboard build to publish. Profile selection fails closed rather than guessing, which is why a PR touching no recognized runtime path still reads as stack.

Stack previews are never automatic. A maintainer can provision one for this PR:

gh workflow run preview-stack.yml -f pr=1921                    # provision
gh workflow run preview-stack.yml -f pr=1921 -f action=retire   # free the slot

It runs on the default branch, resolves the head commit from the API, and comments dashboard, Platform, and isolated Backroom URLs once readiness succeeds. profile defaults to auto, which picks stack-copy when the PR touches apps/platform/alembic/. Eight slots cap every active preview; a re-dispatch reuses the slot this PR already holds. Reconciliation keeps it while the PR is open, up to a 24-hour lease cap, and retires it four hours after the PR closes.

Exact SHA: 2b7df4422b10c71036f99a292c0fee75f027ddef

[Peyton-Spencer]

Open Code Review pass completed at 6eac25140b67581be9c437e933dde7f529e4e188 using z-ai/glm-5.3-flash through Ditto’s Code Review endpoint. Coverage: 57/57 added, modified, renamed, or type-changed files in the PR diff.

Model candidates were checked against the source; independently confirmed actionable findings, if any, are posted separately as inline review comments. This was a static review; no PR code or test suite was executed. This comment is not a merge approval.

[Peyton-Spencer]

Open Code Review pass completed at b9f63a4e4d16cdb1083a2723fd73188c7b512dcd using z-ai/glm-5.3-flash through Ditto’s Code Review endpoint. Coverage: 57/57 added, modified, renamed, or type-changed files in the PR diff.

This refresh re-reviewed the changed PR files and carried forward completed coverage for unchanged files. Model candidates were checked against the source; independently confirmed actionable findings, if any, are posted separately as inline review comments. This was a static review; no PR code or test suite was executed. This comment is not a merge approval.

[ai-mountain]

Ready for review. Merge-time note: once #1901's signer test playbook is on main, infra/ansible/tests/coding-hosted-control-signer.yml needs platform_domain: platform.invalid, otherwise the playbook fails with 'platform_domain' is undefined. I push it when #1901 lands. Queue: #1940 (comment)

[ditto-code]

Ditto Review · Beta

Review completed for 2b7df4422b10c71036f99a292c0fee75f027ddef.

No actionable findings in the reviewed files. This is not an approval or proof that the PR is bug-free.

Configure Ditto Review. Inference is billed through your Ditto Router endpoint; targeted verification is included during beta.