Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
769 changes: 769 additions & 0 deletions .agent/execplans/013_operator_validation_command_center.md

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Switchboard is a reference implementation for coordinating multiple agents again
- **Lease-based ownership** — agents claim work with expiry and heartbeat semantics that reduce duplicate execution.
- **Trusted local execution** — approved exact-SHA work orders are claimed by an outbound local worker using fixed reviewed argv, read-only disposable worktrees, bounded output, cancellation, and cleanup.
- **Compact validation evidence** — `validate-switchboard@1` records strict step outcomes, parsed test/coverage/security summaries, dependency-lock hashes, retained artifact hashes, and a deterministic fingerprint without returning full local logs.
- **Validation Broker workspace** — operators can configure local-worker routing, resolve a GitHub pull request to an exact head, approve and queue it, distinguish fresh execution from exact reuse, publish current or stale evidence, and inspect bounded history without assembling API calls by hand.
- **Live state synchronization** — plan changes are broadcast to the dashboard and clients over WebSockets.
- **Live-file hosting** — agents can fetch mutable documents by URL; mutation endpoints can be protected with an admin token.
- **Operational visibility** — health, readiness, diagnostics, metrics hooks, structured logs, and rate limiting.
Expand Down Expand Up @@ -59,6 +60,12 @@ python scripts/run_uvicorn.py

Open [http://localhost:8000/](http://localhost:8000/) to view the operator dashboard.

The **Validation Broker** workspace uses the same optional admin-token boundary as
the execution APIs. Its comparison units are operator-authored routing values,
not currency, provider credits, or measured savings. See the
[command-center operations guide](docs/operations/validation-command-center.md)
for the end-to-end workflow and trust boundaries.

### 3. Create a task

```bash
Expand Down Expand Up @@ -152,6 +159,7 @@ See [SECURITY.md](SECURITY.md) and [docs/configuration.md](docs/configuration.md
- **[Configuration](docs/configuration.md)** — environment variables and runtime settings.
- **[Agent Integration](docs/ai-interface.md)** — how agents interact with Switchboard.
- **[Local Worker Operations](docs/operations/local-worker.md)** — trusted repository mapping, worker configuration, execution, evidence retention, and limitations.
- **[Validation Command Center](docs/operations/validation-command-center.md)** — browser workflow, bounded projections, exact-reuse metrics, and publication controls.
- **[Public Status](docs/reports/status.md)** — current developer-preview posture and release boundaries.
- **[Two-Agent Workflow](docs/visuals/TWO_AGENT_WORKFLOW.md)** — dependency-unlock sequence.
- **[Documentation Index](docs/index.md)** — full navigation.
Expand Down
348 changes: 341 additions & 7 deletions client/python/tests/test_execution_worker_server_smoke.py

Large diffs are not rendered by default.

70 changes: 56 additions & 14 deletions docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Use this page as the concise endpoint index; use [ai-interface.md](ai-interface.
| `/api/execution/routing-profiles` | `GET`, `POST` | List or create privileged operator-owned local-worker cost, quota, and priority profiles. |
| `/api/execution/routing-profiles/{worker_id}` | `GET`, `PUT` | Read or revision-protected replace one worker routing profile. |
| `/api/execution/routing-profiles/{worker_id}/quota-reset` | `POST` | Apply an explicit monotonic, revision-protected quota replacement. |
| `/api/execution/workers` | `POST` | Register or refresh a read-only worker capability declaration. |
| `/api/execution/workers` | `GET`, `POST` | Read a bounded, redacted operator worker/profile projection or register/refresh a read-only worker capability declaration. |
| `/api/execution/workers/{worker_id}/heartbeat` | `POST` | Refresh a registered worker heartbeat and availability state. |
| `/api/execution/checkout` | `POST` | Atomically assign one capability-compatible queued work order to one worker. |
| `/api/execution/runs` | `GET` | List historical execution attempts; filter with `work_order_id`. |
Expand All @@ -44,8 +44,11 @@ Use this page as the concise endpoint index; use [ai-interface.md](ai-interface.
| `/api/execution/runs/{id}/complete` | `POST` | Record `succeeded`, `failed`, `timed_out`, or `cancelled` after ownership validation. |
| `/api/execution/leases/expire` | `POST` | Timeout stale runs, release worker capacity, and safely requeue their work orders. |
| `/api/execution/github/pull-requests/validate` | `POST` | Resolve an allowlisted GitHub PR to one exact head and create or return one normal pending work order. |
| `/api/execution/github/requests` | `GET` | List a bounded, stably ordered adapter projection with repository, lifecycle, reuse, and publication filters. |
| `/api/execution/github/requests/{request_id}` | `GET` | Return bounded adapter identity and execution/publication lifecycle. |
| `/api/execution/github/requests/{request_id}/publish` | `POST` | Recheck the PR head and synchronously create or update one bounded managed comment as current or stale. |
| `/api/execution/operator/overview` | `GET` | Return database-derived request, run, reuse, publication, avoided-work, and worker counts for a bounded day window. |
| `/api/execution/operator/history` | `GET` | Return bounded, paginated, newest-first request/work-order/latest-run history with redacted route and evidence fields. |
| `/api/plan` | `GET` | Return current plan snapshot used by agents and dashboard. |
| `/api/execplans/index` | `GET` | Return ExecPlan registry index in JSON (default) or YAML based on query/header negotiation. |
| `/health/live` | `GET` | Liveness probe returning process and probe observations. |
Expand Down Expand Up @@ -116,7 +119,10 @@ operator `routing_priority`, and finally lexical worker ID. An explicit
`preferred_executor` is a hard pin to a known worker. It overrides ranking but
never approval, liveness, polling, status, capacity, capability, network,
read-only, cost, quota, or profile-enabled checks, and it never falls back to a
different worker when unavailable.
different worker when unavailable. An unknown hard pin submitted through the
GitHub validation route returns bounded `404 preferred_executor_not_found`; the
request transaction is rolled back before either an adapter request or work
order exists.

Every checkout by a known authenticated worker records only that requester's
`last_checkout_poll_at` using server time. Poll freshness and heartbeat
Expand Down Expand Up @@ -170,18 +176,51 @@ retention is not changed.
## GitHub exact-PR adapter

The manual outbound adapter reuses `SWITCHBOARD_ADMIN_TOKEN` authentication.
Its create request accepts only `repository_full_name`, `pull_request_number`,
and a trusted manifest `name`/`version`. Stable GitHub identities, exact head
SHA, base provenance, manifest digest, work-order identity, terminal evidence,
comment identity, and publication state are server-owned. Unknown fields,
including commands, URLs, paths, status, hashes, worker IDs, and comment IDs,
return `422`.

An identical authenticated actor + stable PR + exact head + trusted manifest
request returns the same adapter and work-order identities. A new head or
credential actor creates a distinct request. Stable actor ownership identifiers
remain server-owned and are not returned. The work order remains
`pending_approval` until the normal explicit approval route is called.
Its create request accepts `repository_full_name`, `pull_request_number`, a
trusted manifest `name`/`version`, and optional strict `reuse_policy`,
`routing_policy`, `maximum_cost_units`, `required_quota_units`, and
`preferred_executor` fields. Defaults preserve the original `never` plus
`first_available` behavior. Stable GitHub identities, exact head SHA, base
provenance, manifest digest, work-order identity, terminal evidence, comment
identity, and publication state are server-owned. Unknown or executable-shaped
fields still return `422`.

An identical authenticated actor + stable PR + exact head + trusted manifest +
complete execution-policy request returns the same adapter and work-order
identities. Every accepted execution-policy field participates in the new
idempotency identity. An all-default request also recognizes the exact legacy
pre-command-center identity, so an existing default request is returned without
mutation or duplication; a non-default request never falls back to that legacy
identity. Policy remains authoritative on the linked work order rather than on
adapter-owned schema columns. A new head, credential actor, or material policy
creates a distinct request. The work order remains `pending_approval` until the
normal explicit approval route is called.

The operator projections accept bounded pagination (`limit` at most `100`,
`offset` at most `10000`) and the overview accepts a day window at most `365`.
History has stable newest-first ordering and joins only the latest run per
request. It never returns commands, argv, logs, environment dumps, local paths,
credentials, candidate lists, or complete worker capabilities. Avoided-work
counts include successful reused runs only; reference seconds come from each
linked successful source run's persisted start/finish interval, and comparison
units come from the reused run's persisted route estimate. Missing values are
excluded rather than guessed.

`GET /api/execution/workers` derives one activity label using safety-first
precedence: unavailable, stale, capacity constrained, then active. Its worker
summary contains only safe typed declarations (OS/architecture, Python/Node,
Docker, up to eight bounded browser names, GPU, Unity, desktop automation,
network posture, and the false repository-write capability) plus bounded profile
and freshness state. It never returns the arbitrary capability document. Profile
summaries distinguish a missing quota reset from a scheduled timestamp.

The command center combines the exact request status with existing bounded
`GET /api/execution/work-orders/{id}/route-assessment` and
`GET /api/execution/runs/{id}` reads. The former supplies a non-mutating queued
candidate decision; the latter supplies persisted route/quota provenance,
timestamps, cleanup, reuse source run/fingerprint, and compact evidence after a
run exists. The browser never requests full logs, commands, argv, environment
values, local paths, or unbounded candidate data.

Publication requires terminal compact evidence and re-resolves the PR
immediately before its managed comment is written. A moved or unavailable head
Expand All @@ -201,6 +240,9 @@ See
for credential permissions, marker recovery, local commit availability, and
transport limits.

See [Validation command center](operations/validation-command-center.md) for
the browser workflow and projection semantics.

## Related Docs

- [ai-interface.md](ai-interface.md)
Expand Down
57 changes: 57 additions & 0 deletions docs/architecture/local-execution-broker.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,63 @@ Provider budgets, external remaining-rate-limit ingestion, and paid-agent
routing remain later concerns. Deterministic local validation must not invoke a
paid coding agent.

### Operator validation command center (#136)

The dashboard adds a browser workspace over the accepted execution and GitHub
adapter boundaries; it does not create a second scheduler or execution path.
Lifecycle actions still call the explicit approve, queue, cancel, expire, and
publish routes. Only the selected active request is polled, with one replaceable
timer that is cleared when selection changes or the page unloads.

Server-owned projections assemble the operator view with bounded, stably ordered
queries. The browser does not join unbounded request, work-order, run, worker,
profile, evidence, and publication lists. History selects only the latest run
per adapter request and returns compact scalar provenance. Worker summaries
combine the declared worker with its operator-owned profile without exposing
capability dumps, local roots, or private connectivity data.

The selected-request surface makes two bounded point reads in addition to its
request/history projection: current route assessment while queued and the exact
run after assignment. This exposes candidate count, reason, hard-pin state,
profile revision, quota reservation state, timestamps, duration, cleanup, and
reuse source provenance without broadening list responses or returning worker
logs. Worker cards render explicit declared status, derived activity,
OS/architecture, capacity, heartbeat/poll freshness, and profile state. Derived
activity has deterministic precedence: draining/offline and malformed records
are unavailable; expired heartbeat or poll freshness is stale even when the
worker is busy/full; a remaining busy/full worker is capacity constrained; and
only a fresh valid worker with spare capacity is active. The projection adds
only typed allowlisted Python/Node, Docker, bounded browser, GPU, Unity, desktop,
network, and read-only repository declarations. Arbitrary capability documents
remain redacted. Worker profiles expose the next quota-reset timestamp or an
explicit unscheduled state.

The selected-request view keeps repository, pull-request number, and configured
reuse/routing policy beside the exact tested SHA and bounded route/run evidence.
These are identity fields from the persisted request/work order, not browser-side
guesses or joins.

The browser acceptance uses synthetic completion only to drive visible lifecycle
states. It is not a worker trust proof. A separate file-backed acceptance uses
the real outbound `ExecutionClient` and `LocalWorker`, executes the trusted
manifest on the deterministically selected lower-cost worker, verifies retained
evidence locally on reuse, and proves the step runner is not called a second
time.

Adapter request identity binds every accepted result-affecting execution policy.
For compatibility, an all-default request may resolve the exact pre-#136 legacy
identity; that row and linked work order are returned unchanged. A non-default
request never uses legacy fallback. The linked work order remains the single
authoritative persistence location for reuse, routing, cost, quota, and preferred
executor policy, so no adapter schema migration is required.

Avoided-work metrics are projections, not a mutable savings ledger. One
successful reused run counts as one deterministic execution avoided. Reference
seconds are the non-negative persisted duration of its linked successful source
run, when available. Comparison units are the reused run's persisted route cost,
when available. These units are local routing comparisons only and are not money,
credits, provider usage, or verified spend reduction.

## Execution-Plane Domain

### WorkOrder
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
15 changes: 15 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,14 @@ These values are local operator comparison units only: Switchboard does not
interpret them as currency, credits, spend, savings, billing, or a provider
rate limit. No provider credential or paid-agent configuration is introduced.

The Validation Broker workspace edits the same persisted profiles through the
revision-protected APIs and reads active/stale worker state through a bounded
projection. It does not introduce browser-owned routing configuration. The
browser reads the existing admin token from local storage only when sending an
authenticated request; the token is never returned by a projection or rendered
into the workspace. Overview windows are request parameters bounded from 1
through 365 days and do not mutate worker heartbeat or checkout-poll freshness.

## Outbound GitHub adapter

The manual exact-PR adapter uses server-only environment configuration:
Expand All @@ -69,6 +77,13 @@ configuration surface. See
[GitHub exact pull-request validation](operations/github-exact-pr-validation.md)
for the complete operator and security contract.

Validation requests may add strict per-request reuse and routing policy. These
values are persisted on the linked execution work order, whose schema already
owns the authoritative policy fields. The adapter table therefore needs no new
policy columns or startup migration. Existing all-default adapter identities
remain discoverable through the exact prior idempotency calculation; non-default
requests use the complete policy-bound identity.

## Exact evidence reuse

Evidence reuse has no global environment-variable switch and no
Expand Down
6 changes: 6 additions & 0 deletions docs/operations/local-worker.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,12 @@ See
[GitHub exact pull-request validation](github-exact-pr-validation.md)
for the operator workflow.

The dashboard's [Validation command center](validation-command-center.md)
provides the same request, approval, routing-profile, lifecycle, evidence, and
publication operations through bounded server projections. It does not change
the worker's outbound-only pull loop, trust model, repository mapping, or local
evidence retention rules.

## Runtime behavior and limits

Phase 1 is deliberately single-concurrency. Configuration must set
Expand Down
Loading
Loading