Skip to content

feat(operator): ship end-to-end validation command center - #137

Merged
Nobodyworld merged 13 commits into
mainfrom
feat/operator-validation-command-center
Aug 9, 2026
Merged

feat(operator): ship end-to-end validation command center#137
Nobodyworld merged 13 commits into
mainfrom
feat/operator-validation-command-center

Conversation

@Nobodyworld

@Nobodyworld Nobodyworld commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Summary

Implements issue #136 as a complete operator-visible Validation Broker slice: routing-profile management, exact GitHub pull-request validation requests, explicit lifecycle controls, cheapest-capable local execution or exact evidence reuse, explicit current/stale publication, bounded history, truthful avoided-work metrics, documentation, and strict browser acceptance.

base / merge base: 223df7752716dd6ad35e75ba7613eeb03cfb2887
branch: feat/operator-validation-command-center
current head: ba9d0a8184448a1ae0c30357a18a7b5962dea94d
ahead of main: 13
behind main: 0
state: connector-reviewed; ready for owner merge decision; unmerged

Operator-visible workflow

worker/profile setup
    -> GitHub PR request
    -> exact head resolution
    -> explicit approval and queueing
    -> cheapest-capable fresh execution or worker-local exact reuse
    -> compact current/stale publication
    -> bounded history and avoided-work reporting

Delivered behavior includes:

  • bounded authenticated overview, history, request, worker, route, run, and profile projections;
  • strict additive GitHub request inputs for reuse, routing, integer cost/quota policy, and optional hard worker pin;
  • full policy-bound idempotency plus exact all-default legacy-key recovery;
  • explicit approve, approve-and-queue, queue, cancel, expire, refresh, and publish controls;
  • revision-safe routing-profile create/replace/quota-reset workflow;
  • truthful request identity, configured policy, selected route, quota, run timing, reuse-source, compact evidence, and publication detail;
  • bounded allowlisted worker runtime summary with arbitrary capability-document redaction;
  • deterministic worker activity classification, including normal busy/full workers as capacity constrained;
  • fresh/reused/current/stale history, one selected-request polling loop, bounded pagination, responsive containment, keyboard coverage, and public-safe documentation/screenshot.

Trust and scope boundaries

  • Approval and GitHub publication remain explicit.
  • The GitHub adapter resolves an immutable head and immediately rechecks it before publication.
  • Exact reuse remains same-worker and requires retained local cryptographic proof.
  • Full logs, artifact bytes, commands, argv, local paths, environments, credentials, and arbitrary capability documents remain outside normal APIs and the browser UI.
  • Comparison units are operator-defined local routing values, not money, provider credits, spend, or measured financial savings.
  • No paid provider, coding agent, MCP, browser/RPA worker, repository write, automatic approval/publication, workflow dispatch, merge, release, deployment, or production/public-internet claim is added.
  • Classification remains PUBLIC DEVELOPER PREVIEW — NOT PRODUCTION READY.

End-to-end evidence

A file-backed server-backed acceptance uses the real ExecutionClient, LocalWorker, WorkerConfig, EvidenceStore, mocked GitHub transport, and FastAPI routes with two routed workers:

  • the expensive worker cannot claim either request;
  • fresh run 1 executes all 7 trusted manifest steps and retains ownership/result/log plus all 14 declared artifacts;
  • reused run 2 links run 1 and its immutable evidence fingerprint, performs worker-local verification once, executes zero additional trusted steps, records steps=[], and leaves source evidence unchanged;
  • first publication is current; publication after the mocked head moves is stale while the historical tested SHA remains unchanged;
  • overview/history report 1 fresh success, 1 reused success, 1 deterministic execution avoided, nonzero reference execution time avoided, 3 comparison units avoided, 50% reuse, 1 current publication, 1 stale publication, and 2 history rows;
  • leases and capacity return to zero and the canonical repository remains clean.

Final validation

Exact final head ba9d0a8184448a1ae0c30357a18a7b5962dea94d:

  • focused backend/concurrency/activity matrix: 100 passed;
  • focused worker matrix: 37 passed;
  • dedicated real-worker fresh-then-reuse acceptance: 1 passed;
  • complete pytest: 587 passed, 5 documented platform skips;
  • strict Playwright: 3 passed, zero skips;
  • aggregate coverage: 93%; all 16 configured module thresholds passed;
  • clean Python 3.11 pip check, pre-commit, TODO policy, Ruff, Black, Mypy, compatible-interpreter Bandit, pip-audit, Gitleaks, Lychee, cleanup, public hygiene, and git diff --check passed.

Hosted exact-head evidence:

  • Commitlint 31308384963success;
  • CI 31308384938success;
  • lint, typecheck, test, security, Secrets audit, Link check, Coverage, and Browser UI tests all succeeded.

Connector review

Final connector review 4892165398 found no remaining blocker. The final patch addresses reviews 4890406546 and 4890873592, and no unresolved inline review threads remain.

This PR is ready for an explicit owner merge decision at the exact head above. It must not merge automatically.

Closes #136 only after an explicit owner-authorized squash merge.

@Nobodyworld Nobodyworld left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Connector planning review complete for issue #136. The draft PR is correctly isolated from exact main 223df7752716dd6ad35e75ba7613eeb03cfb2887, changes only the living ExecPlan, and has a fully successful planning matrix. The scope is intentionally one large vertical product slice: operator profile setup, exact-PR request, explicit approval, routed fresh execution or exact reuse, explicit publication, bounded history, truthful avoided-work metrics, strict browser acceptance, screenshot, documentation, and complete validation. No implementation blocker was found in the plan. Keep the PR draft and unmerged throughout implementation; final readiness requires a new connector review of the ultimate implementation head.

Nobodyworld commented Aug 8, 2026

Copy link
Copy Markdown
Owner Author

Codex delivery evidence — final connector review pending corrections

Planning review: 4888524384

Delivered head: 5bb0ec7a696cb512708e3f60e05c2e281b6255b0
Required base and merge base: 223df7752716dd6ad35e75ba7613eeb03cfb2887

Codex reported the implementation, local matrix, hosted matrix, clean worktree, and public-hygiene checks complete for this head:

  • focused backend/adapter/startup: 173 passed; worker: 36 passed;
  • complete pytest: 576 passed, 5 documented platform skips;
  • strict Playwright: 3 passed, zero skips;
  • aggregate coverage: 93%; all 16 configured module gates passed;
  • Commitlint 31279746749: success;
  • CI 31279746751, attempt 2: success;
  • all required hosted jobs succeeded.

Final connector review 4890406546 identified three remaining corrections:

  1. replace the direct-completion fresh/reuse proof with one server-backed proof that actually runs the trusted LocalWorker/ExecutionClient path and worker-local evidence verification;
  2. map an unknown preferred_executor to a bounded API error instead of allowing ExecutionNotFoundError to escape the GitHub router;
  3. complete the selected-request and worker panels with the required route assessment, quota/provenance, duration/source, freshness, and capacity details.

This comment is delivery evidence, not final approval. PR #137 remains draft, open, unmerged, and without auto-merge.

@Nobodyworld Nobodyworld left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Connector review of exact head 5bb0ec7a696cb512708e3f60e05c2e281b6255b0 is complete. Hosted Commitlint 31279746749 and CI 31279746751 are green, and the general architecture, bounded projections, policy-bound idempotency, explicit lifecycle controls, redaction, responsive layout, and documentation are strong. Three blockers remain before readiness:

  1. The required end-to-end proof bypasses the trusted local worker. test_routed_github_validation_is_fresh_then_reused_and_projected directly calls checkout/heartbeat/candidate resolution/completion and constructs both fresh and reused evidence with steps=[]. web/tests/ui_test_app.py uses the same direct-completion shortcut. This does not prove that the fresh request executes a trusted manifest through LocalWorker, that retained evidence is written locally, or that the second request passes worker-local cryptographic verification before skipping steps. Add one file-backed, server-backed test using the real ExecutionClient/LocalWorker harness, mocked GitHub transport, two routed workers/profiles, fresh execution, exact local reuse, current/stale publication, and overview/history assertions. Keep the browser fixture for UI acceptance, but do not treat it as the worker trust proof.

  2. Unknown hard pins can escape as an unhandled execution-domain exception. The new adapter accepts preferred_executor, then ExecutionService.create_work_order can raise ExecutionNotFoundError("preferred_executor_not_found"). The GitHub router catches only GitHubAdapterError, so this operator input can become a 500 instead of a bounded 404/409. Map the execution-domain error at the adapter boundary, roll back, and add an API regression proving no request or work order is created.

  3. The selected-request and worker panels omit required operator state. The UI never calls the existing route-assessment or run-detail surfaces. It therefore does not show queued route assessment, route reason, quota reservation state, candidate count, run duration, reused source run ID/source evidence fingerprint, or request creation/resolution time. Worker cards collapse status/freshness/capacity into one badge and omit the requested heartbeat, active-poll, capacity, and safe platform details. Add the bounded detail reads/rendering and strict browser assertions. Use the locked label Reference execution time avoided consistently.

Also correct the existing PR evidence comment: review 4888524384 was the planning-head review, not the ultimate-head connector review. Keep PR #137 draft and unmerged through the correction, complete local matrix, final hosted matrix, and connector re-review.

Copy link
Copy Markdown
Owner Author

Connector correction delivery

Final connector review 4890406546 is addressed at ultimate head 4f3613e29cc4aca133b815ec935bbf76f6e89ad9.

Corrections

  • Unknown preferred_executor now rolls back and returns bounded HTTP 404 preferred_executor_not_found; the API regression proves zero GitHub validation requests, zero execution work orders, no open transaction, and no internal/transport data in the response.
  • A file-backed server-backed acceptance now uses real ExecutionClient, LocalWorker, WorkerConfig, EvidenceStore, mocked GitHub transport, and FastAPI execution/GitHub routes. Two real routed workers prove the expensive worker cannot claim either request. Fresh run 1 executes all 7 trusted manifest steps and retains ownership/result/log plus 14 declared artifacts. Reused run 2 verifies the immutable source evidence locally, links run 1 and its fingerprint, invokes zero trusted steps, has steps=[], and leaves source evidence unchanged. Publication is current before the mocked head moves and stale afterward; overview/history report 1 fresh, 1 reused, 1 avoided execution, nonzero reference time, 3 comparison units, 50% reuse, 1 current, 1 stale, and 2 rows. Leases/capacity return to zero and repositories/worktree roots are clean.
  • The selected-request surface now shows bounded queued route assessment and terminal route/quota/timing/reuse-source/evidence/publication details. Worker cards show bounded status/activity/platform/freshness/capacity/profile/quota/priority state. The exact metric label is Reference execution time avoided.
  • The public screenshot was recaptured from synthetic offline data and passed visual/public-hygiene inspection.

Corrected-head validation

  • focused backend: 95 passed;
  • focused worker: 37 passed;
  • strict Playwright: 3 passed, zero skips;
  • complete pytest: 578 passed, 5 documented platform skips;
  • coverage: 93% aggregate; all 16 configured module thresholds passed;
  • clean Python 3.11 pip check, pre-commit, TODO policy, Ruff, Black, Mypy (177 files), Bandit 1.8.6/Python 3.11.14, pip-audit, Gitleaks (255 commits), Lychee, diff check, cleanup, and public-hygiene audit passed.

Hosted ultimate head

  • Commitlint 31298956357 — success;
  • CI 31298956352 — success;
  • lint, typecheck, test, security, Secrets audit, Link check, Coverage, and Browser UI tests all succeeded.

Local and remote branch SHAs are equal. PR #137 remains open, draft, unmerged, and without auto-merge.

@Nobodyworld Nobodyworld left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Connector re-review of exact head 4f3613e29cc4aca133b815ec935bbf76f6e89ad9 confirms the three prior blockers from review 4890406546 are addressed: the real ExecutionClient/LocalWorker fresh-then-reuse proof is present, unknown preferred executors return bounded HTTP 404 with rollback, and the route/quota/timing/source plus worker freshness/capacity details are now rendered and covered. Hosted Commitlint 31298956357 and CI 31298956352 are fully green.

One correctness blocker and several directly related accepted-contract omissions remain:

  1. _worker_activity_state returns unavailable for every non-online worker before checking capacity. Authoritative capacity reservation sets a full worker to busy, so a normally full worker is never classified as capacity_constrained; the overview capacity-constrained metric and worker badge can be wrong. Classify busy or active_run_count >= max_concurrency as capacity constrained before the generic unavailable branch, and add projection/API/UI regressions.
  2. The selected-request detail still omits the repository/PR identity and the configured reuse policy, although issue #136 requires those after exact-head resolution. Render the existing bounded request fields.
  3. Worker cards still omit the accepted safe high-level capability summary and the profile quota-reset timestamp. Add a bounded allowlisted capability projection (versions/booleans/bounded browsers; never the arbitrary capability document) and render quota_reset_at truthfully.

Keep PR #137 draft and unmerged. These are narrow projection/UI corrections; no routing, execution, evidence, provider, MCP, RPA, repository-write, or approval/publication redesign is requested.

Copy link
Copy Markdown
Owner Author

Continuation checkpoint — no new correction head

The latest Codex report repeats the prior correction pass at 4f3613e29cc4aca133b815ec935bbf76f6e89ad9; the remote PR head has not moved.

Connector re-review 4890873592 therefore remains authoritative. The remaining work is still the narrow final correction already recorded in the PR body:

  1. classify ordinary fresh busy/full workers as capacity_constrained instead of unavailable, with explicit stale/capacity precedence and projection/API regressions;
  2. render repository, PR number, and configured reuse policy in selected-request detail;
  3. expose only an allowlisted bounded worker capability summary and render routing-profile quota_reset_at;
  4. rerun focused/full/browser/security validation and return with a new exact head.

PR #137 remains open, draft, mergeable, unmerged, and without auto-merge. No connector-side source change or merge was performed.

@Nobodyworld

Nobodyworld commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Final connector correction — review 4890873592

Corrected head: ba9d0a8184448a1ae0c30357a18a7b5962dea94d

  • Worker activity now has deterministic safety-first precedence: unavailable, stale, capacity constrained, active. The persisted six-worker matrix passes in both insertion orders through direct and HTTP projections (9 focused tests; overview totals partition exactly).
  • Selected request detail now exposes repository, PR number, configured reuse/routing policy, and the existing bounded route/run/quota/source evidence.
  • Worker cards expose only allowlisted typed runtime fields plus scheduled/unscheduled quota reset; arbitrary capability documents remain redacted. Scheduled reset timestamps survive profile replacement as timezone-aware API values.
  • The synthetic public screenshot was recaptured and inspected; no token, path, workstation identity, private URL, credential, or financial claim is present.

Local validation:

  • exact focused backend/adapter/startup/routing/concurrency/activity matrix: 100 passed
  • focused worker trust/reuse/strict order: 37 passed
  • dedicated real ExecutionClient/LocalWorker fresh-then-reuse acceptance: 1 passed
  • complete pytest: 587 passed, 5 documented platform skips
  • strict browser: 3 passed, zero skips
  • aggregate configured coverage: 93%; all 16 module thresholds passed
  • pre-commit, TODO policy, Ruff, Black, Mypy, Bandit, pip-audit, Gitleaks, Lychee, diff check, cleanup, and public hygiene: passed

Ultimate-head workflows:

PR remains open, draft, unmerged, and without auto-merge.

@Nobodyworld Nobodyworld left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final connector re-review of exact head ba9d0a8184448a1ae0c30357a18a7b5962dea94d is complete.

The remaining findings from review 4890873592 are addressed:

  • normal fresh busy/full workers now project as capacity_constrained, with deterministic unavailable → stale → capacity-constrained → active precedence and persisted lifecycle/API regressions;
  • selected-request detail now distinguishes repository/PR identity, configured reuse and routing policy, selected route, and terminal reuse decision;
  • worker projections expose only bounded allowlisted runtime fields and preserve arbitrary capability-document redaction;
  • worker cards show profile quota-reset state truthfully;
  • strict browser coverage verifies the corrected request and worker surfaces, conflict recovery, keyboard flow, narrow-width containment, token non-rendering, and zero unexpected console errors.

Exact-head hosted evidence is green: Commitlint 31308384963; CI 31308384938; lint, typecheck, test, security, Secrets audit, Link check, Coverage, and Browser UI tests all succeeded. The branch is 13 commits ahead and 0 behind current main, with merge base 223df7752716dd6ad35e75ba7613eeb03cfb2887. No unresolved review threads remain.

No remaining connector-review blocker was found. PR #137 is ready for owner merge consideration at this exact head. This review does not authorize merge, release, deployment, automatic approval/publication, provider execution, MCP, RPA, repository writes, or production/public-internet claims.

@Nobodyworld
Nobodyworld marked this pull request as ready for review August 9, 2026 18:28
@Nobodyworld
Nobodyworld merged commit 33a5836 into main Aug 9, 2026
9 checks passed
@Nobodyworld
Nobodyworld deleted the feat/operator-validation-command-center branch August 9, 2026 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(operator): ship end-to-end validation command center

1 participant