feat(operator): ship end-to-end validation command center - #137
Conversation
Nobodyworld
left a comment
There was a problem hiding this comment.
Connector planning review complete for issue #136. The draft PR is correctly isolated from exact main 223df7752716dd6ad35e75ba7613eeb03cfb2887, changes only the living ExecPlan, and has a fully successful planning matrix. The scope is intentionally one large vertical product slice: operator profile setup, exact-PR request, explicit approval, routed fresh execution or exact reuse, explicit publication, bounded history, truthful avoided-work metrics, strict browser acceptance, screenshot, documentation, and complete validation. No implementation blocker was found in the plan. Keep the PR draft and unmerged throughout implementation; final readiness requires a new connector review of the ultimate implementation head.
Codex delivery evidence — final connector review pending correctionsPlanning review: Delivered head: Codex reported the implementation, local matrix, hosted matrix, clean worktree, and public-hygiene checks complete for this head:
Final connector review
This comment is delivery evidence, not final approval. PR #137 remains draft, open, unmerged, and without auto-merge. |
Nobodyworld
left a comment
There was a problem hiding this comment.
Connector review of exact head 5bb0ec7a696cb512708e3f60e05c2e281b6255b0 is complete. Hosted Commitlint 31279746749 and CI 31279746751 are green, and the general architecture, bounded projections, policy-bound idempotency, explicit lifecycle controls, redaction, responsive layout, and documentation are strong. Three blockers remain before readiness:
-
The required end-to-end proof bypasses the trusted local worker.
test_routed_github_validation_is_fresh_then_reused_and_projecteddirectly calls checkout/heartbeat/candidate resolution/completion and constructs both fresh and reused evidence withsteps=[].web/tests/ui_test_app.pyuses the same direct-completion shortcut. This does not prove that the fresh request executes a trusted manifest throughLocalWorker, that retained evidence is written locally, or that the second request passes worker-local cryptographic verification before skipping steps. Add one file-backed, server-backed test using the realExecutionClient/LocalWorkerharness, mocked GitHub transport, two routed workers/profiles, fresh execution, exact local reuse, current/stale publication, and overview/history assertions. Keep the browser fixture for UI acceptance, but do not treat it as the worker trust proof. -
Unknown hard pins can escape as an unhandled execution-domain exception. The new adapter accepts
preferred_executor, thenExecutionService.create_work_ordercan raiseExecutionNotFoundError("preferred_executor_not_found"). The GitHub router catches onlyGitHubAdapterError, so this operator input can become a 500 instead of a bounded 404/409. Map the execution-domain error at the adapter boundary, roll back, and add an API regression proving no request or work order is created. -
The selected-request and worker panels omit required operator state. The UI never calls the existing route-assessment or run-detail surfaces. It therefore does not show queued route assessment, route reason, quota reservation state, candidate count, run duration, reused source run ID/source evidence fingerprint, or request creation/resolution time. Worker cards collapse status/freshness/capacity into one badge and omit the requested heartbeat, active-poll, capacity, and safe platform details. Add the bounded detail reads/rendering and strict browser assertions. Use the locked label
Reference execution time avoidedconsistently.
Also correct the existing PR evidence comment: review 4888524384 was the planning-head review, not the ultimate-head connector review. Keep PR #137 draft and unmerged through the correction, complete local matrix, final hosted matrix, and connector re-review.
Connector correction deliveryFinal connector review Corrections
Corrected-head validation
Hosted ultimate head
Local and remote branch SHAs are equal. PR #137 remains open, draft, unmerged, and without auto-merge. |
Nobodyworld
left a comment
There was a problem hiding this comment.
Connector re-review of exact head 4f3613e29cc4aca133b815ec935bbf76f6e89ad9 confirms the three prior blockers from review 4890406546 are addressed: the real ExecutionClient/LocalWorker fresh-then-reuse proof is present, unknown preferred executors return bounded HTTP 404 with rollback, and the route/quota/timing/source plus worker freshness/capacity details are now rendered and covered. Hosted Commitlint 31298956357 and CI 31298956352 are fully green.
One correctness blocker and several directly related accepted-contract omissions remain:
_worker_activity_statereturnsunavailablefor every non-onlineworker before checking capacity. Authoritative capacity reservation sets a full worker tobusy, so a normally full worker is never classified ascapacity_constrained; the overview capacity-constrained metric and worker badge can be wrong. Classifybusyoractive_run_count >= max_concurrencyas capacity constrained before the generic unavailable branch, and add projection/API/UI regressions.- The selected-request detail still omits the repository/PR identity and the configured reuse policy, although issue #136 requires those after exact-head resolution. Render the existing bounded request fields.
- Worker cards still omit the accepted safe high-level capability summary and the profile quota-reset timestamp. Add a bounded allowlisted capability projection (versions/booleans/bounded browsers; never the arbitrary capability document) and render
quota_reset_attruthfully.
Keep PR #137 draft and unmerged. These are narrow projection/UI corrections; no routing, execution, evidence, provider, MCP, RPA, repository-write, or approval/publication redesign is requested.
Continuation checkpoint — no new correction headThe latest Codex report repeats the prior correction pass at Connector re-review
PR #137 remains open, draft, mergeable, unmerged, and without auto-merge. No connector-side source change or merge was performed. |
Final connector correction — review 4890873592Corrected head:
Local validation:
Ultimate-head workflows:
PR remains open, draft, unmerged, and without auto-merge. |
Nobodyworld
left a comment
There was a problem hiding this comment.
Final connector re-review of exact head ba9d0a8184448a1ae0c30357a18a7b5962dea94d is complete.
The remaining findings from review 4890873592 are addressed:
- normal fresh
busy/full workers now project ascapacity_constrained, with deterministic unavailable → stale → capacity-constrained → active precedence and persisted lifecycle/API regressions; - selected-request detail now distinguishes repository/PR identity, configured reuse and routing policy, selected route, and terminal reuse decision;
- worker projections expose only bounded allowlisted runtime fields and preserve arbitrary capability-document redaction;
- worker cards show profile quota-reset state truthfully;
- strict browser coverage verifies the corrected request and worker surfaces, conflict recovery, keyboard flow, narrow-width containment, token non-rendering, and zero unexpected console errors.
Exact-head hosted evidence is green: Commitlint 31308384963; CI 31308384938; lint, typecheck, test, security, Secrets audit, Link check, Coverage, and Browser UI tests all succeeded. The branch is 13 commits ahead and 0 behind current main, with merge base 223df7752716dd6ad35e75ba7613eeb03cfb2887. No unresolved review threads remain.
No remaining connector-review blocker was found. PR #137 is ready for owner merge consideration at this exact head. This review does not authorize merge, release, deployment, automatic approval/publication, provider execution, MCP, RPA, repository writes, or production/public-internet claims.
Summary
Implements issue #136 as a complete operator-visible Validation Broker slice: routing-profile management, exact GitHub pull-request validation requests, explicit lifecycle controls, cheapest-capable local execution or exact evidence reuse, explicit current/stale publication, bounded history, truthful avoided-work metrics, documentation, and strict browser acceptance.
Operator-visible workflow
Delivered behavior includes:
Trust and scope boundaries
PUBLIC DEVELOPER PREVIEW — NOT PRODUCTION READY.End-to-end evidence
A file-backed server-backed acceptance uses the real
ExecutionClient,LocalWorker,WorkerConfig,EvidenceStore, mocked GitHub transport, and FastAPI routes with two routed workers:steps=[], and leaves source evidence unchanged;Final validation
Exact final head
ba9d0a8184448a1ae0c30357a18a7b5962dea94d:pip check, pre-commit, TODO policy, Ruff, Black, Mypy, compatible-interpreter Bandit, pip-audit, Gitleaks, Lychee, cleanup, public hygiene, andgit diff --checkpassed.Hosted exact-head evidence:
31308384963— success;31308384938— success;Connector review
Final connector review
4892165398found no remaining blocker. The final patch addresses reviews4890406546and4890873592, and no unresolved inline review threads remain.This PR is ready for an explicit owner merge decision at the exact head above. It must not merge automatically.
Closes #136 only after an explicit owner-authorized squash merge.