Repository navigation
[Chore] Show the Zoo Code panel in the Solheim smoke video and bound the apt step - #2195
zoomote2[bot] wants to merge 1 commit into
Conversation
Add the sidebar-runner.cjs extension-tests module so the smoke recording shows the Zoo Code chat panel, hold teardown three seconds after the completion message, and bound the display dependency install at five minutes. No production extension changes.
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one): 1
Please check the settings in the CodeRabbit UI or the
 |
|
1 issue outstanding. See session
Reviewed 760b849 |
Review statusThis PR was opened by an automated account. A human maintainer must verify the change intent, provenance, and validation before merging. Current step: Mark the PR ready. Required CI must pass before CodeRabbit starts. Review-state labels are managed by this workflow; do not edit them manually. |
![zoomote2[bot]](https://avatars.githubusercontent.com/u/278826416?s=60&v=4){kind=small}
| } catch (error) { | ||
| // Focus is best effort for the recording. A rejected run() would end | ||
| // the test host and cut the smoke short, so log and stay pending. | ||
| console.warn("[solheim-smoke] sidebar focus failed:", error && error.message ? error.message : error) |
There was a problem hiding this comment.
This console.warn is never visible anywhere the driver runs: createHostRuntime spawns the host with stdio: "ignore" (scripts/agent-evals/launcher.ts:98), and the isolated child environment strips everything except the display/PATH allowlist, so the module cannot reach SOLHEIM_SMOKE_OUT_DIR either. A focus failure therefore leaves no diagnostic anywhere — the verdict, trace, and uploaded artifacts never record it, and the only symptom would be an empty panel in the video again, which is exactly the failure mode this PR exists to fix. Consider writing a small failure marker the driver can check into the artifacts (for example a file under the inherited TMPDIR that runSmoke copies into verdict.json/the trace), or capturing child stderr into the artifacts dir, so a future focus regression is diagnosable from the uploaded evidence.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Related GitHub Issue
Closes: #2158
Description
The smoke video showed an empty VS Code window. It never showed the Zoo Code panel or the model answer. The IPC start path skips
SidebarProvider.focuson purpose, so the webview never opened. The same run also showed that the apt install step had no time limit. A slow mirror consumed the whole 25-minute job.This change is test and workflow code only. No production extension code changed. The
focusSidebar: falseguard stays.scripts/solheim-smoke/sidebar-runner.cjs. It runs inside the extension host through--extensionTestsPath. It focuseszoo-code.SidebarProvider.focus, waits forisReady(), and keepsrun()pending. VS Code exits whenrun()settles, sorun()must never settle.testsPathtobuildLaunchArgsandcreateHostRuntime. It appends--extensionTestsPath. It is off by default. Only the smoke driver sets it.holdBeforeTeardownMs). Only the smoke sets it.timeout-minutes: 5to the "Install display dependencies" step. A slow apt mirror now fails the job in five minutes.The verdict gates and the trace shape stay the same. The smoke duration grows by the three-second hold on the success path. The
record.shstop budget stays at 30 seconds, which still covers the worst case.Test Procedure
All commands ran from the repository root on this branch:
pnpm agent-evals:check-types— pass.pnpm solheim-smoke:check-types— pass.pnpm test:agent-evals:unit— 47 tests pass. Two new tests cover the teardown hold: one proves the hold runs after the completion message and beforeCloseTask, one proves no hold runs without a completion message.pnpm test:solheim-smoke:unit— 40 tests pass. New coverage provesbuildLaunchArgsadds--extensionTestsPathonly when a tests path is supplied, and the smoke duration includes the hold.pnpm test:solheim-smoke-ci— 25 tests pass. A new policy test asserts the display dependency step hastimeout-minutes: 5.node --check scripts/solheim-smoke/sidebar-runner.cjs— pass.The workflow job runs only from
main, so it cannot run before merge. Issue #2158 records a local run with the real provider: verdictOK, all six gates true, and a 9.2-second video that shows the Zoo Code panel, the prompt, and "Task Completed: Provider smoke completed."Pre-Submission Checklist
Visual Snapshots
Not applicable. This change is CI and test tooling. No webview or extension UI changed. The first post-merge workflow run will publish
smoke.mp4, which will show the Zoo Code chat panel and the model answer.Videos
Not applicable. The relevant recording is the smoke video itself. The workflow produces it only on
main, so it will appear as a workflow artifact after merge.Documentation Updates
The documentation repository does not cover internal smoke tooling. The in-repo READMEs
scripts/solheim-smoke/README.mdandscripts/agent-evals/README.mddescribe the new behavior.Additional Notes
Reviewers must check two points with care. First,
sidebar-runner.cjsmust keeprun()pending forever. A settledrun()exits VS Code and cuts the smoke short. Second, the teardown hold must stay opt-in in the shared controller. The smoke is the only caller that sets it today.Get in Touch
Discord: 0x7777777_