Skip to content

test(core): run Code Mode's executor on plain workerd, and check what core keeps after many runs - #392

Merged
nickruigrok merged 4 commits into
mainfrom
test/code-mode-lifecycle
Oct 11, 2026
Merged

nickruigrok merged 4 commits into
mainfrom
test/code-mode-lifecycle

Conversation

@nickruigrok

@nickruigrok nickruigrok commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Fifth and last PR of the GRA-349 stack, on top of #390. It runs Code Mode's executor on plain workerd (decision D2) and checks what core keeps after many runs.

Retention: what is measured, and what can't be

  • Not measured: what the runtime holds for a run (its isolate, and the RPC sessions of its stubs). workerd offers no signal to read:
    • I counted sessions of the seven code APIs by incrementing a counter in their constructors and decrementing it in [Symbol.dispose]. After 30 calls the counter read 30 opened and 30 still open, and was unchanged 3 s later. The runtime never calls [Symbol.dispose] on these loopback entrypoints.
    • Worker Loader exposes no count of loaded isolates.
    • The stubs core passes have no Symbol.dispose to call.
    • So this PR doesn't claim measured handle retention. The earlier HeldHandles registry is removed: it counted core's own bookkeeping and disposed nothing.
  • Measured: core's own state. After 40 runs over 4 turns (20 completed, 10 threw, 10 failed to load; 30 of them calling an API), Workspace.codeRunRetention() reads 0 open leases and 0 outputs held, with 40 closed leases (bounded by endedRunsKept). The run records hold 20 completed, 10 threw and 10 load_failed.

Plain workerd (D2)

  • vp run smoke:code-mode (scripts/workerd-code-mode.ts) bundles the same runCode core runs into a small Worker (apps/core/test/workerd/code-mode-worker.ts).
    • It bundles with Wrangler's dry run, as smoke:workerd does, so a frozen install is all it needs.
    • It serves the Worker with workerd serve --experimental and a workerLoader binding, with no Miniflare.
    • It retries only a refused connection, within an overall 120 s timeout.
  • These 12 cases end as they do in core:
Case Ends as
complete module (helpers, self === null, ctx with only waitUntil, two parallel capability calls) completed, exactly 2 calls taken
fetch threw
syntax error, non-function export, module that throws while loading (its log kept) load_failed
function that throws threw
module over 200,000 characters refused
promise that never settles wall_expired
60 s wait under a 1 s limit wall_expired, within 5 s
cancelled after 300 ms cancelled
5,000 one-line logs completed, exactly 1,000 lines streamed
flood of output threw, at most 64 KiB, cut with the marker, lines streamed (no more than 1,000), its error ending kept
  • The runtime reports {"runtime":"workerd","cpuLimit":"cpu_limit_unsupported"}.
  • CI runs it in the workerd job, right after smoke:workerd.
  • From a fresh frozen install: I made a new worktree, ran vp install --frozen-lockfile (which leaves no esbuild in the root .bin), then vp run smoke:code-mode. It passed all 11 cases (the line-cap case came after).

Finding for GRA-382 and the on-prem profile. On plain workerd (1.20260921.1), a synchronous busy loop in a loaded Worker holds the thread the host's wall-clock timer needs, so the host never lets go. Until the CPU limit is proven on the deployed profile, on-prem must not offer Code Mode.

Evidence (top of the stack)

  • vp check: passes.
  • Core: 230 passed.
  • Screen-config tests: 47 passed.
  • Web: 12 passed.
  • smoke:code-mode: passes, 12 cases (c2a75f3), and 11 cases from a fresh frozen install.
  • check:migrations: passes.

@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from f5809b6 to daf0cb4 Compare October 11, 2026 03:06
@nickruigrok
nickruigrok force-pushed the test/code-mode-lifecycle branch from 87f19b8 to 541f664 Compare October 11, 2026 03:06
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from daf0cb4 to 74e22af Compare October 11, 2026 03:28
@nickruigrok
nickruigrok force-pushed the test/code-mode-lifecycle branch from 541f664 to 20bd3da Compare October 11, 2026 03:28
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from 74e22af to c7d3cdd Compare October 11, 2026 04:01
@nickruigrok
nickruigrok force-pushed the test/code-mode-lifecycle branch from 20bd3da to fa819d9 Compare October 11, 2026 04:01
@nickruigrok nickruigrok changed the title test(core): measure what Code Mode runs hold, and run its executor on plain workerd test(core): run Code Mode's executor on plain workerd, and check what core keeps after many runs Oct 11, 2026
@nickruigrok
nickruigrok force-pushed the test/code-mode-lifecycle branch from fa819d9 to c2a75f3 Compare October 11, 2026 04:14
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from 2815b1e to b53544a Compare October 11, 2026 11:07
@nickruigrok
nickruigrok force-pushed the test/code-mode-lifecycle branch from c2a75f3 to 77050d1 Compare October 11, 2026 11:07
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from b53544a to bb17abe Compare October 11, 2026 11:32
@nickruigrok
nickruigrok force-pushed the test/code-mode-lifecycle branch from 77050d1 to d90f191 Compare October 11, 2026 11:32
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from bb17abe to 8456366 Compare October 11, 2026 12:04
@nickruigrok
nickruigrok force-pushed the test/code-mode-lifecycle branch from d90f191 to ab962fe Compare October 11, 2026 12:04
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from 8456366 to 80a69d5 Compare October 11, 2026 12:49
@nickruigrok
nickruigrok force-pushed the test/code-mode-lifecycle branch from ab962fe to b01f3ba Compare October 11, 2026 13:09
Base automatically changed from feat/code-mode-output-stream to main October 11, 2026 13:19
… plain workerd

Each Workspace object now holds the entrypoint and env stubs of its code
runs in one measured registry, released when the run ends, and reports
what runs hold. A new smoke run serves the same executor with a Worker
Loader on plain workerd and checks the module contract, its failures,
limits, cancellation, streamed output and the output budget there, and
that the runtime records cpu_limit_unsupported. CI runs it next to the
on-prem smoke run.
…ly what is read

The plain-workerd smoke run bundles its Worker with the Wrangler dry run
the on-prem smoke run uses, so a frozen install is all it needs. It
retries only a refused connection, within an overall timeout, and checks
that a flood keeps its ending and that a run past its time ends at its
limit. The handle registry goes: it counted core bookkeeping and
disposed nothing, and workerd gives no signal of the isolates or RPC
sessions a run leaves (it never calls a loopback entrypoint dispose).
What is measured after many runs is core own state: no open lease and
no output.
…a run may log

A run that logs five thousand lines streams exactly the thousand a run
may log, and a flood never streams more. The header says Wrangler only
bundles the Worker.
@nickruigrok
nickruigrok force-pushed the test/code-mode-lifecycle branch from b01f3ba to 1499b35 Compare October 11, 2026 13:20
@nickruigrok
nickruigrok marked this pull request as ready for review October 11, 2026 13:20
@nickruigrok

Copy link
Copy Markdown
Contributor Author

@greptileai review

@greptile-apps

greptile-apps Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

Adds a plain-workerd smoke test for Code Mode and checks the state core keeps after repeated runs.

  • Code Mode’s executor runs through a smoke check on plain workerd.
  • Core can report how many code-run leases and outputs it keeps.

Reviews (2) · Last reviewed commit: "test(core): check the closed-lease cap, ..." · Reviewed by Greptile

Comment thread apps/core/test/agent.test.ts
Comment thread scripts/workerd-code-mode.ts Outdated
Comment thread scripts/workerd-code-mode.ts Outdated
…ode Mode smoke run

A new test closes more leases than a Workspace object keeps and checks
that the oldest go. The smoke run makes its folder and bundles inside
the scope that removes it and stops workerd, and reports a signal that
ended workerd, whose exit code stays null.
@nickruigrok

Copy link
Copy Markdown
Contributor Author

@greptileai review

@nickruigrok
nickruigrok merged commit 1e33099 into main Oct 11, 2026
9 checks passed
@nickruigrok
nickruigrok deleted the test/code-mode-lifecycle branch October 11, 2026 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant