Skip to content

feat(core): stream Code Mode output to the chat within one 64 KiB budget - #390

Merged
nickruigrok merged 4 commits into
mainfrom
feat/code-mode-output-stream
Oct 11, 2026
Merged

nickruigrok merged 4 commits into
mainfrom
feat/code-mode-output-stream

Conversation

@nickruigrok

@nickruigrok nickruigrok commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Fourth PR of the GRA-349 stack, on top of #389. It adds bounded output and output that streams before the tool completes (section 21.1, step 5).

What changes

  • One output budget: 64 KiB of UTF-8, shared by logs, streamed progress, result and error.
    • 1 KiB is kept for how the run ended, so a flood of logs never hides the error.
    • Cut text ends with an explicit marker: … (cut: over 65536 bytes of output).
    • This replaces the 32 K-character model view.
    • Text is cut with TextEncoder.encodeInto, so a cut never lands inside a character.
  • Streaming:
    • The harness sends each logged line to core's callback in order, one after another, and waits for every line to arrive before a call returns. The code never sees the callback.
    • The callback is handed to verify and to run separately, because an RPC stub only works in the call that brought it.
    • Core checks each line like the run's result (a string, bounded) and drops it once the run has ended.
    • Each line becomes a Pi tool update (onUpdate; Pi ignores updates after the step settles). The turn hands it to the Workspace DO.
    • The DO takes a line only while the run's lease is open. It counts and bounds the lines itself (1,000 lines, 128 KiB of raw text a run), whatever the harness claims.
    • A line is shown only after a read of revocations that began after the line arrived finds none. Those reads happen at most once a second a run and fail closed. So nothing logged after a revocation is ever shown, even if the code makes no further call.
    • The shown text is the whole output redacted at once, then cut at the budget less the reserve, so a credential split across lines is caught. Watchers get it through ChatUpdate.output, which grows incrementally like partial and is left out when unchanged.
    • The output stays until the step's result is stored, so it doesn't flicker away first. A revocation, or the end of the turn, drops it at once.
  • Redaction. Bearer and basic credentials, sk-/pk-/rk- keys, JWTs, PEM private keys and AWS access keys become [redacted]. This applies to every streamed line and to what the model reads and the transcript stores. A run holds no secret of its own; this guards against an outside system's data carrying one.
  • Correlation. The tool result's details now carry the run ID along with the status, tying the transcript to the run's record.
  • Web.
    • ChatView.output holds the running step's output, built with applyOutput.
    • The code step shows an "Output" section while it runs, replaced by its result.
    • Translations are added for nl, de, fr and es.

Evidence

New tests in apps/core/test/agent.test.ts ("chat agent code output"), through the real DO watch, Loader and Pi loop:

  • Streaming:
    • before the step's result message reaches the watcher, it has seen first\nkey [redacted] and then …\nafter the wait;
    • the output goes back to null once the step ends;
    • the model's text is redacted the same way;
    • a line sent through codeRunOutput after the run closed shows nowhere.
  • Budget: a flood of 5,000 lines of two-byte characters followed by a throw:
    • the final text is at most 64 KiB, is cut with the marker and still ends with Error (threw): Error: the end;
    • the streamed output never passes 64 KiB less the 1 KiB reserve.
  • Pure helpers: byte cuts with é and emoji, and the redaction patterns.
  • Updated tests: the existing flood and forged-prototype tests now check the 64 KiB budget and the new marker.

In apps/web/src/chat/follow-chat.test.ts: the output grows, starts afresh for another step, and drops when no step runs.

Commands:

  • vp check: passes.
  • vp test run test/agent.test.ts test/chats.test.ts test/run-fixes.test.ts test/agent-connections.test.ts: 90 passed.
  • vp test run -c vite.screens.config.ts test/agent-builds.test.ts test/agent-apps.test.ts: 32 passed.
  • In apps/web, vp test run src/chat/follow-chat.test.ts src/locales/locales.test.ts: 12 passed.
  • vp build (web, React Compiler): passes.

Restacked on the #387 and #389 review fixes (daf0cb4)

  • The harness's streaming is merged into the hardened harness. The output callback, captured Promise.prototype.then and Promise.resolve, and the in-order delivery are unchanged, and every outcome is still returned rather than thrown.
  • A lease closed by a revocation drops output that no watcher has had yet. The model gets no logs of a revoked run (feat(core): record every Code Mode run, and close its lease on revocation #389, M2).

Review fixes (c7d3cdd)

Must-fix: output kept streaming after a mid-run revocation. A line is now shown only once a read of revocations begun after it arrived finds none. Those reads run at most once a second a run, through the same revokedSince read, and fail closed. A revocation found there:

  • closes the lease as revoked;
  • drops the output, shown or pending;
  • records one agent.run_revoked (method: output);
  • streams nothing more.

Should-fix:

  • Redaction now also covers Stripe sk_/pk_/rk_ live and test keys, GitHub ghp_/gho_/ghs_/ghu_/ghr_/github_pat_, Slack xox[abpr]-, Google AIza, and the value of api_key, apikey, token, secret, password or authorization written as name=value or name: value. The docstring lists exactly what is covered and says it is a net, not a guarantee.
  • The stream redacts like the model view: core keeps each run's lines and redacts all of them as one text.
  • Core counts lines and raw bytes itself, and doesn't trust the harness's totals.

Nits:

  • The output stays until the step's result is stored (no flicker).
  • output is left out of an update when it didn't change (applyOutput keeps what was shown).
  • aria-live is dropped.

New tests:

  • Revocation with no call: the code logs, waits and logs again with no call; the admin revokes in between. No line logged after the revoke ever reaches the chat, and the run ends revoked.
    • Mutation check: with the stream's revocation read disabled, this test fails.
  • Clearing when a call finds the revocation (revokeCodeRun): the shown output goes to null before the step's result.
  • A PEM private key logged line by line never shows its body in any streamed text, and the model reads Logs:\n[redacted]\ndone.
  • 1,500 lines sent straight to codeRunOutput show as 1,000.
  • The redaction table covers the new shapes. The token-shaped fixtures are put together at runtime, so secretlint finds no token in the repository.

Evidence (top of the stack):

  • vp check: passes.
  • Core: 230 passed.
  • Screen-config tests: 47 passed.
  • Web: 12 passed, and the web build passes.

Delta review fix (2815b1e)

Must-fix: quadratic JWT redaction. The pattern started at \b, and a flood like eyJ-eyJ-… has a word boundary at every step. Redacting about 128K characters of that took about 4 s, inside the Workspace object, which re-redacts output about once a second.

  • The pattern now starts at (?<![\w-]). A real JWT still matches.
  • A new test redacts floods of about 128K characters for every credential shape, each with tails of "", "x", "-" and ".". All must finish within 250 ms.
  • With the old pattern, that test fails (13.8 s). With the fix, it passes. No other pattern was slow under the same floods.

Nits:

  • A revocation read that fails is now recorded with its own reason, agent.revocation_read_failed, both by the output check and at the end of a run.
  • New tests for the output check:
    • Failed read: nothing is shown, the run ends revoked, and one agent.revocation_read_failed is recorded.
    • Slow read: the read takes 3 s, longer than the run, so the line is never shown before the step's result, which shows it.

Evidence: vp check passes; vp test run test/agent.test.ts test/chats.test.ts passes (103).

@nickruigrok
nickruigrok force-pushed the feat/code-mode-execution-leases branch from ce8ffa6 to cc14ff5 Compare October 11, 2026 03:06
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from f5809b6 to daf0cb4 Compare October 11, 2026 03:06
@nickruigrok
nickruigrok force-pushed the feat/code-mode-execution-leases branch from cc14ff5 to d14824f Compare October 11, 2026 03:28
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from daf0cb4 to 74e22af Compare October 11, 2026 03:28
@nickruigrok
nickruigrok force-pushed the feat/code-mode-execution-leases branch from d14824f to b04298a Compare October 11, 2026 04:01
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from 74e22af to c7d3cdd Compare October 11, 2026 04:01
@nickruigrok
nickruigrok force-pushed the feat/code-mode-execution-leases branch from b04298a to dbde2ff Compare October 11, 2026 11:07
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from 2815b1e to b53544a Compare October 11, 2026 11:07
@nickruigrok
nickruigrok force-pushed the feat/code-mode-execution-leases branch from dbde2ff to 210f56e Compare October 11, 2026 11:32
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from b53544a to bb17abe Compare October 11, 2026 11:32
@nickruigrok
nickruigrok force-pushed the feat/code-mode-execution-leases branch from 210f56e to d33b9d7 Compare October 11, 2026 11:44
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from bb17abe to 8456366 Compare October 11, 2026 12:04
nickruigrok added a commit that referenced this pull request Oct 11, 2026
…tion (#389)

Third PR of the GRA-349 stack, on top of #387. **Security-relevant:** it
changes how long the agent's code holds authority. It needs a person's
review before merge.

## Threat model (written before the code)
Each row is a way stale or stolen authority could act, and what stops it
now.

| # | Failure or attack | What stops it | Test |
|---|---|---|---|
| T1 | Code keeps running after its run ended (cancelled, past the wall
clock, or detached work after it returned) and calls an API | The lease
closes as the run ends; every call is counted and checked against it
first (`requireOpenRun`) | existing cancel test; new wall-clock write
test |
| T2 | A call admitted while the lease was open writes **after** the
lease closed (a slow write that races the end of the run) | Admitted
again when the write commits. Draft writes, discards and App creates
check the lease **inside the Workspace object's write transaction**.
Commits elsewhere (propose, permission and dependency requests,
connection requests and calls, USER.md saves) call `admitEffect` right
before the effect. | DO-level fence test |
| T3 | A permission of the agent is revoked while code is under way |
Every call and every write admission checks for any revocation of the
agent's permissions since the run opened (one indexed D1 read). The
first one that finds it closes the lease as `revoked`, and the run ends
`revoked` whatever its code returned. Its result isn't handed to the
model, since it may rest on what was revoked. | revocation test |
| T4 | A core restart mid-run leaves a usable lease, or the module runs
again | Leases live only in memory. After a restart the run is unknown
and answers `ended`. The record is marked `interrupted` on wake. Turns
are never replayed, so a follow-up is a new run with a new ID. | restart
test (the object is aborted with `ctx.abort`) |
| T5 | Forged or guessed run IDs, or one chat's run reaching another
chat | The run ID is a random UUID in props the code can't read or
change, and leases are keyed `<chat>/<run>` | existing env/import tests;
unknown-run case |
| T6 | Flooding the audit log by calling after the end | One refusal is
audited per run. The call that finds a revocation audits it, and later
calls don't. | existing 100-call test; revocation test |
| T7 | Erasing what already happened | Effects admitted before the close
keep their receipts (pending actions, proposed versions). Nothing is
rolled back or described as undone. | (by construction) |

Accepted residual: revoking **any** permission of the org agent ends
every run of that agent under way, not only runs that used it. A run
lasts at most 30 s, so this errs safe and costs little. It follows the
posture of allowing and auditing, without per-permission bookkeeping.

## What changes
- **`chat_code_runs`**, a new Workspace DO SQLite table. It is the
chat's existing persistence, not a separate Code Mode store. Each row
holds:
- the run ID, tool call ID, SHA-256 and length of the module (never the
module itself);
- `agent` (`@earendil-works/pi-agent-core@<exact version>`, imported
from the package; pi-ai is pinned to the same version in the catalog)
and the compatibility date;
  - `runtime` and `cpu_limit`;
- `status` (`running`, then the terminal status, or `interrupted`) and
its times.
- Rows are deleted with the chat. The migration is
`0008_chat_code_runs`; it is new, so this is a hard cut.
- **D2.** Only `RUNTIME_PROFILE=cloudflare` records `enforced`. Anything
else, including unset or a typo, records `cpu_limit_unsupported`. Tests,
`vp run dev` and the on-prem smoke config set `workerd`. The CPU proof
is GRA-382's. **Follow-up:** the console must set
`RUNTIME_PROFILE=cloudflare` on Cloudflare deployments. Until it does,
their records say `cpu_limit_unsupported`, which errs safe.
- **Leases.** They now carry `openedAt`. New `revokeCodeRun` and
`admitFromCodeRun` close a lease and check one without counting a call.
Draft writes and discards now name their run: `saveDraft(..., runId)`
and `dropDraft(..., { runId })`. A proposal's follow-through drop goes
by `{ revision }`, because its commit was already admitted.
`takeCreate(..., runId)` also names its run.
- **New error code** `agent.run_revoked`.

## Evidence
New tests in `apps/core/test/agent.test.ts` ("chat agent code run
leases"), through the real Workspace DO, Worker Loader, Pi loop and D1:
- **The record:** call ID, source hash and length, the exact Pi version,
compatibility date, `workerd` with `cpu_limit_unsupported`, and status
`completed`.
- **The DO fence:**
  - a write while the run is open is admitted;
- after `closeCodeRun` (`wall_expired`), a write and a discard are
refused with `agent.run_ended`;
- after `revokeCodeRun`, a write and a create are refused with
`agent.run_revoked`;
  - an unknown run is refused;
  - the revoked run's close returns `revoked`;
- the draft holds only the admitted write, and the records read
`wall_expired` and `revoked`.
- **Revocation mid-run:** the code waits, an admin revokes the agent's
grant, and the next call is refused. The tool result is `Error
(revoked)`, the record says `revoked`, and exactly one
`agent.run_revoked` audit refusal is written.
- **Restart mid-run:** `ctx.abort("restart")` while the code waits. The
turn fails, and on wake the record says `interrupted` and the run's ID
answers `ended`. 6 s later there is still one tool call, no result, one
run record and one model request: nothing ran again.

In `apps/core/test/agent-builds.test.ts`: code that waits 2 s past its 1
s wall-clock limit and then writes the draft is refused once
(`agent.run_ended`), and the draft stays empty.

Commands:
- `vp check`: passes.
- `vp test run test/agent.test.ts test/agent-connections.test.ts
test/chat-connections.test.ts test/agent-instructions.test.ts
test/agent-knowledge.test.ts test/chats.test.ts test/memory.test.ts
test/run-fixes.test.ts test/packages.test.ts test/dependencies.test.ts`:
184 passed.
- `vp test run -c vite.screens.config.ts test/agent-builds.test.ts
test/preview-repairs.test.ts test/app-preview.test.ts
test/agent-apps.test.ts`: 46 passed.



## Security review fixes (cc14ff5)
**M1. A revocation with no later call went unnoticed.** As a run ends,
`#endCodeRun` now reads revocations once more. If it finds one, it
closes the run as `revoked` and records a single `agent.run_revoked`
(`method: executeCode`).

**M2. A revoked run's logs still reached the model.** A revoked run now
hands on neither its result nor its logs. In #390, output that no
watcher has had yet is dropped when the lease is revoked.

**M3. Any revoked row of the org agent ended every run.** `revokedSince`
now counts only revoked permissions with `granted_at IS NOT NULL AND
(chat_id IS NULL OR chat_id = <this chat>)`. A declined request that was
never granted, or a revocation in another chat, leaves the run alone.
The comparison is `revoked_at >= opened_at`.
- I added no skew margin. With a 2 s margin, a run opened just after an
unrelated revoke ended as revoked; the existing held-outcome connection
test showed it. Both timestamps come from Workers clocks.

**S1.** `saveDraft`, `dropDraft({ runId })` and `takeCreate` now run
`admitEffect` first, and keep the lease check inside their own
transaction.

**S2.** `admitEffect` takes three steps in this order:
- a cheap lease read, to get `openedAt` and fail fast;
- the D1 revocation read;
- the lease check, last, right before the effect.

**S3.**
- `connections.call` and `memory.saveUser` record their `admitEffect`
refusals with `admitEffect(env, scope, method)`. Calls wrapped in
`auditedCall` are recorded by it.
- A refusal is recorded only by the call that closed the lease
(`revocationOf` returns `closed` or `closedBefore`), which fixes the
double audit.
- Remaining case: a call admitted at its start, then refused at commit
because another call had already revoked the lease, is recorded by
`auditedCall` too.
- The `admitEffect` in `memory.saveUser` stays. It is the commit-time
check, which `requireOpenRun` at call start doesn't cover.

**S4. New tests:**
- M1: the code calls once and waits, an admin revokes, and the code
returns without another call. The result and logs are withheld, the
status is `revoked`, and one refusal is recorded.
- M3: a declined, never-granted request and another chat's revocation,
stored mid-run, leave the run `completed`.
- `admitEffect` end to end, using a slow outside dependency: every read
of core's files is delayed 2 s against a 1 s run limit.
- A proposal whose run expired while it was checked commits nothing, and
`build.propose` is refused as `agent.run_ended`.
- A draft write let in at once outlives its run while it reads the App's
files, and is refused at commit.
- Mutation checks:
  - an `admitEffect` that does nothing fails the proposal test;
  - dropping the granted and chat filters fails the M3 test;
  - dropping the end-of-run revocation read fails the M1 test.

**S5.** `RUNTIME_PROFILE` now fails closed (see D2 above). There is a
test of how each profile value maps.

**S6.** New migration `0056_permissions_revoked_idx` adds
`permissions(subject_type, subject_id, status, revoked_at)`.

**Nits:**
- `recordSources` refuses a revoked lease with `agent.run_revoked`.
- `interrupted` is a `CodeRunStatus`, and the statuses live in
`code-run-statuses.ts`.
- `chat_code_runs` (migration `0008`, regenerated since it isn't on
main) gains a CHECK on `status`, a CHECK on `tool_call_id` length (1 to
200, cut to fit before insert), and a partial index `WHERE status =
'running'`. A chat keeps its latest 1000 records.
- Not recorded: pi-ai's version, which exports no `package.json` to read
(it is pinned to pi-agent-core's version in the catalog), and the
workerd build version, which the runtime doesn't expose.
- `RUNTIME_PROFILE` is added to the platform's reserved binding names.

**Evidence at the top of the stack:**
- `vp check`: passes.
- `vp test run` on agent, connections, chat-connections, instructions,
knowledge, chats, memory, run-fixes, permissions, packages and
dependencies: 220 passed.
- `vp test run -c vite.screens.config.ts` on agent-builds,
preview-repairs, app-preview and agent-apps: 47 passed.


## Delta review fixes (d14824f)
**Must-fix 1, fail open on T1:** the end-of-run revocation read could
throw before the lease closed. **Must-fix 2, regression:** the lease
stayed open through that D1 round trip after wall expiry or cancel.
- `#endCodeRun` now closes the lease and records the status
synchronously and unconditionally first. Only then does it read
revocations.
- If a revocation is found, the record is revised to `revoked`, one
`agent.run_revoked` is recorded, and the result and logs are withheld.
- A read that **throws** counts as revoked (fail closed). It is logged
as `agent.revocation_read_failed`.

New tests (only the revocation query is faulted, through a database
proxy; everything else is real):
- **A throwing revocation read:** the result and logs are withheld
(`Error (revoked)`), the record says `revoked`, and the lease reads
`ended`.
- **A call during the end-of-run read** (the read is slowed 2 s; the run
ends at its 1 s wall clock while the code waits, then calls): refused
once as `agent.run_ended`, with the run recorded as `wall_expired`.
- Mutation check: against the previous `#endCodeRun`, both tests fail.

**Nits:**
- A comment now states that cross-machine clock skew is the accepted
residual of the margin-free comparison.
- The `admitEffect` doc now says refusals are recorded per refused call.
- A comment ties `granted_at IS NOT NULL` to how permissions become
active: both grant paths set it, so only a request declined before it
was granted lacks it.

Evidence at the top of the stack:
- `vp check`: passes.
- Core test files: 226 passed (11 files).
- Screen-config test files: 47 passed (4 files).
- `smoke:code-mode`: passes.


## Greptile fix (1b7fd82): admission right before the write
`admitEffect` ran before slow preparation, not right before the write,
so work that outlasted the run, or a revocation meanwhile, still let the
write land after the lease closed. I confirmed this with a probe for
dependency resolution.

Each write helper now takes an admission callback and calls it right
before its batch, after any slow preparation. The early check stays as a
cheap refusal. Where the late check runs:

| Helper | Late check runs |
|---|---|
| `resolveDependencies` | after `resolveGraph` and `inspectGraph`,
before `proposeResolved` |
| `proposeDraft` | after `storeTree`, before `auditedBatch` |
| `saveUserMemory` | as `writeVersion`'s `lastCheck` |
| `createApp`, `createFromBlueprint`, `requestPermission`, `askInChat` |
before their batch |

Unchanged:
- `saveDraft`, `dropDraft` and `takeCreate` already check the lease
inside the Workspace transaction.
- `connections.call` is admitted right before `CONNECT.call`.

New tests, each slowing an outside step past a 1 s run limit:

| Slowed step | Result |
|---|---|
| the registry metadata fetch | no dependency request or lock is stored,
and the refusal is recorded as `build.resolveDependencies` |
| the upload of the version's tree | no pending version |
| the knowledge reads | no USER.md is saved, and one `memory.saveUser`
refusal is recorded |

Mutation check: removing the late admission from any one of the three
helpers makes its test fail.

Evidence:
- `vp check`: passes.
- Agent, connections, chat-connections, knowledge, chats, memory,
permissions, dependencies, packages, app-blueprints and apps tests: 236
passed.
- `agent-builds` and `agent-apps` (screen config): 34 passed.
Base automatically changed from feat/code-mode-execution-leases to main October 11, 2026 12:49
What the code logs goes to the chat as it runs, through Pi tool updates,
as the running step output: redacted, tied to its run and step, and
dropped once the run closes, so a late line reopens nothing. Logs,
result and error share one 64 KiB UTF-8 budget, with 1 KiB kept for how
the run ended and an explicit marker where output was cut. The chat
shows the output under the running code step.
A logged line reaches the chat only once a read of revocations begun
after it arrived finds none, at most once a second a run, failing
closed. One found closes the lease as revoked and drops the output.
Core counts and bounds the lines it takes itself, redacts the whole
output so a credential split across lines is caught, covers more
credential shapes, keeps the output until the step result is stored,
and sends output only when it changed.
…le revocations

The JWT pattern started at a word boundary, which a flood like
"eyJ-eyJ-..." offers at every step: seconds of matching on a run full
of output, in the Workspace object, every second. It now starts after
anything but a word character or "-". A revocation read that fails is
recorded with its own reason, agent.revocation_read_failed.
@nickruigrok
nickruigrok force-pushed the feat/code-mode-output-stream branch from 8456366 to 80a69d5 Compare October 11, 2026 12:49
@nickruigrok
nickruigrok marked this pull request as ready for review October 11, 2026 12:49
@nickruigrok

Copy link
Copy Markdown
Contributor Author

@greptileai review

@greptile-apps

greptile-apps Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the latest changes address the previous findings without an established new failure.

Summary

The PR streams Code Mode logs into chat, bounds output by UTF-8 bytes, and redacts credentials before showing or saving results.

  • Code logs appear in the chat before the step finishes.
  • Code output fits one UTF-8 budget and hides credential-shaped text.

Diagram

sequenceDiagram
  participant Code as Code run
  participant Core as Core
  participant Watch as Chat watcher
  participant Page as Chat page
  Code->>Core: Logged line
  Core->>Core: Keep bounded text
  Core->>Core: Read revocations after arrival
  alt Run still allowed
    Core->>Core: Redact and cut whole output
    Core->>Watch: Output change
    Watch->>Page: Add or replace text
  else Revoked or read fails
    Core->>Watch: Clear output
  end
  Core->>Watch: Stored result and cleared output
Loading

Reviews (2) · Last reviewed commit: "fix(core): redact whole credential value..." · Reviewed by Greptile

Comment thread apps/core/src/code-mode.ts Outdated
Comment thread apps/core/src/chat-watch.ts
Comment thread apps/core/src/workspace.ts Outdated
…ch, and cut an oversized line

An assignment of a credential now redacts its whole value: a quoted one
to its closing quote, spaces included, and an unquoted one with any
Bearer or Basic scheme it carries, of any length. A watch first update
always says what output there is, none included, so a page that kept
output from an earlier watch drops it. A line too large for the bytes
left keeps what fits, never half a character, and the cut is shown.
@nickruigrok

Copy link
Copy Markdown
Contributor Author

@greptileai review

@nickruigrok
nickruigrok merged commit aec9572 into main Oct 11, 2026
9 checks passed
@nickruigrok
nickruigrok deleted the feat/code-mode-output-stream branch October 11, 2026 13:19
nickruigrok added a commit that referenced this pull request Oct 11, 2026
… core keeps after many runs (#392)

Fifth and last PR of the GRA-349 stack, on top of #390. It runs Code
Mode's executor on plain workerd (decision D2) and checks what core
keeps after many runs.

## Retention: what is measured, and what can't be
- **Not measured: what the runtime holds for a run** (its isolate, and
the RPC sessions of its stubs). workerd offers no signal to read:
- I counted sessions of the seven code APIs by incrementing a counter in
their constructors and decrementing it in `[Symbol.dispose]`. After 30
calls the counter read 30 opened and 30 still open, and was unchanged 3
s later. The runtime never calls `[Symbol.dispose]` on these loopback
entrypoints.
  - Worker Loader exposes no count of loaded isolates.
  - The stubs core passes have no `Symbol.dispose` to call.
- So this PR **doesn't claim measured handle retention**. The earlier
`HeldHandles` registry is removed: it counted core's own bookkeeping and
disposed nothing.
- **Measured: core's own state.** After 40 runs over 4 turns (20
completed, 10 threw, 10 failed to load; 30 of them calling an API),
`Workspace.codeRunRetention()` reads 0 open leases and 0 outputs held,
with 40 closed leases (bounded by `endedRunsKept`). The run records hold
20 `completed`, 10 `threw` and 10 `load_failed`.

## Plain workerd (D2)
- `vp run smoke:code-mode` (`scripts/workerd-code-mode.ts`) bundles the
same `runCode` core runs into a small Worker
(`apps/core/test/workerd/code-mode-worker.ts`).
- It bundles **with Wrangler's dry run**, as `smoke:workerd` does, so a
frozen install is all it needs.
- It serves the Worker with `workerd serve --experimental` and a
`workerLoader` binding, with no Miniflare.
- It retries only a refused connection, within an overall 120 s timeout.
- These 12 cases end as they do in core:

| Case | Ends as |
|---|---|
| complete module (helpers, `self === null`, `ctx` with only
`waitUntil`, two parallel capability calls) | `completed`, exactly 2
calls taken |
| `fetch` | `threw` |
| syntax error, non-function export, module that throws while loading
(its log kept) | `load_failed` |
| function that throws | `threw` |
| module over 200,000 characters | `refused` |
| promise that never settles | `wall_expired` |
| 60 s wait under a 1 s limit | `wall_expired`, within 5 s |
| cancelled after 300 ms | `cancelled` |
| 5,000 one-line logs | `completed`, exactly 1,000 lines streamed |
| flood of output | `threw`, at most 64 KiB, cut with the marker, lines
streamed (no more than 1,000), its error ending kept |

- The runtime reports
`{"runtime":"workerd","cpuLimit":"cpu_limit_unsupported"}`.
- CI runs it in the `workerd` job, right after `smoke:workerd`.
- **From a fresh frozen install:** I made a new worktree, ran `vp
install --frozen-lockfile` (which leaves no `esbuild` in the root
`.bin`), then `vp run smoke:code-mode`. It passed all 11 cases (the
line-cap case came after).

**Finding for GRA-382 and the on-prem profile.** On plain workerd
(1.20260921.1), a synchronous busy loop in a loaded Worker holds the
thread the host's wall-clock timer needs, so the host never lets go.
Until the CPU limit is proven on the deployed profile, on-prem must not
offer Code Mode.

## Evidence (top of the stack)
- `vp check`: passes.
- Core: 230 passed.
- Screen-config tests: 47 passed.
- Web: 12 passed.
- `smoke:code-mode`: passes, 12 cases (c2a75f3), and 11 cases from a
fresh frozen install.
- `check:migrations`: passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant