Repository navigation
feat(core): stream Code Mode output to the chat within one 64 KiB budget - #390
Merged
Merged
Conversation
nickruigrok
force-pushed
the
feat/code-mode-execution-leases
branch
from
October 11, 2026 03:06
ce8ffa6 to
cc14ff5
Compare
nickruigrok
force-pushed
the
feat/code-mode-output-stream
branch
from
October 11, 2026 03:06
f5809b6 to
daf0cb4
Compare
nickruigrok
force-pushed
the
feat/code-mode-execution-leases
branch
from
October 11, 2026 03:28
cc14ff5 to
d14824f
Compare
nickruigrok
force-pushed
the
feat/code-mode-output-stream
branch
from
October 11, 2026 03:28
daf0cb4 to
74e22af
Compare
nickruigrok
force-pushed
the
feat/code-mode-execution-leases
branch
from
October 11, 2026 04:01
d14824f to
b04298a
Compare
nickruigrok
force-pushed
the
feat/code-mode-output-stream
branch
from
October 11, 2026 04:01
74e22af to
c7d3cdd
Compare
nickruigrok
force-pushed
the
feat/code-mode-execution-leases
branch
from
October 11, 2026 11:07
b04298a to
dbde2ff
Compare
nickruigrok
force-pushed
the
feat/code-mode-output-stream
branch
from
October 11, 2026 11:07
2815b1e to
b53544a
Compare
nickruigrok
force-pushed
the
feat/code-mode-execution-leases
branch
from
October 11, 2026 11:32
dbde2ff to
210f56e
Compare
nickruigrok
force-pushed
the
feat/code-mode-output-stream
branch
from
October 11, 2026 11:32
b53544a to
bb17abe
Compare
nickruigrok
force-pushed
the
feat/code-mode-execution-leases
branch
from
October 11, 2026 11:44
210f56e to
d33b9d7
Compare
nickruigrok
force-pushed
the
feat/code-mode-output-stream
branch
from
October 11, 2026 12:04
bb17abe to
8456366
Compare
nickruigrok
added a commit
that referenced
this pull request
Oct 11, 2026
…tion (#389) Third PR of the GRA-349 stack, on top of #387. **Security-relevant:** it changes how long the agent's code holds authority. It needs a person's review before merge. ## Threat model (written before the code) Each row is a way stale or stolen authority could act, and what stops it now. | # | Failure or attack | What stops it | Test | |---|---|---|---| | T1 | Code keeps running after its run ended (cancelled, past the wall clock, or detached work after it returned) and calls an API | The lease closes as the run ends; every call is counted and checked against it first (`requireOpenRun`) | existing cancel test; new wall-clock write test | | T2 | A call admitted while the lease was open writes **after** the lease closed (a slow write that races the end of the run) | Admitted again when the write commits. Draft writes, discards and App creates check the lease **inside the Workspace object's write transaction**. Commits elsewhere (propose, permission and dependency requests, connection requests and calls, USER.md saves) call `admitEffect` right before the effect. | DO-level fence test | | T3 | A permission of the agent is revoked while code is under way | Every call and every write admission checks for any revocation of the agent's permissions since the run opened (one indexed D1 read). The first one that finds it closes the lease as `revoked`, and the run ends `revoked` whatever its code returned. Its result isn't handed to the model, since it may rest on what was revoked. | revocation test | | T4 | A core restart mid-run leaves a usable lease, or the module runs again | Leases live only in memory. After a restart the run is unknown and answers `ended`. The record is marked `interrupted` on wake. Turns are never replayed, so a follow-up is a new run with a new ID. | restart test (the object is aborted with `ctx.abort`) | | T5 | Forged or guessed run IDs, or one chat's run reaching another chat | The run ID is a random UUID in props the code can't read or change, and leases are keyed `<chat>/<run>` | existing env/import tests; unknown-run case | | T6 | Flooding the audit log by calling after the end | One refusal is audited per run. The call that finds a revocation audits it, and later calls don't. | existing 100-call test; revocation test | | T7 | Erasing what already happened | Effects admitted before the close keep their receipts (pending actions, proposed versions). Nothing is rolled back or described as undone. | (by construction) | Accepted residual: revoking **any** permission of the org agent ends every run of that agent under way, not only runs that used it. A run lasts at most 30 s, so this errs safe and costs little. It follows the posture of allowing and auditing, without per-permission bookkeeping. ## What changes - **`chat_code_runs`**, a new Workspace DO SQLite table. It is the chat's existing persistence, not a separate Code Mode store. Each row holds: - the run ID, tool call ID, SHA-256 and length of the module (never the module itself); - `agent` (`@earendil-works/pi-agent-core@<exact version>`, imported from the package; pi-ai is pinned to the same version in the catalog) and the compatibility date; - `runtime` and `cpu_limit`; - `status` (`running`, then the terminal status, or `interrupted`) and its times. - Rows are deleted with the chat. The migration is `0008_chat_code_runs`; it is new, so this is a hard cut. - **D2.** Only `RUNTIME_PROFILE=cloudflare` records `enforced`. Anything else, including unset or a typo, records `cpu_limit_unsupported`. Tests, `vp run dev` and the on-prem smoke config set `workerd`. The CPU proof is GRA-382's. **Follow-up:** the console must set `RUNTIME_PROFILE=cloudflare` on Cloudflare deployments. Until it does, their records say `cpu_limit_unsupported`, which errs safe. - **Leases.** They now carry `openedAt`. New `revokeCodeRun` and `admitFromCodeRun` close a lease and check one without counting a call. Draft writes and discards now name their run: `saveDraft(..., runId)` and `dropDraft(..., { runId })`. A proposal's follow-through drop goes by `{ revision }`, because its commit was already admitted. `takeCreate(..., runId)` also names its run. - **New error code** `agent.run_revoked`. ## Evidence New tests in `apps/core/test/agent.test.ts` ("chat agent code run leases"), through the real Workspace DO, Worker Loader, Pi loop and D1: - **The record:** call ID, source hash and length, the exact Pi version, compatibility date, `workerd` with `cpu_limit_unsupported`, and status `completed`. - **The DO fence:** - a write while the run is open is admitted; - after `closeCodeRun` (`wall_expired`), a write and a discard are refused with `agent.run_ended`; - after `revokeCodeRun`, a write and a create are refused with `agent.run_revoked`; - an unknown run is refused; - the revoked run's close returns `revoked`; - the draft holds only the admitted write, and the records read `wall_expired` and `revoked`. - **Revocation mid-run:** the code waits, an admin revokes the agent's grant, and the next call is refused. The tool result is `Error (revoked)`, the record says `revoked`, and exactly one `agent.run_revoked` audit refusal is written. - **Restart mid-run:** `ctx.abort("restart")` while the code waits. The turn fails, and on wake the record says `interrupted` and the run's ID answers `ended`. 6 s later there is still one tool call, no result, one run record and one model request: nothing ran again. In `apps/core/test/agent-builds.test.ts`: code that waits 2 s past its 1 s wall-clock limit and then writes the draft is refused once (`agent.run_ended`), and the draft stays empty. Commands: - `vp check`: passes. - `vp test run test/agent.test.ts test/agent-connections.test.ts test/chat-connections.test.ts test/agent-instructions.test.ts test/agent-knowledge.test.ts test/chats.test.ts test/memory.test.ts test/run-fixes.test.ts test/packages.test.ts test/dependencies.test.ts`: 184 passed. - `vp test run -c vite.screens.config.ts test/agent-builds.test.ts test/preview-repairs.test.ts test/app-preview.test.ts test/agent-apps.test.ts`: 46 passed. ## Security review fixes (cc14ff5) **M1. A revocation with no later call went unnoticed.** As a run ends, `#endCodeRun` now reads revocations once more. If it finds one, it closes the run as `revoked` and records a single `agent.run_revoked` (`method: executeCode`). **M2. A revoked run's logs still reached the model.** A revoked run now hands on neither its result nor its logs. In #390, output that no watcher has had yet is dropped when the lease is revoked. **M3. Any revoked row of the org agent ended every run.** `revokedSince` now counts only revoked permissions with `granted_at IS NOT NULL AND (chat_id IS NULL OR chat_id = <this chat>)`. A declined request that was never granted, or a revocation in another chat, leaves the run alone. The comparison is `revoked_at >= opened_at`. - I added no skew margin. With a 2 s margin, a run opened just after an unrelated revoke ended as revoked; the existing held-outcome connection test showed it. Both timestamps come from Workers clocks. **S1.** `saveDraft`, `dropDraft({ runId })` and `takeCreate` now run `admitEffect` first, and keep the lease check inside their own transaction. **S2.** `admitEffect` takes three steps in this order: - a cheap lease read, to get `openedAt` and fail fast; - the D1 revocation read; - the lease check, last, right before the effect. **S3.** - `connections.call` and `memory.saveUser` record their `admitEffect` refusals with `admitEffect(env, scope, method)`. Calls wrapped in `auditedCall` are recorded by it. - A refusal is recorded only by the call that closed the lease (`revocationOf` returns `closed` or `closedBefore`), which fixes the double audit. - Remaining case: a call admitted at its start, then refused at commit because another call had already revoked the lease, is recorded by `auditedCall` too. - The `admitEffect` in `memory.saveUser` stays. It is the commit-time check, which `requireOpenRun` at call start doesn't cover. **S4. New tests:** - M1: the code calls once and waits, an admin revokes, and the code returns without another call. The result and logs are withheld, the status is `revoked`, and one refusal is recorded. - M3: a declined, never-granted request and another chat's revocation, stored mid-run, leave the run `completed`. - `admitEffect` end to end, using a slow outside dependency: every read of core's files is delayed 2 s against a 1 s run limit. - A proposal whose run expired while it was checked commits nothing, and `build.propose` is refused as `agent.run_ended`. - A draft write let in at once outlives its run while it reads the App's files, and is refused at commit. - Mutation checks: - an `admitEffect` that does nothing fails the proposal test; - dropping the granted and chat filters fails the M3 test; - dropping the end-of-run revocation read fails the M1 test. **S5.** `RUNTIME_PROFILE` now fails closed (see D2 above). There is a test of how each profile value maps. **S6.** New migration `0056_permissions_revoked_idx` adds `permissions(subject_type, subject_id, status, revoked_at)`. **Nits:** - `recordSources` refuses a revoked lease with `agent.run_revoked`. - `interrupted` is a `CodeRunStatus`, and the statuses live in `code-run-statuses.ts`. - `chat_code_runs` (migration `0008`, regenerated since it isn't on main) gains a CHECK on `status`, a CHECK on `tool_call_id` length (1 to 200, cut to fit before insert), and a partial index `WHERE status = 'running'`. A chat keeps its latest 1000 records. - Not recorded: pi-ai's version, which exports no `package.json` to read (it is pinned to pi-agent-core's version in the catalog), and the workerd build version, which the runtime doesn't expose. - `RUNTIME_PROFILE` is added to the platform's reserved binding names. **Evidence at the top of the stack:** - `vp check`: passes. - `vp test run` on agent, connections, chat-connections, instructions, knowledge, chats, memory, run-fixes, permissions, packages and dependencies: 220 passed. - `vp test run -c vite.screens.config.ts` on agent-builds, preview-repairs, app-preview and agent-apps: 47 passed. ## Delta review fixes (d14824f) **Must-fix 1, fail open on T1:** the end-of-run revocation read could throw before the lease closed. **Must-fix 2, regression:** the lease stayed open through that D1 round trip after wall expiry or cancel. - `#endCodeRun` now closes the lease and records the status synchronously and unconditionally first. Only then does it read revocations. - If a revocation is found, the record is revised to `revoked`, one `agent.run_revoked` is recorded, and the result and logs are withheld. - A read that **throws** counts as revoked (fail closed). It is logged as `agent.revocation_read_failed`. New tests (only the revocation query is faulted, through a database proxy; everything else is real): - **A throwing revocation read:** the result and logs are withheld (`Error (revoked)`), the record says `revoked`, and the lease reads `ended`. - **A call during the end-of-run read** (the read is slowed 2 s; the run ends at its 1 s wall clock while the code waits, then calls): refused once as `agent.run_ended`, with the run recorded as `wall_expired`. - Mutation check: against the previous `#endCodeRun`, both tests fail. **Nits:** - A comment now states that cross-machine clock skew is the accepted residual of the margin-free comparison. - The `admitEffect` doc now says refusals are recorded per refused call. - A comment ties `granted_at IS NOT NULL` to how permissions become active: both grant paths set it, so only a request declined before it was granted lacks it. Evidence at the top of the stack: - `vp check`: passes. - Core test files: 226 passed (11 files). - Screen-config test files: 47 passed (4 files). - `smoke:code-mode`: passes. ## Greptile fix (1b7fd82): admission right before the write `admitEffect` ran before slow preparation, not right before the write, so work that outlasted the run, or a revocation meanwhile, still let the write land after the lease closed. I confirmed this with a probe for dependency resolution. Each write helper now takes an admission callback and calls it right before its batch, after any slow preparation. The early check stays as a cheap refusal. Where the late check runs: | Helper | Late check runs | |---|---| | `resolveDependencies` | after `resolveGraph` and `inspectGraph`, before `proposeResolved` | | `proposeDraft` | after `storeTree`, before `auditedBatch` | | `saveUserMemory` | as `writeVersion`'s `lastCheck` | | `createApp`, `createFromBlueprint`, `requestPermission`, `askInChat` | before their batch | Unchanged: - `saveDraft`, `dropDraft` and `takeCreate` already check the lease inside the Workspace transaction. - `connections.call` is admitted right before `CONNECT.call`. New tests, each slowing an outside step past a 1 s run limit: | Slowed step | Result | |---|---| | the registry metadata fetch | no dependency request or lock is stored, and the refusal is recorded as `build.resolveDependencies` | | the upload of the version's tree | no pending version | | the knowledge reads | no USER.md is saved, and one `memory.saveUser` refusal is recorded | Mutation check: removing the late admission from any one of the three helpers makes its test fail. Evidence: - `vp check`: passes. - Agent, connections, chat-connections, knowledge, chats, memory, permissions, dependencies, packages, app-blueprints and apps tests: 236 passed. - `agent-builds` and `agent-apps` (screen config): 34 passed.
What the code logs goes to the chat as it runs, through Pi tool updates, as the running step output: redacted, tied to its run and step, and dropped once the run closes, so a late line reopens nothing. Logs, result and error share one 64 KiB UTF-8 budget, with 1 KiB kept for how the run ended and an explicit marker where output was cut. The chat shows the output under the running code step.
A logged line reaches the chat only once a read of revocations begun after it arrived finds none, at most once a second a run, failing closed. One found closes the lease as revoked and drops the output. Core counts and bounds the lines it takes itself, redacts the whole output so a credential split across lines is caught, covers more credential shapes, keeps the output until the step result is stored, and sends output only when it changed.
…le revocations The JWT pattern started at a word boundary, which a flood like "eyJ-eyJ-..." offers at every step: seconds of matching on a run full of output, in the Workspace object, every second. It now starts after anything but a word character or "-". A revocation read that fails is recorded with its own reason, agent.revocation_read_failed.
nickruigrok
force-pushed
the
feat/code-mode-output-stream
branch
from
October 11, 2026 12:49
8456366 to
80a69d5
Compare
nickruigrok
marked this pull request as ready for review
October 11, 2026 12:49
Contributor
Author
|
@greptileai review |
|
…ch, and cut an oversized line An assignment of a credential now redacts its whole value: a quoted one to its closing quote, spaces included, and an unquoted one with any Bearer or Basic scheme it carries, of any length. A watch first update always says what output there is, none included, so a page that kept output from an earlier watch drops it. A line too large for the bytes left keeps what fits, never half a character, and the cut is shown.
Contributor
Author
|
@greptileai review |
nickruigrok
added a commit
that referenced
this pull request
Oct 11, 2026
… core keeps after many runs (#392) Fifth and last PR of the GRA-349 stack, on top of #390. It runs Code Mode's executor on plain workerd (decision D2) and checks what core keeps after many runs. ## Retention: what is measured, and what can't be - **Not measured: what the runtime holds for a run** (its isolate, and the RPC sessions of its stubs). workerd offers no signal to read: - I counted sessions of the seven code APIs by incrementing a counter in their constructors and decrementing it in `[Symbol.dispose]`. After 30 calls the counter read 30 opened and 30 still open, and was unchanged 3 s later. The runtime never calls `[Symbol.dispose]` on these loopback entrypoints. - Worker Loader exposes no count of loaded isolates. - The stubs core passes have no `Symbol.dispose` to call. - So this PR **doesn't claim measured handle retention**. The earlier `HeldHandles` registry is removed: it counted core's own bookkeeping and disposed nothing. - **Measured: core's own state.** After 40 runs over 4 turns (20 completed, 10 threw, 10 failed to load; 30 of them calling an API), `Workspace.codeRunRetention()` reads 0 open leases and 0 outputs held, with 40 closed leases (bounded by `endedRunsKept`). The run records hold 20 `completed`, 10 `threw` and 10 `load_failed`. ## Plain workerd (D2) - `vp run smoke:code-mode` (`scripts/workerd-code-mode.ts`) bundles the same `runCode` core runs into a small Worker (`apps/core/test/workerd/code-mode-worker.ts`). - It bundles **with Wrangler's dry run**, as `smoke:workerd` does, so a frozen install is all it needs. - It serves the Worker with `workerd serve --experimental` and a `workerLoader` binding, with no Miniflare. - It retries only a refused connection, within an overall 120 s timeout. - These 12 cases end as they do in core: | Case | Ends as | |---|---| | complete module (helpers, `self === null`, `ctx` with only `waitUntil`, two parallel capability calls) | `completed`, exactly 2 calls taken | | `fetch` | `threw` | | syntax error, non-function export, module that throws while loading (its log kept) | `load_failed` | | function that throws | `threw` | | module over 200,000 characters | `refused` | | promise that never settles | `wall_expired` | | 60 s wait under a 1 s limit | `wall_expired`, within 5 s | | cancelled after 300 ms | `cancelled` | | 5,000 one-line logs | `completed`, exactly 1,000 lines streamed | | flood of output | `threw`, at most 64 KiB, cut with the marker, lines streamed (no more than 1,000), its error ending kept | - The runtime reports `{"runtime":"workerd","cpuLimit":"cpu_limit_unsupported"}`. - CI runs it in the `workerd` job, right after `smoke:workerd`. - **From a fresh frozen install:** I made a new worktree, ran `vp install --frozen-lockfile` (which leaves no `esbuild` in the root `.bin`), then `vp run smoke:code-mode`. It passed all 11 cases (the line-cap case came after). **Finding for GRA-382 and the on-prem profile.** On plain workerd (1.20260921.1), a synchronous busy loop in a loaded Worker holds the thread the host's wall-clock timer needs, so the host never lets go. Until the CPU limit is proven on the deployed profile, on-prem must not offer Code Mode. ## Evidence (top of the stack) - `vp check`: passes. - Core: 230 passed. - Screen-config tests: 47 passed. - Web: 12 passed. - `smoke:code-mode`: passes, 12 cases (c2a75f3), and 11 cases from a fresh frozen install. - `check:migrations`: passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fourth PR of the GRA-349 stack, on top of #389. It adds bounded output and output that streams before the tool completes (section 21.1, step 5).
What changes
… (cut: over 65536 bytes of output).TextEncoder.encodeInto, so a cut never lands inside a character.verifyand torunseparately, because an RPC stub only works in the call that brought it.onUpdate; Pi ignores updates after the step settles). The turn hands it to the Workspace DO.ChatUpdate.output, which grows incrementally likepartialand is left out when unchanged.sk-/pk-/rk-keys, JWTs, PEM private keys and AWS access keys become[redacted]. This applies to every streamed line and to what the model reads and the transcript stores. A run holds no secret of its own; this guards against an outside system's data carrying one.detailsnow carry the run ID along with the status, tying the transcript to the run's record.ChatView.outputholds the running step's output, built withapplyOutput.Evidence
New tests in
apps/core/test/agent.test.ts("chat agent code output"), through the real DO watch, Loader and Pi loop:first\nkey [redacted]and then…\nafter the wait;nullonce the step ends;codeRunOutputafter the run closed shows nowhere.Error (threw): Error: the end;éand emoji, and the redaction patterns.In
apps/web/src/chat/follow-chat.test.ts: the output grows, starts afresh for another step, and drops when no step runs.Commands:
vp check: passes.vp test run test/agent.test.ts test/chats.test.ts test/run-fixes.test.ts test/agent-connections.test.ts: 90 passed.vp test run -c vite.screens.config.ts test/agent-builds.test.ts test/agent-apps.test.ts: 32 passed.vp test run src/chat/follow-chat.test.ts src/locales/locales.test.ts: 12 passed.vp build(web, React Compiler): passes.Restacked on the #387 and #389 review fixes (daf0cb4)
Promise.prototype.thenandPromise.resolve, and the in-order delivery are unchanged, and every outcome is still returned rather than thrown.Review fixes (c7d3cdd)
Must-fix: output kept streaming after a mid-run revocation. A line is now shown only once a read of revocations begun after it arrived finds none. Those reads run at most once a second a run, through the same
revokedSinceread, and fail closed. A revocation found there:revoked;agent.run_revoked(method: output);Should-fix:
sk_/pk_/rk_live and test keys, GitHubghp_/gho_/ghs_/ghu_/ghr_/github_pat_, Slackxox[abpr]-, GoogleAIza, and the value ofapi_key,apikey,token,secret,passwordorauthorizationwritten asname=valueorname: value. The docstring lists exactly what is covered and says it is a net, not a guarantee.Nits:
outputis left out of an update when it didn't change (applyOutputkeeps what was shown).aria-liveis dropped.New tests:
revoked.revokeCodeRun): the shown output goes tonullbefore the step's result.Logs:\n[redacted]\ndone.codeRunOutputshow as 1,000.Evidence (top of the stack):
vp check: passes.Delta review fix (2815b1e)
Must-fix: quadratic JWT redaction. The pattern started at
\b, and a flood likeeyJ-eyJ-…has a word boundary at every step. Redacting about 128K characters of that took about 4 s, inside the Workspace object, which re-redacts output about once a second.(?<![\w-]). A real JWT still matches."","x","-"and".". All must finish within 250 ms.Nits:
agent.revocation_read_failed, both by the output check and at the end of a run.revoked, and oneagent.revocation_read_failedis recorded.Evidence:
vp checkpasses;vp test run test/agent.test.ts test/chats.test.tspasses (103).