feat(history): file-backed conversation history streamed to providers (Fixes #854) by acoliver · Pull Request #3852 · vybestack/llxprt-code

acoliver · 2026-10-11T05:52:18Z

TLDR

Conversation history now lives in an on-disk journal instead of in memory. HistoryService is a facade over that journal, and subagents get their own journal files. On every provider send, rows are streamed from the journal through the provider transformers into the request body. The only request-sized data is the per-request body, released when the call ends. Long sessions no longer keep a context-sized copy of history in the heap between turns.

Request bytes match main for the same history. Resume, --continue, chat restore and package import/export stream from disk.

Dive Deeper

Requirements and how they are covered (from project-plans/issue854/finish-plan.md):

R1, no context-sized structure at rest. History membership, pending tickets, the provider normalization snapshot, density folds and the resume resolver are all disk-backed. The in-memory state between turns is scalars plus state bounded by mutation events.
R2, journal-fed default send. The array send route and its enforcement are deleted. Every provider family (OpenAI Responses/Codex, OpenAI chat, Anthropic, Gemini plugin, Vercel, load balancing) prepares from requestRows. Stored media references are resolved one row at a time and released at call cleanup.
R3, retained-heap proof with traps. Three files in packages/agents/src/core/__tests__/ run a real ChatSession with compression, one worker per file so each fits the 300 s per-file budget:
- journal-retained-growth.test.ts is the normal run. It uses about 4 KB rows and a 30k-token context, which gives roughly 90 compressions in about 850 requests. After a warm-up that ends when settled heap converges (around turn 510), it takes six settled checkpoints 50 turns apart. Every checkpoint must stay under the 1 MiB allowance, and the fitted growth over checkpoints 3 to 6 must stay under 262,144 B. Locally the late fitted growth ranged from about -4 KB to +28 KB.
- journal-retained-growth-retaining-trap.test.ts keeps every row alive and must fail the allowance.
- journal-retained-growth-release-trap.test.ts retains rows and releases them before the end. It must fail an intermediate checkpoint and end under half the allowance.
An earlier single-file version, which ran three workers concurrently, measured a one-time step of about 170 KB around turns 1,050 to 1,100 on CI. Instrumentation traced it to the bounded request-shape memory filling to its 2,000-entry cap and to JIT tier-up; no product type grew per turn over 1,750 turns. The split design measures after both have settled.
R4, unchanged bytes. Byte-equivalence suites (providerBody*.p05b4, disk-text and source differential tests) compare against independent oracles. Reasoning-ID and active-parent tool-output cases are compared against bytes captured from origin/main.
R5, streamed resume/continue/import. Package import has a 64 MiB per-line limit. The 100k history-count limit now also applies on export. Persisted-state JSON is parsed one file at a time. Blobs are copied one object at a time.
R6, gates. See the test plan below.

Durability. A general history replacement (restore, import-driven replacement, compensation) is written as a replace_begin / rows / replace_commit transaction. Every journal reader, including the CLI scrollback pager, treats an incomplete transaction as absent. A crash during a replacement therefore resumes with the previous history, not an empty or partial one. Single-record shapes (a suffix cut, a clear, a compression) are written as one record. The recorder rejects journal admission when it is inactive or disposed, instead of reporting a write that never happened as durable.

Speed. Measured on this machine:

Prompt estimation at 167k tokens takes about 250 ms, down from 5.9 s earlier on the branch. Main takes 154 ms. Counts are identical.
An exact pre-tokenizer-chunked o200k counter replaces the slow disk BPE path. The official GLM, MiniMax and Kimi tokenizers are counted in bounded chunks cut only at boundaries proven to be additive.
A warm send under the context limit is still slower than main: 349 ms against 104 ms at 1,000 rows, and about 2.9 s against 1.3 s at 8,192 rows. Most of that cost is rebuilding the provider normalization snapshot. Incremental provider normalization across sends (journal-backed history send cost grows with row count) #3849 tracks making it incremental.

Merge with main's owner/port refactor (#3842). The merge had 219 conflicts. Journal-backed code now uses the new owners and ports (session settings, hook and media owners, the runtime tokenizer factory, instance-owned telemetry). Problems the merge exposed and this branch fixes:

Profile adoption is two-phase. A switch writes replace_begin, then commits or rewinds synchronously, rolls back with replace_abort, and fails if history changes concurrently.
Journal records carry their origin, and recorders receive only their own origin's rows. Unowned rewind and density records still reach every mirror, and a resumed recorder in fan-out mode becomes the visible primary.
/clear followed by --continue no longer writes a zero-length rewind into the new session.
Recorded media is hydrated from the archive on export and on resume.
Load-balancer member sends keep the agent system instruction.
Telemetry flush and close no longer wait forever on an export that never acknowledges. After the 5 s acknowledgement timeout, the requesting source gets ExportAcknowledgementMissingError. The root does not report that export a second time at flush or close; other export failures are still reported there.
Client replacement copies carried history with one detached admission instead of two.
Production providers code no longer imports the full bun types. That import put Bun's globals into the program the runtime-boundary analyzer walks and pushed its DOM walk from 9,160 to 11,170 nodes, past the 10,000 cap. Wrapper fetches forward preconnect through a small local helper instead.

Main's #3507 tests were replaced by nine equivalents on the send seam. The p05b4 retry capture count went from 3 to 2, because main's auth policy allows one refreshed retry. The listener-count assertions in recording-finalizer-join were dropped, because the journal has no listeners.

Behaviour changes reviewers should know about:

An abort during a source send now surfaces the user's AbortError instead of the provider's "terminated".
/set retries 0 on Responses providers used to send nothing and return an empty stream. It now means one attempt. A 200 response with no body is an error.
MissingSnapshotHookCallbackError is raised for HookSystem subclass overrides on the source route. That is not a supported extension point.
OpenAI chat and Responses bodies are produced on demand, so uploads are chunked with no Content-Length. OpenAI/Responses requests now upload chunked without Content-Length (on-demand body from #854) #3847 covers what to do if an endpoint rejects that.
The new file-driven scrollback pager (ui.scrollbackPagerEnabled) ships defaulted off. It projects only journal rows, so it cannot yet show tool resultDisplay text, UI-only rows or the live pending turn. With it off, the transcript uses the fallback ledger, which is bounded by configured item and byte limits. Unified scrollback to the start of history: one disk-paged surface with the live turn, on by default #3856 tracks finishing the pager.
Model hooks (BeforeModel/AfterModel) now run on the disk snapshot path and emit hook_call telemetry from there, with stdout/stderr capped at 4000 characters.
Responses request bodies are built in chunks that start at 1 KiB and grow, and tool calls are paired in one pass. Before this, an 8,192-row tool-heavy send rescanned history for every tool call and over-allocated chunk buffers.
Density changes are recorded as density_mutation records of at most 64 rows or 64 KiB each.
The per-turn system prompt is now read at send time. The race it fixes also exists on main.
CI partitions changed: agents run in 3 shards and scripts in 4, balanced by measured duration (scripts/lib/scripts-test-weights.ts).
SessionRecordingService uses backpressure instead of throwing at the queue limit. While merging main (fix(recording): keep one unreadable session from breaking --continue (Fixes #3732) #3838), the header-rebind tests that asserted throw-on-limit or prepared-batch rollback were rewritten as commit-based cases or dropped, because that behaviour no longer exists.

Review. One full review found 2 blockers and 14 major findings. All were fixed. A findings-only follow-up review confirmed 11 of them resolved and 5 partially resolved; those 5 were then completed:

the scalar length during an open transaction;
the bounded official and legacy estimators;
tool-group carry in the pager;
a rewind cut inside a tool group;
a missing failure-path test.

Known open item. HistoryJournalStore.dispose() swallows recorder.dispose errors and removes scratch asynchronously.

Follow-ups filed:

Incremental provider normalization across sends (journal-backed history send cost grows with row count) #3849: incremental provider normalization across sends.
Remove array-route leftovers and #854 heap harnesses after the journal-fed send route #3848: remove array-route leftovers and the older heap harnesses.
OpenAI/Responses requests now upload chunked without Content-Length (on-demand body from #854) #3847: chunked uploads without Content-Length.
Unified scrollback to the start of history: one disk-paged surface with the live turn, on by default #3856: scrollback pager live-turn rendering.
Last-resort tool-response truncation is O(candidates x rows) on the journal-backed send path #3857: last-resort truncation costs O(candidates × rows).
Responses progressive input materializes the whole history when a tool row is present #3858: the Responses progressive input materializes the whole history when a tool row is present. This is request-scoped, but adds about 344 MB at peak with 8,192 rows.

Reviewer Test Plan

npm run typecheck, npm run lint, npm run build, npm run test, npm run test:scripts.
Gemini plugin: cd plugins/google-gemini && npx tsc --noEmit && bun test.
Memory proof: cd packages/agents && bun test src/core/__tests__/journal-retained-growth.test.ts --timeout 300000, then the same command for journal-retained-growth-retaining-trap.test.ts and journal-retained-growth-release-trap.test.ts. Each passes; the trap files pass by observing their trap fail its bound.
Runtime boundary: npm run build:types && npm run check:runtime-boundary.
Smoke: bun scripts/start.ts --profile-load lunahigh "write me a haiku and nothing else". Also try a tool round trip such as "list the files in the current directory".
Manual: run a long session with tool calls, then /chat restore to an earlier point. Quit, --continue, and page back through scrollback. Rows before the cut should remain visible and rows after it should not.

Testing Matrix

	🍏	🪟	🐧
npm run	✅	❓	❓
npx	❓	❓	❓
Docker	❓	❓	❓
Podman	❓	-	-
Seatbelt	❓	-	-

Local macOS: typecheck, lint, build, the Gemini plugin and both smoke profiles (lunahigh, and haiku with a tool round trip) pass. The Interactive UI tmux suite passed twice locally (19/19). On this machine several timing-sensitive tests fail only when the full suite runs alongside other sessions, and pass when run alone. CI on Linux is the authority for the full suite.

Linked issues / bugs

Fixes #854

…Density action

…ession)

…ion)

…e disk round trip

…ssion)

…sion)

Remove the paired-estimator memory harnesses (2 h / 4 h runner budgets, one hung at 0% CPU) and the serial acceptance scheduling that existed only for them. Runner and bun-test-policy files return to origin/main. The repeated-turn journal-retained-growth proof replaces them; behavioral childaccept, wholepeak, media-peak and target tests stay.

They required BODY_EVIDENCE_TEST_SCRATCH, which nothing sets. Each file now creates and removes its own real-path scratch directory. The evidence writer stays because about 25 surviving byte-equivalence tests import it.

Compression summaries call the provider with request rows outside the chat send seam, so openai-responses failed with 'requires a prepared projection token'. The provider now prepares the same projection itself. The one-shot and middle-out summary BODY byte tests pass again (12 each).

…per test It asserted original-row identity across a paused writer, the old contract the journal no longer provides. The four GC-retention cases still prove the trap holds the full fold. The helper stays: three child workloads use it.

…eclaim cases to the durable contract

… paused writer

…ation path uses

…quest uses

…p owner census

…riter

…live writer; retire forensic detached memory harness

…hed copies)

…tion control

…t runtime

… in-place caller mutation case

…h root

…string oracle to sizes tiktoken can encode

The chat projection no longer parses the spooled body into a request object on every estimate. The same pass that writes each message to the spool writes it to the estimator-form messages file, tools follow, and the estimator reads the sealed Gpt56SourceProjection. Unsupported media is a ledger appended per row instead of a scan over an array. Token counts are unchanged (72 parity cases over three chat routes), so PROJECTION_REVISION stays 4. The send peak probe now covers the estimate step, with a trap that holds the history while estimating and must fail.

…ends a second spool D3c: the chat request dump is written from the spool one chunk at a time (re-indented by prettyJsonBytes) instead of parsing the whole body, so a dump no longer holds the body as a value. D4: when a streaming reply holds tool calls and no text, the continuation request is built as a second spool from the same rows plus the acknowledgement tail, and sent with Content-Length equal to the sealed spool size, instead of re-parsing the first body. The deleted OpenAIChatSpooledBody reader is gone from the production path. Request and dump bytes are unchanged; the chatDumpGolden and chatContinuationGolden fixtures were captured against the code at D3b and pass against it. The sendPeak chat measurement is split into sendPeak.openaiChatTextOnly and sendPeak.openaiChatToolHeavy. sharedSendPeak runs one child process for the shapes a file lists, and the first test awaited that whole run, so one test carried both shapes against the 170 s per-test limit and timed out on a loaded machine (the D3b code timed out the same way). One shape per file gives each child half the work. Assertions, estimate, rounds and bounds are unchanged. Lint fixes in touched files: prettyJsonBytes state moved into a class, golden tests assert in the test body, continuation text loop moved to OpenAIStreamChunkText to keep OpenAIProvider under the line limits.

…ed sends The retained-growth gate compared the settled reading after send 6 with the one after send 1, so two readings decided the outcome. Under load (load average 25) sendPeak.openaiChatTextOnly read 1,099,512 bytes on 1 of 3 runs, 51 KB over the 1 MiB bound, from noise in one reading. The probe already takes a settled reading after each of the 6 sends. The gate now uses the Theil-Sen slope of those readings (median of all pairwise slopes) times sends minus one, taken for heap and arrayBuffers separately, and judges the larger. A steady per-send leak gives a steady positive slope and still breaks the bound; one outlier moves few of the 15 pairwise slopes and cannot move the median much. SEND_PEAK_BOUND_BYTES, the peak-growth gate and the traps are unchanged. verdict.retainedGrowth is now the trend value and verdict.retainedEndpointGrowth keeps the old first-to-last difference, so the SEND_PEAK_LOG jsonl records both for comparison. A unit test checks the estimator on a known series, a flat series with one large outlier, and a steady leak of bound/(K-2) per send.

… through a request-scoped disk index

…/res id indexes; the dump stream uses it

…ble one row at a time

… table

… message table The Anthropic request body is built into the disk-backed message table one row at a time, finished and cache-marked on the table, and emitted into a request-scoped spool posted with Content-Length equal to its sealed size. No history array or whole request object is built on the send path. Bytes are unchanged (wire golden corpus, cache 1h and off). sendPeak Anthropic bounds now pass (+18.8/+25.7 MB before); the probe is split per shape so each file has headroom under the per-test limit.

…iles The Anthropic projection writes estimator-form segment files from the same table pass that fills the spool, estimates through the source projection's anthropic-messages protocol, and records unsupported media per row. Dumps stream from the spool. Token counts are identical (estimate parity over the golden histories, caching and OAuth variants) and dump bytes are unchanged (dump golden captured from the previous code). The Anthropic peak probes now include the estimate step, with an estimate trap that must fail.

…get instead of keeping a body copy The recovery retry no longer reads the sent body back, rewrites it and keeps the copy in the shared retry context. The context holds the recovery budget (a number or two); the retry, and any later outer attempt, builds the spool again from the rows and replaces oversized images in the finished message table before the body is written. Retry bytes and dumps are unchanged (golden from the old code). A later outer attempt used to send the kept body after the first attempt's lease had spliced its arrays, i.e. an empty messages array; it now sends the retry bytes.

…nd the Gemini plugin still collect

…irst root

… new ones

…nts in one pass (closes #3857) Replacements are ranked and decided in one pass over a disk-backed running estimate and published as one change, instead of re-projecting the candidate and writing the journal after every replacement. When shedding every tool response cannot reach the limit, the stage fails fast with ContextOverflowError. Results equal the previous per-replacement stage on the differential corpus. Measured on the bench fixture: 1,024 rows went from 722 s to 84 ms; 2,048 and 8,192 rows never finished before and now take 227 ms and 537 ms.

… append before tests unlink the journal prepareParity left the last row's appendFile in flight. When a test then unlinked the journal, the late append recreated the path, which failed the existsSync assertion and orphaned the paused fold (its scratch root was removed, surfacing as ENOENT in the next test). Reproduced 3/60 on the unmodified code; 0/200 after waiting for durability.

…crementally instead of rescanning the journal

…stead of once per resolved row Every row a semantic purge record resolves points at the same journal line, and the origin lookup parsed that whole line per row, so the bytes parsed grew with the square of the row count (16x for 4x rows). The reader now keeps the origin of the latest and of the longest line.

…iteSync per JSON piece

…s instead of four small I/O calls per row

…ow batch and reader start instead of statting on every row

…and rehash through resident pages instead of one syscall per slot

…I/O tests typecheck without casts

…nd-only log read by position instead of one hashed key lookup per row

…ag keys from a fixed-size membership filter instead of probing the index

…tead of three zod objects per lookup

…slice and awaits only on a full chunk or an oversized piece

…ount already fits the token limit A BPE token covers at least one byte, so output with no more bytes than the effective token limit cannot exceed it. The check is exact and avoids encoding every small tool result on the send path.

acoliver added 30 commits October 10, 2026 13:57

test(#854): density call-site count follows the source-stage optimize…

e6a56a0

…Density action

fix(#854): keep injected history service readable for typecheck (regr…

96383d3

…ession)

fix(#854): durable history clear records a single cut rewind (regress…

fba42c8

…ion)

test(#854): token usage lifecycle exercises real disk compression

0526e55

test(#854): contract history round trip asserts entry values after th…

732a5a1

…e disk round trip

fix(#854): restore compression and rebuild recording from main (regre…

28288f9

…ssion)

test(#854): kimi media pre-pass receives normalized request rows

84e0537

test(#854): codex stateful options carry normalized request rows

2609a1b

fix(#854): restore stateful media failure surfacing from main (regres…

54bb2f5

…sion)

test(#854): kimi cache transport asserts chunked on-demand body

3a646d7

test(#854): make body-evidence tests self-contained

19eb1fb

They required BODY_EVIDENCE_TEST_SCRATCH, which nothing sets. Each file now creates and removes its own real-path scratch directory. The evidence writer stays because about 25 surviving byte-equivalence tests import it.

test(#854): convert paused-writer, owner-count and same-millisecond r…

bceb874

…eclaim cases to the durable contract

test(#854): stop pending-prefix ownership tests from deadlocking on a…

e563bee

… paused writer

test(#854): pause deferred commit test on the durability wait the mut…

89c057d

…ation path uses

test(#854): observe curation reads on the snapshot path the source re…

e3bd18f

…quest uses

test(#854): drop heap-delta gate from source HTTP ownership test; kee…

f4d1825

…p owner census

test(#854): run pending-window and fallback value tests with a live w…

b8aca24

…riter

test(#854): convert held snapshot and tool repair pending tests to a …

850bda2

…live writer; retire forensic detached memory harness

test(#854): compare pending fold rows by value (stored rows are detac…

85e3847

…hed copies)

test(#854): count the session start append when injecting write failures

c3e5b61

test(#854): retire forensic queue peak statistic test; keep the reten…

85c1610

…tion control

fix(#854): export the core subpaths that production packages import a…

892895b

…t runtime

test(#854): compare density history rows by value

1c11607

test(#854): convert chat pending reader test to a live writer; retire…

601e126

… in-place caller mutation case

test(#854): observe request snapshot workspaces in the process scratc…

f5e8e69

…h root

test(#854): observe snapshots in the scratch root; bound o200k whole-…

f53fb1f

…string oracle to sizes tiktoken can encode

test(#854): measure ownership on the source projection real sends use

38b5916

acoliver force-pushed the issue854-journal branch from ef596b7 to 5f7e7aa Compare October 14, 2026 06:56

acoliver added 11 commits October 14, 2026 04:25

fix(#854): D5 Kimi file pre-pass reads rows one at a time; uploads go…

00b7178

… through a request-scoped disk index

fix(#854): E1 indexed Anthropic message table with prev links and use…

847950c

…/res id indexes; the dump stream uses it

fix(#854): E2 build Anthropic wire blocks into the indexed message ta…

905be01

…ble one row at a time

fix(#854): E3 post-table passes and cache controls run on the message…

a33c66f

… table

fix(#854): F1 remove the remaining history-array seams; only Vercel a…

58149a1

…nd the Gemini plugin still collect

acoliver mentioned this pull request Oct 14, 2026

OpenAI/Responses requests now upload chunked without Content-Length (on-demand body from #854) #3847

Open

acoliver added 16 commits October 14, 2026 12:47

fix(#854): sweep dead scratch roots once when a process creates its f…

9db6293

…irst root

fix(#854): move bun:test kits out of production src and guard against…

cbcb654

… new ones

fix(#854): history statistics, context range and last content read in…

fa08a81

…crementally instead of rescanning the journal

perf(#854): prompt-key writer fills a 64 KiB buffer instead of one wr…

c8247bc

…iteSync per JSON piece

perf(#854): detached journal gathers appends into bounded tail buffer…

9552ebe

…s instead of four small I/O calls per row

perf(#854): provider normalization checks the scratch directory per r…

0811634

…ow batch and reader start instead of statting on every row

perf(#854): provider normalization index probes read one slot window …

d1b0567

…and rehash through resident pages instead of one syscall per slot

test(#854): count buffered I/O through call-through spies so the new …

e7f58fa

…I/O tests typecheck without casts

perf(#854): provider normalization keeps each stage's rows in an appe…

4185448

…nd-only log read by position instead of one hashed key lookup per row

perf(#854): provider normalization answers lookups of never-stored fl…

c4b62cd

…ag keys from a fixed-size membership filter instead of probing the index

perf(#854): provider normalization builds its counter schema once ins…

5a78cd5

…tead of three zod objects per lookup

perf(#854): chunked o200k counting takes runs of small pieces as one …

a671205

…slice and awaits only on a full chunk or an oversized piece

acoliver mentioned this pull request Oct 14, 2026

Incremental provider normalization across sends (journal-backed history send cost grows with row count) #3849

Open

Navigation Menu

Repository navigation

feat(history): file-backed conversation history streamed to providers (Fixes #854) - #3852

feat(history): file-backed conversation history streamed to providers (Fixes #854)#3852
acoliver wants to merge 381 commits into
mainfrom
issue854-journal

acoliver commented Oct 11, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

acoliver commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

Dive Deeper

Reviewer Test Plan

Testing Matrix

Linked issues / bugs

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

acoliver commented Oct 11, 2026 •

edited

Loading