Skip to content

Slice 014: the prompt window is the recent conversation, not the first 500 rows - #95

Merged
HackTuah merged 2 commits into
mainfrom
slice/014-prompt-window-is-recent
Sep 24, 2026
Merged

HackTuah merged 2 commits into
mainfrom
slice/014-prompt-window-is-recent

Conversation

@HackTuah

Copy link
Copy Markdown
Member

A session holding more than five hundred messages was building every subsequent request from its first five hundred.

Message |> where(...) |> order_by([m], asc: m.seq) |> limit(^limit) |> offset(^offset)

Ascending plus a limit returns the oldest rows. build_request/1 called it with limit: 500. Past that point the assistant answered from the opening of the session with the current exchange absent from the request entirely. Compaction does not delete rows, so any long-running session reached that point by design rather than by accident.

It also explains a second symptom: Compactor.plan/1 derives covered_to from the latest compaction in the window it is handed, so a stale window let a plan re-cover ground and record two compactions with the same to_seq — a model call spent to learn nothing.

The fix is a second accessor, not a change to the first

recent_history/2 takes the newest rows descending and reverses them, so the prompt reads oldest-to-newest as a conversation must. history/2 keeps its forward paging for the interface and the export, and AC3 is a test asserting that at a row count where the two differ.

Renaming or reordering history/2 would have been the smaller diff and the worse change: two callers with opposite needs sharing one function is how this happened.

Why no test caught it

Every existing session test runs below the window. The longest ran 200 turns and only crossed the line once compaction itself added 195 rows. The new test works at 520 rows throughout, because below the window the defect is invisible by definition.

Proven by mutation, not by the fix passing

Reverting recent_history/2 to ascending makes the new test report the defect in its own failure message:

the prompt window ends at seq 500 of 520. Past the window the assistant answers from the
opening of the session with the current exchange missing

How it was found

By accident, at slice 110. Registering one tool raised the tool definitions from 3,254 to 3,516 tokens against a 4,200 soft threshold in the test environment, which took compaction from 38 passes over 200 turns to 195, which pushed a session past 500 rows for the first time in any test.

The tool surface is not the problem and this slice does not treat it as one. That arithmetic is an artefact of a deliberately small test window; the defect was there at any window size. Slice 110 is parked until this lands, because burying a correctness defect in a sandbox diff would hide it.

mix gate
Result: 750 passed (14 properties, 736 tests), 18 excluded
trinity.coverage: 004 80.31% vs 120 80.31%: OK

🤖 Generated with Claude Code

…t 500 rows

A session holding more than five hundred messages was building every subsequent
request from its first five hundred. Store.history/2 orders ascending and then
limits, so it returns the oldest rows; build_request/1 called it with limit: 500.
Past that point the assistant answered from the opening of the session with the
current exchange absent from the request entirely, and compaction does not delete
rows, so any long session reached that point by design rather than by accident.

It explains a second symptom. Compactor.plan/1 derives covered_to from the latest
compaction in the window it is handed, so a window pinned to the oldest rows let a
plan re-cover ground and record two compactions with the same to_seq, spending a
model call to learn nothing.

The fix is a second accessor rather than a change to the first. recent_history/2
takes the newest rows descending and reverses them, so the prompt reads oldest to
newest as a conversation must; history/2 keeps its forward paging for the
interface and the export, with a test asserting it at a row count where the two
differ. Two callers with opposite needs sharing one function is how this happened.

No existing test reached above the window, which is why it went unseen: the
longest ran 200 turns and only crossed the line once compaction itself added 195
rows. The new test works at 520 rows throughout, because below the window the
defect is invisible by definition.

Proven by mutation rather than by the fix passing. Reverting recent_history/2 to
ascending makes the new test report the defect in its own failure message: the
prompt window ends at seq 500 of 520.

Found by accident at slice 110, which registered one tool and pushed a small test
window past a threshold. The tool surface is not the problem and this slice does
not treat it as one: the defect was there at any window size, waiting for a
session to pass five hundred rows.

mix gate: 750 passed (14 properties, 736 tests), 18 excluded.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
The FIPS leg failed on slice 014's branch while its sibling run of the same commit
passed. That is R26's signature, and R26's own counter-example entry says the
signature is not the diagnosis, so the stack trace was read first. It was not R26.

Trinity.TelemetryTest, two turns are two traces, sent its second user message as
soon as it received the llm call stop event from the first. That event fires when
the model call ends, not when the turn does: the session is still thinking while
it writes the result, so the second message returned {:error, {:busy, :thinking}}.

It has been racy since slice 090 and passed on the ordinary gate for days because
that leg is faster where it matters. The FIPS leg is about half again as slow,
which is enough to lose the race reliably.

The test now waits for idle, which is what it meant: two turns is a claim about
two turns, and the end of a model call is not the end of a turn. A poll against a
deadline rather than a sleep, so it flunks naming the state it was stuck in rather
than trading one race for a slower one.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
@HackTuah
HackTuah merged commit a1738db into main Sep 24, 2026
8 checks passed
@HackTuah
HackTuah deleted the slice/014-prompt-window-is-recent branch September 24, 2026 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant