Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,22 @@
background logs, and paths the task names). bash is not blocked; this is a
budget control, not isolation.

### Fixed

- **Two lean-session defects (lean-sessions study, PR #50).** When the kept
user turns are mostly Maat's notes, a shed cut on user turns must now free
at least 25% of the history, or it cuts on recent messages instead: Maat's own notes (acceptance criteria, bar refusals) are
user messages, so a shed could cost the whole prompt cache for 0.7%
(60,788 -> 60,387 tokens), or come back empty on every step so auto-shed
never fired again that turn. Superseded tool results are now elided per
call, not per step: rewriting `dur.py` no longer drops `test_dur.py` read in
the same step. A bash rerun that came back different supersedes the earlier
output (a rerun with different options, such as a longer `timeout_s`, is a
different call), and a plain `cat`/`head`/`tail`/`nl`/`sed -n` of one file
counts as a read a later write makes stale. A shed that cuts on recent
messages keeps the acceptance criteria and the live bar refusal verbatim.
See `docs/lean.md`.

## 0.2.2 — subscription backends removed

### Removed
Expand Down
30 changes: 30 additions & 0 deletions docs/lean.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,36 @@ To see them: `npx tsc -p tsconfig.test.json && node --test dist-test/test/lean-b
- The numbers are deterministic (scripted turns, no clock in the bodies), so
1.5x leaves room for small prompt edits, not for noise.

## Shed and elision rules the study fixed

From the lean-sessions study (PR #50); tests in `test/lean-fixes.test.ts`.

- **Shed min-free** (`SHED_MIN_FREE` = 0.25 in `src/transcript.ts`). When
the user turns a shed would keep are mostly Maat's own notes (acceptance
criteria, bar refusals, nudges, all tagged in `molt`), the cut on user turns
must free at least a quarter of the history, or `planShed` cuts on recent
messages instead. A shed costs the whole prompt cache, and those notes are
user messages, so the user-turn cut could land near the start of a long
turn (a real run: 60,788 -> 60,387 tokens) or return no plan at all, step
after step, so auto-shed never fired again. An interactive session whose
kept turns are the person's own is cut on user turns as before.
- **Criteria and refusal survive a recent-message cut.** On a cut on recent
messages, the latest criteria note (`molt.criteria`) and the live bar
refusal (the latest non-stale `molt.barFailure`) stay verbatim right after
the digest instead of being excerpted to 300 characters in it.
- **Per-call elision.** `elideSupersededReads` elides the superseded call's
own result, not every result of its step. A bash rerun with new output
supersedes the earlier output; the same command with different options
(`timeout_s`, compared with keys sorted) is a different call, and a
`background: true` run is never superseded (its result is the job handle).
A plain `cat`/`head`/`tail`/`nl`/`sed -n` of one file (no pipe, redirect or
second command) is a read that a later write of that file makes stale; a
bash rerun answered with a "same call" pointer is not a new read, so the
copy it points at stays the one a write invalidates.

The flag-gated experiments from the study (shed threshold, ageing) are not
part of this.

## The bench cost alarm (`bench/local/run.py`)

After each run, run.py reads the run's cost (`spend.costUsd`) and prompt
Expand Down
2 changes: 1 addition & 1 deletion src/engine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6062,7 +6062,7 @@ export class Engine {
"never reported as verified — do not describe one as passing. You cannot",
"edit these; attempting to is itself a failure.",
];
this.transcript.push({ role: "user", content: lines.join("\n") });
this.transcript.push({ role: "user", content: lines.join("\n"), molt: { criteria: true } });
}
};
if (!pendingCriteria) yield* sealCriteria(opts.taskChecks ?? [], opts.taskNotes ?? []);
Expand Down
Loading
Loading