perf: reuse immutable history and reduce runtime loading - #281
Merged
Merged
Conversation
SaladDay
marked this pull request as draft
September 21, 2026 08:48
SaladDay
marked this pull request as ready for review
September 21, 2026 12:22
SaladDay
added this pull request to stack #291
September 21, 2026 12:32
yujiachen-y
previously approved these changes
Sep 21, 2026
yujiachen-y
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, thanks for the effort!
stack merge was automatically disabled
September 21, 2026 13:07
Pull Request is not mergeable
SaladDay
force-pushed
the
fix/history-encoding-overhead
branch
from
September 21, 2026 13:07
442c06b to
b2418c0
Compare
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Turns repeatedly decode old history and retain copies of the same messages. Broad imports and large bundles also add startup cost.
Changes
Behavior
Tools, prompts, token estimation, fingerprints, recovery, persistence, external-file visibility and replay eviction remain unchanged. Public mutable results remain detached. The benchmark and perf CI are unchanged. Release-scan exceptions remain limited to exact non-credential constants/expressions and bundle paths.
A small startup regression is an acceptable tradeoff for reproducible sustained-session gains. This PR is stacked on #277.
Base
1d44498; candidates54b2ab5and442c06bhave identical runtime code. Each run used Apple M1 (Virtual), 3 CPUs, 7 GiB; Node 22.23.2 / Bun 1.4.2, one warmup per side/scenario and three alternating serial pairs. Both completed all 24 runs. Across all six pairs, every 100-turn and 300-turn duration, CPU and RSS measurement improved. The smallest paired duration reductions were 21.7% and 37.5%, respectively.Both full CI results are INCONCLUSIVE, not passing. Run 1 exceeds the 30% spread limit in baseline startup duration (33.3%). Run 2 exceeds it in candidate 100-turn duration/CPU (33.4%/37.6%), although every pair still improves. One startup pair in run 1 is slower; both startup medians improve. No thresholds or samples were changed.
All 14 applicable local functional checks passed; independent blind review found no evidenced in-scope issue. Source verification, release audit and CLI release checks pass on
442c06b. Local timing is excluded because of competing machine load. The separate tokenizer-loading experiment is not included.Full performance CI 1 · Full performance CI 2