Skip to content

perf: reuse immutable history and reduce runtime loading - #281

Merged
SaladDay merged 10 commits into
mainfrom
fix/history-encoding-overhead
Sep 21, 2026
Merged

SaladDay merged 10 commits into
mainfrom
fix/history-encoding-overhead

Conversation

@SaladDay

@SaladDay SaladDay commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

Why

Turns repeatedly decode old history and retain copies of the same messages. Broad imports and large bundles also add startup cost.

Changes

Area Change
History reads Verify freshly read bytes and reuse a bounded decoded prefix, private validation results and revisions
Snapshots Share unchanged immutable messages while preserving aliases, isolation and logical replay size
Startup Use focused pi package exports and minify bundled whitespace
Encoding and queries Reuse fixed frame prefixes, UTF-8 lengths and prepared queries

Behavior

Tools, prompts, token estimation, fingerprints, recovery, persistence, external-file visibility and replay eviction remain unchanged. Public mutable results remain detached. The benchmark and perf CI are unchanged. Release-scan exceptions remain limited to exact non-credential constants/expressions and bundle paths.

A small startup regression is an acceptable tradeoff for reproducible sustained-session gains. This PR is stacked on #277.

Full CI Scenario Duration CPU Peak RSS
1 Startup -8.4% -4.5% -17.4%
1 100 turns -29.4% -30.3% -20.5%
1 300 turns -38.8% -38.4% -57.6%
2 Startup -5.7% -10.4% -21.5%
2 100 turns -22.3% -24.9% -30.8%
2 300 turns -42.6% -43.7% -58.0%

Base 1d44498; candidates 54b2ab5 and 442c06b have identical runtime code. Each run used Apple M1 (Virtual), 3 CPUs, 7 GiB; Node 22.23.2 / Bun 1.4.2, one warmup per side/scenario and three alternating serial pairs. Both completed all 24 runs. Across all six pairs, every 100-turn and 300-turn duration, CPU and RSS measurement improved. The smallest paired duration reductions were 21.7% and 37.5%, respectively.

Both full CI results are INCONCLUSIVE, not passing. Run 1 exceeds the 30% spread limit in baseline startup duration (33.3%). Run 2 exceeds it in candidate 100-turn duration/CPU (33.4%/37.6%), although every pair still improves. One startup pair in run 1 is slower; both startup medians improve. No thresholds or samples were changed.

All 14 applicable local functional checks passed; independent blind review found no evidenced in-scope issue. Source verification, release audit and CLI release checks pass on 442c06b. Local timing is excluded because of competing machine load. The separate tokenizer-loading experiment is not included.

Full performance CI 1 · Full performance CI 2

@SaladDay SaladDay added the perf:full Run the full performance suite; required for performance-sensitive changes label Sep 21, 2026
@SaladDay
SaladDay marked this pull request as draft September 21, 2026 08:48
@SaladDay SaladDay changed the title perf: reduce semantic encoding and session query overhead perf: reuse immutable history and reduce runtime loading Sep 21, 2026
@SaladDay
SaladDay marked this pull request as ready for review September 21, 2026 12:22
@SaladDay
SaladDay added this pull request to stack #291 September 21, 2026 12:32
yujiachen-y
yujiachen-y previously approved these changes Sep 21, 2026

@yujiachen-y yujiachen-y left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the effort!

stack merge was automatically disabled September 21, 2026 13:07

Pull Request is not mergeable

Base automatically changed from fix/history-processing-overhead to main September 21, 2026 13:07

@yujiachen-y yujiachen-y left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stamping!

@SaladDay
SaladDay merged commit 35f73f7 into main Sep 21, 2026
23 checks passed
@SaladDay
SaladDay deleted the fix/history-encoding-overhead branch September 21, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

perf:full Run the full performance suite; required for performance-sensitive changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants