Repository navigation
Local agent: latency, context isolation, reliable lists - #93
Merged
Merged
Conversation
Endpoint lab (probe/cache/replay) measures oMLX prefix-cache behavior with synthetic prompts. The rig runs the real agent image against a demo vault, a lab famstack-api, and a logging proxy that records the byte-exact context per LLM call. Review and measurement log included. - lab.py: TTFT/usage measurement, deterministic prompts - rig: nanobot agent turns without Matrix, fabricated demo data only - findings: 4096-token cache blocks, 430-460 tok/s prefill, 11.2K prefix
Measured in the agent-lab rig: simple questions dropped from 42-60 s to 23-24 s, warm TTFT from ~10 s to ~4 s, all replies still correct. - config.json: disable unused nanobot builtin skills (memory, my) - entrypoint.sh: seed minimal USER.md, replaces the ~1K-char template - AGENTS.md, SKILL.md: token-lean rewrite; brief-pointer-first and one-search rule; prefix 11,281 -> 9,184 tokens - memory_tool.py: build an OR-regex from model keywords; removes the --nl rewrite, a second LLM call inside every multi-word search - sitecustomize.py: AGENT_LEAN_STATE=0 switch for append-only A/B
Store matches the named item, changes one line, refuses ambiguity. Allowlisted for the agent; 8 module tests.
…kill Thread-scoped session keys, 6 unused tool schemas dropped, batched memory_search, cache-pad alignment, max_messages 40.
Bulk clear/reset in one call; edit verbs return the touched page's /go link for the reply's Sources. German rig run green.
Single items[] param, one commit, changed and missed items reported together. Rig runs the production transform. Validated end-to-end.
Stub gains ArraySchema, tools.registry, real _handle_message signature. Removal-via-write now refused (goes through list_edit).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the local chat agent's 3-5 minute responses, context bleed between
threads, and unreliable list editing. All validated in an isolated rig
(real agent image, demo vault, logging proxy to oMLX); see
docs/design/agent/review-2026-09.mdandagent-improvement-log.md.Latency
Simple grounded questions dropped from 42-60 s to ~17-24 s, warm TTFT from
~10 s to ~2-4 s.
--nlkeywordrewrite); the model supplies keywords, joined as an OR-regex.
~8.2K prefix).
zz-cache-padskill aligns the prefix just past oMLX's 4096-token cacheblock boundary (measured: misalignment doubles per-call prefill).
memory_search.Context isolation
transcript.
max_messagescap for long threads.Lists
stack memory list-editverb: add/tick/untick/remove/clear-done/reset,one or more items per call, one commit; the store matches items and
refuses ambiguity instead of guessing.
remove list items.
Tooling
tools/agent-lab/: endpoint lab + end-to-end rig for measuring agentlatency and context in isolation (synthetic data only).
Env kill switches on each behavior change (
AGENT_LEAN_STATE,AGENT_TOOL_TRIM,AGENT_THREAD_SESSIONS). Not yet tested on live Matrix.