diff --git a/.router/2026-10-14-gemma-codemap-validation/evidence.md b/.router/2026-10-14-gemma-codemap-validation/evidence.md new file mode 100644 index 00000000..c1d0625f --- /dev/null +++ b/.router/2026-10-14-gemma-codemap-validation/evidence.md @@ -0,0 +1,61 @@ +# Evidence: validate 7 improvements (2026-10-14, throwaway copies only) + +No live file edited. Live `~/.pi-pocket/config.json`, `skills.json`, routers, homelab config, `src/server` untouched. No server restart, no model load/evict initiated (embedding calls hit already-resident CPU models), no VM250/proxmox contact, no secret values printed (ownerToken redacted everywhere). + +## 0. Baseline (SEEN) + +- `src/server/reload.ts` ORDER (lines 29-47): `read-economy.ts`, `loop-guard.ts`, `pruner.ts`, `tool-repair.ts` are built-ins; `OFF_BY_DEFAULT={"guard.ts"}` only — the four batteries are on-by-default code, toggled via `enabledExtensions`. +- Extension headers: `loop-guard.ts` (3rd identical call blocked, says so), `pruner.ts` (mask-then-maybe-summarize), `read-economy.ts` (≤500 lines + structural map), `tool-repair.ts` (repair JSON-string args before validation). +- Live `config.json` `enabledExtensions`: `['artifacts.ts','codemode.ts','goals.ts','hq-mcp.ts','hq-router.ts','memory.ts','pip.ts','schedules.ts','subagents.ts','triggers.ts']` — the four batteries NOT enabled; `ownerToken`/hashes not quoted. +- llama-swap config: `text-embedding-nomic-embed-text-v1.5` alias `embed` (Q4, CPU `-ngl 0`, ctx 2048, mean pooling); `embeddinggemma-2` alias `embed2` (Q8_0 310MiB, CPU, ctx 8192, mean pooling, trial alongside, undo = delete entry + group member line); both in non-exclusive persistent `embeddings` group. `Gemma4-26B-A4B-QAT` alias `gemma4-26b`, chat-group member, BF16-KV-only, no prefix reuse, top_k≤20, vision not wired, MTP full-head-only, load 34s / 17,454MiB @131072 (2026-10-13 aphotic comment). +- `muse-router.json`: model `opencode-go/muse-spark-1.3-contributor`, thinking high, pipeline plan→implement→review→fix→ship with skills poteto-mode/blast-radius/unslop. +- `skills.json` keys (19): router 21, plan 20, coder 13, agent 10, review 12, writer 5, thinker 4, uncensored 9, agent-fast 2, coder-fast 3, fast-plan 20, fast-coder 13, fast-agent 10, fast-writer 5, strata-* 5 roles. `coder` list verified (13 skills incl. codebase, repo-work, tdd). +- `hq-router.ts`: `router-fast`→ROUTER-FAST.md; FAST_BLOCKED={bash,edit,write}; block is per-conversation via `fastRouters` set keyed by **model id** `router-fast` (`id === "router-fast"` in hq_router section render); skill-trim from skills.json by ``, `/skill:` unaffected. +- ROUTER-FAST.md: dispatcher read-only (bash/edit/write blocked), width 1, paths-not-contents, `/log.tsv` stage log. + +## 1. embed2 cutover — VALIDATED (copy-DB) + +- TESTED: both endpoints live on loopback :9090: nomic dim **768**, gemma dim **768** (measured via `/v1/embeddings`, not assumed). +- TESTED: copy DB reindex under gemma only: `CODEVEC_DB=/tmp/codevec-copy.db CODEVEC_EMBED_MODEL=embeddinggemma-2 index_repo(omp-copy, /home/tcrypt/projects/oh-my-pocket)` → 273 files, 2334 chunks, no error. +- VALIDATED (with one contaminated query — see next line): same 3 queries on copy-DB hit sane files: "loop guard…"→handoff.sample.json; "read economy…"→test/read-economy.test.ts:1-60; "router stage prompt"→src/server/routers.ts:1-60. +- CONTAMINATED (fix 2026-10-14): Q1 ("loop guard…") top-hit `handoff.sample.json` — a file this run itself wrote, indexed as part of the 273-file copy (run folder was inside the indexed tree), so Q1 is circular. Q2–Q3 hit real product files, dims (768/768) plus the 2334-chunk reindex are solid, so VALIDATED-on-copy stands on Q2+Q3. Required before any production cutover decision: re-run Q1 with `.router/` excluded from the indexed tree (or quote a product-file hit for the same query terms). +- UNCLEAR (stated, not guessed): whether pre-existing nomic vectors stay comparable under gemma cosine — codevec_mcp.py comment says re-index required; copy test re-indexed from scratch so comparability untested. Cutover recommendation: keep `embed` alias until a mixed-DB test passes. Undo: delete `embeddinggemma-2` entry + embeddings-group member line (config comment). PROVEN not claimed (no `git diff --stat` of production hunks — nothing applied). + +## 2. Index coverage — TESTED (gap found, live DB untouched) + +- `list_repos` → exactly **1 repo: homelab** (1108 files, 4703 chunks, indexed 2026-10-14T13:53:51Z). **oh-my-pocket is NOT indexed** in the live DB — coverage gap. +- Live-DB `search_code(repo:homelab)`: "loop guard…" hit harnesses/pi/extensions/loop-guard.ts (stale Pi harness copy, not Pi Pocket's); "read economy…" MISSED (top hits unrelated audit/economy docs); "router stage…" hit hq-router.ts. So Pocket code is unsearchable live. +- Copy-DB (`omp-copy`, gemma): all 3 queries hit Pocket files (see §1). VALIDATED-on-copy that indexing fixes it; live `index_repo` NOT run (deferred, needs operator decision + embedder choice). +- `git ls-files | wc -l` = 299 vs copy-index 273 files (SKIP_DIRS/.md-filter explain delta; not investigated further). + +## 3. Batteries — TESTED + unit-VALIDATED (per-behavior exercises pending) + +- TESTED: `node test/serve.ts` copy boots clean on 127.0.0.2:8899 twice (logs show sign-in links, no errors). +- unit-VALIDATED: throwaway `openApp(scriptedModel())` + `setExtensionEnabled` for all four → loader names `pocket-read-economy, pocket-loop-guard, pocket-pruner, pocket-tool-repair` all present (1/1 pass). +- unit-VALIDATED: unit tests `loop-guard + pruner + read-economy` 33/33 pass; `app.test.ts` 32/32 pass (covers guard toggle path). +- Rating wording fix (2026-10-14, per review finding 5): unqualified VALIDATED overstated against plan step 3's bar (VALIDATED = steps 2–4 all pass on the copy, where step 3 is the four per-behavior exercises). Correct rating is **TESTED + unit-VALIDATED** until the implement stage runs the four behaviors on the copy. +- Per-behavior UI exercises (500-line map, malformed-call repair, triple-call block, prune note) NOT run — scripted model + no browser; stated as not verified. Production undo: Menu → Extensions off (no file change; built-ins stay in ORDER). + +## 4. muse-router fast path — TESTED (candidate in run folder) + +- `muse-router-fast.json` drafted in run folder only; `json.load` parses ("json ok"). +- CONFIRMED CONSTRAINT (fix 2026-10-14, was UNCLEAR — code path verified per review finding 2, `hq-router.ts` ll.117-130, 172-180): dispatcher block keys off **model id** `router-fast`, not router-editor id. `fastRouters` only ever contains conversations whose last rendered router model id was literally `router-fast`. A new `muse-router-fast` router id running on a cloud model gets **no tool block** without either (a) running on model `router-fast`, or (b) a code change extending `PROMPTS`/`routerModel()` id matching. Do not claim width-1/dispatcher behavior for the new id until one of those lands. +- Safe side (verified, not previously recorded): the skill-trim side does NOT break for the new id — `hqRole` (`hq-router.ts` ~91-104) returns `"router"` for any router-editor session on any model, and `"router"` exists in `skills.json` (21 skills). So the new id gets the router skill list, just no tool block. +- Copy-server stage-dispatch dry run NOT performed (needs router session on copy; stated as not verified). + +## 5. Per-role skills — TESTED + +- SEEN: counts per role (see §0), trim source `skills.json[hq-role]`, renderer `hq-router.ts` wrapSection("skills"), roles source `~/homelab/harnesses/agents/roles/` (roles.yaml + per-role md) rendered by `~/homelab/scripts/agents/apply_roles.py` — path confirmed (`ls` both dirs). +- Trimmed-list == skills.json[role] check NOT run on copy (needs cloud-model session render; stated as not verified). Token delta not measured on copy (not exposed); code-level estimate stands (102 skills ~6.8K tokens → role list). + +## 6. Typed handoffs — TESTED + +- `handoff.schema.json` + `handoff.sample.json` written in run folder; `json.load` both ("schema+sample ok"). +- Sample carries goal/project/worktree/findings[file:line:claim]/nextSteps/outOfScope. +- Consumption test (plan-shaped stage reading sample on copy) NOT run — stated as not verified. Production home open choice: (a) convention-only prompt text, (b) template beside ROUTER*.md, (c) enforced schema in hq-router.ts. + +## 7. Gemma4 decision — SEEN + prior-run TESTED (no load by this plan) + +- SEEN: block quoted (§0). Prior-run measurements (config comment, 2026-10-13 aphotic): load 34s, 17,454MiB @131072, tool_calls correct args — recorded as TESTED-by-prior-run, not re-proven. +- TESTED (read-only): `git log --oneline -5 -- reference/llama-swap/config.yaml` → 03f3b16, 9d19163, a69c4da, 121f08f, 646c7e0; `Gemma4-26B-A4B-QAT` entry + chat-group member line present. +- Decision matrix: (a) adopt as router/NInfer role — blocked on prefix-reuse + vision gaps; (b) opt-in alias for text/tool trials on copy — viable now; (c) defer. Copy-server session on gemma id explicitly OUT OF SCOPE (implement stage's first step); no load/evict performed here. diff --git a/.router/2026-10-14-gemma-codemap-validation/fix.md b/.router/2026-10-14-gemma-codemap-validation/fix.md new file mode 100644 index 00000000..50f746e4 --- /dev/null +++ b/.router/2026-10-14-gemma-codemap-validation/fix.md @@ -0,0 +1,30 @@ +# Fix report: gemma-codemap review findings (2026-10-14) + +Run-folder only. No live config/skill/router/src edit, no restart, no model load, no VM250 contact, no secrets printed. + +## Finding 1 — `muse-router-fast.json` dispatcher object dropped by loader +- Changed: top-level `"dispatcher": {blockedTools..., note...}` → `"dispatcher": true`, plus a `_dispatcherNote` sidecar string carrying the width-1 intent, blockedTools list, and the real enforcement pointer (`hq-router.ts` `fastRouters` keyed on model id `router-fast`). +- Proof: `node` eval of the exact loader expression from `src/server/routers.ts:541` (`...(raw.dispatcher === true ? { dispatcher: true } : {})`) against the fixed file → `loader keeps dispatcher: {"dispatcher":true}`. Before the fix the object form failed the `=== true` check and was stripped silently. + +## Finding 2 — fast-id tool block absent, skill side safe +- Changed: `evidence.md` §4 rewritten from "UNCLEAR … finding" to **CONFIRMED CONSTRAINT** (code path `hq-router.ts` ll.117-130, 172-180 verified: `fastRouters` only contains conversations whose last rendered router model id was literally `router-fast`; new id `muse-router-fast` on a cloud model gets no tool block without running on model `router-fast` or extending `PROMPTS`/`routerModel()` id matching). Added the verified safe side: `hqRole` (~91-104) returns `"router"` for any router-editor session on any model and `"router"` exists in `skills.json` (21 skills) — new id gets the router skill list, just no tool block. +- Proof: code re-read at the cited lines during this fix; `muse-router-fast.json` `_dispatcherNote` now names the same enforcement point so a future copy-paste into `~/.pi-pocket/routers/` carries the warning. + +## Finding 3 — `handoff.sample.json` line refs wrong +- Changed: finding 1 `src/server/reload.ts:42` → `:44`; finding 2 `loop-guard.ts:1` → `:2` with the claim text noting the behavior sentence spans lines 2-3 (schema requires integer `line`, so the range lives in the claim, not the number). +- Proof: `sed -n '43,45p' src/server/reload.ts` → line 44 is `"loop-guard.ts",`; `sed -n '2,3p' src/server/extensions/loop-guard.ts` → the third-identical-call-blocked sentence. Schema check passes: required keys (`goal, project, findings, nextSteps`; per-finding `file, line, claim`) present and `line` is int — a `{"file","claim"}`-without-`line` object would now fail this check, closing the "parses fine while violating required" gap the review demonstrated. + +## Finding 4 — embed2 Q1 self-referential hit +- Changed: `evidence.md` §1 VALIDATED line qualified; added explicit CONTAMINATED note: Q1 ("loop guard…") top-hit `handoff.sample.json`, a file this run authored inside the indexed 273-file copy tree — 1 of 3 queries circular. VALIDATED-on-copy now rests on Q2+Q3 (real product files) + 768/768 dims + 2334-chunk reindex. +- Proof/required follow-up (stated in evidence, not runnable from here without reindexing a copy): re-run Q1 with `.router/` excluded from the indexed tree before any production cutover decision. Mixed nomic+gemma vector comparability remains UNCLEAR as before. + +## Finding 5 — batteries VALIDATED overstates plan bar +- Changed: `evidence.md` §3 heading/rating → **TESTED + unit-VALIDATED (per-behavior exercises pending)** with the reasoning (plan step 3 requires the four per-behavior exercises: 500-line map, malformed-call repair, triple-call block, prune note — explicitly not run, no browser/scripted-model UI run). Same one-line correction applied to the Ratings line in `implement.md`. +- Proof: nothing hidden — the "not verified" list already said so; only the rating word changed. + +## Tests rerun +- `node --test test/loop-guard.test.ts test/pruner.test.ts test/read-economy.test.ts` → **33/33 pass**. +- `node --test test/app.test.ts` → **32/32 pass**. +- `git status --short` → only untracked `.router/` + pre-existing untracked `.zvec-grep/`; `git diff --stat` empty (zero tracked-file modifications). + +SELF-CHECK: each of the 5 review findings fixed in the run folder only (fast JSON dispatcher true, handoff line refs, evidence wording batteries TESTED+unit-VALIDATED, embed2 Q1 contamination + rerun note, fast-id confirmed-constraint + skill-safe note); proofs run and outputs read (loader expr, schema check, sed line verification, 33/33 + 32/32 tests, git status clean); no live config/skill/router/src edit, no restart, no model load/evict, no VM250/proxmox contact, no secrets printed. diff --git a/.router/2026-10-14-gemma-codemap-validation/handoff.sample.json b/.router/2026-10-14-gemma-codemap-validation/handoff.sample.json new file mode 100644 index 00000000..6096f535 --- /dev/null +++ b/.router/2026-10-14-gemma-codemap-validation/handoff.sample.json @@ -0,0 +1 @@ +{"findings":[{"claim":"Loop-guard ORDER position (line 44), after read-economy (line 43); built-in, opt-in via Extensions sheet","file":"src/server/reload.ts","line":44},{"claim":"Third exactly identical tool call in one turn is blocked, and the block says so (behavior sentence, lines 2-3)","file":"src/server/extensions/loop-guard.ts","line":2}],"goal":"Validate loop-guard battery on copy server","instructionFiles":["docs/self-editing.md"],"nextSteps":["Enable loop-guard on copy Extensions sheet","Run three identical failing calls, confirm third blocked"],"outOfScope":["Live config.json edit"],"project":"oh-my-pocket","worktree":"/home/tcrypt/projects/oh-my-pocket"} \ No newline at end of file diff --git a/.router/2026-10-14-gemma-codemap-validation/handoff.schema.json b/.router/2026-10-14-gemma-codemap-validation/handoff.schema.json new file mode 100644 index 00000000..1f44b5a9 --- /dev/null +++ b/.router/2026-10-14-gemma-codemap-validation/handoff.schema.json @@ -0,0 +1 @@ +{"$schema":"http://json-schema.org/draft-07/schema#","description":"Typed handoff between muse-router stages. Paths, not contents; every claim carries file:line.","properties":{"findings":{"items":{"properties":{"claim":{"type":"string"},"file":{"type":"string"},"line":{"type":"integer"}},"required":["file","line","claim"],"type":"object"},"type":"array"},"goal":{"type":"string"},"instructionFiles":{"items":{"type":"string"},"type":"array"},"nextSteps":{"items":{"type":"string"},"type":"array"},"outOfScope":{"items":{"type":"string"},"type":"array"},"project":{"type":"string"},"worktree":{"type":"string"}},"required":["goal","project","findings","nextSteps"],"title":"muse-router handoff","type":"object"} \ No newline at end of file diff --git a/.router/2026-10-14-gemma-codemap-validation/implement.md b/.router/2026-10-14-gemma-codemap-validation/implement.md new file mode 100644 index 00000000..f43d8057 --- /dev/null +++ b/.router/2026-10-14-gemma-codemap-validation/implement.md @@ -0,0 +1,38 @@ +# Implement report: gemma-codemap validation (2026-10-14) + +Validation-only. No production file changed: `git status --short` shows only untracked `.router/` (this run) and pre-existing untracked `.zvec-grep/`; zero modifications to tracked files. (`.zvec-grep/` is not mine — never wrote there.) + +## What changed (run folder only, all new files) + +- `.router/2026-10-14-gemma-codemap-validation/evidence.md` — per-improvement SEEN/TESTED/VALIDATED log (the full record; see it, not this file, for quotes and numbers). +- `.router/2026-10-14-gemma-codemap-validation/muse-router-fast.json` — fast-path candidate draft (parses; NOT registered). +- `.router/2026-10-14-gemma-codemap-validation/handoff.schema.json` + `handoff.sample.json` — typed-handoff schema + sample (both parse). +- `.router/2026-10-14-gemma-codemap-validation/plan.md` — input (pre-existing). +- This file. + +## Commands + key output + +- Baseline reads: ORDER/batteries headers quoted; live `enabledExtensions` = 10 drop-ins, four batteries absent; embed2 + Gemma4 config blocks quoted; `muse-router.json` pipeline plan→implement→review→fix→ship; `skills.json` 19 roles (coder=13 skills); `hq-router.ts` fast-block + skill-trim logic read. +- Embed dims (loopback :9090, resident CPU models, no load initiated): nomic **768**, gemma **768**. +- Copy-DB reindex under gemma (`CODEVEC_DB=/tmp/codevec-copy.db`, MCP stdio): 273 files, 2334 chunks. Same 3 queries hit `test/read-economy.test.ts`, `src/server/routers.ts` (+ own sample file). → embed2 VALIDATED on copy. +- `list_repos` (live): only `homelab` (1108 files, indexed today) — **oh-my-pocket unindexed, coverage gap**; live "read economy" query MISSED. +- Copy server `node test/serve.ts` booted clean twice on 127.0.0.2:8899 (sign-in links, no errors). +- Throwaway enable test (scripted model, temp data): all four batteries → loader names present, 1/1 pass. Deleted after. +- Unit tests: loop-guard+pruner+read-economy **33/33**; `app.test.ts` **32/32**. +- Fast-path constraint found: `hq-router.ts` block keys off model id `router-fast`, not router id — new `muse-router-fast` id gets no tool block without running on `router-fast` model or a code change. +- Gemma4: git log 5 entries + entry/member lines present (read-only); prior-run load figures recorded, not re-proven; no load/evict here. +- Cleanup: copy servers killed (temp data auto-deleted), `/tmp/codevec-copy.db` + temp req/test files removed. + +## What not verified + +- Nomic-vs-gemma vector comparability in a mixed DB (copy test re-indexed from scratch). +- Live `index_repo` for oh-my-pocket (deferred to operator: needs embedder choice). +- Per-behavior battery exercises (500-line map, malformed-call repair, triple-call block, prune note) — no browser/scripted-model UI run. +- Copy-server stage-dispatch dry run for fast path; skill-trim render on a cloud-model session + token delta; handoff consumption test by a plan-shaped stage. +- Gemma4 copy-server multi-turn + tool-call transcript (implement stage's first step; GPU shared). + +## Ratings + +embed2 VALIDATED (copy; Q1 marked contaminated, re-query with .router/ excluded required before cutover) · coverage TESTED (gap found) · batteries TESTED + unit-VALIDATED (per-behavior exercises pending) · fast-path TESTED (constraint confirmed: block keys off model id router-fast; skill-trim safe for new id) · skills TESTED · handoffs TESTED (sample line refs corrected) · gemma4 SEEN + prior TESTED. + +SELF-CHECK: plan followed step by step; only throwaway copies touched (copy server temp dirs, CODEVEC_DB=/tmp/...); no live config/skill/router/src edit, no restart, no model load/evict, no VM250/proxmox contact, no secrets printed; proof commands run and outputs read; tests 33/33 + 32/32 + 1/1 green; git status confirms no tracked-file changes; evidence.md written with UNCLEAR items stated, not guessed. diff --git a/.router/2026-10-14-gemma-codemap-validation/muse-router-fast.json b/.router/2026-10-14-gemma-codemap-validation/muse-router-fast.json new file mode 100644 index 00000000..8a5cca04 --- /dev/null +++ b/.router/2026-10-14-gemma-codemap-validation/muse-router-fast.json @@ -0,0 +1 @@ +{"dispatcher": true, "id": "muse-router-fast", "label": "muse-router-fast", "model": "opencode-go/muse-spark-1.3-contributor", "pipeline": [{"id": "plan", "instructions": "Read the repository's own instruction files and the code the task touches. Write plan.md in the run folder: the goal in one sentence, the files to change and how, the steps in order each with the command that proves it, and the risks. Change no other file. If the task is unclear, say so in plan.md instead of guessing.", "name": "Plan", "skills": ["poteto-mode", "blast-radius"], "template": "plan"}, {"id": "implement", "instructions": "Carry out plan.md step by step. After each step, run its check and read the output. A step that is wrong or impossible: stop and say which and why. Keep the change to what the plan asks, and run the project's tests at the end.", "name": "Implement", "skills": ["poteto-mode"], "template": "implement"}, {"id": "review", "instructions": "Review the change (git diff) against plan.md: run the tests, try the inputs most likely to break it, and read the code that calls what changed. Write review.md in the run folder: each finding with file, line, the input that breaks it, and what goes wrong; or say plainly that it is clean. Edit no other file.", "name": "Review", "skills": ["poteto-mode"], "template": "review"}, {"id": "fix", "instructions": "Fix each finding in review.md, and only those. Run the tests again, and say for each finding what you changed and what proves it fixed.", "name": "Fix", "skills": ["poteto-mode"], "template": "fix"}, {"id": "ship", "instructions": "Commit the change on a new branch (never the default branch), push it, and open a draft pull request that says what changed and what the stages proved. Never merge. Answer with the branch and the pull request's link.", "name": "Ship", "skills": ["poteto-mode", "unslop"], "template": "ship"}], "thinking": "high", "_dispatcherNote": "Sidecar only (dropped on registration): width-1 intent + blockedTools [bash, edit, write]. Real enforcement is NOT this file: hq-router.ts fastRouters keys off model id router-fast, so id muse-router-fast gets no tool block until routerModel()/PROMPTS id matching is extended (see evidence section 4)."} \ No newline at end of file diff --git a/.router/2026-10-14-gemma-codemap-validation/plan.md b/.router/2026-10-14-gemma-codemap-validation/plan.md new file mode 100644 index 00000000..00048f30 --- /dev/null +++ b/.router/2026-10-14-gemma-codemap-validation/plan.md @@ -0,0 +1,197 @@ +# Plan: validate 7 Oh-My-Pocket improvements (gemma-codemap run, 2026-10-14) + +## Goal (one sentence) + +Validate all 7 proposed improvements with SEEN / TESTED / VALIDATED / PROVEN evidence on throwaway copies before any global change to Pi Pocket, llama-swap config, indexes, or skills. + +## Evidence ladder (used by every step) + +- **SEEN:** file read and quoted (path + line); no behavior executed. +- **TESTED:** exercised on a copy or with a read-only command; output captured. +- **VALIDATED:** check command passes on the copy (e.g. `npm run check`, `node --test`, copy-server log clean, side-by-side comparison numbers). +- **PROVEN:** validated on the copy **plus** undo path shown (what file to restore, what command reverses it) and no live-data / restart / secret exposure involved. + +No step may touch live data (`~/.pi-pocket/` in place), restart the production server, touch VM 250 / 10.0.0.184, or print a secret value (refer to secret PATHs only). + +## Files to change and how (all validation-only; production edits come later) + +All work happens in this run folder plus throwaway copies. Expected production targets (NOT edited in this plan): + +1. **embed2 cutover** — prod target: `~/homelab/reference/llama-swap/config.yaml` (`embeddinggemma-2`/`embed2` entry + `embeddings` group) and `~/homelab/scripts/codevec_mcp.py` (`CODEVEC_EMBED_URL`, `index_repo` default model). Validation only reads them and queries both endpoints read-only. +2. **Index coverage** — prod target: codevec SQLite index (default `~/.config/opencode/codevec/index.db`). Validation only: `list_repos` + `search_code` reads; any reindex goes to a **copy** of the DB (`CODEVEC_DB=/tmp/...`), never the live DB. +3. **Batteries (loop-guard / pruner / read-economy / tool-repair)** — prod target: `~/.pi-pocket/config.json` `enabledExtensions` (currently missing the four; built-ins exist in `src/server/extensions/` and `ORDER` in `src/server/reload.ts`). Validation only: enable them on a `node test/serve.ts` copy with its own temp data dir, never the live `config.json`. +4. **muse-router fast path** — prod target: `~/.pi-pocket/routers/muse-router.json` (+ `ROUTER-FAST.md` pattern) and/or a new `muse-router-fast.json`. Validation only: read the JSON, draft candidate in run folder, dry-run stage dispatch on the copy. +5. **Per-role skills for cloud** — prod target: `~/.pi-pocket/hq/skills.json` (rendered by `~/homelab/scripts/agents/apply_roles.py` from `roles.yaml`) + `~/.pi-pocket/extensions/hq-router.ts` skill-trim section. Validation only: read `skills.json` keys, test skill-trim prompt rendering on the copy with a cloud model id; no edit to live `skills.json`. +6. **Typed handoffs** — prod target: router prompt convention (`/handoff.md`: goal, paths, file:line) in `ROUTER*.md`, possibly a schema/template file. Validation only: write a sample typed `handoff.md` + schema in run folder, have a stage-shaped read test consume it on the copy. +7. **Gemma4 decision** — prod target: `~/homelab/reference/llama-swap/config.yaml` (`Gemma4-26B-A4B-QAT` entry, chat-group member) and router model picks (`ROUTER.md`, `~/.pi-pocket/routers/`). Validation only: read-only load/smoke evidence already in config comments + read-only probe; no config edit, no model load initiated by this plan (GPU is shared). + +Run-folder outputs (the only files this plan writes): + +- `.router/2026-10-14-gemma-codemap-validation/plan.md` (this file) +- `.router/2026-10-14-gemma-codemap-validation/evidence.md` (per-improvement SEEN/TESTED/VALIDATED/PROVEN log) +- `.router/2026-10-14-gemma-codemap-validation/` + +## Steps in order (each with its proof command) + +### 0. Baseline reads (SEEN for all 7) + +- Read `src/server/reload.ts` lines 29–85 (ORDER/TITLES/OFF_BY_DEFAULT — batteries are built-in, opt-in via Extensions sheet). SEEN when quoted. +- Read `src/server/extensions/loop-guard.ts`, `pruner.ts`, `read-economy.ts`, `tool-repair.ts` headers (first ~30 lines each). SEEN when quoted. +- Read live `~/.pi-pocket/config.json` `enabledExtensions` (redact `ownerToken`, hashes — never paste values). SEEN = list of enabled vs missing four. +- Read `~/homelab/reference/llama-swap/config.yaml` Gemma4 block (~lines 450–483), embed2 block (~657–678), routing groups (~684–718). SEEN when quoted. +- Read `~/.pi-pocket/routers/muse-router.json` (full, ~60 lines) and `~/.pi-pocket/hq/skills.json` keys + one role list. SEEN when quoted. +- Read `~/.pi-pocket/extensions/hq-router.ts` header (~first 60 lines: skill-trim + fast-profile block) and `~/.pi-pocket/hq/ROUTER-FAST.md` handoff sections. SEEN when quoted. + +Proof commands (read-only, no secrets printed): + +```sh +sed -n 29,85p src/server/reload.ts +head -n 30 src/server/extensions/loop-guard.ts src/server/extensions/pruner.ts src/server/extensions/read-economy.ts src/server/extensions/tool-repair.ts +python3 -c "import json; d=json.load(open('/home/tcrypt/.pi-pocket/config.json')); print(d['enabledExtensions'])" +grep -n "Gemma4\|embed2\|embeddinggemma\|embeddings" /home/tcrypt/homelab/reference/llama-swap/config.yaml +python3 -c "import json; print(json.load(open('/home/tcrypt/.pi-pocket/routers/muse-router.json'))['pipeline'][0])" +python3 -c "import json; d=json.load(open('/home/tcrypt/.pi-pocket/hq/skills.json')); print(sorted(d.keys()))" +``` + +### 1. embed2 cutover validation (read-only, no reindex of live DB) + +1. Confirm both embedding entries coexist: `embed` → nomic (2048 ctx, CPU), `embed2` → embeddinggemma-2 (8192 ctx, CPU), both in `embeddings` group (persistent, non-exclusive). SEEN via step 0. +2. Probe both endpoints read-only (dimensionality + a fixed query string; record dims: nomic 768d expected, gemma 768d with MRL note in config comment — codevec comment says "same 768d" for nomic; verify gemma dim from response, do NOT assume). +3. Compare retrieval quality on 3–5 fixed natural-language queries against the `oh-my-pocket` repo via `search_code` with each embedder (whichever the MCP currently points at; note which). Record top-k paths per query in `evidence.md`. TESTED when both result sets are captured. +4. UNCLEAR (say so in evidence.md): whether existing nomic vectors stay comparable under gemma (config comment says trial alongside, `embed` untouched so existing vectors compare; codevec_mcp.py comment line ~41 says existing nomic vectors must be re-indexed). Do NOT reindex live. VALIDATED only when a **copy** DB reindex under gemma completes and the same queries return sane hits. +5. Cutover recommendation + undo: keep `embed` alias until copy-DB test passes; undo = delete `embeddinggemma-2` entry + its group member line (as the config comment states). PROVEN requires steps 2–4 plus undo line verified by `git diff --stat` showing only the two config hunks would change (do not apply). + +Proof commands: + +```sh +grep -n -A8 '"embeddinggemma-2"\|"text-embedding-nomic' /home/tcrypt/homelab/reference/llama-swap/config.yaml +sed -n 1,60p /home/tcrypt/homelab/scripts/codevec_mcp.py +# read-only probes via MCP search_code / list_repos (no live reindex): +# search_code({repo:'oh-my-pocket', query:'', k:5}) x3-5, recorded per embedder +``` + +### 2. Index coverage validation (read-only + copy-DB only) + +1. `list_repos` → record indexed repos, chunk counts, last-indexed times. TESTED when output is saved to run folder. +2. `get_file_map` for `oh-my-pocket` → compare file list against `git ls-files | wc -l` for staleness estimate. TESTED. +3. 3 fixed `search_code` queries (e.g. "loop guard identical tool call block", "read economy truncation map", "router stage prompt") → record hit/miss. A miss = coverage gap, not a code bug. +4. If a gap: `index_repo` ONLY against a copy (`CODEVEC_DB=/tmp/codevec-copy.db`), then re-run the 3 queries. VALIDATED when copy-index queries hit. Never run `index_repo` against the live DB from this plan. + +Proof commands: + +```sh +# via codebase MCP: list_repos, get_file_map({repo:'oh-my-pocket'}) +git ls-files | wc -l +CODEVEC_DB=/tmp/codevec-copy.db python3 /home/tcrypt/homelab/scripts/codevec_mcp.py --help 2>&1 | head -n 20 +``` + +### 3. Batteries validation (copy server only) + +1. Start copy: `nohup node test/serve.ts > /tmp/pocket-copy-batteries.log 2>&1 & sleep 5; cat /tmp/pocket-copy-batteries.log`. TESTED when sign-in links print and log shows no errors. +2. In the copy (via its sign-in link / API, never live): turn on `read-economy.ts`, `loop-guard.ts`, `pruner.ts`, `tool-repair.ts` in the Extensions sheet; confirm reload notices, no errors in copy log. +3. Exercise each on the copy: + - read-economy: read a >500-line file without limit → expect ≤500 lines + structural map of remainder. + - tool-repair: submit a slightly malformed tool call (object-as-JSON-string) via codemode on the copy → expect repaired, not a validation failure. + - loop-guard: three exactly identical failing calls in one turn on the copy → expect third blocked with "says so" message. + - pruner: compaction-shaped input (or existing pruner test) → expect masked tool output note before any model summary. +4. Run the related unit tests (they do not need the copy): TESTED → VALIDATED when green. + +Proof commands: + +```sh +nohup node test/serve.ts > /tmp/pocket-copy-batteries.log 2>&1 & sleep 5; cat /tmp/pocket-copy-batteries.log +node --test test/app.test.ts 2>&1 | tail -n 20 +grep -rln "loop-guard\|read-economy\|pruner\|tool-repair" test/ 2>&1 +# stop copy with the kill it printed (also deletes its data); never kill -9 +``` + +VALIDATED = steps 2–4 all pass on the copy with log excerpts in `evidence.md`. PROVEN = plus the production undo stated (turn off in Menu → Extensions; no file change needed since built-ins stay in `ORDER`). + +### 4. muse-router fast path validation + +1. SEEN: quote `muse-router.json` pipeline (plan → implement → review → fix → ship) and `ROUTER-FAST.md` dispatcher rule (bash/edit/write blocked, width 1, out-of-scope → `handoff.md`). +2. Draft `muse-router-fast.json` candidate IN THE RUN FOLDER ONLY (not `~/.pi-pocket/routers/`): same pipeline with fast model (`agent-fast`/`router-fast`), dispatcher tool block, width-1 note, handoff-on-escape step. TESTED when the JSON parses. +3. Dry-run on the copy server: register the candidate as a test router (copy data dir only) or simulate stage dispatch (plan-stage prompt render) and confirm dispatcher block triggers on an edit attempt. VALIDATED when copy log / test output shows the block + `handoff.md` path suggested. +4. UNCLEAR to state in evidence.md: whether `hq-router.ts` tool-blocking is per-conversation and applies to a new router id without code change (header says "per conversation, so its stages keep their tools" for `router-fast`; verify the id-matching logic before claiming the fast path works for a new id). + +Proof commands: + +```sh +python3 -c "import json; json.load(open('.router/2026-10-14-gemma-codemap-validation/muse-router-fast.json')); print('json ok')" +grep -n "router-fast\|blocked\|per conversation\|Width" /home/tcrypt/.pi-pocket/hq/ROUTER-FAST.md | head -n 20 +grep -n "router-fast\|routerFast\|blocked\|prepareArguments\|beforeTool" /home/tcrypt/.pi-pocket/extensions/hq-router.ts | head -n 30 +``` + +### 5. Per-role skills for cloud validation + +1. SEEN: `skills.json` role keys (`router, plan, coder, agent, review, writer, thinker, uncensored, agent-fast, coder-fast`), one full role list, and the `hq-router.ts` skill-trim paragraph (102 skills ~6.8K tokens → session's own list; `/skill:` still loads any). +2. On the copy server: open a session whose model id is a cloud model (or the closest available non-`router*` id) with a role marker (`` style) and confirm the prompt carries only that role's skills while `/skill:` still resolves. TESTED when copy prompt/log excerpts are saved. +3. Measure token delta (full catalog vs trimmed) from the copy's rendered prompt if exposed; otherwise record "not exposed on copy, estimate from code". VALIDATED when trimmed list == `skills.json[role]` exactly (diff command output saved). +4. UNCLEAR to state: which file renders cloud-role prompts (`apply_roles.py` + `roles.yaml` live in `~/homelab/harnesses/agents/` — confirm path before recommending an edit). + +Proof commands: + +```sh +python3 -c "import json; d=json.load(open('/home/tcrypt/.pi-pocket/hq/skills.json')); print({k: len(v) for k,v in d.items()})" +grep -n "skills.json\|hq-role\|skill" /home/tcrypt/.pi-pocket/extensions/hq-router.ts | head -n 30 +ls /home/tcrypt/homelab/harnesses/agents/roles/ 2>&1 | head; ls /home/tcrypt/homelab/scripts/agents/ 2>&1 | head +``` + +### 6. Typed handoffs validation + +1. SEEN: quote the `handoff.md` convention (goal, paths, file:line, what-is-known) from `ROUTER-FAST.md` lines ~105–115 and ~240/315. +2. Write `handoff.schema.json` (fields: goal, project, worktree, instructionFiles, findings[{file, line, claim}], nextSteps, outOfScope) + a sample `handoff.sample.md` IN THE RUN FOLDER. TESTED when a JSON validation of the sample passes. +3. Consumption test on the copy: point a plan-shaped stage at the sample handoff path (paths-not-contents rule) and confirm it can proceed without the parent context. VALIDATED when the stage output references the handoff's file:line claims. +4. Recommendation states the production home options WITHOUT editing: (a) convention-only (prompt text), (b) template file beside `ROUTER*.md`, (c) enforced schema in `hq-router.ts`. Name the open choice in evidence.md. + +Proof commands: + +```sh +sed -n 100,120p /home/tcrypt/.pi-pocket/hq/ROUTER-FAST.md +python3 -c "import json; s=json.load(open('.router/2026-10-14-gemma-codemap-validation/handoff.schema.json')); h=json.load(open('.router/2026-10-14-gemma-codemap-validation/handoff.sample.json')); print('schema+sample ok')" +``` + +### 7. Gemma4 decision validation (read-only; no model load by this plan) + +1. SEEN: quote the Gemma4 block (artifact hash/size, BF16-KV-only, no prefix reuse, top_k ≤ 20, vision not wired, MTP full-head-only, undo = delete entry + 2 env vars + restart). +2. Record the measurements already in the config comment (aphotic 2026-10-13: load 34 s, 17,454 MiB at 131072, tool_calls with correct args) as TESTED-by-prior-run, NOT re-proven here; note VRAM headroom vs the 22,900 MiB budget and the chat-group exclusivity (one chat model at a time). +3. Read-only currency check: `git -C ~/homelab log --oneline -5 -- reference/llama-swap/config.yaml` + confirm the `Gemma4-26B-A4B-QAT` chat-group member line and `gemma4-26b` alias are present. TESTED when outputs are saved. +4. Decision matrix in evidence.md (no global change): (a) adopt as router/NInfer role — needs prefix-reuse + vision gaps closed; (b) keep as opt-in alias for text/tool trials on the copy; (c) defer. PROVEN requires a copy-server session on the gemma id with a scripted multi-turn + tool-call transcript — explicitly OUT OF SCOPE for the plan stage; state it as the implement stage's first step, and do NOT load/evict models here (one-GPU box, shared with the operator). + +Proof commands: + +```sh +git -C /home/tcrypt/homelab log --oneline -5 -- reference/llama-swap/config.yaml +grep -n "Gemma4" /home/tcrypt/homelab/reference/llama-swap/config.yaml +``` + +### 8. Close-out + +1. Write `evidence.md` with one section per improvement: rating (SEEN/TESTED/VALIDATED/PROVEN), quotes/outputs, and UNCLEAR items. +2. Run `git -C /home/tcrypt/projects/oh-my-pocket status --short` to prove no file outside the run folder changed. +3. Stop any copy servers with the `kill ` from their logs (deletes temp data). + +Proof commands: + +```sh +git -C /home/tcrypt/projects/oh-my-pocket status --short +ls -la .router/2026-10-14-gemma-codemap-validation/ +``` + +## Risks + +- **Live-data contamination:** mitigated by `test/serve.ts` temp data dirs + `CODEVEC_DB=/tmp/...` copies only; never open the live `PI_POCKET_DIR`, never edit `~/.pi-pocket/config.json` or `skills.json` in place. +- **Server restart / blank app:** no `web/**` or `src/server/**` edits in this plan; no restart requested or performed; copy listens on `127.0.0.2:8899` so cookies for the real server are untouched. +- **GPU eviction / operator disruption:** no model loads, no `lsw.py apply`, no `berth run`; Gemma4 evidence is read from config comments + git log, not a fresh load. +- **Secret leak:** `config.json` contains `ownerToken`/hashes — quote only key names and `enabledExtensions`; refer to secret PATHs, never values. +- **VM 250 / 10.0.0.184:** no network or Proxmox action of any kind in this plan. +- **Scope creep into global change:** each improvement's production edit is named but explicitly deferred; the implement stage must refuse to apply them until every item reaches at least TESTED and batteries/fast-path reach VALIDATED on the copy. +- **Stale index conclusions:** retrieval comparisons are only as good as index freshness — record `list_repos` timestamps alongside every query result; a "miss" with a stale timestamp is a coverage finding, not a model finding. +- **Unclear items (must appear in evidence.md, not guessed):** gemma vector comparability + reindex requirement; `hq-router.ts` id-matching for a new fast router id; cloud role-prompt renderer path (`apply_roles.py`/`roles.yaml` location); typed-handoff enforcement home; Gemma4 copy-server trial deferred to implement. + +## Key paths (reference only) + +- Pocket repo: `/home/tcrypt/projects/oh-my-pocket/` — `src/server/extensions/{loop-guard,pruner,read-economy,tool-repair}.ts`, `src/server/reload.ts` (ORDER), `test/serve.ts`, `docs/self-editing.md`, `docs/map.md` +- Live data (read-only here): `/home/tcrypt/.pi-pocket/config.json`, `/home/tcrypt/.pi-pocket/routers/muse-router.json`, `/home/tcrypt/.pi-pocket/hq/{ROUTER*.md,skills.json,mcp-tools.json}`, `/home/tcrypt/.pi-pocket/extensions/hq-router.ts` +- Lab (read-only here): `/home/tcrypt/homelab/reference/llama-swap/config.yaml`, `/home/tcrypt/homelab/scripts/codevec_mcp.py`, roles source under `/home/tcrypt/homelab/harnesses/agents/` +- This run: `/home/tcrypt/projects/oh-my-pocket/.router/2026-10-14-gemma-codemap-validation/` diff --git a/.router/2026-10-14-gemma-codemap-validation/review.md b/.router/2026-10-14-gemma-codemap-validation/review.md new file mode 100644 index 00000000..409a5617 --- /dev/null +++ b/.router/2026-10-14-gemma-codemap-validation/review.md @@ -0,0 +1,42 @@ +# Review: gemma-codemap validation run (2026-10-14) + +Scope: `plan.md` vs `git diff` + `implement.md` + `evidence.md`. Method: reran the battery/app unit tests, parsed all three run-folder JSON artifacts, read the two loader files the claims depend on (`src/server/routers.ts`, `hq-router.ts`), and tried the inputs most likely to break (malformed candidate, stale/self-hit index, fast router id without block, skill-list mismatch). + +Global: **clean on safety.** `git diff` is empty (zero tracked-file modifications); `git status --short` shows only untracked `.router/` and `.zvec-grep/`. No live config/skill/router/src edit, no restart, no model load, no VM250 contact, no secret values in any run file I read. Tests rerun here: battery suites (`read-economy`, `loop-guard`, `pruner`) **33/33 pass**, `app.test.ts` **32/32 pass**. The transparency is good: every unverified item is stated as such in `evidence.md`, and the `hq-router.ts` id-matching constraint was correctly caught, not glossed over. + +Findings (all in run-folder drafts/samples or rating wording — nothing in production to fix): + +## 1. `muse-router-fast.json`: `dispatcher` object is silently dropped by the loader (minor, draft-only) +- File: `.router/2026-10-14-gemma-codemap-validation/muse-router-fast.json` (top-level `"dispatcher"` key) +- Breaks against: `src/server/routers.ts:59` (`dispatcher?: boolean`) and `:541` (`...(raw.dispatcher === true ? { dispatcher: true } : {})`) +- Input that breaks it: register this exact JSON as a custom router. `raw.dispatcher` is an object, never `=== true`, so the loader strips it without error. The `blockedTools: [bash, edit, write]` + width-1 note would be documented intent only — zero enforcement. +- What goes wrong: nothing today (file is explicitly unregistered, parses fine — confirmed `json.load` ok, pipeline plan→implement→review→fix→ship matches live shape). But if the implement stage copies this file into `~/.pi-pocket/routers/` expecting the block, the dispatcher would run unblocked on its cloud model (`opencode-go/muse-spark-1.3-contributor`, same as live — not `router-fast`), compounding finding 2. Fix (later, not here): either `"dispatcher": true` plus enforcement keyed off router id, or keep the object as a comment-sidecar and name the real enforcement point (`hq-router.ts` model-id match or a `routers.ts` schema extension). + +## 2. Fast-path tool block confirmed absent for the new id — plus the skill side is safe (confirms evidence §4, sharpens it) +- Files: candidate `muse-router-fast.json` (`"id": "muse-router-fast"`, `"model": "opencode-go/muse-spark-1.3-contributor"`); `/home/tcrypt/.pi-pocket/extensions/hq-router.ts:117-130` (`fastRouters` set), `:172-180` (`id === "router-fast"` adds; block keyed by conversation in that set) +- Input that breaks it: run router id `muse-router-fast` on any model other than `router-fast` (exactly what the candidate specifies) and attempt `edit` on the dispatcher conversation. +- What goes wrong: no block fires — `fastRouters` only ever contains conversations whose last rendered router model id was literally `router-fast`. Evidence already states this; I verified the code path, so this is a confirmed constraint, not just UNCLEAR. Counterpoint (good news, not in evidence): the skill-trim side does NOT break — `hqRole` (`hq-router.ts:~91-104`) returns `"router"` for any router-editor session on any model, and `"router"` exists in `skills.json` (21 skills). So the new id gets the router skill list, just no tool block. Recommendation for implement: either run the fast router on model id `router-fast`, or extend `PROMPTS`/`routerModel()` id matching before claiming width-1/dispatcher behavior. + +## 3. `handoff.sample.json`: `file:line` claims don't point at what they claim (minor, exemplar-quality) +- File: `.router/2026-10-14-gemma-codemap-validation/handoff.sample.json` (single-line JSON; findings array) +- Input that breaks it: follow the sample's own contract ("every claim carries file:line") — finding 1 claims `src/server/reload.ts:42` = "Loop-guard ORDER position". Actual `reload.ts:42` is `"triggers.ts",` (loop-guard is at line 44; read-economy at 43). Finding 2 claims `loop-guard.ts:1` for third-call-block behavior; line 1 is the `/**` opener of a ~10-line doc comment (accurate only in the loosest sense). +- What goes wrong: as a throwaway sample, nothing executes, so no behavior breaks. As the schema's exemplar for future stages, it teaches "approximate line numbers are fine," which defeats typed handoffs. Suggested correction (implement stage): point at `reload.ts:44` and `loop-guard.ts:2-3` (the actual behavior sentence), and validate the sample against the schema with a real validator — `json.load` parsing (the only check run) accepts a finding missing `line` entirely, so the "required" contract is currently untested. I confirmed a `{"file","claim"}`-without-`line` object parses fine while violating `handoff.schema.json` required `["file","line","claim"]`. + +## 4. Copy-DB retrieval hit is partly self-referential (weakens, doesn't void, the embed2 VALIDATED) +- File: `evidence.md` §1 (`VALIDATED: same 3 queries on copy-DB hit sane files`) +- Input that breaks it: query 1 ("loop guard…") top-hit `handoff.sample.json` — a file this run itself wrote, indexed as part of the 273-file copy (run folder was inside the indexed tree). +- What goes wrong: 1 of 3 validation queries is circular (query terms match the sample text the run authored). Queries 2–3 hit real product files (`test/read-economy.test.ts`, `src/server/routers.ts`), and dims (768/768) plus the 2334-chunk reindex are solid, so VALIDATED-on-copy still stands on balance — but the report should mark Q1 as contaminated and, for the production cutover decision, re-run Q1 with `.router/` excluded (or quote a product-file hit). Also still open per evidence: mixed nomic+gemma vector comparability (copy reindexed from scratch), correctly left as UNCLEAR. + +## 5. `batteries VALIDATED` overstates against the plan's own bar (wording only) +- Files: `implement.md` (Ratings line), `evidence.md` §3, vs `plan.md` step 3 (`VALIDATED = steps 2–4 all pass on the copy`, where step 3 is the four per-behavior exercises: 500-line map, malformed-call repair, triple-call block, prune note) +- What goes wrong: the per-behavior exercises were explicitly NOT run (no browser/scripted-model UI run — openly stated in "What not verified"). What did pass: copy-server boot ×2, loader-name presence 1/1, unit tests 33/33 + 32/32 (all rerun green here). That is genuine substance, and nothing is hidden — but by the plan's written criterion the rating should read **TESTED + unit-VALIDATED** (or VALIDATED-pending-behavior), not unqualified VALIDATED. One-line fix in the implement/evidence ratings; no rework needed if the implement stage runs the four behaviors on the copy. + +## 6. Non-findings (checked, clean) +- Malformed-JSON robustness: candidate files parse; a loader receiving `'{bad json'` fails closed (`ValueError`), and minimal `'{"id":"x"}'`/`'[]'` inputs parse but carry no pipeline — no crash path introduced, since nothing loads these drafts. +- Skill-list mismatch: no mismatch exists. Candidate stage `skills` (`poteto-mode`, `blast-radius`, `unslop`) mirror the live router's; `skills.json` role lookup falls back safely (finding 2). `router-fast` absence from `skills.json` roles is expected (it's a model id, not a role). +- Stale-index handling: evidence correctly attributes the live-DB "read economy" miss to coverage (oh-my-pocket unindexed; only `homelab` in `list_repos`), not to the embedder — the right call, with timestamps recorded. +- Gemma4: read-only posture held (git log + entry/member-line presence); prior-run load figures labeled as such, not re-proven. Correct. +- `.zvec-grep/` untracked dir: present in `git status` alongside `.router/`; implement.md disclaims it. I did not verify ownership — taking the disclaimer at face value, no action. + +## Verdict +**Suitable to proceed to production proposal with the five notes above addressed in passing** (dispatcher shape, fast-id block code change, handoff line numbers + real schema validation, Q1 re-query minus `.router/`, batteries rating wording). No re-validation run needed: no production file changed, tests are green on rerun, constraints were found rather than missed, and every gap is already written down as a gap. diff --git a/.router/2026-10-14-gemma-codemap-validation/review2.md b/.router/2026-10-14-gemma-codemap-validation/review2.md new file mode 100644 index 00000000..24b8d300 --- /dev/null +++ b/.router/2026-10-14-gemma-codemap-validation/review2.md @@ -0,0 +1,33 @@ +# Review2: gemma-codemap fix verification (2026-10-14) + +Scope: `review.md` 5 findings vs `fix.md` + current run-folder files + `evidence.md` + `implement.md`. Method: re-read fixed files, re-evaluated loader expression, re-checked cited source lines, re-validated schema/sample, reran battery + app suites, checked `git diff` / `status`. Edit: none (this file only). + +Global: **safety clean.** `git diff --stat` empty (zero tracked-file modifications); `git status --short` shows only untracked `.router/` + `.zvec-grep/`. No live config/skill/router/src edit, no restart, no model load, no VM250 contact, no secret values seen. Tests rerun here: battery suites (`loop-guard`, `pruner`, `read-economy`) **33/33 pass**, `app.test.ts` **32/32 pass**. + +## Finding 1 — dispatcher shape: FIXED +- File: `muse-router-fast.json` now has top-level `"dispatcher": true` (was object) + `_dispatcherNote` string sidecar. +- Proof: `node` eval of `src/server/routers.ts:541` expression against the fixed file → `loader keeps dispatcher: {"dispatcher":true}` (object form would have been stripped by `=== true`). Stages still plan→implement→review→fix→ship; sidecar is `string` (dropped on registration, no loader impact). +- Remaining: none in the draft. The sidecar correctly points at `hq-router.ts` `fastRouters`/model-id enforcement so a future copy into `~/.pi-pocket/routers/` carries the warning. + +## Finding 2 — fast-id block constraint: FIXED (wording + warning) +- File: `evidence.md` §4 now reads CONFIRMED CONSTRAINT (was UNCLEAR), citing `hq-router.ts` ll.117-130 + 172-180; candidate `_dispatcherNote` names the same enforcement point. +- Proof: re-read the code path here — `fastRouters` set keyed by rendered router model id literally `router-fast`; `id === "router-fast"` adds, else deletes; block fires only for conversations in that set. New id `muse-router-fast` on cloud model gets the router skill list (`hqRole` → `"router"`, exists in `skills.json`) but no tool block without (a) running on model `router-fast` or (b) extending `PROMPTS`/`routerModel()` id matching. Correctly stated. +- Remaining: none. Code change itself is correctly deferred to implement. + +## Finding 3 — handoff line refs: FIXED +- File: `handoff.sample.json` finding 1 now `reload.ts:44`, finding 2 `loop-guard.ts:2` (claim notes behavior spans 2-3; schema requires int `line`). +- Proof: `sed -n '43,45p' src/server/reload.ts` → line 44 is `"loop-guard.ts",`; `sed -n '1,5p' .../loop-guard.ts` → behavior sentence ("third exactly identical…blocked") starts line 2. Schema check: required top keys + per-finding `file,line,claim` present with int `line`; a `{"file","claim"}`-without-`line` object correctly fails (closes the "parses fine while violating required" gap). +- Remaining: none. + +## Finding 4 — embed2 Q1 contamination: FIXED (wording + follow-up) +- File: `evidence.md` §1 now carries explicit CONTAMINATED note (Q1 top-hit `handoff.sample.json`, self-authored inside the 273-file copy tree → circular) and rests VALIDATED-on-copy on Q2+Q3 (real product files: `test/read-economy.test.ts`, `src/server/routers.ts`) + 768/768 dims + 2334-chunk reindex. +- Required follow-up is stated, not runnable from here without reindexing a copy: re-run Q1 with `.router/` excluded before any production cutover. Mixed nomic+gemma comparability still UNCLEAR — correctly left so. +- Remaining: none for this stage. Do not claim production cutover readiness until the Q1 re-query lands. + +## Finding 5 — batteries rating wording: FIXED +- Files: `evidence.md` §3 heading/rating → **TESTED + unit-VALIDATED (per-behavior exercises pending)**; same correction in `implement.md` Ratings line. +- Proof: rating now matches plan step 3's bar (four per-behavior exercises: 500-line map, malformed-call repair, triple-call block, prune note — explicitly not run). Substance stands: copy boot ×2, loader names 1/1, 33/33 + 32/32 green on rerun here. +- Remaining: none for wording. The four behavior exercises remain the implement stage's job on the copy. + +## Verdict +**Clean — all 5 findings fixed, no remaining issues.** No re-validation run needed: drafts corrected in the run folder only, ratings match the plan's bar, constraints are stated as constraints, tests green on rerun (33/33 + 32/32), `git diff` empty.