[naamanhirschfeld-armis]

Part of #87. This PR completes phases A2 and A3 of the layered index and the full A2–A8 specification. It does not close the issue, because A4–A8 are not started.

What's in it

Spec (ADR-0016). Adds the Specification section for A2–A8. The section was checked against the code, which corrected several assumptions in the plan:

• A Rev scan has no resolve pass, so the base builder scans an exported tree as a working tree.
• code_chunks is keyed (scope, path) in Lance.
• repo_group_key hashes the main worktree root, not the common dir.

The section also records these owner decisions:

• A2 migrates through the row version and needs no re-index.
• With no daemon, layered mode degrades like a lost lock: Daemon route, else InRam.
• Base storage stays pluggable behind a BaseReader. An immutable mmapped sorted-run base is prototyped and benchmarked before A4. Fjall 3.1 takes an exclusive flock even read-only, so a Fjall base can be held by only one process.

A critical review added merge-contract rules (path masks rather than tombstone rows, NameDict union, snapshot handle, Meta deltas), crash-safe publish ordering, a disk budget, observability, and per-phase acceptance criteria with fault injection. docs/layered-index-design.md now defers to the ADR where they differ.

A2: grep_bloom keyed by content hash.

• Paths with identical bytes share one row.
• The skip guard uses the indexed (size, mtime_ns) from the manifest.
• Old rows migrate by ROW_VERSION 2. Rows of another version read as absent, which means the file is a candidate and gets scanned. INDEX_SCHEMA_VER is unchanged, so no user re-indexes.
• The sweep runs on working-tree scans only. It reads keys only in steady state, plus a one-time version sweep behind a meta marker.
• The watcher (scan_paths) deletes a row once no manifest entry holds its hash, via a per-hash refcount on Store. Without this, a long-lived daemon grows the keyspace on every edit.
• Racy stamps: a file whose mtime is within 2 s of its read is stored with an untrusted stamp (mtime_ns = 0), the same approach as git's racy-clean handling. Such a file is never skipped and is restamped on the next scan. This closes a same-size, same-tick edit hole that could cause a false skip.

A3: IndexView trait.

• Every index read goes through &dyn IndexView, and IndexRoute::Local holds a StandaloneView.
• The trait is backend-neutral. Values are a crate-owned IndexBytes, a zero-copy wrapper over fjall::Slice today. range takes Bound<&[u8]>.
• The named reads (references_to, definition_of, bm25_stats, …) are default methods over the generic get/prefix/range/iter layer, so a layered view inherits them. Their IndexDb copies are deleted.
• Keyspace fields are pub(in crate::index), so a read that bypasses the view does not compile.

Testing

• cargo test passes: 962 lib tests and all integration targets.
• cargo clippy --workspace --all-targets --tests -- -D warnings is clean on default features and on comms,code-search. cargo fmt --check is clean.
• Red-green: the watcher delete, the key-only sweep, the racy-stamp guard, the generic-layer bounds and the cap check each made their new tests fail when mutated, then were restored.
• A recording view asserts the exact read log for the call and impl scans, the grep prefilter, both search lanes and IndexRoute::Local.
• Migration tests write real v1 rows (path-keyed with the 17-byte header, and hash-keyed v1). They assert no false skip, and that the next incremental scan sweeps the old rows and rebuilds v2 rows.

Known gaps

• Perf: no criterion bench exists for references/callers. A probe over a 995-file corpus showed no consistent change, but its run-to-run noise is about ±20%. The change is zero-copy by construction, at one boxed iterator per scan and one virtual call per row. A bench should land before A4.
• Defaults:
  • advance_stall_ms is 2000. A CoW clone of the 1.7 GB armis index takes 0.36 s.
  • max_group_bytes is auto, 4× the newest generation: about 7 GB on armis.
  • Retention is 2 generations, or 3 when the cap allows.
  • The visited-row budget is scan_cap × 64.

Also fixed

• tests/cli_daemon.rs is gated on comms only, but it exercises memory mode. It therefore failed under --features comms,code-search without memory. The memory step now runs only when that feature is built.

🤖 Generated with Claude Code