Skip to content

fix(indexer): keep session indexing incremental on append - #176

Open
HengYangDS wants to merge 1 commit into
obra:mainfrom
HengYangDS:fix/index-session-high-water
Open

HengYangDS wants to merge 1 commit into
obra:mainfrom
HengYangDS:fix/index-session-high-water

Conversation

@HengYangDS

@HengYangDS HengYangDS commented Sep 26, 2026 •

Copy link
Copy Markdown

Problem

indexSession re-embeds and replaces every previously indexed exchange when a transcript grows. Its archive copy is also only created once, so a newly indexed exchange can point to a stale archive that does not contain the appended lines.

Change

  • Resume session indexing after the database's MAX(line_end) high-water mark, matching the existing incremental-indexing path.
  • Reuse the atomic archive copy helper and refresh when source size changes even if filesystem timestamps tie.
  • Leave summary generation and the existing append-only transcript assumption unchanged.

Verification

  • Added a regression that failed on the upstream base because last_indexed of an existing row was replaced. It now verifies the old row is preserved, the appended exchange is indexed, and the archive contains that exchange even when source and archive mtimes match.
  • Full local suite: 351 tests passed across 63 files. TypeScript compilation, esbuild bundling, bundle syntax checks, and a redacted secret scan of this commit passed.
  • Local tests used already-installed development dependencies and a pre-existing local model cache. A fresh local npm install did not finish within a 300-second bound; this is not a cold-install claim. The fork PR's Node 22/24 CI is the cold-install gate.

Scope and risks

The change applies to the session-specific indexing path; it does not invoke repair or re-summarize historical conversations. It relies on the existing append-only transcript contract and does not reconcile rewritten or truncated sessions. The generated MCP bundle has a large mechanical diff because upstream does not lock transitive dependency layout; the source-level change is in src/indexer.ts and src/sync.ts.

Related to the high-water fixes discussed in #84 and #152; both issues are already closed, so this PR does not claim to close them.

Additional local Node matrix (2026-09-26)

On macOS arm64, this current PR head passed npm run build, node --check dist/mcp-server.js, and the complete test suite on Node 22.23.3 and Node 24.21.0 (351/351 tests on each). Each run used a fresh source snapshot and the corresponding per-Node native dependency tree, with a pre-seeded local embedding-model cache and an isolated OMP_HOME. The dependency trees were installed from PR #179 (all five PRs have the same package.json); this is not a separate empty-cache install for each PR, an Ubuntu result, or an upstream CI pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant