Skip to content

feat(admission): enforce record exclusions across indexing and recall - #180

Open
HengYangDS wants to merge 6 commits into
obra:mainfrom
HengYangDS:feat/record-admission
Open

HengYangDS wants to merge 6 commits into
obra:mainfrom
HengYangDS:feat/record-admission

Conversation

@HengYangDS

@HengYangDS HengYangDS commented Sep 26, 2026 •

Copy link
Copy Markdown

Problem

A new record-level exclusion cannot be enforced by the existing project filter or whole-conversation marker alone. Direct parser/index/database paths, summaries, MCP/display reads, and retained search rows can otherwise disagree about what is admissible. A previously indexed row must not remain visible merely because it predates a changed operator policy.

Change

  • Add an optional, validated record-exclusions.json in the existing configuration directory. It binds inclusive physical line ranges and native tool-call IDs to session ID plus transcript basename, not any host-specific absolute path. No policy file or personal exclusions are included in this PR.
  • Apply one fail-closed admission boundary across archive derivation, parsing, indexing, direct insertions, summaries, display/MCP reads, and search. Preserve original transcripts and physical line coordinates. Recheck the current policy and source snapshot for retained search rows; excluded candidates do not consume the requested limit.
  • Decode user instruction text for conversation opt-out markers while ignoring assistant quotations and tool results. Make policy-enabled summaries transcript-only, without resuming excluded source context or silently changing a Codex summary to Claude.
  • Support optional local Gitleaks screening with an operator-supplied absolute executable and SHA-256 pin, bounded per-record execution, strict report validation, no inherited scanner configuration, and no secret-bearing diagnostics. The normal path has no scanner requirement.
  • Classify screening rejections and unavailable scanner state separately from transcript corruption, even when summaries are missing. --repair refuses to start while either screening failure remains; invalid record policy fails verification before a false healthy result.

Verification

  • From upstream main at 7e06519: npm run build, node --check dist/mcp-server.js, and the current full local suite passed (448/448 on Node 26.10.0).
  • Screening contract tests also passed without Gitleaks in PATH (29 passed; the one real-Gitleaks integration test skipped); the actual Gitleaks integration test passed locally when available. New verifier and CLI regressions failed before the repair and pass on the updated head.
  • Commit-range Gitleaks scan and host-specific path scan passed. No private policy, local absolute path, credential, package version change, or lockfile is included.
  • A same-mtime source append regression now proves the derived archive refreshes instead of dropping an incremental exchange. A temporary, unpushed integration of the final fix(indexer): keep session indexing incremental on append #176–feat(admission): enforce record exclusions across indexing and recall #180 heads passed build, bundle syntax, and 466/466 tests on both Node 22 and Node 24.

Review and CI limits

  • A follow-up regression first failed on the initial PR head because SQLite resolved distance to a vec0 column rather than the computed alias. The query now uses retrieval_distance; the two ordering regressions and the full suite pass on the updated head.
  • Search uses a complete ordered candidate scan so exclusions beyond vec0's KNN candidate ceiling cannot hide later safe results. This favors admission correctness; performance on large indexes has not been benchmarked here and should be reviewed.
  • A Codex transcript without a source session ID previously reached Claude even with no record policy. New failing-then-passing regressions route it through an isolated ephemeral Codex thread, prevent borrowing another harness’s session ID, and prevent Codex failures from invoking Claude. Session-ID summaries still use an ephemeral Codex fork.
  • The isolated Codex summary path is covered by fake app-server tests, not a real provider/credential run. The five-PR integration required conflict resolution and an integration-only update to fix(summarizer): keep Codex summaries on the Codex route #178’s older sessionless-error assertion; no integration branch was pushed. This PR does not backfill historical summaries, rewrite session history, or turn on background sync.
  • The generated dist/mcp-server.js diff is large because upstream does not lock transitive dependency layout. Source/test changes are the reviewable semantic core.
  • The local run reused existing dependencies and model data. Cold npm install and official Node 22/24 CI are not verified. The upstream CI run is action_required pending maintainer approval; no Node 22/24 job executed.

Additional local Node matrix (2026-09-26)

On macOS arm64, this current PR head passed npm run build, node --check dist/mcp-server.js, and the complete test suite on Node 22.23.3 and Node 24.21.0 (448/448 tests on each). Each run used a fresh source snapshot and the corresponding per-Node native dependency tree, with a pre-seeded local embedding-model cache and an isolated OMP_HOME. The dependency trees were installed from PR #179 (all five PRs have the same dependency declarations); this is not a separate empty-cache install for each PR, an Ubuntu result, or an upstream CI pass.

@HengYangDS

Copy link
Copy Markdown
Author

The upstream CI run for this fork PR is currently action_required, so the declared Node 22/24 cold-install matrix has not executed. Could a maintainer approve the workflow run when convenient? The related independent PRs #176–#179 are in the same state. I will treat their CI as unverified until checks actually run, and address any failures or rebases that follow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant