Skip to content

perf(index): leave secret redaction to push - #296

Merged
dacorvo merged 18 commits into
mainfrom
perf/index-many-sessions
Nov 13, 2026
Merged

dacorvo merged 18 commits into
mainfrom
perf/index-many-sessions

Conversation

@dacorvo

@dacorvo dacorvo commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

Indexing spawned TruffleHog for every batch of 32 sessions, so for every agent turn a hook indexed: about 1 s of CPU and 160 MB each, for a few kilobytes of text. It also rewrote the whole index state after every session. This PR leaves secret redaction to push, where a session leaves the machine, and writes the state once per batch.

Behavior

  • funes index stores sessions as written and only elides data: URIs. It no longer needs TruffleHog.
  • funes push redacts the secrets its gate finds, the way funes scrub does, where it held their blocks back. The redacted chunks are published under the stored rows' ids, and the local memory keeps its rows as they are. A block it can't redact in place is held back whole: a secret in an encoded region, a redaction that still scans dirty, or one that re-chunks into a different number of chunks.
  • The gate and scrub also redact the block's tool_name column. Scrub on main redacted the name in the text only, so a secret in a tool name reached the next push in that column.
  • Indexing skips a block whose first chunk the memory holds, where it skipped chunk by chunk. A session an older funes indexed holds its blocks with the secrets cut out. Read again raw, such a block can split into more chunks, and the extra ones would have stored the end of the secret.
  • The index state (state.json, index-coverage.json) is written once per batch of up to 32 sessions instead of after every session. Each write holds every session the run has seen, so the cost grew with the square of their count. A run that stops mid-batch re-reads that batch, and the stored chunk ids keep its rows from being written twice.

The local memory now holds credentials as written, like the transcripts it reads. SECURITY.md, docs/push.md and docs/automation.md say so.

Benchmarks

  • bench_index (new) times the part of funes index that writes rows, without the embedding.
  • bench_backends is split into bench_models, which times the embedder and the reranker on one backend, and score_models, which scores blas and metal against ONNX. Both cover the Metal embedder.
  • bge-reranker-base is dropped, with the XLM-R position offsets and roberta. weight prefix that only it used. bench_backends was the last thing that ran it.

bench_index row-writing time on an M2 MacBook Air, each change against the commit before it:

corpus change before after
5,000 LongMemEval-S sessions one state write per batch 181.6 s 167.6 s
no secret scan at index 179.4 s 20.7 s
2,000 generated sessions one state write per batch 64.9 s 64.1 s
no secret scan at index 65.4 s 4.9 s

🤖 Generated with Claude Code

dacorvo and others added 18 commits November 12, 2026 23:02
bench_backends timed every compiled backend side by side, with ONNX as
the reference. ONNX is no speed target, but it is a sound accuracy
reference. So bench_models times the embedder and the reranker on one
backend chosen with --backend (blas, or metal in a macOS build), and
score_models scores blas and metal against ONNX, which it requires.
Both now cover the Metal embedder, and neither runs bge-reranker-base,
the speed baseline of the switch to gte. The workloads move to
benchmark/workloads.rs, which both include.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
funes reranks with gte-reranker-modernbert-base on every backend, and
bench_backends was the last thing that ran bge-reranker-base, as the
speed baseline of the switch to gte. The BERT encoder's XLM-R position
offsets and roberta. weight prefix go with it, since only that model
used them. The embedders still match ONNX as before in score_models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An index run writes rows, embeds them, then compacts the memory, and
the embedding takes long enough to hide the rest. index::write_rows
runs everything but the embedding, and bench_index times it in a
scratch memory, over generated sessions or a directory of turns files.
It reports the rows written and their rate, the fragments the writes
leave for the compaction, and the time the end of the run takes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
record rewrote state.json and index-coverage.json after every unit,
and each write holds every unit the run has seen, so a run over many
small sessions paid a cost that grew with the square of their count.
The state is now persisted once per batch, after a record, and the
batch's spool files are drained after that write. A run that fails
mid-batch re-reads that batch on its next run, where the stored chunk
ids keep its rows from being written twice.

bench_index, rows written before and after:
- 5,000 day-zero sessions: 181.6 s, then 167.6 s
- 2,000 generated sessions: 64.9 s, then 64.1 s

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The push gate held back every block that still held a secret, and its
rows waited for funes scrub. It now redacts each such block as scrub
does: the secret excised from the block's text, the text re-split by
the same chunker, the result scanned again and re-embedded. The pieces
are published under their rows' ids, so the remote holds the ids the
local memory does, and the local memory keeps its rows as they are. A
block that can't be redacted in place is still held back whole: a
secret in an encoded region, a redaction that still scans dirty, or
one that re-splits into a different number of pieces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Indexing spawned TruffleHog for every batch of 32 sessions, and so for
every agent turn the hook indexed: about 1 s of CPU and 160 MB each,
for a few kilobytes of text. Push redacts what it publishes now, so
indexing stores sessions as written and only elides data URIs. The
local memory is no publication boundary, and push and scrub still
refuse to run without a scanner.

bench_index, the two builds alternated, rows written before and after:
- 5,000 day-zero sessions: 179.4 s, then 20.7 s (294.6 s, then 34.6 s
  in a busier first round)
- 2,000 generated sessions: 65.4 s, then 4.9 s
The end of the run, merging and indexing, takes the same time in both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…written

SECURITY.md still said credentials are redacted at index time, and
that the gate withholds every row holding a secret. Indexing now
stores sessions as written, and the gate redacts what it can and
withholds the rest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The GPU embedder embeds a repeated text once, and four workloads
repeated theirs: 30×~500tok is one text, 100×real and 256×real hold 32,
256×mixed 17. bench_models --backend metal timed those few texts while
it reported the whole workload, up to 30x too fast. Each document of
those workloads now starts with its number.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
workloads.rs opened by naming bench_models and score_models, its two
users. It also called 256 documents the production batch size, which
holds on the CPU only: an indexing call on the GPU takes 256 to 2,048
texts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its name and doc said a directory is indexed a batch at a time, which
no assertion has checked since indexing stopped spawning the scanner
once per batch. take_batch's unit test covers how a batch fills. The
test now says what it asserts, that indexing runs no secret scanner,
and its file is named for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bench_index's docs and RowsWritten listed scanning for secrets among
the work of writing rows, which indexing no longer does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gate's comments said it redacts the secrets the rows "still"
hold, from when indexing redacted first. Skipped said rows are held
back for holding a secret, when they are held back only where their
block can't be redacted in place. The escaped-key test told the story
of the leak it once caught, and the gate's doc kept an em dash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Several described their callers instead: write_rows named the bench
that times it, record and persist said how often write_units writes
the state, and write_unit said what write_units does with its spool
file. Why the state is written once a batch now sits where it is
written. write_unit no longer calls its turns sanitized, since only
their data URIs are elided. BATCH_UNITS gives no reason for 32: the
reason was the TruffleHog spawn, which indexing no longer runs. The
two lines the branch touched with em dashes lose them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The field was called drained, but it holds the spool files of a batch
that are not drained yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gate's result named them held, as the push coverage's field does,
and run_push renamed them skipped on the way out. One name now runs
through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dropping bge-reranker-base from it left a short line mid-paragraph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Indexing deduplicated chunk by chunk, so a block read again with a
longer text kept its stored pieces and took the extra ones. funes
1.12 and older stored a block with its secrets cut out, so this
branch re-reading such a session stored the raw tail of the secret
behind the redacted head. The push gate rebuilds the block from both,
finds no secret without the head, and would publish the tail.

A block is now deduplicated by its first piece: one already in the
memory, or met earlier in the unit, takes none of the new pieces. An
edit re-emitted under its turn_uuid is then deduped away whole, as
docs/funes-jsonl.md says.

Reproduced with the v1.12.0 release and a 4096-bit RSA key: 33 of
its 47 lines were stored after the re-read, and scrub found nothing.
With this commit, none are stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A block's text renders its tool name, which is also a column of its
own. The push gate cut a secret found there out of the text, then
re-split the text under the original tool name, so the redacted rows
uploaded the secret in that column, out of the rescan's sight. Scrub
re-splits the same way, so a scrubbed block carried such a secret to
the next push too.

resplit now takes the tool name, and both callers pass it with the
same findings cut out as from the text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dacorvo
dacorvo merged commit 1e7b2b7 into main Nov 13, 2026
12 checks passed
@dacorvo
dacorvo deleted the perf/index-many-sessions branch November 13, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant