[Aschenbath]

Each identity-document extraction calls name_vectors::drop_for_document. The only existing fact_evidence index starts with fact_id, so a lookup for one document scans every evidence row, once per pushed document. The source identity-flag update has the same reverse lookup through drop_for_source.

Add fact_evidence (document_id) in migration 0119 and advance the CLI's migration count to 106. The two DELETE statements are unchanged and shared with a plan regression, which copies the migrated tables/indexes into temporary tables and checks actual rows visited for selective document/source lookups. It uses the normal planner and no timing assertions.

Fixes #1408.

Measured on base 14bbb76d53e3a58419bab9ae16cebee38b66b72d, with PostgreSQL 16.15 on Linux x86_64, a four-CPU container quota, 4 GiB memory and tmpfs storage (shared_buffers = 128 MiB). Two synthetic bases contain 400,000 facts/evidence rows, 20,000 documents and 40,000 name vectors: 20 facts and two names per document. Each value below is the median of five executions after warmup, using the original production SQL. Individual statements use EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON); the batch uses the server clock. Every sample rolls back, and each deletion leaves the same vector IDs before and after the index.

Operation	Before	With index	Evidence rows visited, before → after
One document	23.77 ms	0.17 ms	400,000 → 20
Source with 50 documents	52.95 ms	2.52 ms	400,000 → 1,000
Source with 5,000 documents	103.25 ms	102.97 ms	400,000 → 400,000
1,000 consecutive document deletions	18.44 s	61.31 ms	repeated DELETE statements
Insert 20,000 evidence rows	558.83 ms	574.22 ms	write-cost probe

The index occupied 3.25 MiB in this fixture. The large-source query still chooses a sequential scan. The batch measures deletion statements in one transaction, not complete ingestion, and the INSERT probe retains the real ownership/foreign-key triggers. Like the existing index migrations, this builds the index in the startup migration transaction.

Validation on 1ac9fdef5078302ba69277f416b0b750602dc050: the regression fails with the new index removed and passes with it restored; both name-vector plan tests pass. The full Rust workspace passes 1,694 tests on Linux x86_64, with six existing live/opt-in tests ignored and the database/PDF prerequisites required. Workspace formatting and all-target clippy with warnings denied pass. The actual migration binary succeeds twice on a fresh database, recording 106 migrations with maximum version 119 and the new index present. Frozen frontend install, 323 tests, style guard, TypeScript and production build pass on Windows.

The upstream CI run for this commit passes all six jobs: lint, test, build, backend, migrations and web.

Implemented and tested with AI assistance.

[gedaiu]

Thanks for this, the numbers are great. One small request: could the migration use CREATE INDEX IF NOT EXISTS? A fork we run (re-cinq) has had this exact index, same name, since 2026-10-17, so a plain CREATE INDEX would stop its startup migration with "relation already exists". With IF NOT EXISTS it's a no-op there and unchanged everywhere else.

[WaylandYang]

Thanks, clear numbers and a test that fails without the index. Please make it CREATE INDEX IF NOT EXISTS as gedaiu asked (same as 0043/0094), and keep it in the migration transaction. Not CONCURRENTLY: 0094 explains why, and a failed concurrent build would leave an invalid index that IF NOT EXISTS then skips. Heads-up: #1386 also sets CURRENT_SCHEMA_VERSION to 106 and draft #1359 also uses 0119, so whichever lands second renumbers.

[Aschenbath]

Thanks both. Changed in 4dbf36f: the migration is now CREATE INDEX IF NOT EXISTS, still in the migration transaction and without CONCURRENTLY. A short comment in the file explains the fork case and points to 0094. I also merged current dev (212c8e4). dev already has 0119 and 0120, and open #1428/#1450 both use 0121, so this migration is now 0122_fact_evidence_is_found_by_document.sql and CURRENT_SCHEMA_VERSION is 108, matching 108 migration files. The Rust and test changes are the same as before.

[WaylandYang]

Thanks — IF NOT EXISTS inside the migration transaction, as asked. Merging.