Cover the async retrieval path, and record a de-duplication asymmetry

[adamjohnwright]

An adversarial pass over the retriever — the code every question goes through, and the thing spec 009 proposes changing.

The gap

HybridRetriever implements retrieval twice: retrieve_documents and aretrieve_documents, the second gathering coroutines so collections are queried concurrently. Nothing exercised the async one — and that's the one the application serves through, while bin/retrieval_baseline drives the sync one.

That asymmetry is worse than ordinary duplication. retrieval_baseline is the tool the constitution names for measuring retrieval changes, and spec 009's whole verification plan rests on it. If the paths drift, every measurement is of a path no user takes.

They don't drift today — same documents, same order, across three query shapes. Now pinned.

Two things this test had to be argued out of claiming

It first reported the two paths returning different document sets. That was FakeEmbeddings, which returns a fresh random vector per call, so the same query embedded differently on the second run. DeterministicFakeEmbedding fixes it. The near-miss is in the docstring, because the false result is more instructive than the true one.

It does not cover the per-collection cap, and says so. Deterministic fake vectors carry no relation to the text, so every query retrieves much the same set and fusion lands on exactly the cap however many queries are supplied — changing the cap on one path alone stays invisible. I tried 12 and 60 documents per collection with one, three and seven queries: ten per collection every time. Rather than leave apparent coverage, the limit is written down; covering it needs real embeddings, which is retrieval_baseline's job.

A finding I measured and chose not to act on

dedupe_by_entity keys on st_id, falling back to page_content. But BM25 documents come from CSVLoader, whose metadata is {source, row} — no st_id at all. Verified against the real bundle: 0 of the first 2,000 reaction documents carry one.

So the two halves of the pipeline de-duplicate by different identities. Vector hits collapse to one row per stable ID; BM25 hits don't, and reactions has 17,004 rows for 16,107 distinct reactions, so the same reaction appears under several pathways.

Measured on real questions rather than argued about:

question	distinct reactions in BM25's top 10
What does CDK5 phosphorylate in Alzheimer disease?	10/10
How does TP53 regulate PTEN transcription?	9/10
cholesterol transport	10/10
glycolysis regulation	10/10
interferon signalling	9/10

Worst case one slot in ten. Real, minor, and not worth a blind change to retrieval — "this should be better" is not a finding. Recorded here so spec 009 knows it's there, since routing work touches exactly this code.

329 tests, ruff and mypy clean.

🤖 Generated with Claude Code