Skip to content

Stage 3: the caller supplies the bundle and the context budget - #185

Merged
adamjohnwright merged 2 commits into
mainfrom
feat/retriever-stage3
Sep 9, 2026
Merged

adamjohnwright merged 2 commits into
mainfrom
feat/retriever-stage3

Conversation

@adamjohnwright

Copy link
Copy Markdown
Contributor

Closes out the retriever rewrite (spec FR-005, FR-006, SC-006). Two module-level facts become arguments.

The bundle

Each create_*_rag had

embeddings_directory: Path = EmbeddingEnvironment.get_dir("reactome"),  # noqa: B008

A default argument is evaluated at import time, so importing a rag module required an installed bundle — which is why these were untestable. And get_dir returns Path | None, so None flowed into a parameter typed Path; four mypy baseline entries existed only to suppress that. All five suppressions are deleted, not re-annotated.

EmbeddingEnvironment.require_dir replaces the lookup at the three call sites that cannot run without a bundle. It also checks the directory exists — which is not tidiness:

get_dir builds a path without touching the disk, and Chroma creates a missing persist_directory rather than complaining. An embeddings/current left pointing at a deleted bundle gave a chatbot that answered every question from an empty collection, confidently, with no error anywhere.

Measured against main:

missing-bundle failure
before AttributeError: 'NoneType' object has no attribute 'glob'
after No embeddings bundle installed for 'uniprot' (installed: reactome). Run ./bin/embeddings_manager install <embedding-id>.

The userguide bundle is deliberately optional; react_to_me.py already resolved it itself and degraded on absence. That path keeps get_dir, and require_dir raises FileNotFoundError specifically so the existing except FileNotFoundError there still means what it meant.

The budget

MAX_DOCUMENTS_PER_COLLECTION becomes DEFAULT_MAX_DOCUMENTS_PER_COLLECTION, the default of a constructor argument. As a module constant, comparing two budgets meant editing the file and restarting — so the answer-quality evaluation meant to settle the number could not be run.

Verification

Against the Release95 reactome bundle:

  • the assembled chain answers "What role does TP53 play in apoptosis?" identically through invoke and ainvoke, 40 documents each (constitution Article I)
  • budget=3 over 4 collections → 12 documents; budget=10 → 40
  • all three profiles construct or refuse correctly: ReactToMe builds, CrossDatabase and PlantReactome refuse with the message above

Each new test was checked to be non-vacuous by perturbing the thing it pins — ignoring the budget argument, reinstating a default, and removing the is_dir check each make exactly the relevant test fail.

Two module-level facts become arguments, closing out the retriever
rewrite (FR-005, FR-006, SC-006).

The bundle. Each create_*_rag had

    embeddings_directory: Path = EmbeddingEnvironment.get_dir("reactome")

as a default argument, so it was evaluated at import time: importing a
rag module required an installed bundle, which is why they could not be
unit tested. get_dir returns `Path | None`, and None travelled into a
parameter typed `Path` -- four mypy baseline entries existed solely to
suppress that, and a ruff B008 suppression sat on each line. All five
suppressions are now deleted rather than annotated.

EmbeddingEnvironment.require_dir replaces the lookup at the three call
sites that cannot proceed without a bundle, and also checks the
directory exists. That second check is not tidiness: get_dir built a
path without touching the disk, and Chroma CREATES a missing
persist_directory rather than complaining, so an `embeddings/current`
left pointing at a deleted bundle produced a chatbot that answered
every question from an empty collection. Verified against main, the
missing-bundle failure was "AttributeError: 'NoneType' object has no
attribute 'glob'"; it now names the database, lists what is installed,
and gives the command to fix it.

The userguide bundle is deliberately optional, and react_to_me.py
already resolved it itself and degraded on absence -- that path keeps
get_dir, and require_dir raises FileNotFoundError precisely so the
existing `except FileNotFoundError` there still means what it meant.

The budget. MAX_DOCUMENTS_PER_COLLECTION is now
DEFAULT_MAX_DOCUMENTS_PER_COLLECTION, the default of a constructor
argument. As a module constant, comparing two budgets meant editing the
file and restarting -- so the answer-quality evaluation that is
supposed to settle the number could not be run at all.

Verified end to end against the Release95 bundle: the assembled chain
answers a real question identically through invoke and ainvoke (40 docs
each), budget=3 over 4 collections yields 12 documents and budget=10
yields 40, and all three profiles construct or refuse correctly.
@adamjohnwright
adamjohnwright merged commit ee7c897 into main Sep 9, 2026
10 checks passed
@adamjohnwright
adamjohnwright deleted the feat/retriever-stage3 branch September 9, 2026 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant