Correct a claim I made three times today: 009 has not landed - #253
Merged
Merged
Conversation
I marked spec 010's T021 done, wrote in the contract that collection routing "already did most of the work on retrieval", and told Adam it was the main reason first-token fell from ~36s to 9.6s. All three are wrong. What landed is *source* routing: `resolve_active_sources` sends a question to the Reactome bundle, the user guide, or a live lookup. Spec 009 is *collection* routing -- choosing among the five collections inside the Reactome bundle -- and it is not implemented. `QueryIntent` has no `collections` field, `resolve_collections` does not exist, and `retrieve_documents` still loops over every collection it has. I conflated the two because both narrow retrieval and both were described to me as routing. The check that would have caught it is the one I eventually ran: grepping for the function the spec names rather than for a concept. The honest position on the latency improvement is now recorded as partly unexplained. Source routing accounts for the fast user-guide answers, two-round preprocessing for about 2.5s, and the rest may be that the old figures came from one question and were never representative. Substituting a new confident explanation for the wrong one would repeat the mistake. T021 is open again, and spec 009's task list is accurate as it stands. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documentation only. I marked spec 010's T021 done, wrote in the contract that collection routing "already did most of the work on retrieval", and told Adam it was the main reason first-token fell from ~36s to 9.6s.
All three are wrong.
What actually landed
Source routing.
resolve_active_sourcessends a question to the Reactome bundle, the user guide, or a live lookup. That is real and it works.Spec 009 is collection routing — choosing among the five collections inside the Reactome bundle — and it is not implemented:
collectionsfield onQueryIntentsourceexistsresolve_collections(selected, available)collection_retrieversby selectionretrieve_documentsloops over all fiveI conflated the two because both narrow retrieval and both get called "routing". The check that caught it is the one I eventually ran: grepping for the function the spec names, rather than for a concept.
The latency explanation is now honestly incomplete
Rather than substituting a new confident explanation for the wrong one:
That is recorded in the contract too, since the website team was told the stronger version.
Consequence
T021 is open again, and spec 009's task list — which showed 24 open items I had assumed were stale bookkeeping — is accurate as it stands. Collection routing remains real, unstarted work, and it is still the lever the latency analysis points at.
🤖 Generated with Claude Code