Skip to content

SEAB-7771 Page get_tool's versions through TRS /versions - #14

Merged
denis-yuen merged 3 commits into
developfrom
feature/pagination_versions
Oct 1, 2026
Merged

denis-yuen merged 3 commits into
developfrom
feature/pagination_versions

Conversation

@denis-yuen

@denis-yuen denis-yuen commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Description
See what an MCP with a modified TRS dockstore/tool-registry-service-schemas#24 with pagination would function like. Some modest improvements.

Generated description:
Dockstore's TRS API now pages a tool's versions (SEAB-7771, dockstore/dockstore#6374), so get_tool fetches the tool with includeVersions=false and one page of /tools/{id}/versions in parallel. It returns up to version_limit versions (default 100) plus next_version_offset, which it reads from the next_page header. Against a local webservice with SEAB-7771 and a production data copy, ReblockGVCF's first page takes 0.16 s and 56 KB, down from 0.62 s and 749 KB for all 1,439 versions. Going through every version takes 15 calls at the default page size, or 2 calls with summary and version_limit=1000. A Dockstore without SEAB-7771 gets a clear error rather than every version. get_tool_version now also returns the version's own description, which only the single-version endpoint fills in. list_tools caps limit at 100, the most /tools serves per page: with a larger limit its total came out wrong, 70,027 instead of 7,027.

Review Instructions
After the linked PR is merged, https://github.com/dockstore/dockstore/pull/6374/changes#r4138373882 it will be possible to run the prompts in #12 (comment) and see a modest improvement in response size.

Not a huge difference, but an ok exercise to see how updating TRS for new parameters might look.

Issue
https://ucsc-cgl.atlassian.net/browse/SEAB-7771

Security and Privacy

None

  • Security and Privacy assessed

e.g. Does this change...

  • Any user data we collect, or data location?
  • Access control, authentication or authorization?
  • Encryption features?

Please make sure that you've checked the following before submitting your pull request. Thanks!

  • Check that you pass the basic style checks and unit tests by running make check
  • Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
  • If you are changing dependencies, check the Snyk status check or the dashboard to ensure you are not introducing new high/critical vulnerabilities
  • Assume that arguments passed to a tool can be malicious, and sanitize and/or check for Denial of Service type values, e.g., massive sizes
  • Do not log or return secrets/credentials in a tool's response, error message, or debug output
  • If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.

🤖 Generated with Claude Code

get_tool now fetches the tool with includeVersions=false and one page of
/tools/{id}/versions at the same time, returning up to version_limit
versions (default 100) and the next page's offset, read from Dockstore's
next_page header. A page can be short or empty before the last one, so
models are told to keep going until next_version_offset is unset. A
Dockstore without SEAB-7771 sends no paging headers, and get_tool now fails
there rather than returning every version.

get_tool_version also returns the version's own description, which
Dockstore only fills in for a single version.

list_tools now caps limit at 100, the most Dockstore's /tools serves in a
page. Above that, its total and next_offset came out wrong.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@denis-yuen denis-yuen self-assigned this Sep 29, 2026
Comment thread README.md
| ----------------------------- | ----------- | ------------------------------------------------------------------------------------ |
| `get_trs_info` | ✅ | Reports instance and version; unless `local_only`, also TRS info and tool classes. |
| `list_tools` | ✅ | Lists one page of TRS tools, optionally filtered by name, class, language, etc. |
| `get_tool` | ✅ | Retrieves one TRS tool by id, with all of its versions in full or summarized. |

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty much the bottom line here, a little faster here by default

ToolPage becomes PageOfTools, and ToolDetail becomes
ToolWithPageOfVersions. ToolPage read like a kind of tool rather than a
page of them, and ToolDetail read like the full form of ToolSummary,
though the two come from different tools and a ToolDetail's versions can
themselves be summaries. The private get_tool_page helper becomes
get_page_of_tools to match. No tool's output fields change, only the
titles in its output schema.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@denis-yuen

denis-yuen commented Sep 29, 2026 •

Copy link
Copy Markdown
Member Author

This comment was generated by Claude (Claude Code).

Replaying the #12 evaluation prompts against this branch and develop

The prompts are the ones from #12 (comment). All of them work with this branch. The only error in any run is search_entries, which isn't implemented and fails the same way on develop. Response sizes match develop byte for byte except in three places: D is much smaller per call, A grows by 2.2 KB, and E grows by 66 bytes. The renames in 05faa79 changed nothing in the output: every size is identical to the run before them.

Both versions ran in-process against a local webservice with SEAB-7771 (dockstore/dockstore#6374) and a production copy of the data: 5 alternating runs each, after a warm-up. Times are medians and include Dockstore but not MCP transport. Sizes are the text the agent receives. RB = #workflow/github.com/broadinstitute/warp/ReblockGVCF.

Prompts A–G, as recorded

Prompt develop This branch Change
A. "using Local_mcp_2_dockstore_prod can you find the reblockGVCF pipeline from the Broad, I want to work with the develop version of it" 2.09 s, 9,163 B 2.16 s, 11,374 B +2,211 B, all from the version's own description in get_tool_version (1,425 → 3,636 B). The time difference is noise: it's all in the unchanged list_tools step (1.67 → 1.72 s).
B. "Can you tell me potential issues with the wdl?" 0.09 s, 39,484 B 0.09 s, 39,484 B none
C. "run the lint tools on ReblockGVCF, show me any MCP calls that you make" 0.09 s, 9,994 B 0.09 s, 9,994 B none
D. "Ok, let's try something different. Can you list all the versions for ReblockGVCF, sort them all in a couple of interesting ways. Show me the MCP calls that you make along with response sizes and timings" 0.62 s, 748,530 B, all 1,439 versions 0.15 s, 55,757 B, first 100 versions 75% faster, 93% smaller, but only one page
D again, with summary on its get_tool call 0.48 s, 145,231 B, all 1,439 0.14 s, 12,991 B, first 100 71% faster, 91% smaller, one page
E. "before that, do you understand TRS ids? Can you use the MCP to find #workflow/github.com/iwc-workflows/sars-cov-2-variation-reporting/COVID-19-VARIATION-REPORTING for example? Show me the MCP calls made and their timings" 0.10 s, 6,924 B 0.09 s, 6,990 B +66 B, the three version-paging fields
F. "Look at Local_mcp_2_dockstore_prod, can you summarize two pages of tools when listing them, display the mcp calls that you make" 0.34 s, 17,498 B 0.33 s, 17,498 B none
G. "Get me a list of RNAseq workflows from Dockstore and group by language, but also find out how many versions of each workflow there are" 3.97 s, 30,273 B 3.96 s, 30,273 B none

Prompt D when every version is needed

With this branch, getting all 1,439 versions takes several get_tool calls:

How Calls Time Size Compared with develop's single call
Default page size (100), full versions 15 2.26 s 786,028 B 3.7× slower, 5% bigger than the full call
Default page size, summary 15 1.96 s 182,729 B 4.1× slower, 26% bigger than the summary call
summary, version_limit=1000 2 0.72 s 147,972 B 1.5× slower, 2% bigger than the summary call

The extra bytes come from repeating the tool's metadata, including its 2.2 KB README, in every call's response. In a real session each call is also a separate model turn, which these times don't include, so the 15-call routes cost more than the table shows.

What this means

  • Unchanged prompts: the four that don't touch get_tool or get_tool_version (B, C, F and G) return identical bytes in the same time.
  • Single-version questions: looking up a large entry's latest or develop version is now much cheaper. The first page holds the most recently modified versions.
  • "Every version" questions: these cost more calls unless the model uses summary with version_limit=1000. That route is close to develop's cost, but nothing in get_tool's description currently points the model to it. This PR leaves the description as it is; a one-sentence hint could be a follow-up.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
from dockstore_mcp.config import Settings
from dockstore_mcp.models import (
FileWrapper,
PageOfTools,

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed ToolPage to PageOfTools to parallel ToolWithPageOfVersions and emphasize that a ToolPage is not a type of Tool.

``next_version_offset`` as ``version_offset`` (a page number, not an item index).
Keep going until it is unset: a page can come back short, or even empty, before the last.

Paging only speeds up the response for workflows, where Dockstore pages in the database,

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example of how to hint to the agent what MCP tools may be better than others in certain situations

@denis-yuen
denis-yuen marked this pull request as ready for review September 30, 2026 14:40
@denis-yuen
denis-yuen requested review from a team and svonworl and removed request for a team September 30, 2026 14:40
@denis-yuen
denis-yuen merged commit 8d3d600 into develop Oct 1, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants