Skip to content

SEAB-7740 Add slimmed TRS MCP tools - #12

Merged
denis-yuen merged 23 commits into
developfrom
feature/trim_tools
Sep 29, 2026
Merged

denis-yuen merged 23 commits into
developfrom
feature/trim_tools

Conversation

@denis-yuen

@denis-yuen denis-yuen commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Description

Implements TRS tools but removing all the convenience methods that are useful for humans but redundant for AI.
i.e. we had separate methods for getting container files, tests, and descriptors but they were already implemented as simple file retrieval in the web service
The hello method was redundant with TRS server info command (added a mode to just show that the MCP server was up without talking to the webservice) and also merged the process of getting tool versions with getting a tool controlled through arguments. We end up with 5 total implemented tools.

Generated comment follows:
Trims the MCP server from 17 tools to 9 so agents load less and have fewer overlapping choices. The tool definitions the model sees shrink from 18,267 B to 14,451 B (68,673 B → 50,133 B with output schemas, which Claude Code doesn't pass to the model). list_tool_classes and hello fold into get_trs_info, which also reports the Dockstore URL and server version and, with local_only=True, skips Dockstore as a smoke test. get_tool_descriptor, get_tool_tests and get_tool_containerfile fold into get_tool_descriptor_by_path, which fetches the primary descriptor when relative_path is omitted. get_tool_files folds into get_tool_version(files=…), search_tools into list_tools (a proper non-TRS search will come separately), and list_tool_versions into get_tool(summary=…).

Review Instructions
There's a generated comment below with the prompts that I tested the MCP with, try them and see if the MCP as deployed in QA gives useful/realistic results.

Issue

Security and Privacy

None

  • Security and Privacy assessed

e.g. Does this change...

  • Any user data we collect, or data location?
  • Access control, authentication or authorization?
  • Encryption features?

Please make sure that you've checked the following before submitting your pull request. Thanks!

  • Check that you pass the basic style checks and unit tests by running make check
  • Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
  • If you are changing dependencies, check the Snyk status check or the dashboard to ensure you are not introducing new high/critical vulnerabilities
  • Assume that arguments passed to a tool can be malicious, and sanitize and/or check for Denial of Service type values, e.g., massive sizes
  • Do not log or return secrets/credentials in a tool's response, error message, or debug output
  • If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.

🤖 Generated with Claude Code

denis-yuen and others added 12 commits September 24, 2026 14:10
Implements the two unauthenticated, parameterless TRS V2 endpoints
(service-info and toolClasses), plus a shared camelCase-to-snake_case
casing helper for translating their responses into our models.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ckstore

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Adds list_tools, search_tools, get_tool, list_tool_versions, get_tool_version,
get_tool_descriptor, get_tool_descriptor_by_path, get_tool_files, get_tool_tests,
and get_tool_containerfile to trs.py, backed by Dockstore's TRS V2 endpoints,
along with the TRS models they return.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ref comes from DOCKSTORE_MCP_GIT_REF, which the Docker image sets from a
GIT_REF build argument (filled in by the tagged deploy workflow and the
Makefile), falling back to the package version when unset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
list_tools and search_tools now return a ToolPage with the total across every
page and the next page's offset. Dockstore only reports the last page's offset,
computed as floor(total / limit), so the total is counted from the last page's
contents rather than read from the header.

Adds the JUPYTER and SERVICE descriptor types Dockstore's TRS API accepts, so
notebooks' descriptors and files can be fetched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With summary set, each tool comes back as a ToolSummary (id, name,
organization, class, descriptor languages across its versions, version names,
and the first 200 characters of its description) instead of the full TRS
record. The nine RNA-seq name searches against production shrink from about
724 KB to 43 KB this way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Monorepo workflows such as broadinstitute/warp have a version for every
branch and tag of the repository, so a summary of one ReblockGVCF search
was 57 KB, 94% of it version names. Summaries now list at most 10 names,
production-ready versions first, plus version_count and versions_truncated
so callers know when the list is incomplete. The same search is now 5 KB.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Monorepo workflows list a version for every branch and tag of their
repository: broadinstitute/warp's ReblockGVCF has 1,433, a 793 KB
response. With summary, each version is just its name, meta_version and
is_production, which brings that entry down to 151 KB. Paging through the
versions is left to a follow-up ticket, since Dockstore's TRS endpoint
ignores limit and offset.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
list_tool_classes only added a little more about the TRS instance, so an
agent needed two calls to learn what it was talking to. get_trs_info now
fetches service-info and toolClasses concurrently and returns the classes
as tool_classes, and list_tool_classes is removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
get_tool_descriptor, get_tool_tests and get_tool_containerfile each fetch
a file that get_tool_files already lists, and get_tool_descriptor_by_path
returns the same content for every one of them. Remove the three tools and
make relative_path optional, so omitting it still fetches the primary
descriptor without waiting on get_tool_files. That leaves 13 tools and
trims about 5.8 KB from the tool schemas agents load.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
hello only reported which Dockstore instance the server is attached to
and its version, which is what get_trs_info is for. get_trs_info now
always reports dockstore_url and server_version, and local_only=True
returns just those without contacting Dockstore, keeping hello's use as
a smoke test. That leaves 12 tools.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@denis-yuen denis-yuen self-assigned this Sep 24, 2026
@denis-yuen

denis-yuen commented Sep 24, 2026 •

Copy link
Copy Markdown
Member Author

This comment was generated by Claude (Claude Code).

Prompts used to evaluate these tool changes

These are the prompts from today's sessions that used the Dockstore MCP (local_mcp_2_dockstore_prod), plus G from #6's sample run. RB = #workflow/github.com/broadinstitute/warp/ReblockGVCF.

The table is for the 9-tool head (e6ec87d). Each prompt's recorded calls, mapped to replacements for removed tools, were replayed 3 times against Dockstore production in-process. Times are medians, with calls sent together run in parallel, and include Dockstore but not MCP transport. Sizes are the tool result bytes the agent receives.

Prompt MCP calls Response time Response size
A. "using Local_mcp_2_dockstore_prod can you find the reblockGVCF pipeline from the Broad, I want to work with the develop version of it" search_entries, list_tools(toolname=ReblockGVCF, summary, limit 50), then get_tool_version(files=WDL) and get_tool_descriptor_by_path sent together (RB, develop, WDL) 2.06 s: list_tools 1.33 s, then the two together 0.72 s. search_entries fails at once as not yet implemented 9.3 KB: list_tools 4.8 KB, descriptor 3.0 KB, version and files 1.4 KB
B. "Can you tell me potential issues with the wdl?" get_tool_descriptor_by_path × 2 (…/tasks/wdl/GermlineVariantDiscovery.wdl, Qc.wdl) 0.09 s 39.5 KB: 14.8 KB + 24.7 KB
C. "run the lint tools on ReblockGVCF, show me any MCP calls that you make" get_tool_descriptor_by_path (…/tasks/wdl/Utilities.wdl) 0.05 s 10.0 KB
D. "Ok, let's try something different. Can you list all the versions for ReblockGVCF, sort them all in a couple of interesting ways. Show me the MCP calls that you make along with response sizes and timings" get_tool(RB) 0.89 s 754.5 KB, all 1,434 versions in full
E. "before that, do you understand TRS ids? Can you use the MCP to find #workflow/github.com/iwc-workflows/sars-cov-2-variation-reporting/COVID-19-VARIATION-REPORTING for example? Show me the MCP calls made and their timings" get_tool 0.05 s 7.0 KB
F. "Look at Local_mcp_2_dockstore_prod, can you summarize two pages of tools when listing them, display the mcp calls that you make" list_tools(20, 0, summary), list_tools(20, 1, summary) 0.86 s: 0.42 s + 0.46 s 17.5 KB: 9.1 KB + 8.4 KB
G. "Get me a list of RNAseq workflows from Dockstore and group by language, but also find out how many versions of each workflow there are" list_tools × 9 sent together (tool_class=Workflow, summary, limit 100), name and toolname each as rnaseq, RNAseq, rna-seq, RNASeq, plus toolname=RNA-seq 2.06 s, set by the slowest call (name=rnaseq); the others took 0.43–1.57 s 30.3 KB across the 9; 57 workflows after deduplicating by TRS id, 804 versions

Compared with the 17-tool version this PR started from (825c0f2, in alternating runs): the tool definitions the model loads each session shrink from 18,267 B to 14,451 B (−21%), or 68,673 B to 50,133 B (−27%) with output schemas, which Claude Code doesn't pass to the model. The prompts get the same answers from one call fewer. Only A and D change: in A, search_tools becomes list_tools and get_tool_version plus get_tool_files become one get_tool_version(files=WDL); in D, list_tool_versions becomes get_tool. Searches and descriptors are byte-identical, get_tool_version(files=WDL) adds only 28 B for its files and files_error keys, and D's get_tool returns the same 1,434 versions plus 2.6 KB of tool metadata. There is no performance loss: summed medians are 6.26 s for 17 tools and 6.06 s for 9, and no prompt differs by more than the spread between runs, since Dockstore's TRS API sets the response times.

@denis-yuen
denis-yuen changed the base branch from feature/trs_tweaks to develop September 25, 2026 14:05
# Conflicts:
#	README.md
#	src/dockstore_mcp/models.py
#	src/dockstore_mcp/tools/__init__.py
#	src/dockstore_mcp/tools/trs.py
#	tests/test_server.py
#	tests/test_trs.py
get_tool_version takes an optional files descriptor type: given one, it
fetches the version and its file listing in parallel and returns them
together, so the tool list drops from 12 to 11.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@denis-yuen denis-yuen changed the title SEAB-7740 Trim the MCP tool list from 17 to 12 SEAB-7740 Trim the MCP tool list from 17 to 11 Sep 25, 2026
list_tools takes search_tools' filters: with none it pages through every
tool as before. The tool list drops from 11 to 10, and since each tool
carried its own copy of the ToolPage output schema, the schemas shrink
by about 9 KB. A non-TRS search will come separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@denis-yuen denis-yuen changed the title SEAB-7740 Trim the MCP tool list from 17 to 11 SEAB-7740 Trim the MCP tool list from 17 to 10 Sep 25, 2026
get_tool takes list_tool_versions' summary option, which shrinks each
version to its name, meta_version, and is_production. Dockstore's
/tools/{id} already returns every version, so this needs no extra
request, and the tool list drops from 10 to 9.

get_tool returns a new ToolDetail, whose versions are either full or
summarized, rather than a union of models, which FastMCP would wrap
under "result". Also export ToolVersionWithFiles from models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@denis-yuen denis-yuen changed the title SEAB-7740 Trim the MCP tool list from 17 to 10 SEAB-7740 Trim the MCP tool list from 17 to 9 Sep 25, 2026
denis-yuen and others added 2 commits September 25, 2026 10:50
get_tool_version with files used to fail outright if only the file
listing request failed. It now returns the version with files unset and
a files_error saying why; a missing version still fails the call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@denis-yuen denis-yuen changed the title SEAB-7740 Trim the MCP tool list from 17 to 9 SEAB-7740 Add slimmed TRS MCP tools Sep 25, 2026
denis-yuen and others added 3 commits September 25, 2026 11:05
The two differed only in whether versions could be summaries. Tool's
versions now take either, with left_to_right so that a TRS response
always parses as full versions, and get_tool swaps in summaries in
place. Responses are byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/health, get_trs_info's server_version, and the version the server
reports to MCP clients all used the package version, 0.1.0, so none of
them showed which release was running. They now share the User-Agent's
Settings.server_version: the git ref the server was built from (the
release tag, for published images), else the package version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread README.md
## Tools

| Tool | Description |
| ------------------- | ------------------------------------------------------------------------------------- |

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is most useful for a breakdown of what the tools are and what they do

Comment thread Makefile
PIP ?= $(VENV)/bin/pip
IMAGE ?= dockstore/dockstore-mcp:local
# Reported in the User-Agent the server sends to Dockstore.
GIT_REF ?= $(shell git describe --tags --always 2>/dev/null)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This git ref business is to hook-up the user agent version to the mcp version which we will also expose in Slack deploy messages and the footer

description: str | None = Field(default=None, description="Longer explanation of what this class is.")


class TrsDescriptorType(StrEnum):

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These models are basically just TRS models

included_apps: list[str] | None = Field(default=None, description="Apps bundled with this version.")


class ToolVersionSummary(BaseModel):

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This summary business allows the agent to pick to get lists of versions or tools with fewer fields. The content is still transferred from the webservice to the MCP but not over the Internet to the end user

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@denis-yuen
denis-yuen marked this pull request as ready for review September 25, 2026 16:10
@denis-yuen
denis-yuen requested review from a team and svonworl and removed request for a team September 25, 2026 16:10
@denis-yuen
denis-yuen requested review from svonworl and removed request for svonworl September 28, 2026 17:08

@svonworl svonworl left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discussed, I'm not convinced that TRS is the ideal AI interface for Dockstore. The TRS "tool" terminology isn't consistent with Dockstore's notion of a tool ("CommandLineTool"), and some important information about Dockstore entries isn't accessible via TRS. That said, this streamlined version is better than a literal conversion of the entire TRS API, and it'll give us a something to benchmark and compare against the other in-process implementation.

@denis-yuen

Copy link
Copy Markdown
Member Author

The TRS "tool" terminology isn't consistent with Dockstore's notion of a tool ("CommandLineTool"), and some important information about Dockstore entries isn't accessible via TRS.

FWIW, we can/should be making more information accessible via TRS either as official proposals for extension or unofficial extensions.

For terminology, I'm not opposed to renaming/describing entities differently as they flow through MCP interface after being retrieved from the TRS API.

@denis-yuen
denis-yuen merged commit 911346c into develop Sep 29, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants