Skip to content

feat: Add opt-in WebMCP tools for Shiny apps - #2513

Draft
karangattu wants to merge 11 commits into
mainfrom
shiny-webmcp-example
Draft

karangattu wants to merge 11 commits into
mainfrom
shiny-webmcp-example

Conversation

@karangattu

@karangattu karangattu commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

This PR lets browser agents use existing Shiny apps through WebMCP. The agent and the person share the same open app. Both can change its controls and read its results.

The feature is experimental and off by default. Set SHINY_WEBMCP=1 to enable it for an existing app. Core apps can also use App(..., webmcp=True). Browsers without WebMCP still run the app normally.

Browser support

WebMCP browser support is evolving (status as of October 1, 2026). This PR requires document.modelContext.registerTool and a compatible browser agent.

Browser / environment WebMCP support Setup / limitations
Chrome Experimental preview Enable chrome://flags/#enable-webmcp-testing and restart for local testing; an origin trial is available from Chrome 149.
Microsoft Edge Experimental preview An origin trial is available from Edge 150; use a build with WebMCP enabled.
ChatGPT Desktop browser Supported Use its WebMCP-capable browser agent.
Brave Experimental Support is available in Leo AI chat; availability depends on the build.
Firefox No released native support WebMCP tools are unavailable.
Safari No released native support WebMCP tools are unavailable.
Other browsers, or WebMCP disabled Not supported unless the required API is exposed Sharing the Chromium engine alone does not guarantee support.

When the required API is absent, Shiny skips tool registration and the app continues to work normally. This PR does not bundle a WebMCP polyfill. SHINY_WEBMCP=1 enables Shiny's integration; it does not enable the browser API or enroll the site in an origin trial.

Sources: WebMCP browser and agent implementation status, Chrome setup.

When to use each tool

These tools work at different levels. Shiny's Playwright controllers are helpers for browser controls. Choose the tool based on what you need to do.

Tool When to use it Example
WebMCP Let an agent discover named actions and use them in a person's open app. Requires a browser that supports WebMCP. Ask an agent to compare sales and leave the dashboard on the selected region.
Playwright with Shiny controllers Test or automate controls in a real browser, including controls that WebMCP does not support. Select a region and make sure that the browser shows the expected result.
local_server Test Python calculations, input rules, and reactive updates without a browser. It does not test browser controls or page appearance. Set region and channel inputs, then assert that the calculated revenue is 600.

Tools for existing apps

Shiny provides four tools:

  • Describe the app with shiny_describe_app. It lists visible supported inputs, their current values, allowed arguments, available actions, and output status.
  • Change inputs with shiny_set_inputs. It makes sure that values are allowed, updates the actual controls, and returns results after Shiny finishes its queued reactive updates.
  • Read displayed text and code with shiny_read_outputs. Results include status information and a length limit. Visual, HTML, and data-frame outputs report that they are unsupported.
  • Press an action button with shiny_invoke_action. Only buttons marked data-webmcp="action" are available. Use data-webmcp="exclude" to omit an element and its children from automatic discovery.

Each tool has a schema, which defines its allowed arguments. Shiny updates schemas when controls, choices, or limits change. Input IDs keep their module prefixes.

The tools omit hidden controls, password inputs, and file inputs. They reject changes to disabled controls and choices. Calls use the session's existing WebSocket connection between the browser and server.

The browser receives a response after Shiny sends the related output updates. Calls handle lost connections and cancellation, and stop waiting after 30 seconds. They do not wait for background tasks. Cancellation does not undo Python work that already started.

Tools written in Python

While automatic tools let an agent interact with the UI, you can also write custom Python functions that agents can call directly using the @webmcp.tool decorator. This is useful for performing calculations, querying backend databases, or returning structured data.

Basic Example

Decorate your function inside the server function (or at the top level in Shiny Express):

from shiny import webmcp

@webmcp.tool(
    description="Multiply the current quantity by a factor.",
    input_schema={
        "type": "object",
        "properties": {"factor": {"type": "number"}},
        "required": ["factor"],
        "additionalProperties": False,
    },
    read_only=True,
)
def multiply(factor: float):
    return {"result": input.quantity() * factor}

How it works

  • Function requirements: Can be synchronous (def) or asynchronous (async def). Return values must be JSON-serializable (e.g. dicts, lists, numbers, strings).
  • Argument validation: You provide an input_schema (a standard JSON Schema object). Shiny uses jsonschema to validate the agent's arguments before invoking your Python function.
  • Safe reactivity: Your tool function can read reactive inputs and calculations (input.x()) in an isolated context without creating reactive dependencies that trigger unwanted reruns of outputs.
  • Session-scoped & isolated: Tools exist only for the duration of the user's active session and are cleanly unregistered when the session ends. In modules, tool names are automatically namespaced.
  • Agent hints: read_only=True indicates the tool just reads data without changing state, while consequential=True indicates significant side effects. These provide guidance to the agent's planner.
  • Error handling: Errors follow the app's sanitization rules to avoid leaking sensitive server details.

Example: Quickstart & How to Use

This feature allows a person and an AI browser agent to collaborate in the same open Shiny session. The agent has access to two types of tools:

  1. Automatic UI tools (shiny_*): Built-in tools that let the agent discover inputs, modify controls, click exposed buttons, and read outputs.
  2. Custom Python tools (@webmcp.tool): Domain-specific tools you define in Python to calculate metrics or return structured data directly to the agent.

1. Run the sales explorer example

To try the bundled example app (which enables WebMCP via webmcp=True):

shiny run examples/webmcp/app.py --port 8000

(To enable automatic WebMCP tools on any existing Shiny app without code changes, start it with SHINY_WEBMCP=1 shiny run app.py)

2. Interact with the app alongside an agent

Open the app in a browser that supports WebMCP (e.g., Chrome with WebMCP enabled). Both you and the browser agent share the open dashboard.

Ask your browser agent:

"Compare West online sales with retail, and leave the dashboard showing online sales."

Here is what happens step-by-step:

  1. Query data directly (Custom tool): The agent calls the custom tool compare_channels(region="West") to calculate sales on the server without changing the user's dashboard view, returning:
    {"region": "West", "currency": "USD", "Online": 600, "Retail": 300, "online_minus_retail": 300}
  2. Update the live dashboard (Automatic tool): The agent calls shiny_set_inputs(values={"region": "West", "channel": "Online"}).
  3. Reactive synchronization: Shiny updates the dropdown inputs, triggers reactive calculations, and updates the table and summary in the browser. Both you and the agent see the updated results in real time.

3. How to add a custom tool to your own app

Adding a tool for an agent to use in your server function is simple:

from shiny import App, Inputs, Outputs, Session, ui, webmcp

app_ui = ui.page_fluid(
    ui.input_select("region", "Region", ["North", "South", "West"]),
    ui.output_text("result"),
)

def server(input: Inputs, output: Outputs, session: Session):
    @webmcp.tool(
        description="Calculate total revenue for a specific region.",
        input_schema={
            "type": "object",
            "properties": {"region": {"type": "string"}},
            "required": ["region"],
        },
        read_only=True,
    )
    def get_revenue(region: str):
        return {"region": region, "revenue": 600}

app = App(app_ui, server, webmcp=True)

The README and bundled Shiny agent reference explain setup, supported controls, custom tools, and limits.

SkillDiff: Evaluate WebMCP v/s Playwright

Ran SkillDiff with GPT-6 Sol, using this task:

Compare West online and retail revenue, report the difference, and leave the dashboard showing West online sales.

Both agents returned $600 online, $300 retail, +$300 difference and left the correct filters selected.

Metric Playwright WebMCP + reference Observed change
Outcome checks passed 7/7 7/7 Equal correctness
Agent elapsed time 77.63s 39.73s 48.8% less
Browser operations 11 4 63.6% fewer
Tokens, including cached input 245,115 132,937 45.8% fewer
Estimated API-equivalent cost $0.0990 $0.0675 31.8% lower

When running with GPT-6 Luna we got this table

Metric Playwright WebMCP + Shiny reference Observed difference
Task score 100% (5/5) 100% (5/5) Tied in all pairs
Median task time 34.9s 27.2s -8.0s paired mean (95% CI -12.9 to -3.2)
Median browser interactions 8 3 62.5% fewer
Median session tokens 127,441 126,594 -20,205 paired mean (95% CI -55,599 to +16,671)
Estimated API cost, five runs $0.01498 $0.01291 $0.00207 lower

@mconflitti-pbc

Copy link
Copy Markdown

Very glad to see this being worked for shiny! Thanks for taking it on. Excited to use this.

karangattu and others added 4 commits October 1, 2026 09:47
crypto.randomUUID() is unavailable in insecure contexts, so every tool
that flushed failed on plain-http hosts. The flush input is sent with
event priority, which always resends, so a counter is sufficient.

Tool schemas now default additionalProperties to False. Unknown keys
previously passed validation and surfaced as a TypeError from the
keyword call, which the sanitizer turned into an opaque error.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants