feat(appkit): reference agent-app, dev-playground chat UI, docs, and template

[MarioCadenas]

Final layer of the agents feature stack. Everything needed to
exercise, demonstrate, and learn the feature.

Reference application: agent-app

apps/agent-app/ — a standalone app purpose-built around the agents
feature. Demonstrates every major capability in one place:

• Markdown orchestrator (config/agents/assistant.md, default)
  with destructive file tools (upload, delete) for HITL demo, and
  agents: [support, researcher] delegating to both a markdown
  sibling and a code-defined specialist.
• Markdown specialist (config/agents/support.md) — full analytics
  • files toolkits, plus ambient get_weather.
• Code-defined specialist (researcher in server.ts) — defined
  in code specifically because its MCP tool set is conditional on
  runtime env vars, which markdown frontmatter can't express.
  Referenced from assistant.md via the markdown → code cross-reference.
• server.ts — concise: ambient tool() factories, conditional MCP
  wiring, zero-trust host allowlist derived from the same env vars,
  agents() plugin config with autoInheritTools and mcp.trustedHosts.
• Vite + React 19 + TailwindCSS frontend with a chat UI showing
  streaming tokens, tool calls, and an approval card that approves or
  denies destructive tool requests over /api/agents/approve.
• Databricks deployment config (databricks.yml, app.yaml) and
  .env.example for local dev.

dev-playground chat UI + autocomplete agent

apps/dev-playground/client/src/routes/agent.route.tsx — chat UI
with inline autocomplete (hits the autocomplete markdown agent
configured with ephemeral: true) and a full threaded conversation
panel (hits the default agent).

apps/dev-playground/server/index.ts — code-defined helper agent
using fromPlugin(analytics) alongside the markdown-driven
autocomplete agent in config/agents/, demonstrating the mixed
setup against the same plugin list. Route tree (routeTree.gen.ts)
regenerated to include the new /agent route.

Docs

docs/docs/plugins/agents.md — progressive guide covering:

1. Drop a markdown file → it just works.
2. Scope tools via toolkits: / tools: frontmatter.
3. Code-defined agents with fromPlugin().
4. Sub-agents (markdown → markdown, markdown → code, code → code).
5. Standalone runAgent() (no createApp or HTTP).

Plus configuration reference (including approval, limits, mcp
keys), runtime API reference, and a full frontmatter schema table.

docs/docs/api/appkit/ — typedoc regenerated for the full agents
public surface including AgentDefinition.ephemeral,
AgentsPluginConfig.{approval, limits, mcp}, updated
loadAgentFromFile / loadAgentsFromDir signatures, expanded
AgentEvent union, and ToolkitEntry annotations.

Template

template/appkit.plugins.json — adds the agents plugin entry so
npx @databricks/appkit init --features agents scaffolds the plugin
correctly.

Test plan

• Full appkit vitest suite: 1552 tests passing at stack tip
• Typecheck clean across all 8 workspace projects
• pnpm docs:build clean (no broken links)
• pnpm --filter=@databricks/appkit build:package clean, publint clean

Signed-off-by: MarioCadenas MarioCadenas@users.noreply.github.com

PR Stack

1. Shared agent types + LLM adapters — feat(appkit): shared agent types and LLM adapter implementations #301
2. Tool primitives + ToolProvider surfaces — feat(appkit): tool primitives and ToolProvider surfaces on core plugins #302
3. Plugin infrastructure (attachContext + PluginContext) — feat(appkit): plugin infrastructure — attachContext + PluginContext mediator #303
4. agents() plugin + createAgent(def) + markdown-driven agents — feat(appkit): agents() plugin, createAgent(def), and markdown-driven agents #304
5. fromPlugin() DX + runAgent plugins arg + toolkit-resolver — feat(appkit): fromPlugin() DX, runAgent plugins arg, shared toolkit-resolver #305
6. Reference app + dev-playground + docs (this PR)

Demo

agent-demo.mp4

[pkosiec]

(not yet a full review, just some partial comments)

[pkosiec]

Why do we need a name for a tool which is already registered as a unique key?

[pkosiec]

I'm not sure if it isn't a bit confusing to have such distinction - in the end, it's always about tools (but either custom ones or plugin ones.

Maybe we should rename that to: tools and extraTools
or, have something like:

  tools:
    - plugin:analytics
    - plugin:files: { only: [uploads.read] }
    - get_weather

WDYT?

[pkosiec]

That's a lot of the code for the dev playground.

Honestly, I'm not sure if that fits in the dev playground - maybe we should move it somewhere? E.g. Move it as a separate template in app-templates?

IMO this is too good to keep it in the dev playground.
Just saying, it's not a blocker but I think it has a huge potential 👍

[pkosiec]

Actually, as a second thought - the "Smart dashboard" example is kind of reimplementing Metric Views? Correct me if I'm wrong 🤔

[pkosiec]

Wait, this file shouldn't be modified, right? Because there's no breaking change if we haven't released agent plugin yet?

(BTW, why do we have so many "Changelog" headers? 😄 something is wrong with our changelog generation, I guess)

[pkosiec]

Agentic reviews (aggregated report):

Code Review — agent/v2/6-apps-docs

Context

Branch agent/v2/6-apps-docs (based on agent/v2/5-fromplugin-runagent) adds ~22K lines implementing the full agents system: core agent runtime, agents plugin, MCP client, plugin context mediator, tool primitives, ToolProvider surfaces on all core plugins, smart dashboard demo, template updates, and docs. This review covers both correctness bugs and developer experience issues.

---

What works well

• Progressive disclosure — 5 levels (markdown drop-in → frontmatter scoping → code agents → sub-agents → standalone) is a strong pedagogical ladder
• tool() ergonomics — Zod schema drives type inference + JSON Schema generation + runtime validation with LLM-friendly errors
• fromPlugin() pattern — elegant lazy references; no instance coupling, spread-friendly, clear error messages with Available: [...] listing
• Safety defaults — autoInheritTools: false, approval gates, resource limits, MCP host policy, SQL classifier. Two-key operation for auto-inherit is well-designed
• docs/docs/plugins/agents.md — comprehensive, well-structured, covers the full lifecycle with runnable examples at each level

---

P0 — Must fix

1. Doc/behavior contradiction on auto-inherit

File: docs/docs/plugins/agents.md:57-58 vs :233

Level 1 documentation says:

"4. Auto-inherits every registered ToolProvider plugin's tools (analytics.*, files.*, ...)"

But the Configuration Reference says:

"autoInheritTools defaults to { file: false, code: false } — no tools spread into any agent unless the developer explicitly opts in."

These directly contradict each other. A developer following Level 1 will expect their markdown agent to see all plugin tools out of the box, but it won't — they get zero tools. This is the single most confusing thing in the agents DX.

Fix: Either (a) add agents({ autoInheritTools: { file: true } }) to the Level 1 example, or (b) rewrite the Level 1 narrative to say the agent has no tools yet and point to Level 2.

---

P1 — Should fix before merge

2. MCP callTool corrupts results when text is undefined

File: packages/appkit/src/connectors/mcp/client.ts:264-275

McpToolCallResult.content[].text is typed text?: string (line 58), but callTool maps c.text without filtering undefined:

.filter((c) => c.type === "text")
.map((c) => c.text)       // c.text can be undefined
.join("\n");               // produces "undefined" in output

Fix: Change both occurrences (error path line 267 and success path line 274) to filter:

.filter((c): c is { type: string; text: string } => c.type === "text" && typeof c.text === "string")
.map((c) => c.text)

3. MCP sendNotification ignores HTTP error status

File: packages/appkit/src/connectors/mcp/client.ts:362-389

The notifications/initialized fetch doesn't check response.ok. A 4xx/5xx response is silently ignored, making connect() appear successful when the server may not have registered the client.

Fix: Warn (don't throw — MCP spec says notifications are fire-and-forget):

if (!response.ok) {
  logger.warn("MCP notification %s failed: %s %s", method, response.status, response.statusText);
}

4. MCP SSE response body read has no size limit

File: packages/appkit/src/connectors/mcp/client.ts:339-340

response.text() reads the entire SSE body into memory. A malicious or misconfigured MCP server could send unbounded data.

Fix: Read incrementally with a size cap (e.g., 10 MB), or keep response.text() but add a Content-Length check if the header is present and document the 30s AbortSignal.timeout as the backstop.

5. reload() races with in-flight streams

File: packages/appkit/src/plugins/agents/agents.ts:161-168

reload() calls mcpClient.close() and sets it to null. In-flight streams captured this.mcpClient at _streamAgent call time (line 802). The old client is now closed, so sendRpc throws "MCP client is closed" mid-stream.

Fix: Don't close the old client synchronously — let in-flight streams drain. Or simpler: don't close it at all (it has no keep-alive connections; GC collects after in-flight refs drop).

6. tool() return type lies about string-only

Files: packages/appkit/src/core/agent/tools/tool.ts:19, function-tool.ts:19

ToolConfig.execute is typed as (args) => Promise<string> | string, and FunctionTool.execute matches. But the template's helper.ts:29-30 returns objects:

execute: () => ({ now: new Date().toISOString() }),

This works at runtime (the result gets serialized downstream) but the type is wrong. Developers naturally want to return structured objects from tools.

Fix: Widen ToolConfig.execute and FunctionTool.execute return types to unknown | Promise<unknown> (matching defineTool's handler signature), with serialization handled by the runner.

7. SSE parsing boilerplate in template AgentChat

File: template/client/src/pages/agents/AgentChat.tsx:104-125

~20 lines of manual SSE frame splitting, data-line extraction, and JSON parsing. Every developer who scaffolds an app will copy this verbatim. The comment at line 34 acknowledges the gap. Consider shipping a thin parseSSEStream(reader) async generator in @databricks/appkit-ui so the template doesn't teach low-level SSE plumbing as the canonical pattern.

---

P2 — Fix if straightforward

8. Files and Genie plugin tools don't forward signal

Files:

• packages/appkit/src/plugins/files/plugin.ts:1059-1123
• packages/appkit/src/plugins/genie/genie.ts:63-99

All tool handlers in _defineVolumeTools and _defineSpaceTools ignore the signal parameter from ToolEntry.handler. The analytics and lakebase plugins correctly forward it.

Fix: Add signal parameter to each handler. For genie sendMessage, pass it to abort the async iteration.

9. MCP JSON-RPC response shape not validated

File: packages/appkit/src/connectors/mcp/client.ts:349-351

as JsonRpcResponse is an unsafe cast. A malformed server response could pass through without json.error, json.result, or matching json.id.

Fix: Add minimal validation:

if (typeof json !== "object" || json === null || json.jsonrpc !== "2.0") {
  throw new Error(`MCP response for ${method} is not valid JSON-RPC 2.0`);
}

10. Hardcoded 30s tool execution timeout

File: packages/appkit/src/core/plugin-context.ts:187

const timeout = 30_000; is not configurable. Complex SQL queries or batch operations may need more.

Fix: Accept an optional timeoutMs in the executeTool signature with 30_000 as default.

11. Missing approval timeout validation

File: packages/appkit/src/plugins/agents/agents.ts:124

cfg.timeoutMs from user config is not validated. A negative or zero value causes immediate denial.

Fix: timeoutMs: Math.max(cfg.timeoutMs ?? 60_000, 1_000)

12. Inconsistent tool annotation fields across plugins

Files: lakebase.ts:181-185, analytics.ts:290, files/plugin.ts:1067,1108,1119, genie.ts:76,96

The ToolAnnotations type defines a preferred effect enum ("read" | "write" | "update" | "destructive") but no plugin uses it — all use the deprecated readOnly/destructive booleans. The lakebase plugin sets both readOnly AND destructive (inverse of each other) which is redundant.

Fix: Standardize on the effect field across all plugins. Remove redundant pairs in lakebase.

13. Stale comments reference destructive: true instead of effect

Files: shared/src/agent.ts:126, core/agent/types.ts:149, docs/docs/plugins/agents.md:334

These all reference the legacy boolean form. Since effect is now preferred and destructive is @deprecated, docs and comments should lead with effect.

14. createAgent({ name }) precedence unclear

File: packages/appkit/src/core/agent/types.ts:77-78

AgentDefinition.name is optional ("Filled in from the enclosing key"), but the template explicitly sets it. Does explicit name override the key? What if they differ? The template should either omit name or the docs should explain precedence.

15. tool() vs defineTool() callback naming inconsistency

Files: tool.ts, define-tool.ts

Two tool-creation functions for different audiences (agent authors vs plugin authors) use different callback names: execute vs handler. A developer switching from agent-side to plugin-side tool authoring will wonder why the API changed.

---

P3 — Low priority

16. Sub-agent input extraction is lenient

File: packages/appkit/src/plugins/agents/agents.ts:975-980

Falls back to JSON.stringify(args) when args.input isn't a string. An object like { input: null } serializes the entire args object.

17. AbortSignal.any() requires Node 20+

File: packages/appkit/src/connectors/mcp/client.ts:327

No polyfill or feature detection. Document Node 20+ as minimum requirement or add a fallback.

18. MCP connectAll doesn't surface partial failures clearly

File: packages/appkit/src/connectors/mcp/client.ts:103-116

Failed connections are logged but not thrown. Callers can't distinguish "all connected" from "partially connected".

19. tool() description silently falls back to name

File: tool.ts:40 — description: config.description ?? config.name

If a developer omits description, the LLM sees the tool name as the description (e.g., "get_weather"). Consider logging a warning or making description required.

20. Playground server is too dense for a reference

File: apps/dev-playground/server/index.ts — 750 lines

Mixes agent setup, file policy harness, saved-views CRUD, and OBO/SP smoke tests. Consider extracting non-agent concerns into separate files.

21. No shared SSE/stream hook in appkit-ui

The playground has custom hooks (use-agent-stream.ts, use-action-dispatcher.ts) that demonstrate SSE consumption, but these are local — not reusable. The template reimplements the same SSE parsing inline, creating two divergent implementations.

---

Not a bug (reviewer false positives)

• Budget consumed before approval: Intentional security design ("Counted pre-dispatch so a prompt-injected agent cannot drain the budget silently via denied calls")
• EventChannel push/close race: Node.js is single-threaded; these can't execute concurrently
• Stream cleanup ordering: activeStreams.delete() in driver's finally runs before the generator drains, but approval gate entries are already resolved. The 404 to late approvals is correct behavior
• OTel context leak in asUser() dev proxy: Proxy objects and closures are lightweight; GC handles them fine

[pkosiec]

Actually, as a second thought - the "Smart dashboard" example is kind of reimplementing Metric Views? Correct me if I'm wrong 🤔

[pkosiec]

I missed that part on previous PRs, but now I'm testing the flow and I have a small suggestion: can we make the description more high-level? I don't think we need to mention the fromModelServing stuff

Maybe we should also rephrase that the agent will require DATABRICKS_SERVING_ENDPOINT_NAME if the endpoint is not configured - just to improve the language stating that the env is required.

And why Model Serving (agents)? maybe it should be named Model Serving Endpoint for LLM used by Agent or sth shorter?

[pkosiec]

I initialized an app from the template and got

Error: Failed to register agent 'helper' (code): Agent 'helper' tool 'current_time' has an unrecognized shape
    at AgentsPlugin.loadAgents (/private/tmp/0705/agent/pkosiec-agent-local/node_modules/@databricks/appkit/dist/plugins/agents/agents.js:125:10)
    at async AgentsPlugin.setup (/private/tmp/0705/agent/pkosiec-agent-local/node_modules/@databricks/appkit/dist/plugins/agents/agents.js:86:3)
    at async Promise.all (index 0)
    at async Function._createApp (/private/tmp/0705/agent/pkosiec-agent-local/node_modules/@databricks/appkit/src/core/appkit.ts:225:5) {
  [cause]: Error: Agent 'helper' tool 'current_time' has an unrecognized shape
      at AgentsPlugin.buildToolIndex (/private/tmp/0705/agent/pkosiec-agent-local/node_modules/@databricks/appkit/dist/plugins/agents/agents.js:259:10)
      at AgentsPlugin.buildRegisteredAgent (/private/tmp/0705/agent/pkosiec-agent-local/node_modules/@databricks/appkit/dist/plugins/agents/agents.js:167:32)
      at async AgentsPlugin.loadAgents (/private/tmp/0705/agent/pkosiec-agent-local/node_modules/@databricks/appkit/dist/plugins/agents/agents.js:121:23)
      at async AgentsPlugin.setup (/private/tmp/0705/agent/pkosiec-agent-local/node_modules/@databricks/appkit/dist/plugins/agents/agents.js:86:3)
      at async Promise.all (index 0)
      at async Function._createApp (/private/tmp/0705/agent/pkosiec-agent-local/node_modules/@databricks/appkit/src/core/appkit.ts:225:5)
}

Here's what my agent thinks:

 Root cause: The template's tool() calls omit name (e.g. tool({ description, schema, execute })). The tool()
  factory sets name: config.name → undefined. Then isFunctionTool() checks typeof obj.name === "string" →
  false, so it falls through all shape checks and throws "unrecognized shape".

  The irony: buildToolIndex already overrides the name with the object key at line 407: def: {
  ...functionToolToDefinition(tool), name: key }. So name in tool() is redundant — but the type guard rejects
  it before that line is reached.

  Fix: Make name optional in ToolConfig and relax isFunctionTool to not require it, since buildToolIndex always
   assigns it from the key.

[pkosiec]

Does it make sense to have an assistant tab? maybe we can just keep the helper one, as it's the only one that comes with tools?
Or, let me ask a different related question: what's the purpose of the "assistant" agent in the template?

[pkosiec]

Let's note that only streaming model serving endpoints are supported. On the list during the apps init we list all of the resources.

Here's the comparison of the endpoint usage in context of the Model Serving plugin:

Template (non-streaming) vs Dev-Playground (streaming)

Hook / Client approach

Aspect	Template	Dev-Playground
Hook	useServingInvoke	useServingStream
Backend endpoint	POST /api/serving/invoke	POST /api/serving/stream
Transport	Regular HTTP POST → JSON response	HTTP POST → SSE (Server-Sent Events)
State	{ invoke, data, loading, error }	{ stream, chunks, streaming, error, reset }

Backend routes (both registered by ServingPlugin)

• /invoke → calls servingConnector.invoke() → uses the SDK's high-level client.servingEndpoints.query() which returns a single QueryEndpointResponse JSON object.
• /stream → calls servingConnector.stream() → uses the SDK's low-level client.apiClient.request({ raw: true }) hitting /serving-endpoints/{name}/invocations with Accept: text/event-stream and { stream: true } in the body. Returns raw SSE bytes piped directly to the client.

Response parsing

• Template extracts choices[0].message.content — a complete response in one shot.
• Dev-Playground extracts choices[0].delta.content from each SSE chunk and concatenates them in real-time, committing the full message when streaming ends.

What "doesn't support streaming" means

A model serving endpoint "not supporting streaming" means:

1. The underlying model doesn't implement the SSE/chunked response protocol. When you send { stream: true } in the request body, the endpoint either ignores it and returns a full JSON response, or returns an error.
2. Practically: The /serving-endpoints/{name}/invocations endpoint with Accept: text/event-stream won't produce an incremental byte stream — the response.contents ReadableStream will be null or the call will fail (see client.ts:103: throw new Error("Response body is null — streaming not supported")).
3. Which endpoints support it: Foundation Model APIs (e.g. DBRX, Llama, external models like GPT) generally support streaming. Custom model endpoints (e.g. sklearn, MLflow pyfunc models) typically do not — they return a single JSON response.

So the template defaults to useServingInvoke (non-streaming) as the safe, universally compatible choice, with the streaming hook commented out as an opt-in. The dev-playground uses useServingStream because it's a showcase/reference app assumed to point at a chat-capable LLM endpoint.