Skip to content

[MOCK][DO NOT MERGE] Future work. Desktop Mode Agents. - #240

Open
AllTerrainDeveloper wants to merge 3 commits into
trunkfrom
my-agents
Open

[MOCK][DO NOT MERGE] Future work. Desktop Mode Agents.#240
AllTerrainDeveloper wants to merge 3 commits into
trunkfrom
my-agents

Conversation

@AllTerrainDeveloper

@AllTerrainDeveloper AllTerrainDeveloper commented May 18, 2026

Copy link
Copy Markdown
Collaborator

The pitch

A WordPress site is a tool the user operates. Agents make it a tool that operates with the user: durable, addressable workers that live on the site, take orders by chat, drag, hook, or HTTP, and use the same APIs a human admin would.

This PR ships the navigation surface and visual contract. Everything described below is what we build on top of it, in the order described, behind this UI.

What an Agent is — three layers, by design

We split an Agent across three existing WordPress primitives instead of inventing a new one. This is the architectural decision that lets every other piece compose cleanly.

Layer 1 — Identity: a WordPress user

Each Agent is a real row in wp_users with three constraints:

  1. A role (administrator, editor, author, contributor). Capabilities are real WP capabilities; an Agent that runs wp_insert_post() is gated by current_user_can( 'edit_posts' ) exactly like a human.
  2. No login. The authenticate filter rejects them, password resets are disabled, cookie auth refuses them. They exist only as actors the site invokes on its own behalf.
  3. An identity surface (avatar, display name, attribution on revisions, comments, audit logs). When an Agent edits a post, the post's _edit_lock and revision author show the Agent — just like a human collaborator.

Layer 2 — Behavior: the wp_guideline CPT (portable) @artpi

The Agent's wp_guideline post IS its behavior. Every field that shapes how the Agent thinks, every toggle that changes what it can do, every list of supplemental knowledge it can reach for — all of it reads from and writes to a single wp_guideline post. There is no parallel Desktop-Mode table for prompts, no separate options row for the tool list, no shadow registry for skills. One post per Agent, and that post is the agent definition.

This is the CPT the broader Automattic agent ecosystem (Dolly, Push MD, the in-tree Guidelines experiment) already uses, which is why this works.

Concretely, every behavior-shaping change in the Desktop Mode UI lands as a write to this one post:

What the user does in the UI What happens in storage
Edits the system prompt wp_update_post() on the guideline's post_content
Ticks a checkbox to enable an ability (wordpress/list-posts) add_post_meta( $guideline_id, '_agent_abilities', 'wordpress/list-posts' )
Unticks a checkbox to disable an ability delete_post_meta( $guideline_id, '_agent_abilities', 'wordpress/list-posts' )
Attaches a skill (writing/headline-style) Link the child wp_guideline to the parent — Dolly's existing relationship model, no new schema
Detaches a skill Remove that link
Edits a skill's body wp_update_post() on the child guideline (the skill is its own post; the agent just references it)

Nothing about the agent's brain lives outside wp_guideline. Pull up the agent's guideline post in any tool that speaks WP REST (Gutenberg, wp-cli, Push MD, an external script) and you have the entire behavior surface — prompt, tool toggles, attached skills — editable in one place, revisable through the standard editorial UI, auditable via the standard revisions table.

And because skills are themselves wp_guideline posts, this composes recursively: a skill can be used by many agents; an agent can mix-and-match skills from many authors; a single skill update propagates to every agent referencing it. Same pattern Dolly ships.

This layer is fully portable. Nothing in any of these fields is Desktop-Mode-specific — every value is something Claude Code, Codex, Cursor, or any other agent runtime understands natively. pushmd pull the site and the agent's brain materialises into the consumer's skills/ folder verbatim, tool toggles and all.

Layer 3 — Bindings: user meta on the Agent (site-specific)

Everything about how this site invokes the Agent lives as user meta on the Agent's wp_users row. The fields here are intentionally outside wp_guideline because they would be meaningless to consumers that aren't Desktop Mode:

  • Triggers: which WP hook the Agent subscribes to (save_post, wp_insert_comment, …), which REST endpoint it exposes and under what auth, which drop payloads its tile accepts, which agents it chains to.
  • Runtime overrides: per-Agent model selection that beats the platform default; per-Agent rate limits; per-Agent debug flags.

This is the layer Claude Code in a terminal doesn't have an opinion about — it invokes agents directly, no hook subscription, no REST gateway. Keeping bindings out of the guideline means the guideline travels, the bindings don't: clone the site, get the agent's brain; configure how your site invokes it separately. Same agent definition, different invocation policy per environment.

The split, in one line

wp_users row = who. wp_guideline post = what it does (prompt + tools + skills, all of it). user meta = how this site reaches it.

Updates to bindings don't touch behavior; updates to behavior don't touch identity; rename the identity and the other two don't move. And critically: if a user clicks anything in the agent's Define / Tools / Skills UI, the only thing that changes on disk is one wp_guideline post.

Why this split matters: free ecosystem compatibility

Because behavior lives in wp_guideline, every Agent we ship is automatically discoverable by any AI client that already speaks this CPT — including Claude Code, Codex, and anything in the Automattic agent ecosystem. The mechanism (already in production via pushmd.blog): your WordPress site becomes a Git remote, every guideline materialises as wp_guideline/skills/{slug}/SKILL.md with an AGENTS.md alias, and a local git clone of the site drops a working skills/ folder into the consumer's checkout. Claude Code reads it. Codex reads it. Cursor reads it. No bespoke integration, no separate sync layer, no second source of truth.

The agent you build inside Desktop Mode shows up in your terminal the moment you pushmd pull. The agent your developer hand-writes as a .md file in their checkout shows up in Desktop Mode the moment they git push. Same artifact, two front-ends.

How you talk to an Agent — five triggers, one mental model

Every interaction with an Agent is a trigger the user configures up front. The five triggers we plan to ship:

Trigger Looks like Use case
Drag & drop Drop a media tile (or post, user, comment) onto the Agent's tile "Remove the background from these 12 images."
Chat Double-click the Agent → conversation window "Audit this post and tighten the headings."
Hook Subscribe to a WP action (save_post, wp_insert_comment, …) "Moderate every new comment automatically."
REST endpoint Authenticated or anonymous POST /agents/v1/<slug> "Trigger a newsletter send from our build pipeline."
Agent-to-agent One Agent's output feeds another's input "When SEO scoring finishes, hand the verdict to the publishing Agent."

All five collapse to the same loop: a message arrives → the Agent's system prompt + the message become an LLM call → the model picks tools off the allowlist → tools run as the Agent's user → the result is the trigger's return value. Drag-and-drop is a chat with a media payload. A hook subscription is a chat where the message is the hook args. An endpoint is a chat where the body is the message. Same engine, different intake.

Trigger configuration is user meta on the Agent (Layer 3), not part of the guideline. Two reasons: (1) triggers are site-specific — the same "Moderate Comments" guideline can sit on one site that listens to wp_insert_comment and another that only exposes the REST endpoint, with no fork of the underlying agent definition; (2) triggers are a Desktop-Mode concept that wouldn't round-trip cleanly through pushmd / Claude Code / Codex anyway, so they don't belong in the layer those tools consume.

Tools = the WordPress Abilities API

WordPress 6.9 introduced wp_register_ability() — Core's first-party way to expose typed, schema-described actions to AI tooling. Agents read that registry at runtime: every ability becomes a candidate tool with its declared parameters schema converted to the OpenAI / Anthropic / Gemini function-calling shape. The user picks which ones each Agent gets, and the picks live as post meta on the guideline.

Three properties that fall out of this:

  • Ecosystem leverage. Any plugin that registers an ability becomes Agent-compatible. WooCommerce abilities → commerce Agents. Yoast abilities → SEO Agents. No bespoke integration code per plugin.
  • Capability gating for free. Each ability already carries its own permission_callback. The Agent runs as itself (its WP user), so the same checks that protect a human editor protect the Agent.
  • Tool palette UX. The checkbox list is a view over wp_get_abilities(). As Core grows that registry, every Agent's picker grows with it.

For abilities not yet exposed by Core or plugins, the existing desktop_mode_register_ai_tool() registry plugs the gap and feeds the same picker.

This Abilities-to-tools mapping is the piece the broader ecosystem doesn't have yet. It's what turns "an agent that knows things" (skills, instructions) into "an agent that does things" (tool calls against a typed schema).

LLM provider — bring your own

Agents need a model. We already ship a provider registry (desktop_mode_register_ai_provider) that supports OpenAI's Responses API today and is structured for Anthropic, Gemini, and any vendor a plugin author wires up. Two implications for Agents:

  • No LLM key, no Agent creation. The "Create Agent" button greys out with a notice pointing at the OS Settings AI panel. The section still renders — it's a hint, not a hard gate — and existing Agents still appear so users can audit them before installing a key.
  • Per-Agent model overrides. Some Agents are cheap classification jobs (Moderate Comments → Haiku-tier); some need real reasoning (Audit Post → Sonnet-tier). Model choice is a binding (Layer 3, user meta) rather than part of the portable guideline — what model you pay for on this site has no business travelling with the agent definition.

Drag-and-drop is the North Star

This is the feature that proves the desktop metaphor isn't a skin. "Drag this image onto the Remove BG agent" is a sentence non-technical users say out loud and expect to work. The cross-window drag bridge needed for Media-into-Gutenberg is the same machinery needed for tile-into-Agent — so every Agent we ship pulls the bridge closer to finished.

What it requires on the Agent side: an accepted-payload manifest in the drag-trigger binding (MIME types, entity kinds, post types) and a drop handler that converts the dropped payload into the chat message format. The framework already knows how to draw the ghost, hit-test windows, and route the drop — Agents are just one more drop target type.

Security model — boring on purpose

A new actor that can take HTTP requests, listen to hooks, and call tools is a security surface. The boring-on-purpose answer:

  • Agents are users. Every action lands in WordPress's existing audit trail with the Agent's user ID as the actor. No new auth model, no parallel ACL.
  • Capabilities are inherited, not granted. Selecting a tool from the abilities picker does not elevate the Agent. If the Agent's role can't run the underlying ability, the call fails the same way a human's would.
  • Trigger gating is explicit. Each trigger carries its own auth (capability for chat, REST permission for endpoints, hook firing user for actions). The Agent never runs un-gated.
  • No outbound creds in payloads. Tool results are sanitised before going back into the LLM context. The model sees what the editor would see in the wp-admin UI, not what wp-cli would see in the database.
  • Behavior is auditable. Because instructions live as wp_guideline posts, every prompt change is a real revision in wp_postmeta, attributable to a real user, reviewable in the standard editorial UI.

Why ship the UX mock first

This PR is intentionally a navigation point and a visual contract, nothing else. Four hard-coded Agents, a read-only Define / Tools / Triggers right pane, a "+ Create agent" button that says Coming soon. No data model writes, no login block, no real LLM call, no real trigger plumbing.

The reason is concrete: the surface area below is large, and the right argument about which slice to build first is the one that holds up to looking at the actual screen. Shipping the screen unlocks that argument without committing the team to any single backend shape. Now that the storage layer has a clear answer (wp_guideline CPT, ecosystem-compatible from day one), the next slice is obvious — but we still want the screen real before the wiring goes in.

Every architectural choice in this PR is reversible. The files added or touched can be deleted in one commit if the direction changes. What survives is the lesson: this is what it should look like.

The order we'll build it

  1. Behavior layer — adopt wp_guideline. Each Agent gets a guideline post storing prompt + tool allowlist + skill links. This is the load-bearing decision and it goes first because every later layer references it.
  2. Identity layer — synthetic users. Agent role(s) added, authenticate filter rejects synthetic users, password reset disabled, REST cookie auth refuses them, audit-trail attribution works end to end. The user row carries a single piece of meta linking to its guideline.
  3. Abilities bridge. desktop_mode_ai_tools consumer that harvests wp_get_abilities() into LLM-shaped tools, capability-gated, deduped against the existing tool registry, the allowlist meta on the guideline filters down to the per-call tool set.
  4. Push MD compatibility audit. Once 1 + 2 + 3 are in, validate that the Agent guidelines materialise correctly under wp_guideline/skills/{slug}/SKILL.md via pushmd. This is the moment Desktop Mode Agents become natively discoverable by Claude Code, Codex, and the rest of the ecosystem. We don't need to ship pushmd — we just need to not break the shape it expects.
  5. Bindings layer — user meta scaffold. Trigger configuration, model override, rate limits land as structured user meta on the Agent's wp_users row. Nothing wired yet — just the shape, so steps 6–10 can drop in without rework.
  6. Chat trigger. Multi-instance native window per Agent, conversation history per ( user × agent ), streaming via the existing AI Copilot endpoint, tool dispatch on the client.
  7. Hook trigger. Subscription registry derived from the bindings meta, hook → Agent invocation, optional filter-return propagation.
  8. REST endpoint trigger. Per-Agent route registration with auth choice (capability / nonce / anonymous), request body becomes the chat message.
  9. Drag trigger. Agent tiles become drop targets; the cross-window drag bridge already in flight (the North Star) carries the payload.
  10. Agent-to-agent trigger. desktop_mode_agent_completed action, chained invocations, loop detection.
  11. Marketplace. Packaged Agent definitions third parties ship as plugins (or as Push MD guideline collections), same way block patterns work today.

Each step ships behind feature flags so plugin authors can test against trunk without us cutting a stable release until the contract settles.

Where this leads

The endgame is a WordPress site where the workflow looks like this:

The author finishes a draft, drags the post onto the Audit Agent. The Agent reviews structure, drops a note in the Reviewer queue, hands the draft to the Optimize SEO Agent on success, which proposes edits, asks for one human OK, then hands the draft to the Schedule Agent, which picks a publication time based on traffic analytics and queues the Send to mail list Agent for the moment of publication.

None of those steps need a custom plugin. They're four Agents the user composed by ticking abilities and wiring triggers, using one screen — the one this PR introduces. And because the guidelines live in wp_guideline, the same author can pushmd pull from their laptop and edit the Optimize SEO Agent's system prompt in a code editor, then push it back. Or have Claude Code in their terminal call the same Audit Agent through its REST endpoint while writing a different post. One source of truth, every surface.

What this PR is not

  • Not a feature flag for production users. The Agents section appears in every My WordPress window. That's deliberate — we want to learn from the question "what is this?" before we ship the working version.
  • Not a public API. No docs/hooks-reference.md entries, no docs/examples/agents.md. The contract is the screen. We add the API surface when the backend lands.
  • Not the final design. Define / Tools / Triggers is a starting point, not a finished interaction. Expect iteration on the trigger configuration UI, the abilities picker density, the chat affordance, and the empty state for "no LLM configured."
image image image Open WordPress Playground Preview

@github-actions

github-actions Bot commented May 18, 2026

Copy link
Copy Markdown
Contributor

✅ WordPress Plugin Check Report

✅ Status: Passed

📊 Report

All checks passed! No errors or warnings found.


🤖 Generated by WordPress Plugin Check Action • Learn more about Plugin Check

- Introduced a new Agents entity in the My WordPress section, including an inline SVG icon for visual consistency.
- Implemented a mock renderer for the Agents section, allowing for a UI preview without backend integration.
- Created mock data for four fictional agents, each with defined abilities and triggers.
- Added tests to ensure the integrity of the mock data and the rendering functionality.
- Updated entity registration to include the new Agents kind and adjusted related types accordingly.
- Implement unit tests for the Agents REST API in `agentsRest.php`, covering endpoints for listing, creating, updating, and deleting agents, as well as permission checks.
- Create integration tests for the Agents renderer in `agents-renderer.test.ts`, ensuring proper rendering of agent data and UI interactions.
- Add tests for the Agents REST adapter in `agents-rest.test.ts`, validating fetch requests and response handling.
- Introduce tests for the Agents send-to functionality in `agents-send-to.test.ts`, verifying caching, menu interactions, and event dispatching.
juanlentino added a commit to juanlentino/signal-and-noise-tools that referenced this pull request May 24, 2026
Reading WordPress/desktop-mode PR #240 (Future work. Desktop Mode Agents.)
revealed two architectural facts that retire v3.8.0:

1. The Anthropic provider is GENERIC infrastructure — it contains zero
   Signal & Noise content. It belongs in desktop-mode itself, not in
   our plugin. PR #240's §"LLM provider — bring your own" explicitly
   names Anthropic as the kind of provider "a plugin author wires up,"
   but the better path is upstream contribution since the work has no
   SN-specific surface area.

2. The 26 manual `desktop_mode_register_ai_tool()` registrations we
   planned for Tasks 5-7 will be obsoleted by step 3 of PR #240's
   Agents framework, which auto-harvests `wp_register_ability()`
   registrations into LLM-shaped tools. Our 12 theme abilities (theme
   v9.1.1) + 17 plugin abilities (plugin v3.7.3) are already
   future-compatible — no plugin-side work needed for them to surface
   in the Agents framework when it lands.

What this commit changes:
  - Deletes inc/ai-copilot/ (the 3 anthropic-* files + .gitkeep
    scaffold from Tasks 1-4, originally committed in 6425ab9,
    d3d89cc, 92e39cc, a1275b2)
  - Deletes tests/anthropic-provider.php (71 assertions of provider
    coverage, ported to the upstream PR's PHPUnit tests)
  - Removes the conditional require_once block from
    signal-and-noise-tools.php — back to its pre-Task-1 state
  - Annotates the v3.8.0 spec + plan with CANCELLED headers pointing
    to the upstream contribution path

What stays:
  - Theme v9.1.1 + plugin v3.7.3 production state — unchanged
  - The 12 launcher commands in inc/desktop-mode-integration.php from
    commit b3430cc (display-only ⌘K entries, harmless)
  - All 9 legacy test suites still pass — 550 assertions across
    admin-tabs, ai-bootstrap, bot-detection, cron-dashboard,
    cron-history, health-checks, insights, theme-ability-commands,
    webhooks

Upstream contribution work continues in the fork at
juanlentino/desktop-mode (cloned to ../desktop-mode/). The provider
code itself is preserved in git history at the SHAs listed above and
will be ported to the PR with desktop-mode's `desktop_mode_ai_*`
function-prefix conventions modeled on includes/ai-copilot/openai.php.

Reference: WordPress/openstation#240
           WordPress/openstation#271
epeicher added a commit that referenced this pull request Jul 29, 2026
The PR #240 North Star, wired through the existing drag machinery. A
drop is a chat whose message carries the dropped entity: the shared
dispatch engine (src/agents-dispatch.ts) normalizes the drag payload
('shortcut' from My WordPress tiles / wpd-tile drag-out,
'desktop-file' from wallpaper tiles) into { kind, id, title },
composes the invocation message, seeds the cross-bundle chat store,
surfaces the Agent chat window, and runs /invoke with source=drag so
the conversation shows the run live.

Three intake surfaces:
- Agent rows in the My WordPress Agents section (drop targets
  re-registered per paint, pruned per agent, torn down with the
  mount).
- Agent user tiles on the wallpaper, opted in through the files
  layer's tile-payload-handler seam. Gating is payload-driven: the
  user-file payload now inlines isAgent + agentDragKinds (the drag
  trigger's entityKinds; null = no drag trigger, [] = all kinds), so
  accept() stays synchronous with no REST roundtrip.
- The open Agent chat window, which accepts drops for the active
  agent without trigger gating — dropping into an open conversation
  is explicit intent, like typing.

The invoke route gains a source param (chat|drag|send-to) that lands
in the completed action's context for audit and future chaining.
Self-drops (an agent's own user tile) are always rejected.
epeicher added a commit that referenced this pull request Jul 31, 2026
…t with persisted conversations, drag & drop and Send to triggers (#428)

* feat: AI Agents framework, Phase A (opt-in)

Agents are durable workers that live on the site as real WordPress
users and act through the Abilities API under their own role. An agent
is a login-blocked wp_users row plus a _desktop_mode_agent_* user-meta
family holding the whole definition: description, instructions
(system prompt), ability allowlist, triggers, model override, rate
limit. No wp_guideline dependency; the audit trail for definition
changes is the desktop_mode_agent_{created,updated,deleted} actions,
each carrying before/after values.

Behind the new 'agents' extended option (default off, games-style
module gating). Phase A ships:

- includes/agents/: store (meta CRUD + orchestrators), identity
  (login/password-reset/app-password blocks, bot avatar, Users list
  column), abilities bridge (desktop-mode/get-post + update-post,
  picker catalogue with readonly badges), runner on the Core AI
  Client (8-turn cap, per-agent hourly rate limit, identity switch in
  try/finally, provider-safe tool-schema normalization shared with
  the Copilot), REST CRUD + /invoke + catalogues, privacy
  exporter/eraser, Agent chat native window, My WordPress entity.
- Client: 'agent' entity-kind renderer in the My WordPress bundle
  (list, Define/Tools/Triggers panes, create flow, live AI-provider
  probe) and a new lazy agent-run-window bundle fed through the
  cross-bundle desktop-mode/agents-chat shared store.
- 45 PHPUnit + 20 vitest tests; docs (hooks-reference,
  javascript-reference, api-index, architecture, rest README,
  examples/agents.md) and the implementation plan under docs/plans/.

Chat is the only wired trigger; send-to/drag, hook, endpoint, and
agent-to-agent intakes are declared in the trigger-kind catalogue and
land in later phases.

* fix: provider-safe tool schemas + transcript replay in the agents runner

Two Gemini-surfaced fixes in the agent invocation path:

- Strip the WordPress-only arg-schema keys (sanitize_callback,
  validate_callback, arg_options) from tool schemas at every depth, in
  the shared desktop_mode_ai_normalize_tool_schema() so the Copilot
  benefits too. Strict providers reject any unknown field and 400 the
  whole request over one property. The walk is structure-aware:
  property NAMES are never stripped, only schema-level keys; covers
  properties, patternProperties, single and tuple items, array-shaped
  additionalProperties, and nested combinators.

- Stop replaying assistant functionCall turns to the provider. Each
  generate turn now sends one user message: the original request plus
  a text transcript of executed tool calls and their results.
  Replaying functionCall parts requires provider-specific signatures
  (Gemini thought_signature, Anthropic thinking signatures) that the
  current provider plugins do not round-trip, and one missing
  signature 400s the request. Text transcripts carry the same
  information with no signature or pairing constraints on any
  provider.

* feat: desktop-mode/get-media ability (read media details)

Agents (and the Copilot, via the readonly annotation) can now read a
media library item by id: file URL, mime type, dimensions, alt text,
caption, and the post it is attached to. Closes the gap where no
ability on a stock site could read attachments, so image-referencing
prompts dead-ended.

Permission gates on upload_files (author+), deliberately not on
read_post: for inherit-status attachments that check defers to the
parent (and effectively requires edit rights when unattached), which
wrongly blocks read-only access to media whose file URL is public on
a standard site anyway.

* feat: Remove Background extension (media-tools/remove-background ability)

New standalone extension plugin under extensions/, riding the agents
PR as the first real consumer of the framework: an ability that
removes the background from a media library image and sideloads the
result as a NEW png attachment (original untouched), authored by the
calling user — for agents, the agent's own account, completing the
attribution story end to end.

Mutating ability (no readonly annotation): invisible to the Copilot,
reachable only through an agent's explicit allowlist. Pluggable
backends behind desktop_mode_remove_background_backends: remove.bg
(default, key required), a self-hosted rembg server, and an
experimental WordPress AI Client generative-editing backend that
reuses the site's configured connectors. Settings live natively on
Settings -> Media. A desktop_mode_remove_background_pre short-circuit
filter keeps PHPUnit network-free; seven tests cover registration,
the execute lifecycle, permissions, and runner dispatch with
agent attribution.

* refactor: drop the Remove Background settings UI

The extension's only user-facing surface is the ability itself — no
Settings -> Media section. Configuration is code/CLI-level, resolved
option -> constants (DESKTOP_MODE_REMOVE_BG_{BACKEND,API_KEY,ENDPOINT})
-> desktop_mode_remove_background_settings filter, with an unknown
backend slug falling back to the default. README documents all three
paths; error messages point at them instead of the removed screen.

* feat: default Remove Background to the AI Client backend

The ai backend rides the site's existing Connectors credentials, so a
stock install needs no extension-specific key at all — remove.bg and
rembg become the opt-in paths for mask-based quality. The AI Client
resolves the model from the prompt's modalities (an input image
requires an image-input-capable model), so providers that only do
text-to-image are never silently picked for an edit.

* fix: talk to the WP prompt-builder wrapper in the AI backend

wp_ai_client_prompt() returns WordPress's snake_case wrapper
(WP_AI_Client_Prompt_Builder), not the SDK PromptBuilder: generating
methods are generate_* and failures come back as WP_Error instead of
exceptions. The camelCase generateImageResult() call fell through the
wrapper's fluent path and returned the builder itself, which the
backend then reported as 'no readable image data' regardless of what
the provider said. Now calls generate_image_result(), propagates
WP_Error verbatim (so provider errors like quota exhaustion surface
readably in the agent's tool trace), and keeps the try/catch for
result-shape surprises. Regression test drives the real builder path
with no connector configured.

* fix: agent avatar as a real file URL + list-route collection headers

Three related symptoms, one root cause for two of them: the agent
avatar shipped as a data: URI, and 'data' is not in
wp_allowed_protocols(), so every consumer that runs avatars through
esc_url() (wp-admin's get_avatar(), the desktop user-tile icon, the
My WordPress entity tile) stripped it to an empty string — broken
avatar in the Users screen, icon-less desktop tile when dragging an
agent user to the wallpaper, and the letter-badge fallback on the
Agents folder tile. The bot avatar now ships as a static SVG
(assets/images/agent-avatar.svg, light disc + dark glyph so it reads
on light and dark surfaces) and every PHP surface uses its URL.

The Agents folder also showed 'Agents · 0': the root grid derives
folder counts from the X-WP-Total collection header, which the agents
list route never sent. It now emits X-WP-Total / X-WP-TotalPages.

* feat: drag & drop trigger — drop entities onto agents

The PR #240 North Star, wired through the existing drag machinery. A
drop is a chat whose message carries the dropped entity: the shared
dispatch engine (src/agents-dispatch.ts) normalizes the drag payload
('shortcut' from My WordPress tiles / wpd-tile drag-out,
'desktop-file' from wallpaper tiles) into { kind, id, title },
composes the invocation message, seeds the cross-bundle chat store,
surfaces the Agent chat window, and runs /invoke with source=drag so
the conversation shows the run live.

Three intake surfaces:
- Agent rows in the My WordPress Agents section (drop targets
  re-registered per paint, pruned per agent, torn down with the
  mount).
- Agent user tiles on the wallpaper, opted in through the files
  layer's tile-payload-handler seam. Gating is payload-driven: the
  user-file payload now inlines isAgent + agentDragKinds (the drag
  trigger's entityKinds; null = no drag trigger, [] = all kinds), so
  accept() stays synchronous with no REST roundtrip.
- The open Agent chat window, which accepts drops for the active
  agent without trigger gating — dropping into an open conversation
  is explicit intent, like typing.

The invoke route gains a source param (chat|drag|send-to) that lands
in the completed action's context for audit and future chaining.
Self-drops (an agent's own user tile) are always rejected.

* fix: agent tile drop used the files REST base; URL entity icons

Two live-verified fixes from browser-testing the drag intake on a
real site:

- The wallpaper tile drop handler built its invoke URL from the files
  layer's injected baseUrl, which already ends in
  desktop-mode/v1/files — the request went to
  .../files/desktop-mode/v1/agents/{id}/invoke and 404ed
  (rest_no_route). The handler now reads the shell config's restUrl
  (rest_url()) directly; a regression test pins that the URL never
  contains /files/.

- My WordPress root entity tiles ran every icon through the class
  sanitizer, so a URL icon (the Agents entity's bot SVG) was mangled
  into an invalid dashicon class and fell back to the letter badge.
  URL- and data-URI-shaped icons now pass through untouched, per
  wpd-tile's documented contract.

Verified end to end in the browser: dragging a media tile from
My WordPress onto the agent's desktop tile shows the Send-to-agent
chip, opens the chat window, and completes the invocation.

* fix(ui-core): dispose nested parts when a slot switches templates

Deleting the last agent left the dead agent's tabs + Define form
painted above the fresh 'No agents yet' empty state. The renderer
bug is general, not agents-specific: disposing a template instance
removed only the nodes cloned at mount (state.nodes), but a child
part whose anchor sits at the instance's TOP level inserts its
content as SIBLINGS of those nodes — so everything such slots had
rendered leaked whenever an outer slot switched to a different
template. The agents detail pane (head + top-level tab/pane slots
swapping against the empty state) was the first in-tree shape to
trip it.

disposeChildState now recursively disposes the instance's own child
parts before removing the cloned nodes. Regression test covers the
switch in both directions; verified live in the browser on the
reported delete flow.

* fix: replay conversation history into agent invocations

Every chat message was a stateless run: the client posted only the
current message, so a follow-up like "Yes, please" reached the model
with no idea what had been proposed. Reported symptom, and it is a
data-integrity bug, not a cosmetic one: an agent proposed a TL;DR for
post 973, the user approved, and the agent — starting from nothing —
searched, picked an unrelated post, and wrote to 614 instead.

The invoke route now accepts a  array of prior
{ role: 'user'|'agent', text } turns, capped at the 20 most recent ×
4000 chars, and the runner folds them into the composed prompt ahead
of the new message with an explicit instruction to resolve references
against the conversation rather than a fresh search. Client side, the
chat window's typed path now delegates to the same
invokeAgentIntoTranscript() the drop path uses, which snapshots the
transcript (skipping pending and error rows) before appending the new
message — so both intakes replay identically and neither can
regress independently.

Verified live: turn two of the reported flow now carries the proposed
post id into the prompt.

* feat: update-media and create-post abilities

Two mutating abilities that complete the demo-facing toolset:

- desktop-mode/update-media: alt text / title / caption / description
  on an attachment, gated on the same edit capability wp-admin
  requires. The file itself is never touched. Unlocks accessibility
  agents (write alt text at drag-and-drop speed).
- desktop-mode/create-post: creates a NEW post or page with the
  status hard-forced to draft — it can never publish, whatever the
  model asks for. Authored by the calling user, so agent-created
  drafts carry agent attribution. Page creation additionally gates on
  edit_pages. Unlocks translation/derivative agents that produce
  reviewable drafts without touching any existing content.

Both are unannotated (mutating): invisible to the Copilot, reachable
only through an agent's explicit allowlist. Six new tests cover
registration annotations, the write paths, draft forcing, authorship,
and the capability denials.

* feat: agent user tiles open the Agent chat instead of the profile

Double-clicking an agent's tile on the desktop now starts a
conversation with the agent; human user tiles keep opening the
profile window. Built on a new per-FILE seam in the opener registry:
FileOpenerDef gains an optional appliesTo(file) predicate, honoured
by resolveOpener/getOpenersForType when a file is passed (the open
dispatcher now passes it). Predicate-bearing openers are excluded
from type-level listings where no file exists to test — the
default-apps settings tab never shows them. registerOpener's
normalization also had to learn the field; it silently dropped
unknown keys, which the new tests would have missed without the
failing-first run.

The built-in agent-chat opener (isDefault, sort 5, appliesTo
shape.isAgent) seeds the cross-bundle chat store via the shared
openAgentChatWindow() helper — extracted from the drop dispatch so
tile-open, drop, and chat all surface the window identically. The
user-file payload now inlines agentDescription so the chat header
shows the agent's when-to-use line without a REST roundtrip.

Verified live: the registry resolves agent-chat for agent files and
wp-user-profile for humans on the running site.

* feat: agents UX batch: Send to menu, chat avatars and markdown, profile and desktop shortcuts

- Remove the remove-background extension, its abilities, tests, and
  every reference; the Photo Studio demo agent is gone with it.
- Wire the send-to trigger: agents with a send-to trigger appear as
  'Send to <agent>' entries in the site folder tile context menus
  (posts, pages, media, users), gated by the trigger's entityKinds.
  The users grid menu now runs the same tile-context-menu filter seam
  as posts and media. Registration is idempotent on the hooks bus
  because the bundle IIFE can execute twice (boot enqueue plus the
  native-window lazy loader).
- Agent chat: WhatsApp-style avatars on both sides of the
  conversation, a much wider in-flight bubble, a New chat button that
  clears the transcript, and markdown answers rendered via a new
  shared src/markdown.ts (extracted from the AI assistant, now with
  headings, thematic breaks, and per-line inline tokens so stray
  asterisks cannot pair across lines).
- Agents window: Open profile button (opens the user-edit window for
  the agent) and Send to Desktop button (creates a wallpaper tile on
  the first free grid cell; count-based slotting collided with moved
  tiles and buried existing icons).
- Trigger kinds catalogue now carries a wired flag; the Triggers pane
  renders unwired kinds (hook, endpoint, agent-to-agent) as disabled
  'coming soon' options.

* feat: center the chat loading bubble, drag agents from the list to the desktop

- The in-flight chat bubble centers its label and spinner (the bubble
  is a flex column, so text-align alone left-aligned the spinner).
- Agent rows in the Agents section are draggable out as 'user'
  shortcuts via the shared attachTileDragOut helper: dropping a row on
  the wallpaper creates the same agent tile the Users grid drag
  produces. Attach is guarded per element so a repaint cannot stack a
  second pointerdown listener.

* feat: persist agent chat conversations with a sidebar, raise history replay cap to 50

- New desktop_mode_chat private post type: one post per conversation,
  post_author = the human, messages as JSON in post_content, agent id
  in meta. Strictly owner-only REST CRUD under
  /desktop-mode/v1/agents/conversations (list stays light, message
  bodies fetched per conversation); foreign rows read as 404 so ids
  cannot be probed. Caps: 100 conversations per user (filterable via
  desktop_mode_agent_conversation_cap, prune orders by modified with
  an ID tie-break so same-second creates cannot self-prune), 200
  messages per conversation, tool-call outputs dropped on store.
- Chat window gains a left sidebar: + New chat, past conversations
  (agent avatar + derived title, active highlight, hover delete with
  confirm), clicking a row reloads its transcript and re-targets the
  chat to that agent. The window default width grows to 760px.
- Auto-save after every completed exchange in the shared dispatcher
  (chat, drag, and send-to all persist); failures are swallowed so
  persistence can never break the conversation itself.
- History replay cap raised from 20 to 50 turns, filterable via
  desktop_mode_agent_history_turn_cap; both filters documented in
  docs/hooks-reference.md.

* style: halve the chat loading bubble (176px line, 24px spinner)

* feat: call-to-action buttons for agent confirmations

Agents that need the user's confirmation now return renderable
buttons instead of asking for a typed reply.

- The runner constrains every final answer to a { text,
  call_to_actions } JSON schema via the AI Client's structured output
  (as_json_response), and a system-prompt appendix teaches the
  convention so existing agents pick it up without prompt edits. Each
  action carries id, label, style (primary/secondary/danger), and a
  reply: the literal message sent back as the user's next turn when
  its button is pressed.
- Parsing is lenient: answers that are not the JSON shape (pre-filter
  runtimes, providers that ignore the schema) pass through verbatim
  with no actions, so structured answers only ever degrade to the old
  behavior. Fenced JSON is tolerated. Sanitization caps 4 actions,
  40-char labels, 500-char replies, enforces the style enum.
- The chat window renders the actions as wpd-buttons under the
  agent's bubble. Only the latest message's buttons are live; pressing
  one posts its reply as a visible user message (the stored history
  shows exactly what was approved) and marks the message ctaUsed so
  reopened conversations render them disabled. Buttons persist with
  the conversation.
- Invoke results and the completed action now carry callToActions;
  conversation storage round-trips callToActions + ctaUsed.

* feat: seed five default agents; large section loading spinner

- New includes/agents/defaults.php ships a complete default roster:
  tl;dr, Comment Concierge, Localizer (author role), SEO Medic, and
  Alt Text Librarian — full system prompts, ability allowlists, and
  chat + send-to + drag triggers. Seeded once per site, and ONLY when
  the site has no agents at all; an install that already built its own
  roster gets the seeded flag without any rows. The hook wrapper runs
  on admin_init gated on edit_users so the seeder stays out of
  front-end requests, cron, and the PHPUnit bootstrap. Abilities that
  are not registered on the site (the ai/* family) are skipped by the
  runner at tool-build time, so allowlisting them costs nothing.
- The Agents section loading state uses the standard large loading
  logo (the clamp(96px, 14vw, 192px) scale curve shared with the
  preview loader) instead of the bare 48px spinner default.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant