fix(runtime): an anonymous /mcp caller is answered 401 ahead of the switch's 404 and the service's 501

.changeset/24251-mcp-anonymous-first.md

@@ -0,0 +1,13 @@
+---
+'@objectstack/runtime': patch
+---
+
+fix(runtime): an anonymous caller of `/api/v1/mcp` is answered `401 UNAUTHENTICATED` whatever `OS_MCP_SERVER_ENABLED` or the `mcp` service says
+
+Clause-②: no
+
+ADR-0138 D2 takes the anonymous decision once per domain, as the domain's first statement. The `/mcp` domain checked the `OS_MCP_SERVER_ENABLED` switch and the `mcp` service before it, so an anonymous caller was answered `404 RESOURCE_NOT_FOUND` with the switch off and `501 NOT_IMPLEMENTED` with no MCP service installed, and learned whether the deployment runs MCP before it was refused. Measured on both hosts, the standalone server (`HonoServerPlugin` with the dispatcher plugin) and a `createHonoApp` host.
+
+- An anonymous request on `/api/v1/mcp` (any verb) is now refused `401 UNAUTHENTICATED` first, in every composition. Only the order of the refusals moves: no request that was refused before is accepted.
+- A signed-in caller, or a system principal, keeps the switch's `404` and the service's `501`, in that order.
+- The `401` advertises RFC 9728 resource metadata (`WWW-Authenticate: Bearer … resource_metadata="…"`) only where the resource is served: the switch on, an `mcp` service that can take the request, and an auth service that answers a metadata URL. With the switch off or no service the `401` is the plain API-key one, so it never points a client at a protected resource the deployment does not serve.

content/docs/ai/connect-mcp.mdx

@@ -12,8 +12,9 @@ about the only thing left to do: **connecting a client and proving it works**.
 
 <Callout type="info">
 To turn the surface off, set `OS_MCP_SERVER_ENABLED=false` — the endpoint then
-returns 404 and the **Connect an Agent** page disappears with it, along with
-both its Setup and Account navigation entries. See
+returns 404 to a caller with an API key or token (an anonymous caller still gets
+401 first, with no OAuth metadata advertised), and the **Connect an Agent** page
+disappears with it, along with both its Setup and Account navigation entries. See
 [environment variables](/docs/deployment/environment-variables#mcp-server).
 </Callout>
 
@@ -380,8 +381,8 @@ skill and a guided `/objectstack:connect` command.
 | **✘ Failed to connect** / `ConnectionRefused` on a server that used to work | The app is not running. Start it — the client reconnects on its own, no need to re-register |
 | The server is missing from `/mcp` in a running session | Sessions read MCP config at start — open a new one. If it is still missing, you registered at **local** scope from another directory; re-add with `-s user` |
 | `Object 'x' not found` on `query_records` / `get_record` | Wrong object name — pass the API name from `list_objects` in **`objectName`** (`crm_opportunity`, not `opportunity`) |
-| `404` on `/api/v1/mcp` | The HTTP surface is disabled — unset `OS_MCP_SERVER_ENABLED` (default is on) |
-| `501 Not Implemented` | The MCP plugin isn't part of this build — check your stack's plugins |
+| `404` on `/api/v1/mcp` | The HTTP surface is disabled — unset `OS_MCP_SERVER_ENABLED` (default is on). A client with an API key or token sees the `404`; an anonymous client, including an OAuth client before it signs in, sees a plain `401` with no `resource_metadata` in `WWW-Authenticate` |
+| `501 Not Implemented` | The MCP plugin isn't part of this build — check your stack's plugins. A client with an API key or token sees the `501`; an anonymous client sees a plain `401` with no `resource_metadata` |
 | `mcp` missing from `GET /api/v1/discovery` and no Connect-an-Agent card, but `OS_MCP_SERVER_ENABLED` is on | The same cause as the `501` above, seen from the other side: the surface is *enabled* but not *serveable*, so discovery declines to advertise a route that would 501 rather than over-promising it (`declared === enforced`). Load the MCP plugin — `os serve` / `os dev` do it for you; a host that embeds `@objectstack/rest` directly must add `@objectstack/mcp` itself |
 | stdio won't start / boot fails closed | `OS_MCP_STDIO_ENABLED=true` but `OS_MCP_STDIO_API_KEY` is missing, unknown, revoked, or expired — fail-closed by design (ADR-0101). Set a valid `osk_` key; there is no unscoped or `system` fallback |
 | `401` on every call | Anonymous or invalid credentials. Interactive clients: complete the browser login (the `WWW-Authenticate` header advertises the OAuth metadata). Headless: check the `osk_` key and header spelling |

content/docs/deployment/environment-variables.mdx

@@ -268,7 +268,7 @@ OS_MCP_STDIO_ENABLED=true OS_MCP_STDIO_API_KEY=osk_... os start   # start the lo
 
 | Variable | Type | Default | Description |
 |:---|:---|:---|:---|
-| `OS_MCP_SERVER_ENABLED` | boolean | `true` | The MCP **HTTP** surface (`/api/v1/mcp`) is a core capability and defaults **on**. Set `false` to disable it (endpoint 404s, the Connect-an-Agent page disappears). |
+| `OS_MCP_SERVER_ENABLED` | boolean | `true` | The MCP **HTTP** surface (`/api/v1/mcp`) is a core capability and defaults **on**. Set `false` to disable it (the endpoint answers 404 to a caller with an API key or token and 401 to an anonymous one, and the Connect-an-Agent page disappears). |
 | `OS_MCP_STDIO_ENABLED` | boolean | `false` | Auto-start the long-lived **stdio** transport at boot. Opt-in and **stricter** than the HTTP surface. **Requires `OS_MCP_STDIO_API_KEY`** — stdio runs as that key's identity with RLS/FLS/tenant applied; if the key is missing or invalid, boot **fails closed** (stdio refuses to start). See ADR-0101. |
 | `OS_MCP_STDIO_API_KEY` | string | — | The `osk_...` API key the **stdio** transport runs as. Resolved through the same verify chain as the HTTP/REST surfaces, so reads are scoped to that identity's permissions. Mint one from the **Connect an Agent** page — **Account → Developer** for any signed-in user, **Setup → Connect an Agent** for platform admins — or `POST /api/v1/keys`. For full authority, mint a key on a platform-admin or dedicated **service** identity — there is deliberately no `system`/unscoped bypass. |
 | `OS_MCP_SERVER_NAME` | string | `objectstack` | Server name advertised to MCP clients. |

packages/qa/http-conformance/src/hono-mcp-anonymous-first.conformance.test.ts

@@ -0,0 +1,189 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#24251] On a `createHonoApp` host, an anonymous caller of `/api/v1/mcp` is
+ * answered `401 UNAUTHENTICATED` whatever the MCP switch or the `mcp` service
+ * says — through the host's own pipeline, and the same as the standalone
+ * server answers it.
+ *
+ * ## The defect
+ *
+ * ADR-0138 D2 takes the anonymous decision "once per domain, as the domain's
+ * first statement". `handleMcpRequest` (`@objectstack/runtime`,
+ * `domains/mcp.ts`) checked `OS_MCP_SERVER_ENABLED` (`404`) and the `mcp`
+ * service (`501`) ahead of it, so an anonymous caller learned whether the
+ * deployment runs MCP before it was refused. Measured on `9822b053e` through
+ * this file's hosted door: anonymous `POST /api/v1/mcp` answered `404
+ * RESOURCE_NOT_FOUND` with the switch off and `501 NOT_IMPLEMENTED` with no
+ * service. The host adds no gate of its own on `/mcp`: its catch-all hands the
+ * request to `dispatch()`, so the domain's order was the host's answer.
+ *
+ * ## What is pinned, per composition
+ *
+ *  1. Hosted (`app.request` into the Hono app `createHonoApp` returns): an
+ *     anonymous `POST` is `401 UNAUTHENTICATED` and never reaches the
+ *     transport; the `401` advertises RFC 9728 resource metadata only where
+ *     the resource is served (switch on, `mcp` installed). The fake `auth`
+ *     slot answers a metadata URL in every composition, as plugin-auth does
+ *     with the switch off when `OS_OIDC_PROVIDER_ENABLED=true`.
+ *  2. Hosted, signed in: the domain's own answer is unchanged — `200` from the
+ *     transport, `501 NOT_IMPLEMENTED`, `404 RESOURCE_NOT_FOUND`.
+ *  3. The hosted door answers the standalone server's status, `code` and
+ *     `WWW-Authenticate`, anonymous and signed in.
+ *
+ * `LiteKernel`, fake services only (no storage): the `auth` slot answers a
+ * session only for a request carrying `x-test-session`. The switch is read per
+ * request, so each row sets it for its own requests.
+ */
+
+import { describe, it, expect, beforeAll, afterAll, afterEach } from 'vitest';
+import { LiteKernel } from '@objectstack/core';
+import type { Plugin, PluginContext } from '@objectstack/core';
+import { HonoServerPlugin } from '@objectstack/plugin-hono-server';
+import { createDispatcherPlugin } from '@objectstack/runtime';
+import { createHonoApp } from '@objectstack/hono';
+import type { IHttpServer } from '@objectstack/spec/contracts';
+
+const PREFIX = '/api/v1';
+const SESSION_HEADER = 'x-test-session';
+const METADATA_URL = 'https://mcp-anonymous-first.test/.well-known/oauth-protected-resource';
+const JSON_RPC = JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'tools/list' });
+
+/** How many requests reached a fake `/mcp` transport, on either door. */
+let transportCalls = 0;
+
+function fakeServicesPlugin(withMcp: boolean): Plugin {
+    return {
+        name: `com.objectstack.test.fake-services-hono-mcp-anonymous-first-${withMcp ? 'mcp' : 'no-mcp'}`,
+        version: '1.0.0',
+        init: async (ctx: PluginContext) => {
+            ctx.registerService('auth', {
+                api: {
+                    getSession: async ({ headers }: { headers?: Headers }) =>
+                        (headers?.get?.(SESSION_HEADER) ? { user: { id: 'u_hono_mcp_anonymous_first' } } : undefined),
+                },
+                getMcpResourceMetadataUrl: () => METADATA_URL,
+            });
+            if (withMcp) {
+                ctx.registerService('mcp', {
+                    handleHttpRequest: async () => {
+                        transportCalls += 1;
+                        return new Response('{}', { status: 200, headers: { 'content-type': 'application/json' } });
+                    },
+                    renderSkill: () => '# skill',
+                });
+            }
+        },
+    };
+}
+
+interface Composition {
+    name: string;
+    withMcp: boolean;
+    switchOff: boolean;
+    /** The anonymous 401 carries `resource_metadata`. */
+    advertises: boolean;
+    /** What a signed-in `POST` is answered: the domain's own answer. */
+    signedIn: { status: number; code?: string };
+}
+
+const COMPOSITIONS: Composition[] = [
+    { name: 'MCP on and installed', withMcp: true, switchOff: false, advertises: true, signedIn: { status: 200 } },
+    { name: 'no mcp service installed', withMcp: false, switchOff: false, advertises: false, signedIn: { status: 501, code: 'NOT_IMPLEMENTED' } },
+    { name: 'OS_MCP_SERVER_ENABLED=false', withMcp: true, switchOff: true, advertises: false, signedIn: { status: 404, code: 'RESOURCE_NOT_FOUND' } },
+];
+
+interface Reading { status: number; code: string | undefined; httpStatus: number | undefined; www: string | null }
+async function read(res: Response): Promise<Reading> {
+    const json = (await res.json()) as { error?: { code?: string; httpStatus?: number } };
+    return { status: res.status, code: json.error?.code, httpStatus: json.error?.httpStatus, www: res.headers.get('www-authenticate') };
+}
+
+describe('[#24251] an anonymous /mcp caller is answered 401 first on a createHonoApp host (conformance)', () => {
+    const kernels: LiteKernel[] = [];
+    const hosted = new Map<boolean, (init: RequestInit) => Promise<Response>>();
+    const standalone = new Map<boolean, (init: RequestInit) => Promise<Response>>();
+    const prevSwitch = process.env.OS_MCP_SERVER_ENABLED;
+
+    beforeAll(async () => {
+        for (const withMcp of [true, false]) {
+            const hostedKernel = new LiteKernel();
+            hostedKernel.use(fakeServicesPlugin(withMcp));
+            await hostedKernel.bootstrap();
+            kernels.push(hostedKernel);
+            const app = createHonoApp({ kernel: hostedKernel as any, prefix: PREFIX, cors: false });
+            hosted.set(withMcp, async (init) => app.request(`http://hosted.test${PREFIX}/mcp`, init));
+
+            const standaloneKernel = new LiteKernel();
+            standaloneKernel.use(fakeServicesPlugin(withMcp));
+            standaloneKernel.use(new HonoServerPlugin({ port: 0, cors: false }));
+            standaloneKernel.use(createDispatcherPlugin({ prefix: PREFIX, securityHeaders: false }));
+            await standaloneKernel.bootstrap();
+            kernels.push(standaloneKernel);
+            const base = `http://127.0.0.1:${standaloneKernel.getService<IHttpServer>('http.server').getPort!()}`;
+            standalone.set(withMcp, (init) => fetch(`${base}${PREFIX}/mcp`, init));
+        }
+    }, 30_000);
+
+    afterAll(async () => {
+        for (const kernel of kernels) {
+            await Promise.race([kernel.shutdown(), new Promise<void>((resolve) => setTimeout(resolve, 10_000))]);
+        }
+    }, 30_000);
+
+    afterEach(() => {
+        if (prevSwitch === undefined) delete process.env.OS_MCP_SERVER_ENABLED;
+        else process.env.OS_MCP_SERVER_ENABLED = prevSwitch;
+    });
+
+    function post(c: Composition, signedIn: boolean): RequestInit {
+        process.env.OS_MCP_SERVER_ENABLED = c.switchOff ? 'false' : 'true';
+        return {
+            method: 'POST',
+            headers: {
+                'content-type': 'application/json',
+                accept: 'application/json, text/event-stream',
+                ...(signedIn ? { [SESSION_HEADER]: '1' } : {}),
+            },
+            body: JSON_RPC,
+        };
+    }
+
+    for (const c of COMPOSITIONS) {
+        it(`${c.name}: hosted anonymous POST /mcp → 401 UNAUTHENTICATED, the transport never reached`, async () => {
+            const before = transportCalls;
+            const r = await read(await hosted.get(c.withMcp)!(post(c, false)));
+
+            expect(r.status).toBe(401);
+            expect(r.code).toBe('UNAUTHENTICATED');
+            expect(r.httpStatus).toBe(401);
+            expect(transportCalls, 'an anonymous caller must never reach the MCP transport').toBe(before);
+            if (c.advertises) {
+                expect(r.www).toContain(`resource_metadata="${METADATA_URL}"`);
+            } else {
+                expect(r.www).toBeNull();
+            }
+        });
+
+        it(`${c.name}: hosted signed-in POST /mcp keeps the domain's own ${c.signedIn.status}${c.signedIn.code ? ` ${c.signedIn.code}` : ''}`, async () => {
+            const res = await hosted.get(c.withMcp)!(post(c, true));
+            expect(res.status).toBe(c.signedIn.status);
+            if (c.signedIn.code) {
+                const r = await read(res);
+                expect(r.code).toBe(c.signedIn.code);
+                expect(r.httpStatus).toBe(c.signedIn.status);
+            }
+        });
+
+        for (const signedIn of [false, true]) {
+            it(`${c.name}: the hosted door answers the standalone server's ${signedIn ? 'signed-in' : 'anonymous'} POST /mcp`, async () => {
+                const viaHost = await hosted.get(c.withMcp)!(post(c, signedIn));
+                const viaStandalone = await standalone.get(c.withMcp)!(post(c, signedIn));
+                const a = signedIn && !c.signedIn.code ? { status: viaHost.status } : await read(viaHost);
+                const b = signedIn && !c.signedIn.code ? { status: viaStandalone.status } : await read(viaStandalone);
+
+                expect(a).toEqual(b);
+            });
+        }
+    }
+});

packages/runtime/src/dispatcher-body-after-anonymous-floor.hono.integration.test.ts

@@ -44,8 +44,8 @@ import { createDispatcherPlugin } from './dispatcher-plugin.js';
  * every answer here comes before a domain touches storage. The `auth` slot
  * answers a session only when the request carries `x-test-session`. A stub
  * `analytics` slot makes the standalone server mount `/analytics`, and a stub
- * `mcp` slot lets `/mcp` reach its anonymous decision (which follows its
- * enabled and available checks).
+ * `mcp` slot lets a signed-in `/mcp` caller past the domain's available check
+ * to the body read (its anonymous decision comes first, #24251).
  */
 
 const SESSION_HEADER = 'x-test-session';