Skip to content

Commit a12cdcf

Browse files
committed
Merge remote-tracking branch 'origin/main' into claude/issue-23050-s6-security-endpoints
2 parents 96504da + f40d877 commit a12cdcf

7 files changed

Lines changed: 379 additions & 12 deletions
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
'@objectstack/objectql': patch
3+
---
4+
5+
fix(objectql): on a kernel with an `environmentId` (`os dev`, `os serve`, `os start`, the cloud single-environment runtime), a runtime-authored object now has a metadata-service copy, so both MCP doors refuse an object whose API exposure is off
6+
7+
Clause-②: no
8+
9+
On a kernel built with an `environmentId`, the `metadata` service is sourced from the artifact and the boot bridge does not run. Until now, the copy follow that keeps a runtime-authored object's copy current also returned early on that shape. So the service held **no** copy of **any** object saved through Studio, the REST metadata door or an AI author, and the MCP exposure gates read that absence as "no declaration". Measured on the real `os dev` (`env_local`), where the REST data door refused the same objects with `OBJECT_API_DISABLED`:
10+
11+
- **Saved with `apiEnabled: false`, or turned off after a save.** The HTTP MCP door served `query_records` and `create_record`, and the create wrote a row. The MCP stdio door did the same.
12+
- **Saved with `apiEnabled: false`, then a restart.** Still served: the hydrated object had no copy, and no save fired after boot.
13+
- **Exposed objects.** `list_objects` did not name them, and `describe_object` answered "not found".
14+
15+
The fix is at the copy's producer, and it bounds what is written the same way the no-`environmentId` shape already does:
16+
17+
- **The copy follow runs on every kernel shape.** A save refreshes the copy before it returns, through `registerInMemory` (in memory, no persistence, no announce). A delete removes it once the stored row is gone and the registry no longer serves the name.
18+
- **A kernel that hydrates its own `sys_metadata` (`hydrateMetadataFromDb`) copies its hydrated runtime-authored objects once at boot.** That option is set only on a kernel whose registry is its own. Only objects whose registry owner row is tenant-authored are copied, so a code package's objects still reach the service from the artifact alone. The full boot bridge stays off on this shape.
19+
- **A boot copy that cannot land is logged at `error`.** The line names the objects, the consequence (the MCP exposure gates serve them even when their exposure is off) and the way out.
20+
21+
Both exposure gates read this service: the MCP stdio door's `enforceApiExposure`, and `refuseUnexposedDataAction`, which the HTTP MCP data verbs run through. Both now answer `OBJECT_API_DISABLED` / 404 for such an object, which is what `apiEnabled: false` declares. Neither gate's fall-open rule for an absent definition changes. The kernel shape without an `environmentId` is unchanged.

‎packages/objectql/src/plugin-runtime-object-metadata-copy.test.ts‎

Lines changed: 130 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@
4646
*/
4747

4848
import { randomUUID } from 'node:crypto';
49-
import { mkdtempSync, rmSync } from 'node:fs';
49+
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
5050
import { tmpdir } from 'node:os';
5151
import { join } from 'node:path';
5252
import { afterEach, describe, expect, it, vi } from 'vitest';
@@ -320,3 +320,132 @@ describe('the metadata service\'s copy of a runtime-authored object follows the
320320
expect((await copyOf(k, OTHER)).copy).toMatchObject({ enable: { apiEnabled: true } });
321321
});
322322
});
323+
324+
/**
325+
* The same copy on a kernel with an `environmentId` that hydrates its own
326+
* `sys_metadata`: `createStandaloneStack`'s composition (`os dev` as
327+
* `env_local`, `os serve`, `os start`) and the cloud single-environment
328+
* runtime. Its `metadata` service is sourced from the artifact and the boot
329+
* bridge does not run, so before the fix a runtime-authored object had NO copy
330+
* there, exposed or not, and both MCP exposure gates served one saved with
331+
* `apiEnabled: false` (measured through the real `os dev`).
332+
*/
333+
describe('on a kernel with an environmentId, a runtime-authored object has a copy that follows the registry', () => {
334+
const ENV = 'env_local';
335+
const live: ObjectKernel[] = [];
336+
const roots: string[] = [];
337+
afterEach(async () => {
338+
while (live.length) {
339+
const kernel = live.pop()!;
340+
if (kernel.getState() === 'running') await kernel.shutdown();
341+
}
342+
while (roots.length) rmSync(roots.pop()!, { recursive: true, force: true });
343+
});
344+
345+
/**
346+
* One boot over `stores`, with `createStandaloneStack`'s two options:
347+
* `MetadataPlugin` reading an (empty) artifact under the environment, and
348+
* `ObjectQLPlugin` with the `environmentId` and `hydrateMetadataFromDb`.
349+
*/
350+
async function boot(stores: Stores, beforeBoot?: (engine: ObjectQL) => void): Promise<Booted> {
351+
const kernel = new ObjectKernel({ logger: { level: 'silent' }, gracefulShutdown: false });
352+
live.push(kernel);
353+
const root = mkdtempSync(join(tmpdir(), 'os-23689-'));
354+
roots.push(root);
355+
const artifact = join(root, 'objectstack.json');
356+
writeFileSync(artifact, JSON.stringify({
357+
manifest: { id: 'com.qa.empty', name: 'empty', version: '0.0.0', type: 'app' },
358+
objects: [], views: [], apps: [], flows: [], requires: [],
359+
}));
360+
const engine = new ObjectQL();
361+
engine.registerDriver(makeMemoryDriver(stores), true);
362+
await kernel.use(new MetadataPlugin({
363+
watch: false, artifactWatch: false, environmentId: ENV,
364+
artifactSource: { mode: 'local-file', path: artifact }, rootDir: root,
365+
}));
366+
await kernel.use(new ObjectQLPlugin({ ql: engine, environmentId: ENV, hydrateMetadataFromDb: true }));
367+
beforeBoot?.(engine);
368+
await kernel.bootstrap();
369+
return { kernel, engine, protocol: kernel.getService('protocol'), metadata: kernel.getService('metadata') };
370+
}
371+
372+
const save = (k: Booted, name: string, label: string, enable?: Record<string, unknown>) =>
373+
k.protocol.saveMetaItem({ type: 'object', name, item: objectBody(name, label, enable) });
374+
375+
async function copyOf(k: Booted, name: string) {
376+
const listed = ((await k.metadata.listObjects()) as Array<{ name?: string }>).find((o) => o?.name === name);
377+
return { copy: await k.metadata.getObject(name), listed };
378+
}
379+
380+
const withShippedPackage = (engine: ObjectQL) => {
381+
engine.registerApp({
382+
id: SHIPPED_PKG, name: SHIPPED_PKG, version: '1.0.0', type: 'app',
383+
objects: [objectBody(SHIPPED, 'Shipped')],
384+
});
385+
};
386+
387+
it('created after boot with exposure off: a copy at once, carrying its declaration', async () => {
388+
const k = await boot(new Map());
389+
expect((await save(k, NAME, 'Gadget', { apiEnabled: false })).success).toBe(true);
390+
391+
const { copy, listed } = await copyOf(k, NAME);
392+
expect(copy).toMatchObject({ label: 'Gadget', enable: { apiEnabled: false }, _provenance: 'org' });
393+
expect(listed).toBeDefined();
394+
});
395+
396+
it('saved exposed, then turned off: the copy says so before the save returns', async () => {
397+
const k = await boot(new Map());
398+
expect((await save(k, NAME, 'Gadget', { apiEnabled: true })).success).toBe(true);
399+
expect((await save(k, NAME, 'Gadget Q', { apiEnabled: false })).success).toBe(true);
400+
401+
expect((await copyOf(k, NAME)).copy)
402+
.toMatchObject({ label: 'Gadget Q', enable: { apiEnabled: false }, _provenance: 'org' });
403+
});
404+
405+
it('saved with exposure off, then a restart: the boot gives the hydrated object its copy (no mutation fires for it)', async () => {
406+
const stores: Stores = new Map();
407+
const k1 = await boot(stores);
408+
expect((await save(k1, NAME, 'Gadget', { apiEnabled: false })).success).toBe(true);
409+
expect((await save(k1, OTHER, 'Other', { apiEnabled: true })).success).toBe(true);
410+
await k1.kernel.shutdown();
411+
412+
const k2 = await boot(stores);
413+
expect(k2.engine.registry.getObject(NAME)).toMatchObject({ enable: { apiEnabled: false } });
414+
expect((await copyOf(k2, NAME)).copy)
415+
.toMatchObject({ label: 'Gadget', enable: { apiEnabled: false }, _provenance: 'org' });
416+
expect((await copyOf(k2, OTHER)).copy).toMatchObject({ enable: { apiEnabled: true } });
417+
});
418+
419+
it('deleted: the copy is gone and the listing no longer names it', async () => {
420+
const stores: Stores = new Map();
421+
const k1 = await boot(stores);
422+
expect((await save(k1, NAME, 'Gadget', { apiEnabled: true })).success).toBe(true);
423+
await k1.kernel.shutdown();
424+
const k = await boot(stores);
425+
expect((await copyOf(k, NAME)).copy).toBeDefined();
426+
427+
expect(await k.protocol.deleteMetaItem({ type: 'object', name: NAME })).toMatchObject({ success: true, reset: true });
428+
429+
await vi.waitFor(async () => {
430+
const { copy, listed } = await copyOf(k, NAME);
431+
expect(copy).toBeUndefined();
432+
expect(listed).toBeUndefined();
433+
});
434+
});
435+
436+
it('a code package\'s object gets no copy from the boot pass or from a runtime edit beside it', async () => {
437+
const stores: Stores = new Map();
438+
const k1 = await boot(stores, withShippedPackage);
439+
expect((await save(k1, NAME, 'Gadget', { apiEnabled: true })).success).toBe(true);
440+
await k1.kernel.shutdown();
441+
442+
const k = await boot(stores, withShippedPackage);
443+
expect(k.engine.registry.getObject(SHIPPED)).toMatchObject({ label: 'Shipped' });
444+
// The code package's objects reach this service from the artifact, never from this pass.
445+
expect(await k.metadata.getObject(SHIPPED)).toBeUndefined();
446+
expect((await save(k, NAME, 'Gadget Q', { apiEnabled: false })).success).toBe(true);
447+
await new Promise((resolve) => setTimeout(resolve, 150));
448+
expect(await k.metadata.getObject(SHIPPED)).toBeUndefined();
449+
expect((await copyOf(k, NAME)).copy).toMatchObject({ enable: { apiEnabled: false } });
450+
});
451+
});

‎packages/objectql/src/plugin.ts‎

Lines changed: 63 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1107,6 +1107,10 @@ export class ObjectQLPlugin implements Plugin {
11071107
// skip it in that case.
11081108
if (this.environmentId === undefined) {
11091109
await this.bridgeObjectsToMetadataService(ctx);
1110+
} else if (this.hydrateMetadataFromDb) {
1111+
// The runtime-authored objects the hydration above just put in this
1112+
// kernel's own registry. See {@link bridgeRuntimeObjectsToMetadataService}.
1113+
await this.bridgeRuntimeObjectsToMetadataService(ctx);
11101114
}
11111115
// Arm the incremental per-manifest bridge for everything after start() —
11121116
// marketplace install / rehydrate register through the `manifest`
@@ -2436,6 +2440,56 @@ export class ObjectQLPlugin implements Plugin {
24362440
}
24372441
}
24382442

2443+
/**
2444+
* The boot half of {@link followRuntimeObjectCopy} on a kernel with an
2445+
* `environmentId` that hydrates its own `sys_metadata`
2446+
* (`hydrateMetadataFromDb`: `os dev`, `os serve`, `os start` and the cloud
2447+
* single-environment runtime): copy each RUNTIME-AUTHORED object the
2448+
* hydration put in the registry into the `metadata` service.
2449+
*
2450+
* The full {@link bridgeObjectsToMetadataService} stays off on this shape
2451+
* (cross-project leakage, and the code packages' objects come from the
2452+
* artifact). But nothing else gives a hydrated runtime object a copy, and no
2453+
* mutation fires for it after boot: an object saved with `apiEnabled: false`
2454+
* and then restarted was served by both exposure gates, which read an absent
2455+
* copy as "no declaration". `hydrateMetadataFromDb` is set only on a kernel
2456+
* whose registry is its own, so every object this pass copies is one this
2457+
* environment authored. Same write as the follow: tenant-authored owner rows
2458+
* only, through `registerInMemory` (no persistence, no announce).
2459+
*/
2460+
private async bridgeRuntimeObjectsToMetadataService(ctx: PluginContext): Promise<void> {
2461+
if (!this.ql?.registry) return;
2462+
let metadataService: IMetadataService | undefined;
2463+
try {
2464+
metadataService = ctx.getService<IMetadataService>('metadata');
2465+
} catch {
2466+
return; // no metadata service on this kernel — there is no copy to keep
2467+
}
2468+
if (!metadataService) return;
2469+
const failed: string[] = [];
2470+
let copied = 0;
2471+
for (const obj of this.ql.registry.getAllObjects()) {
2472+
try {
2473+
if (await this.mirrorRuntimeObject(metadataService, obj.name)) copied++;
2474+
} catch {
2475+
failed.push(obj.name);
2476+
}
2477+
}
2478+
if (copied > 0) {
2479+
ctx.logger.debug('Copied runtime-authored objects into the metadata service', { count: copied });
2480+
}
2481+
if (failed.length > 0) {
2482+
ctx.logger.error(
2483+
`[ObjectQLPlugin] the metadata service holds no copy of runtime-authored object(s) ${failed.map((n) => `'${n}'`).join(', ')}: `
2484+
+ 'the registry, the data door and REST serve them, but the MCP exposure gates read an absent copy as "no '
2485+
+ 'declaration" and serve them even when their API exposure is off, and MCP does not list them. Fix: restart '
2486+
+ 'this process; if it recurs, check that the metadata service accepts in-memory registration.',
2487+
undefined,
2488+
{ objects: failed },
2489+
);
2490+
}
2491+
}
2492+
24392493
/**
24402494
* Keep the metadata service's copy of a RUNTIME-AUTHORED object current after
24412495
* boot, and take it out when the object is deleted.
@@ -2478,12 +2532,17 @@ export class ObjectQLPlugin implements Plugin {
24782532
* Afterwards the registry is asked again: a re-creation that landed while
24792533
* the removal was awaiting its loaders has its copy put back.
24802534
*
2481-
* Armed on the kernel shape the bridge runs on (no `environmentId`): a
2482-
* project-scoped kernel's service is sourced from the artifact and holds no
2483-
* copy of the registry to keep current.
2535+
* Armed on every kernel shape. On a kernel with an `environmentId` (`os dev`
2536+
* as `env_local`, the cloud single-environment runtime) the service is
2537+
* sourced from the artifact and the boot bridge does not run, so a
2538+
* runtime-authored object had no copy there at all: both exposure gates read
2539+
* that absence as "no declaration" and served an object saved with
2540+
* `apiEnabled: false`. The copy written here is the object THIS kernel's
2541+
* authoring door just saved, so the cross-project leakage the full bridge is
2542+
* gated against does not arise. Its boot half on that shape is
2543+
* {@link bridgeRuntimeObjectsToMetadataService}.
24842544
*/
24852545
private followRuntimeObjectCopy(ctx: PluginContext, evt: { name: string; state: string }): void {
2486-
if (this.environmentId !== undefined) return;
24872546
if (typeof evt?.name !== 'string' || evt.name === '') return;
24882547
// An async function runs synchronously up to its first `await`: the
24892548
// refresh of a served runtime object is done before this call returns.
Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* Test scaffolding for the two `storage-upload-*.wall.test.ts` files: each pays
5+
* its booted wall's FIRST load at module scope, with
6+
* `await warmUpUploadWallBoot()`, never inside a clocked window.
7+
*
8+
* The convention is the repo's: "clocked windows measure behaviour, never
9+
* loading — a test that boots a real plugin chain pays its first load at
10+
* module top" (AGENTS.md, Build & Test). vitest clocks a file's hooks and test
11+
* bodies and reports their sum as the file's duration — the module line the
12+
* hourly timing table reads — while the module's own evaluation runs during
13+
* COLLECTION, which no clock covers.
14+
*
15+
* Why these files need it: both boot the same walled stack in their
16+
* `beforeAll`, and the first `bootStack` in a process is 2.3 to 2.8 s slower
17+
* than every later one (measured per boot, 4 vCPU: 3.2 to 3.8 s against 0.9 to
18+
* 1.2 s). The difference is first use, not the boot: the default datasource's
19+
* driver and better-auth's entries are `import()`ed lazily from inside the
20+
* boot, sql.js compiles its WASM, and every path runs cold. Without this, that
21+
* cost lands in the `beforeAll`, and the timing table charges it to the file.
22+
*
23+
* It is `@objectstack/verify`'s `warmUpBoot` (`boot-warm-up.testkit.ts`, which
24+
* that package does not export) with this package's boot: the same two plugins
25+
* the files mount, under the same two environment variables. So the lazily
26+
* loaded code is loaded under the environment the files' own boots load it
27+
* under (`bootStack` sets `NODE_ENV` itself, first, in both), and a module
28+
* that reads its environment once, when it is first loaded, reads the same
29+
* values either way. Afterwards the WHOLE environment is put back as it was —
30+
* the two variables, and every variable the boot set (`NODE_ENV` among them) —
31+
* so the file's first hook starts from the environment it would have started
32+
* from without this.
33+
*
34+
* ⛔ It shares nothing a case asserts on. It boots an app of its own (no
35+
* objects), on its own in-memory database and its own storage root, with
36+
* plugin instances of its own, and stops it before the file's first hook runs.
37+
* What stays warm is process-level: the module registry, sql.js's compiled
38+
* WASM and the JIT.
39+
*
40+
* ⛔ Call it at module scope, on a line of its own, never in a hook: vitest
41+
* clocks a hook with `hookTimeout` exactly as it clocks a test with
42+
* `testTimeout`. And never answer a recurrence by raising a timeout.
43+
*/
44+
45+
import { mkdtempSync, rmSync } from 'node:fs';
46+
import { tmpdir } from 'node:os';
47+
import { join } from 'node:path';
48+
import { bootStack } from '@objectstack/verify';
49+
import { StorageServicePlugin } from '@objectstack/service-storage';
50+
import { OrganizationsPlugin } from '../organizations-plugin.js';
51+
52+
/** The environment both files' `beforeAll` sets before its boot. */
53+
const WALLED_BOOT_ENV = [
54+
['OS_TENANCY_POSTURE', 'isolated'],
55+
['OS_AUTH_MEMBERSHIP_POLICY', 'invite-only'],
56+
] as const;
57+
58+
/**
59+
* Boot a throwaway app on the walled stack the upload-wall files boot, sign
60+
* the seeded admin in, sign one member up, and stop.
61+
*/
62+
export async function warmUpUploadWallBoot(): Promise<void> {
63+
const envBefore = { ...process.env };
64+
for (const [name, value] of WALLED_BOOT_ENV) process.env[name] = value;
65+
const rootDir = mkdtempSync(join(tmpdir(), 'upload-wall-warm-up-'));
66+
try {
67+
// A fresh configuration object per call: `bootStack`'s instance rule keys
68+
// on identity, and nothing here may be shared with the file's own boot.
69+
const stack = await bootStack(
70+
{
71+
manifest: {
72+
id: 'com.example.uploadwallwarmup',
73+
namespace: 'uploadwallwarmup',
74+
version: '0.0.1',
75+
type: 'app',
76+
name: 'Upload Wall Warm-up',
77+
},
78+
objects: [],
79+
} as never,
80+
{
81+
extraPlugins: [
82+
new OrganizationsPlugin(),
83+
new StorageServicePlugin({ adapter: 'local', local: { rootDir }, bindToSettings: false }),
84+
],
85+
},
86+
);
87+
try {
88+
await stack.signIn();
89+
await stack.signUp('upload-wall-warm-up@verify.test');
90+
} finally {
91+
await stack.stop();
92+
}
93+
} finally {
94+
rmSync(rootDir, { recursive: true, force: true });
95+
for (const name of Object.keys(process.env)) {
96+
if (!(name in envBefore)) delete process.env[name];
97+
}
98+
for (const [name, value] of Object.entries(envBefore)) {
99+
if (value !== undefined && process.env[name] !== value) process.env[name] = value;
100+
}
101+
}
102+
}

‎packages/plugins/organizations/src/storage-upload-door-ownership.wall.test.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,13 @@ import { join } from 'node:path';
5252
import { bootStack, type VerifyStack } from '@objectstack/verify';
5353
import { StorageServicePlugin } from '@objectstack/service-storage';
5454
import { OrganizationsPlugin } from './organizations-plugin.js';
55+
import { warmUpUploadWallBoot } from './__tests__/upload-wall-warm-up.testkit.js';
56+
57+
// The walled boot's first load (lazily imported drivers and better-auth entries,
58+
// sql.js's WASM compile) is paid here, during collection, so the `beforeAll`
59+
// below clocks the boot and not the loading (AGENTS.md: clocked windows
60+
// measure behaviour, never loading). See the helper for what it shares: nothing.
61+
await warmUpUploadWallBoot();
5562

5663
const APP = {
5764
manifest: {

0 commit comments

Comments
 (0)