Skip to content

Commit 1ac2bb5

Browse files
perf(metadata-protocol,rest,runtime): /meta/app judges its navigation on the execution face; diagnostics verdicts are memoized by content and schema (#25941)
<!-- claude-projects-attribution: {"github_login":"os-elon-musk"} --> _Requested by **Elon** · [project thread](https://claude.ai/code/project/chan_01WedDzV26nniwxv7jC88B9D?thread=cmsg_01WedDzV26nniwxv7jC88B9D7ZazGN9DENufYRJ7p13Wia)_ Closes #25783 Clause-②: yes (widening): the exported `RestProtocol` type of `@objectstack/rest` gains one optional member, `getMetaItemsForExecution?`. Measured on the built `dist/index.d.ts` at `cb0a2fcf` and at `22f886e1`: rest's export list is unchanged (67 names) and, outside doc comments, that member is the only changed line; the `@objectstack/metadata-protocol` and `@objectstack/runtime` declarations are byte-identical. Metadata-protocol: none A read-path cost change. No accept set, served body, route parameter or error code changes, and nothing under `packages/spec` or `ui/` moves. The one published type that changes is `RestProtocol`, by the optional member above (changeset `.changeset/25783-rest-meta-app-execution-face.md`); all three packages stay `patch`, as `Metadata-protocol: none` requires. The served bytes of six metadata types were hashed before and after and are identical (below). ## What changes **1. `GET /meta/app` judges its navigation on the protocol's execution face** (`packages/rest/src/rest-server.ts`, `packages/runtime/src/domains/meta.ts`). The nav-servability gate (`resolveNavServability`, #24031) reads each object's `enable` to prune the nav entries whose object refuses `list`. It read them through `loadObjectItems` → `metaListSource` → the served `getMetaItems`, which stamps `_diagnostics` on every object, so each `/meta/app` read ran a full zod `safeParse` of every object definition only to read `enable`. Measured at 63 of 73 ms CPU per read. `RestServer.metaListSource` now asks `getMetaItemsForExecution` for `object` when the protocol has it: the same list, without the serving decorations (the #24031 / #24535 precedent for `/data`). The dispatcher's `metaItemReadGateSources.listMetaItems` does the same (seat amendment 6407727649), so `/meta/app` through the `${prefix}/*` catch-all gets the same cut. A protocol without the execution face is asked `getMetaItems` as before. `RestProtocol` gains the optional `getMetaItemsForExecution` member; `MetaDomainProtocol` gains it in its existing partial pick of the protocol type. Book and doc lists stay on the served face (their readers want the served shape). **2. Diagnostics verdicts are memoized by schema and content** (`packages/metadata-protocol/src/metadata-diagnostics.ts`). `computeMetadataDiagnostics` keeps one bounded LRU (4096 entries) per schema object, held in a `WeakMap`, keyed by a SHA-256 of the stripped document. The key encodes everything the parse can tell apart: plain JSON documents key on `JSON.stringify`; a document holding `undefined` (a strict schema refuses an undeclared key whose value is `undefined`, and an array hole parses as `undefined`) keys on an encoding that spells `undefined` out; a document holding anything without a faithful key (a function, a class instance, `NaN`/`Infinity`, `-0`, nesting past 256) is parsed every time, as before. A `registerMetadataTypeSchema` replacement is a new schema object and starts from nothing. Each caller gets its own copy of the verdict. The inputs are only the schema and the document: no global zod error map is installed anywhere, and the node-config contract maps the refinements read are fixed after first build. This is not a cache of any read's answer; it sits below every merge, on a pure function of its own input, so a metadata change is a different document and a different key. `protocol.ts` is untouched (0 lines). ## Measured Showcase app (`examples/app-showcase`, 108 objects), local, signed in as the seeded admin. Main-thread CPU per request from `/proc/PID/task/PID/stat` over 40 serial requests after 5 warm-ups; shares from a V8 CPU profile at 250 µs sampling over 40 more. Shared box (4 cores), so read the ratios. | Endpoint | CPU before (`cb0a2fcf`) | CPU after (`a8f08c97`) | `computeMetadataDiagnostics` share before → after | |:--|--:|--:|--:| | `GET /api/v1/meta/app` | 73.25 ms | 16.75 ms | 59.7% → 1.3% | | `GET /api/v1/meta/object` | 85.75 ms | 37.00 ms | 50.7% → 15.6% | | `GET /api/v1/meta/view` | 13.25 ms | 11.25 ms | 11.8% → 3.1% | Served bytes, SHA-256 prefix of each `GET /api/v1/meta/:type` body, before vs after on the first read after boot and on a warm read: | Type | Bytes | Before | After, cold | After, warm | |:--|--:|:--|:--|:--| | app | 17,559 | `68ea5cb73ccd` | `68ea5cb73ccd` | `68ea5cb73ccd` | | object | 694,011 | `3fe564c0fe63` | `3fe564c0fe63` | `3fe564c0fe63` | | view | 23,954 | `71352ec4f6a6` | `71352ec4f6a6` | `71352ec4f6a6` | | flow | 52,153 | `d35da2f62bcd` | `d35da2f62bcd` | `d35da2f62bcd` | | dashboard | 12,075 | `ef3c4c852a3d` | `ef3c4c852a3d` | `ef3c4c852a3d` | | page | 115,021 | `bf75d678c85d` | `bf75d678c85d` | `bf75d678c85d` | ## Tests - `packages/rest/src/meta-app-nav-servability-execution-read.test.ts` (new): the real REST handlers over the real `ObjectStackProtocolImplementation` and `SchemaRegistry`, with the `object` schema behind a recording proxy (the `rest-exposure-gate-execution-read.test.ts` instrument). Unscoped and environment-scoped: the app list and the app by name prune the unlistable entry and keep the listable one with **zero** object diagnostics; FALLBACK, a protocol without the execution list face, gets the same prune and diagnoses every object (so the zeros are not vacuous); CONTROL, `GET /meta/object` still serves `_diagnostics` on every object. On `cb0a2fcf`'s `rest-server.ts` the four zero rows fail (`expected [ 'listed_widget', 'unlisted_widget' ] to deeply equal []`). - `packages/runtime/src/domains/meta-app-nav-object-execution-face.test.ts` (new): through `HttpDispatcher.dispatch()`, `/meta/app` and `/meta/app/:name` read `object` on `getMetaItemsForExecution` and never on `getMetaItems`, with the same prune; without the execution face it falls back to `getMetaItems`. The two execution-face rows fail on `cb0a2fcf`'s `meta.ts`. - `packages/metadata-protocol/src/metadata-diagnostics.memo.test.ts` (new): one parse per document per schema, a structurally equal copy and the decorated served body included; a changed document, a replaced schema each re-parse; a missing key vs the key holding `undefined`, `null` vs an array hole, `null` vs `NaN` each get their own verdict in both orders; a function, `-0`, a 300-deep nesting and a `Date` parse every time; callers get independent copies; the 4096 LRU bound evicts the least recently used. On the real `object` schema, a memo hit serves byte-identical `decorateMetadataItem` output to a fresh parse, for a valid and an invalid document. Mutation check: with a naive `JSON.stringify` key all six collision rows fail. - `packages/rest/src/rest-exposure-gate-execution-read.test.ts` (re-pinned): its recording proxy is now registered per test (`beforeEach`), because verdicts are memoized per schema object; five rows recorded zero parses under one shared proxy. Full suites: `@objectstack/metadata-protocol` 264 files / 21,505 tests, `@objectstack/rest` 328 / 6,730, `@objectstack/runtime` 448 / 8,011, all green. `typecheck` green for the three packages (test layers included for rest and runtime). ## Gates `node scripts/pm/dispatch-gates.mjs --commands` derives 71 commands for this diff; all 71 ran with exit 0, and `--ran` reconciles 71 run, 0 NOT-MEASURED, 0 UNRUN. Four were red on the first pass and green after: - `check-rest-log-spy-declared`: the new rest test spied on `console.warn` without declaring `OS_REST_LOG`. The spies asserted nothing, so they are gone. - `check:engine-double-contract`: the new rest test carried a `findOne` double no path reached. Dropped, so the pinned ledger is unchanged. - `check-plugin-teardown-shape --self-test`: shallow clone missing the pinned fixture commit; fetched it. - `check:dual-build-cjs-loads`: prerequisite, seven packages had no `dist/`; built them. `pnpm lint` green. ## Acceptance notes - **`GET /meta/app` lands at 16.75 ms against the card's ≤15 ms.** What remains is not diagnostics (1.3%): the profile puts about 4.4 ms in artifact resolution (`lookupArtifactItem` / `foldExtenders`) and about 7.8 ms in fixed per-request overhead (`(program)`, routing, auth). Neither is on this claim's file surface. The maintainer relaxed both targets to these numbers (ruling 6408692677, letter A). - **`GET /meta/object` lands at 15.6% in `computeMetadataDiagnostics` against the card's ≤5%.** Warm reads run no zod parse at all; the remaining 9.3 ms is the content key itself, one `JSON.stringify` plus SHA-256 over 694 KB of definitions (about 6 ms in an isolated bench) and the shape walk. A content key costs about one serialization, so content keying cannot reach 5%. The card's other route does: Studio opts in to `_diagnostics` with a query switch, and plain `/meta/object` reads skip them. That changes `/meta`'s parameters and the served body for every caller that does not opt in, and about a dozen `_diagnostics` readers in `ui/` would have to send the switch: a contract change, outside this claim (claim 6407697525). Reported to the seat as a finding; not dispatched (ruling 6408692677). - **Deviation:** the before/after servers were started with `pnpm --filter @objectstack/example-showcase dev -- --fresh -p 38425` rather than `pnpm dev`, because `check-dev-prereqs` refused `pnpm dev` with 45 unbuilt packages in this worktree. The same command and port served both sides. --- _Generated by [Claude Code](https://claude.ai/code/session_019d4JoyFgaZCRubmNWh91uZ)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 69fa23d commit 1ac2bb5

10 files changed

Lines changed: 712 additions & 6 deletions
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
'@objectstack/metadata-protocol': patch
3+
'@objectstack/runtime': patch
4+
---
5+
6+
perf(metadata-protocol,runtime): served metadata lists parse each unchanged document once, and the dispatcher's `/meta/app` reads objects on the execution face
7+
8+
Clause-②: no
9+
Metadata-protocol: none
10+
11+
- **Diagnostics verdicts are memoized.** `computeMetadataDiagnostics` remembers a verdict per schema object and per document content (a SHA-256 of an encoding that keeps apart everything the parse keeps apart), so a served list of unchanged documents parses each one once instead of on every read. A replaced type schema starts from nothing, each caller gets its own copy, and a document with no faithful key is parsed every time as before. The served bytes are unchanged. Measured on the showcase app (108 objects, local, main-thread CPU per request): `GET /meta/object` 85.75 ms → 37 ms.
12+
- **The dispatcher's `/meta/app`** (the `${prefix}/*` catch-all) reads each object's `enable` for its nav prune through `getMetaItemsForExecution` where the protocol has it, as `RestServer` does, and `getMetaItems` otherwise.
13+
- **No public surface moves.** The built `dist/index.d.ts` of both packages is byte-identical at `cb0a2fcf` and at this head.
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
'@objectstack/rest': patch
3+
---
4+
5+
perf(rest): `GET /meta/app` judges its navigation on the protocol's execution face, so it no longer schema-parses every object on each read
6+
7+
Clause-②: yes (widening): the exported `RestProtocol` type gains one optional member, `getMetaItemsForExecution?`. Measured on the built `dist/index.d.ts` at `cb0a2fcf` (before) and at this head (after): the export list is unchanged (67 names), and outside doc comments the one changed line is that member. Every other member signature is byte-identical. The member is optional, so nothing a protocol must provide changes.
8+
9+
Metadata-protocol: none
10+
11+
- **What.** The nav-servability gate (#24031) reads each object's `enable` to prune the nav entries whose object refuses `list`. It read them through the served `getMetaItems`, which stamps `_diagnostics` on every object, so every `/meta/app` read ran a full schema parse of every object definition. `RestServer.metaListSource` now asks `getMetaItemsForExecution` for `object` when the protocol has it (the same list, without the serving decorations) and `getMetaItems` otherwise. The batch route's exposure gate and the by-name gate's list fallback read the same source. The `book` and `doc` lists stay on the served face.
12+
- **Measured** on the showcase app (108 objects, local, main-thread CPU per request): `GET /meta/app` 73.25 ms → 16.75 ms.
13+
- **Unchanged.** Every `/meta` answer: the same nav prune, and `GET /meta/object` still serves `_diagnostics` on every object.
Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* #25783 — the diagnostics verdict memo answers what a fresh parse answers,
5+
* and parses a document once per schema however many reads serve it.
6+
*
7+
* Every served metadata read stamps `_diagnostics` on every item, and that
8+
* stamp was a full `safeParse` per item per read: 60 ms of an 86 ms
9+
* `GET /meta/object` over 108 unchanged objects. `computeMetadataDiagnostics`
10+
* now remembers a verdict by the schema object and the document's content
11+
* ("The verdict memo" in `metadata-diagnostics.ts`). These rows pin the two
12+
* halves of that promise:
13+
*
14+
* - **The served bytes do not move.** A memo hit serves, byte for byte, what
15+
* a fresh parse of the same document under a schema with no memo serves —
16+
* a valid document and an invalid one, on the real `object` schema.
17+
* - **Nothing the parse can tell apart shares a key.** A changed document, a
18+
* replaced schema, `undefined` against a missing key, an array hole against
19+
* `null`, `NaN` against `null` — each is parsed again and gets its own
20+
* verdict. A document with no faithful key (a function, `-0`, a nesting past
21+
* the keyable depth) is parsed every time, as before the memo.
22+
*
23+
* The instrument: a schema behind a proxy that counts its `safeParse` calls,
24+
* registered for a probe type through `registerMetadataTypeSchema`. A fresh
25+
* proxy per test is a fresh memo, because the memo is held per schema object.
26+
*/
27+
import { beforeEach, describe, expect, it } from 'vitest';
28+
import { z } from 'zod';
29+
import { getMetadataTypeSchema, registerMetadataTypeSchema } from '@objectstack/spec/kernel';
30+
import { computeMetadataDiagnostics, decorateMetadataItem } from './metadata-diagnostics.js';
31+
32+
/** A type no build registers, so binding it touches no real type's schema. */
33+
const PROBE = 'memo_probe_25783';
34+
35+
const PROBE_SCHEMA = z.object({
36+
name: z.string(),
37+
tags: z.array(z.string().nullable()).optional(),
38+
weight: z.number().nullable().optional(),
39+
nested: z.unknown().optional(),
40+
handler: z.unknown().optional(),
41+
}).strict();
42+
43+
let parses = 0;
44+
45+
/** Bind `schema` to `type` behind a proxy that counts parses; returns the proxy. */
46+
function bindCounting(type: string, schema: z.ZodType): z.ZodType {
47+
const counting = new Proxy(schema, {
48+
get(target, prop, receiver) {
49+
if (prop === 'safeParse') {
50+
return (...args: unknown[]) => {
51+
parses += 1;
52+
return (target.safeParse as (...a: unknown[]) => unknown).apply(target, args);
53+
};
54+
}
55+
return Reflect.get(target, prop, receiver);
56+
},
57+
});
58+
registerMetadataTypeSchema(type, counting);
59+
return counting;
60+
}
61+
62+
beforeEach(() => {
63+
parses = 0;
64+
bindCounting(PROBE, PROBE_SCHEMA);
65+
});
66+
67+
describe('#25783 the verdict memo — one parse per document per schema', () => {
68+
it('answers a repeated document from the memo, a structurally equal copy included', () => {
69+
const doc = { name: 'a', tags: ['x'] };
70+
expect(computeMetadataDiagnostics(PROBE, doc)).toEqual({ valid: true });
71+
expect(computeMetadataDiagnostics(PROBE, doc)).toEqual({ valid: true });
72+
// A row read again from the store is a new object with the same content.
73+
expect(computeMetadataDiagnostics(PROBE, JSON.parse(JSON.stringify(doc)))).toEqual({ valid: true });
74+
expect(parses).toBe(1);
75+
});
76+
77+
it('keys the document without its read decorations, as the parse sees it', () => {
78+
const doc = { name: 'a' };
79+
computeMetadataDiagnostics(PROBE, doc);
80+
// The served body of the same document, decorations and all.
81+
computeMetadataDiagnostics(PROBE, { ...doc, _diagnostics: { valid: true } });
82+
expect(parses).toBe(1);
83+
});
84+
85+
it('parses a changed document again', () => {
86+
expect(computeMetadataDiagnostics(PROBE, { name: 'a' })?.valid).toBe(true);
87+
expect(computeMetadataDiagnostics(PROBE, { name: 'a', bogus: 1 })?.valid).toBe(false);
88+
expect(computeMetadataDiagnostics(PROBE, { name: 'b' })?.valid).toBe(true);
89+
expect(parses).toBe(3);
90+
});
91+
92+
it('starts from nothing when the type schema is replaced', () => {
93+
const doc = { name: 'a', extra: 1 };
94+
expect(computeMetadataDiagnostics(PROBE, doc)?.valid).toBe(false);
95+
bindCounting(PROBE, PROBE_SCHEMA.passthrough());
96+
expect(computeMetadataDiagnostics(PROBE, doc)?.valid).toBe(true);
97+
expect(parses).toBe(2);
98+
});
99+
100+
it('hands each caller its own copy of a verdict', () => {
101+
const doc = { name: 'a', bogus: 1 };
102+
const first = computeMetadataDiagnostics(PROBE, doc)!;
103+
const pristine = JSON.stringify(first);
104+
first.valid = true;
105+
first.errors![0]!.message = 'edited by a caller';
106+
first.errors!.push({ path: 'x', message: 'pushed by a caller', code: 'custom' } as never);
107+
expect(JSON.stringify(computeMetadataDiagnostics(PROBE, doc))).toBe(pristine);
108+
expect(parses).toBe(1);
109+
});
110+
111+
it('keeps at most 4096 verdicts per schema, the least recently used out first', () => {
112+
computeMetadataDiagnostics(PROBE, { name: 'kept' });
113+
computeMetadataDiagnostics(PROBE, { name: 'evicted' });
114+
for (let i = 0; i < 4094; i++) computeMetadataDiagnostics(PROBE, { name: `n${i}` });
115+
// 4096 held. Touch `kept`, so `evicted` is now the least recently used.
116+
computeMetadataDiagnostics(PROBE, { name: 'kept' });
117+
computeMetadataDiagnostics(PROBE, { name: 'one more' });
118+
const before = parses;
119+
computeMetadataDiagnostics(PROBE, { name: 'kept' });
120+
expect(parses - before).toBe(0);
121+
computeMetadataDiagnostics(PROBE, { name: 'evicted' });
122+
expect(parses - before).toBe(1);
123+
});
124+
});
125+
126+
describe('#25783 the verdict memo — no two documents the parse tells apart share a verdict', () => {
127+
// Each pair is run in both orders: whichever is diagnosed first, the
128+
// second must get its own parse and its own verdict.
129+
const pairs: Array<[label: string, accepted: () => unknown, refused: () => unknown]> = [
130+
['a missing key against the same key holding `undefined` (strict schema)',
131+
() => ({ name: 'a' }),
132+
() => ({ name: 'a', undeclared: undefined })],
133+
['`null` against an array hole',
134+
() => ({ name: 'a', tags: [null, 'x'] }),
135+
() => {
136+
const tags: unknown[] = [];
137+
tags[1] = 'x'; // index 0 is a hole: JSON.stringify spells it `null`
138+
return { name: 'a', tags };
139+
}],
140+
['`null` against `NaN`',
141+
() => ({ name: 'a', weight: null }),
142+
() => ({ name: 'a', weight: Number.NaN })],
143+
];
144+
145+
for (const [label, accepted, refused] of pairs) {
146+
for (const order of ['accepted first', 'refused first'] as const) {
147+
it(`${label} — ${order}`, () => {
148+
if (order === 'accepted first') {
149+
expect(computeMetadataDiagnostics(PROBE, accepted())?.valid).toBe(true);
150+
expect(computeMetadataDiagnostics(PROBE, refused())?.valid).toBe(false);
151+
} else {
152+
expect(computeMetadataDiagnostics(PROBE, refused())?.valid).toBe(false);
153+
expect(computeMetadataDiagnostics(PROBE, accepted())?.valid).toBe(true);
154+
}
155+
expect(parses).toBe(2);
156+
});
157+
}
158+
}
159+
160+
it('parses a document with no faithful key every time: a function, `-0`, a nesting past the keyable depth', () => {
161+
let deep: Record<string, unknown> = {};
162+
const root = deep;
163+
for (let i = 0; i < 300; i++) deep = (deep.next = {}) as Record<string, unknown>;
164+
const unkeyable = [
165+
{ name: 'a', handler: () => 1 },
166+
{ name: 'a', weight: -0 },
167+
{ name: 'a', nested: root },
168+
{ name: 'a', nested: new Date(0) },
169+
];
170+
for (const doc of unkeyable) {
171+
const before = parses;
172+
const first = computeMetadataDiagnostics(PROBE, doc);
173+
const second = computeMetadataDiagnostics(PROBE, doc);
174+
expect(second).toEqual(first);
175+
expect(parses - before).toBe(2);
176+
}
177+
});
178+
});
179+
180+
describe('#25783 the verdict memo — a hit serves the bytes a fresh parse serves (real `object` schema)', () => {
181+
const OBJECT_SCHEMA = getMetadataTypeSchema('object')!;
182+
const documents: Array<[label: string, valid: boolean, doc: Record<string, unknown>]> = [
183+
['a valid object', true, {
184+
name: 'memo_widget', label: 'Memo Widget',
185+
fields: { title: { name: 'title', type: 'text' } },
186+
enable: { apiMethods: ['get', 'list'] },
187+
}],
188+
['an invalid object', false, {
189+
name: 'memo_widget', label: 'Memo Widget',
190+
fields: { title: { name: 'title', type: 'no_such_field_type' } },
191+
bogusTopLevelKey: true,
192+
}],
193+
];
194+
195+
for (const [label, valid, doc] of documents) {
196+
it(label, () => {
197+
try {
198+
// A schema with no memo: the reference answer.
199+
bindCounting('object', OBJECT_SCHEMA);
200+
parses = 0;
201+
const fresh = JSON.stringify(decorateMetadataItem('object', structuredClone(doc)));
202+
// The same schema object again: the second read is a memo hit.
203+
const hit = JSON.stringify(decorateMetadataItem('object', structuredClone(doc)));
204+
expect(JSON.parse(fresh)._diagnostics.valid).toBe(valid);
205+
expect(parses).toBe(1);
206+
expect(hit).toBe(fresh);
207+
// And a further fresh schema answers it the same way.
208+
bindCounting('object', OBJECT_SCHEMA);
209+
expect(JSON.stringify(decorateMetadataItem('object', structuredClone(doc)))).toBe(fresh);
210+
expect(parses).toBe(2);
211+
} finally {
212+
registerMetadataTypeSchema('object', OBJECT_SCHEMA);
213+
}
214+
});
215+
}
216+
});

0 commit comments

Comments
 (0)