Skip to content

Commit 78c13bd

Browse files
fix(metadata-protocol): a republish of a view container retires the views it dropped, through one reconcile (#23234)
Part of #23198 Clause-②: no ## What was wrong A view container (a `defineView` document such as `showcase_task`) registers its own name plus one view per entry, under `OBJECT.VIEWKEY`, in the process registry. A republish of the container registered the views it expanded to and retired none it no longer expanded to. After a publish that dropped `listViews.added_by_overlay`, `GET /api/v1/meta/view`, the `?object=` list and the by-name read kept serving `showcase_task.added_by_overlay` until a restart. Measured before the change, on this PR's pins (red at `0219bb6a0`) and over the real `SchemaRegistry` (see Verification). ## The change (`packages/metadata-protocol/src/protocol.ts`) One private reconcile, `reconcileViewExpansions(type, write, organizationId)`. After any write that changes a container's stored row, it leaves the registry holding the views that row expands to now: 1. It registers each expanded view, except a name another package ships ([#21980], [#22057]). That name is never recorded. 2. It replaces the record of the rows the write addresses with the names it just registered. The record is PR #23196's, now kept per stored row. 3. It heals each name the rows registered before and do not register now, through PR #23196's per-name walk (`restoreArtifactRegistryView`). A packaged view of that name is un-shadowed, a baseline is re-registered on a control-plane kernel, and a name no layer serves is retired. Both sides call it, and nothing else registers an expansion or touches the record: - **The registration.** `hydrateExpandedViewItems` passes the row's binding and its expansion. `hydrateOverlayIntoRegistry` reaches it on both of its registering paths. That covers every hydration: the write-through (save-publish, publish, rollback, revert, a replica's convergence), the list read and the boot. - **The heal.** `restoreArtifactRegistryView` passes `removed: 'every-row'` (an empty set) ahead of its tier walk. A reset, a delete, a revert's removal and a replica's removal reach it there. The walk reads the MetadataService, so the reconcile is async. `hydrateOverlayIntoRegistry` and `applyRegistryWriteThrough` became `async`, and all eight call sites await them. A publish therefore answers after the heal. The `#9111` type assert still runs before the first `await`, and its refusal reaches the same `catch` as before. `protocol.hydrate-overlay-canonical-type.test.ts` awaits the hydrator now. Its four affected cases assert what they asserted before: code, status, message and nothing minted. **Why the record is per row (zone 2, hypothesis 2, measured).** One container name can have a package-less row and a package-bound row (ADR-0048), and the list read hydrates both on every call. If the record is keyed on the name alone, each row reconciles against the other row's set. The first list read then retired the package-less row's own member (ablation B below). Keyed per row, a republish of one row heals only what that row dropped. A view both rows expand to, which one row drops, leaves the registry with that row. Both doors answer it from the other row's stored row, and the next list read registers it again (pinned). A reset still heals every row of the name, because the heal is handed no binding (#23196's behaviour, unchanged). ## Zone 2 hypotheses 1. **The record is a union. Confirmed.** Pin: a reset after a republish leaves alone a name the container dropped earlier. Before the change, that reset healed a bare entry held by another source. 2. **Two rows of one name. Confirmed for a name-only key.** Answered with per-row keys (above). 3. **Callers, enumerated from the tree.** `hydrateOverlayIntoRegistry` has three callers: the list read's hydration in `getMetaItems` (not the cluster peer), `applyRegistryWriteThrough`, and `loadMetaFromDb`. The peer reaches the registry through `applyRegistryWriteThrough` or `restoreArtifactRegistryView`. The completeness pin reads that population from the source. 4. **The gates.** The organization gate is in the reconcile itself and in both callers. Re-registration stays control-plane only, inside the walk. The another-package skip is inside the reconcile's registration loop. #23196's pins for the organization gate and the another-package skip pass unchanged. 5. **A degraded tier-2 read.** Inherited from the code, not pinned by a run. The walk stops, so that name stays registered (and is no longer recorded) until a restart. No retry was added. ## Pins `protocol.view-container-reset-expansions.test.ts`, a new `[#23198]` block of 5 cases on #23196's doubles (so no new engine-double rows). Each case asserts the list read, the `?object=` list and the by-name read together: - a republish adds a member; - the card's repro: a republish drops two members, so the one no package ships is served by no door, and the packaged `showcase_task.in_progress` is served again, once; - a reset after a republish; - a delete of a container no package ships, after a republish that dropped a member; - two rows of one container name. `protocol.view-expansion-reconcile-call-sites.test.ts`, the family's closing check (13 cases, source-reading, no CI gate). It covers three populations: - **Stored-row writes.** Every repository write verb and raw `sys_metadata` write in `protocol.ts` (9 sites) carries a recorded route to the write-through or the heal. A same-member route must sit after its write and before the member's next write. A write with no route turns it red, naming the write. - **The chain.** The write-through reaches the hydrator. Every registering path of the hydrator reaches the expansion hydration. The heal reconciles ahead of its walk, past its organization gate. Every call into the chain is awaited. - **The record and the verbs.** Only the reconcile reads or writes the record, and only the walk calls the heal verbs. The data door cannot write `sys_metadata`: its `apiMethods` are `get` and `list`. `protocol.register-item-call-sites.test.ts` (#22057): the expansion's `registerItem(` row moves to `reconcileViewExpansions`. ## Ablation (fix committed first; `scripts/ablation-replace.mjs` WRAP mode; these suites import `./protocol.js`, so they resolve from `src/` and need no build) Every leg was restored with blob equal to HEAD and an empty `git diff HEAD`. - **A. The heal of a dropped name made a no-op** (anchor 1 to 0, blob `a9b1a41c7` to `5faee5a1f`). 9 red, 2 green: the dashboard control and "a republish adds a member". - **B. The record keyed on the container name alone** (blob to `70a74e603`). Only the two-row case is red, at `read 1: expected undefined to be 'A only'`. - **C. The bound-row path of the hydrator registers without reconciling.** This leg took three attempts: 1. Replacing the call and its return with a commented `return true;` went red, but only at the closing check's path count. So the count now reads a trailing comment too (`675c31319`). 2. A bare `return true;` was refused by the tool: its replacement restated part of the anchor, and the replace count stayed 5 to 5. Nothing ran in that attempt. 3. Deleting the `await this.hydrateExpandedViewItems` line (blob to `c85a10a1c`) turned only the closing check red, in 2 cases. All 11 behaviour pins stayed green, so this bypass is visible to the closing check alone. - **D. The delete's heal after `repo.delete` removed** (blob to `e1658dc6f`). The census is red: `deleteMetaItem: repo.delete( → restoreArtifactRegistryView`. - **E. The rollback's write-through not awaited** (at `99dd8fda3`, blob `e5645d2f9` to `a6c6cec44`). The await case is red. Legs A to D ran at `a4b879a33` and `675c31319`, on the `protocol.ts` blob `a9b1a41c7` that the merge left unchanged. After them, `protocol.ts` changed only in comments. ## Verification (HEAD `99dd8fda35`) - Full `@objectstack/metadata-protocol` suite on the merged tree `4035e9baee` (after `main` at `75cada0ea`), run in two halves: 116 files passed and 3 skipped (25931 tests), then 118 files passed (2370 tests). VERDICT command-exit 0 for both. Before the merge, at `a4b879a33`: 234 files passed, 3 skipped, 28301 tests. `99dd8fda35` adds comment lines and one closing-check case, which ran green with typecheck. - `pnpm --filter @objectstack/metadata-protocol typecheck`: exit 0. `tsc --listFiles` lists all four changed or new test files. - **Real composition** (scratch file in `packages/objectql/src`, not committed, over the rebuilt `dist/`, with the real `SchemaRegistry`). The three legs are the card's repro then a reset, the delete, and the two rows. They are 3/3 green. Against the base `protocol.ts` (rebuilt, then restored to HEAD and rebuilt) they are 3/3 red, at `Added` and `Closed` still served and `a_only` still registered. 12 files and 178 tests are green across them plus the 11 `objectql` suites that drive this heal and hydration. - **Gates.** `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` at `99dd8fda35` derives 65 commands. `--ran` with exit codes reports 64 run with exit 0 and 1 NOT MEASURED. `check:lean-entry-closure` first exited 3 because `objectql/dist` was absent; after the `objectql` build it exited 0. **NOT MEASURED: `check:dual-build-cjs-loads`**, reason: exit 3, 60+ packages unbuilt in this container. Narrow stand-in: `require` of `packages/metadata-protocol/dist/index.cjs` loads `ObjectStackProtocolImplementation` (a function). - **Lint, a declared narrowing.** All 5 changed `.ts` files are in `eslint.config.mjs`'s population (`--print-config` resolves each). `eslint --no-inline-config --format json` reports 5 files, 0 errors and 0 warnings. Type-aware linting is not enabled: the resolved config has no `parserOptions.project` or `projectService`. So this diff cannot move the verdict of an untouched file. The full `pnpm lint` is CI's. - **Built declarations.** `ObjectStackProtocolImplementation` gains the untyped private name `reconcileViewExpansions` and loses `recordViewExpansion` and `takeViewExpansions`. `ViewContainerWrite` is module-private and absent from `dist/index.d.ts`. No public member, export, schema or accept-set changes, so `Clause-②: no` stands. - Changed lines: +635 / -102 in 6 files against merge base `75cada0ea`. ## Acceptance notes - **Orphan adoption bypasses the reconcile, and the closing check names it** (recorded as a shrink-only `bypass` row). `POST /api/v1/packages/ID/adopt-orphans` (`reassignOrphanedMetadata`) rebinds a package-less view container row to another package with a raw `sys_metadata` update and touches no registry entry. This was measured over the real `SchemaRegistry` with this fix built. After adopting an overlay of `showcase_task` into another package, the row expands to `showcase_task.showcase_task*` for that package. Meanwhile the package-less registration's bare `showcase_task`, `showcase_task.default` (`All Tasks (X)`, 6) and `showcase_task.in_progress` stay registered, and all three doors keep serving `All Tasks (X)` for `showcase_task.default` until a restart. It is the same family, but a fifth write kind outside this claim's region. It is reported for the seat to route; see the report on #23198. - **Residual, documented in the reconcile's docblock.** A dropped name whose walk stops on a degraded MetadataService read stays registered, no longer recorded, until a restart. Before this change the same residual was reachable from a reset only. - **Two rows of one name.** A reset or delete of either row heals both rows' views. The remaining row is hydrated again on the next list read. This is #23196's behaviour, unchanged. - **Test hygiene, not addressed.** `protocol.code-defined-datasource-door.test.ts` calls the now-async hydrator without `await` in its `hydrate` helper. It stays correct because a `datasource` row registers before the first `await`. - Not done: a live API run on `examples/app-showcase`. The real-registry scratch stands in. --- _Generated by [Claude Code](https://claude.ai/code/session_01P88ehrZ6hM19hBzdpsQTTo)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 197fbe2 commit 78c13bd

6 files changed

Lines changed: 635 additions & 102 deletions
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
"@objectstack/metadata-protocol": patch
3+
---
4+
5+
Republishing a view container without one of its views now stops serving that view, without a restart
6+
7+
Clause-②: no
8+
9+
- A view container (a `defineView` document, such as a package's `showcase_task` views) expands to one view per entry, registered in the process registry under `<object>.<viewKey>`. A publish of the container registered the views it expanded to and removed none it no longer expanded to. So after `POST /api/v1/meta/view/NAME/publish` of a container whose `listViews` lost an entry, `GET /api/v1/meta/view`, `GET /api/v1/meta/view?object=OBJECT` and the by-name read of that view kept serving it until the process restarted.
10+
- Every write that changes a view container's stored row now leaves the registry holding the views that row expands to: a publish, a save that publishes, a rollback, a revert, a replica's convergence, a reset and a delete. So does the list read's and the boot's hydration of a stored row. The write answers after this, so a read that follows it serves the published set.
11+
- A view the container no longer expands to is healed the way a reset heals it: where a package ships a view of that name, the package's view is served again, and a name nothing else serves is removed. As for a reset, a name whose MetadataService read fails stays registered until a restart.
12+
- When one container name has two stored rows (one bound to no package, one bound to a package), each row is compared with what that row registered before. A republish of one row removes only the views that row dropped, and the list read leaves both rows' views registered however often it runs. A view both rows expand to, which one row drops, is served from the other row's stored row and registered again by the next list read. A reset or delete of the name heals every row's views, as before.
13+
- Unchanged: an organization-scoped write touches no registry entry, a baseline is re-registered on a control-plane kernel only, and a view name another package ships is never registered or healed for the container. Every other type registers and resets as before.
14+
- No public export, signature, schema or accept-set change. In the built entry declarations, `ObjectStackProtocolImplementation` gains one untyped `private` member name and loses two `private static` ones.

‎packages/metadata-protocol/src/protocol.hydrate-overlay-canonical-type.test.ts‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -231,11 +231,11 @@ describe('[#9111] the manifest map is tolerant AND incomplete where the URL map
231231
// ═══════════════════════════════════════════════════════════════════════════
232232

233233
describe('[#9111] `hydrateOverlayIntoRegistry` asserts its type is canonical', () => {
234-
it('refuses a manifest-BLIND plural with code + status, and mints nothing', () => {
234+
it('refuses a manifest-BLIND plural with code + status, and mints nothing', async () => {
235235
const { protocol, registeredItems } = makeUnscopedProtocol([]);
236236
let caught: any;
237237
try {
238-
protocol.hydrateOverlayIntoRegistry('fields', { name: 'showcase_task.title' }, {
238+
await protocol.hydrateOverlayIntoRegistry('fields', { name: 'showcase_task.title' }, {
239239
packageId: null,
240240
organizationId: null,
241241
});
@@ -248,45 +248,45 @@ describe('[#9111] `hydrateOverlayIntoRegistry` asserts its type is canonical', (
248248
expect(registeredItems).toHaveLength(0);
249249
});
250250

251-
it('refuses a manifest-PRESENT plural too — the #8862 residue this card was filed on', () => {
251+
it('refuses a manifest-PRESENT plural too — the #8862 residue this card was filed on', async () => {
252252
// `'objects'` folds in both maps, so no producer delivers it today.
253253
// The assert is the COMPLETE contract rather than the narrow
254254
// at-rest predicate, so a future unfolded caller is refused whichever
255255
// plural it holds. Ablation 3 is what keeps this distinct from §3.
256256
const { protocol, registeredItems } = makeUnscopedProtocol([]);
257-
expect(() => protocol.hydrateOverlayIntoRegistry('objects', { name: 'ticket' }, {
257+
await expect(protocol.hydrateOverlayIntoRegistry('objects', { name: 'ticket' }, {
258258
packageId: null,
259259
organizationId: null,
260-
})).toThrowError(/Refusing to register a SchemaRegistry overlay entry/);
260+
})).rejects.toThrowError(/Refusing to register a SchemaRegistry overlay entry/);
261261
expect(registeredItems).toHaveLength(0);
262262
});
263263

264-
it('is judged on the SPELLING, ahead of every no-op return', () => {
264+
it('is judged on the SPELLING, ahead of every no-op return', async () => {
265265
// An org-scoped row returns `false` without registering. The assert
266266
// still fires: a caller that stopped folding must not be able to hide
267267
// behind a body this call happened not to register anyway.
268268
const { protocol } = makeUnscopedProtocol([]);
269-
expect(() => protocol.hydrateOverlayIntoRegistry('fields', { name: 'x' }, {
269+
await expect(protocol.hydrateOverlayIntoRegistry('fields', { name: 'x' }, {
270270
packageId: null,
271271
organizationId: 'org_alpha',
272-
})).toThrowError(/Refusing to register a SchemaRegistry overlay entry/);
272+
})).rejects.toThrowError(/Refusing to register a SchemaRegistry overlay entry/);
273273
// …and a body with no `name` at all.
274-
expect(() => protocol.hydrateOverlayIntoRegistry('fields', { label: 'no name' }, {
274+
await expect(protocol.hydrateOverlayIntoRegistry('fields', { label: 'no name' }, {
275275
packageId: null,
276276
organizationId: null,
277-
})).toThrowError(/Refusing to register a SchemaRegistry overlay entry/);
277+
})).rejects.toThrowError(/Refusing to register a SchemaRegistry overlay entry/);
278278
});
279279

280-
it('cannot refuse a canonical type or a plugin-registered kind', () => {
280+
it('cannot refuse a canonical type or a plugin-registered kind', async () => {
281281
const { protocol, registeredItems } = makeUnscopedProtocol([]);
282282
// Canonical — registers, as before.
283-
expect(protocol.hydrateOverlayIntoRegistry('view', { name: 'grid' }, {
283+
expect(await protocol.hydrateOverlayIntoRegistry('view', { name: 'grid' }, {
284284
packageId: null, organizationId: null,
285285
})).toBe(true);
286286
// A kind the platform has never heard of: `canonicalMetaType` is the
287287
// identity for anything the static map does not carry, so this gate is
288288
// positive-control-by-construction and can never fire on a plugin kind.
289-
expect(protocol.hydrateOverlayIntoRegistry('acme_widget', { name: 'w' }, {
289+
expect(await protocol.hydrateOverlayIntoRegistry('acme_widget', { name: 'w' }, {
290290
packageId: null, organizationId: null,
291291
})).toBe(true);
292292
expect(registeredItems.map((r) => r.type)).toEqual(['view', 'acme_widget']);

‎packages/metadata-protocol/src/protocol.register-item-call-sites.test.ts‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -103,12 +103,13 @@ const SITES: ReadonlyArray<Site & {
103103
viewOnlyBy: 'const bound = canonicalType === \'view\' ? boundPackageOf(options.packageId) : undefined;',
104104
},
105105
{
106-
member: 'hydrateExpandedViewItems', args: 'type, item, \'name\' as any',
106+
// [#23198] Moved from hydrateExpandedViewItems into the one reconcile, which that hydration calls.
107+
member: 'reconcileViewExpansions', args: 'type, item, \'name\' as any',
107108
disposition: 'yes, view only: skips a name another package ships',
108-
path: 'hydrateOverlayIntoRegistry, for each expansion of a stored view container',
109+
path: 'hydrateExpandedViewItems (from hydrateOverlayIntoRegistry), for each expansion of a stored view container',
109110
why: 'an expansion: not registered where another package ships its name (#21980), and the by-name read answers it '
110111
+ 'from its stored container row',
111-
viewOnlyBy: 'if ((PLURAL_TO_SINGULAR[type] ?? type) !== \'view\') return [];',
112+
viewOnlyBy: 'if (canonicalMetaType(type) !== \'view\') return;',
112113
},
113114
{
114115
member: 'restoreArtifactRegistryView', args: 'type, baseline.data, \'name\'',

0 commit comments

Comments
 (0)