Skip to content

Commit 5677add

Browse files
fix(metadata-protocol): two locales of one email template survive a restart — sys_metadata overlay rows keyed by bundle member (#22877, stage 1) (#22954)
Part of #22877 Clause-②: no Stage 1 of 2 of ruling 6116165299 (letter A): the overlay store keys a bundled type's row by its member, so two locales of one `email_template` saved through the metadata door are two `sys_metadata` rows and both survive a restart. Stage 2 (the by-name member selector on the `/meta` doors, `Clause-②: yes`) is a separate PR; #22877 remains open for it. The ruling, quoted: > **A: the overlay store carries the bundled type's member as a key column; one table, one hydration path; each member addressable by name.** > > `sys_metadata` gains one nullable, type-neutral discriminator column. NULL means the canonical member, so every existing row and every type whose identity is `name` alone stays byte-identical. The value is taken from the body through `itemDiscriminator` (`ITEM_KEY_DISCRIMINATORS`), the one place the identity is declared. ## What changed - **Column** (`metadata-core` `sys-metadata.object.ts`): `bundle_member`, nullable text (`maxLength: 100`). NULL is the canonical member and every row of a name-only type. The declared `idx_sys_metadata_overlay_active` gains it. - **Index** (`metadata-protocol` `migrations/overlay-index.ts`): the runtime key is `(type, name, organization_id, COALESCE(package_id, ''), COALESCE(bundle_member, ''))`, active and draft. The change rides the existing ADR-0120 D4 probe-then-replace order unchanged: it drops and rebuilds the real name on every first write, so the next boot replaces the four-part index. A key part only relaxes uniqueness, so no rows a store already holds can block the rebuild. The duplicate-listing query, the dialect fallback and the `os migrate duplicates` probes all read the same builders. - **Backfill** (same module, `backfillOverlayBundleMembers`): through the engine, each `email_template` row with a NULL member whose body names a non-canonical locale gets that locale. It is run by boot hydration (`loadMetaFromDb`, after its own read succeeded) and by the first write. It is idempotent, never throws, and reports a failure at `error` with the consequence and the fix. A missing table counts as benign (`isMissingTableError`). - **Repository** (`sys-metadata-repository.ts`): `put` takes the member from the body (`overlayMemberOfBody` → `itemDiscriminator`), never from the caller. The write head (inherited draft base and orphan adoption included), the row payload, `get`, `headAt`, `delete`, `promoteDraft` (and its drain), `restoreVersion` and `history` all address one member. An omitted `member` means the canonical one. For a name-only type, no `where` and no row payload ever carries the column. - **Protocol** (`protocol.ts`): the save door addresses its stored head and its credential carry-forward by the body's member. The by-name reads that address rows directly (`findServedOverlayRow`, the publish door's draft read, the diff's active read and history rows) address the canonical member. A bare-name `getMetaItem` with no canonical stored row answers the code layer's canonical member, ahead of the registry's tier-first bare read. ## Pins (the ruling's, on real code) `packages/objectql/src/environment-overlay-email-locale-restart.test.ts` (real `ObjectQL` + real protocol, `sys_metadata` registered from `SysMetadataObject`, a restart is a new engine and protocol over the same stores): - two locales saved, then restart → each serves its own wording (the list the send path reads), and a re-save updates its own row; - CONTROL: a single-locale save is unchanged (one row, member NULL; the other locale keeps the package wording); - CONTROL: a package-shipped bundle is unchanged across a restart; - a bare-name read answers the canonical member, also when only `zh-CN` is customized; - a by-name reset removes the canonical member's overlay only; - a row stored before the column existed is backfilled at boot from its body, and that locale's next save updates it. `packages/metadata-protocol/src/migrations/overlay-index.test.ts` (real SQLite, `node:sqlite`): - the four-part index refuses a second locale (the defect at the index level); - after the run the key is replaced, so two members coexist and a duplicate `(type, name, member)` is refused `UNIQUE constraint failed`; - the backfill writes exactly the non-canonical bodies, and every other type's rows are byte-identical (all columns) before and after the backfill and the index rebuild. `packages/metadata-protocol/src/sys-metadata-repository.bundle-member.test.ts`: the repository by member. It covers two rows, canonical defaults for read, delete, history and promote, a revert refused across members, and a CONTROL that no name-only payload or `where` carries the column. `packages/qa/dogfood/test/email-template-two-locales-survive-boot.dogfood.test.ts`: the showcase on a real SQLite file through `PUT /meta`, real `sendTemplate` per locale, and a cold boot. ## Evidence Every heavy run went through `scripts/pm/os-verify-lock.sh` (`VERDICT command-exit 0` unless noted). Each exit code was captured before any pipe. - **Patch round 1, final head `f21b6f9d24`** (three commits on `288ef6fb69`; no rebase and no force-push; os-dev-report 6119103318 on #22877): - the `operatorFacingErrorText` caller census (`Test Core (1/6)` red): `overlay-index.ts` keeps its call. It gains a census row and a sentinel pin in `raw-exec-operator-detail-16657.test.ts` (read and write legs, and the sentinel reaches neither `result.detail` nor the error report) (`a9f79a360c`); - `check:i18n` drift (`Type Check · consumer gates` red): the 7 `platform-objects` translation bundles regenerated with `node scripts/check-i18n-bundles.mjs --write`, committed alone (`10f3b686f4`); - the `os migrate duplicates` pre-flight on a pre-column store (`Test Core (3/6)` red on `10f3b686f4`), as the residual bullet below describes (`f21b6f9d24`); - gates at `f21b6f9d24`: 96 commands, all exit 0 (81 derived, reconciled `--ran` 81 of 81, plus the declared wide families and the i18n and census checks). `cli` duplicates files, both tiers: 6 files, 37 passed. `types`: `test:repo` 11 passed, `local` 754 passed. `platform-objects`: 1083 passed. `metadata-protocol` `src/migrations/`: 232 passed. Full packages build: exit 0. - CI on `f21b6f9d24`: every job is green except the two timing gates this PR does not cause: `Test Core (6/6)`'s drift step (#22966) and the `Test Core` aggregator's suite-duration ceilings (#22949: `cloud-connection`, `client`, `service-analytics`, `service-package`, `knowledge-ragflow`). - **Head `288ef6fb69`**, before patch round 1 (this branch merged with `origin/main` `e357a441c6`; the incoming commits overlap none of the files here, and in `metadata-core` they add only `meta-write-capability.ts`): - `turbo run build --filter=...@objectstack/objectql`: exit 0. - `metadata-core` full suite: `17 passed`, `394 passed`. - `metadata-protocol` typecheck: exit 0. - `metadata-protocol` targeted (`overlay-index`, `sys-metadata-repository.bundle-member`, `runtime-index-preflight`, `lookup-artifact-item-call-sites`, `i18n-bundle-list-merge`, `item-name-every-door`): `6 passed`, `72 passed`. - `objectql` targeted (`environment-overlay-email-locale-restart`, `environment-overlay-five-types`, `engine-security-catalog-package-door`, `registry-i18n-bundle-key`): `4 passed`, `65 passed`. - The derived gate union, `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` at `288ef6fb69`: **79 commands, 79 exit 0**. Reconciled with `--ran` (exit codes recorded): `79 derived, 79 run, 0 NOT-MEASURED, 0 UNRUN`. - **Before the merge:** - `metadata-protocol` full suite (`c7f399ba7f`): `Test Files 1 failed | 226 passed | 3 skipped (230)`, `Tests 2 failed | 28183 passed`. The one red file was the `lookupArtifactItem` call-site census, which needed the new site recorded (`59841e1570`); after that it is `4 passed`. - `objectql` full suite, both projects (`59841e1570`): `Test Files 404 passed (404)`, `Tests 7864 passed`. - At `139a2cbc9d`: - `metadata` full suite: `58 passed`, `871 passed`. - Typecheck of `metadata-core`, `metadata-protocol`, `objectql` (with `check:test-typecheck`) and `dogfood`: all exit 0. - The dogfood closure build (`turbo`, 63 tasks): exit 0. - `email-template-two-locales-survive-boot.dogfood.test.ts`: `3 passed (3)`. - `packages/cli` is not touched. The repo-wide eslint sweep is CI's. ## Ablation The member was dropped from the repository's write head: `const member = overlayMemberOfBody(ref.type, body);` became an expression that is `undefined` at runtime, carrying the marker `__ABLATION_22877_MEMBER_DROPPED`. The change was made through `scripts/ablation-replace.mjs` on committed HEAD. The anchor count went 1 → 0, the marker count 0 → 1, and the blob `314afe10d8` → `a28c0832a6`. `metadata-protocol` was rebuilt (objectql resolves it through `dist/`), and `ablation-dist-preflight` confirmed the marker in `dist`. - Mutated: the restart pin file goes `3 failed | 3 passed (6)`. The two-locale restart pin, the bare-name canonical pin and the by-name reset pin are red (the second save updates the first locale's row). Both CONTROLs (single locale, package bundle) and the backfill case stay green, as predicted. - Restored: blob back to `314afe10d8`, `git diff HEAD` empty, `dist` rebuilt, `ablation-dist-preflight --absent` exit 0, and the file goes `6 passed (6)`. ## Deviations and residuals, stated - **Lineage counter stays per `(org, type, name)`**, not per member. `sys_metadata_history` declares `unique (organization_id, type, name, version)`, and a per-member counter would mint colliding versions under it, which would need a second index ceremony on a second table. With one counter per name, a version names exactly one snapshot. `history` and `restoreVersion` carry the member: history filters to one member, and a revert to another member's version answers `404 VERSION_NOT_FOUND`. - **History rows carry no member column.** A history row's member is read off its own body (and a tombstone's off the row it removed, by `previous_checksum`) through the same `overlayMemberOfBody`. No second table and no history migration. - **Registry heal on a by-name reset** (`removeRuntimeShadow`) still sweeps the bundle's whole hydrated overlay group in the registry: the other member's REGISTRY entry drops until the next boot. Its row stays and is still served: the list and the send path read the store, as the reset pin shows. Member-aware registry heal belongs with stage 2's member-addressed DELETE. - **Package-batch doors** (`publishPackageDrafts`, `discardPackageDrafts`, `revertCommit`, the cluster mutation fan-out) address the canonical member. Before this PR two members' drafts could not coexist, so nothing they did before changes. A non-canonical member's draft is published by a save naming its locale, or by stage 2's selector. - **`os migrate duplicates` on a store this version has not booted yet** (a `sys_metadata` table without the member column): the overlay pre-flight first reads whether the column is present. When it is not, it lists duplicates with every row read as the empty member, so a live duplicate is still reported `blocked` and is never reported `unreadable`. Both statements live in `overlay-index.ts`. The choice between them is made in `runtime-index-preflight.ts`, which reports the statement that ran as the entry's `duplicateSql`. The report's keys are unchanged. On such a store, an `email_template` pair whose bodies name different locales can be reported and then not block once the serving boot's backfill has run. That errs in the strict direction. (Patch round 1, `f21b6f9d24`. This fixes the `packages/cli` duplicates integration red on `10f3b686f4`.) - **A driver without partial indexes** (the ceremony's fallback arm, MySQL): the rebuild is refused there, as before, and is reported in the server log at the first metadata write. A store that booted before keeps its four-column declared UNIQUE under its name, because `syncDeclaredIndexes` skips by name. That index has never constrained an environment-wide overlay row: the metadata door stores no organization (ADR-0131 D6), and a UNIQUE key does not collide NULLs, in either the direct index or driver-sql's hash-shadow form. So two languages of one template coexist there too, and no save is refused. This was measured on SQLite (two members with `organization_id` NULL are both accepted; CONTROL: a non-NULL organization refuses the second member). It is not measured on a live MySQL. `os migrate plan` lists the declaration change as a destructive `recreate_index`, applied with `os migrate apply --allow-destructive`. A fresh store gets the five-column declaration. (Corrected after patch round 2: the earlier wording here said such a save is refused, which the dev's measurement 6119963033 falsified.) ## Acceptance notes - **i18n bundles:** the regenerated `zh-CN`, `ja-JP` and `es-ES` bundles carry the English source text for the new `bundle_member` keys, which still need translating. `check:i18n-coverage` stays green. - **Caller census:** `packages/types/src/driver-error-classification.callers.test.ts` (a `domain:cli` file) gains one row for `overlay-index.ts`, the edit class the seat named. The sentinel pin sits in `metadata-protocol`. - **Surface:** `metadata-protocol/src/migrations/runtime-index-preflight.ts` (and its test) is outside the claim's listed files but inside its package and directory. It is the module that reads `overlay-index.ts`'s statements. The edit is one function and a private table, and the seat accepted it as an in-lane deviation. - `packages/drivers/driver-sql/src/sql-driver-overlay-index-drift.test.ts` keeps its own four-part fixture. It tests the drift detector's reading of a `COALESCE` key part and imports nothing changed here, so it stays a valid detector fixture. - Two `objectql` harnesses (`environment-overlay-five-types.test.ts`, `engine-security-catalog-package-door.test.ts`) hand-declare `sys_metadata`. They gain the `bundle_member` field the platform object declares, because the engine refuses an undeclared field in a `where`. - `protocol.lookup-artifact-item-call-sites.test.ts` records the new `lookupArtifactItem` site with its disposition. `scripts/engine-double-contract.pinned.json` records the new test file's pinned doubles (`--write`, additions only). --- _Generated by [Claude Code](https://claude.ai/code/session_01ADzJtzYTLUfgrRZHxkagkX)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 6a7eca6 commit 5677add

24 files changed

Lines changed: 1755 additions & 70 deletions
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
---
2+
'@objectstack/metadata-protocol': patch
3+
'@objectstack/metadata-core': patch
4+
---
5+
6+
fix(metadata-protocol): two locales of one email template saved through the metadata door both survive a restart — an overlay row is keyed by its bundle member (#22877)
7+
8+
Clause-②: no
9+
10+
**What an administrator sees.** Saving the same email template in two languages through the metadata door (`PUT /api/v1/meta/email_template/:name`, the Studio editor's save) keeps both. Before, the store kept one overlay row per template name, so the second language's save replaced the first language's row: both read back until the next restart, and then the first language mailed the package wording again. Each language now keeps its own wording across restarts, and each send in that language uses it.
11+
12+
**What changed in the store.**
13+
14+
- `sys_metadata` gains one nullable column, `bundle_member`. It holds the member of a metadata type whose identity is a pair — today `email_template`, identified by `(name, locale)`. NULL is the canonical member (`en-US`, the locale a send falls back to). Every other row stays NULL, so rows of every other type, and an `en-US` template row, are byte-identical to before.
15+
- The value is taken from the saved body's `locale`; it is never authored. A body that declares no `locale` is the `en-US` member.
16+
- The overlay uniqueness index becomes `(type, name, organization_id, COALESCE(package_id, ''), COALESCE(bundle_member, ''))`, for active rows and for draft rows, and the declared index in `metadata-core` carries the column too. A second row for the same template, language, package and state is refused by the index.
17+
18+
**What a read or a door addresses.** A save addresses the language its body names. A read, delete, publish, history or revert by name that names no language addresses the `en-US` member: `GET /meta/email_template/:name` serves the `en-US` wording (the stored one when it was customized, else the package's), where before it served whichever row the name had. The layered list and the send path serve every language as before. A non-`en-US` language's own customization is reached through the list, and is replaced by a save that names its `locale`; a by-name reset (`DELETE`) removes the `en-US` customization only. The lineage counter stays one per template name, so a version number names one saved snapshot, and a revert by name restores only a version of the `en-US` member. Resetting, publishing or reading a non-`en-US` language by name needs the by-name member selector, which arrives with stage 2 of #22877; until then those doors address the `en-US` member.
19+
20+
**What happens on the first boot after upgrading.** Nothing to configure.
21+
22+
1. The schema sync adds the `bundle_member` column.
23+
2. Boot hydration gives every `email_template` row stored before the column existed the member its body declares, so a row an earlier save left holding a second language's wording becomes that language's row, not the `en-US` one. Rows of every other type are not read or written.
24+
3. The first metadata write rebuilds the overlay index with the new key, through the existing probe-then-replace order. Adding a key part only relaxes uniqueness, so no rows a store already holds can block the rebuild; a store that was already blocked is reported as before, with the duplicate-listing query and `os migrate duplicates`.
25+
26+
**On a database without partial indexes (MySQL).** The rebuild above is refused there, as before, and reported in the server log at the first metadata write. A store that booted an earlier version keeps the four-column declared index under `idx_sys_metadata_overlay_active`, because the schema sync skips an index whose name already exists. That index has never constrained an environment-wide overlay row: those rows store no organization, and a UNIQUE key does not collide NULLs. So two languages of one template coexist there too, and no save is refused. `os migrate plan` lists the declaration change on that index as a destructive `recreate_index`. To bring the index to the five-column declaration, run `os migrate apply --allow-destructive`. A fresh store gets the five-column declaration.
27+
28+
`os migrate duplicates` boots read-only and adds no column. Run against a store this version has not booted yet, it lists the two overlay indexes with every row read as the empty member — the value the column will hold — so a duplicate that will block the rebuild is still reported, and the entry's `duplicateSql` is the member-less statement that ran there.

‎packages/metadata-core/src/objects/sys-metadata.object.ts‎

Lines changed: 33 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,27 @@ export const SysMetadataObject = ObjectSchema.create({
9393
defaultValue: 'platform',
9494
}),
9595

96+
/**
97+
* The bundle member this row overlays, for a metadata type whose identity
98+
* is a pair (`ITEM_KEY_DISCRIMINATORS` in `../item-key-discriminators.ts`,
99+
* today `email_template`'s `(name, locale)`).
100+
*
101+
* NULL is the type's CANONICAL member, which is also every row of a type
102+
* whose identity is `name` alone — so those rows never carry a value and
103+
* stay byte-identical. A non-canonical member stores the value the body
104+
* declares (`itemDiscriminator`), so two locales of one template are two
105+
* rows. Written by the overlay repository from the body, never authored;
106+
* part of the overlay uniqueness key below and of the runtime index
107+
* (`metadata-protocol`'s `migrations/overlay-index.ts`).
108+
*/
109+
bundle_member: Field.text({
110+
label: 'Bundle Member',
111+
required: false,
112+
maxLength: 100,
113+
description:
114+
'Bundle member (e.g. the locale of an email template) this overlay row customizes. Null = the canonical member.',
115+
}),
116+
96117
/** JSON payload — the actual metadata configuration */
97118
metadata: Field.textarea({
98119
label: 'Metadata',
@@ -200,11 +221,13 @@ export const SysMetadataObject = ObjectSchema.create({
200221

201222
indexes: [
202223
// ADR-0005 (revised 2026-05) + ADR-0048: overlay uniqueness is scoped by
203-
// (type, name, organization_id, package_id). `package_id` is part of the
204-
// discriminator so two installed packages shipping the same `type`/`name`
205-
// each get their OWN customization row (a package-less / global overlay
206-
// uses NULL). environment_id is deprecated and not part of the
207-
// discriminator.
224+
// (type, name, organization_id, package_id, bundle_member). `package_id` is
225+
// part of the discriminator so two installed packages shipping the same
226+
// `type`/`name` each get their OWN customization row (a package-less /
227+
// global overlay uses NULL). `bundle_member` is the bundled type's member
228+
// (#22877): two locales of one email template are two overlay rows; NULL
229+
// is the canonical member and every undiscriminated type's row.
230+
// environment_id is deprecated and not part of the discriminator.
208231
//
209232
// ⚠️ The active-row restriction is NOT declared here, and never was in any
210233
// effective sense: this entry carried `partial: "state = 'active'"` until
@@ -217,15 +240,16 @@ export const SysMetadataObject = ObjectSchema.create({
217240
// What actually delivers active-row scoping is the runtime layer:
218241
// `protocol.ts ensureOverlayIndex` issues a DROP-then-CREATE migration for
219242
// `CREATE UNIQUE INDEX … ON sys_metadata (type, name, organization_id,
220-
// COALESCE(package_id,'')) WHERE state = 'active'` — which additionally
221-
// fixes NULL-distinctness for package-less rows, something this
222-
// declaration could not express either. This entry is therefore the
243+
// COALESCE(package_id,''), COALESCE(bundle_member,'')) WHERE state =
244+
// 'active'` — which additionally fixes NULL-distinctness for package-less
245+
// rows and canonical members, something this declaration could not
246+
// express either. This entry is therefore the
223247
// coarser fallback that a driver without that runtime migration gets: it
224248
// still prevents duplicate ACTIVE overlays, at the cost of also colliding
225249
// with archived/reset rows.
226250
{
227251
name: 'idx_sys_metadata_overlay_active',
228-
fields: ['type', 'name', 'organization_id', 'package_id'],
252+
fields: ['type', 'name', 'organization_id', 'package_id', 'bundle_member'],
229253
unique: 'global',
230254
},
231255
{ name: 'idx_sys_metadata_org_type', fields: ['organization_id', 'type'] },

0 commit comments

Comments
 (0)