diff --git a/.changeset/15212-8b-single-platform-insert-stamped.md b/.changeset/15212-8b-single-platform-insert-stamped.md index 482d7ecf440..a04dc9de9c6 100644 --- a/.changeset/15212-8b-single-platform-insert-stamped.md +++ b/.changeset/15212-8b-single-platform-insert-stamped.md @@ -12,6 +12,6 @@ Clause-②: no (narrowing) - **Before.** A system-context insert (`isSystem`, no `tenantId`) into a `sys_` / `cloud_` / `ai_` object that carries an organization column, but that the tenancy inventory had not admitted, skipped the organization rule entirely, under every posture. The row landed with `organization_id` NULL. Thirty-five platform objects in this repository were in that population, among them `sys_business_unit`, `sys_business_unit_member`, `sys_member`, `sys_team`, `sys_user_permission_set`, `sys_user_position`, `sys_sharing_rule`, `sys_share_link`, `sys_comment`, `sys_attachment`, `sys_activity`, `sys_email`, `sys_inbox_message`, `sys_secret`, `sys_setting`, `sys_setting_audit`, `sys_webhook` and `sys_flow_credential`. - **Now, under `single`.** That insert goes through the same decision every application object already gets. With exactly one organization (the Default Organization), the row is stamped with it. With none, or with several, the insert is refused with `ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED` (status 500), and the message names the missing organization. Nothing is written. -- **Unchanged.** Under a walled posture (`group` / `isolated`) the per-object exclusion stands as before; the walled half of this change ships in a later 18.x release. A write that carries an organization, on the execution context or on the record, is never re-stamped or refused. Objects with no organization column, objects declaring `tenancy: { enabled: false }`, and federated objects are outside the rule. Updates are untouched. The engine's diagnostics-only tenant-audit mute is unchanged. +- **Unchanged.** Under a walled posture (`group` / `isolated`) the per-object exclusion stands as before; the walled half of this change is the companion `@objectstack/objectql` changeset on system inserts under a walled posture. A write that carries an organization, on the execution context or on the record, is never re-stamped or refused. Objects with no organization column, objects declaring `tenancy: { enabled: false }`, and federated objects are outside the rule. Updates are untouched. The engine's diagnostics-only tenant-audit mute is unchanged. **The remedy.** If a refusal names your insert, carry the organization on the write: `ql.insert(object, { ...row, organization_id: orgId }, { context: { isSystem: true } })`, or `{ context: { isSystem: true, tenantId: orgId } }`. On a fresh `single` deployment the auth plugin creates the Default Organization at boot, so a refusal reading `no-organization` means the write ran before it, or in a composition that registers `sys_organization` but mounts no auth plugin to create the Default Organization. diff --git a/.changeset/15212-8b-walled-platform-insert-refused.md b/.changeset/15212-8b-walled-platform-insert-refused.md index 592df71f840..ff9e5806edd 100644 --- a/.changeset/15212-8b-walled-platform-insert-refused.md +++ b/.changeset/15212-8b-walled-platform-insert-refused.md @@ -6,12 +6,12 @@ feat(objectql)!: under a walled posture (`group` / `isolated`), a system insert Clause-②: yes (narrowing) - + -**BREAKING** accept-set narrowing, shipped as `minor` under the repo's launch-window convention for breaking changes (ADR-0131 D9, D11). This is the walled half of the change whose `single` half shipped in 18.0. +**BREAKING** accept-set narrowing, shipped as `minor` under the repo's launch-window convention for breaking changes (ADR-0131 D9, D11). This is the walled half of the change whose `single` half is the companion `@objectstack/objectql` changeset on system inserts under `single`. - **Before.** Under `group` or `isolated`, a system-context insert (`isSystem`, no `tenantId`) into a `sys_` / `cloud_` / `ai_` object that carries an organization column, but that the tenancy inventory had not admitted, skipped the organization rule. The row landed with `organization_id` NULL, a row no organization owns and the wall hides from every organization. -- **Now.** That insert goes through the same decision every application object already gets under a wall: no organization is derivable, so it is refused with `ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED` (status 500, reason `walled-posture`), and nothing is written. Apart from the two tables named below, the posture no longer changes which objects the rule covers. -- **Unchanged.** A write that carries an organization, on the execution context or on the record, is never re-stamped or refused. Objects with no organization column (`systemFields: { tenant: false }`), objects declaring `tenancy: { enabled: false }`, and federated objects are outside the rule. `sys_secret` and `sys_flow_credential` keep their organization-less system inserts under a wall, as before: a global-scope encrypted setting, a datasource credential and a flow credential are deployment-level, and both tables drop their organization column in a later 18.x release, which retires this bridge (`WALLED_ORGANIZATION_LESS_BRIDGE`). Updates are untouched. The engine's diagnostics-only tenant-audit mute still reads the inventory. +- **Now.** That insert goes through the same decision every application object already gets under a wall: no organization is derivable, so it is refused with `ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED` (status 500, reason `walled-posture`), and nothing is written. The posture no longer changes which objects the rule covers. +- **Unchanged.** A write that carries an organization, on the execution context or on the record, is never re-stamped or refused. Objects with no organization column (`systemFields: { tenant: false }`), objects declaring `tenancy: { enabled: false }`, and federated objects are outside the rule. `sys_secret` and `sys_flow_credential` are among them: a global-scope encrypted setting, a datasource credential and a flow credential are deployment-level, so both declare `systemFields: { tenant: false }` and their organization-less system inserts land under a wall. Updates are untouched. The engine's diagnostics-only tenant-audit mute still reads the inventory. **The remedy.** If a refusal names your insert, carry the organization on the write: `ql.insert(object, { ...row, organization_id: orgId }, { context: { isSystem: true } })`, or `{ context: { isSystem: true, tenantId: orgId } }`. If the row is deployment-level by design, the object should declare no organization column: `systemFields: { tenant: false }`. diff --git a/.changeset/15212-walled-posture-boots.md b/.changeset/15212-walled-posture-boots.md index 1d3c0737e5d..5db963c0385 100644 --- a/.changeset/15212-walled-posture-boots.md +++ b/.changeset/15212-walled-posture-boots.md @@ -6,8 +6,8 @@ feat(cli): `os serve` (and so `os dev` and `os start`) boots a walled tenancy po Clause-②: yes (widening) -18.0 refused a walled posture at boot, because the walled half of the platform-object organization rule had not shipped. It ships in this release (see the `@objectstack/objectql` changeset on walled system inserts), so the refusal is removed. +The 18.0 pre-releases refused a walled posture at boot, because the walled half of the platform-object organization rule had not shipped. It ships in this release (see the `@objectstack/objectql` changeset on walled system inserts), so the refusal is removed. - **Before.** With `OS_TENANCY_POSTURE=group` or `isolated`, or `OS_MULTI_ORG_ENABLED=true`, `os serve` exited `1` with a FATAL `NOT_IMPLEMENTED` before it read a config. `os doctor` printed a `Tenancy posture` warning saying so, and the unrecognised-posture fix list marked both walled values as refused in 18.0. -- **Now.** A walled posture boots like `single`, under the same prerequisites as before 18.0: the enterprise `@objectstack/organizations` runtime, and `OS_PLATFORM_OWNER_EMAIL`. `os doctor` reports no row for a valid posture, and the fix list offers `group` and `isolated` without the 18.0 note. -- **Unchanged.** An unrecognised posture is still refused at boot. Upgrading an existing database still waits for the upgrade ceremony, which still refuses in this release. +- **Now.** A walled posture boots like `single`, under the same prerequisites as before the 18.0 pre-releases: the enterprise `@objectstack/organizations` runtime, and `OS_PLATFORM_OWNER_EMAIL`. `os doctor` reports no row for a valid posture, and the fix list offers `group` and `isolated` without the refusal note. +- **Unchanged.** An unrecognised posture is still refused at boot. Upgrading an existing database is the ownership ceremony's: `os migrate organization-ownership --out PLAN`, then `--apply --plan-file PLAN --backup-taken`. diff --git a/.changeset/15214-c11-fresh-install-messages.md b/.changeset/15214-c11-fresh-install-messages.md deleted file mode 100644 index d3e3a6b38ee..00000000000 --- a/.changeset/15214-c11-fresh-install-messages.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -'@objectstack/cli': patch -'@objectstack/runtime': patch ---- - -fix(cli, runtime): the 18.0 operator messages say that 18.0 supports fresh `single` installs only - -Clause-②: no - -- **The v18 boot refusal on a v17-shaped database** now says first that 18.0 does not upgrade a v17 database: keep it on a 17.x release until the 18.x release that carries the upgrade path. Before, it named only `os migrate organization-ownership`, which refuses in 18.0 as well, so the operator reached the answer in two hops. -- **The fix list for an unrecognized `OS_TENANCY_POSTURE`** (printed by `os serve` and `os doctor`) still lists `group` and `isolated`, and now says beside each that 18.0 refuses it at boot. -- **`os doctor` reports a walled posture.** Under `group` or `isolated` it prints a `Tenancy posture` warning: 18.0 supports `single` only, so the next `os serve` (and so `os dev` and `os start`) refuses the posture at boot. The row is a warning, so doctor's exit code does not change. Before, doctor printed nothing about a valid posture. - -Nothing else changes. The refusals themselves, their codes and their exit codes are as before. diff --git a/.changeset/24529-declared-webhooks-from-registry.md b/.changeset/24529-declared-webhooks-from-registry.md index 4fad5d6836d..0162a7c1e4b 100644 --- a/.changeset/24529-declared-webhooks-from-registry.md +++ b/.changeset/24529-declared-webhooks-from-registry.md @@ -27,4 +27,4 @@ On a database an 18.0 install created (expand first — nothing is moved or dele - A seeded row whose declaration was removed keeps delivering, as it did on 18.0. - Redelivering an old delivery of a pristine seeded row is judged by the declaration that now answers for it, so a webhook switched off through the ledger is not re-sent. -Deleting the seeded rows, and re-pointing their `sys_http_delivery` history at the declaration, is a step of the ownership migration in a later 18.x release. +Deleting the seeded rows, and re-pointing their `sys_http_delivery` history at the declaration, is the ownership ceremony's `declared-webhooks` step (`os migrate organization-ownership`). diff --git a/.changeset/24567-drop-gate-refusal-18-0.md b/.changeset/24567-drop-gate-refusal-18-0.md index e28c7313491..6da5a00994c 100644 --- a/.changeset/24567-drop-gate-refusal-18-0.md +++ b/.changeset/24567-drop-gate-refusal-18-0.md @@ -2,12 +2,10 @@ '@objectstack/cli': patch --- -fix(cli): `os migrate apply --allow-destructive`'s organization-ownership refusal says what 18.0 supports for a v17 database, and names no step of the command 18.0 refuses +fix(cli): `os migrate apply --allow-destructive`'s organization-ownership refusal states the ceremony's record in its `--json` `detail` Clause-②: no -- **`os migrate apply --allow-destructive` on a v17-shaped database.** When the run would drop `organization_id` from a table the organization-ownership ceremony owns, and the ceremony has not verified on the database, the refusal now says that 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path, whose ceremony drops these columns itself. The v18 boot refusal and `os migrate organization-ownership`'s own refusal say the same. Before, it told the operator to run `os migrate organization-ownership --out PLAN` and then `--apply --plan-file PLAN`, which 18.0 refuses on every database (`NOT_IMPLEMENTED`). The `--json` refusal's `detail` no longer names that command either: it reads "No run of the organization-ownership ceremony is recorded on this database.", or names the last unverified run with its date and blocking count. -- **`os migrate organization-ownership --help`** shows what the command answers in 18.0: exit `1` with `NOT_IMPLEMENTED`, nothing read or written, and the same refusal as JSON. Before, it showed six plan and apply examples that 18.0 refuses. -- **`os dev` on a v17-shaped development database.** When it does not upgrade the database in place (the ceremony's plan is refused, or the database is not a local SQLite file), it now ends with the same position: keep a v17 development database on a 17.x release until the 18.x release that carries the upgrade path, and upgrade it there. Before, it told the developer to run `os migrate organization-ownership --dev` and then the same command with `--apply`. +- **`os migrate apply --allow-destructive --json` on a database the organization-ownership ceremony has not verified.** When the run would drop `organization_id` from a table the ceremony owns, the refusal's `detail` reads "No run of the organization-ownership ceremony is recorded on this database.", or names the last unverified run with its date and blocking count. The human message names the ceremony's plan and apply, which drop these columns themselves. -Nothing else changes. The refusals still hold back the whole run, and their message code (`refused_organization_ownership_unmigrated`), `tables`, `marker`, exit code `1` and every flag are as before. +Nothing else changes. The refusal still holds back the whole run, and its message code (`refused_organization_ownership_unmigrated`), `tables`, `marker`, exit code `1` and every flag are as before. diff --git a/.changeset/24581-organization-ownership-ceremony-open.md b/.changeset/24581-organization-ownership-ceremony-open.md new file mode 100644 index 00000000000..16f888daa70 --- /dev/null +++ b/.changeset/24581-organization-ownership-ceremony-open.md @@ -0,0 +1,30 @@ +--- +'@objectstack/cli': minor +'@objectstack/runtime': minor +'@objectstack/spec': minor +'@objectstack/driver-sql': minor +'@objectstack/driver-turso': minor +'@objectstack/objectql': minor +'@objectstack/plugin-webhooks': minor +'@objectstack/platform-objects': minor +'@objectstack/service-automation': minor +'@objectstack/service-settings': minor +--- + +feat(cli,runtime)!: `os migrate organization-ownership` upgrades a database 17.x or an 18.0 pre-release created; a server serves only a database it created or one the ceremony certified + +Clause-②: yes (narrowing) + + + +**BREAKING** for a database with no birth marker — one 17.x or an 18.0 pre-release (`18.0.0-next.*`) created — shipped as `minor` under the repo's launch-window convention for breaking changes (ADR-0131 D10). + +- **The ceremony is open.** `os migrate organization-ownership` runs again (the 18.0 pre-releases refused it with `NOT_IMPLEMENTED`): a bare run writes the plan, and `--apply --plan-file PLAN --backup-taken` applies that reviewed plan. Its new steps carry what an 18.0 pre-release database still owes, each selected by the rows' shape, never by a NULL organization: + - `sibling-moves`: the global settings-audit rows move to `sys_platform_setting_audit`, and identity mail (reset, verification, account invitation, change-email notice, magic link) to `sys_identity_email`, id for id. Their queued mail jobs and their activity rows follow them. The organization invitation stays in `sys_email`. + - `declared-webhooks`: each `sys_webhook` copy a package's seeder wrote under a name a package still declares is settled. A pristine copy is deleted once its deliveries name the declaration (`declared:NAME`), with the cipher it referenced. A copy customized only in its switch becomes the declaration's activation row. Any other edit becomes the administrator's own webhook under a new name (`NAME_custom`), and the declaration is switched off. + - `declared-rule-copies`: each stored sharing rule under a declared rule's name becomes a clone under a new name, its grants kept, and the declaration is switched off. + - `not-null`: after the column drops, every table the post-step plan clears takes `organization_id` NOT NULL. The ceremony owns that nullability, so the schema differ no longer reports it as `relax_not_null` drift. +- **The boot guard decides on a marker.** The first boot that creates a datastore from empty writes a birth marker (`sys_migration` row `adr-0131-organization-ownership`, `details.birth.shapeGeneration`). Every CLI data boot writes it too, so a fresh install whose first command is `os migrate apply` or `os seed` still boots. On the Turso remote face (`libsql://`), `getSchemaSyncStats` now answers what its schema doors created and found (it answered `{ created: 0, existing: 0 }` before), so a fresh remote database is observed as one and gets its marker. The ceremony's verified marker carries the same `shapeGeneration` (now `1`, `ORGANIZATION_OWNERSHIP_SHAPE_GENERATION` in `@objectstack/spec/system`). A server refuses a database with no marker (one 17.x or an 18.0 pre-release created), one certified at an older generation, or an uncertified one. The refusal names the command and no longer says to stay on 17.x. The v17-shape probe also reads `tenancy: { enabled: false }` and `systemFields: false`. +- **`sys_secret` and `sys_flow_credential` are deployment-level** (`systemFields: { tenant: false }`). Their `organization_id` is dropped by the ceremony's column-drops step. The settings service no longer writes one, and `SettingsSecretStore.insert` no longer takes it. With no organization column, their organization-less system inserts land under a walled posture on the engine's own rule, so `@objectstack/objectql` no longer exports `WALLED_ORGANIZATION_LESS_BRIDGE` or `isWalledOrganizationLessBridgeObject`, and the system-write refusal's remedy names `systemFields: { tenant: false }`. + +**What an operator does.** Fresh installs are supported under every tenancy posture and need nothing. Before serving a database 17.x or an 18.0 pre-release created with this release: start this release against it once (the boot is refused, and its schema sync has already created the tables this release adds, which the ceremony moves rows into), take a backup, run `os migrate organization-ownership --out plan.json`, review the plan, then run `os migrate organization-ownership --apply --plan-file plan.json --backup-taken`, and start the server. `os dev` does this for a local SQLite development database. On a walled database, a customized copy of a declared webhook or sharing rule blocks its step, naming the row: rename it to a name no package declares, or delete it, then re-run. diff --git a/.changeset/24637-spec-upgrade-entries-fresh-install-only.md b/.changeset/24637-spec-upgrade-entries-fresh-install-only.md deleted file mode 100644 index 6107e9ce91e..00000000000 --- a/.changeset/24637-spec-upgrade-entries-fresh-install-only.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -'@objectstack/spec': patch ---- - -fix(spec): the 18.0 upgrade-guide entries that name `os migrate organization-ownership` say first that 18.0 supports fresh installs only - -Clause-②: no - -In 18.0, `os migrate organization-ownership` refuses plan and apply alike on every database (exit 1, `NOT_IMPLEMENTED`, nothing read or written). Seven ADR-0087 semantic entries still told an operator to run `--out PLAN`, then `--apply --plan-file PLAN`, as if 18.0 upgraded a v17 database. Each now says, in the same words as the CLI and the boot refusal: 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. The plan and apply steps stay, marked as the 18.x release's ceremony. - -- `sys-metadata-organization-column-retired`, `sys-metadata-audit-organization-column-retired`, `sys-metadata-commit-organization-column-retired`, `sys-metadata-history-organization-column-retired`, `ups-permission-set-id-column-retired`: the replacement and the acceptance criteria. -- `sharing-rule-declared-mirror-retired`: the replacement and the acceptance criteria. -- `cli-organization-ownership-apply-plan-file-required`: the replacement now says that 18.0 refuses this invocation, as it refuses every invocation of the command, and keeps the plan-file rule as the 18.x release's. - -Only the text changes. Every entry keeps its `id`, its surface and its step. diff --git a/.changeset/24651-boot-refusal-fresh-installs-only.md b/.changeset/24651-boot-refusal-fresh-installs-only.md deleted file mode 100644 index a3a38f2f744..00000000000 --- a/.changeset/24651-boot-refusal-fresh-installs-only.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -'@objectstack/runtime': patch ---- - -fix(runtime): the organization-ownership boot refusals say what 18.0 supports for a v17 database and name no step of `os migrate organization-ownership`, which 18.0 refuses on every database - -Clause-②: no (refusal text only: the refusal code `ORGANIZATION_OWNERSHIP_MIGRATION_REQUIRED`, the refused tables and the boot outcome do not move) - -In 18.0, `os migrate organization-ownership` refuses plan and apply on every database (`NOT_IMPLEMENTED`): 18.0 supports fresh installs only. Three branches of the v18 boot refusal still told the operator to run that command's plan and its `--apply --plan-file PLAN --backup-taken`, so following them led to a second refusal. - -Each branch now closes on the sentence `os migrate apply`'s drop gate and `os dev` already use: "18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path, and upgrade it there." - -- **A table still in the v17 shape** (an object that declares no organization column, over a table that still carries `organization_id`). This branch already said that 18.0 does not upgrade a v17 database. It no longer follows that with the command's plan and apply steps. -- **No table left in the v17 shape, and the ceremony's marker is not verified.** The refusal no longer prescribes a re-plan and a re-apply, which 18.0 cannot run. -- **No table left in the v17 shape, and the marker could not be read.** The refusal now says to fix the read and start again: a database no ceremony ran on boots once the read answers. It no longer prescribes a re-plan and a re-apply. -- **The column read itself failed.** The refusal still says to fix the read. It no longer offers the command as the alternative. - -A development boot (`OrganizationOwnershipBootGuardPlugin({ dev: true })`, as `os serve --dev` composes it) says "a v17 development database" in place of "a v17 database". Before, it named the command with `--dev`. diff --git a/.changeset/24652-report-v18-upgrade-text.md b/.changeset/24652-report-v18-upgrade-text.md deleted file mode 100644 index 49839d91ac2..00000000000 --- a/.changeset/24652-report-v18-upgrade-text.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -'@objectstack/plugin-security': patch ---- - -fix(plugin-security): the boot report's unnamed-grant line says what 18.0 supports for a v17 database, not a command 18.0 refuses - -Clause-②: no - -- **The error line for `sys_user_permission_set` grants whose `permission_set` is empty** used to end with a fix: plan with `os migrate organization-ownership --out plan.json`, then run `os migrate organization-ownership --apply --backup-taken --plan-file plan.json`, restart, with a warning that granting the set again by name before it runs leaves a second grant for the ceremony to report. 18.0 refuses that command on every database, so the operator was sent to a refusal. The line now ends with the sentence the drop-gate refusal of `os migrate apply --allow-destructive` carries: "18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path." -- **Unchanged:** the count, the "permission_set is empty" fact, the ADR-0131 D4 reason ("each confers nothing"), "Nothing was changed.", the structured meta, the log level and the report's result. Such a grant still confers nothing and is still listed. -- **Removed from the line:** the two command steps and the restart step, and the advice about granting the set again by name. They described the 18.x upgrade ceremony, which 18.0 does not carry. diff --git a/.changeset/24709-platform-admin-withheld-18-text.md b/.changeset/24709-platform-admin-withheld-18-text.md deleted file mode 100644 index 2dc210351ba..00000000000 --- a/.changeset/24709-platform-admin-withheld-18-text.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -'@objectstack/plugin-security': patch ---- - -fix(plugin-security): the platform-admin promotion WITHHELD warning says what 18.0 supports for a v17 database, not a migration 18.0 refuses - -Clause-②: no - -- **The boot warning "[security] platform-admin promotion WITHHELD: an unscoped grant exists whose permission-set name (sys_user_permission_set.permission_set) was never written"** used to end with a way on: "The organization-ownership migration's id-to-name rewrite (ADR-0131 D10) names it; the next pass after that reads it by name." 18.0 refuses that migration on every database, so the operator was sent to a refusal. The warning now ends with the sentence the boot refusal and the drop-gate refusal of `os migrate apply --allow-destructive` carry: "18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path." -- **Unchanged:** the withholding itself (an unscoped grant that names no set is still read as an administrator who may already exist, and no second platform-admin grant is minted), the facts the warning states (which column was never written, that the grant predates it, that the set it holds can no longer be read), the log level and the `info` fallback when the logger has no `warn`, and the returned `admin_grant_unnamed` result. -- **Removed from the warning:** the pointer to the organization-ownership migration's rewrite and the promise that the next pass reads the grant by name afterwards. Both described the 18.x upgrade path, which 18.0 does not carry. diff --git a/.changeset/24710-ownership-refusal-text-fresh-installs-only.md b/.changeset/24710-ownership-refusal-text-fresh-installs-only.md deleted file mode 100644 index a8afad52c74..00000000000 --- a/.changeset/24710-ownership-refusal-text-fresh-installs-only.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -'@objectstack/metadata-core': patch -'@objectstack/metadata-protocol': patch ---- - -fix(metadata-core, metadata-protocol): two refusals say what 18.0 supports for a v17 database, not `os migrate organization-ownership`, which 18.0 refuses on every database - -Clause-②: no (text only: no refusal code, outcome, accept set or served shape moves) - -In 18.0, `os migrate organization-ownership` refuses plan and apply on every database (`NOT_IMPLEMENTED`): 18.0 supports fresh installs only. Two refusals still told the operator to run it, so following them led to a second refusal. - -- **`LEGACY_ORGANIZATION_ROWS_UNREADABLE`** (`LegacyOrganizationRowsUnreadable`, `@objectstack/metadata-core`). It used to close with "Run `os migrate organization-ownership` on this database first." This read runs on every SQL database, a fresh 18.0 one included, and several of its refusals are the driver's (a dialect it cannot name, no raw-SQL seam, a column list that is not a list or comes back empty). For those, the command was never the remedy. The message now closes: "A fresh 18.0 install holds no legacy organization rows: there, the cause is the one named above, and the fix is in the SQL driver or the caller that met it. A database that does hold them is a v17 one. 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path." -- **The seed refusal for a natural key held by a row with no organization** (`SeedLoaderService`, `@objectstack/metadata-protocol`). It used to close with "Attribute the unowned rows first (`os migrate organization-ownership`, then the same command with --apply), then re-run this seed". Even before 18.0, that bare `--apply` was refused, since the command requires `--plan-file`. The message now closes: "A row with no organization predates organization stamping, so it is a v17 database's. 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path." -- **Unchanged:** both refusal codes, "Nothing was listed or written.", the seed refusal's opening sentences (what was refused and why), its `errored` count, and the dropped record. diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index a6ac802cfea..6b7f4c3f059 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -1081,6 +1081,18 @@ jobs: continue-on-error: true run: pnpm check:watch-hint-literal + # No shipped refusal, upgrade text or pending changeset says the + # organization-ownership ceremony is closed (#24581). The 18.0 + # pre-releases refused the command and taught a family of surfaces to say + # "fresh installs only / keep it on 17.x"; the ceremony is open, and one + # such sentence left anywhere sends an operator away from the command + # that upgrades their database. Tests are out of the population (they + # spell the sentence to pin its absence). Repo-wide text sweep; ~1s. + - name: No shipped text says the ownership ceremony is closed + id: closed-ceremony-text + continue-on-error: true + run: pnpm check:closed-ceremony-text + # ADR-0087 D4's per-release correctness gate (#17080). The REAL run needs # two published tarballs and npm, so it lives in the release lane and # cannot run on a PR; what runs here is its `--self-test`, which is diff --git a/content/docs/automation/webhooks.mdx b/content/docs/automation/webhooks.mdx index 2b0499a3c0d..5b642798588 100644 --- a/content/docs/automation/webhooks.mdx +++ b/content/docs/automation/webhooks.mdx @@ -100,7 +100,7 @@ A webhook reaches the dispatcher one of two ways: - **Authored by an organization** — a `sys_webhook` row (§3.1), switched by its own `active` field. -On a database an 18.0 install created, the rows that release's boot seeder +On a database an 18.0 pre-release (`18.0.0-next.*`) created, the rows that release's boot seeder wrote for declared webhooks (`managed_by: 'package'`) are still there. Until the ownership migration removes them, a pristine one is ignored in favour of its declaration, so the webhook is never delivered twice; one an administrator diff --git a/content/docs/concepts/metadata-lifecycle.mdx b/content/docs/concepts/metadata-lifecycle.mdx index a3f1b64ef78..a562fd41b0d 100644 --- a/content/docs/concepts/metadata-lifecycle.mdx +++ b/content/docs/concepts/metadata-lifecycle.mdx @@ -106,7 +106,7 @@ later ships. See [ADR-0070](https://github.com/objectstack-ai/objectstack/blob/m **Most metadata types must not be overlaid by an environment** — overlaying a packaged item would break the physical schema or the behaviour its package ships. The whitelist lives in **one** place: `MetadataTypeRegistryEntry.allowEnvironmentOverlay` in `packages/spec/src/kernel/metadata-plugin.zod.ts` (spelled `allowOrgOverride` before protocol 18; `GET /api/v1/meta/types` serves it per entry as `allowEnvironmentOverlay`). -> **ADR-0131 D6 — the per-organization overlay axis is retired.** The `/meta` doors (REST and the runtime dispatcher) carry no organization into a metadata write or read: a Studio or `PUT /api/v1/meta/...` write of the five ✅ types below lands **environment-wide** (since ADR-0131 D7 the `sys_metadata` family declares no organization column) and is served to every organization, and an organization admin holding only the retired `manage_org_presentation` capability is refused (`403`). The protocol itself refuses every organization-scoped write, and every read — item, list, layers, cached, history, diff, audit, drafts and the commit timeline — is environment → code, whatever organization a caller names. An overlay row an earlier release stored under an organization — under the `single` posture, every Studio save of these types, filed under the Default Organization — is served by no read, and on a SQL database (SQLite, PostgreSQL, MySQL) still holding one a v18 `os serve` refuses to start. 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. In that release, the v18 migration ceremony (ADR-0131 C7: `os migrate organization-ownership --out PLAN`, then `--apply --plan-file PLAN`) carries such a row to the environment layer and drops the column; nothing deletes or rewrites it before then. The anonymous form doors read the environment alone too: in that release the ceremony carries every organization-layer public-form withdrawal onto the environment, fail-closed. The table still records which types take an overlay of a shipped item. +> **ADR-0131 D6 — the per-organization overlay axis is retired.** The `/meta` doors (REST and the runtime dispatcher) carry no organization into a metadata write or read: a Studio or `PUT /api/v1/meta/...` write of the five ✅ types below lands **environment-wide** (since ADR-0131 D7 the `sys_metadata` family declares no organization column) and is served to every organization, and an organization admin holding only the retired `manage_org_presentation` capability is refused (`403`). The protocol itself refuses every organization-scoped write, and every read — item, list, layers, cached, history, diff, audit, drafts and the commit timeline — is environment → code, whatever organization a caller names. An overlay row an earlier release stored under an organization — under the `single` posture, every Studio save of these types, filed under the Default Organization — is served by no read, and on a SQL database (SQLite, PostgreSQL, MySQL) still holding one a v18 `os serve` refuses to start until the v18 migration ceremony (ADR-0131 C7: `os migrate organization-ownership --out PLAN`, then `--apply --plan-file PLAN`) has carried such a row to the environment layer and dropped the column; nothing deletes or rewrites it before then. The anonymous form doors read the environment alone too: the ceremony carries every organization-layer public-form withdrawal onto the environment, fail-closed. The table still records which types take an overlay of a shipped item. | Type | `allowEnvironmentOverlay` | Rationale | | :--- | :---: | :--- | diff --git a/content/docs/deployment/cli.mdx b/content/docs/deployment/cli.mdx index 7d2d4adba97..8227b1fcfb2 100644 --- a/content/docs/deployment/cli.mdx +++ b/content/docs/deployment/cli.mdx @@ -196,24 +196,13 @@ os dev --database file:./data/test.db --auth-secret $(openssl rand -hex 32) | `--seed-admin` / `--no-seed-admin` | — | Seed a dev admin (`admin@objectos.ai` / `admin123`) on an empty DB — default on; override with `--admin-email` / `--admin-password` | | `-v, --verbose` | — | Verbose output | -##### A 17.x development database +##### A 17.x or 18.0 pre-release development database is upgraded in place -A v18 server refuses to start on a database still in the v17 -organization-ownership shape until the [ownership ceremony](#os-migrate-organization-ownership) -has run on it. - - -Fresh installs are supported under every tenancy posture; the upgrade path arrives in a later 18.x release. Until then the ceremony refuses, so `os dev` stops at step 2 with its -refusal and changes nothing. -18.0 does not upgrade a v17 database: keep a v17 database on a 17.x release -until the 18.x release that carries the upgrade path. For a local SQLite file, -`os dev` in 18.0 still takes step 1's copy first and prints where it is, -beside the ceremony's `NOT_IMPLEMENTED` refusal; any other database gets no -copy and the same answer. - - -In the 18.x release that carries the upgrade path, `os dev` runs that ceremony -for you when the database is a local SQLite file: +Fresh installs are supported under every tenancy posture. A v18 server +refuses to start on a database with no birth marker, which 17.x and the 18.0 +pre-releases (`18.0.0-next.*`) created, until the [ownership ceremony](#os-migrate-organization-ownership) +has run on it. `os dev` runs that ceremony for you when the database is a local +SQLite file: 1. It copies the database file and its `telemetry` sibling (each with its `-wal` / `-shm`) into `/backups/organization-ownership-/`. @@ -222,14 +211,14 @@ for you when the database is a local SQLite file: beside the copy. 3. It prints the ceremony's warnings and starts the server again. -In that release, if a step blocks, or the post-check still holds rows (a row whose owner cannot +If a step blocks, or the post-check still holds rows (a row whose owner cannot be derived, named per table with its remedy), `os dev` stops and prints what holds it, where the copy is, and the command that resumes the run. It tries once per session. A database that is not a local SQLite file (PostgreSQL, a remote libSQL, `:memory:`) is not upgraded in place: run the ceremony yourself. -`os serve` and `os start` never run the ceremony: they refuse to start, and on -a v17 database their 18.0 refusal says that 18.0 does not upgrade it. +`os serve` and `os start` never run the ceremony: they refuse to start, and the +refusal names the command. ##### Serving dev over https @@ -1040,12 +1029,9 @@ safe. No flag skips these refusals: `sys_metadata` family among them. Dropping one first would turn every legacy organization row into an environment row. Under `--json` the refusal answers `message: 'refused_organization_ownership_unmigrated'` and lists the `tables`. - 18.0 itself never records a verified run, so on a v17 database this refusal - holds on every run. 18.0 supports fresh installs only and does not upgrade a - v17 database: keep a v17 database on a 17.x release until the 18.x release - that carries the upgrade path. In that release, the ceremony's - `os migrate organization-ownership --apply --plan-file FILE` run drops these - columns itself, in its column-drops step, before it records the run verified. + The ceremony's `os migrate organization-ownership --apply --plan-file FILE` + run drops these columns itself, in its column-drops step, before it records + the run verified. Each refusal holds the whole `--allow-destructive` run: nothing in it is created, applied or dropped, the run's other changes included. A run without @@ -1216,7 +1202,7 @@ where the data lives. | `os migrate value-shapes` | Scan stored reference and structured-JSON field values against the platform's value contract, and record the deployment's migration flag when clean | | `os migrate summary-nulls` | Backfill roll-up `count` / `sum` columns still stored as `NULL` on parent rows created before the insert-time seed. Repairs values; no flag, nothing depends on it having run | | `os migrate comment-reactions` | Carry the emoji reactions a 17.x deployment stored in the retired `sys_comment.reactions` column into `sys_comment_reaction` records, list every stored value that does not read, and record the verified run that lets `os migrate apply --allow-destructive` drop the column | -| `os migrate organization-ownership` | The v18 ownership ceremony (ADR-0131 D10): write a plan of what each table's rows will become, then `--apply` that reviewed plan. A v18 server refuses to start until it has run. Fresh installs are supported under every tenancy posture; the upgrade path arrives in a later 18.x release. Until then the command refuses. See [below](#os-migrate-organization-ownership) | +| `os migrate organization-ownership` | The v18 ownership ceremony (ADR-0131 D10): write a plan of what each table's rows will become, then `--apply` that reviewed plan. It upgrades a database 17.x or an 18.0 pre-release (`18.0.0-next.*`) created; a v18 server refuses to start on one until it has run. See [below](#os-migrate-organization-ownership) | | `os migrate activity-changed-fields` | Stamp the two read-rule columns (`changed_fields`, `milestone_key`) on the `sys_activity` rows written before they existed. Until it runs, those rows are withheld from every reader but the system, so run it at once after upgrading. No flag; idempotent | | `os migrate audit-mail-bodies` | Mask the mail bodies (`body_text`, `body_html`) that `sys_audit_log` and `sys_activity` copies of `sys_email` / `sys_identity_email` rows still hold in clear, written before the audit writer masked them. Reports counts only. No flag; idempotent. See [below](#os-migrate-audit-mail-bodies) | | `os migrate comment-parent-pair` | Stamp the parent pair (`object_name`, `record_id`) on the `sys_comment` rows written before it existed. Until it runs, a broad comment read past 2,000 candidate rows omits comments the reader may see, and its total agrees with the short list, so run it at once after upgrading. No flag; idempotent | @@ -1521,18 +1507,32 @@ upgrading rather than wait for that. #### `os migrate organization-ownership` -The v18 ownership ceremony (ADR-0131 D10). A v18 `os serve` / `os start` refuses -to start on a database still in the v17 organization-ownership shape until this -has run. In the 18.x release that carries the upgrade path, `os dev` runs it on -its own for a local SQLite database (see [`os dev`](#os-dev)). - - -Fresh installs are supported under every tenancy posture; the upgrade path arrives in a later 18.x release. Until then this command refuses, plan and apply alike: it exits `1` with -`NOT_IMPLEMENTED` (under `--json`, `code` beside `error`) and reads and -writes nothing. Keep a v17 database on a 17.x release until the 18.x release -that carries the upgrade path. The invocations below, and what the list after -them says the ceremony does, are that 18.x release's. - +The v18 ownership ceremony (ADR-0131 D10), and the upgrade path of a database +17.x or an 18.0 pre-release (`18.0.0-next.*`) created. +Fresh installs are supported under every tenancy posture and never need it. +A v18 `os serve` / `os start` serves a database this release created itself, or one this ceremony +certified; it refuses any other until the ceremony has run. Under `os dev` it +runs on its own for a local SQLite database (see [`os dev`](#os-dev)). + +How a database is recognised: the first boot that creates a datastore from +empty writes a **birth marker** into `sys_migration` (row +`adr-0131-organization-ownership`) naming the ownership shape generation it was +born in. The ceremony writes the same row, verified, when its post-check +passes. A server decides on that generation alone: a database with no birth +marker, which 17.x and the 18.0 pre-releases created, or one a shorter ceremony +certified at an older generation, is refused, and the refusal names this +command. + +The sequence on an existing database: + +1. Start this release against it once (`os serve`). The boot is refused, but + its schema sync has already created the tables this release adds, the + sibling logs the ceremony moves rows into among them. A plan drawn before + that boot blocks its `sibling-moves` step, naming the missing table and this + remedy. +2. Take a backup. +3. Write the plan, review it, then apply that reviewed plan (below). +4. Start the server again. ```bash os migrate organization-ownership --out plan.json # the plan: reads only diff --git a/content/docs/deployment/tenancy-modes.mdx b/content/docs/deployment/tenancy-modes.mdx index ffc0d6cdc49..4f17d88f413 100644 --- a/content/docs/deployment/tenancy-modes.mdx +++ b/content/docs/deployment/tenancy-modes.mdx @@ -16,10 +16,12 @@ it generalizes the two-valued *tenancy mode* of whose `tenancy` service, membership lifecycle and degraded-tenancy boot guard all still apply — widened to the three postures. - + Fresh installs are supported under every posture: `single`, `group` and -`isolated`. The upgrade path for an existing database arrives in a later 18.x -release, and a database an 18.0 install created takes the same path as a v17 one. +`isolated`. A database with no birth marker, which 17.x and the 18.0 +pre-releases (`18.0.0-next.*`) created, is refused at boot until +[`os migrate organization-ownership`](/docs/deployment/cli#os-migrate-organization-ownership) +has upgraded it. diff --git a/content/docs/kernel/contracts/metadata-service.mdx b/content/docs/kernel/contracts/metadata-service.mdx index c082d2d5b3d..4e2ddfd6eb2 100644 --- a/content/docs/kernel/contracts/metadata-service.mdx +++ b/content/docs/kernel/contracts/metadata-service.mdx @@ -415,13 +415,10 @@ read is environment → code, whatever organization a caller names. An overlay row an earlier release stored under an organization is served by no read. On a SQL database (SQLite, PostgreSQL, MySQL) still holding one, a v18 `os serve` refuses to start, because the `sys_metadata` family there still carries its -`organization_id` column. 18.0 supports fresh installs only and does not -upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x -release that carries the upgrade path. In that release, the v18 migration -ceremony (ADR-0131 C7) carries such a row to the environment layer. The -anonymous form doors read the environment alone too; in that release the -ceremony carries every organization-layer public-form withdrawal onto the -environment, fail-closed. The per-user, +`organization_id` column, until the v18 migration ceremony (ADR-0131 C7) has +carried such a row to the environment layer. The anonymous form doors read the +environment alone too; the ceremony carries every organization-layer +public-form withdrawal onto the environment, fail-closed. The per-user, per-field patch overlay a previous revision of this page taught here was removed in #13135 — it was never served by any route. diff --git a/content/docs/permissions/capabilities.mdx b/content/docs/permissions/capabilities.mdx index 847f32c0bb1..2e9220c786f 100644 --- a/content/docs/permissions/capabilities.mdx +++ b/content/docs/permissions/capabilities.mdx @@ -217,10 +217,7 @@ and resource `requiredPermissions` match capability names **as strings**. The boot writes no row for a capability, declared or curated: both are served by the registry. The `sys_capability` object itself is retired and no longer registered (ADR-0131 D3): a row an older database still holds is served by no -route and read by nothing. In the 18.x release that carries the upgrade path -(18.0 does not upgrade a v17 database: see -[`os migrate organization-ownership`](/docs/deployment/cli#os-migrate-organization-ownership)), -`os migrate organization-ownership --apply` reports the authored rows and +route and read by nothing. [`os migrate organization-ownership --apply`](/docs/deployment/cli#os-migrate-organization-ownership) reports the authored rows and deletes the seeded copies (ADR-0131 D10). Withdraw a capability by withdrawing the **grant**. diff --git a/content/docs/permissions/permission-sets.mdx b/content/docs/permissions/permission-sets.mdx index f3710de1cc1..a0a2e189b8f 100644 --- a/content/docs/permissions/permission-sets.mdx +++ b/content/docs/permissions/permission-sets.mdx @@ -396,10 +396,7 @@ posture, the platform operator. This has three author-visible consequences: package. Uninstall the owning package to remove a packaged set entirely. An older database keeps the `sys_permission_set` table and its rows, and -nothing serves or reads them. In the 18.x release that carries the upgrade -path (18.0 does not upgrade a v17 database: see -[`os migrate organization-ownership`](/docs/deployment/cli#os-migrate-organization-ownership)), -`os migrate organization-ownership --apply` +nothing serves or reads them. [`os migrate organization-ownership --apply`](/docs/deployment/cli#os-migrate-organization-ownership) (ADR-0131 D10) converts the sets an organization authored in Setup into environment metadata under `single` and deletes the seeded copies. The shipped sets that an install created before 2026-08-15 stored as `managed_by: 'admin'` diff --git a/content/docs/references/index.mdx b/content/docs/references/index.mdx index 178df51d814..7dad531e931 100644 --- a/content/docs/references/index.mdx +++ b/content/docs/references/index.mdx @@ -1,7 +1,7 @@ --- title: Protocol reference — every schema by module navTitle: Protocol Reference -description: Every schema published by @objectstack/spec — 1541 schemas across 14 protocol modules +description: Every schema published by @objectstack/spec — 1542 schemas across 14 protocol modules --- {/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} @@ -32,9 +32,9 @@ counts are sums of the rows they head. Regenerate with | [Security Protocol](/docs/references/security) | 5 | 31 | Permission sets, row-level security, sharing rules, tenancy posture. | | [Shared Protocol](/docs/references/shared) | 10 | 31 | Primitives used across every protocol — identifiers, HTTP, expressions, error maps, enums. | | [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. | -| [System Protocol](/docs/references/system) | 36 | 284 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. | +| [System Protocol](/docs/references/system) | 36 | 285 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. | | [UI Protocol](/docs/references/ui) | 16 | 169 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. | -| **Total** | **197** | **1541** | 14 protocol modules | +| **Total** | **197** | **1542** | 14 protocol modules | --- @@ -317,7 +317,7 @@ Studio designer metadata — the authoring surfaces for the protocols above. ## System Protocol -**Source:** `packages/spec/src/system/` · **Import:** `@objectstack/spec/system` · **36 pages, 284 schemas** +**Source:** `packages/spec/src/system/` · **Import:** `@objectstack/spec/system` · **36 pages, 285 schemas** The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. @@ -344,7 +344,7 @@ The runtime environment — logging, jobs, cache, metrics, notifications, i18n a | [`logging.zod.ts`](/docs/references/system/logging) | `ConsoleDestinationConfig`, `ExtendedLogLevel`, `ExternalServiceDestinationConfig`, `FileDestinationConfig`, `HttpDestinationConfig`, `LogDestination`, `LogDestinationType`, `LogEnrichmentConfig`, `LogEntry`, `LogFormat`, `LogLevel`, `LoggerConfig`, `LoggingConfig`, `StructuredLogEntry` | | [`metadata-persistence.zod.ts`](/docs/references/system/metadata-persistence) | `MetadataCollectionInfo`, `MetadataDiffResult`, `MetadataFallbackStrategy`, `MetadataFormat`, `MetadataHistoryQueryOptions`, `MetadataHistoryQueryResult`, `MetadataHistoryRecord`, `MetadataHistoryRetentionPolicy`, `MetadataLoadOptions`, `MetadataLoadResult`, `MetadataLoaderContract`, `MetadataManagerConfig`, `MetadataRecord`, `MetadataSaveOptions`, `MetadataSaveResult`, `MetadataScope`, `MetadataSource`, `MetadataState`, `MetadataStats`, `MetadataWatchEvent`, `PackagePublishResult` | | [`metrics.zod.ts`](/docs/references/system/metrics) | `HistogramBucketConfig`, `MetricAggregationConfig`, `MetricAggregationType`, `MetricDataPoint`, `MetricDefinition`, `MetricExportConfig`, `MetricLabels`, `MetricType`, `MetricUnit`, `MetricsConfig`, `ServiceLevelIndicator`, `ServiceLevelObjective`, `TimeSeries`, `TimeSeriesDataPoint` | -| [`migration.zod.ts`](/docs/references/system/migration) | `AddFieldOperation`, `ChangeSet`, `CreateObjectOperation`, `DataMigrationFlag`, `DeleteObjectOperation`, `ExecuteSqlOperation`, `MigrationDependency`, `MigrationJournalEvent`, `MigrationOperation`, `ModifyFieldOperation`, `RemoveFieldOperation`, `RenameObjectOperation` | +| [`migration.zod.ts`](/docs/references/system/migration) | `AddFieldOperation`, `ChangeSet`, `CreateObjectOperation`, `DataMigrationFlag`, `DeleteObjectOperation`, `ExecuteSqlOperation`, `MigrationDependency`, `MigrationJournalEvent`, `MigrationOperation`, `ModifyFieldOperation`, `OrganizationOwnershipBirthMarker`, `RemoveFieldOperation`, `RenameObjectOperation` | | [`notification.zod.ts`](/docs/references/system/notification) | `NotificationChannel` | | [`object-storage.zod.ts`](/docs/references/system/object-storage) | `AccessControlConfig`, `BucketConfig`, `FileMetadata`, `LifecycleAction`, `LifecyclePolicyConfig`, `LifecyclePolicyRule`, `MultipartUploadConfig`, `ObjectMetadata`, `ObjectStorageConfig`, `PresignedUrlConfig`, `StorageAcl`, `StorageClass`, `StorageConnection`, `StorageProvider`, `StorageScope` | | [`registry-config.zod.ts`](/docs/references/system/registry-config) | `RegistryConfig`, `RegistrySyncPolicy`, `RegistryUpstream` | diff --git a/content/docs/references/system/migration.mdx b/content/docs/references/system/migration.mdx index 63ac5b26c39..04854497121 100644 --- a/content/docs/references/system/migration.mdx +++ b/content/docs/references/system/migration.mdx @@ -27,8 +27,8 @@ irreversibly on migrated data gate on the flag instead of the version. ## TypeScript Usage ```typescript -import { AddFieldOperation, ChangeSetSchema, CreateObjectOperation, DataMigrationFlagSchema, DeleteObjectOperation, ExecuteSqlOperation, MigrationDependencySchema, MigrationJournalEventSchema, MigrationOperationSchema, ModifyFieldOperation, RemoveFieldOperation, RenameObjectOperation } from '@objectstack/spec/system'; -import type { ChangeSet, DataMigrationFlag, DeleteObjectOperation, ExecuteSqlOperation, MigrationDependency, MigrationJournalEvent, MigrationOperation, ModifyFieldOperation, RemoveFieldOperation, RenameObjectOperation } from '@objectstack/spec/system'; +import { AddFieldOperation, ChangeSetSchema, CreateObjectOperation, DataMigrationFlagSchema, DeleteObjectOperation, ExecuteSqlOperation, MigrationDependencySchema, MigrationJournalEventSchema, MigrationOperationSchema, ModifyFieldOperation, OrganizationOwnershipBirthMarkerSchema, RemoveFieldOperation, RenameObjectOperation } from '@objectstack/spec/system'; +import type { ChangeSet, DataMigrationFlag, DeleteObjectOperation, ExecuteSqlOperation, MigrationDependency, MigrationJournalEvent, MigrationOperation, ModifyFieldOperation, OrganizationOwnershipBirthMarker, RemoveFieldOperation, RenameObjectOperation } from '@objectstack/spec/system'; // Validate data const result = AddFieldOperation.parse(data); @@ -728,6 +728,22 @@ Modify properties of an existing field | **changes** | `Record` | ✅ | Partial field definition updates | +--- + +## OrganizationOwnershipBirthMarker + +Birth marker of a datastore created from empty: the ownership shape generation it was born in + +### Properties + +| Property | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| **shapeGeneration** | `integer` | ✅ | The organization-ownership shape generation the store was created in; the only field the boot guard decides on | +| **platformVersion** | `string` | ✅ | The exact platform version that created the store (provenance only, never compared) | +| **commit** | `string` | optional | The source commit of that build, when known (provenance only) | +| **createdAt** | `string` | ✅ | When the guard observed the store created from empty | + + --- ## RemoveFieldOperation diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.counts/system.md b/docs/audits/2026-07-unknown-key-strictness-ledger.counts/system.md index 6aaedaaf246..fee2a67d822 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.counts/system.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.counts/system.md @@ -19,4 +19,4 @@ hand-patch a number here** — fix the code or the verdict and regenerate. | Dir | Sites | |---|---| -| `system/` | 374 | +| `system/` | 375 | diff --git a/docs/qa/platform-checklist/areas/cli.json b/docs/qa/platform-checklist/areas/cli.json index f5d45a286b3..b85f263afc3 100644 --- a/docs/qa/platform-checklist/areas/cli.json +++ b/docs/qa/platform-checklist/areas/cli.json @@ -1871,10 +1871,10 @@ }, { "id": "cli.organization-ownership-ceremony", - "title": "`os migrate organization-ownership` in 18.0, which supports fresh installs only: plan and apply alike refuse NOT_IMPLEMENTED before reading anything and leave the database unchanged, and `os migrate apply --allow-destructive` still withholds every organization_id drop on a v17-shaped database; the ceremony class the refusal keeps underneath (plan, refusals, checkpoints, drop gate) and `os migrate comment-reactions --apply` stay pinned — the v17 → v18 ceremony itself ships in an 18.x release", + "title": "`os migrate organization-ownership` upgrades a database 17.x or an 18.0 pre-release (18.0.0-next.*) created (18.x): a bare run writes the reviewed plan and changes nothing; `--apply --plan-file PLAN --backup-taken` runs every step — attribution through the moves, the declared webhook and sharing-rule settlements, the column drops and NOT NULL — and certifies the marker at this release's ownership shape generation; the served boot refuses an uncertified database, naming the command, and boots a certified one; `os migrate apply --allow-destructive` still withholds every organization_id drop the ceremony owns, and `os migrate comment-reactions --apply` stays pinned", "since": "v18", "status": "active", - "revision": 2, + "revision": 3, "priority": "P0", "surface": "cli", "personas": [ @@ -1887,55 +1887,62 @@ "a COPY of a v17-shaped SQLite DB (platform-core.v18-boot-refusals names how to get one) — never the original" ], "knownGaps": [ - "WITHDRAWN IN 18.0 and deliberately not scored here: the ceremony's own arms — the plan file a bare run writes, its plan-file-required / plan-*-mismatch / database-busy refusals, the sys_migration adr-0131-organization-ownership checkpoint a reviewed apply writes, and a migrated copy booting under v18 — are not clauses of this revision. QA run #24551: \"ruling records 6259608309 (item 1) and 6259698357 then cut 18.0 to fresh installs on the single posture\" — carried, the record goes on, by the changeset 15212-180-guards-fresh-single-only (consumed into 18.0.0; source lists it with the others), the ADR-0131 release-scope note and organization-ownership.ts; the registered command (packages/cli/src/commands/migrate/organization-ownership.ts#MigrateOrganizationOwnership) refuses before the ceremony runs. A runner scores none of these arms — neither `fail` (QA run #24551's reading of revision 1) nor `skipped` (QA run #24556's, c1/c2/c4): they are not on the item. The 18.x release that ships the upgrade path restores them as clauses in the revision that re-registers the ceremony; their revision-1 wording is in this file's git history", - "acceptance[3] scores the drop gate's exit code, `message` and `tables`, not its prose: that prose still prescribes `os migrate organization-ownership --out PLAN` and `--apply --plan-file PLAN`, which 18.0 refuses — a product defect tracked on #24567, not a fail of this clause" + "a walled database (group or isolated posture) holding a CUSTOMIZED copy of a declared webhook or sharing rule blocks at its settlement step, by design: the declaration's switch is deployment-wide, so the ceremony does not switch it off for every organization on the operator's behalf — that arm is scored by the unit pins (organization-ownership-apply.generation-1.test.ts), not by a step here" ] }, "steps": [ - "`os migrate organization-ownership --database-url file:COPY` on the v17-shaped copy, plain and with `--json`; checksum the DB before and after, and list the working directory for organization-ownership-plan-*.json", - "`--apply` without --plan-file; `--apply --plan-file P --backup-taken` with P any path (18.0 writes no plan, and the refusal reads none); `--apply` with the DB held open by a second process; each with `--json` too", - "`os migrate apply --allow-destructive --yes --json` on the v17-shaped copy; checksum the DB before and after", + "`os migrate organization-ownership --database-url file:COPY --out /tmp/RUN/plan.json` on the v17-shaped copy, in the showcase's directory; checksum the DB before and after", + "`--apply` without --plan-file; then `--apply --plan-file /tmp/RUN/plan.json --backup-taken --out /tmp/RUN/apply.json --json`; then read `SELECT verified_at, blocking, details FROM sys_migration WHERE id = 'adr-0131-organization-ownership'`", + "boot the showcase (`os serve`) over the migrated copy, and over a second, unmigrated copy", + "`os migrate apply --allow-destructive --yes --json` on a fresh unmigrated v17-shaped copy; checksum the DB before and after", "`os migrate comment-reactions --apply` twice" ], "acceptance": [ { - "clause": "the 18.0 release refusal of the registered command, and the ceremony class it keeps whole underneath (plan, refusals, checkpoints, drop gate, reactions conversion) — the permanent pins pass at the run subject sha (RUNNER rule 6: cite their output, never re-derive by hand)", + "clause": "the ceremony and the guard — plan, refusals, checkpoints, the generation-1 steps, the drop gate, the birth marker and the reactions conversion — the permanent pins pass at the run subject sha (RUNNER rule 6: cite their output, never re-derive by hand)", "oracle": "test", "verify": "run each file named in automated.ref with its package's vitest (`pnpm --filter PKG exec vitest run --maxWorkers=2 FILE`, never through a bare `--`); every named file reports ≥1 test and 0 failed. A file that reports 0 tests or is not found is NOT a pass — it is a stale ref, fix the item", "evidence": "the vitest summary line per pin file, with the package name and the subject sha" }, { - "clause": "a bare run (plan mode, plain or `--json`) is refused before it reads anything: exit 1 with NOT_IMPLEMENTED — `--json` answers {error: plan_refused, reason: upgrade-unavailable, code: NOT_IMPLEMENTED, detail}, the plain form prints the refusal with `(NOT_IMPLEMENTED)` — no organization-ownership-plan-*.json is written, and the DB checksum is unchanged", + "clause": "a bare run writes the plan file and changes nothing: exit 0, the plan JSON names every table's fate, and the DB checksum is unchanged; `--apply` without --plan-file is refused before the boot (exit 1, plan-file-required)", "oracle": "build", - "verify": "exit code + the JSON object (or the printed refusal) per form + a directory listing + checksums before/after", - "evidence": "the refusal per form, the listing, the checksum pair" + "verify": "exit codes + the plan file's kind/ceremony/inventoryDigest + checksums before/after", + "evidence": "the plan excerpt, the refusal line, the checksum pair" }, { - "clause": "every --apply arm — no --plan-file, a --plan-file with --backup-taken, the DB held open by a second process — is refused the same way before reading anything: exit 1, `--json` {error: apply_refused, reason: upgrade-unavailable, code: NOT_IMPLEMENTED}; none answers the ceremony's own plan-file-required, plan-*-mismatch or database-busy, which nothing reaches in 18.0, and nothing is applied", + "clause": "`--apply --plan-file PLAN --backup-taken` completes: exit 0, every step done (the report's steps), blocking 0, and the sys_migration marker is verified with details.checkpoint.shapeGeneration equal to this release's generation", "oracle": "build", - "verify": "exit code + error / reason / code per arm + checksums before/after", - "evidence": "per-arm lines" + "verify": "the apply report's steps/complete/blocking + the marker row", + "evidence": "the report excerpt and the marker row" + }, + { + "clause": "the served boot refuses the unmigrated copy with ORGANIZATION_OWNERSHIP_MIGRATION_REQUIRED, naming `os migrate organization-ownership` and no 17.x release to stay on, and boots the migrated copy", + "oracle": "build", + "verify": "the refused boot's error line + the migrated boot's ready line", + "evidence": "both lines" }, { - "clause": "the drop gate still holds on a v17-shaped DB: `os migrate apply --allow-destructive` refuses every organization_id drop the ceremony owns — exit 1, `--json` message refused_organization_ownership_unmigrated with `tables` naming them — and applies nothing", + "clause": "the drop gate still holds on an unmigrated v17-shaped DB: `os migrate apply --allow-destructive` refuses every organization_id drop the ceremony owns — exit 1, `--json` message refused_organization_ownership_unmigrated with `tables` naming them — and applies nothing", "oracle": "build", "verify": "exit code + the JSON message and tables + checksums before/after", "evidence": "the excerpt and the checksum pair" } ], "negative": [ - "any ceremony arm that alters the DB while refusing is a data-integrity FAIL" + "any ceremony arm that alters the DB while refusing, or a plan run that alters it at all, is a data-integrity FAIL" ], "traps": [ "destructive-in-place" ], "automated": { "kind": "unit", - "ref": "packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts + packages/cli/src/utils/organization-ownership-apply.test.ts + packages/cli/src/commands/migrate/organization-ownership.occupancy.integration.test.ts + packages/cli/src/commands/migrate/organization-ownership.dialect.test.ts + packages/cli/src/commands/migrate/apply-organization-ownership-gate.integration.test.ts + packages/cli/src/commands/migrate/comment-reactions.integration.test.ts + packages/cli/src/utils/organization-ownership-host-supplement.integration.test.ts + packages/cli/src/commands/migrate/organization-ownership.release-refusal.test.ts" + "ref": "packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts + packages/cli/src/utils/organization-ownership-apply.test.ts + packages/cli/src/commands/migrate/organization-ownership.occupancy.integration.test.ts + packages/cli/src/commands/migrate/organization-ownership.dialect.test.ts + packages/cli/src/commands/migrate/apply-organization-ownership-gate.integration.test.ts + packages/cli/src/commands/migrate/comment-reactions.integration.test.ts + packages/cli/src/utils/organization-ownership-host-supplement.integration.test.ts + packages/cli/src/utils/organization-ownership-apply.generation-1.test.ts + packages/runtime/src/organization-ownership-boot-guard.test.ts" }, "source": [ "packages/cli/src/commands/migrate/organization-ownership.ts", - "packages/cli/src/commands/migrate/organization-ownership.ts#ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE", + "packages/runtime/src/organization-ownership-boot-guard.ts", + "packages/spec/src/system/migration.zod.ts#ORGANIZATION_OWNERSHIP_SHAPE_GENERATION", "packages/spec/src/system/fate-supplement.zod.ts#OrganizationOwnershipSupplementSchema", "packages/cli/src/utils/organization-ownership-drop-gate.ts", "docs/adr/0131-total-organization-ownership-no-null-organization-id.md", @@ -1953,6 +1960,12 @@ "date": "2026-10-19", "change": "re-pointed to the 18.0 release refusal (QA run #24551 acceptance[1]–[4] fail, disposition stale-spec, RUNNER rule 7 agreeing; parked on #24579 §1). 18.0 was narrowed to fresh installs by ruling — #24551: \"ruling records 6259608309 (item 1) and 6259698357 then cut 18.0 to fresh installs on the single posture\" — and the registered `os migrate organization-ownership` refuses plan and apply alike with NOT_IMPLEMENTED before reading anything. acceptance[1] and [2] now score that refusal (no plan written, the DB unchanged, every apply arm refused the same way); acceptance[3] keeps the drop-gate half and drops the sys_migration checkpoint half; revision 1's acceptance[4] (the migrated copy boots) is removed. The ledger has no clause-level skip, so the ceremony's own arms are recorded in fixtures.knownGaps[0] as withdrawn and deliberately not scored, the shape knownGaps already carries for a half an item does not score. That is the one reading for both runs: a runner scores those arms neither `fail` (#24551, revision 1) nor `skipped` (QA run #24556, c1/c2/c4), and both runs captured the refusal each re-pointed clause now asserts. automated.ref gains organization-ownership.release-refusal.test.ts, the pin of the registered command (the other pins drive the ceremony class beneath it); acceptance[0], the title, the steps and source follow; re-derived from main 5a871e4e41, where every source cited here is byte-identical to the run subject 5b14b7f3", "ref": "#24622" + }, + { + "revision": 3, + "date": "2026-10-19", + "change": "re-pointed to the opened ceremony (18.x steps 5 and 6, one landing by ruling record 6270035914 on the v18 epic): the registered command runs the ceremony again, the 18.0 refusal and its pin (organization-ownership.release-refusal.test.ts) are deleted, and the boot guard recognises a database by its birth marker and its ownership shape generation (ruling record 6269955919, ① A). Revision 1's arms are restated against it: the plan written, the reviewed apply certifying at the current generation, the served boot refusing an unmigrated copy and booting the migrated one; the drop-gate clause is kept; fixtures.knownGaps now records the one deliberate block (a walled customized copy). automated.ref drops the deleted refusal pin and gains the generation-1 steps' pins and the guard's", + "ref": "#24681" } ] }, diff --git a/docs/qa/platform-checklist/areas/platform-core.json b/docs/qa/platform-checklist/areas/platform-core.json index bed72a18245..3a59e9b9694 100644 --- a/docs/qa/platform-checklist/areas/platform-core.json +++ b/docs/qa/platform-checklist/areas/platform-core.json @@ -3043,10 +3043,10 @@ }, { "id": "platform-core.v18-boot-refusals", - "title": "A v18 server refuses to boot rather than serve wrongly: against a 17.x-shaped database (a physical organization_id on a non-tenant table, no verified ADR-0131 ceremony marker) it stops before listening with ORGANIZATION_OWNERSHIP_MIGRATION_REQUIRED, naming the tables and saying that 18.0 supports fresh installs only, and with security on but no identity objects it stops with AuthzIdentityObjectsMissingError", + "title": "A v18 server refuses to boot rather than serve wrongly: against a 17.x-shaped database (a physical organization_id on a non-tenant table, no verified ADR-0131 ceremony marker) it stops before listening with ORGANIZATION_OWNERSHIP_MIGRATION_REQUIRED, naming the tables and the ownership ceremony's plan and apply, and with security on but no identity objects it stops with AuthzIdentityObjectsMissingError", "since": "v18", "status": "active", - "revision": 4, + "revision": 5, "priority": "P0", "surface": "cli", "personas": [ @@ -3071,9 +3071,9 @@ "evidence": "the vitest summary line per pin file, with the package name and the subject sha" }, { - "clause": "the v18 serve against the v17 DB exits 1 before kernel:listening with the organization-ownership refusal, which names the tables and ends with the 18.0 sentence (\"18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path, and upgrade it there.\"), and names neither `os migrate organization-ownership` nor any step of it (no `--out PLAN`, no `--apply --plan-file`): 18.0 refuses that command on every database (NOT_IMPLEMENTED). ORGANIZATION_OWNERSHIP_MIGRATION_REQUIRED is that refusal's `code` (packages/runtime/src/organization-ownership-boot-guard.ts#OrganizationOwnershipBootRefusal) and the boot never prints it — `os serve` prints the refusal's message only — so the code is the pins' evidence (acceptance[0]), not this log's", + "clause": "the v18 serve against the v17 DB exits 1 before kernel:listening with the organization-ownership refusal, which opens \"This database carries no organization-ownership marker\", names the tables still carrying organization_id, and names the ceremony's two steps (`os migrate organization-ownership --out PLAN`, then `os migrate organization-ownership --apply --plan-file PLAN --backup-taken`). It does not tell the operator to stay on 17.x. ORGANIZATION_OWNERSHIP_MIGRATION_REQUIRED is that refusal's `code` (packages/runtime/src/organization-ownership-boot-guard.ts#OrganizationOwnershipBootRefusal) and the boot never prints it — `os serve` prints the refusal's message only — so the code is the pins' evidence (acceptance[0]), not this log's", "oracle": "log", - "verify": "boot output + exit code + no listening line: the printed refusal names each table still carrying organization_id and ends with \"18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path, and upgrade it there.\"; the refusal excerpt has 0 hits each for `--out PLAN`, `--plan-file` and `os migrate organization-ownership`. ⛔ Do not grep the log for the code literal — it is not printed (packages/cli/src/commands/serve.ts prints the refusal's message)", + "verify": "boot output + exit code + no listening line: the printed refusal opens \"This database carries no organization-ownership marker\", names each table still carrying organization_id, and contains `os migrate organization-ownership --out PLAN` and `--apply --plan-file PLAN --backup-taken`; the refusal excerpt has 0 hits for `17.x release` and for `fresh installs only`. ⛔ Do not grep the log for the code literal — it is not printed (packages/cli/src/commands/serve.ts prints the refusal's message)", "evidence": "the refusal excerpt" }, { @@ -3130,6 +3130,12 @@ "date": "2026-10-19", "change": "title and acceptance[1] (clause, verify) re-worded for 18.0's boot refusal text (card #24651). In 18.0 `os migrate organization-ownership` refuses plan and apply on every database (NOT_IMPLEMENTED: 18.0 supports fresh installs only), so the boot refusal no longer names that command or its `--out PLAN` / `--apply --plan-file PLAN --backup-taken` steps. Every branch now ends with \"18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path, and upgrade it there.\" (packages/runtime/src/organization-ownership-boot-guard.ts, freshInstallsOnly; `os serve --dev` says \"a v17 development database\"). The clause asserted that the refusal names the command, so a run against the new text would have recorded FAIL on a correct product. It now asserts the tables, that sentence, and the absence of the command and its steps. The ⛔ on the code literal is kept. acceptance[0], acceptance[2], acceptance[3] and both steps are unchanged", "ref": "#24682" + }, + { + "revision": 5, + "date": "2026-10-20", + "change": "title and acceptance[1] (clause, verify) re-worded for the open ownership ceremony (#24581). `os migrate organization-ownership` plans and applies again, and the boot guard decides on the ownership marker (a birth marker a fresh datastore writes, or the ceremony's verified one): a v17 database has none, so the refusal opens \"This database carries no organization-ownership marker\", names the tables still carrying organization_id, and names the ceremony's plan and apply (`--out PLAN`, then `--apply --plan-file PLAN --backup-taken`). Revision 4 asserted the 18.0 sentence and the absence of the command, so a run against the new text would have recorded FAIL on a correct product. It now asserts the marker sentence, the tables and the two steps, and that the refusal says nothing of staying on 17.x. The ⛔ on the code literal is kept. acceptance[0], acceptance[2], acceptance[3] and both steps are unchanged", + "ref": "#24681" } ] }, diff --git a/package.json b/package.json index 6457158513a..c059066f759 100644 --- a/package.json +++ b/package.json @@ -78,6 +78,7 @@ "check:pm-label-desc-cap": "node scripts/pm/check-label-desc-cap.mjs --self-test && node scripts/pm/check-label-desc-cap.mjs", "check:pm-dispatch-gates": "node scripts/pm/check-dispatch-gates.mjs --self-test && node scripts/pm/check-dispatch-gates.mjs", "check:watch-hint-literal": "node scripts/check-watch-hint-literal.mjs --self-test && node scripts/check-watch-hint-literal.mjs", + "check:closed-ceremony-text": "node scripts/check-closed-ceremony-text.mjs --self-test && node scripts/check-closed-ceremony-text.mjs", "check:declared-population-live": "node scripts/check-declared-population-live.mjs --self-test && node scripts/check-declared-population-live.mjs", "check:pm-half-states": "node scripts/pm/check-half-states.mjs --self-test", "check:issue-citations": "node scripts/check-issue-citations.mjs --self-test", diff --git a/packages/cli/src/commands/migrate/apply-organization-ownership-gate.integration.test.ts b/packages/cli/src/commands/migrate/apply-organization-ownership-gate.integration.test.ts index 9510bb0aa3b..38d867f9be2 100644 --- a/packages/cli/src/commands/migrate/apply-organization-ownership-gate.integration.test.ts +++ b/packages/cli/src/commands/migrate/apply-organization-ownership-gate.integration.test.ts @@ -104,8 +104,6 @@ function caseDb(columns: readonly string[]): string { const dropArgv = (file: string) => ['--database-url', `file:${file}`, '--allow-destructive', '--yes', '--json']; -/** A step of `os migrate organization-ownership`, which 18.0 refuses on every database. */ -const WITHDRAWN_COMMAND = /os migrate organization-ownership|--plan-file|organization-ownership --out/; beforeAll(async () => { for (const key of [...OVERRIDING_ENV, 'NODE_ENV'] as const) savedEnv[key] = process.env[key]; @@ -155,8 +153,6 @@ describe('os migrate apply --allow-destructive waits for the organization-owners marker: null, }); expect(payload.detail).toMatch(/organization-ownership ceremony/); - // 18.0 refuses `os migrate organization-ownership` on every database: the refusal names no step of it. - expect(payload.detail).not.toMatch(WITHDRAWN_COMMAND); expect(columnsOf(file, 'sys_platform_setting')).toContain('organization_id'); }, CASE_TIMEOUT_MS); @@ -169,7 +165,6 @@ describe('os migrate apply --allow-destructive waits for the organization-owners expect(exitCode).toBe(1); expect(payload).toMatchObject({ message: 'refused_organization_ownership_unmigrated', tables: ['sys_platform_setting'] }); expect(payload.detail).toMatch(/is not verified \(2 blocking/); - expect(payload.detail).not.toMatch(WITHDRAWN_COMMAND); expect(columnsOf(file, 'sys_platform_setting')).toContain('organization_id'); }, CASE_TIMEOUT_MS); diff --git a/packages/cli/src/commands/migrate/apply.organization-ownership-refusal.test.ts b/packages/cli/src/commands/migrate/apply.organization-ownership-refusal.test.ts index 0cd96b6978c..08fa2c97e86 100644 --- a/packages/cli/src/commands/migrate/apply.organization-ownership-refusal.test.ts +++ b/packages/cli/src/commands/migrate/apply.organization-ownership-refusal.test.ts @@ -3,15 +3,13 @@ /** * [ADR-0131 D10 item 5] What `os migrate apply --allow-destructive` tells the * operator when it holds back an `organization_id` drop the organization-ownership - * ceremony owns, in 18.0. + * ceremony owns. * - * 18.0 supports fresh installs only, and its registered - * `os migrate organization-ownership` refuses on every database, plan and apply - * alike (`organization-ownership.release-refusal.test.ts`). So the drop gate's - * refusal, the human face and the `--json` `detail` both, names no step of that - * command: it says what 18.0 supports for a v17 database, the 17.x release to - * keep it on and the 18.x release that carries the upgrade path, as the boot - * refusal does (`organizationOwnershipRefusalMessage`, `@objectstack/runtime`). + * The ceremony is the way on (18.x step 6 opened it), so the human face names + * its steps — the plan, then the reviewed apply with the backup acknowledged — + * as the boot refusal does (`organizationOwnershipRefusalMessage`, + * `@objectstack/runtime`), and names no release line to stay on. The `--json` + * `detail` names the ceremony and states the marker it read. * * ## What is real here * @@ -69,8 +67,11 @@ const ORGANIZATION_DROP = { description: 'sys_platform_setting.organization_id: column exists in the database but not in metadata (orphaned)', }; -/** A step of `os migrate organization-ownership`, which 18.0 refuses on every database. */ -const WITHDRAWN_COMMAND = /os migrate organization-ownership|--plan-file|organization-ownership --out/; +/** The ceremony's two steps, as the refusal prescribes them. */ +const CEREMONY_STEPS = [ + 'os migrate organization-ownership --out PLAN', + 'os migrate organization-ownership --apply --plan-file PLAN --backup-taken', +]; const UNVERIFIED = { id: 'adr-0131-organization-ownership', last_run_at: '2026-10-01T00:00:00.000Z', verified_at: null, blocking: 2 }; const VERIFIED = { ...UNVERIFIED, verified_at: '2026-10-01T00:00:00.000Z', blocking: 0 }; @@ -141,9 +142,9 @@ async function runApply(argv: string[]) { const ARGV = ['--database-url', 'file:v17.db', '--allow-destructive', '--yes']; -describe('os migrate apply --allow-destructive: the organization-ownership refusal names no command 18.0 refuses', () => { +describe('os migrate apply --allow-destructive: the organization-ownership refusal names the ceremony\'s steps', () => { for (const [state, flag] of [['no ceremony run recorded', null], ['an unverified run recorded', UNVERIFIED]] as const) { - it(`${state}: the human face refuses, exit 1, saying what 18.0 supports, and nothing is written`, async () => { + it(`${state}: the human face refuses, exit 1, naming the ceremony's steps, and nothing is written`, async () => { bootWithOrganizationDrop(); mocks.readOrganizationOwnershipMarker.mockResolvedValue({ verified: false, flag }); @@ -152,15 +153,14 @@ describe('os migrate apply --allow-destructive: the organization-ownership refus expect(exit).toBe(1); expect(refusal, 'the gate printed its refusal').toBeDefined(); expect(refusal).toContain('sys_platform_setting'); - expect(refusal).not.toMatch(WITHDRAWN_COMMAND); - expect(refusal).toContain('17.x release'); - expect(refusal).toContain('18.x release'); + for (const step of CEREMONY_STEPS) expect(refusal).toContain(step); + expect(refusal).not.toMatch(/17\.x|18\.0 supports/); if (flag) expect(refusal).toContain('2 blocking'); expect(flushSchemaDdl).not.toHaveBeenCalled(); expect(applyMigrationEntries).not.toHaveBeenCalled(); }, RUN_TIMEOUT); - it(`${state}: under --json the refusal's detail names no command 18.0 refuses either`, async () => { + it(`${state}: under --json the refusal's detail names the ceremony and the marker it read`, async () => { bootWithOrganizationDrop(); mocks.readOrganizationOwnershipMarker.mockResolvedValue({ verified: false, flag }); @@ -174,7 +174,7 @@ describe('os migrate apply --allow-destructive: the organization-ownership refus created: [], marker: flag, }); - expect(payload.detail).not.toMatch(WITHDRAWN_COMMAND); + expect(payload.detail).toMatch(/organization-ownership ceremony/); expect(applyMigrationEntries).not.toHaveBeenCalled(); }, RUN_TIMEOUT); } diff --git a/packages/cli/src/commands/migrate/apply.ts b/packages/cli/src/commands/migrate/apply.ts index 7a6414669b7..2de13adab01 100644 --- a/packages/cli/src/commands/migrate/apply.ts +++ b/packages/cli/src/commands/migrate/apply.ts @@ -571,15 +571,9 @@ export default class MigrateApply extends Command { ); const marker = await readOrganizationOwnershipMarker(engine as never); if (!marker.verified) { - // 18.0 supports fresh installs only, and its `os migrate - // organization-ownership` refuses on every database, plan and apply - // alike (`ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE`, ruling record - // 6259608309 on the v18 epic, item 1). So neither the `detail` nor the - // refusal names a step of that command: they say what 18.0 supports - // for a v17 database, in the boot refusal's terms - // (`organizationOwnershipRefusalMessage`, `@objectstack/runtime`). - // The 18.x release that registers the ceremony names its steps here - // again. + // The ceremony is the way on (18.x step 6): the refusal names its + // steps, as the boot refusal does (`organizationOwnershipRefusalMessage`, + // `@objectstack/runtime`). const detail = marker.flag ? `The last run of the organization-ownership ceremony recorded on this database (${marker.flag.last_run_at}) ` + `is not verified (${marker.flag.blocking} blocking item(s)).` @@ -594,10 +588,10 @@ export default class MigrateApply extends Command { printError( `Refusing this --allow-destructive run: it would drop organization_id from ${organizationDrops.join(', ')}, a drop the organization-ownership ceremony ` + `owns, and the ceremony has not verified on this database. ${detail} Dropping the column first would turn ` - + 'every legacy organization row into an environment row without the ceremony\'s promotion or report. 18.0 ' - + 'supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release ' - + 'until the 18.x release that carries the upgrade path, whose ceremony drops these columns itself, in its ' - + 'column-drops step, before it records the run verified. Nothing in this run was applied.', + + 'every legacy organization row into an environment row without the ceremony\'s promotion or report. Run ' + + '"os migrate organization-ownership --out PLAN" to write and review its plan, take a backup, then ' + + '"os migrate organization-ownership --apply --plan-file PLAN --backup-taken"; it drops these columns ' + + 'itself, in its column-drops step, before it records the run verified. Nothing in this run was applied.', ); this.exit(1); return; diff --git a/packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts b/packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts index b17cdff8df0..3816bac4702 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts @@ -28,13 +28,13 @@ vi.mock('../../utils/migrate-occupancy-gate.js', () => ({ probeMigrationTarget: vi.fn(async () => ({ status: 'free' })), })); -type Command = (typeof import('./organization-ownership.js'))['OrganizationOwnershipCeremony']; +type Command = (typeof import('./organization-ownership.js'))['default']; let MigrateOrganizationOwnership: Command; // The command's module graph is large, and oclif reads the package's config on // the first run: both are paid outside the 5 s default (CASE_TIMEOUT_MS). beforeAll(async () => { - ({ OrganizationOwnershipCeremony: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); + ({ default: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); }, 60_000); const CASE_TIMEOUT_MS = 60_000; diff --git a/packages/cli/src/commands/migrate/organization-ownership.declared-rule-mirrors.integration.test.ts b/packages/cli/src/commands/migrate/organization-ownership.declared-rule-mirrors.integration.test.ts index 96eb4e942f2..e5d14c95370 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.declared-rule-mirrors.integration.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.declared-rule-mirrors.integration.test.ts @@ -57,7 +57,7 @@ import { DECLARED_SHARING_RULE_SEEDER_RUNS } from '../../utils/organization-owne import '@objectstack/objectql'; import '@objectstack/metadata-protocol'; -type Command = (typeof import('./organization-ownership.js'))['OrganizationOwnershipCeremony']; +type Command = (typeof import('./organization-ownership.js'))['default']; let MigrateOrganizationOwnership: Command; const CLI_ROOT = fileURLToPath(new URL('../../..', import.meta.url)); @@ -158,7 +158,7 @@ describe('os migrate organization-ownership on a database whose packages declare savedEnv[key] = process.env[key]; delete process.env[key]; } - ({ OrganizationOwnershipCeremony: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); + ({ default: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); dir = mkdtempSync(join(tmpdir(), 'os-ceremony-declared-rules-')); file = join(dir, 'fixture.db'); // The project the command runs in: the ceremony boots ITS composition, so diff --git a/packages/cli/src/commands/migrate/organization-ownership.dialect.test.ts b/packages/cli/src/commands/migrate/organization-ownership.dialect.test.ts index 7a4c21eeb45..7e0f8266b1b 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.dialect.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.dialect.test.ts @@ -44,13 +44,13 @@ vi.mock('../../utils/organization-ownership-apply.js', async (importOriginal) => applyOrganizationOwnership: apply, })); -type Command = (typeof import('./organization-ownership.js'))['OrganizationOwnershipCeremony']; +type Command = (typeof import('./organization-ownership.js'))['default']; let MigrateOrganizationOwnership: Command; // The command's module graph is large, and oclif reads the package's config on // the first run: both are paid outside the 5 s default (CASE_TIMEOUT_MS). beforeAll(async () => { - ({ OrganizationOwnershipCeremony: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); + ({ default: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); }, 60_000); const CASE_TIMEOUT_MS = 60_000; diff --git a/packages/cli/src/commands/migrate/organization-ownership.fate-supplement.integration.test.ts b/packages/cli/src/commands/migrate/organization-ownership.fate-supplement.integration.test.ts index fd185cdd090..d300a7b90cf 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.fate-supplement.integration.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.fate-supplement.integration.test.ts @@ -34,13 +34,13 @@ vi.mock('../../utils/organization-ownership-apply.js', async (importOriginal) => applyOrganizationOwnership: apply, })); -type Command = (typeof import('./organization-ownership.js'))['OrganizationOwnershipCeremony']; +type Command = (typeof import('./organization-ownership.js'))['default']; let MigrateOrganizationOwnership: Command; // The command's module graph is large, and oclif reads the package's config on // the first run: both are paid outside the 5 s default (CASE_TIMEOUT_MS). beforeAll(async () => { - ({ OrganizationOwnershipCeremony: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); + ({ default: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); }, 60_000); const CASE_TIMEOUT_MS = 60_000; diff --git a/packages/cli/src/commands/migrate/organization-ownership.journal-mode.integration.test.ts b/packages/cli/src/commands/migrate/organization-ownership.journal-mode.integration.test.ts index 9a0b629c3e3..e09e732bb5f 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.journal-mode.integration.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.journal-mode.integration.test.ts @@ -32,13 +32,13 @@ import { JOURNAL_MODE_SWITCH_NOTE } from '../../utils/sqlite-occupancy.js'; import '@objectstack/runtime'; import '@objectstack/metadata-protocol'; -type Command = (typeof import('./organization-ownership.js'))['OrganizationOwnershipCeremony']; +type Command = (typeof import('./organization-ownership.js'))['default']; let MigrateOrganizationOwnership: Command; // The command's module graph is large, and oclif reads the package's config on // the first run: both are paid outside the case timeout. beforeAll(async () => { - ({ OrganizationOwnershipCeremony: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); + ({ default: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); }, 60_000); const CASE_TIMEOUT_MS = 60_000; diff --git a/packages/cli/src/commands/migrate/organization-ownership.no-organization.integration.test.ts b/packages/cli/src/commands/migrate/organization-ownership.no-organization.integration.test.ts index 7f95510d87a..661896a2e69 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.no-organization.integration.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.no-organization.integration.test.ts @@ -61,7 +61,7 @@ import '@objectstack/objectql'; import '@objectstack/metadata-protocol'; import '@objectstack/plugin-auth'; -type Command = (typeof import('./organization-ownership.js'))['OrganizationOwnershipCeremony']; +type Command = (typeof import('./organization-ownership.js'))['default']; let MigrateOrganizationOwnership: Command; const CLI_ROOT = fileURLToPath(new URL('../../..', import.meta.url)); @@ -162,7 +162,7 @@ describe('os migrate organization-ownership on a single-posture database that ho savedEnv[key] = process.env[key]; delete process.env[key]; } - ({ OrganizationOwnershipCeremony: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); + ({ default: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); dir = mkdtempSync(join(tmpdir(), 'os-ceremony-no-organization-')); file = join(dir, 'fixture.db'); diff --git a/packages/cli/src/commands/migrate/organization-ownership.occupancy.integration.test.ts b/packages/cli/src/commands/migrate/organization-ownership.occupancy.integration.test.ts index 13832f2173f..179a65dfaa8 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.occupancy.integration.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.occupancy.integration.test.ts @@ -40,13 +40,13 @@ vi.mock('../../utils/schema-migrate.js', async (importOriginal) => ({ bootSchemaStack: boot, })); -type Command = (typeof import('./organization-ownership.js'))['OrganizationOwnershipCeremony']; +type Command = (typeof import('./organization-ownership.js'))['default']; let MigrateOrganizationOwnership: Command; // The command's module graph is large, and oclif reads the package's config on // the first run: both are paid outside the case timeout. beforeAll(async () => { - ({ OrganizationOwnershipCeremony: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); + ({ default: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); }, 60_000); const CASE_TIMEOUT_MS = 60_000; diff --git a/packages/cli/src/commands/migrate/organization-ownership.release-refusal.test.ts b/packages/cli/src/commands/migrate/organization-ownership.release-refusal.test.ts deleted file mode 100644 index d700cba367b..00000000000 --- a/packages/cli/src/commands/migrate/organization-ownership.release-refusal.test.ts +++ /dev/null @@ -1,163 +0,0 @@ -// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. - -/** - * `os migrate organization-ownership` refuses in 18.0, plan and apply alike - * (ruling record 6259608309 on the v18 epic, item 1). 18.0 supports fresh - * installs only. Its ceremony is knowingly incomplete, and an incomplete - * ceremony must never certify an old database. - * - * The refusal comes first, so it reads nothing. These cases set every input - * the ceremony would judge before its boot to a value the ceremony refuses: - * an unrecognised posture, a fate supplement that does not exist, a plan file - * that does not exist. They also stub the occupancy probe and the boot so that - * either one throws. Only the release refusal can answer. - * - * The ceremony itself (`OrganizationOwnershipCeremony`) is pinned by the - * sibling `organization-ownership.*.test.ts` files, which run it directly. - */ - -import { afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'; -import { existsSync, mkdtempSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { fileURLToPath } from 'node:url'; - -const boot = vi.hoisted(() => vi.fn(async () => { - throw new Error('the database was opened'); -})); -const probe = vi.hoisted(() => vi.fn(async () => { - throw new Error('the database was probed'); -})); -vi.mock('../../utils/schema-migrate.js', () => ({ bootSchemaStack: boot })); -vi.mock('../../utils/migrate-occupancy-gate.js', () => ({ - OCCUPANCY_STOP_HINT: 'occupancy hint', - probeMigrationTarget: probe, -})); - -type Module = typeof import('./organization-ownership.js'); -let mod: Module; - -// The command's module graph is large, and oclif reads the package's config on -// the first run: both are paid outside the 5 s default. -beforeAll(async () => { - mod = await import('./organization-ownership.js'); -}, 60_000); - -const CASE_TIMEOUT_MS = 60_000; -const CLI_ROOT = fileURLToPath(new URL('../../..', import.meta.url)); -const DIR = mkdtempSync(join(tmpdir(), 'os-ownership-release-refusal-')); -const DB_FILE = join(DIR, 'v17.db'); -const OUT = join(DIR, 'out.json'); -const MISSING_SUPPLEMENT = join(DIR, 'missing-supplement.json'); -const MISSING_PLAN = join(DIR, 'missing-plan.json'); - -let savedPosture: string | undefined; -beforeEach(() => { - savedPosture = process.env.OS_TENANCY_POSTURE; - // The ceremony refuses an unrecognised posture before anything else; the release refusal never reads it. - process.env.OS_TENANCY_POSTURE = 'not-a-posture'; -}); -afterEach(() => { - if (savedPosture === undefined) delete process.env.OS_TENANCY_POSTURE; - else process.env.OS_TENANCY_POSTURE = savedPosture; - boot.mockClear(); - probe.mockClear(); - vi.restoreAllMocks(); -}); - -async function run(argv: string[]): Promise<{ stdout: string; exitCode: unknown }> { - const chunks: string[] = []; - const savedExit = process.exitCode; - vi.spyOn(process.stdout, 'write').mockImplementation(((chunk: unknown, ...rest: unknown[]) => { - chunks.push(String(chunk)); - const cb = rest.find((a) => typeof a === 'function') as (() => void) | undefined; - if (cb) cb(); - return true; - }) as typeof process.stdout.write); - vi.spyOn(process.stderr, 'write').mockImplementation((() => true) as typeof process.stderr.write); - vi.spyOn(console, 'log').mockImplementation((...args: unknown[]) => { - chunks.push(`${args.map(String).join(' ')}\n`); - }); - let exitCode: unknown; - try { - await mod.default.run(argv, { root: CLI_ROOT }); - exitCode = process.exitCode; - } catch (error) { - const oclif = (error as { oclif?: { exit?: unknown } }).oclif; - if (oclif === undefined) throw error; - exitCode = oclif.exit; - } - process.exitCode = savedExit; - vi.restoreAllMocks(); - return { stdout: chunks.join(''), exitCode }; -} - -function expectNothingTouched(): void { - expect(boot).not.toHaveBeenCalled(); - expect(probe).not.toHaveBeenCalled(); - expect(existsSync(DB_FILE), 'the database file was brought into existence').toBe(false); - expect(existsSync(OUT), 'a plan or report file was written').toBe(false); -} - -describe('os migrate organization-ownership — refused in 18.0', () => { - it('the plan refuses with NOT_IMPLEMENTED, exit 1, reading and writing nothing', async () => { - const { stdout, exitCode } = await run([ - '--database-url', `file:${DB_FILE}`, '--out', OUT, '--fate-supplement', MISSING_SUPPLEMENT, '--json', - ]); - expect(JSON.parse(stdout.trim())).toEqual({ - error: 'plan_refused', - reason: 'upgrade-unavailable', - code: 'NOT_IMPLEMENTED', - detail: mod.ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE, - }); - expect(exitCode).toBe(1); - expectNothingTouched(); - }, CASE_TIMEOUT_MS); - - it('the apply refuses with NOT_IMPLEMENTED, exit 1, reading and writing nothing', async () => { - const { stdout, exitCode } = await run([ - '--apply', '--plan-file', MISSING_PLAN, '--backup-taken', - '--database-url', `file:${DB_FILE}`, '--out', OUT, '--fate-supplement', MISSING_SUPPLEMENT, '--json', - ]); - expect(JSON.parse(stdout.trim())).toMatchObject({ - error: 'apply_refused', - reason: 'upgrade-unavailable', - code: 'NOT_IMPLEMENTED', - }); - expect(exitCode).toBe(1); - expectNothingTouched(); - }, CASE_TIMEOUT_MS); - - it('without --json it prints the refusal with its code and exits 1', async () => { - const { stdout, exitCode } = await run(['--database-url', `file:${DB_FILE}`, '--out', OUT]); - expect(stdout).toContain('✗'); - expect(stdout).toContain(mod.ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE); - expect(stdout).toContain('(NOT_IMPLEMENTED)'); - expect(exitCode).toBe(1); - expectNothingTouched(); - }, CASE_TIMEOUT_MS); - - it('the refusal names the release that will carry the upgrade and what 18.0 supports', () => { - expect(mod.ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE).toMatch(/^os migrate organization-ownership is not available in 18\.0\./); - expect(mod.ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE).toContain('fresh installs only'); - expect(mod.ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE).toContain('an 18.x release'); - }); - - it('its --help examples show the refusal: none is a plan or an apply 18.0 refuses on every database', () => { - const examples = mod.default.examples.map((e) => (typeof e === 'string' ? { command: e, description: '' } : e)); - expect(examples.length).toBeGreaterThan(0); - for (const { command, description } of examples) { - expect(command).toMatch(/migrate organization-ownership/); - expect(command).not.toMatch(/--out|--apply|--plan-file|--fate-supplement/); - expect(description).toContain('NOT_IMPLEMENTED'); - } - // The ceremony class keeps its own examples, for the 18.x release that registers it. - expect(mod.OrganizationOwnershipCeremony.examples.some((e) => String(e).includes('--apply --plan-file'))).toBe(true); - }); - - it('the registered command keeps the ceremony class whole underneath: same flags, its own run', () => { - expect(Object.getPrototypeOf(mod.default)).toBe(mod.OrganizationOwnershipCeremony); - expect(mod.default.flags).toBe(mod.OrganizationOwnershipCeremony.flags); - expect(mod.default.prototype.run).not.toBe(mod.OrganizationOwnershipCeremony.prototype.run); - }); -}); diff --git a/packages/cli/src/commands/migrate/organization-ownership.remote-face.integration.test.ts b/packages/cli/src/commands/migrate/organization-ownership.remote-face.integration.test.ts index 46889c5777c..ad7fa489d5e 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.remote-face.integration.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.remote-face.integration.test.ts @@ -59,7 +59,7 @@ import { TursoDriver } from '@objectstack/driver-turso'; import { ORGANIZATION_OWNERSHIP_MIGRATION_ID } from '@objectstack/spec/system'; import { bootSchemaStack } from '../../utils/schema-migrate.js'; import { isExitSignal } from '../../utils/format.js'; -import { OrganizationOwnershipCeremony as MigrateOrganizationOwnership } from './organization-ownership.js'; +import MigrateOrganizationOwnership from './organization-ownership.js'; // Paid at module load, not inside a clocked case (`check:test-source-alias`). import '@objectstack/runtime'; diff --git a/packages/cli/src/commands/migrate/organization-ownership.remote-posture.test.ts b/packages/cli/src/commands/migrate/organization-ownership.remote-posture.test.ts index 5ae3f969d55..fba272b1e5d 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.remote-posture.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.remote-posture.test.ts @@ -31,11 +31,11 @@ vi.mock('../../utils/migrate-occupancy-gate.js', () => ({ probeMigrationTarget: vi.fn(async () => ({ status: 'free' })), })); -type Command = (typeof import('./organization-ownership.js'))['OrganizationOwnershipCeremony']; +type Command = (typeof import('./organization-ownership.js'))['default']; let MigrateOrganizationOwnership: Command; beforeAll(async () => { - ({ OrganizationOwnershipCeremony: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); + ({ default: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); }, 60_000); const CASE_TIMEOUT_MS = 60_000; diff --git a/packages/cli/src/commands/migrate/organization-ownership.report-file.integration.test.ts b/packages/cli/src/commands/migrate/organization-ownership.report-file.integration.test.ts index 6db272bc3ef..2e3756a30bd 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.report-file.integration.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.report-file.integration.test.ts @@ -53,7 +53,7 @@ vi.mock('../../utils/schema-migrate.js', async (importOriginal) => { }; }); -type Command = (typeof import('./organization-ownership.js'))['OrganizationOwnershipCeremony']; +type Command = (typeof import('./organization-ownership.js'))['default']; let MigrateOrganizationOwnership: Command; const HERE = dirname(fileURLToPath(import.meta.url)); @@ -114,6 +114,15 @@ async function bootServed(projectDir: string, dbFile: string): Promise { await kernel.use(new PlatformObjectsPlugin() as any); await runtime.start(); await kernel.shutdown(); + // This boot of THIS tree created the store from empty, so it wrote this + // release's birth marker; the database it stands in for (an earlier + // release's) carries none, and the cases below read the marker's absence. + const db = new Database(dbFile); + try { + db.prepare('DELETE FROM sys_migration WHERE id = ?').run(ORGANIZATION_OWNERSHIP_MIGRATION_ID); + } finally { + db.close(); + } } interface Prepared { @@ -182,7 +191,7 @@ const apply = (prepared: Prepared, out: string): Promise => ]); beforeAll(async () => { - ({ OrganizationOwnershipCeremony: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); + ({ default: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); for (const key of [...OVERRIDING_ENV, 'NODE_ENV', 'OS_SECRET_KEY'] as const) savedEnv[key] = process.env[key]; for (const key of OVERRIDING_ENV) delete process.env[key]; // The deployed posture: no telemetry sibling, and the crypto provider's key. diff --git a/packages/cli/src/commands/migrate/organization-ownership.report-reservation.test.ts b/packages/cli/src/commands/migrate/organization-ownership.report-reservation.test.ts index 89b2f6d505c..cd533c69ccf 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.report-reservation.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.report-reservation.test.ts @@ -28,11 +28,11 @@ vi.mock('../../utils/migrate-occupancy-gate.js', () => ({ probeMigrationTarget: vi.fn(async () => ({ status: 'free' })), })); -type Command = (typeof import('./organization-ownership.js'))['OrganizationOwnershipCeremony']; +type Command = (typeof import('./organization-ownership.js'))['default']; let MigrateOrganizationOwnership: Command; beforeAll(async () => { - ({ OrganizationOwnershipCeremony: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); + ({ default: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); }, 60_000); const CASE_TIMEOUT_MS = 60_000; diff --git a/packages/cli/src/commands/migrate/organization-ownership.ts b/packages/cli/src/commands/migrate/organization-ownership.ts index 0eac0d5700b..ea223b9e5d7 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.ts @@ -204,11 +204,10 @@ class ReportFileReservation { } /** - * The v18 organization-ownership ceremony behind `os migrate - * organization-ownership` — in 18.0 the registered command refuses before - * this class runs (see {@link MigrateOrganizationOwnership} at the end of this - * file). This class is the ceremony itself, kept whole for the 18.x release - * that carries the upgrade path; its tests run it directly. + * `os migrate organization-ownership` — the v18 organization-ownership + * ceremony, the upgrade path of a v17 or an 18.0 database (18.x step 6): the + * boot guard refuses such a database until this ceremony has verified it at the + * current ownership shape generation. * * The ceremony's preflight (ADR-0131 D10, ceremony item 1): per table, the fate, * the row counts each fate will touch, the rows whose owner cannot be derived @@ -261,8 +260,10 @@ class ReportFileReservation { * through plugin-auth's boot invariant, as the plan's warnings say; ADR-0131 * D3), the authored catalog conversion, * the promotion of organization customizations and public-form withdrawals, - * the template promotion, the settings global-rung move, the verified id→name - * rewrite, mirror deletion, column drops — through `../../utils/organization-ownership-apply.ts`, + * the template promotion, the settings global-rung move, the sibling moves, + * the settlement of declared webhooks and of declared sharing rules' stored + * copies, the verified id→name rewrite, mirror deletion, column drops, NOT + * NULL on `organization_id` — through `../../utils/organization-ownership-apply.ts`, * checkpointing each finished step in the deployment ledger (`sys_migration`, * row {@link ORGANIZATION_OWNERSHIP_MIGRATION_ID}) so an interrupted run * resumes where it stopped, and ends with the post-check. The ledger row is @@ -316,16 +317,16 @@ class ReportFileReservation { * the drops go through the remote face's own drift detector and migration * door (`@objectstack/driver-turso`). */ -export class OrganizationOwnershipCeremony extends Command { +export default class OrganizationOwnershipCeremony extends Command { static override description = 'The v18 organization-ownership ceremony (ADR-0131 D10). Without --apply, the read-only plan: per table, the fate, ' + 'the row counts, the rows whose owner cannot be derived (by id), and which tables will receive NOT NULL, written ' + 'to a file — never a row or a column changed; it refuses, naming it, any table it cannot enumerate. With --apply ' + '(the reviewed plan file in --plan-file, and a backup), only that plan: attribution, the authored catalog ' + 'conversion, the promotion of organization customizations and form withdrawals, the template promotion, the ' + - 'settings move, the ' + - 'verified id-to-name rewrite, mirror deletion and column drops, in that order, ' + - 'resumable from a recorded checkpoint, then the post-check.'; + 'settings move, the sibling moves (global settings audit, identity mail), the settlement of declared webhooks and ' + + 'of declared sharing rules\' stored copies, the verified id-to-name rewrite, mirror deletion, column drops and ' + + 'NOT NULL on organization_id, in that order, resumable from a recorded checkpoint, then the post-check.'; static override examples: Command.Example[] = [ '$ os migrate organization-ownership', @@ -357,8 +358,9 @@ export class OrganizationOwnershipCeremony extends Command { }), apply: Flags.boolean({ description: - 'Perform the ceremony: attribution, the verified id-to-name rewrite, mirror deletion, column drops, then the ' + - 'post-check. Idempotent; an interrupted run resumes from its recorded checkpoint. Requires a backup', + 'Perform the ceremony: attribution, the moves and settlements, the verified id-to-name rewrite, mirror deletion, ' + + 'column drops and NOT NULL, then the post-check. Idempotent; an interrupted run resumes from its recorded ' + + 'checkpoint. Requires a backup', default: false, }), 'plan-file': Flags.string({ @@ -830,6 +832,60 @@ export class OrganizationOwnershipCeremony extends Command { return judgeDeclaredSharingRules(items); }, }, + declaredWebhooks: { + // [18.x step 5a] What plugin-webhooks serves: the registry's declared + // webhooks, else the metadata service's, each parsed by the spec's + // own schema — an item the plugin cannot parse is not served, so a + // copy is never settled onto it. + list: async () => { + const { WebhookSchema } = await import('@objectstack/spec/automation'); + const registry = (engine.registry ?? (engine as { _registry?: unknown })._registry) as + | { listItems?(type: string): unknown[] | undefined } + | undefined; + let items: unknown[] = registry?.listItems?.('webhook') ?? []; + if (items.length === 0) { + const metadata = kernel.getService('metadata') as { list?(type: string): unknown } | undefined; + const listed = await metadata?.list?.('webhook'); + items = Array.isArray(listed) ? listed : []; + } + const out = new Map(); + for (const item of items) { + const parsed = WebhookSchema.safeParse(item); + if (!parsed.success || out.has(parsed.data.name)) continue; + const packageId = (item as { _packageId?: unknown })._packageId; + out.set(parsed.data.name, typeof packageId === 'string' && packageId ? packageId : null); + } + return [...out].map(([name, packageId]) => ({ name, packageId })); + }, + }, + notNull: { + // [18.x step 5d] NOT NULL through the drift apply door of the driver + // that holds the table, for a table this composition declares. + apply: async (columns) => { + const { StorageNameMapping } = await import('@objectstack/spec/system'); + const { rotationBaseOf } = await import('../../utils/unmanaged-tables.js'); + const { applyNotNullThroughDrift } = await import('../../utils/organization-ownership-not-null.js'); + const declared = new Set( + (liveStack.allObjects() as Array<{ name?: unknown } | null>) + .map((o) => o?.name) + .filter((name): name is string => typeof name === 'string' && name.length > 0) + .map((name) => StorageNameMapping.resolveTableName({ name })), + ); + return applyNotNullThroughDrift( + { + isDeclared: (table) => declared.has(rotationBaseOf(table) ?? table), + shardBaseOf: (table) => rotationBaseOf(table) ?? null, + driverFor: (database) => + (database === undefined + ? liveStack.driver + : database === TELEMETRY_DATASOURCE && liveStack.telemetryDatabase + ? liveStack.drivers[1] + : undefined) as never, + }, + columns, + ); + }, + }, door: { save: async (type, name, body, target) => { if (typeof protocol?.saveMetaItem !== 'function') { @@ -1093,73 +1149,3 @@ export class OrganizationOwnershipCeremony extends Command { if (verdictCode !== 0) this.exit(1); } } - -/** - * What `os migrate organization-ownership` answers in 18.0, plan and apply - * alike (ruling record 6259608309 on the v18 epic, item 1). - * - * 18.0 supports fresh installs only. Its ceremony is knowingly incomplete: - * several fates, and the moves they carry, were deferred to 18.x. An - * incomplete ceremony must never certify an old database, because the boot - * guard (`OrganizationOwnershipBootGuardPlugin`) would then serve it. So the - * registered command refuses before it reads anything. It opens no database, - * reads no plan, supplement or report file, writes nothing, and boots nothing. - * The code is the standard `NOT_IMPLEMENTED` (ADR-0112): the capability is - * not in this release. No flag and no environment variable lifts the refusal. - */ -export const ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE = - 'os migrate organization-ownership is not available in 18.0. 18.0 supports fresh installs only; the v17 → v18 ' + - 'upgrade path ships in an 18.x release. Keep a v17 database on a 17.x release until then. Nothing was read or written.'; - -/** The ADR-0112 code the 18.0 refusal carries: the standard code for a capability this release does not implement. */ -export const ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE_CODE = 'NOT_IMPLEMENTED' as const; - -/** - * `os migrate organization-ownership` in 18.0: the ceremony's flags and help, - * and a refusal instead of the ceremony - * ({@link ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE}). The 18.x release that - * carries the upgrade path deletes this class and registers the ceremony. - */ -export default class MigrateOrganizationOwnership extends OrganizationOwnershipCeremony { - static override description = - 'Refused in 18.0, which supports fresh installs only: the v17 → v18 upgrade path ships in an 18.x release. ' + - OrganizationOwnershipCeremony.description; - - /** - * The ceremony's own examples show a plan and an apply, which 18.0 refuses on - * every database, fresh or not, so `--help` shows what an invocation answers - * in this release instead. The ceremony class keeps its examples for the 18.x - * release that registers it. - */ - static override examples: Command.Example[] = [ - { - description: - 'Refused in 18.0, plan and apply alike: exit 1 with NOT_IMPLEMENTED, nothing read or written. Keep a v17 ' + - 'database on a 17.x release until the 18.x release that carries the upgrade path', - command: '<%= config.bin %> migrate organization-ownership', - }, - { - description: 'The same refusal as JSON: error, reason upgrade-unavailable, code NOT_IMPLEMENTED, and the detail', - command: '<%= config.bin %> migrate organization-ownership --json', - }, - ]; - - override async run(): Promise { - const { flags } = await this.parse(MigrateOrganizationOwnership); - if (flags.json) { - await emitJson( - { - error: flags.apply ? 'apply_refused' : 'plan_refused', - reason: 'upgrade-unavailable', - code: ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE_CODE, - detail: ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE, - }, - 1, - { compact: true }, - ); - return; - } - printError(`${ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE} (${ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE_CODE})`); - this.exit(1); - } -} diff --git a/packages/cli/src/commands/migrate/sqlite-wasm.integration.test.ts b/packages/cli/src/commands/migrate/sqlite-wasm.integration.test.ts index 7dde27230b6..4b1f4dc4d0d 100644 --- a/packages/cli/src/commands/migrate/sqlite-wasm.integration.test.ts +++ b/packages/cli/src/commands/migrate/sqlite-wasm.integration.test.ts @@ -36,7 +36,7 @@ import { fileURLToPath } from 'node:url'; import { isExitSignal } from '../../utils/format.js'; import MigrateApply from './apply.js'; import MigratePlan from './plan.js'; -import { OrganizationOwnershipCeremony as MigrateOrganizationOwnership } from './organization-ownership.js'; +import MigrateOrganizationOwnership from './organization-ownership.js'; // [#10126] Pay the first transform of the dist-resolved workspace deps the // commands reach through dynamic `import()`s at MODULE LOAD, not inside a case. diff --git a/packages/cli/src/utils/dev-organization-ownership-upgrade.test.ts b/packages/cli/src/utils/dev-organization-ownership-upgrade.test.ts index 16980324527..932c82412a4 100644 --- a/packages/cli/src/utils/dev-organization-ownership-upgrade.test.ts +++ b/packages/cli/src/utils/dev-organization-ownership-upgrade.test.ts @@ -86,8 +86,9 @@ describe('os dev — the in-place upgrade of a v17-shape development database', expect(result.lines[0]).toContain('✗ Refused (sys_mystery)'); expect(result.lines.at(-1)).toContain('nothing was changed'); expect(result.lines.at(-1)).toContain(BACKUP()); - expect(result.lines.at(-1)).toContain('17.x release'); - expect(result.lines.at(-1)).not.toContain('--plan-file'); + // The ceremony is open (18.x step 6): the way on is running it by hand. + expect(result.lines.at(-1)).toContain('os migrate organization-ownership --dev'); + expect(result.lines.at(-1)).not.toContain('17.x'); }); it('a blocked step boots nothing: each blocker is printed, and how to resume', () => { @@ -120,15 +121,13 @@ describe('os dev — the in-place upgrade of a v17-shape development database', expect(result.lines.at(-1)).toContain(`--apply --plan-file ${join(BACKUP(), 'plan.json')} --backup-taken`); }); - it('a database that is not a local SQLite file is not upgraded in place: 18.0\'s position is named instead', () => { + it('a database that is not a local SQLite file is not upgraded in place: the ceremony is named instead', () => { const result = upgradeDevelopmentDatabase(input(fakeRun({}), { sqliteFile: null, databaseUrl: 'postgres://db/app' })); expect(result.ok).toBe(false); expect(calls).toEqual([]); expect(existsSync(join(dir, 'backups'))).toBe(false); - expect(result.lines[0]).toContain('17.x release'); - // 18.0 refuses `os migrate organization-ownership` on every database: the way on names no step of it. - expect(result.lines[0]).not.toContain('os migrate organization-ownership'); - expect(result.lines[0]).not.toContain('--plan-file'); + expect(result.lines[0]).toContain('os migrate organization-ownership --dev'); + expect(result.lines[0]).not.toContain('17.x'); }); it('the IPC message is recognised by its type alone', () => { diff --git a/packages/cli/src/utils/dev-organization-ownership-upgrade.ts b/packages/cli/src/utils/dev-organization-ownership-upgrade.ts index 47a5a0318ec..c9f652245df 100644 --- a/packages/cli/src/utils/dev-organization-ownership-upgrade.ts +++ b/packages/cli/src/utils/dev-organization-ownership-upgrade.ts @@ -118,15 +118,11 @@ function refusalLines(logFile: string): string[] { * Synchronous on purpose — nothing else may open the files while it runs. */ export function upgradeDevelopmentDatabase(input: DevelopmentUpgradeInput): DevelopmentUpgradeResult { - // The way on, when os dev does not upgrade the database itself. 18.0 supports - // fresh installs only, and its `os migrate organization-ownership` refuses on - // every database (`ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE`), so this names - // no step of that command: it says what 18.0 supports for a v17 database, in - // the boot refusal's terms. The 18.x release that registers the ceremony names - // its steps here again. + // The way on, when os dev does not upgrade the database itself: the ceremony, + // run by hand (18.x step 6 opened it). const manual = - '18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 development database on a ' + - '17.x release until the 18.x release that carries the upgrade path, and upgrade it there.'; + 'Run `os migrate organization-ownership --dev` (review its plan), then the same command with --apply ' + + '--plan-file FILE --backup-taken, and start os dev again.'; if (!input.sqliteFile) { return { ok: false, diff --git a/packages/cli/src/utils/organization-ownership-apply.declared-rule-mirrors.test.ts b/packages/cli/src/utils/organization-ownership-apply.declared-rule-mirrors.test.ts index b2f5abbf81e..bc68b8c32fa 100644 --- a/packages/cli/src/utils/organization-ownership-apply.declared-rule-mirrors.test.ts +++ b/packages/cli/src/utils/organization-ownership-apply.declared-rule-mirrors.test.ts @@ -9,8 +9,11 @@ * * The fixture is HotCRM's measured shape: eleven uncustomized declared rules * the seeder wrote organization-less (`managed_by 'package'`, `customized 0`), - * beside an organization-authored rule and a customized declared rule with no - * organization, plus a column-drop table still carrying `organization_id` — + * beside an organization-authored rule and a customized seeded rule with no + * organization whose declaration its package has since dropped (a customized + * copy of a rule still declared is the `declared-rule-copies` step's, pinned in + * `organization-ownership-apply.declared-rule-copies.test.ts`), plus a + * column-drop table still carrying `organization_id` — * and the grants those rules materialised, each naming its rule row by id. * * - The mirror-deletion step renames every grant naming a mirror to the @@ -123,8 +126,8 @@ function ledger(): CeremonyLedgerPort & { verdicts: Verdict[] } { }; } -/** Every rule the fixture's packages declare: the eleven mirrors' names and the customized one's. */ -const DECLARED = [...Array.from({ length: MIRRORS }, (_, i) => `share_declared_${i + 1}`), 'share_custom']; +/** Every rule the fixture's packages declare: the eleven mirrors' names (the customized rule's is no longer declared). */ +const DECLARED = Array.from({ length: MIRRORS }, (_, i) => `share_declared_${i + 1}`); /** * Plan, then apply that reviewed plan, under `single`, as the command does. @@ -352,11 +355,26 @@ describe('the declared sharing-rule mirrors, once their seeder has retired', () const l = ledger(); const report = await apply(l, undefined, null); expect(report.complete).toBe(false); + // The customized seeded rule stops it first: whether it is a copy of a + // declared rule is read from the same registry. + expect(report.steps.find((s) => s.step === 'declared-rule-copies')).toMatchObject({ + status: 'blocked', + blockers: [expect.stringMatching(/no registry to read them from/)], + }); + expect(report.steps.find((s) => s.step === 'mirror-deletion')).toMatchObject({ status: 'not-reached' }); + expect(rows(`SELECT COUNT(*) AS n FROM sys_sharing_rule`)).toEqual([{ n: MIRRORS + 2 }]); + }); + + it('with no customized copy to stop it first, the mirror-deletion step blocks on the missing registry itself', async () => { + db.exec(`DELETE FROM sys_sharing_rule WHERE id = 'rule_customized'`); + const report = await apply(ledger(), undefined, null); + expect(report.complete).toBe(false); + expect(report.steps.find((s) => s.step === 'declared-rule-copies')).toMatchObject({ status: 'done' }); expect(report.steps.find((s) => s.step === 'mirror-deletion')).toMatchObject({ status: 'blocked', blockers: [expect.stringMatching(/no registry to read the declared sharing rules from/)], }); - expect(rows(`SELECT COUNT(*) AS n FROM sys_sharing_rule`)).toEqual([{ n: MIRRORS + 2 }]); + expect(rows(`SELECT COUNT(*) AS n FROM sys_sharing_rule`)).toEqual([{ n: MIRRORS + 1 }]); }); it('an organization-stamped mirror (a walled seeder\'s copy) goes too, its grant renamed; a customized one stays', async () => { diff --git a/packages/cli/src/utils/organization-ownership-apply.generation-1.test.ts b/packages/cli/src/utils/organization-ownership-apply.generation-1.test.ts new file mode 100644 index 00000000000..a58296de0e4 --- /dev/null +++ b/packages/cli/src/utils/organization-ownership-apply.generation-1.test.ts @@ -0,0 +1,433 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The four ceremony steps of ownership shape generation 1 (18.x steps 5a–5d), + * pinned in the ceremony's own terms, over an 18.0-SHAPED database: every row + * stamped with the Default Organization (line fact F3), so each step must + * select by shape — a NULL selector would find nothing here. + * + * - `sibling-moves` (5b): the global settings-audit rows and identity mail + * (reset by `related_object`, the magic link by its sign-in link) move id + * for id into their siblings, the organization invitation stays; a queued + * 18.0 mail job is pointed at the sibling first; the activity rows about a + * moved row name its new object. + * - `declared-webhooks` (5a): a pristine copy is deleted, its deliveries + * re-sourced to `declared:NAME` and its orphaned cipher deleted; a + * switch-only edit becomes the declaration's activation row; any other + * edit becomes a clone under a new name, the declaration switched off, its + * deliveries still its own; a seeded row no package declares stays. + * - `declared-rule-copies` (5c, 9c): a customized copy and an admin rule under + * a declared name become clones, the declaration switched off, their grants + * kept; the stale name-keyed grant goes. + * - `not-null` (5d, 8e): every cleared table receives NOT NULL through the + * port, after the column drops. + * + * Plus: the checkpoint carries this release's shape generation; a walled + * posture blocks on a customized copy rather than switch a declaration off + * for every organization; every step is idempotent. + */ + +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import Database from 'better-sqlite3'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { ORGANIZATION_OWNERSHIP_SHAPE_GENERATION } from '@objectstack/spec/system'; +import { declaredWebhookRefId } from '@objectstack/plugin-webhooks'; +import { buildOrganizationOwnershipPlan, inventoryDigest } from './organization-ownership-plan.js'; +import { + applyOrganizationOwnership, + CEREMONY_STEPS, + DECLARED_WEBHOOK_REF_PREFIX, + reviewedConflictsOf, + type ApplyExecutor, + type CeremonyCheckpoint, + type CeremonyLedgerPort, + type ColumnDropPort, + type NotNullColumn, + type NotNullPort, + type OrganizationOwnershipApplyReport, +} from './organization-ownership-apply.js'; + +const ORG = 'org_default'; + +let dir: string; +let db: Database.Database; +let writes: string[]; +let tightened: NotNullColumn[]; + +function executor(): ApplyExecutor { + return { + dialect: 'sqlite', + client: 'better-sqlite3', + async query(sql, params = []) { + return db.prepare(sql).all(...(params as unknown[])) as Record[]; + }, + async execute(sql, params = []) { + writes.push(sql); + db.prepare(sql).run(...(params as unknown[])); + }, + }; +} + +const columnDrops: ColumnDropPort = { + async drop(columns) { + return columns.map(({ table, column }) => { + writes.push(`ALTER TABLE ${table} DROP COLUMN ${column}`); + db.exec(`ALTER TABLE "${table}" DROP COLUMN "${column}"`); + return { table, column, outcome: 'dropped' as const }; + }); + }, +}; + +/** The schema door: records what it was asked, and answers `applied`. */ +const notNull: NotNullPort = { + async apply(columns) { + tightened.push(...columns); + return columns.map((c) => ({ ...c, outcome: 'applied' as const })); + }, +}; + +function ledger(): CeremonyLedgerPort & { stored: () => CeremonyCheckpoint | null } { + let stored: CeremonyCheckpoint | null = null; + return { + stored: () => stored, + async read() { + return stored ? (JSON.parse(JSON.stringify(stored)) as CeremonyCheckpoint) : null; + }, + async write(checkpoint) { + stored = JSON.parse(JSON.stringify(checkpoint)) as CeremonyCheckpoint; + }, + }; +} + +/** The webhooks and sharing rules the fixture's packages declare. */ +const WEBHOOKS = [ + { name: 'task_changed', packageId: 'com.example.app' }, + { name: 'deal_won', packageId: 'com.example.app' }, + { name: 'lead_scored', packageId: 'com.example.app' }, +]; +const RULES = ['share_custom', 'share_collides']; + +async function apply( + l: ReturnType = ledger(), + opts: { posture?: string; webhooks?: boolean; rules?: boolean; notNull?: boolean } = {}, +): Promise { + const posture = opts.posture ?? 'single'; + const database = join(dir, 'fixture.db'); + const reviewed = await buildOrganizationOwnershipPlan({ reader: executor(), posture, database }); + return applyOrganizationOwnership({ + executor: executor(), + posture, + database, + reviewedPlan: { + file: 'plan.json', + generatedAt: reviewed.generatedAt, + database, + inventoryDigest: inventoryDigest(), + posture, + conflicts: reviewedConflictsOf(reviewed), + }, + catalog: { async resolve() { return undefined; } }, + ...(opts.webhooks === false ? {} : { declaredWebhooks: { async list() { return WEBHOOKS; } } }), + ...(opts.rules === false + ? {} + : { declaredSharingRules: { async rules() { return RULES.map((name) => ({ name, refused: null, packageId: 'com.example.app' })); } } }), + ...(opts.notNull === false ? {} : { notNull }), + door: { + async save(type, name) { + throw new Error(`no environment save is expected here (${type} ${name})`); + }, + }, + templates: { async find() { return undefined; } }, + views: { async list() { return []; } }, + lock: { async verdict() { return { editable: true }; } }, + columnDrops, + ledger: l, + now: () => new Date('2026-10-19T00:00:00Z'), + }); +} + +const rows = (sql: string): Record[] => db.prepare(sql).all() as Record[]; +const step = (report: OrganizationOwnershipApplyReport, name: string) => report.steps.find((s) => s.step === name)!; + +/** A seeded 18.0 webhook row: the seeder's mapping of the envelope, plus the snapshot it wrote. */ +function seededWebhook(id: string, name: string, over: Record = {}): void { + const envelope = { name, label: `Label ${name}`, object: 'crm_deal', triggers: ['create', 'update'], url: `https://hooks.example/${name}`, method: 'POST', timeoutMs: 30000, isActive: false, _provenance: 'package', _packageId: 'com.example.app', _packageVersion: '1.0.0' }; + const row: Record = { + id, organization_id: ORG, name, label: envelope.label, object_name: 'crm_deal', triggers: JSON.stringify(envelope.triggers), + url: envelope.url, method: 'post', description: null, active: 0, definition_json: JSON.stringify(envelope), + headers_secret: null, signing_secret: null, managed_by: 'package', customized: 0, ...over, + }; + const keys = Object.keys(row); + db.prepare(`INSERT INTO sys_webhook (${keys.join(', ')}) VALUES (${keys.map(() => '?').join(', ')})`).run(...keys.map((k) => row[k])); +} + +beforeEach(() => { + dir = mkdtempSync(join(tmpdir(), 'os-d10-generation-1-')); + db = new Database(join(dir, 'fixture.db')); + writes = []; + tightened = []; + db.exec(` + CREATE TABLE sys_organization (id TEXT PRIMARY KEY, slug TEXT); + INSERT INTO sys_organization VALUES ('${ORG}', 'default'); + CREATE TABLE crm_deal (id TEXT PRIMARY KEY, organization_id TEXT, name TEXT); + INSERT INTO crm_deal VALUES ('deal_1', '${ORG}', 'Acme'); + CREATE TABLE sys_user (id TEXT PRIMARY KEY, email TEXT); + INSERT INTO sys_user VALUES ('usr_b', 'b@example.test'); + + CREATE TABLE sys_metadata_activation (id TEXT PRIMARY KEY, created_at TEXT, updated_at TEXT, metadata_type TEXT, + name TEXT, package_id TEXT, active INTEGER DEFAULT 1, UNIQUE (metadata_type, name)); + + CREATE TABLE sys_setting_audit (id TEXT PRIMARY KEY, created_at TEXT, organization_id TEXT, namespace TEXT, key TEXT, + scope TEXT, action TEXT, new_hash TEXT, encrypted INTEGER); + INSERT INTO sys_setting_audit VALUES + ('aud_tenant', '2026-10-19T12:00:00Z', '${ORG}', 'branding', 'workspace_name', 'tenant', 'set', 'fnv32:1', 0), + ('aud_global', '2026-10-19T12:00:01Z', '${ORG}', 'auth', 'api_key_policy', 'global', 'set', 'fnv32:2', 0), + ('aud_global_enc', '2026-10-19T12:00:02Z', '${ORG}', 'ai', 'gateway_api_key', 'global', 'set', 'hmac:3', 1); + CREATE TABLE sys_platform_setting_audit (id TEXT PRIMARY KEY, created_at TEXT, namespace TEXT, key TEXT, scope TEXT, + action TEXT, new_hash TEXT, encrypted INTEGER); + + CREATE TABLE sys_email (id TEXT PRIMARY KEY, organization_id TEXT, subject TEXT, body_text TEXT, body_html TEXT, + status TEXT, related_object TEXT, related_id TEXT); + INSERT INTO sys_email VALUES + ('mail_invite', '${ORG}', 'You are invited to Default Organization', 'join', NULL, 'sent', 'sys_invitation', 'inv_1'), + ('mail_reset', '${ORG}', 'Reset your password', 'reset at /reset-password/tok', NULL, 'sent', 'sys_user', 'usr_b'), + ('mail_magic', '${ORG}', 'Your sign-in link', 'open https://app.example/api/auth/magic-link/verify?token=abc', NULL, 'pending', NULL, NULL), + ('mail_app', '${ORG}', 'Your invoice', 'see attached', NULL, 'sent', 'crm_deal', 'deal_1'); + CREATE TABLE sys_identity_email (id TEXT PRIMARY KEY, subject TEXT, body_text TEXT, body_html TEXT, status TEXT, + related_object TEXT, related_id TEXT); + + CREATE TABLE sys_job_queue (id TEXT PRIMARY KEY, queue TEXT, idempotency_key TEXT, payload_json TEXT, metadata_json TEXT, status TEXT); + INSERT INTO sys_job_queue VALUES + ('job_magic', 'email.send.async', 'sys_email:mail_magic', '{"rowId":"mail_magic"}', '{"object":"sys_email","rowId":"mail_magic"}', 'pending'), + ('job_app', 'email.send.async', 'sys_email:mail_app', '{"rowId":"mail_app"}', '{"object":"sys_email","rowId":"mail_app"}', 'pending'), + ('job_done', 'email.attachment.reclaim', 'sys_email_attachments:mail_reset', '{"rowId":"mail_reset","keys":[]}', NULL, 'completed'); + + CREATE TABLE sys_activity (id TEXT PRIMARY KEY, organization_id TEXT, object_name TEXT, record_id TEXT, summary TEXT); + INSERT INTO sys_activity VALUES + ('act_reset', '${ORG}', 'sys_email', 'mail_reset', 'mail sent'), + ('act_audit', '${ORG}', 'sys_setting_audit', 'aud_global', 'setting changed'), + ('act_deal', '${ORG}', 'crm_deal', 'deal_1', 'deal updated'); + + CREATE TABLE sys_webhook (id TEXT PRIMARY KEY, organization_id TEXT, name TEXT, label TEXT, object_name TEXT, + triggers TEXT, url TEXT, method TEXT, description TEXT, active INTEGER, definition_json TEXT, headers_secret TEXT, + signing_secret TEXT, managed_by TEXT, customized INTEGER); + CREATE TABLE sys_http_delivery (id TEXT PRIMARY KEY, organization_id TEXT, source TEXT, ref_id TEXT, dedup_key TEXT, status TEXT); + CREATE TABLE sys_secret (id TEXT PRIMARY KEY, organization_id TEXT, namespace TEXT, key TEXT, ciphertext TEXT); + INSERT INTO sys_secret VALUES + ('sec_pristine', '${ORG}', 'sys_webhook', 'signing_secret', 'c1'), + ('sec_setting', '${ORG}', 'ai', 'gateway_api_key', 'c2'); + + CREATE TABLE sys_sharing_rule (id TEXT PRIMARY KEY, organization_id TEXT, name TEXT, object_name TEXT, managed_by TEXT, customized INTEGER); + INSERT INTO sys_sharing_rule VALUES + ('rule_custom', '${ORG}', 'share_custom', 'crm_deal', 'package', 1), + ('rule_collides', '${ORG}', 'share_collides', 'crm_deal', 'admin', 0), + ('rule_own', '${ORG}', 'mine', 'crm_deal', 'admin', 0); + CREATE TABLE sys_record_share (id TEXT PRIMARY KEY, organization_id TEXT, object_name TEXT, record_id TEXT, source TEXT, source_id TEXT); + INSERT INTO sys_record_share VALUES + ('grant_custom', '${ORG}', 'crm_deal', 'deal_1', 'rule', 'rule_custom'), + ('grant_stale', '${ORG}', 'crm_deal', 'deal_1', 'rule', 'share_custom'), + ('grant_own', '${ORG}', 'crm_deal', 'deal_1', 'rule', 'rule_own'); + `); + // A pristine copy holding a cipher; a switch-only edit; an edit beyond the switch; a row no package declares. + seededWebhook('whk_pristine', 'task_changed', { signing_secret: 'secret:sec_pristine' }); + seededWebhook('whk_switched', 'deal_won', { active: 1, customized: 1 }); + seededWebhook('whk_edited', 'lead_scored', { url: 'https://hooks.example/admin-moved-it', customized: 1 }); + seededWebhook('whk_orphan', 'retired_hook'); + db.exec(` + INSERT INTO sys_http_delivery VALUES + ('dlv_1', '${ORG}', 'webhook', 'whk_pristine', 'whk_pristine:e1', 'pending'), + ('dlv_2', '${ORG}', 'webhook', 'whk_switched', 'whk_switched:e2', 'failed'), + ('dlv_3', '${ORG}', 'webhook', 'whk_edited', 'whk_edited:e3', 'sent'), + ('dlv_4', '${ORG}', 'webhook', 'whk_orphan', 'whk_orphan:e4', 'sent'); + `); +}); + +afterEach(() => { + db.close(); + rmSync(dir, { recursive: true, force: true }); +}); + +describe('ownership shape generation 1 — the ceremony steps an 18.0 database owes', () => { + it('runs every step in order, certifies, and records this release\'s shape generation in the checkpoint', async () => { + const l = ledger(); + const report = await apply(l); + expect(report.steps.map((s) => [s.step, s.status])).toEqual(CEREMONY_STEPS.map((s) => [s, 'done'])); + expect([report.complete, report.blocking]).toEqual([true, 0]); + expect(l.stored()?.shapeGeneration).toBe(ORGANIZATION_OWNERSHIP_SHAPE_GENERATION); + // The new steps run where the ruling put them: the moves after settings-move, NOT NULL after the column drops. + expect(CEREMONY_STEPS.indexOf('sibling-moves')).toBe(CEREMONY_STEPS.indexOf('settings-move') + 1); + expect(CEREMONY_STEPS.at(-1)).toBe('not-null'); + expect(CEREMONY_STEPS.indexOf('not-null')).toBeGreaterThan(CEREMONY_STEPS.indexOf('column-drops')); + }); + + it('5b ⭐ the global audit rows and identity mail move id for id — selected by shape, though stamped — and the rest stays', async () => { + const report = await apply(); + expect(step(report, 'sibling-moves')).toMatchObject({ status: 'done', summary: { moved: 4, alreadyMoved: 0, jobsRewritten: 1, activityRepointed: 2 } }); + expect(rows(`SELECT id FROM sys_platform_setting_audit ORDER BY id`).map((r) => r.id)).toEqual(['aud_global', 'aud_global_enc']); + expect(rows(`SELECT id FROM sys_setting_audit`).map((r) => r.id)).toEqual(['aud_tenant']); + expect(rows(`SELECT encrypted, new_hash FROM sys_platform_setting_audit WHERE id = 'aud_global_enc'`)).toEqual([{ encrypted: 1, new_hash: 'hmac:3' }]); + // Identity mail: the reset (a person's mail) and the magic link (its sign-in link); the invitation and app mail stay. + expect(rows(`SELECT id FROM sys_identity_email ORDER BY id`).map((r) => r.id)).toEqual(['mail_magic', 'mail_reset']); + expect(rows(`SELECT id FROM sys_email ORDER BY id`).map((r) => r.id)).toEqual(['mail_app', 'mail_invite']); + }); + + it('5b ⭐ a queued 18.0 mail job names its moved row\'s sibling; jobs of mail that stays, and finished jobs, are untouched', async () => { + await apply(); + expect(rows(`SELECT id, idempotency_key, payload_json, metadata_json FROM sys_job_queue ORDER BY id`)).toEqual([ + { id: 'job_app', idempotency_key: 'sys_email:mail_app', payload_json: '{"rowId":"mail_app"}', metadata_json: '{"object":"sys_email","rowId":"mail_app"}' }, + { id: 'job_done', idempotency_key: 'sys_email_attachments:mail_reset', payload_json: '{"rowId":"mail_reset","keys":[]}', metadata_json: null }, + { + id: 'job_magic', + idempotency_key: 'sys_identity_email:mail_magic', + payload_json: '{"rowId":"mail_magic","object":"sys_identity_email"}', + metadata_json: '{"object":"sys_identity_email","rowId":"mail_magic"}', + }, + ]); + // The job is rewritten BEFORE its row leaves sys_email. + const job = writes.findIndex((w) => /^UPDATE "sys_job_queue"/.test(w)); + const leave = writes.findIndex((w) => /^DELETE FROM "sys_email"/.test(w)); + expect(job).toBeGreaterThan(-1); + expect(job).toBeLessThan(leave); + }); + + it('5b the activity rows about a moved row name its new object; the rest of the timeline is untouched', async () => { + await apply(); + expect(rows(`SELECT id, object_name FROM sys_activity ORDER BY id`)).toEqual([ + { id: 'act_audit', object_name: 'sys_platform_setting_audit' }, + { id: 'act_deal', object_name: 'crm_deal' }, + { id: 'act_reset', object_name: 'sys_identity_email' }, + ]); + }); + + it('5b a run stopped between copy and delete finishes: the row already in the sibling is not copied twice', async () => { + db.exec(`INSERT INTO sys_identity_email (id, subject, status, related_object, related_id) VALUES ('mail_reset', 'Reset your password', 'sent', 'sys_user', 'usr_b')`); + const report = await apply(); + expect(step(report, 'sibling-moves')).toMatchObject({ status: 'done', summary: { moved: 3, alreadyMoved: 1 } }); + expect(rows(`SELECT COUNT(*) AS n FROM sys_identity_email WHERE id = 'mail_reset'`)).toEqual([{ n: 1 }]); + expect(rows(`SELECT COUNT(*) AS n FROM sys_email WHERE id = 'mail_reset'`)).toEqual([{ n: 0 }]); + }); + + it('5a ⭐ a pristine copy is deleted once its deliveries name the declaration, and its orphaned cipher goes with it', async () => { + const report = await apply(); + expect(step(report, 'declared-webhooks')).toMatchObject({ + status: 'done', + summary: { mirrorsDeleted: 2, activationRows: 2, clones: 1, deliveriesResourced: 2, secretsDeleted: 1, undeclaredKept: 1 }, + }); + expect(rows(`SELECT ref_id FROM sys_http_delivery WHERE id = 'dlv_1'`)).toEqual([{ ref_id: declaredWebhookRefId('task_changed') }]); + expect(rows(`SELECT id FROM sys_secret ORDER BY id`).map((r) => r.id)).toEqual(['sec_setting']); + // The re-source precedes the delete. + const resource = writes.findIndex((w) => /^UPDATE "sys_http_delivery"/.test(w)); + const remove = writes.findIndex((w) => /^DELETE FROM "sys_webhook"/.test(w)); + expect(resource).toBeGreaterThan(-1); + expect(resource).toBeLessThan(remove); + }); + + it('5a ⭐ a switch-only edit becomes the declaration\'s activation row; an edit beyond it a clone, the declaration switched off', async () => { + await apply(); + expect(rows(`SELECT metadata_type, name, package_id, active FROM sys_metadata_activation ORDER BY name`)).toEqual([ + { metadata_type: 'webhook', name: 'deal_won', package_id: 'com.example.app', active: 1 }, + { metadata_type: 'webhook', name: 'lead_scored', package_id: 'com.example.app', active: 0 }, + { metadata_type: 'sharing_rule', name: 'share_collides', package_id: 'com.example.app', active: 0 }, + { metadata_type: 'sharing_rule', name: 'share_custom', package_id: 'com.example.app', active: 0 }, + ].sort((a, b) => a.name.localeCompare(b.name))); + expect(rows(`SELECT ref_id FROM sys_http_delivery WHERE id = 'dlv_2'`)).toEqual([{ ref_id: 'declared:deal_won' }]); + // The clone keeps its id, its edit and its deliveries; it is the administrator's own now. + const [clone] = rows(`SELECT id, name, url, managed_by, customized, definition_json FROM sys_webhook WHERE id = 'whk_edited'`); + expect(clone).toMatchObject({ name: 'lead_scored_custom', url: 'https://hooks.example/admin-moved-it', managed_by: 'admin', customized: 0 }); + const def = JSON.parse(String(clone!.definition_json)); + expect(def.name).toBe('lead_scored_custom'); + expect(def).not.toHaveProperty('_packageId'); + expect(rows(`SELECT ref_id FROM sys_http_delivery WHERE id = 'dlv_3'`)).toEqual([{ ref_id: 'whk_edited' }]); + // A seeded row no package declares stays as it is, dispatching as itself. + expect(rows(`SELECT id, name, managed_by FROM sys_webhook ORDER BY id`)).toEqual([ + { id: 'whk_edited', name: 'lead_scored_custom', managed_by: 'admin' }, + { id: 'whk_orphan', name: 'retired_hook', managed_by: 'package' }, + ]); + expect(rows(`SELECT ref_id FROM sys_http_delivery WHERE id = 'dlv_4'`)).toEqual([{ ref_id: 'whk_orphan' }]); + }); + + it('5a the delivery spelling is plugin-webhooks\' own', () => { + expect(`${DECLARED_WEBHOOK_REF_PREFIX}x`).toBe(declaredWebhookRefId('x')); + }); + + it('5a a credential on a customized copy is never read as switch-only: it is cloned, which keeps it', async () => { + db.exec(`UPDATE sys_webhook SET headers_secret = 'secret:sec_setting' WHERE id = 'whk_switched'`); + await apply(); + expect(rows(`SELECT name, headers_secret FROM sys_webhook WHERE id = 'whk_switched'`)).toEqual([{ name: 'deal_won_custom', headers_secret: 'secret:sec_setting' }]); + expect(rows(`SELECT id FROM sys_secret WHERE id = 'sec_setting'`)).toHaveLength(1); + }); + + it('5c ⭐ a customized copy and an admin rule under a declared name become clones, their grants theirs; the stale name grant goes', async () => { + const report = await apply(); + expect(step(report, 'declared-rule-copies')).toMatchObject({ status: 'done', summary: { clones: 2, activationRows: 2, grantsRemoved: 1 } }); + expect(rows(`SELECT id, name, managed_by, customized FROM sys_sharing_rule ORDER BY id`)).toEqual([ + { id: 'rule_collides', name: 'share_collides_custom', managed_by: 'admin', customized: 0 }, + { id: 'rule_custom', name: 'share_custom_custom', managed_by: 'admin', customized: 0 }, + { id: 'rule_own', name: 'mine', managed_by: 'admin', customized: 0 }, + ]); + expect(rows(`SELECT id, source_id FROM sys_record_share ORDER BY id`)).toEqual([ + { id: 'grant_custom', source_id: 'rule_custom' }, + { id: 'grant_own', source_id: 'rule_own' }, + ]); + }); + + it('5c/5a under a wall a customized copy blocks, named, and nothing of it is written: the switch is deployment-wide', async () => { + db.exec(`DELETE FROM sys_webhook WHERE customized = 1`); + const report = await apply(ledger(), { posture: 'isolated' }); + expect(report.complete).toBe(false); + const rules = step(report, 'declared-rule-copies'); + expect(rules.status).toBe('blocked'); + expect(rules.blockers[0]).toMatch(/Under the isolated posture the declaration's switch is deployment-wide/); + expect(rows(`SELECT name FROM sys_sharing_rule WHERE id = 'rule_custom'`)).toEqual([{ name: 'share_custom' }]); + expect(rows(`SELECT COUNT(*) AS n FROM sys_metadata_activation WHERE metadata_type = 'sharing_rule'`)).toEqual([{ n: 0 }]); + }); + + it('5a without the declared webhooks to read, a seeded row blocks the step and nothing is deleted', async () => { + const report = await apply(ledger(), { webhooks: false }); + expect(step(report, 'declared-webhooks')).toMatchObject({ status: 'blocked', blockers: [expect.stringMatching(/no registry to read them from/)] }); + expect(rows(`SELECT COUNT(*) AS n FROM sys_webhook`)).toEqual([{ n: 4 }]); + }); + + it('5d ⭐ after the column drops, every cleared table is handed to the schema door for NOT NULL — the moved-from tables among them', async () => { + const report = await apply(); + expect(step(report, 'not-null').status).toBe('done'); + const tables = tightened.map((c) => c.table).sort(); + for (const table of ['crm_deal', 'sys_email', 'sys_setting_audit', 'sys_http_delivery', 'sys_webhook', 'sys_sharing_rule', 'sys_record_share', 'sys_activity']) { + expect(tables, table).toContain(table); + } + expect(tightened.every((c) => c.column === 'organization_id')).toBe(true); + // A column-drop table is never handed to it: its column is gone. + expect(tables).not.toContain('sys_secret'); + expect(rows(`SELECT name FROM pragma_table_info('sys_secret') WHERE name = 'organization_id'`)).toEqual([]); + // The door is asked only once the drops have run. + expect(writes.some((w) => /^ALTER TABLE sys_secret DROP COLUMN organization_id$/.test(w))).toBe(true); + }); + + it('5d a table that still holds a NULL organization row is not handed to the door; the cleared ones are', async () => { + // Under a wall nothing owns an orphan row; the copies a wall would block on are gone. + db.exec(` + INSERT INTO crm_deal VALUES ('deal_orphan', NULL, 'nobody'); + DELETE FROM sys_webhook WHERE customized = 1; + DELETE FROM sys_sharing_rule WHERE id IN ('rule_custom', 'rule_collides'); + `); + const report = await apply(ledger(), { posture: 'isolated' }); + expect(step(report, 'not-null').status).toBe('done'); + expect(tightened.map((c) => c.table)).not.toContain('crm_deal'); + expect(tightened.map((c) => c.table)).toContain('sys_email'); + }); + + it('5d a run with no schema door applies nothing and says so; it does not block', async () => { + const report = await apply(ledger(), { notNull: false }); + expect(step(report, 'not-null')).toMatchObject({ status: 'done', notes: [expect.stringMatching(/no schema door to apply it: nothing was applied/)] }); + }); + + it('every new step is idempotent: a second run on a fresh checkpoint writes nothing to the tables they own', async () => { + await apply(); + writes = []; + const again = await apply(); + expect([again.complete, again.blocking]).toEqual([true, 0]); + expect(writes.filter((w) => /sys_webhook|sys_http_delivery|sys_identity_email|sys_platform_setting_audit|sys_job_queue|sys_sharing_rule|sys_metadata_activation/.test(w))).toEqual([]); + }); +}); diff --git a/packages/cli/src/utils/organization-ownership-apply.test.ts b/packages/cli/src/utils/organization-ownership-apply.test.ts index 82ee81fae79..40b3a0181d7 100644 --- a/packages/cli/src/utils/organization-ownership-apply.test.ts +++ b/packages/cli/src/utils/organization-ownership-apply.test.ts @@ -530,10 +530,14 @@ describe('ADR-0131 D10 --apply — the acceptance fixture', () => { ['metadata-promotion', 'done'], ['template-promotion', 'done'], ['settings-move', 'done'], + ['sibling-moves', 'done'], + ['declared-webhooks', 'done'], + ['declared-rule-copies', 'done'], ['rewrite', 'done'], ['mirror-deletion', 'done'], ['history-renumber', 'done'], ['column-drops', 'done'], + ['not-null', 'done'], ]); expect(report.complete).toBe(true); // Ruling C: the fate-4 orphan is the one discrepancy — the marker is not verified. @@ -781,7 +785,7 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { const again = await apply(c, l); expect(again.resumedFrom).toEqual([ - 'attribution', 'authored-conversion', 'metadata-promotion', 'template-promotion', 'settings-move', 'rewrite', 'mirror-deletion', 'history-renumber', 'column-drops', + 'attribution', 'authored-conversion', 'metadata-promotion', 'template-promotion', 'settings-move', 'sibling-moves', 'declared-webhooks', 'declared-rule-copies', 'rewrite', 'mirror-deletion', 'history-renumber', 'column-drops', 'not-null', ]); expect(again.steps.every((s) => s.status === 'checkpointed')).toBe(true); expect(again.complete).toBe(true); @@ -793,14 +797,14 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { const c = catalog(); const l = ledger(); await expect(apply(c, l, { executor: executor({ failOnceOn: /^DELETE FROM "sys_capability"/ }) })).rejects.toThrow(/killed mid-step/); - expect(Object.keys(l.checkpoint!.completed)).toEqual(['attribution', 'authored-conversion', 'metadata-promotion', 'template-promotion', 'settings-move', 'rewrite']); + expect(Object.keys(l.checkpoint!.completed)).toEqual(['attribution', 'authored-conversion', 'metadata-promotion', 'template-promotion', 'settings-move', 'sibling-moves', 'declared-webhooks', 'declared-rule-copies', 'rewrite']); expect(rows('SELECT COUNT(*) AS n FROM sys_position_permission_set')).toEqual([{ n: 3 }]); writes = []; const resumed = await apply(c, l); - expect(resumed.resumedFrom).toEqual(['attribution', 'authored-conversion', 'metadata-promotion', 'template-promotion', 'settings-move', 'rewrite']); - expect(resumed.steps.map((s) => s.status)).toEqual(['checkpointed', 'checkpointed', 'checkpointed', 'checkpointed', 'checkpointed', 'checkpointed', 'done', 'done', 'done']); + expect(resumed.resumedFrom).toEqual(['attribution', 'authored-conversion', 'metadata-promotion', 'template-promotion', 'settings-move', 'sibling-moves', 'declared-webhooks', 'declared-rule-copies', 'rewrite']); + expect(resumed.steps.map((s) => s.status)).toEqual([...Array(9).fill('checkpointed'), 'done', 'done', 'done', 'done']); expect(writes.some((w) => /^UPDATE/.test(w))).toBe(false); expect(resumed.complete).toBe(true); expect(rows('SELECT COUNT(*) AS n FROM sys_position_permission_set')).toEqual([{ n: 0 }]); @@ -1067,7 +1071,7 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { // (a declared object the differ proposed no drop for). const notProposed = await answering('not-proposed'); expect(notProposed.resumedFrom).toEqual([ - 'attribution', 'authored-conversion', 'metadata-promotion', 'template-promotion', 'settings-move', 'rewrite', 'mirror-deletion', 'history-renumber', + 'attribution', 'authored-conversion', 'metadata-promotion', 'template-promotion', 'settings-move', 'sibling-moves', 'declared-webhooks', 'declared-rule-copies', 'rewrite', 'mirror-deletion', 'history-renumber', ]); const control = notProposed.steps.find((s) => s.step === 'column-drops')!; expect(control.status).toBe('blocked'); diff --git a/packages/cli/src/utils/organization-ownership-apply.ts b/packages/cli/src/utils/organization-ownership-apply.ts index 2fdb0c395bd..380ef3b0ae2 100644 --- a/packages/cli/src/utils/organization-ownership-apply.ts +++ b/packages/cli/src/utils/organization-ownership-apply.ts @@ -1,6 +1,7 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. -import { ORGANIZATION_OWNERSHIP_MIGRATION_ID } from '@objectstack/spec/system'; +import { randomUUID } from 'node:crypto'; +import { ORGANIZATION_OWNERSHIP_MIGRATION_ID, ORGANIZATION_OWNERSHIP_SHAPE_GENERATION } from '@objectstack/spec/system'; import { convertPositionBindingRows, permissionSetDefinitionFromRow, @@ -27,6 +28,7 @@ import { columnNameOf, predicateSql, PRIMARY_DATABASE, + requireColumn, Sql, underivableDepartureOf, type AttributionSink, @@ -103,6 +105,19 @@ import type { DeclaredSharingRuleSkip } from '@objectstack/plugin-sharing'; * copied, by the database itself, into `sys_platform_setting`, read back, * and only then removed from `sys_setting`. * A key the new table already holds with a different value is a blocker. + * 2d. `sibling-moves` (18.x step 5b) — the deployment-level rows an 18.0 + * database keeps in a tenant table move, id for id, into the sibling this + * release writes them to: the settings audit trail's global rung and + * identity mail, selected by shape, the mail's queued jobs and the rows' + * activity timeline pointed at the sibling first ({@link runSiblingMoves}). + * 2e. `declared-webhooks` (18.x step 5a) — each `sys_webhook` copy a seeder + * wrote under a declared name is settled: a pristine one deleted once its + * deliveries name the declaration, a switch-only edit kept as the + * declaration's activation row, any other edit kept as a clone under a new + * name with the declaration switched off ({@link runDeclaredWebhooks}). + * 2f. `declared-rule-copies` (18.x step 5c, C3 stage 9c) — each stored + * sharing rule under a declared rule's name becomes a clone under a new + * name, the declaration switched off ({@link runDeclaredRuleCopies}). * 3. `rewrite` — the verified id→name rewrite (D4): * `sys_user_permission_set.permission_set` is named from the set row its * `permission_set_id` points at, then EVERY grant is verified to name a set @@ -137,6 +152,10 @@ import type { DeclaredSharingRuleSkip } from '@objectstack/plugin-sharing'; * (C7b), once the rewrite has read it and no two grants hold one set for * one user in one organization (the unique index that replaces the old * one would refuse them; they are reported, never deleted). + * 6. `not-null` (18.x step 5d, 8e) — every table the plan now clears + * receives NOT NULL on `organization_id`, through the schema door + * ({@link NotNullPort}); the ceremony owns that nullability and the + * differ reads it as no drift (ruling record 6269955919, ② A). * * Then the post-check re-runs the plan and reports, per table, what remains. * A departure no step of a release can perform — the declared sharing-rule @@ -189,7 +208,9 @@ import type { DeclaredSharingRuleSkip } from '@objectstack/plugin-sharing'; * * ## What it never does * - * ⛔ Delete a row outside fate 2 and a declared `underivableDeparture`. ⛔ Give an unattributable row an owner: + * ⛔ Delete a row outside fate 2, a declared `underivableDeparture`, a moved + * row once its copy reads back, a declared webhook's copy, and the grants of a + * declaration it switched off. ⛔ Give an unattributable row an owner: * owners come only from the plan's anchors and, under `single`, the Default * Organization. ⛔ Create any organization but the Default Organization, and * that one only under `single` on a database that holds none. ⛔ Run at boot: @@ -205,10 +226,14 @@ export const CEREMONY_STEPS = [ 'metadata-promotion', 'template-promotion', 'settings-move', + 'sibling-moves', + 'declared-webhooks', + 'declared-rule-copies', 'rewrite', 'mirror-deletion', 'history-renumber', 'column-drops', + 'not-null', ] as const; export type CeremonyStep = (typeof CEREMONY_STEPS)[number]; @@ -269,7 +294,8 @@ export interface MetadataLockPort { * reason. */ export interface DeclaredSharingRulesPort { - rules(): Promise>; + /** `packageId`: the declaring package, for the switch row a settled copy writes (`declared-rule-copies`). */ + rules(): Promise>; } /** @@ -278,14 +304,17 @@ export interface DeclaredSharingRulesPort { * {@link DeclaredSharingRulesPort}: a nameless item is no rule to rename onto * and is left out. */ -export async function judgeDeclaredSharingRules(items: readonly unknown[]): Promise> { +export async function judgeDeclaredSharingRules( + items: readonly unknown[], +): Promise> { const { translateDeclaredSharingRule } = await import('@objectstack/plugin-sharing'); - const out: Array<{ name: string; refused: string | null }> = []; + const out: Array<{ name: string; refused: string | null; packageId: string | null }> = []; for (const item of items) { const name = (item as { name?: unknown } | null)?.name; if (typeof name !== 'string' || name === '') continue; const { skip } = translateDeclaredSharingRule(item); - out.push({ name, refused: skip ? describeDeclaredRuleSkip(skip) : null }); + const packageId = (item as { _packageId?: unknown })._packageId; + out.push({ name, refused: skip ? describeDeclaredRuleSkip(skip) : null, packageId: typeof packageId === 'string' && packageId ? packageId : null }); } return out; } @@ -345,6 +374,13 @@ export interface ColumnDropPort { export interface CeremonyCheckpoint { ceremony: 'ADR-0131 D10'; inventoryDigest: string; + /** + * The organization-ownership shape generation this ceremony's step set + * reaches ({@link ORGANIZATION_OWNERSHIP_SHAPE_GENERATION}): what the boot + * guard decides on once the marker is verified. A checkpoint an older + * ceremony wrote carries none and reads as generation 0. + */ + shapeGeneration?: number; completed: Partial }>>; /** * The `--metadata-choice` rows whose conflict this ceremony resolved, recorded @@ -596,6 +632,13 @@ export interface ApplyOptions { * that finds a declared-rule mirror standing blocks instead of deleting it. */ declaredSharingRules?: DeclaredSharingRulesPort; + /** + * [18.x step 5a] The declared webhooks, by name. Absent, a run that finds a + * seeded `sys_webhook` row blocks instead of settling it. + */ + declaredWebhooks?: DeclaredWebhooksPort; + /** [18.x step 5d] The schema door the NOT NULL step applies through. Absent, it applies nothing and says so. */ + notNull?: NotNullPort; /** The served views, for carrying form withdrawals. */ views: ServedViewsPort; /** The door's lock verdict, for the one write the ceremony makes in place. */ @@ -2925,6 +2968,797 @@ async function runColumnDrops(run: Run): Promise { }; } +// ── 18.x, shape generation 1: what an 18.0 database owes too ──────────────── +// +// The four steps below select their rows BY SHAPE, never by `organization_id +// IS NULL`: an 18.0 single-posture database stamped every one of them with the +// Default Organization (line fact F3 of the 18.x plan), so a NULL selector +// would find nothing there and certify a database that still owes the step. + +const WEBHOOKS = 'sys_webhook'; +const DELIVERIES = 'sys_http_delivery'; +const SECRETS = 'sys_secret'; +const ACTIVATIONS = 'sys_metadata_activation'; +const JOB_QUEUE = 'sys_job_queue'; +const ACTIVITY = 'sys_activity'; +/** The provenance a seeded copy of a declaration carries (`managed_by`). */ +const SEEDED_PROVENANCE = ['package', 'platform'] as const; +/** + * How a delivery names a declared webhook: plugin-webhooks' + * `declaredWebhookRefId` (`declared:NAME`). Spelled here because the CLI does + * not load the plugin to re-source rows; `organization-ownership-apply.declared-webhooks.test.ts` + * holds the two equal. + */ +export const DECLARED_WEBHOOK_REF_PREFIX = 'declared:'; +/** The `name` column's length on `sys_webhook` and `sys_sharing_rule`. */ +const NAME_MAX = 100; +/** The stored form of an encrypted field's value: `secret:` (objectql's `SECRET_REF_PREFIX`). */ +const SECRET_REF_PREFIX = 'secret:'; + +const truthy = (value: unknown): boolean => value === true || value === 1 || value === '1' || value === 'true' || value === 't'; + +/** The physical tables of `object` the plan this run started from found present. */ +function presentTables(run: Run, object: string): string[] { + const table = run.plan.tables.find((t) => t.object === object); + return table?.physical === 'present' ? table.tables : []; +} + +const blockedStep = (step: CeremonyStep, summary: Record, blockers: string[], notes: string[] = []): StepReport => + ({ step, status: 'blocked', summary, blockers, notes }); + +/** + * Set the deployment's switch for a declared item — the `sys_metadata_activation` + * row (ADR-0131 D14) keyed `(metadata_type, name)` — to `active`, read back. + * `false` when the row already said so (nothing written). + */ +async function setActivation(run: Run, type: string, name: string, packageId: string | null, active: boolean): Promise { + const [table] = presentTables(run, ACTIVATIONS); + if (!table) { + throw new Error( + `${ACTIVATIONS} does not exist on this database, so the switch of the declared ${type} ${name} cannot be written. ` + + 'Boot this release against it once (its schema sync creates the table), then re-run --apply.', + ); + } + const cols = (await columnsOf(run, table))!; + const { sql } = databaseFor(run, table); + const q = (c: string): string => sql.q(c, table); + const read = (): Promise[]> => + sql.read( + table, + `SELECT ${q('id')} AS ${q('id')}, ${q('active')} AS ${q('active')} FROM ${q(table)} WHERE ${q('metadata_type')} = ? AND ${q('name')} = ?`, + [type, name], + ); + const [row] = await read(); + if (row && truthy(row.active) === active) return false; + const at = (run.opts.now ?? (() => new Date()))().toISOString(); + if (row) { + await execute( + run, + table, + `UPDATE ${q(table)} SET ${q('active')} = ?${cols.has('updated_at') ? `, ${q('updated_at')} = ?` : ''} WHERE ${q('id')} = ?`, + [sql.bind(active), ...(cols.has('updated_at') ? [at] : []), row.id], + ); + } else { + const values: Record = { + id: `act_${randomUUID()}`, + metadata_type: type, + name, + package_id: packageId, + active: sql.bind(active), + created_at: at, + updated_at: at, + }; + const keys = Object.keys(values).filter((c) => cols.has(c)); + await execute( + run, + table, + `INSERT INTO ${q(table)} (${keys.map(q).join(', ')}) VALUES (${keys.map(() => '?').join(', ')})`, + keys.map((k) => values[k]), + ); + } + const [after] = await read(); + if (!after || truthy(after.active) !== active) { + throw new Error(`The switch of the declared ${type} ${name} did not read back ${active ? 'on' : 'off'} in ${ACTIVATIONS}.`); + } + return true; +} + +/** + * A name for a clone: `_custom`, then `_custom_2`, … — one no row of + * `table` holds and no package declares (`taken`), within {@link NAME_MAX}. + */ +async function freeName(run: Run, table: string, base: string, taken: Set): Promise { + const { sql } = databaseFor(run, table); + for (let n = 1; ; n++) { + const suffix = n === 1 ? '_custom' : `_custom_${n}`; + const name = `${base.slice(0, NAME_MAX - suffix.length)}${suffix}`; + if (taken.has(name)) continue; + const held = await sql.count(table, `SELECT COUNT(*) AS n FROM ${sql.q(table, table)} WHERE ${sql.q('name', table)} = ?`, [name]); + if (held === 0) { + taken.add(name); + return name; + } + } +} + +// ── Step 2e: declared webhooks (18.x step 5a; ADR-0131 D2, D10 fate 2) ────── + +/** + * [ADR-0131 D2] The webhooks the deployment's packages DECLARE, by name, with + * the package that declares each — what plugin-webhooks serves from the + * registry (the command: the booted engine's registry, else the metadata + * service). + */ +export interface DeclaredWebhooksPort { + list(): Promise>; +} + +/** A `sys_webhook` row's `definition_json`, parsed, or `null`. */ +function webhookDefinition(row: Record): Record | null { + const raw = row.definition_json; + if (typeof raw !== 'string' || raw === '') return null; + try { + const parsed = JSON.parse(raw) as unknown; + return parsed && typeof parsed === 'object' && !Array.isArray(parsed) ? (parsed as Record) : null; + } catch { + return null; + } +} + +const triggerList = (value: unknown): string => { + let list: unknown = value; + if (typeof value === 'string') { + try { + list = JSON.parse(value); + } catch { + list = [value]; + } + } + return JSON.stringify(Array.isArray(list) ? list.map(String).sort() : []); +}; + +/** + * Whether a customized seeded webhook differs from the copy the 18.0 seeder + * last wrote ONLY in its switch. `definition_json` is that copy: the seeder + * rewrote it on every boot while the row was pristine, and the first edit + * stopped it. Every column the seeder mapped from the envelope must still + * read as the envelope says. A row holding a credential is never read as + * switch-only: a stored secret cannot be compared without decrypting it, so it + * is settled as a clone, which keeps it — the conservative reading. + */ +function isSwitchOnlyEdit(row: Record): boolean { + const def = webhookDefinition(row); + if (!def) return false; + if (text(row.headers_secret) !== null || text(row.signing_secret) !== null) return false; + const same = (a: unknown, b: unknown): boolean => text(a) === text(b); + return ( + same(row.label, def.label ?? def.name) && + same(row.object_name, def.object ?? null) && + triggerList(row.triggers) === triggerList(def.triggers) && + same(row.url, def.url) && + same(String(row.method ?? 'post').toLowerCase(), String(def.method ?? 'POST').toLowerCase()) && + same(row.description, def.description ?? null) + ); +} + +const secretIdsOf = (row: Record): string[] => + [row.headers_secret, row.signing_secret] + .map((v) => (typeof v === 'string' && v.startsWith(SECRET_REF_PREFIX) ? v.slice(SECRET_REF_PREFIX.length) : '')) + .filter((id) => id !== ''); + +/** + * [18.x step 5a; ADR-0131 D2, D10 fate 2] Settle every `sys_webhook` row a + * package's seeder wrote (`managed_by` `package` or `platform`) under a name a + * package still declares — plugin-webhooks serves that declaration from the + * registry, so the row is a copy: + * + * - a PRISTINE copy (not `customized`) is deleted, after every delivery that + * names it (`sys_http_delivery.ref_id` = its row id) is re-sourced to the + * declaration (`declared:NAME`), so history and redelivery follow the + * declaration; + * - a copy customized only in its switch ({@link isSwitchOnlyEdit}) becomes + * the declaration's activation row with that switch (regime C, ruling Q2 A + * on the declared-webhook card: an activation row for an `active` change), + * its deliveries are re-sourced, and it is deleted; + * - a copy customized in anything else becomes a CLONE under a new name — + * renamed in place, `managed_by 'admin'`, so its id, its credentials and + * every delivery naming it stay its own (the deliveries' tail already names + * the clone's row id) — and the declaration is switched off. + * + * The `sys_secret` rows a deleted copy's credential columns referenced are + * deleted with it, once no other webhook row references them. A seeded row + * whose name no package declares any more is an ordinary webhook — it + * dispatches as itself, as it did in 18.0 — and stays, reported. + * + * Under a wall the declaration's switch is deployment-wide (there is no + * per-organization switch-off), so a customized copy is not settled on the + * operator's behalf: it blocks, named, with the remedy. Idempotent: a second + * run finds no seeded row under a declared name. + */ +async function runDeclaredWebhooks(run: Run): Promise { + const step: CeremonyStep = 'declared-webhooks'; + const summary: Record = { mirrorsDeleted: 0, activationRows: 0, clones: 0, deliveriesResourced: 0, secretsDeleted: 0, undeclaredKept: 0 }; + const notes: string[] = []; + const [table] = presentTables(run, WEBHOOKS); + const cols = table ? await columnsOf(run, table) : null; + if (!table || !cols?.has('managed_by') || !cols.has('name')) return { step, status: 'done', summary, blockers: [], notes }; + const { sql } = databaseFor(run, table); + const q = (c: string): string => sql.q(c, table); + const seeded = predicateSql(sql, table, cols, [{ column: 'managed_by', in: SEEDED_PROVENANCE }]); + const select = ['name', 'managed_by', 'customized', 'active', 'label', 'object_name', 'triggers', 'url', 'method', 'description', 'definition_json', 'headers_secret', 'signing_secret']; + const rows = await readAll(run, table, select.filter((c) => cols.has(c)), seeded); + if (rows.length === 0) return { step, status: 'done', summary, blockers: [], notes }; + + if (!run.opts.declaredWebhooks) { + return blockedStep(step, summary, [ + `${table} holds ${rows.length} row(s) a package's seeder wrote, settled only against the webhooks the ` + + 'deployment declares, and this run has no registry to read them from. Run the command against the ' + + "deployment's own composition, then re-run.", + ]); + } + const declared = new Map((await run.opts.declaredWebhooks.list()).map((d) => [d.name, d.packageId] as const)); + const held = rows.filter((r) => declared.has(String(r.name))); + summary.undeclaredKept = rows.length - held.length; + if (summary.undeclaredKept > 0) { + notes.push( + `${table}: ${summary.undeclaredKept} seeded row(s) name a webhook no installed package declares any more ` + + `(${rows.filter((r) => !declared.has(String(r.name))).map((r) => String(r.id)).sort().join(', ')}). Each is an ` + + 'ordinary webhook — it dispatches as itself, as it did in 18.0 — and stays.', + ); + } + const customized = held.filter((r) => truthy(r.customized)); + if (customized.length > 0 && run.opts.posture !== 'single') { + return blockedStep(step, summary, [ + `${table} holds ${customized.length} customized copy(ies) of a declared webhook ` + + `(${customized.map((r) => `${String(r.id)} ${String(r.name)}`).sort().join(', ')}). Under the ${run.opts.posture} ` + + 'posture the declaration\'s switch is deployment-wide — there is no per-organization switch-off — so which ' + + 'organizations keep the declared webhook is not decided on the operator\'s behalf, and nothing was written. ' + + 'Rename each copy to a name no package declares (it becomes that organization\'s own webhook), or delete it; ' + + 'switch the declared webhook on or off through the activation door as the deployment wants; then re-run.', + ], notes); + } + + const deliveryTables: string[] = []; + for (const physical of presentTables(run, DELIVERIES)) { + const dcols = await columnsOf(run, physical); + if (dcols?.has('source') && dcols.has('ref_id')) deliveryTables.push(physical); + } + const resource = async (from: string, to: string): Promise => { + let n = 0; + for (const physical of deliveryTables) { + const { sql: dsql } = databaseFor(run, physical); + const where = `${dsql.q('source', physical)} = ? AND ${dsql.q('ref_id', physical)} = ?`; + const count = (): Promise => dsql.count(physical, `SELECT COUNT(*) AS n FROM ${dsql.q(physical, physical)} WHERE ${where}`, ['webhook', from]); + const before = await count(); + if (before === 0) continue; + await execute(run, physical, `UPDATE ${dsql.q(physical, physical)} SET ${dsql.q('ref_id', physical)} = ? WHERE ${where}`, [to, 'webhook', from]); + const left = await count(); + if (left !== 0) throw new Error(`${left} delivery(ies) on ${physical} still name the webhook row ${from} after the re-source; nothing was deleted.`); + n += before; + } + return n; + }; + const deleteCopy = async (row: Record): Promise => { + const where = `${q('id')} = ? AND (${seeded.text})`; + await execute(run, table, `DELETE FROM ${q(table)} WHERE ${where}`, [row.id, ...seeded.params]); + if ((await sql.count(table, `SELECT COUNT(*) AS n FROM ${q(table)} WHERE ${q('id')} = ?`, [row.id])) !== 0) { + throw new Error(`The declared-webhook copy ${String(row.id)} on ${table} was not deleted; nothing past it was.`); + } + }; + + const orphaned: string[] = []; + const taken = new Set(declared.keys()); + for (const row of held) { + const name = String(row.name); + const id = String(row.id); + const def = webhookDefinition(row); + const packageId = (typeof def?._packageId === 'string' && def._packageId ? def._packageId : null) ?? declared.get(name) ?? null; + if (!truthy(row.customized) || isSwitchOnlyEdit(row)) { + if (truthy(row.customized) && (await setActivation(run, 'webhook', name, packageId, truthy(row.active)))) summary.activationRows++; + summary.deliveriesResourced += await resource(id, `${DECLARED_WEBHOOK_REF_PREFIX}${name}`); + await deleteCopy(row); + orphaned.push(...secretIdsOf(row)); + summary.mirrorsDeleted++; + continue; + } + const clone = await freeName(run, table, name, taken); + const { _provenance: _p, _packageId: _k, _packageVersion: _v, ...own } = def ?? {}; + const sets: Array<[string, unknown]> = [ + ['name', clone], + ['managed_by', 'admin'], + ...(cols.has('customized') ? [['customized', sql.bind(false)] as [string, unknown]] : []), + ...(def && cols.has('definition_json') ? [['definition_json', JSON.stringify({ ...own, name: clone })] as [string, unknown]] : []), + ]; + await execute( + run, + table, + `UPDATE ${q(table)} SET ${sets.map(([c]) => `${q(c)} = ?`).join(', ')} WHERE ${q('id')} = ? AND (${seeded.text})`, + [...sets.map(([, v]) => v), row.id, ...seeded.params], + ); + const [landed] = await sql.read(table, `SELECT ${q('name')} AS ${q('name')} FROM ${q(table)} WHERE ${q('id')} = ?`, [row.id]); + if (text(landed?.name) !== clone) throw new Error(`The clone of the declared webhook ${name} (${id}) did not read back as ${clone}.`); + if (await setActivation(run, 'webhook', name, packageId, false)) summary.activationRows++; + summary.clones++; + notes.push( + `${table} ${id}: customized beyond its switch, so it is now the administrator's own webhook ${clone} (its ` + + `credentials and deliveries with it), and the declared ${name} is switched off.`, + ); + } + + // The ciphers the deleted copies referenced, once nothing else does. + const [secrets] = presentTables(run, SECRETS); + if (secrets && orphaned.length > 0) { + const { sql: ssql } = databaseFor(run, secrets); + for (const secretId of [...new Set(orphaned)].sort()) { + const ref = `${SECRET_REF_PREFIX}${secretId}`; + const credentialCols = ['headers_secret', 'signing_secret'].filter((c) => cols.has(c)); + const still = credentialCols.length === 0 + ? 0 + : await sql.count(table, `SELECT COUNT(*) AS n FROM ${q(table)} WHERE ${credentialCols.map((c) => `${q(c)} = ?`).join(' OR ')}`, credentialCols.map(() => ref)); + if (still > 0) continue; + const where = `${ssql.q('id', secrets)} = ?`; + const before = await ssql.count(secrets, `SELECT COUNT(*) AS n FROM ${ssql.q(secrets, secrets)} WHERE ${where}`, [secretId]); + if (before === 0) continue; + await execute(run, secrets, `DELETE FROM ${ssql.q(secrets, secrets)} WHERE ${where}`, [secretId]); + summary.secretsDeleted += before; + } + } + + // Verified: no seeded row holds a declared name any more. + const names = [...declared.keys()]; + for (let i = 0; i < names.length; i += IN_CHUNK) { + const chunk = names.slice(i, i + IN_CHUNK); + const left = await sql.count( + table, + `SELECT COUNT(*) AS n FROM ${q(table)} WHERE (${seeded.text}) AND ${q('name')} IN (${chunk.map(() => '?').join(', ')})`, + [...seeded.params, ...chunk], + ); + if (left !== 0) throw new Error(`${left} seeded ${table} row(s) still hold a declared webhook's name after the settlement.`); + } + run.deleted.set(WEBHOOKS, (run.deleted.get(WEBHOOKS) ?? 0) + summary.mirrorsDeleted); + if (summary.deliveriesResourced > 0) { + notes.push( + `${DELIVERIES}: ${summary.deliveriesResourced} delivery(ies) now name their declared webhook (declared:NAME). A ` + + 'pending one keeps the partition it was enqueued under: it is still claimed and sent, by that partition\'s worker.', + ); + } + run.opts.onProgress?.( + `declared webhooks: ${summary.mirrorsDeleted} copy(ies) deleted, ${summary.clones} cloned, ` + + `${summary.deliveriesResourced} delivery(ies) re-sourced`, + ); + return { step, status: 'done', summary, blockers: [], notes }; +} + +// ── Step 2d: the sibling moves (18.x step 5b; ADR-0131 D7) ────────────────── + +/** One id-preserving move of deployment-level rows into the sibling that holds them now. */ +interface SiblingMove { + /** The inventory departure on `from` that names these rows. */ + id: string; + from: string; + to: string; + /** The rows that move, by SHAPE — never by a NULL organization (F3). */ + where(sql: Sql, table: string, cols: ReadonlySet): { text: string; params: unknown[] }; +} + +/** + * The body of a magic-link mail: the sign-in URL better-auth's magic-link + * plugin builds (`/magic-link/verify?token=`). An 18.0 magic-link row carries + * no related record and no template column, so its link is the one structural + * trace of what it is; every rendering of `auth.magic_link`, customized or + * not, carries it, because the mail exists to deliver it. + */ +const MAGIC_LINK_BODY = '%/magic-link/verify?token=%'; + +export const SIBLING_MOVES: readonly SiblingMove[] = [ + { + id: 'global-audit-move', + from: 'sys_setting_audit', + to: 'sys_platform_setting_audit', + where: (sql, table, cols) => predicateSql(sql, table, cols, [{ column: 'scope', equals: 'global' }]), + }, + { + // Identity mail: mail ABOUT a person (reset, verification, account + // invitation, change-email notice — `related_object = 'sys_user'`, the + // four plugin-auth sites that name one) and the magic link, which names no + // record. The organization invitation (`related_object = 'sys_invitation'`) + // is tenant mail and stays. + id: 'identity-mail-move', + from: 'sys_email', + to: 'sys_identity_email', + where: (sql, table, cols) => { + requireColumn(table, cols, 'related_object'); + const c = (name: string): string => sql.q(name, table); + const bodies = ['body_text', 'body_html'].filter((b) => cols.has(b)); + const magic = bodies.length && cols.has('related_id') + ? ` OR (${c('related_object')} IS NULL AND ${c('related_id')} IS NULL AND (${bodies.map((b) => `${c(b)} LIKE ?`).join(' OR ')}))` + : ''; + return { text: `(${c('related_object')} = ?${magic})`, params: ['sys_user', ...(magic ? bodies.map(() => MAGIC_LINK_BODY) : [])] }; + }, + }, +]; + +/** The mail queues whose 18.0 jobs name a `sys_email` row by id alone (plugin-email). */ +const MAIL_JOB_QUEUES = ['email.send.async', 'email.attachment.reclaim'] as const; + +/** + * Before identity mail leaves `sys_email`: point every pending or running mail + * job that names a moving row at the sibling. An 18.0 job carries `{ rowId }` + * with no `object`, which its consumer reads as `sys_email` — a send would find + * no row and never deliver, and an attachment reclaim would take the row-gone + * branch and delete the mail's attachment content. The job gains + * `object: 'sys_identity_email'`, in its payload and metadata, and its + * idempotency key the sibling's spelling (`:`, `_attachments:`). + */ +async function rewriteMailJobs(run: Run, from: string, to: string, ids: ReadonlySet): Promise { + let rewritten = 0; + for (const physical of presentTables(run, JOB_QUEUE)) { + const cols = await columnsOf(run, physical); + if (!cols?.has('queue') || !cols.has('payload_json')) continue; + const { sql } = databaseFor(run, physical); + const q = (c: string): string => sql.q(c, physical); + const statusClause = cols.has('status') ? ` AND ${q('status')} IN (?, ?)` : ''; + const jobs = await readAll( + run, + physical, + ['payload_json', ...['metadata_json', 'idempotency_key'].filter((c) => cols.has(c))], + { + text: `${q('queue')} IN (${MAIL_JOB_QUEUES.map(() => '?').join(', ')})${statusClause}`, + params: [...MAIL_JOB_QUEUES, ...(statusClause ? ['pending', 'running'] : [])], + }, + ); + for (const job of jobs) { + const payload = parseJsonObject(job.payload_json); + const rowId = payload?.rowId; + if (!payload || typeof rowId !== 'string' || !ids.has(rowId) || (payload.object !== undefined && payload.object !== from)) continue; + const sets: Array<[string, unknown]> = [['payload_json', JSON.stringify({ ...payload, object: to })]]; + const metadata = parseJsonObject(job.metadata_json); + if (metadata) sets.push(['metadata_json', JSON.stringify({ ...metadata, object: to })]); + const key = text(job.idempotency_key); + if (key?.startsWith(`${from}:`)) sets.push(['idempotency_key', `${to}:${key.slice(from.length + 1)}`]); + else if (key?.startsWith(`${from}_attachments:`)) sets.push(['idempotency_key', `${to}_attachments:${key.slice(from.length + 13)}`]); + await execute(run, physical, `UPDATE ${q(physical)} SET ${sets.map(([c]) => `${q(c)} = ?`).join(', ')} WHERE ${q('id')} = ?`, [...sets.map(([, v]) => v), job.id]); + rewritten++; + } + } + return rewritten; +} + +function parseJsonObject(value: unknown): Record | null { + let parsed: unknown = value; + if (typeof value === 'string') { + try { + parsed = JSON.parse(value); + } catch { + return null; + } + } + return parsed && typeof parsed === 'object' && !Array.isArray(parsed) ? (parsed as Record) : null; +} + +/** + * [18.x step 5b; ADR-0131 D7] Move the deployment-level rows an 18.0 database + * still keeps in a tenant table into the sibling this release writes them to + * ({@link SIBLING_MOVES}): the settings audit trail's global rung + * (`sys_setting_audit WHERE scope = 'global'` → `sys_platform_setting_audit`) + * and identity mail (`sys_email` → `sys_identity_email`). Each row keeps its + * id: it is copied (the columns both tables carry — never `organization_id`, + * which the sibling does not have), read back, and only then deleted from its + * old table, by id with its selector re-asserted. A row already in the sibling + * (a run stopped between copy and delete) is not copied twice. + * + * Before identity mail moves, the mail jobs naming it are pointed at the + * sibling ({@link rewriteMailJobs}). After each move, the activity timeline + * rows about a moved row (`sys_activity.object_name`, every rotation shard) + * name its new object, so the record's history follows it; the audit ledger + * (`sys_audit_log`) is a ledger and is not rewritten. + */ +async function runSiblingMoves(run: Run): Promise { + const step: CeremonyStep = 'sibling-moves'; + const summary: Record = { moved: 0, alreadyMoved: 0, jobsRewritten: 0, activityRepointed: 0 }; + const blockers: string[] = []; + for (const move of SIBLING_MOVES) { + const [from] = presentTables(run, move.from); + const fromCols = from ? await columnsOf(run, from) : null; + if (!from || !fromCols) continue; + const { sql } = databaseFor(run, from); + const where = move.where(sql, from, fromCols); + const ids = (await readAll(run, from, [], where)).map((r) => String(r.id)); + if (ids.length === 0) continue; + const [to] = presentTables(run, move.to); + const toCols = to ? await columnsOf(run, to) : null; + if (!to || !toCols) { + blockers.push( + `${ids.length} ${move.from} row(s) move to ${move.to}, which this database does not have. Boot this release ` + + 'against it once (its schema sync creates the table), then re-run.', + ); + continue; + } + const { sql: tsql } = databaseFor(run, to); + const shared = [...fromCols].filter((c) => toCols.has(c) && c !== ORG); + if (move.from === 'sys_email') summary.jobsRewritten += await rewriteMailJobs(run, move.from, move.to, new Set(ids)); + for (let i = 0; i < ids.length; i += IN_CHUNK) { + const chunk = ids.slice(i, i + IN_CHUNK); + const inChunk = (s: Sql, table: string): string => `${s.q('id', table)} IN (${chunk.map(() => '?').join(', ')})`; + const there = new Set( + (await tsql.read(to, `SELECT ${tsql.q('id', to)} AS ${tsql.q('id', to)} FROM ${tsql.q(to, to)} WHERE ${inChunk(tsql, to)}`, chunk)) + .map((r) => String(r.id)), + ); + const missing = chunk.filter((id) => !there.has(id)); + summary.alreadyMoved += chunk.length - missing.length; + if (missing.length > 0) { + const source = await sql.read( + from, + `SELECT ${shared.map((c) => `${sql.q(c, from)} AS ${sql.q(c, from)}`).join(', ')} FROM ${sql.q(from, from)} ` + + `WHERE ${sql.q('id', from)} IN (${missing.map(() => '?').join(', ')})`, + missing, + ); + for (const row of source) { + await execute( + run, + to, + `INSERT INTO ${tsql.q(to, to)} (${shared.map((c) => tsql.q(c, to)).join(', ')}) VALUES (${shared.map(() => '?').join(', ')})`, + shared.map((c) => row[c] ?? null), + ); + } + } + // The source leaves only once every row reads back in the sibling. + const landed = await tsql.count(to, `SELECT COUNT(*) AS n FROM ${tsql.q(to, to)} WHERE ${inChunk(tsql, to)}`, chunk); + if (landed !== chunk.length) { + throw new Error(`${chunk.length - landed} ${move.from} row(s) did not read back in ${move.to}; they were kept in ${move.from}.`); + } + summary.activityRepointed += await repointActivity(run, move.from, move.to, chunk); + await execute(run, from, `DELETE FROM ${sql.q(from, from)} WHERE (${where.text}) AND ${inChunk(sql, from)}`, [...where.params, ...chunk]); + const left = await sql.count(from, `SELECT COUNT(*) AS n FROM ${sql.q(from, from)} WHERE ${inChunk(sql, from)}`, chunk); + if (left !== 0) throw new Error(`${left} moved ${move.from} row(s) were not removed after they read back in ${move.to}.`); + summary.moved += missing.length; + run.movedOut.set(move.from, (run.movedOut.get(move.from) ?? 0) + chunk.length); + run.movedIn.set(move.to, (run.movedIn.get(move.to) ?? 0) + missing.length); + } + run.opts.onProgress?.(`sibling move: ${ids.length} ${move.from} row(s) now in ${move.to}`); + } + return { step, status: blockers.length ? 'blocked' : 'done', summary, blockers, notes: [] }; +} + +/** Point the activity rows about `ids` of `from` at `to` (every rotation shard); the count re-pointed. */ +async function repointActivity(run: Run, from: string, to: string, ids: readonly string[]): Promise { + let n = 0; + for (const physical of presentTables(run, ACTIVITY)) { + const cols = await columnsOf(run, physical); + if (!cols?.has('object_name') || !cols.has('record_id')) continue; + const { sql } = databaseFor(run, physical); + const where = `${sql.q('object_name', physical)} = ? AND ${sql.q('record_id', physical)} IN (${ids.map(() => '?').join(', ')})`; + const count = (object: string): Promise => + sql.count(physical, `SELECT COUNT(*) AS n FROM ${sql.q(physical, physical)} WHERE ${where}`, [object, ...ids]); + const before = await count(from); + if (before === 0) continue; + await execute(run, physical, `UPDATE ${sql.q(physical, physical)} SET ${sql.q('object_name', physical)} = ? WHERE ${where}`, [to, from, ...ids]); + if ((await count(from)) !== 0) throw new Error(`Activity rows on ${physical} still name ${from} for a moved row after the re-point.`); + n += before; + } + return n; +} + +// ── Step 2f: the declared sharing rules' stored copies (18.x step 5c, 9c) ─── + +/** + * [ADR-0131 C3 stage 9c; ruling A on the sharing-rule regime card, item 3] + * Every `sys_sharing_rule` row that holds the name of a rule a package + * declares — a CUSTOMIZED copy the retired seeder wrote, or an administrator's + * rule a later package declaration collided with — answers for that name + * today, while plugin-sharing withholds the declaration and reports it at + * load. The regime is a clone under a new name plus the base switched off, so + * each becomes exactly that, with what it grants unchanged: + * + * - the row is renamed in place to a free name (`_custom`, …) and owned + * by the administrator (`managed_by 'admin'`, not `customized`) — its id is + * unchanged, so every grant naming it (`sys_record_share.source_id`, a row + * id) stays its own; + * - the declaration is switched off: its `sys_metadata_activation` row + * (`sharing_rule`, NAME) says `active: false`, so it does not start granting + * beside its clone — the withholding the row caused, now recorded; + * - grants still naming the declaration itself (`source_id` = NAME) are + * removed, as the switch's own reconcile would remove them. + * + * The UNCUSTOMIZED copies are not touched here: they are mirrors, deleted by + * the mirror-deletion step once their grants name the rule. Under a wall the + * switch is deployment-wide (ruling A item 1: no per-organization switch-off), + * so the copies block, named, instead of switching a rule off for every + * organization on the operator's behalf. Idempotent: a second run finds no + * row under a declared name. + */ +async function runDeclaredRuleCopies(run: Run): Promise { + const step: CeremonyStep = 'declared-rule-copies'; + const summary: Record = { clones: 0, activationRows: 0, grantsRemoved: 0 }; + const notes: string[] = []; + const tables: Array<{ physical: string; cols: ReadonlySet }> = []; + for (const physical of presentTables(run, SHARING_RULES)) { + const cols = await columnsOf(run, physical); + if (cols?.has('name')) tables.push({ physical, cols }); + } + if (tables.length === 0) return { step, status: 'done', summary, blockers: [], notes }; + + // A copy is a row that is not a pristine mirror. + const copyWhere = (physical: string, cols: ReadonlySet): { text: string; params: unknown[] } => { + const { sql } = databaseFor(run, physical); + if (!cols.has('managed_by')) return { text: '1 = 1', params: [] }; + const mirror = predicateSql(sql, physical, cols, cols.has('customized') + ? [{ column: 'managed_by', in: SEEDED_PROVENANCE }, { column: 'customized', notTrue: true }] + : [{ column: 'managed_by', in: SEEDED_PROVENANCE }]); + return { text: `(CASE WHEN ${mirror.text} THEN 1 WHEN NOT (${mirror.text}) THEN 0 ELSE 1 END) = 0`, params: [...mirror.params, ...mirror.params] }; + }; + if (!run.opts.declaredSharingRules) { + let customized = 0; + for (const { physical, cols } of tables) { + if (!cols.has('customized') || !cols.has('managed_by')) continue; + const { sql } = databaseFor(run, physical); + const where = predicateSql(sql, physical, cols, [{ column: 'managed_by', in: SEEDED_PROVENANCE }, { column: 'customized', equals: true }]); + customized += await sql.count(physical, `SELECT COUNT(*) AS n FROM ${sql.q(physical, physical)} WHERE ${where.text}`, where.params); + } + if (customized === 0) return { step, status: 'done', summary, blockers: [], notes }; + return blockedStep(step, summary, [ + `${SHARING_RULES} holds ${customized} customized copy(ies) of a declared sharing rule, settled only against the ` + + 'rules the deployment declares, and this run has no registry to read them from. Run the command against the ' + + "deployment's own composition, then re-run.", + ]); + } + const declared = await run.opts.declaredSharingRules.rules(); + const packageOf = new Map(declared.map((r) => [r.name, r.packageId ?? null] as const)); + const names = [...packageOf.keys()]; + if (names.length === 0) return { step, status: 'done', summary, blockers: [], notes }; + + const copies: Array<{ physical: string; cols: ReadonlySet; id: string; name: string }> = []; + for (const { physical, cols } of tables) { + const { sql } = databaseFor(run, physical); + const copy = copyWhere(physical, cols); + for (let i = 0; i < names.length; i += IN_CHUNK) { + const chunk = names.slice(i, i + IN_CHUNK); + const rows = await readAll(run, physical, ['name'], { + text: `(${copy.text}) AND ${sql.q('name', physical)} IN (${chunk.map(() => '?').join(', ')})`, + params: [...copy.params, ...chunk], + }); + for (const row of rows) copies.push({ physical, cols, id: String(row.id), name: String(row.name) }); + } + } + if (copies.length === 0) return { step, status: 'done', summary, blockers: [], notes }; + if (run.opts.posture !== 'single') { + return blockedStep(step, summary, [ + `${SHARING_RULES} holds ${copies.length} stored rule(s) under the name of a rule a package declares ` + + `(${copies.map((c) => `${c.id} ${c.name}`).sort().join(', ')}). Under the ${run.opts.posture} posture the ` + + 'declaration\'s switch is deployment-wide — there is no per-organization switch-off — so it is not switched ' + + 'off for every organization on the operator\'s behalf, and nothing was written. Rename each row to a name no ' + + 'package declares (it stays that organization\'s own rule), switch the declared rule on or off through the ' + + 'activation door as the deployment wants, then re-run.', + ]); + } + + const shares: string[] = []; + for (const physical of presentTables(run, RECORD_SHARES)) { + const cols = await columnsOf(run, physical); + if (cols?.has('source') && cols.has('source_id')) shares.push(physical); + } + const taken = new Set(names); + for (const copy of copies) { + const { sql } = databaseFor(run, copy.physical); + const q = (c: string): string => sql.q(c, copy.physical); + const clone = await freeName(run, copy.physical, copy.name, taken); + const sets: Array<[string, unknown]> = [ + ['name', clone], + ...(copy.cols.has('managed_by') ? [['managed_by', 'admin'] as [string, unknown]] : []), + ...(copy.cols.has('customized') ? [['customized', sql.bind(false)] as [string, unknown]] : []), + ]; + await execute( + run, + copy.physical, + `UPDATE ${q(copy.physical)} SET ${sets.map(([c]) => `${q(c)} = ?`).join(', ')} WHERE ${q('id')} = ? AND ${q('name')} = ?`, + [...sets.map(([, v]) => v), copy.id, copy.name], + ); + const [landed] = await sql.read(copy.physical, `SELECT ${q('name')} AS ${q('name')} FROM ${q(copy.physical)} WHERE ${q('id')} = ?`, [copy.id]); + if (text(landed?.name) !== clone) throw new Error(`The clone of the declared sharing rule ${copy.name} (${copy.id}) did not read back as ${clone}.`); + summary.clones++; + notes.push(`${SHARING_RULES} ${copy.id}: now the rule ${clone}, its grants with it; the declared ${copy.name} is switched off.`); + } + for (const name of [...new Set(copies.map((c) => c.name))].sort()) { + if (await setActivation(run, 'sharing_rule', name, packageOf.get(name) ?? null, false)) summary.activationRows++; + for (const physical of shares) { + const { sql } = databaseFor(run, physical); + const where = `${sql.q('source', physical)} = ? AND ${sql.q('source_id', physical)} = ?`; + const count = (): Promise => sql.count(physical, `SELECT COUNT(*) AS n FROM ${sql.q(physical, physical)} WHERE ${where}`, ['rule', name]); + const before = await count(); + if (before === 0) continue; + await execute(run, physical, `DELETE FROM ${sql.q(physical, physical)} WHERE ${where}`, ['rule', name]); + if ((await count()) !== 0) throw new Error(`Grants of the switched-off declared rule ${name} were left on ${physical}.`); + summary.grantsRemoved += before; + } + } + run.opts.onProgress?.(`declared sharing rules: ${summary.clones} stored copy(ies) now clones, their declarations switched off`); + return { step, status: 'done', summary, blockers: [], notes }; +} + +// ── Step 6: organization_id NOT NULL (18.x step 5d, 8e) ───────────────────── + +/** One NOT NULL the ceremony applies: the tenant column of a table it cleared. */ +export interface NotNullColumn { + table: string; + column: string; + /** The database beside the primary that holds the table (`ApplySibling.name`); absent for the primary. */ + database?: string; +} + +/** One NOT NULL, as the schema door answered it. */ +export interface NotNullOutcome extends NotNullColumn { + /** + * `applied`; `not-declared` — the composition this run booted declares no + * object for the table, so nothing rebuilds it against a declaration; + * `unsupported` — the datasource's transport cannot apply it (the Turso + * remote face cannot rebuild a table); `failed`. + */ + outcome: 'applied' | 'not-declared' | 'unsupported' | 'failed'; + detail?: string; +} + +/** + * Applies NOT NULL to a tenant column through the schema-drift ceremony's own + * apply (ADR-0120 D4), never a hand `ALTER`: on SQLite the table rebuild, on + * PostgreSQL / MySQL `ALTER COLUMN … SET NOT NULL`. The differ owns no + * proposal for it — the column's nullability is the ceremony's (ruling record + * 6269955919 on the v18 epic, ② A), and the differ reads it as no drift. + */ +export interface NotNullPort { + apply(columns: readonly NotNullColumn[]): Promise; +} + +/** + * [18.x step 5d; ADR-0131 8e; ruling ② A] After the column drops, every table + * whose fate keeps `organization_id` and whose post-step plan reads it ready + * for NOT NULL (`TablePlan.notNull.willReceive`, zero NULL rows) receives it, + * on every physical table (rotation shards included). The ceremony owns that + * nullability; no metadata says it. + * + * A table the plan does not clear — rows still NULL and reported, a fate-4 + * table whose column is not the tenancy anchor — is not touched and its reason + * is reported. A run with no schema door (`notNull` absent) applies nothing + * and says so; the command always passes one. A transport that cannot apply + * it is reported, never certified as applied; any other failure blocks. + */ +async function runNotNull(run: Run): Promise { + const step: CeremonyStep = 'not-null'; + const summary: Record = { applied: 0, due: 0, notDeclared: 0, unsupported: 0 }; + const blockers: string[] = []; + const notes: string[] = []; + const plan = await planFor(run); + const due: NotNullColumn[] = []; + for (const table of plan.tables) { + if (table.fate !== 'attribution' && table.fate !== 'report') continue; + if (table.physical !== 'present' || table.organizationColumn !== 'present' || !table.rows) continue; + if (!table.notNull.willReceive || table.rows.organizationNull !== 0) continue; + due.push(...table.tables.map((t) => ({ table: t, column: ORG, ...(table.database ? { database: table.database } : {}) }))); + } + summary.due = due.length; + if (due.length === 0) return { step, status: 'done', summary, blockers, notes }; + if (!run.opts.notNull) { + notes.push(`${due.length} table(s) are ready for NOT NULL on ${ORG}, and this run has no schema door to apply it: nothing was applied.`); + return { step, status: 'done', summary, blockers, notes }; + } + for (const outcome of await run.opts.notNull.apply(due)) { + if (outcome.outcome === 'applied') summary.applied++; + else if (outcome.outcome === 'not-declared') { + summary.notDeclared++; + notes.push( + `${outcome.table}.${outcome.column} stays nullable: the composition this run booted declares no object for ` + + `${outcome.table}. Its rows hold no NULL; re-run the ceremony where the deployment's composition is declared to apply it.`, + ); + } else if (outcome.outcome === 'unsupported') { + summary.unsupported++; + notes.push(`${outcome.table}.${outcome.column} stays nullable: ${outcome.detail ?? 'this transport cannot apply NOT NULL'}.`); + } else { + blockers.push(`NOT NULL on ${outcome.table}.${outcome.column} failed: ${outcome.detail ?? 'no detail'}. Re-run once the cause is fixed.`); + } + } + if (summary.applied) run.opts.onProgress?.(`not null: ${summary.applied} table(s) now hold ${ORG} NOT NULL`); + return { step, status: blockers.length ? 'blocked' : 'done', summary, blockers, notes }; +} + // ── The run ────────────────────────────────────────────────────────────────── const RUNNERS: Record Promise> = { @@ -2933,10 +3767,14 @@ const RUNNERS: Record Promise> = { 'metadata-promotion': runMetadataPromotion, 'template-promotion': runTemplatePromotion, 'settings-move': runSettingsMove, + 'sibling-moves': runSiblingMoves, + 'declared-webhooks': runDeclaredWebhooks, + 'declared-rule-copies': runDeclaredRuleCopies, rewrite: runRewrite, 'mirror-deletion': runMirrorDeletion, 'history-renumber': runHistoryRenumber, 'column-drops': runColumnDrops, + 'not-null': runNotNull, }; /** @@ -3118,8 +3956,8 @@ export async function applyOrganizationOwnership(opts: ApplyOptions): Promise { + dir = mkdtempSync(join(tmpdir(), 'os-d10-not-null-')); + driver = new SqlDriver({ client: 'better-sqlite3', connection: { filename: join(dir, 'db.sqlite') }, useNullAsDefault: true }); + (driver as any).logger = { warn: () => {}, info: () => {}, error: () => {}, debug: () => {} }; + knex = (driver as any).knex; + await driver.initObjects([ + { name: 'crm_deal', fields: { name: { type: 'text' }, organization_id: { type: 'lookup', reference: 'sys_organization', required: false } } }, + ] as never); + await knex('crm_deal').insert([ + { id: 'deal_1', name: 'Acme', organization_id: 'org_default' }, + { id: 'deal_2', name: 'Globex', organization_id: 'org_default' }, + ]); + await knex.schema.createTable('legacy_table', (t: any) => { + t.string('id').primary(); + t.string('organization_id'); + }); +}); + +afterEach(async () => { + await driver.disconnect?.(); + rmSync(dir, { recursive: true, force: true }); +}); + +const nullable = async (table: string, column: string): Promise => (await knex(table).columnInfo())[column].nullable; + +describe('organization_id NOT NULL through the drift apply door', () => { + it('⭐ applies it on a declared table, keeps its rows, and the next drift scan reports nothing for it', async () => { + expect(await nullable('crm_deal', 'organization_id')).toBe(true); + const outcomes = await applyNotNullThroughDrift( + { isDeclared: (t) => t === 'crm_deal', driverFor: () => driver as never }, + [{ table: 'crm_deal', column: 'organization_id' }], + ); + expect(outcomes).toEqual([{ table: 'crm_deal', column: 'organization_id', outcome: 'applied' }]); + expect(await nullable('crm_deal', 'organization_id')).toBe(false); + expect(await knex('crm_deal').select('id', 'organization_id').orderBy('id')).toEqual([ + { id: 'deal_1', organization_id: 'org_default' }, + { id: 'deal_2', organization_id: 'org_default' }, + ]); + const drift = (await driver.detectManagedDrift()).filter((d) => d.table === 'crm_deal' && d.kind === 'nullability_mismatch'); + expect(drift).toEqual([]); + // A NULL can no longer land. + await expect(knex('crm_deal').insert({ id: 'deal_3', name: 'Initech', organization_id: null })).rejects.toThrow(/NOT NULL/); + }); + + it('a table the booted composition does not declare is not-declared, and nothing touches it', async () => { + const outcomes = await applyNotNullThroughDrift( + { isDeclared: (t) => t === 'crm_deal', driverFor: () => driver as never }, + [{ table: 'legacy_table', column: 'organization_id' }], + ); + expect(outcomes).toEqual([{ table: 'legacy_table', column: 'organization_id', outcome: 'not-declared' }]); + expect(await nullable('legacy_table', 'organization_id')).toBe(true); + }); + + it('a transport whose apply door cannot perform it is unsupported, never attempted', async () => { + let attempted = 0; + const remote = { + driftApplyLimit: (op: string) => (op === 'tighten_not_null' ? 'the remote face runs no SQLite table rebuild' : null), + applyMigrationEntries: async () => { + attempted++; + return { applied: [], skipped: [] }; + }, + }; + const outcomes = await applyNotNullThroughDrift( + { isDeclared: () => true, driverFor: () => remote as never }, + [{ table: 'crm_deal', column: 'organization_id', database: 'telemetry' }], + ); + expect(outcomes).toEqual([ + { table: 'crm_deal', column: 'organization_id', outcome: 'unsupported', detail: 'the remote face runs no SQLite table rebuild' }, + ]); + expect(attempted).toBe(0); + }); + + it('a rotation shard is unsupported, never rebuilt: it has no declaration of its own', async () => { + let attempted = 0; + const outcomes = await applyNotNullThroughDrift( + { + isDeclared: () => true, + shardBaseOf: (t) => (t.startsWith('sys_activity__') ? 'sys_activity' : null), + driverFor: () => ({ applyMigrationEntries: async () => { attempted++; return { applied: [], skipped: [] }; } }) as never, + }, + [{ table: 'sys_activity__r20261019', column: 'organization_id' }], + ); + expect(outcomes).toEqual([ + { table: 'sys_activity__r20261019', column: 'organization_id', outcome: 'unsupported', detail: expect.stringMatching(/rotation shard of sys_activity/) }, + ]); + expect(attempted).toBe(0); + }); + + it('no driver for the table\'s database fails, naming the database', async () => { + const [outcome] = await applyNotNullThroughDrift( + { isDeclared: () => true, driverFor: () => undefined }, + [{ table: 'crm_deal', column: 'organization_id', database: 'telemetry' }], + ); + expect(outcome).toMatchObject({ outcome: 'failed', detail: expect.stringMatching(/database telemetry/) }); + }); +}); diff --git a/packages/cli/src/utils/organization-ownership-not-null.ts b/packages/cli/src/utils/organization-ownership-not-null.ts new file mode 100644 index 00000000000..056dc2262be --- /dev/null +++ b/packages/cli/src/utils/organization-ownership-not-null.ts @@ -0,0 +1,84 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import type { NotNullColumn, NotNullOutcome } from './organization-ownership-apply.js'; +import type { SqlDriverLike } from './schema-migrate.js'; + +type DriftEntry = Parameters[0][number]; + +export interface NotNullTarget { + /** The driver of the database the plan placed the table in (`database` absent: the primary). */ + driverFor(database: string | undefined): Pick | undefined; + /** Whether the booted composition declares an object for this table (a rotation shard judged by its base). */ + isDeclared(table: string): boolean; + /** The declared table a rotation shard belongs to (`sys_activity__r20261019` → `sys_activity`), or `null`. */ + shardBaseOf?(table: string): string | null; +} + +/** + * The ceremony's `notNull` port (18.x step 5d; ruling record 6269955919 on the + * v18 epic, ② A): each tenant column goes NOT NULL through the driver's own + * drift apply door — the SQLite table rebuild, or `ALTER COLUMN … SET NOT + * NULL` — never a hand `ALTER`. + * + * The differ proposes no entry for it (the column's nullability is the + * ceremony's, not a declaration's), so the entry is the ceremony's own + * `tighten_not_null`, handed to the one driver that holds the table. A table + * the composition does not declare is not rebuilt (`not-declared`); a + * transport whose apply door cannot perform the op (the Turso remote face, + * `driftApplyLimit`) is `unsupported`, never attempted; so is a rotation + * shard: the apply door rebuilds a table against its own declaration, which a + * shard does not have, and every shard rotated in later is created from the + * base's declaration, nullable — the shard family holds NOT NULL nowhere. + */ +export async function applyNotNullThroughDrift(target: NotNullTarget, columns: readonly NotNullColumn[]): Promise { + const outcomes: NotNullOutcome[] = []; + for (const { table, column, database } of columns) { + if (!target.isDeclared(table)) { + outcomes.push({ table, column, outcome: 'not-declared' }); + continue; + } + const base = target.shardBaseOf?.(table) ?? null; + if (base !== null && base !== table) { + outcomes.push({ + table, + column, + outcome: 'unsupported', + detail: `${table} is a rotation shard of ${base}: the drift apply door rebuilds only a declared table, and each new shard is created with ${base}'s nullable column`, + }); + continue; + } + const driver = target.driverFor(database); + if (!driver) { + outcomes.push({ table, column, outcome: 'failed', detail: `no driver serves the database ${database ?? 'primary'} the table is in` }); + continue; + } + const limit = driver.driftApplyLimit?.('tighten_not_null') ?? null; + if (limit) { + outcomes.push({ table, column, outcome: 'unsupported', detail: limit }); + continue; + } + const entry = { + kind: 'nullability_mismatch', + remoteName: table, + table, + column, + expected: 'NOT NULL', + actual: 'NULL', + severity: 'error', + category: 'destructive', + op: { type: 'tighten_not_null', table, column }, + message: `${table}.${column}: NOT NULL, applied by the organization-ownership ceremony (ADR-0131 D10, 8e).`, + } as unknown as DriftEntry; + try { + const { applied, skipped } = await driver.applyMigrationEntries([entry], { allowDestructive: true }); + outcomes.push( + applied.length === 1 + ? { table, column, outcome: 'applied' } + : { table, column, outcome: 'failed', detail: `the drift apply door did not apply it (${skipped.length} skipped)` }, + ); + } catch (error) { + outcomes.push({ table, column, outcome: 'failed', detail: error instanceof Error ? error.message : String(error) }); + } + } + return outcomes; +} diff --git a/packages/cli/src/utils/organization-ownership-platform-global-column-drop.integration.test.ts b/packages/cli/src/utils/organization-ownership-platform-global-column-drop.integration.test.ts index 71da34a2ad2..1e8723c05ff 100644 --- a/packages/cli/src/utils/organization-ownership-platform-global-column-drop.integration.test.ts +++ b/packages/cli/src/utils/organization-ownership-platform-global-column-drop.integration.test.ts @@ -67,7 +67,7 @@ import { fileURLToPath, pathToFileURL } from 'node:url'; import { ORGANIZATION_OWNERSHIP_MIGRATION_ID } from '@objectstack/spec/system'; import { isExitSignal } from './format.js'; import MigrateApply from '../commands/migrate/apply.js'; -import { OrganizationOwnershipCeremony as MigrateOrganizationOwnership } from '../commands/migrate/organization-ownership.js'; +import MigrateOrganizationOwnership from '../commands/migrate/organization-ownership.js'; // Paid at module load, not inside a clocked case (`check:test-source-alias`). import '@objectstack/runtime'; @@ -193,6 +193,15 @@ async function bootV17(dbFile: string): Promise { await kernel.use(osgcObjectsPlugin() as any); await runtime.start(); await kernel.shutdown(); + // This boot of THIS tree created the store from empty, so it wrote this + // release's birth marker; the v17 database it stands in for carries none, + // and the drop gate below reads that absence. + const db = new Database(dbFile); + try { + db.prepare(`DELETE FROM sys_migration WHERE id = 'adr-0131-organization-ownership'`).run(); + } finally { + db.close(); + } } /** A supplement row: `column-drop` for `drop`, `report` for every other host object. */ @@ -437,8 +446,6 @@ describe('os migrate apply --allow-destructive: the organization_id drop on a ho marker: null, }); expect(payload.detail).toMatch(/organization-ownership ceremony/); - // 18.0 refuses `os migrate organization-ownership` on every database: the refusal names no step of it. - expect(payload.detail).not.toMatch(/os migrate organization-ownership|--plan-file|organization-ownership --out/); // The drops the refusal held back are the ones the drift offered. expect(organizationDropsIn(payload.skipped)).toEqual([...DECLARED].sort()); for (const table of HOST_OBJECTS) expect(hasOrganizationColumn(dbFile, table), table).toBe(true); diff --git a/packages/cli/src/utils/schema-migrate.integration.test.ts b/packages/cli/src/utils/schema-migrate.integration.test.ts index 19c9b4a31fb..eb734fa9c0c 100644 --- a/packages/cli/src/utils/schema-migrate.integration.test.ts +++ b/packages/cli/src/utils/schema-migrate.integration.test.ts @@ -42,7 +42,8 @@ describe('bootSchemaStack + migrate engine (integration)', () => { name: 'mig_biz_unit', fields: { name: { type: 'text', required: true }, - organization_id: { type: 'text', required: false }, // optional now + organization_id: { type: 'text', required: false }, + region: { type: 'text', required: false }, // optional now code: { type: 'text', unique: true }, // tenant-scoped since #3696 }, }, @@ -50,8 +51,10 @@ describe('bootSchemaStack + migrate engine (integration)', () => { }), ); - // Seed a "legacy" DB: organization_id NOT NULL (the #2178 shape) plus the - // platform-wide unique index on `code` the pre-#3696 driver emitted. + // Seed a "legacy" DB: an optional field's column NOT NULL (the #2178 shape, + // re-keyed off `organization_id`, whose NOT NULL is the ownership + // ceremony's and never drift) plus the platform-wide unique index on + // `code` the pre-#3696 driver emitted. const seed = new SqlDriver({ client: 'better-sqlite3', connection: { filename: dbFile }, useNullAsDefault: true }); const k = (seed as any).knex; await k.schema.createTable('mig_biz_unit', (t: any) => { @@ -59,11 +62,12 @@ describe('bootSchemaStack + migrate engine (integration)', () => { t.timestamp('created_at'); t.timestamp('updated_at'); t.string('name').notNullable(); - t.string('organization_id').notNullable(); + t.string('organization_id'); + t.string('region').notNullable(); t.string('code'); }); await k.raw('CREATE UNIQUE INDEX mig_biz_unit_code_unique ON mig_biz_unit (code)'); - await k('mig_biz_unit').insert({ id: '1', name: 'Acme', organization_id: 'org1', code: 'BU-00001' }); + await k('mig_biz_unit').insert({ id: '1', name: 'Acme', organization_id: 'org1', region: 'emea', code: 'BU-00001' }); await k.destroy(); savedEnv.OS_ARTIFACT_PATH = process.env.OS_ARTIFACT_PATH; @@ -87,8 +91,8 @@ describe('bootSchemaStack + migrate engine (integration)', () => { // Boot changed nothing — this is exactly what `os migrate plan` renders. const drift = await stack.driver!.detectManagedDrift(); - const org = drift.find((d) => d.table === 'mig_biz_unit' && d.column === 'organization_id'); - expect(org, 'expected drift on mig_biz_unit.organization_id').toBeDefined(); + const org = drift.find((d) => d.table === 'mig_biz_unit' && d.column === 'region'); + expect(org, 'expected drift on mig_biz_unit.region').toBeDefined(); // ADR-0113: a column stricter than its declaration is needs_confirm — // reported by plan, applied by an explicit `os migrate apply` (the human // IS the confirmation), but never by dev auto-reconcile. @@ -111,7 +115,7 @@ describe('bootSchemaStack + migrate engine (integration)', () => { expect(skipped).toHaveLength(0); const after = await stack.driver!.detectManagedDrift(); - expect(after.find((d) => d.table === 'mig_biz_unit' && d.column === 'organization_id')).toBeUndefined(); + expect(after.find((d) => d.table === 'mig_biz_unit' && d.column === 'region')).toBeUndefined(); expect(after.find((d) => d.op.type === 'replace_unique_index')).toBeUndefined(); // The seeded row survived the reconcile. diff --git a/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts b/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts index af35d2323fe..b63ede0b718 100644 --- a/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts +++ b/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts @@ -65,7 +65,7 @@ import MigrateDuplicates from '../commands/migrate/duplicates.js'; import MigrateFilesToReferences from '../commands/migrate/files-to-references.js'; import MigrateMeta from '../commands/migrate/meta.js'; import MigrateMultiValueColumns from '../commands/migrate/multi-value-columns.js'; -import { OrganizationOwnershipCeremony as MigrateOrganizationOwnership } from '../commands/migrate/organization-ownership.js'; +import MigrateOrganizationOwnership from '../commands/migrate/organization-ownership.js'; import MigratePlan from '../commands/migrate/plan.js'; import MigrateRecordedBy from '../commands/migrate/recorded-by.js'; import MigrateResume from '../commands/migrate/resume.js'; diff --git a/packages/cli/src/utils/schema-migrate.sql-driver-lookup.test.ts b/packages/cli/src/utils/schema-migrate.sql-driver-lookup.test.ts index b4a8932615f..0ec2adc531d 100644 --- a/packages/cli/src/utils/schema-migrate.sql-driver-lookup.test.ts +++ b/packages/cli/src/utils/schema-migrate.sql-driver-lookup.test.ts @@ -53,13 +53,13 @@ import { findSqlDriverForKernel } from './schema-migrate.js'; import '@objectstack/metadata-protocol'; import '@objectstack/platform-objects/system'; -type Command = (typeof import('../commands/migrate/organization-ownership.js'))['OrganizationOwnershipCeremony']; +type Command = (typeof import('../commands/migrate/organization-ownership.js'))['default']; let MigrateOrganizationOwnership: Command; // The command's module graph is large, and oclif reads the package's config on // the first run: both are paid outside the case timeout. beforeAll(async () => { - ({ OrganizationOwnershipCeremony: MigrateOrganizationOwnership } = await import('../commands/migrate/organization-ownership.js')); + ({ default: MigrateOrganizationOwnership } = await import('../commands/migrate/organization-ownership.js')); }, 60_000); const CASE_TIMEOUT_MS = 60_000; diff --git a/packages/cli/vitest.config.ts b/packages/cli/vitest.config.ts index f7245834057..373b69f73e3 100644 --- a/packages/cli/vitest.config.ts +++ b/packages/cli/vitest.config.ts @@ -806,6 +806,14 @@ export default defineConfig({ // driver is its storage. Resolved to source for the reason the entry above // gives: a verdict about the checkout, not about the last build. Anchored // like its neighbours (driver-sqlite-wasm publishes only `"."`). + // [18.x step 5a] `utils/organization-ownership-apply.generation-1.test.ts` + // pins the ceremony's `declared:NAME` spelling equal to plugin-webhooks' + // own `declaredWebhookRefId`; resolved to source, so the pin is a verdict + // about the checkout's spelling, not the last build's. + { + find: /^@objectstack\/plugin-webhooks$/, + replacement: path.resolve(__dirname, '../plugins/plugin-webhooks/src/index.ts'), + }, { find: /^@objectstack\/driver-sqlite-wasm$/, replacement: path.resolve(__dirname, '../drivers/driver-sqlite-wasm/src/index.ts'), diff --git a/packages/drivers/driver-sql/src/schema-drift.ts b/packages/drivers/driver-sql/src/schema-drift.ts index 3d59867fd4e..7aed35b83a2 100644 --- a/packages/drivers/driver-sql/src/schema-drift.ts +++ b/packages/drivers/driver-sql/src/schema-drift.ts @@ -927,8 +927,20 @@ export function diffManagedTable(args: { * as the #15771 corruption it still is on an unmoved deployment. */ fileColumnsMoved?: boolean; + /** + * The table's tenant column (`SqlDriver.resolveTenantField`): the one column + * whose nullability the v18 organization-ownership ceremony owns (ruling + * record 6269955919 on the v18 epic, ② A). The ceremony applies NOT NULL to + * it on a database it upgraded, and no declaration says so — the injected + * tenant field is `required: false` with no `storage` — so a NOT NULL on it + * is the ceremony's verdict, never drift: no `relax_not_null` is proposed + * for it, and nothing here undoes the ceremony's work. Omitted (the + * default), every column is judged against its declaration as before. + */ + ceremonyOwnedNullability?: string | null; }): ManagedDriftEntry[] { const { table, fields, columns, dialect, keyedColumns, varcharColumnChars } = args; + const ceremonyOwnedNullability = args.ceremonyOwnedNullability ?? null; const fileColumnsMoved = args.fileColumnsMoved === true; const out: ManagedDriftEntry[] = []; @@ -950,7 +962,7 @@ export function diffManagedTable(args: { // explicit physical constraint — NOT against `required`, which is the // write-time contract and implies nothing about the column) ────────── const expectNullable = field.storage?.notNull !== true; - if (expectNullable && !col.nullable && field.required !== true) { + if (expectNullable && !col.nullable && field.required !== true && fieldName !== ceremonyOwnedNullability) { // Column STRICTER than its declaration — and the field is not even // write-gated, so an omitting write reaches the DB and dies as a raw // driver error instead of a clean validation 400. That surprise is diff --git a/packages/drivers/driver-sql/src/sql-driver-11722-sqlite-rebuild-applied-honesty.test.ts b/packages/drivers/driver-sql/src/sql-driver-11722-sqlite-rebuild-applied-honesty.test.ts index 2d0a4da7298..7bb8fd726c0 100644 --- a/packages/drivers/driver-sql/src/sql-driver-11722-sqlite-rebuild-applied-honesty.test.ts +++ b/packages/drivers/driver-sql/src/sql-driver-11722-sqlite-rebuild-applied-honesty.test.ts @@ -159,15 +159,15 @@ describe('SqlDriver SQLite reconcile reports only what the rebuild honoured (#11 t.timestamp('created_at'); t.timestamp('updated_at'); t.string('name'); - t.string('organization_id').notNullable(); + t.string('region_code').notNullable(); }); - await knexInstance('biz_unit').insert({ id: '1', name: 'Acme', organization_id: 'org1' }); + await knexInstance('biz_unit').insert({ id: '1', name: 'Acme', region_code: 'emea' }); await driver.initObjects([ { name: 'biz_unit', fields: { name: { type: 'string' }, - organization_id: { type: 'string', required: false }, + region_code: { type: 'string', required: false }, }, }, ]); @@ -192,10 +192,10 @@ describe('SqlDriver SQLite reconcile reports only what the rebuild honoured (#11 // The rebuild genuinely ran: the constraint is gone and the row survived. const info = await knexInstance('biz_unit').columnInfo(); - expect(info.organization_id.nullable).toBe(true); + expect(info.region_code.nullable).toBe(true); expect(info).toHaveProperty('name'); expect(await knexInstance('biz_unit').select('*')).toMatchObject([ - { id: '1', name: 'Acme', organization_id: 'org1' }, + { id: '1', name: 'Acme', region_code: 'emea' }, ]); }); }); diff --git a/packages/drivers/driver-sql/src/sql-driver-schema-drift.test.ts b/packages/drivers/driver-sql/src/sql-driver-schema-drift.test.ts index 26e45041764..3d15987ac8a 100644 --- a/packages/drivers/driver-sql/src/sql-driver-schema-drift.test.ts +++ b/packages/drivers/driver-sql/src/sql-driver-schema-drift.test.ts @@ -30,38 +30,57 @@ describe('SqlDriver managed-schema drift (#2186)', () => { await knexInstance?.destroy(); }); - // Build an "existing DB" table where organization_id is NOT NULL (the shape - // a DB created while the field was `required: true` has — the #2178 repro). + // Build an "existing DB" table where an optional field's column is NOT NULL + // (the shape a DB created while the field was `required: true` has — the + // #2178 repro, re-keyed off `region_code`: the tenant column's + // nullability is the ownership ceremony's, never drift — pinned below). const seedLegacyTable = async (driver: SqlDriver) => { await knexInstance.schema.createTable('biz_unit', (t: any) => { t.string('id').primary(); t.timestamp('created_at'); t.timestamp('updated_at'); t.string('name').notNullable(); - t.string('organization_id').notNullable(); + t.string('region_code').notNullable(); }); - await knexInstance('biz_unit').insert({ id: '1', name: 'Acme', organization_id: 'org1' }); + await knexInstance('biz_unit').insert({ id: '1', name: 'Acme', region_code: 'emea' }); }; - // Metadata after #2178: organization_id is now optional. + // Metadata after #2178: the field is now optional. const relaxedMeta = [ { name: 'biz_unit', fields: { name: { type: 'string', required: true }, - organization_id: { type: 'string', required: false }, + region_code: { type: 'string', required: false }, }, }, ]; describe('detectManagedDrift', () => { + it('a NOT NULL on the table\'s tenant column is no drift: its nullability is the ownership ceremony\'s (ruling ② A)', async () => { + const driver = makeDriver(); + await knexInstance.schema.createTable('biz_owned', (t: any) => { + t.string('id').primary(); + t.timestamp('created_at'); + t.timestamp('updated_at'); + t.string('organization_id').notNullable(); + t.string('note').notNullable(); + }); + await driver.initObjects([ + { name: 'biz_owned', fields: { organization_id: { type: 'string', required: false }, note: { type: 'string', required: false } } }, + ]); + const drift = (await driver.detectManagedDrift()).filter((d) => d.table === 'biz_owned' && d.kind === 'nullability_mismatch'); + // The ceremony's NOT NULL on organization_id is not reported; the same constraint on any other column still is. + expect(drift.map((d) => [d.column, d.op.type])).toEqual([['note', 'relax_not_null']]); + }); + it('flags a NOT NULL column on a non-required optional field as needs_confirm (ADR-0113 — ratify or relax, never auto-applied)', async () => { const driver = makeDriver(); await seedLegacyTable(driver); await driver.initObjects(relaxedMeta); const drift = await driver.detectManagedDrift(); - const orgDrift = drift.find((d) => d.column === 'organization_id'); + const orgDrift = drift.find((d) => d.column === 'region_code'); expect(orgDrift).toBeDefined(); expect(orgDrift!.kind).toBe('nullability_mismatch'); expect(orgDrift!.category).toBe('needs_confirm'); @@ -148,16 +167,16 @@ describe('SqlDriver managed-schema drift (#2186)', () => { // The column is NOT auto-relaxed... const info = await knexInstance('biz_unit').columnInfo(); - expect(info.organization_id.nullable).toBe(false); + expect(info.region_code.nullable).toBe(false); // ...the pre-existing row is untouched... const rows = await knexInstance('biz_unit').select('*'); expect(rows).toHaveLength(1); - expect(rows[0]).toMatchObject({ id: '1', name: 'Acme', organization_id: 'org1' }); + expect(rows[0]).toMatchObject({ id: '1', name: 'Acme', region_code: 'emea' }); // ...and the divergence is reported for a human decision. const drift = await driver.detectManagedDrift(); - const d = drift.find((x) => x.column === 'organization_id'); + const d = drift.find((x) => x.column === 'region_code'); expect(d?.category).toBe('needs_confirm'); expect(d?.op.type).toBe('relax_not_null'); }); @@ -188,7 +207,7 @@ describe('SqlDriver managed-schema drift (#2186)', () => { await seedLegacyTable(driver); await driver.initObjects(relaxedMeta); const info = await knexInstance('biz_unit').columnInfo(); - expect(info.organization_id.nullable).toBe(false); // unchanged + expect(info.region_code.nullable).toBe(false); // unchanged } finally { process.env.NODE_ENV = prev; } @@ -198,8 +217,8 @@ describe('SqlDriver managed-schema drift (#2186)', () => { describe('in-place ALTER SQL (Postgres / MySQL)', () => { // No live PG/MySQL in CI — assert the generated DDL by spying on knex.raw. const opFor = (type: string): any => ({ - kind: 'x', table: 'biz_unit', column: 'organization_id', severity: 'warning', - category: 'safe', message: 'm', op: { type, table: 'biz_unit', column: 'organization_id', to: 120 }, + kind: 'x', table: 'biz_unit', column: 'region_code', severity: 'warning', + category: 'safe', message: 'm', op: { type, table: 'biz_unit', column: 'region_code', to: 120 }, }); const spyDriver = (client: string, columnInfo?: any) => { @@ -227,7 +246,7 @@ describe('SqlDriver managed-schema drift (#2186)', () => { }); it('MySQL MODIFY reconstructs char length so a nullability change keeps it', async () => { - const { d, calls } = spyDriver('mysql2', { organization_id: { type: 'varchar', maxLength: 255 } }); + const { d, calls } = spyDriver('mysql2', { region_code: { type: 'varchar', maxLength: 255 } }); expect(await (d as any).applyDriftOpInPlace(opFor('relax_not_null').op)).toBe(true); expect(calls[0]).toMatch(/MODIFY \?\? varchar\(255\) NULL/); }); @@ -243,7 +262,7 @@ describe('SqlDriver managed-schema drift (#2186)', () => { const { applied, skipped } = await driver.applyMigrationEntries(drift, { allowDestructive: false }); expect(applied.some((d) => d.op.type === 'relax_not_null')).toBe(true); expect(skipped).toHaveLength(0); - expect((await knexInstance('biz_unit').columnInfo()).organization_id.nullable).toBe(true); + expect((await knexInstance('biz_unit').columnInfo()).region_code.nullable).toBe(true); }); it('skips destructive drops unless allowDestructive, then drops with it', async () => { diff --git a/packages/drivers/driver-sql/src/sql-driver.ts b/packages/drivers/driver-sql/src/sql-driver.ts index 7cf751abe1e..28ad32650fd 100644 --- a/packages/drivers/driver-sql/src/sql-driver.ts +++ b/packages/drivers/driver-sql/src/sql-driver.ts @@ -13800,6 +13800,9 @@ export class SqlDriver implements IDataDriver { // ADR-0104: the media family's column shape is this deployment's fact, // not the type's — the differ must ask the arm the emitter just used. fileColumnsMoved: this.fileColumnsMoved, + // ADR-0131 D10 (ruling record 6269955919, ② A): the tenant column's + // NOT NULL is the organization-ownership ceremony's, not drift. + ceremonyOwnedNullability: this.resolveTenantField(tableName), }); out.push(...(await this.detectTableIndexDrift(tableName, fields, declaredIndexes, new Set(cols.map((c) => c.name))))); return out; diff --git a/packages/drivers/driver-turso/src/remote-transport.ts b/packages/drivers/driver-turso/src/remote-transport.ts index 039d3cd860c..43b26640187 100644 --- a/packages/drivers/driver-turso/src/remote-transport.ts +++ b/packages/drivers/driver-turso/src/remote-transport.ts @@ -1038,6 +1038,16 @@ function isRemoteReadableIdsSemiJoin(value: unknown): value is RemoteReadableIds */ export class RemoteTransport { private client: Client | null = null; + /** + * The tables this transport's schema doors created, and the ones they found + * already present (one not created here first), since construction — the + * remote face's answer to `IDataDriver.getSchemaSyncStats`, counted from the + * `sqlite_master` probe each door already makes, exactly as + * `SqlDriver.initObjects` counts from its `hasTable` probe. A table counts as + * created only once its DDL was applied. + */ + private readonly tablesCreatedHere = new Set(); + private readonly tablesFoundExisting = new Set(); /** * Factory function for lazy (re)connection. @@ -2084,9 +2094,11 @@ export class RemoteTransport { args: [tableName], }); const exists = checkResult.rows.length > 0; + if (exists) this.noteFoundExisting(tableName); if (!exists) { await this.client!.execute(this.buildCreateTableSQL(tableName, objectDef)); + this.tablesCreatedHere.add(tableName); // [#8413 · #17609] The table was created empty microseconds ago, so its // declared indexes cannot fail on existing data — no isolation needed, // and any error here is a real DDL fault that should surface. @@ -2253,6 +2265,9 @@ export class RemoteTransport { if (ddlStatements.length > 0) { await this.client!.batch(ddlStatements, 'write'); } + // The batch is one transaction: its CREATE TABLEs landed together, or none did. + for (const { object } of newSchemas) this.tablesCreatedHere.add(object); + for (const { object } of existingSchemas) this.noteFoundExisting(object); // Phase 4 [#8413 · #17609]: retrofit the existing tables' missing declared // indexes, outside the batch above for the blast-radius reason stated at @@ -2266,6 +2281,21 @@ export class RemoteTransport { await this.client!.execute(`DROP TABLE IF EXISTS "${object}"`); } + /** + * Tables the schema doors created vs found since construction — the remote + * face's `IDataDriver.getSchemaSyncStats` (`TursoDriver` answers with it). + * `existing === 0 && created > 0` is the observation that the remote + * database was empty when this process reached it. + */ + schemaSyncStats(): { created: number; existing: number } { + return { created: this.tablesCreatedHere.size, existing: this.tablesFoundExisting.size }; + } + + /** A table found present, unless this transport created it itself (a re-sync is not a find). */ + private noteFoundExisting(table: string): void { + if (!this.tablesCreatedHere.has(table)) this.tablesFoundExisting.add(table); + } + // =================================== // Internal Helpers // =================================== diff --git a/packages/drivers/driver-turso/src/turso-driver.ts b/packages/drivers/driver-turso/src/turso-driver.ts index 1a3a5461af6..fb56e0d7998 100644 --- a/packages/drivers/driver-turso/src/turso-driver.ts +++ b/packages/drivers/driver-turso/src/turso-driver.ts @@ -1078,9 +1078,11 @@ export const REMOTE_FACE_ANSWERS = { deferredSchemaObjectCount: 'inherited', previewDeferredSchemaWork: 'inherited', flushDeferredSchemaDdl: 'inherited', - // `{ created: 0, existing: 0 }`: the remote doors count nothing, and both - // counts at zero is the `IDataDriver` contract's "cannot say". - getSchemaSyncStats: 'inherited', + // The remote schema doors count what their `sqlite_master` probe answers — + // created vs found — so a fresh remote database is observed as one (the + // organization-ownership birth marker reads it). A remote boot that ran no + // schema door still answers `{0, 0}`, the contract's "cannot say". + getSchemaSyncStats: 'remote', // [#23816] Drift and migration: the shared differ over the remote catalog, // and the index / column-drop entries applied through the libSQL client (see // "Remote schema drift" above for which entries, and why only those). @@ -4199,6 +4201,16 @@ export class TursoDriver extends SqlDriver { * statement and no round-trip. The parameter repeats the base's declared * shape key for key (`check:object-def-param-keys` arm C). */ + /** + * `IDataDriver.getSchemaSyncStats` — on the REMOTE face, what the transport's + * schema doors counted (`RemoteTransport.schemaSyncStats`); the local and + * embedded-replica faces answer with the inherited Knex count. + */ + override getSchemaSyncStats(): { created: number; existing: number } { + if (this.isRemote) return this.remoteTransport!.schemaSyncStats(); + return super.getSchemaSyncStats(); + } + override registerObjectMetadata( objects: Array<{ name: string; fields?: Record; tenancy?: any; indexes?: any[] }>, ): void { diff --git a/packages/drivers/driver-turso/src/turso-remote-inherited-members.test.ts b/packages/drivers/driver-turso/src/turso-remote-inherited-members.test.ts index aab6f9cac2e..061bf6e7625 100644 --- a/packages/drivers/driver-turso/src/turso-remote-inherited-members.test.ts +++ b/packages/drivers/driver-turso/src/turso-remote-inherited-members.test.ts @@ -402,6 +402,27 @@ describe('members whose remote answer is a declaration', () => { }); }); +describe('getSchemaSyncStats on the remote face: what its schema doors counted', () => { + it('a fresh remote database is observed as created from empty; the same database reached again is found; the local control counts too', async () => { + const file = nextFile(); + expect((await seeded('remote', file)).getSchemaSyncStats()).toEqual({ created: 2, existing: 0 }); + // A second process over the same remote database: every table is found. + const again = new TursoDriver({ url: 'libsql://probe.example.turso.io', client: createClient({ url: `file:${file}` }) }); + open.push({ driver: again, client: null }); + await again.connect(); + await again.initObjects([PROBE, TENANT]); + expect(again.getSchemaSyncStats()).toEqual({ created: 0, existing: 2 }); + // A re-sync of the tables this process created is not a find. + await again.syncSchemasBatch([{ object: 'probe_t', schema: PROBE }]); + expect(again.getSchemaSyncStats()).toEqual({ created: 0, existing: 2 }); + expect((await seeded('local')).getSchemaSyncStats()).toEqual({ created: 2, existing: 0 }); + }); + + it("a remote boot that ran no schema door says nothing: both counts at zero, the contract's \"cannot say\"", () => { + expect(bareRemote().getSchemaSyncStats()).toEqual({ created: 0, existing: 0 }); + }); +}); + describe('members inherited unchanged, and true on the remote face', () => { it('the deferred-DDL readers report nothing deferred, and arming stays refused', async () => { const remote = await seeded('remote'); @@ -411,10 +432,6 @@ describe('members inherited unchanged, and true on the remote face', () => { await expect(remote.flushDeferredSchemaDdl()).resolves.toEqual([]); }); - it("getSchemaSyncStats: both counts at zero, the contract's \"cannot say\"; the local control counts", async () => { - expect((await seeded('remote')).getSchemaSyncStats()).toEqual({ created: 0, existing: 0 }); - expect((await seeded('local')).getSchemaSyncStats()).toEqual({ created: 2, existing: 0 }); - }); it('the bookkeeping and dialect members answer as on the local face', async () => { const remote = await seeded('remote'); diff --git a/packages/metadata-core/src/legacy-organization-row.test.ts b/packages/metadata-core/src/legacy-organization-row.test.ts index ddff9cf1197..defc878c0b2 100644 --- a/packages/metadata-core/src/legacy-organization-row.test.ts +++ b/packages/metadata-core/src/legacy-organization-row.test.ts @@ -115,26 +115,26 @@ describe('readLegacyOrganizationRows — fails closed', () => { await expect(readLegacyOrganizationRows({ dialectName: 'sqlite' })).rejects.toThrow(/no raw-SQL seam/); }); - // What 18.0 supports, said to both readers: a driver-shaped refusal, which a - // fresh install meets too, and one only a database still carrying the - // column reaches. The ceremony 18.0 refuses is never the remedy. + // Said to both readers: a driver-shaped refusal, which a fresh install meets + // too, and one only a database still carrying the column reaches. The + // remedy for a database that holds them is the ceremony, which is open. it.each([ ['driver-shaped: no raw seam (any database, a fresh one included)', { dialectName: 'sqlite' }], ['database-shaped: the column is on disk, the legacy select answered nothing', sqlDriver({ dialectName: 'sqlite', columns: V17_COLUMNS, rows: null })], - ])('the refusal says what 18.0 supports, never the ceremony (%s)', async (_label, driver) => { + ])('the refusal tells a fresh install apart from a 17.x database, and names the ceremony for the second (%s)', async (_label, driver) => { const error = await readLegacyOrganizationRows(driver).then(() => null, (e: unknown) => e); expect(error).toBeInstanceOf(LegacyOrganizationRowsUnreadable); const refusal = error as LegacyOrganizationRowsUnreadable; expect(refusal.code).toBe('LEGACY_ORGANIZATION_ROWS_UNREADABLE'); expect(refusal.message).toContain('Nothing was listed or written.'); expect(refusal.message).toContain( - 'A fresh 18.0 install holds no legacy organization rows: there, the cause is the one named above, ' + + 'A fresh install holds no legacy organization rows: there, the cause is the one named above, ' + 'and the fix is in the SQL driver or the caller that met it.', ); expect(refusal.message).toMatch( - /A database that does hold them is a v17 one\. 18\.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17\.x release until the 18\.x release that carries the upgrade path\.$/, + /A database that does hold them was created by 17\.x: run `os migrate organization-ownership` on it first\.$/, ); - expect(refusal.message).not.toContain('os migrate organization-ownership'); + expect(refusal.message).not.toMatch(/fresh installs only|on a 17\.x release|carries the upgrade path/); }); it('a table outside the family is refused before anything runs', async () => { diff --git a/packages/metadata-core/src/legacy-organization-row.ts b/packages/metadata-core/src/legacy-organization-row.ts index e19b19399a0..c8179efb9f0 100644 --- a/packages/metadata-core/src/legacy-organization-row.ts +++ b/packages/metadata-core/src/legacy-organization-row.ts @@ -75,10 +75,9 @@ export class LegacyOrganizationRowsUnreadable extends Error { constructor(readonly table: string, detail: string) { super( `Could not tell ${table}'s legacy organization rows from the environment's: ${detail}. ` + - 'Nothing was listed or written. A fresh 18.0 install holds no legacy organization rows: there, the ' + + 'Nothing was listed or written. A fresh install holds no legacy organization rows: there, the ' + 'cause is the one named above, and the fix is in the SQL driver or the caller that met it. A database ' + - 'that does hold them is a v17 one. 18.0 supports fresh installs only and does not upgrade a v17 ' + - 'database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path.', + 'that does hold them was created by 17.x: run `os migrate organization-ownership` on it first.', ); this.name = 'LegacyOrganizationRowsUnreadable'; } diff --git a/packages/metadata-protocol/src/seed-loader-null-organization-holder.test.ts b/packages/metadata-protocol/src/seed-loader-null-organization-holder.test.ts index 0278178435e..54a75980ebd 100644 --- a/packages/metadata-protocol/src/seed-loader-null-organization-holder.test.ts +++ b/packages/metadata-protocol/src/seed-loader-null-organization-holder.test.ts @@ -135,15 +135,15 @@ describe('[#24301] a scoped load over a NULL-organization holder of a natural ke const [refusal] = refusalsOf(result); expect(refusal).toMatchObject({ sourceObject: 'tw_widget', targetField: 'sku', attemptedValue: 'W-HELD', recordIndex: 0 }); expect(refusal.message).toContain("'w_unowned'"); - // What 18.0 supports, never the ceremony it refuses; the first sentences + // The ceremony is the remedy, with its reviewed plan; the first sentences // (what was refused, and why) stay. expect(refusal.message).toContain('which has no organization (organization_id is NULL)'); expect(refusal.message).toContain('decided by the organization-ownership ceremony, not by a seed load.'); - expect(refusal.message).toMatch( - /A row with no organization predates organization stamping, so it is a v17 database's\. 18\.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17\.x release until the 18\.x release that carries the upgrade path\.$/, + expect(refusal.message).toContain( + 'Attribute the unowned rows first (`os migrate organization-ownership --out PLAN`, then the same command with ' + + '--apply --plan-file PLAN --backup-taken), then re-run this seed', ); - expect(refusal.message).not.toContain('os migrate organization-ownership'); - expect(refusal.message).not.toContain('--apply'); + expect(refusal.message).not.toMatch(/fresh installs only|on a 17\.x release|carries the upgrade path/); }); it("the organization's own row wins: a key held both by it and by an unowned row is matched, not refused", async () => { diff --git a/packages/metadata-protocol/src/seed-loader.ts b/packages/metadata-protocol/src/seed-loader.ts index a1382d99c46..3b0cb448f35 100644 --- a/packages/metadata-protocol/src/seed-loader.ts +++ b/packages/metadata-protocol/src/seed-loader.ts @@ -1848,10 +1848,10 @@ export class SeedLoaderService implements ISeedLoaderService { `which has no organization (organization_id is NULL), and this seed load writes for organization ` + `${config.organizationId}. Inserting the record would put a twin beside that row, and updating the ` + `row would hand it to this organization: which organization an unowned row belongs to is decided by ` + - `the organization-ownership ceremony, not by a seed load. A row with no organization predates ` + - `organization stamping, so it is a v17 database's. 18.0 supports fresh installs only and does not ` + - `upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries ` + - `the upgrade path.`, + `the organization-ownership ceremony, not by a seed load. Attribute the unowned rows first ` + + `(\`os migrate organization-ownership --out PLAN\`, then the same command with ` + + `--apply --plan-file PLAN --backup-taken), then re-run this seed: the row is this organization's own ` + + `by then, and the upsert matches it.`, }; errors.push(error); allErrors.push(error); diff --git a/packages/objectql/src/engine.ts b/packages/objectql/src/engine.ts index d00a9567287..491bfce234a 100644 --- a/packages/objectql/src/engine.ts +++ b/packages/objectql/src/engine.ts @@ -214,7 +214,6 @@ import { resolveSystemWriteOrganization, resolveTenantFieldName, carriesOrganization, - isWalledOrganizationLessBridgeObject, SystemWriteOrganizationRequiredError, ORGANIZATION_OBJECT, } from './tenancy/system-write-organization.js'; @@ -7396,16 +7395,10 @@ export class ObjectQL implements IObjectQLEngine { // organization or refused below — never left NULL. The walled half landed // in 18.x, after the stages that gave the walled writers (identity mail, // global audit, declared webhooks) an organization or a sibling of their - // own. ⛔ Never re-introduce a per-table or per-posture cut here — with - // ONE cited, temporary exception: - // - // [BRIDGE, epic ruling record 6270035914 on #15194 item 2, #24582] Under a - // wall, `sys_secret` and `sys_flow_credential` keep their organization-less - // system inserts until the steps 5+6 landing flips them column-less. See - // `WALLED_ORGANIZATION_LESS_BRIDGE`; delete this with the flips. - if (isWalledOrganizationLessBridgeObject(object) && this.resolveEnginePosture() !== 'single') { - return undefined; - } + // own. ⛔ Never re-introduce a per-table or per-posture cut here: a table + // whose rows are deployment-level declares no organization column + // (`sys_secret` and `sys_flow_credential` since the steps 5+6 landing, + // #24582) and exits below, on the missing tenant field. const objectSchema = this._registry.getObject(object) as any; // A federated object's schema is the REMOTE's (ADR-0015); the platform's // injected column says nothing about it, which is the same reason diff --git a/packages/objectql/src/index.ts b/packages/objectql/src/index.ts index 13b37b68420..efde3c270c9 100644 --- a/packages/objectql/src/index.ts +++ b/packages/objectql/src/index.ts @@ -545,8 +545,6 @@ export { resolveSystemWriteOrganization, resolveTenantFieldName, isPlatformNamespaceObject, - WALLED_ORGANIZATION_LESS_BRIDGE, - isWalledOrganizationLessBridgeObject, carriesOrganization, isSystemWriteOrganizationRequiredError, SystemWriteOrganizationRequiredError, diff --git a/packages/objectql/src/system-write-organization.test.ts b/packages/objectql/src/system-write-organization.test.ts index 55fa6290a80..f186611c03b 100644 --- a/packages/objectql/src/system-write-organization.test.ts +++ b/packages/objectql/src/system-write-organization.test.ts @@ -317,7 +317,7 @@ describe('#8844 multi-organization — the write is REFUSED, never defaulted', ( expect(refusal.message).toContain('dispatch_order'); expect(refusal.message).toContain('__global__'); expect(refusal.message).toContain('tenantId'); - expect(refusal.message).toContain('tenancy: { enabled: false }'); + expect(refusal.message).toContain('systemFields: { tenant: false }'); // ⛔ NOT defaulted and ⛔ not silently skipped — nothing reached the driver. expect(observed.filter((c) => c.object === 'dispatch_order')).toEqual([]); diff --git a/packages/objectql/src/tenancy/system-write-organization.ts b/packages/objectql/src/tenancy/system-write-organization.ts index 4ecde671e93..8cce07e4574 100644 --- a/packages/objectql/src/tenancy/system-write-organization.ts +++ b/packages/objectql/src/tenancy/system-write-organization.ts @@ -84,9 +84,6 @@ * (`seed-tenancy-backfill.ts`) still carries the NAMESPACE form; its * residue attribution is C7's. Read the three together before changing any * of them. - * - **Under a walled posture only, `sys_secret` and `sys_flow_credential`** — - * a cited, temporary bridge ({@link WALLED_ORGANIZATION_LESS_BRIDGE}) until - * the steps 5+6 landing flips both tables column-less. * - **Writes that already carry an organization** — on the execution context * or on the row itself. That IS "carrying an explicit organization"; the * ruling asks for nothing more. @@ -171,34 +168,6 @@ export const DEFAULT_TENANT_FIELD = 'organization_id'; */ const PLATFORM_NAMESPACE = /^(sys_|cloud_|ai_)/; -/** - * [ADR-0131 C8 stage 8b, BRIDGE — delete it with the flips] The two platform - * tables whose organization-less SYSTEM inserts keep landing under a walled - * posture, as they did before the walled half of the lift. - * - * Their deployment-level rows (a global-scope encrypted setting's and a - * datasource credential's `sys_secret` ciphertext, a flow's - * `sys_flow_credential`) are organization-less by ruling until those tables - * drop the column: epic ruling record 6270035914 on #15194, item 2, moved the - * deferred `sys_secret` / `sys_flow_credential` `systemFields: { tenant: false }` - * flips into the steps 5+6 landing (#24582), because declaring them column-less - * any earlier refuses every 18.0 database at boot. Until then the walled lift - * would turn each of those writes into a 500 `ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED`. - * - * It is a walled-posture bridge only: under `single` these inserts take the - * Default Organization like any other. ⛔ Add no table here. When the flips - * land, the tables have no tenant column, the engine exits on that before this - * list is read, and the tripwire pin in `@objectstack/service-automation` - * (`walled-organization-less-bridge.test.ts`) turns red until this list, its - * reader in `Engine.resolveSystemInsertOrganization` and that pin are deleted. - */ -export const WALLED_ORGANIZATION_LESS_BRIDGE: readonly string[] = Object.freeze(['sys_flow_credential', 'sys_secret']); - -/** Is `object` one of {@link WALLED_ORGANIZATION_LESS_BRIDGE}'s two tables? */ -export function isWalledOrganizationLessBridgeObject(object: string): boolean { - return WALLED_ORGANIZATION_LESS_BRIDGE.includes(object); -} - /** * Is this object in one of the platform namespaces? * @@ -306,7 +275,7 @@ function buildRefusalMessage( `carrying the organization the way a session write does: pass it on the execution context ` + `({ context: { isSystem: true, tenantId: '' } }), or set ${DEFAULT_TENANT_FIELD} ` + `on the record itself. If rows of '${object}' are genuinely platform-global and belong to no ` + - `organization, declare that on the OBJECT — tenancy: { enabled: false } (ADR-0066) — so it is ` + + `organization, declare that on the OBJECT — systemFields: { tenant: false } (ADR-0131) — so it is ` + `stated once and checkable, rather than decided per write.` ); } diff --git a/packages/platform-objects/src/deployment-level-no-tenant-column.test.ts b/packages/platform-objects/src/deployment-level-no-tenant-column.test.ts index 8461ed39aeb..379b23a357e 100644 --- a/packages/platform-objects/src/deployment-level-no-tenant-column.test.ts +++ b/packages/platform-objects/src/deployment-level-no-tenant-column.test.ts @@ -4,7 +4,7 @@ import { describe, it, expect } from 'vitest'; import { resolveInjectedSystemColumns } from '@objectstack/spec/data'; import { PLATFORM_CAPABILITIES } from '@objectstack/spec/security'; import { SysJob, SysJobRun, SysJobQueue } from './audit/index.js'; -import { SysMigration, SysMigrationJournal, SysPlatformSettingAudit, SysSecret } from './system/index.js'; +import { SysMigration, SysMigrationJournal, SysPlatformSettingAudit, SysSecret, SysSettingAudit } from './system/index.js'; /** * [ADR-0131 D7] The deployment-level plumbing this package declares carries no @@ -16,9 +16,10 @@ import { SysMigration, SysMigrationJournal, SysPlatformSettingAudit, SysSecret } * on `fields` alone is a phantom check: an object that merely omits the field * still gets the column. `resolveInjectedSystemColumns` is the derivation * `applySystemFields` consumes to do the injecting, so it is the authority on - * whether the column exists. The `sys_secret` control below is the same - * question asked of an object that DOES keep the column (its writer stamps the - * caller's organization), so the predicate is shown able to answer `true`. + * whether the column exists. The `sys_setting_audit` control below is the + * same question asked of an object that DOES keep the column (its writer + * stamps the caller's organization), so the predicate is shown able to answer + * `true`. * * ## Why the capability is pinned beside the column * @@ -80,11 +81,20 @@ describe('[ADR-0131 D7] deployment-level plumbing: no tenant column, platform-on }); it('control: a tenant-attributed object keeps the column, so the predicate can answer true', () => { - // `sys_secret` is written with the business write's driver options, so the - // SQL driver stamps the caller's organization on it. It is NOT - // deployment-level today; its fate is decided by the C7 inventory. - const plan = resolveInjectedSystemColumns(SysSecret); + // `sys_setting_audit` keeps a tenant or user setting's history: its writer + // stamps the caller's organization on every row it writes there. + const plan = resolveInjectedSystemColumns(SysSettingAudit); expect(plan.tenant).toBe(true); expect([...plan.names]).toContain('organization_id'); }); + + it('sys_secret is deployment-level too: no tenant column (every read is by id); its reads stay private by access', () => { + // 18.x step 6 (line fact F2): the flip lands with the open upgrade + // ceremony, whose column-drops step takes an upgraded database's column. + const plan = resolveInjectedSystemColumns(SysSecret); + expect(plan.tenant).toBe(false); + expect([...plan.names]).not.toContain('organization_id'); + expect((SysSecret as { systemFields?: unknown }).systemFields).toEqual({ tenant: false }); + expect((SysSecret as { access?: unknown }).access).toEqual({ default: 'private' }); + }); }); diff --git a/packages/platform-objects/src/system/sys-secret.object.ts b/packages/platform-objects/src/system/sys-secret.object.ts index 6c099986490..a5e0418d53a 100644 --- a/packages/platform-objects/src/system/sys-secret.object.ts +++ b/packages/platform-objects/src/system/sys-secret.object.ts @@ -42,6 +42,11 @@ export const SysSecret = ObjectSchema.create({ icon: 'key', isSystem: true, managedBy: 'engine-owned', + // [ADR-0131 D7] Deployment-level: every read is by id under the system + // context, never by organization, so the store carries no organization + // column. The flip lands with the open upgrade ceremony (18.x step 6, line + // fact F2), whose column-drops step takes an upgraded database's column. + systemFields: { tenant: false }, // [ADR-0066 D2/④] Secure-by-default: the environment's encrypted-secrets // store (settings/datasource credentials). Not covered by the wildcard `'*'` // grant — ordinary members get 403 from the generic data layer. Platform diff --git a/packages/plugins/plugin-security/src/bootstrap-platform-admin.ts b/packages/plugins/plugin-security/src/bootstrap-platform-admin.ts index eac7d8f87ba..f9123b5ffce 100644 --- a/packages/plugins/plugin-security/src/bootstrap-platform-admin.ts +++ b/packages/plugins/plugin-security/src/bootstrap-platform-admin.ts @@ -880,8 +880,8 @@ export async function bootstrapPlatformAdmin( '[security] platform-admin promotion WITHHELD: an unscoped grant exists whose permission-set name ' + '(sys_user_permission_set.permission_set) was never written — a grant from a release before that ' + 'column existed. Which set it holds can no longer be read, and it may be the platform administrator\'s, ' - + 'so this pass will not mint a second one. 18.0 supports fresh installs only and does not upgrade a v17 ' - + 'database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path.'; + + 'so this pass will not mint a second one. The organization-ownership migration\'s id-to-name rewrite ' + + '(ADR-0131 D10) names it; the next pass after that reads it by name.'; if (logger?.warn) logger.warn(withheld); else logger?.info?.(withheld); return { diff --git a/packages/plugins/plugin-security/src/catalog-reference-report.test.ts b/packages/plugins/plugin-security/src/catalog-reference-report.test.ts index 20401e850bc..f5ddf50b64f 100644 --- a/packages/plugins/plugin-security/src/catalog-reference-report.test.ts +++ b/packages/plugins/plugin-security/src/catalog-reference-report.test.ts @@ -15,8 +15,8 @@ * * Every assertion reads the structured result, never the log's wording; the * wiring pin finds the plugin's lines by their structured meta. One pin reads - * wording on purpose: the unnamed-grant line is 18.0 release text an operator - * acts on, and that release says one sentence about a v17 database everywhere. + * wording on purpose: the unnamed-grant line is release text an operator acts + * on, and its fix is the ownership ceremony's reviewed plan and apply. */ import { describe, it, expect, afterEach, vi } from 'vitest'; @@ -322,19 +322,16 @@ describe('[ADR-0131 D4] the catalog reference report — references whose names }); }); -describe('[ADR-0131 D4] the unnamed-grant line says what 18.0 supports for a v17 database', () => { - // The one wording pin in this file. 18.0 supports fresh installs only and its - // `os migrate organization-ownership` refuses plan and apply on every - // database, so the line an operator reads at boot must not send them to it; - // it carries the sentence the drop-gate refusal of `os migrate apply - // --allow-destructive` carries (the boot refusal and the command's own - // refusal say the same in their own terms). Everything else in the line is - // the report's own: the count, the field, the D4 reason, and that nothing was - // changed. - const V17_SENTENCE = '18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a ' - + '17.x release until the 18.x release that carries the upgrade path.'; - - it('names the count, the empty field, the reason and "Nothing was changed", then the 18.0 sentence, and prescribes no command', async () => { +describe('[ADR-0131 D4] the unnamed-grant line prescribes the ownership ceremony', () => { + // The one wording pin in this file. The line an operator reads at boot sends + // them to the ceremony (open since #24581): its plan, then the apply of that + // reviewed plan, which names each legacy grant. It never says to keep the + // database on 17.x. Everything else in the line is the report's own: the + // count, the field, the D4 reason, and that nothing was changed. + const FIX = 'Fix: run the v18 ceremony. Plan it first, os migrate organization-ownership --out plan.json, and review ' + + 'the file it writes; then run os migrate organization-ownership --apply --backup-taken --plan-file plan.json'; + + it('names the count, the empty field, the reason and "Nothing was changed", then the ceremony\'s plan and apply', async () => { const b = await boot(); await insertUser(b, 'usr_a'); await insertUser(b, 'usr_b'); @@ -352,11 +349,9 @@ describe('[ADR-0131 D4] the unnamed-grant line says what 18.0 supports for a v17 expect(message).toContain( '[security] 2 sys_user_permission_set grant(s) name no permission set (permission_set is empty), ' - + 'so each confers nothing (ADR-0131 D4). Nothing was changed. ' + V17_SENTENCE, + + 'so each confers nothing (ADR-0131 D4). Nothing was changed. ' + FIX, ); - expect(message.endsWith(V17_SENTENCE), 'the sentence closes the line').toBe(true); - // The refused command and every step of it are gone, not merely qualified. - expect(message).not.toMatch(/organization-ownership|os migrate|--out|--apply|--plan-file|--backup-taken|plan\.json|ceremony/); + expect(message).not.toMatch(/fresh installs only|on a 17\.x release|carries the upgrade path/); }); }); diff --git a/packages/plugins/plugin-security/src/catalog-reference-report.ts b/packages/plugins/plugin-security/src/catalog-reference-report.ts index 14fa2de8049..8683807b938 100644 --- a/packages/plugins/plugin-security/src/catalog-reference-report.ts +++ b/packages/plugins/plugin-security/src/catalog-reference-report.ts @@ -433,9 +433,11 @@ function logReport(report: CatalogReferenceReport, logger?: CatalogReferenceRepo logError( logger, `[security] ${unnamed.count} ${GRANT_OBJECT} grant(s) name no permission set (${GRANT_SET_NAME_FIELD} is ` + - 'empty), so each confers nothing (ADR-0131 D4). Nothing was changed. 18.0 supports fresh installs only and ' + - 'does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries ' + - 'the upgrade path.', + 'empty), so each confers nothing (ADR-0131 D4). Nothing was changed. Fix: run the v18 ceremony. Plan it ' + + 'first, os migrate organization-ownership --out plan.json, and review the file it writes; then run ' + + 'os migrate organization-ownership --apply --backup-taken --plan-file plan.json, which executes that plan: ' + + 'it names each legacy grant from its retired id and lists the ones it cannot name. Then restart. Granting ' + + 'the set again by name before it runs makes a second grant the ceremony will report, not name.', { count: unnamed.count, organizations: unnamed.organizations }, ); } diff --git a/packages/plugins/plugin-security/src/grant-readers-unnamed-grant.test.ts b/packages/plugins/plugin-security/src/grant-readers-unnamed-grant.test.ts index 51892bbd5ba..13633aaf183 100644 --- a/packages/plugins/plugin-security/src/grant-readers-unnamed-grant.test.ts +++ b/packages/plugins/plugin-security/src/grant-readers-unnamed-grant.test.ts @@ -222,17 +222,15 @@ describe('[ADR-0131 D4] a grant that names nothing — the platform-admin bootst expect(await findExistingPlatformAdmin(engine, defaultPermissionSets)).toBeUndefined(); }); - // The one wording pin on the withheld-promotion warning. 18.0 supports fresh - // installs only and its `os migrate organization-ownership` refuses plan and - // apply on every database, so the warning an operator reads at boot must not - // send them to it; it ends with the sentence the unnamed-grant line of the - // catalog reference report and the boot refusal carry, in the same terms. + // The one wording pin on the withheld-promotion warning. It ends with the + // ownership migration's id-to-name rewrite, which names the grant (the + // ceremony is open, #24581); it never says to keep the database on 17.x. // Everything before the sentence is the warning's own: what is withheld, why, // and that this pass mints no second grant. - const V17_SENTENCE = '18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a ' - + '17.x release until the 18.x release that carries the upgrade path.'; + const REWRITE_SENTENCE = 'The organization-ownership migration\'s id-to-name rewrite (ADR-0131 D10) names it; the next ' + + 'pass after that reads it by name.'; - it('unnamed, the withheld warning keeps what is withheld and why, ends with the 18.0 sentence, and prescribes no migration', async () => { + it('unnamed, the withheld warning keeps what is withheld and why, and ends with the rewrite that names the grant', async () => { const engine = await standingAdminUnnamed(); await unname(engine, { id: 'g_admin' }); // An existing database: the platform-bucket rows an earlier release seeded. @@ -252,11 +250,10 @@ describe('[ADR-0131 D4] a grant that names nothing — the platform-admin bootst '[security] platform-admin promotion WITHHELD: an unscoped grant exists whose permission-set name ' + '(sys_user_permission_set.permission_set) was never written — a grant from a release before that ' + 'column existed. Which set it holds can no longer be read, and it may be the platform administrator\'s, ' - + 'so this pass will not mint a second one. ' + V17_SENTENCE, + + 'so this pass will not mint a second one. ' + REWRITE_SENTENCE, ); - expect(message.endsWith(V17_SENTENCE), 'the sentence closes the warning').toBe(true); - // The refused command and the rewrite it performed are gone, not merely qualified. - expect(message).not.toMatch(/organization-ownership|os migrate|--out|--apply|--plan-file|ceremony|id-to-name/); + expect(message.endsWith(REWRITE_SENTENCE), 'the sentence closes the warning').toBe(true); + expect(message).not.toMatch(/fresh installs only|on a 17\.x release|carries the upgrade path/); }); }); diff --git a/packages/plugins/plugin-webhooks/src/declared-webhooks.ts b/packages/plugins/plugin-webhooks/src/declared-webhooks.ts index 2ee34596942..f878e821d4e 100644 --- a/packages/plugins/plugin-webhooks/src/declared-webhooks.ts +++ b/packages/plugins/plugin-webhooks/src/declared-webhooks.ts @@ -65,7 +65,7 @@ * `sys_webhook` row holding a declared name is judged by what the 18.0 seeder * would have done with it on this boot ({@link classifyDeclaredNameHolder}): * - * - **A pristine mirror** (`managed_by: 'package'`, not `customized`): the + * - **A pristine mirror** (`managed_by` `package` or `platform`, not `customized`): the * seeder would have overwritten it from the declaration. The row is ignored * and the declaration answers. * - **Any other row** (`customized: true`, or `managed_by: 'admin'`): the @@ -262,7 +262,18 @@ export async function readWebhookActivation(engine: unknown, logger?: Logger): P * declaration is withheld. */ export function classifyDeclaredNameHolder(row: { managed_by?: unknown; customized?: unknown }): 'mirror' | 'row' { - return row.managed_by === 'package' && row.customized !== true ? 'mirror' : 'row'; + return isSeededProvenance(row.managed_by) && row.customized !== true ? 'mirror' : 'row'; +} + +/** + * The provenance values a seeded copy carries: `package` and `platform`, the + * same pair the provenance hook stamps `customized` on + * (`webhook-provenance.ts`) and the upgrade ceremony settles. One predicate, + * so the mirror skip, the redeliver guard and the ceremony never disagree on + * which rows are copies of a declaration. + */ +export function isSeededProvenance(managedBy: unknown): boolean { + return managedBy === 'package' || managedBy === 'platform'; } /** diff --git a/packages/plugins/plugin-webhooks/src/index.ts b/packages/plugins/plugin-webhooks/src/index.ts index 9f89b7beb22..2f82357f3ea 100644 --- a/packages/plugins/plugin-webhooks/src/index.ts +++ b/packages/plugins/plugin-webhooks/src/index.ts @@ -27,6 +27,9 @@ export { export { AutoEnqueuer, type AutoEnqueuerOptions, type HttpEnqueueFn } from './auto-enqueuer.js'; export { SysWebhook } from './sys-webhook.object.js'; +// [ADR-0131 D2] How a delivery names a declared webhook (`declared:NAME`); the upgrade ceremony +// re-sources an 18.0 copy's deliveries to it and pins its own spelling equal to this one. +export { DECLARED_WEBHOOK_REF_PREFIX, declaredWebhookRefId } from './declared-webhooks.js'; /** * [#7799] The signing-secret seam. Exported so a host that boots the pieces diff --git a/packages/plugins/plugin-webhooks/src/webhook-provenance.ts b/packages/plugins/plugin-webhooks/src/webhook-provenance.ts index 5e542a58497..61166dc41c7 100644 --- a/packages/plugins/plugin-webhooks/src/webhook-provenance.ts +++ b/packages/plugins/plugin-webhooks/src/webhook-provenance.ts @@ -49,6 +49,8 @@ * stamp exists to remember. */ +import { isSeededProvenance } from './declared-webhooks.js'; + interface MinimalEngine { registerHook(event: string, handler: (ctx: any) => any, options?: Record): void; unregisterHooksByPackage(packageId: string): number; @@ -80,7 +82,7 @@ export function bindWebhookProvenanceStamp(engine: MinimalEngine, logger?: Minim // had just read. const previous = ctx?.previous as Record | undefined; if (!previous || typeof previous !== 'object') return; - if ((previous.managed_by === 'package' || previous.managed_by === 'platform') && previous.customized !== true) { + if (isSeededProvenance(previous.managed_by) && previous.customized !== true) { (data as any).customized = true; } }, diff --git a/packages/qa/dogfood/test/identity-mail-read-gate.dogfood.test.ts b/packages/qa/dogfood/test/identity-mail-read-gate.dogfood.test.ts index 2eb2097e9c4..6d4e5819ab3 100644 --- a/packages/qa/dogfood/test/identity-mail-read-gate.dogfood.test.ts +++ b/packages/qa/dogfood/test/identity-mail-read-gate.dogfood.test.ts @@ -1,7 +1,7 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. // // [#24625] Identity-mail bodies are read only by the platform capability, on -// every door, over an 18.0 database. +// every door, over an 18.0 database the ownership ceremony has upgraded. // // ## What was measured // @@ -21,6 +21,18 @@ // gate is on the BODY columns of both mail logs: field `requiredPermissions: // ['manage_platform_settings']`. The mirror writers mask the same columns. // +// ## Over a ceremonied fixture (#24581) +// +// The boot guard serves an 18.0 database only once the ownership ceremony has +// certified it, so the matrix boots a copy the ceremony ran on +// (`ceremoniedFixture`). Its `sibling-moves` step moved the 18.0 identity row +// — the password reset — into `sys_identity_email`, id for id, and re-pointed +// its `sys_activity` rows there; the identity probes follow it. The +// organization invitation is tenant mail and stays in `sys_email`, and the +// pre-fix `sys_audit_log` copies of the reset still name `sys_email`: both keep +// their `sys_email` probes. Every door closed before the ceremony stays +// closed. +// // ## The matrix, principal × door // // - anonymous: refused on the data API; @@ -44,8 +56,8 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { afterAll, beforeAll, describe, expect, it } from 'vitest'; import type { VerifyStack } from '@objectstack/verify'; +import { ceremoniedFixture } from './upgrade-harness/ceremony.js'; import { FIXTURE_PEOPLE, bootFixture } from './upgrade-harness/populate-fixture.js'; -import { loadSqlIntoSqlite } from './upgrade-harness/sqlite-text-dump.js'; const FIXTURE = new URL('./upgrade-harness/fixture-18.0/', import.meta.url); const fixtureSql = readFileSync(new URL('database.sql', FIXTURE), 'utf8'); @@ -76,17 +88,21 @@ function credentialSpanOf(row: Row): string { return longest; } -describe('[#24625] identity-mail bodies over an 18.0 database: principal × door', () => { +describe('[#24625] identity-mail bodies over a ceremonied 18.0 database: principal × door', () => { const dir = mkdtempSync(join(tmpdir(), 'dogfood-identity-mail-gate-')); - const file = join(dir, 'upgraded.db'); let stack: VerifyStack; // eslint-disable-next-line @typescript-eslint/no-explicit-any let ql: any; const tok: Record<'platform' | 'orgAdmin' | 'member', string> = { platform: '', orgAdmin: '', member: '' }; - /** The fixture's two `sys_email` rows: the 18.0 password reset and the organization invitation. */ + /** + * The fixture's two 18.0 mail rows: the password reset, which the ceremony + * moved to `sys_identity_email`, and the organization invitation, still in + * `sys_email`. + */ let reset: Row; let invitation: Row; let resetSpan = ''; + let invitationSpan = ''; const call = async (who: keyof typeof tok | 'anonymous', method: string, path: string, body?: unknown) => { const res = who === 'anonymous' @@ -99,118 +115,152 @@ describe('[#24625] identity-mail bodies over an 18.0 database: principal × door }; beforeAll(async () => { - loadSqlIntoSqlite(fixtureSql, file); + const { file } = await ceremoniedFixture(dir, fixtureSql, 'identity-mail-gate'); stack = await bootFixture(file); ql = await stack.kernel.getServiceAsync('objectql'); const mails: Row[] = await ql.find('sys_email', { where: {}, limit: 50, ...SYS }); - reset = mails.find((m) => m.related_object === 'sys_user')!; + const identity: Row[] = await ql.find('sys_identity_email', { where: {}, limit: 50, ...SYS }); + reset = identity.find((m) => m.related_object === 'sys_user')!; invitation = mails.find((m) => m.related_object === 'sys_invitation')!; - expect(reset, 'the fixture holds the 18.0 password-reset row').toBeTruthy(); - expect(invitation, 'the fixture holds the organization-invitation row').toBeTruthy(); + expect(reset, 'the ceremony moved the 18.0 password-reset row to the identity log').toBeTruthy(); + expect(mails.some((m) => m.id === reset.id), 'the moved row left sys_email').toBe(false); + expect(invitation, 'the organization-invitation row stays in sys_email').toBeTruthy(); resetSpan = credentialSpanOf(reset); + invitationSpan = credentialSpanOf(invitation); // The organization administrator: fixture member A, promoted to the // organization's `admin` role. The membership hook grants the scoped // organization-admin set; nothing grants the platform capability. const [userA] = await ql.find('sys_user', { where: { email: FIXTURE_PEOPLE.memberA.email }, limit: 1, ...SYS }); const [membership] = await ql.find('sys_member', { where: { user_id: userA.id }, limit: 1, ...SYS }); - expect(membership?.organization_id, 'member A belongs to the fixture organization').toBe(reset.organization_id); + expect(membership?.organization_id, 'member A belongs to the fixture organization').toBe(invitation.organization_id); await ql.update('sys_member', { id: membership.id, role: 'admin' }, SYS); tok.platform = await stack.signIn(FIXTURE_PEOPLE.admin.email, FIXTURE_PEOPLE.admin.password); tok.orgAdmin = await stack.signIn(FIXTURE_PEOPLE.memberA.email, FIXTURE_PEOPLE.memberA.password); tok.member = await stack.signIn(FIXTURE_PEOPLE.memberB.email, FIXTURE_PEOPLE.memberB.password); for (const who of ['platform', 'orgAdmin', 'member'] as const) { - await call(who, 'POST', '/auth/organization/set-active', { organizationId: reset.organization_id }); + await call(who, 'POST', '/auth/organization/set-active', { organizationId: invitation.organization_id }); } - }, 240_000); + }, 600_000); afterAll(async () => { await stack?.stop(); rmSync(dir, { recursive: true, force: true }); }); - describe('the data API: list, get and query of sys_email', () => { - it('anonymous is refused', async () => { - expect((await call('anonymous', 'GET', '/data/sys_email')).status).toBe(401); + describe('the data API: list, get and query of the two mail logs', () => { + it.each(['sys_email', 'sys_identity_email'] as const)('anonymous is refused %s', async (object) => { + expect((await call('anonymous', 'GET', `/data/${object}`)).status).toBe(401); }); - it('a member is refused', async () => { - const res = await call('member', 'GET', '/data/sys_email'); + it.each(['sys_email', 'sys_identity_email'] as const)('a member is refused %s', async (object) => { + const res = await call('member', 'GET', `/data/${object}`); expect(res.status).toBe(403); expect(res.text.includes(resetSpan)).toBe(false); + expect(res.text.includes(invitationSpan)).toBe(false); }); - it('the organization admin lists both rows — the identity row and its own invitation — without a body', async () => { + it('the organization admin lists its own invitation without a body, and the moved identity row is not in sys_email', async () => { const res = await call('orgAdmin', 'GET', '/data/sys_email'); expect(res.status).toBe(200); const rows = rowsOf(res.json); - const byId = new Map(rows.map((r) => [r.id, r])); - // The rows are served, with their envelope and state: the gate is on the body only. - for (const mail of [reset, invitation]) { - const row = byId.get(mail.id); - expect(row, 'the row is listed').toBeTruthy(); - expect(row!.subject).toBe(mail.subject); - expect(row!.status).toBe(mail.status); - expect(hasBodyKey(row!)).toBe(false); - } + // The row is served, with its envelope and state: the gate is on the body only. + const row = rows.find((r) => r.id === invitation.id); + expect(row, 'the row is listed').toBeTruthy(); + expect(row!.subject).toBe(invitation.subject); + expect(row!.status).toBe(invitation.status); + expect(hasBodyKey(row!)).toBe(false); + expect(rows.some((r) => r.id === reset.id)).toBe(false); expect(res.text.includes(resetSpan)).toBe(false); + expect(res.text.includes(invitationSpan)).toBe(false); }); - it('the organization admin gets the identity row by id without a body, even naming the body in select', async () => { - for (const path of [`/data/sys_email/${reset.id}`, `/data/sys_email/${reset.id}?select=id,subject,body_text,body_html`]) { + it('the organization admin gets the invitation by id without a body, even naming the body in select', async () => { + for (const path of [`/data/sys_email/${invitation.id}`, `/data/sys_email/${invitation.id}?select=id,subject,body_text,body_html`]) { const res = await call('orgAdmin', 'GET', path); - expect(res.text.includes(resetSpan)).toBe(false); + expect(res.text.includes(invitationSpan)).toBe(false); if (res.status === 200) expect(hasBodyKey(recordOf(res.json))).toBe(false); else expect(res.status).toBe(403); } }); - it('the organization admin queries the identity row without a body, and a filter on the body is refused', async () => { - const byId = await call('orgAdmin', 'POST', '/data/sys_email/query', { where: { id: reset.id } }); + it('the organization admin is refused the moved identity row by id, on either log, naming the body or not', async () => { + for (const path of [ + `/data/sys_identity_email/${reset.id}`, + `/data/sys_identity_email/${reset.id}?select=id,subject,body_text,body_html`, + `/data/sys_email/${reset.id}`, + ]) { + const res = await call('orgAdmin', 'GET', path); + expect(res.status, path.split('?')[0]).not.toBe(200); + expect(res.text.includes(resetSpan)).toBe(false); + } + }); + + it('the organization admin queries the invitation without a body, and a filter on a body is refused', async () => { + const byId = await call('orgAdmin', 'POST', '/data/sys_email/query', { where: { id: invitation.id } }); expect(byId.status).toBe(200); - expect(rowsOf(byId.json).map((r) => r.id)).toEqual([reset.id]); + expect(rowsOf(byId.json).map((r) => r.id)).toEqual([invitation.id]); expect(hasBodyKey(rowsOf(byId.json)[0])).toBe(false); // A body predicate would disclose the body by row presence: refused, not answered. - const probe = await call('orgAdmin', 'POST', '/data/sys_email/query', { where: { body_text: { $contains: resetSpan } } }); - expect(probe.status).toBe(403); + for (const span of [invitationSpan, resetSpan]) { + const probe = await call('orgAdmin', 'POST', '/data/sys_email/query', { where: { body_text: { $contains: span } } }); + expect(probe.status).toBe(403); + } + const identity = await call('orgAdmin', 'POST', '/data/sys_identity_email/query', { where: { body_text: { $contains: resetSpan } } }); + expect(identity.status).toBe(403); }); - it('the platform admin is served the bodies (control)', async () => { - const list = await call('platform', 'GET', '/data/sys_email'); - expect(list.status).toBe(200); - const row = rowsOf(list.json).find((r) => r.id === reset.id); - expect(row && BODY_FIELDS.every((f) => typeof row[f] === 'string' && row[f].length > 0)).toBe(true); - expect(list.text.includes(resetSpan)).toBe(true); - const probe = await call('platform', 'POST', '/data/sys_email/query', { where: { body_text: { $contains: resetSpan } } }); - expect(probe.status).toBe(200); - expect(rowsOf(probe.json).map((r) => r.id)).toEqual([reset.id]); + it('the platform admin is served the bodies on both logs (control)', async () => { + for (const [object, mail, span] of [ + ['sys_identity_email', reset, resetSpan], + ['sys_email', invitation, invitationSpan], + ] as const) { + const list = await call('platform', 'GET', `/data/${object}`); + expect(list.status).toBe(200); + const row = rowsOf(list.json).find((r) => r.id === mail.id); + expect(row && BODY_FIELDS.every((f) => typeof row[f] === 'string' && row[f].length > 0), object).toBe(true); + expect(list.text.includes(span)).toBe(true); + const probe = await call('platform', 'POST', `/data/${object}/query`, { where: { body_text: { $contains: span } } }); + expect(probe.status).toBe(200); + expect(rowsOf(probe.json).map((r) => r.id)).toEqual([mail.id]); + } }); }); describe('global search', () => { - const sysEmailHits = (json: any): string[] => - (Array.isArray(json?.hits) ? json.hits : []).filter((h: any) => h?.object === 'sys_email').map((h: any) => String(h.id)); + const mailHits = (json: any): string[] => + (Array.isArray(json?.hits) ? json.hits : []) + .filter((h: any) => h?.object === 'sys_email' || h?.object === 'sys_identity_email') + .map((h: any) => String(h.id)); + const terms = () => [[resetSpan, reset.id], [invitationSpan, invitation.id]] as const; it.each(['member', 'orgAdmin'] as const)('a body term finds nothing for the %s', async (who) => { - const res = await call(who, 'GET', `/search?q=${encodeURIComponent(resetSpan)}`); - expect(res.status).toBe(200); - expect(sysEmailHits(res.json)).not.toContain(reset.id); - // The hits only: the body echoes the query term itself. - expect(JSON.stringify((res.json as any)?.hits ?? []).includes(resetSpan)).toBe(false); + for (const [span, id] of terms()) { + const res = await call(who, 'GET', `/search?q=${encodeURIComponent(span)}`); + expect(res.status).toBe(200); + expect(mailHits(res.json)).not.toContain(id); + // The hits only: the body echoes the query term itself. + expect(JSON.stringify((res.json as any)?.hits ?? []).includes(span)).toBe(false); + } }); - it('the same term finds the identity row for the platform admin (control)', async () => { - const res = await call('platform', 'GET', `/search?q=${encodeURIComponent(resetSpan)}`); - expect(res.status).toBe(200); - expect(sysEmailHits(res.json)).toContain(reset.id); + it('the same terms find both rows for the platform admin (control)', async () => { + for (const [span, id] of terms()) { + const res = await call('platform', 'GET', `/search?q=${encodeURIComponent(span)}`); + expect(res.status).toBe(200); + expect(mailHits(res.json)).toContain(id); + } }); }); describe('the mirrors: sys_activity and sys_audit_log of the 18.0 identity row', () => { + // The ceremony re-pointed the activity rows at the identity log; the + // pre-fix audit copies keep naming the log the row was written to. + const parentOf = { sys_activity: 'sys_identity_email', sys_audit_log: 'sys_email' } as const; const mirrorPath = (object: 'sys_activity' | 'sys_audit_log') => - `/data/${object}?$filter=${filterOf({ object_name: 'sys_email', record_id: reset.id })}`; + `/data/${object}?$filter=${filterOf({ object_name: parentOf[object], record_id: reset.id })}`; it.each(['sys_activity', 'sys_audit_log'] as const)('%s: the organization admin is served the rows without the body', async (object) => { const res = await call('orgAdmin', 'GET', mirrorPath(object)); @@ -222,7 +272,7 @@ describe('[#24625] identity-mail bodies over an 18.0 database: principal × door ['sys_activity', 'metadata'], ['sys_audit_log', 'new_value'], ] as const)('%s: a body probe on the copy is refused to the organization admin, named parent or not', async (object, column) => { - for (const where of [{ [column]: { $contains: resetSpan } }, { object_name: 'sys_email', [column]: { $contains: resetSpan } }]) { + for (const where of [{ [column]: { $contains: resetSpan } }, { object_name: parentOf[object], [column]: { $contains: resetSpan } }]) { expect((await call('orgAdmin', 'POST', `/data/${object}/query`, { where })).status).toBe(403); } }); diff --git a/packages/qa/dogfood/test/organization-ownership-boot-refusal-order.dogfood.test.ts b/packages/qa/dogfood/test/organization-ownership-boot-refusal-order.dogfood.test.ts index 3f6cd8c38bb..f60f6c2e2be 100644 --- a/packages/qa/dogfood/test/organization-ownership-boot-refusal-order.dogfood.test.ts +++ b/packages/qa/dogfood/test/organization-ownership-boot-refusal-order.dogfood.test.ts @@ -16,36 +16,27 @@ // that declare no organization column) still carrying a physical // `organization_id`: // -// 1. the v18 boot is refused, saying what 18.0 supports for a v17 database and -// naming no step of the command 18.0 refuses — and the refused boot has +// 1. the v18 boot is refused, naming the command — and the refused boot has // ALREADY added the column the ceremony waits for: the guard judges in its // own `start()`, after the engine's `start()`, which is where the schema // sync runs (and before every plugin that writes rows at start, so the // refusal arrives as the `cause` of the kernel's start-failure error); // 2. `os migrate organization-ownership` writes its plan, and // `--apply --plan-file PLAN` completes with no boot in between, and -// verifies its marker; +// verifies its marker — through the real CLI binary, with the arguments, +// working directory and environment an operator's run has; // 3. the next v18 boot succeeds. // -// 18.0 ships for fresh installs only (ruling record 6259608309 on the v18 -// epic): the registered `os migrate organization-ownership` refuses, and -// step 2a pins that through the real CLI binary, reading and writing nothing. -// Step 2 runs the ceremony class the command keeps for 18.x -// (`OrganizationOwnershipCeremony`), from the CLI's built module in its own -// process, with the same arguments, working directory and environment an -// operator's run has. The 18.x release that re-registers the ceremony puts -// step 2 back on the binary and drops 2a. -// // ⚠ Needs the `sys_metadata` family's no-column schema beside the refusal: // against a release whose family still declares the column, step 2 stops at // `column-drops` (the declaration keeps it). The two land together. import { spawnSync } from 'node:child_process'; -import { copyFileSync, existsSync, mkdtempSync, rmSync } from 'node:fs'; +import { copyFileSync, mkdtempSync, rmSync } from 'node:fs'; import { createRequire } from 'node:module'; import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; -import { fileURLToPath, pathToFileURL } from 'node:url'; +import { fileURLToPath } from 'node:url'; import { DatabaseSync } from 'node:sqlite'; import { afterAll, describe, expect, it } from 'vitest'; import { SecurityPlugin, securityDefaultPermissionSets } from '@objectstack/plugin-security'; @@ -58,7 +49,6 @@ import { bootShowcase } from './showcase-boot.js'; const SHOWCASE_DIR = fileURLToPath(new URL('../../../../examples/app-showcase/', import.meta.url)); const CLI_ROOT = dirname(createRequire(import.meta.url).resolve('@objectstack/cli/package.json')); const CLI_BIN = join(CLI_ROOT, 'bin', 'run.js'); -const CEREMONY_MODULE = pathToFileURL(join(CLI_ROOT, 'dist', 'commands', 'migrate', 'organization-ownership.js')).href; const columnsOf = (file: string, table: string): string[] => { const db = new DatabaseSync(file); @@ -110,7 +100,7 @@ describe('[ADR-0131 D10 item 5] a 17.x database upgrades through the boot refusa rmSync(dir, { recursive: true, force: true }); }); - it('1. a v17-shape database is refused, saying what 18.0 supports and naming no step of the refused command', async () => { + it('1. a v17-shape database is refused, naming the command', async () => { const seed = new DatabaseSync(file); seed.exec(` CREATE TABLE sys_user_permission_set (id TEXT PRIMARY KEY, user_id TEXT, permission_set_id TEXT, organization_id TEXT); @@ -122,8 +112,7 @@ describe('[ADR-0131 D10 item 5] a 17.x database upgrades through the boot refusa refusal = refusalIn(await boot(file)); expect(refusal, 'the boot was refused for the v17 shape').toBeDefined(); - expect(String((refusal as Error).message)).toContain('18.0 supports fresh installs only and does not upgrade a v17 database'); - expect(String((refusal as Error).message)).not.toMatch(/os migrate organization-ownership|--plan-file|--out PLAN|--backup-taken/); + expect(String((refusal as Error).message)).toContain('os migrate organization-ownership'); expect((refusal as { tables?: string[] }).tables).toEqual(expect.arrayContaining(['sys_metadata', 'sys_migration_journal'])); }, 240_000); @@ -132,32 +121,11 @@ describe('[ADR-0131 D10 item 5] a 17.x database upgrades through the boot refusa expect(columnsOf(file, 'sys_user_permission_set')).toContain('permission_set'); }); - it('⭐ 2a. in 18.0 the real command refuses with NOT_IMPLEMENTED, and the database is untouched', () => { - const before = columnsOf(file, 'sys_metadata'); - const refused = spawnSync(process.execPath, [ - CLI_BIN, 'migrate', 'organization-ownership', '--database-url', `file:${file}`, '--out', join(dir, 'refused-plan.json'), '--json', - ], { cwd: SHOWCASE_DIR, encoding: 'utf8', timeout: 300_000, env: { ...process.env, NO_COLOR: '1' } }); - expect(refused.status, `${refused.stdout}\n${refused.stderr}`).toBe(1); - expect(JSON.parse(refused.stdout.trim())).toMatchObject({ error: 'plan_refused', code: 'NOT_IMPLEMENTED' }); - expect(existsSync(join(dir, 'refused-plan.json'))).toBe(false); - expect(columnsOf(file, 'sys_metadata')).toEqual(before); - }, 320_000); - it('⭐ 2. the ceremony completes with no boot in between, and verifies its marker', () => { // As an operator's run has it (S4, #23437): the plan first, then the apply - // of that reviewed plan file — through the ceremony class (see the header). + // of that reviewed plan file — through the real binary. const ceremony = (args: string[]) => spawnSync(process.execPath, [ - '--input-type=module', '-e', - // The built module, as `bin/run.js` runs it: oclif's auto-transpile off, - // so an inherited NODE_ENV=test cannot send it to `src/` under tsx. - 'const [root, url, ...argv] = process.argv.slice(1);' + - " const { createRequire } = await import('node:module');" + - " const { pathToFileURL } = await import('node:url');" + - " const oclif = await import(pathToFileURL(createRequire(root + '/package.json').resolve('@oclif/core')).href);" + - ' (oclif.settings ?? oclif.default.settings).enableAutoTranspile = false;' + - ' const { OrganizationOwnershipCeremony } = await import(url);' + - ' await OrganizationOwnershipCeremony.run(argv, { root });', - CLI_ROOT, CEREMONY_MODULE, '--database-url', `file:${file}`, ...args, + CLI_BIN, 'migrate', 'organization-ownership', '--database-url', `file:${file}`, ...args, ], { cwd: SHOWCASE_DIR, encoding: 'utf8', timeout: 300_000, env: { ...process.env, NO_COLOR: '1' } }); const planFile = join(dir, 'plan.json'); const plan = ceremony(['--out', planFile]); diff --git a/packages/qa/dogfood/test/upgrade-18x-harness.dogfood.test.ts b/packages/qa/dogfood/test/upgrade-18x-harness.dogfood.test.ts index 313b9fe89b2..952026f4e68 100644 --- a/packages/qa/dogfood/test/upgrade-18x-harness.dogfood.test.ts +++ b/packages/qa/dogfood/test/upgrade-18x-harness.dogfood.test.ts @@ -1,9 +1,12 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. // -// [#24522] The 18.x line's upgrade harness: a database an 18.0 install created -// upgrades to THIS commit, boots, and keeps its data (R2), and a v17-shaped one -// is still refused (R3). Ruling record 6266750393 on the v18 epic makes R2 a -// check every PR on the line passes before it merges. +// [#24522] The 18.x line's upgrade harness, RESTATED by 18.x step 6 (ruling +// records 6266750393, 6269955919 and 6270035914 on the v18 epic): a database +// an 18.0 install created takes the upgrade ceremony and then boots THIS +// commit with its data (R2), and so does a v17-shaped one (R3). The ceremony +// is open now, and the boot guard recognises a database by its birth marker: +// 18.0 wrote none, so an 18.0 database is refused until the ceremony has +// certified it at this release's ownership shape generation. // // The fixture (`upgrade-harness/fixture-18.0/`) is a showcase install of the // 18.0 code after real use — sign-ups, an invitation, settings with a secret, a @@ -15,34 +18,47 @@ // // What is asserted, on a fresh copy of the fixture: // -// 1. HEAD boots over it, composed the way `os serve` composes it — the -// ownership boot guard included — and nothing refuses it. +// 1. R2, restated: HEAD refuses it at boot, composed the way `os serve` +// composes it, naming the ceremony — and the refused boot writes no +// marker. +// 1b. `os migrate organization-ownership` — the real binary, in the +// showcase's directory — plans it and applies that plan, and the marker +// verifies at this release's shape generation. +// 1c. HEAD then boots over it, and nothing refuses it. // 2. It serves the fixture's data: the fixture's people sign in with their // 18.0 passwords and read the fixture's own task. -// 3. Once HEAD has stopped, every reshaped table still holds every row the -// fixture gave it, with the same key columns (`organization_id` among -// them): a step that moved, deleted or re-stamped 18.0 rows outside the -// upgrade ceremony goes red here, naming the table and the row. -// 4. The same fixture with one v17 trait added — a physical `organization_id` -// on `sys_metadata`, an object that declares none — is refused at boot -// with the guard's code. Until step 6 opens the ceremony that is the -// answer a v17 database gets; step 6 rewrites this case, by name. +// 2b. The fixture's declared webhook — customized in 18.0 only by switching +// it on — now answers from its declaration, switched on by the activation +// row the ceremony wrote: one delivery per task change, naming the +// declaration. +// 3. Once HEAD has stopped, every 18.0 row of every reshaped table is where +// the ceremony put it, with its key columns: kept in place, moved to its +// sibling (identity mail, the global audit rows), re-sourced (the +// deliveries), or deleted as a copy of a declaration (the webhook) — and +// `organization_id` dropped where the table became deployment-level, NOT +// NULL where it stayed. +// 4. R3, restated: the same fixture with one v17 trait added — a physical +// `organization_id` on `sys_metadata`, an object that declares none — is +// refused at boot with the guard's code, takes the ceremony, and boots. // // ⛔ Additive change passes: a row HEAD's boot ADDS (an activity shard, a // seeder's row) is not a failure, so counts are compared as "at least", and // every fixture row is looked up by id. -import { mkdtempSync, readFileSync, rmSync } from 'node:fs'; +import { copyFileSync, mkdtempSync, readFileSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { DatabaseSync } from 'node:sqlite'; import { afterAll, beforeAll, describe, expect, it } from 'vitest'; import { isOrganizationOwnershipBootRefusal } from '@objectstack/runtime'; +import { ORGANIZATION_OWNERSHIP_SHAPE_GENERATION } from '@objectstack/spec/system'; import type { VerifyStack } from '@objectstack/verify'; -import { columnsOf, keyRowsOf, type FixtureManifest } from './upgrade-harness/fixture-manifest.js'; +import { columnsOf, type FixtureManifest } from './upgrade-harness/fixture-manifest.js'; import { FIXTURE_NAMES, FIXTURE_PEOPLE, bootFixture } from './upgrade-harness/populate-fixture.js'; import { loadSqlIntoSqlite } from './upgrade-harness/sqlite-text-dump.js'; +import { OWNERSHIP_MARKER, runOwnershipCeremony } from './upgrade-harness/ceremony.js'; const FIXTURE = new URL('./upgrade-harness/fixture-18.0/', import.meta.url); +const MARKER = OWNERSHIP_MARKER; const manifest = JSON.parse(readFileSync(new URL('manifest.json', FIXTURE), 'utf8')) as FixtureManifest; const fixtureSql = readFileSync(new URL('database.sql', FIXTURE), 'utf8'); @@ -54,22 +70,59 @@ function refusalIn(error: unknown): unknown { return undefined; } +/** A boot of HEAD over `file`; the stack, or the error that stopped it. */ +async function tryBoot(file: string): Promise<{ stack?: VerifyStack; error?: unknown }> { + try { + return { stack: await bootFixture(file) }; + } catch (error) { + return { error }; + } +} + +/** The ceremony on `file`, as an operator runs it (`runOwnershipCeremony`). */ +function ceremony(dir: string, file: string, tag: string): Record { + return runOwnershipCeremony(dir, file, tag); +} + +/** The ceremony's marker on `file`: verified, blocking 0, at this release's generation. */ +function expectCertified(file: string): void { + const db = new DatabaseSync(file, { readOnly: true }); + try { + const marker = db.prepare('SELECT verified_at, blocking, details FROM sys_migration WHERE id = ?').get(MARKER) as + | { verified_at: unknown; blocking: unknown; details: unknown } + | undefined; + expect(marker?.verified_at, 'the ceremony verified its marker').toBeTruthy(); + expect(Number(marker?.blocking)).toBe(0); + expect(JSON.parse(String(marker?.details)).checkpoint?.shapeGeneration).toBe(ORGANIZATION_OWNERSHIP_SHAPE_GENERATION); + } finally { + db.close(); + } +} + // eslint-disable-next-line @typescript-eslint/no-explicit-any const rowsOf = (body: any): any[] => (Array.isArray(body) ? body : (body?.records ?? body?.data ?? [])); -describe(`[#24522] an 18.0 database (fixture from ${manifest.ref} @ ${manifest.sha.slice(0, 12)}) upgrades to HEAD and keeps its data`, () => { +describe(`[#24522] an 18.0 database (fixture from ${manifest.ref} @ ${manifest.sha.slice(0, 12)}) takes the ceremony, upgrades to HEAD and keeps its data`, () => { const dir = mkdtempSync(join(tmpdir(), 'dogfood-upgrade-18x-')); const file = join(dir, 'upgraded.db'); + /** + * The ceremony's output, copied once it verified, and booted. A refused boot + * lives on in this process, its SQLite driver never torn down, and a later + * boot over the SAME path can meet its stale image (measured on the + * boot-refusal-order dogfood); booting the copied bytes is the operator's + * sequence — the refused `os serve` process has exited. + */ + const migrated = join(dir, 'upgraded.migrated.db'); let stack: VerifyStack | undefined; + let refusedError: unknown; let bootError: unknown; + let report: Record | undefined; beforeAll(async () => { loadSqlIntoSqlite(fixtureSql, file); - try { - stack = await bootFixture(file); - } catch (error) { - bootError = error; - } + const refused = await tryBoot(file); + await refused.stack?.stop(); + refusedError = refused.error ?? new Error('the 18.0 database booted without the ceremony'); }, 240_000); afterAll(async () => { @@ -81,11 +134,36 @@ describe(`[#24522] an 18.0 database (fixture from ${manifest.ref} @ ${manifest.s for (const t of manifest.reshaped) expect(t.count, `${t.table} holds rows in the fixture`).toBeGreaterThan(0); }); - it('1. HEAD boots over the 18.0 database, and no ownership refusal stops it', () => { + it('1. R2, restated: HEAD refuses the 18.0 database at boot, naming the ceremony, and writes no marker', () => { + const refusal = refusalIn(refusedError) as { message?: string } | undefined; + expect(refusal, `the boot was not refused by the ownership guard: ${String((refusedError as Error)?.stack ?? refusedError)}`).toBeDefined(); + expect(refusal!.message).toContain('os migrate organization-ownership'); + expect(refusal!.message).not.toMatch(/on a 17\.x release|stay on 17|every 18\.0/); + const db = new DatabaseSync(file, { readOnly: true }); + try { + expect(db.prepare('SELECT id FROM sys_migration WHERE id = ?').all(MARKER)).toEqual([]); + } finally { + db.close(); + } + }); + + it('1b. R2, restated: the ceremony plans and applies it, and certifies it at this release\'s shape generation', () => { + report = ceremony(dir, file, 'r2'); + expect(report.complete).toBe(true); + expect(report.blocking).toBe(0); + expectCertified(file); + copyFileSync(file, migrated); + }, 640_000); + + it('1c. R2, restated: HEAD then boots over it, and no ownership refusal stops it', async () => { + expect(report, 'the ceremony ran').toBeDefined(); + const booted = await tryBoot(migrated); + stack = booted.stack; + bootError = booted.error; const refusal = refusalIn(bootError); - expect(refusal, `the ownership boot guard refused the 18.0 database: ${String((refusal as Error | undefined)?.message)}`).toBeUndefined(); + expect(refusal, `the ownership boot guard refused the migrated database: ${String((refusal as Error | undefined)?.message)}`).toBeUndefined(); expect(bootError, `the boot failed: ${String((bootError as Error | undefined)?.stack ?? bootError)}`).toBeUndefined(); - }); + }, 240_000); it('2. it serves the 18.0 data: the fixture\'s people sign in, and the fixture\'s task is there', async () => { expect(stack).toBeDefined(); @@ -98,24 +176,17 @@ describe(`[#24522] an 18.0 database (fixture from ${manifest.ref} @ ${manifest.s expect(titles).toContain(`${FIXTURE_NAMES.task} — edited`); }); - // [#24529, step 1] Declared webhooks are read from the registry, and the 18.0 - // seeder's rows stay until the ceremony. The fixture's mirror of - // `showcase_task_changed` was activated in 18.0, which stamped it - // `customized`, so it answers as itself and the declaration of the same name - // is withheld even while the declaration is switched on: one delivery per - // task change, from the row, never a second one from the declaration. - it('2b. the fixture\'s customized declared webhook dispatches exactly once on a task change', async () => { + // [18.x step 5a] The fixture's mirror of `showcase_task_changed` was only + // switched on in 18.0 (customized by its `active` alone), so the ceremony + // kept that switch as the declaration's activation row and deleted the + // copy. The declaration now answers, once: a reader that still served the + // row would deliver a second time. + it('2b. the fixture\'s declared webhook dispatches exactly once on a task change, from its declaration', async () => { expect(stack).toBeDefined(); const adminTok = await stack!.signIn(FIXTURE_PEOPLE.admin.email, FIXTURE_PEOPLE.admin.password); const hooks = rowsOf(await (await stack!.apiAs(adminTok, 'GET', `/data/sys_webhook?name=${FIXTURE_NAMES.webhook}`)).json()) .filter((h) => h?.name === FIXTURE_NAMES.webhook); - expect(hooks, 'the 18.0 mirror row is still there').toHaveLength(1); - expect(hooks[0].customized, 'the fixture activated it in 18.0, which marks it customized').toBe(true); - // Switch the declaration ON as well (the showcase ships it inactive), so a - // reader that served both the row and the declaration WOULD deliver twice: - // without this the case could not fail on the rule it pins. - const on = await stack!.apiAs(adminTok, 'POST', `/security/_activation/webhook/${FIXTURE_NAMES.webhook}`, { enabled: true }); - expect(on.status, await on.clone().text()).toBe(200); + expect(hooks, 'the 18.0 copy is gone').toEqual([]); const deliveries = async () => rowsOf(await (await stack!.apiAs(adminTok, 'GET', '/data/sys_http_delivery?limit=500')).json()) .filter((d) => d?.source === 'webhook'); const before = new Set((await deliveries()).map((d) => String(d.id))); @@ -136,31 +207,60 @@ describe(`[#24522] an 18.0 database (fixture from ${manifest.ref} @ ${manifest.s } await new Promise((r) => setTimeout(r, 1000)); fresh = (await deliveries()).filter((d) => !before.has(String(d.id))); - expect(fresh.map((d) => d.ref_id), 'one delivery, from the customized row').toEqual([hooks[0].id]); + expect(fresh.map((d) => d.ref_id), 'one delivery, from the declaration').toEqual([`declared:${FIXTURE_NAMES.webhook}`]); }); - it('3. every reshaped table keeps every 18.0 row, with its key columns, once HEAD has stopped', async () => { + it('3. every 18.0 row of every reshaped table is where the ceremony put it, with its key columns', async () => { await stack?.stop(); stack = undefined; - const db = new DatabaseSync(file, { readOnly: true }); + const db = new DatabaseSync(migrated, { readOnly: true }); try { + // Where a row may be now: its own table, or the deployment-level sibling it moved to. + const SIBLING: Record = { sys_email: 'sys_identity_email', sys_setting_audit: 'sys_platform_setting_audit' }; + // Tables that became deployment-level: their organization column is gone. + const DEPLOYMENT_LEVEL = new Set(['sys_secret', 'sys_flow_credential']); + const rowById = (table: string, id: string): Record | undefined => + db.prepare(`SELECT * FROM "${table}" WHERE id = ?`).get(id) as Record | undefined; + const project = (row: Record, cols: readonly string[]): Record => + Object.fromEntries(cols.map((c) => [c, row[c] === null || row[c] === undefined ? null : String(row[c])])); for (const t of manifest.reshaped) { - for (const physical of t.physical) { - const cols = columnsOf(db, physical); - expect(cols.length, `${physical} still exists`).toBeGreaterThan(0); - for (const c of t.keyColumns) expect(cols, `${physical}.${c} still exists`).toContain(c); + for (const fixtureRow of t.rows) { + const id = String(fixtureRow.id); + if (t.table === 'sys_webhook') { + // The declared webhook's copy: deleted, its switch kept as the declaration's activation row. + for (const physical of t.physical) expect(rowById(physical, id), `sys_webhook copy ${id} is deleted`).toBeUndefined(); + continue; + } + const home = t.physical.find((physical) => rowById(physical, id)); + const moved = !home && SIBLING[t.table] ? SIBLING[t.table] : undefined; + const now = home ? rowById(home, id) : moved ? rowById(moved, id) : undefined; + expect(now, `${t.table} row ${id} is kept${SIBLING[t.table] ? ` or moved to ${SIBLING[t.table]}` : ''}`).toBeDefined(); + const cols = moved || DEPLOYMENT_LEVEL.has(t.table) ? t.keyColumns.filter((c) => c !== 'organization_id') : t.keyColumns; + const expected = { ...project(fixtureRow, cols) }; + if (t.table === 'sys_http_delivery') expected.ref_id = `declared:${FIXTURE_NAMES.webhook}`; + expect(project(now!, cols), `${t.table} row ${id}`).toEqual(expected); } - const now = keyRowsOf(db, t.physical, t.keyColumns); - const byId = new Map(now.map((r) => [r.id, r])); - expect(now.length, `${t.table}: ${now.length} rows now, ${t.count} in the fixture`).toBeGreaterThanOrEqual(t.count); - for (const row of t.rows) expect(byId.get(row.id), `${t.table} row ${row.id}`).toEqual(row); + } + // The identity mail and the global audit rows moved; tenant mail and tenant audit stayed. + const reset = manifest.reshaped.find((t) => t.table === 'sys_email')!.rows; + expect(reset.filter((r) => rowById('sys_identity_email', String(r.id))).length, 'the reset mail moved').toBeGreaterThan(0); + expect(reset.filter((r) => rowById('sys_email', String(r.id))).length, 'the invitation mail stayed').toBeGreaterThan(0); + expect(db.prepare(`SELECT COUNT(*) AS n FROM sys_setting_audit WHERE scope = 'global'`).get()).toEqual({ n: 0 }); + // The webhook's switch, kept. + expect(db.prepare(`SELECT active FROM sys_metadata_activation WHERE metadata_type = 'webhook' AND name = ?`).get(FIXTURE_NAMES.webhook)).toEqual({ active: 1 }); + // Deployment-level tables lost the column; the tenant tables hold it NOT NULL. + for (const table of DEPLOYMENT_LEVEL) expect(columnsOf(db, table), table).not.toContain('organization_id'); + const notNull = (table: string): unknown => + (db.prepare(`SELECT "notnull" AS nn FROM pragma_table_info(?) WHERE name = 'organization_id'`).get(table) as { nn?: unknown } | undefined)?.nn; + for (const table of ['sys_email', 'sys_setting_audit', 'sys_share_link', 'sys_member', 'sys_user_permission_set']) { + expect(notNull(table), `${table}.organization_id is NOT NULL`).toBe(1); } } finally { db.close(); } }); - it('4. R3: the same database with a v17 trait is still refused at boot, with the guard\'s code', async () => { + it('4. R3, restated: the same database with a v17 trait is refused at boot, takes the ceremony, and boots', async () => { const v17 = join(dir, 'v17-shaped.db'); loadSqlIntoSqlite(fixtureSql, v17); const db = new DatabaseSync(v17); @@ -170,15 +270,26 @@ describe(`[#24522] an 18.0 database (fixture from ${manifest.ref} @ ${manifest.s } finally { db.close(); } - let error: unknown; - try { - const refused = await bootFixture(v17); - await refused.stop(); - } catch (e) { - error = e; - } - const refusal = refusalIn(error); + const refused = await tryBoot(v17); + await refused.stack?.stop(); + const refusal = refusalIn(refused.error); expect(refusal, 'the v17-shaped database was refused by the ownership boot guard').toBeDefined(); expect((refusal as { tables?: string[] }).tables).toContain('sys_metadata'); - }, 240_000); + + const v17Report = ceremony(dir, v17, 'r3'); + expect([v17Report.complete, v17Report.blocking]).toEqual([true, 0]); + expectCertified(v17); + const check = new DatabaseSync(v17, { readOnly: true }); + try { + expect(columnsOf(check, 'sys_metadata')).not.toContain('organization_id'); + } finally { + check.close(); + } + const v17Migrated = join(dir, 'v17-shaped.migrated.db'); + copyFileSync(v17, v17Migrated); + const booted = await tryBoot(v17Migrated); + await booted.stack?.stop(); + expect(refusalIn(booted.error), String((booted.error as Error | undefined)?.message)).toBeUndefined(); + expect(booted.error).toBeUndefined(); + }, 900_000); }); diff --git a/packages/qa/dogfood/test/upgrade-harness/ceremony.ts b/packages/qa/dogfood/test/upgrade-harness/ceremony.ts new file mode 100644 index 00000000000..d17bdae6f46 --- /dev/null +++ b/packages/qa/dogfood/test/upgrade-harness/ceremony.ts @@ -0,0 +1,74 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// [18.x step 6] The upgrade ceremony over a copy of the 18.0 fixture, as an +// operator runs it — the plan, then the apply of that reviewed plan file, +// through the real `os migrate organization-ownership` binary in the +// showcase's directory, so it composes what the deployment serves. Shared by +// the upgrade harness and every suite that boots the 18.0 fixture under the +// ownership boot guard: the guard refuses an 18.0 database until the ceremony +// has certified it. +import { spawnSync } from 'node:child_process'; +import { copyFileSync, readFileSync } from 'node:fs'; +import { createRequire } from 'node:module'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { isOrganizationOwnershipBootRefusal } from '@objectstack/runtime'; +import { bootFixture } from './populate-fixture.js'; +import { loadSqlIntoSqlite } from './sqlite-text-dump.js'; + +const SHOWCASE_DIR = fileURLToPath(new URL('../../../../../examples/app-showcase/', import.meta.url)); +const CLI_ROOT = dirname(createRequire(import.meta.url).resolve('@objectstack/cli/package.json')); +const CLI_BIN = join(CLI_ROOT, 'bin', 'run.js'); + +/** The ceremony's sys_migration row: its marker. */ +export const OWNERSHIP_MARKER = 'adr-0131-organization-ownership'; + +/** + * Run the ceremony on `file`: the plan into `dir/-plan.json`, then + * `--apply --plan-file … --backup-taken`. The apply report. Throws, with both + * runs' output, when either exits non-zero. + */ +export function runOwnershipCeremony(dir: string, file: string, tag: string): Record { + const run = (args: string[]) => spawnSync(process.execPath, [ + CLI_BIN, 'migrate', 'organization-ownership', '--database-url', `file:${file}`, ...args, + ], { cwd: SHOWCASE_DIR, encoding: 'utf8', timeout: 300_000, env: { ...process.env, NO_COLOR: '1' } }); + const planFile = join(dir, `${tag}-plan.json`); + const plan = run(['--out', planFile]); + if (plan.status !== 0) throw new Error(`the ceremony's plan exited ${plan.status}:\n${plan.stdout}\n${plan.stderr}`); + const reportFile = join(dir, `${tag}-apply.json`); + const apply = run(['--apply', '--plan-file', planFile, '--backup-taken', '--out', reportFile, '--json']); + if (apply.status !== 0) throw new Error(`the ceremony's apply exited ${apply.status}:\n${apply.stdout}\n${apply.stderr}`); + return JSON.parse(readFileSync(reportFile, 'utf8')) as Record; +} + +/** + * A copy of the 18.0 fixture the ceremony has certified, at `dir/.db`, + * ready to boot under the guard — reached the way an operator reaches it. + * First a boot of this release over the 18.0 bytes: the guard refuses it, and + * its schema sync has already created the tables this release added (the + * sibling logs the ceremony's `sibling-moves` step moves rows into), which the + * ceremony asks for rather than creates. Then the plan and its apply. The + * ceremony runs on a working copy and the certified bytes are copied out, so + * no boot ever opens the path the ceremony's process held (the stale-image + * hazard the harness measured). Throws when the first boot is not refused by + * the ownership guard. + */ +export async function ceremoniedFixture(dir: string, fixtureSql: string, tag: string): Promise<{ file: string; report: Record }> { + const work = join(dir, `${tag}.ceremony.db`); + loadSqlIntoSqlite(fixtureSql, work); + let refusal: unknown; + try { + const stack = await bootFixture(work); + await stack.stop(); + } catch (error) { + for (let e = error, depth = 0; !!e && typeof e === 'object' && depth < 8; e = (e as { cause?: unknown }).cause, depth++) { + if (isOrganizationOwnershipBootRefusal(e)) refusal = e; + } + if (!refusal) throw error; + } + if (!refusal) throw new Error('the 18.0 fixture booted without the ceremony: the ownership guard did not refuse it'); + const report = runOwnershipCeremony(dir, work, tag); + const file = join(dir, `${tag}.db`); + copyFileSync(work, file); + return { file, report }; +} diff --git a/packages/runtime/package.json b/packages/runtime/package.json index d0a6147c567..a42edced741 100644 --- a/packages/runtime/package.json +++ b/packages/runtime/package.json @@ -61,6 +61,7 @@ "@objectstack/driver-mongodb": "workspace:*" }, "devDependencies": { + "@libsql/client": "^0.18.0", "@objectstack/platform-objects": "workspace:*", "@objectstack/plugin-hono-server": "workspace:*", "@objectstack/plugin-sharing": "workspace:*", diff --git a/packages/runtime/src/index.ts b/packages/runtime/src/index.ts index a82a2c8b2de..31b907cd78a 100644 --- a/packages/runtime/src/index.ts +++ b/packages/runtime/src/index.ts @@ -69,6 +69,9 @@ export { DriverPlugin } from './driver-plugin.js'; export { MigrationRecoveryPlugin, describeInterruptedRun } from './migration-recovery-plugin.js'; export { OrganizationOwnershipBootGuardPlugin, + OrganizationOwnershipBirthMarkerPlugin, + ORGANIZATION_OWNERSHIP_BIRTH_MARKER_PLUGIN_NAME, + recordOrganizationOwnershipBirth, OrganizationOwnershipBootRefusal, ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE, assertOrganizationOwnershipMigrated, @@ -76,7 +79,7 @@ export { isOrganizationOwnershipBootRefusal, organizationOwnershipRefusalMessage, } from './organization-ownership-boot-guard.js'; -export type { OrganizationOwnershipBootGuardOptions } from './organization-ownership-boot-guard.js'; +export type { OrganizationOwnershipBootGuardOptions, OrganizationOwnershipMarkerState } from './organization-ownership-boot-guard.js'; export { DefaultDatasourcePlugin } from './default-datasource-plugin.js'; export type { DefaultDatasourceDefinition, DefaultDatasourcePluginOptions } from './default-datasource-plugin.js'; export { AppPlugin, collectBundleHooks, collectBundleFunctions, collectBundleFunctionEntries, collectBundleActions } from './app-plugin.js'; diff --git a/packages/runtime/src/organization-ownership-boot-guard.kernel.test.ts b/packages/runtime/src/organization-ownership-boot-guard.kernel.test.ts index 51d2e1578d0..97735bd7e55 100644 --- a/packages/runtime/src/organization-ownership-boot-guard.kernel.test.ts +++ b/packages/runtime/src/organization-ownership-boot-guard.kernel.test.ts @@ -13,11 +13,12 @@ * column still carries a physical `organization_id`) is refused, with the * refusal's code, and the kernel never reaches `running`; * 2. the same database with the ceremony's marker verified boots; - * 3. a fresh database boots; + * 3. a fresh database boots, the guard writing its birth marker, and boots + * again on that marker; a database an earlier boot created WITHOUT the + * guard (an 18.0 pre-release install's: no retired column, no marker) is refused; * 4. a database the ceremony ran on to the end (no column left) whose marker * is NOT verified — its post-check found rows still without their - * organization — is refused, saying what 18.0 supports (it cannot re-run - * the ceremony); + * organization — is refused, naming the re-run; * 5. ⭐ a refused boot writes nothing: the plugins that write rows in their * own `start()` — `AppPlugin` (the inline seed) and plugin-auth's * identity-objects plugin (the Default Organization, `AuthPlugin`'s twin @@ -35,6 +36,7 @@ import { ObjectKernel } from '@objectstack/core'; import { SqlDriver } from '@objectstack/driver-sql'; import { ObjectQLPlugin } from '@objectstack/objectql'; import { PlatformObjectsPlugin } from '@objectstack/platform-objects/plugin'; +import { ORGANIZATION_OWNERSHIP_SHAPE_GENERATION } from '@objectstack/spec/system'; import { createIdentityObjectsPlugin } from '@objectstack/plugin-auth'; import { AppPlugin } from './app-plugin.js'; import { DriverPlugin } from './driver-plugin.js'; @@ -88,10 +90,12 @@ function seedV17Shape(file: string, marker?: { verifiedAt: string | null }): voi db.close(); } +/** The ceremony's marker, its checkpoint at this release's shape generation. */ function writeMarker(db: DatabaseSync, verifiedAt: string | null, blocking = 0): void { - db.exec('CREATE TABLE IF NOT EXISTS sys_migration (id TEXT PRIMARY KEY, last_run_at TEXT, verified_at TEXT, applied_at TEXT, blocking INTEGER)'); - db.prepare('INSERT INTO sys_migration (id, last_run_at, verified_at, applied_at, blocking) VALUES (?, ?, ?, ?, ?)') - .run(MARKER, '2026-10-14T00:00:00.000Z', verifiedAt, '2026-10-14T00:00:00.000Z', blocking); + db.exec('CREATE TABLE IF NOT EXISTS sys_migration (id TEXT PRIMARY KEY, last_run_at TEXT, verified_at TEXT, applied_at TEXT, blocking INTEGER, details TEXT)'); + const details = JSON.stringify({ checkpoint: { ceremony: 'ADR-0131 D10', inventoryDigest: 'x', completed: {}, shapeGeneration: ORGANIZATION_OWNERSHIP_SHAPE_GENERATION } }); + db.prepare('INSERT INTO sys_migration (id, last_run_at, verified_at, applied_at, blocking, details) VALUES (?, ?, ?, ?, ?, ?)') + .run(MARKER, '2026-10-14T00:00:00.000Z', verifiedAt, '2026-10-14T00:00:00.000Z', blocking, details); } /** An app whose inline seed writes two rows of an organization-owned object. */ @@ -167,22 +171,38 @@ describe('the v18 boot refusal on a real kernel (ADR-0131 D10 item 5)', () => { expect(refusalIn(error)?.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); }); - it('⭐ a fresh database boots: the guard reads the ledger, finds no marker (absent, never unreadable), and starts', async () => { - const { kernel, error } = await boot(databaseFile()); + it('⭐ R1: a fresh database boots, the guard writing its birth marker; the next boot reads it and boots', async () => { + const file = databaseFile(); + const { kernel, error } = await boot(file); expect(error).toBeNull(); expect(kernel.getState()).toBe('running'); - // The read the guard made is a real one: the ledger object is registered and - // its table answers, so the guard took the read, not the "no ledger" branch, - // and a throwing read would have refused the boot as 'unreadable'. - const engine = kernel.getService('objectql') as { - getObject(name: string): unknown; - findOne(object: string, query: Record): Promise; - }; - expect(engine.getObject('sys_migration')).toBeTruthy(); - await expect(engine.findOne('sys_migration', { where: { id: MARKER }, context: { isSystem: true } })).resolves.toBeNull(); + const [row] = sql<{ verified_at: string | null; blocking: number; details: string }>( + file, + `SELECT verified_at, blocking, details FROM sys_migration WHERE id = '${MARKER}'`, + ); + expect(row?.verified_at).toBeTruthy(); + expect(row?.blocking).toBe(0); + expect(JSON.parse(String(row!.details)).birth).toMatchObject({ shapeGeneration: ORGANIZATION_OWNERSHIP_SHAPE_GENERATION }); + await kernel.shutdown(); + // The same store, found this time: the marker carries it. + expect((await boot(file)).error).toBeNull(); + expect(sql(file, `SELECT id FROM sys_migration WHERE id = '${MARKER}'`)).toHaveLength(1); + }); + + it('⭐ a database an earlier boot created without the guard (an 18.0 pre-release install: no retired column, no marker) is refused', async () => { + const file = databaseFile(); + const db = new DatabaseSync(file); + db.exec('CREATE TABLE sys_migration (id TEXT PRIMARY KEY, last_run_at TEXT, verified_at TEXT, applied_at TEXT, blocking INTEGER)'); + db.close(); + const { kernel, error } = await boot(file); + const refusal = refusalIn(error); + expect(refusal?.code, String(error)).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); + expect(refusal?.message).toMatch(/^This database carries no organization-ownership marker/); + expect(kernel.getState()).not.toBe('running'); + expect(sql(file, `SELECT id FROM sys_migration`)).toEqual([]); }); - it('⭐ a ceremony that ran every step and did not verify (no v17 column left, marker unverified) is refused, saying what 18.0 supports', async () => { + it('⭐ a ceremony that ran every step and did not verify (no v17 column left, marker unverified) is refused, naming the re-run', async () => { const file = databaseFile(); const db = new DatabaseSync(file); writeMarker(db, null, 602); @@ -192,10 +212,7 @@ describe('the v18 boot refusal on a real kernel (ADR-0131 D10 item 5)', () => { expect(refusal?.code, String(error)).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); expect(refusal?.tables).toEqual([]); expect(refusal?.message).toMatch(/^The organization-ownership ceremony has not finished on this database: its marker/); - // 18.0 cannot re-run the ceremony (its `os migrate organization-ownership` refuses on every - // database), so the refusal names no step of it and says what 18.0 supports instead. - expect(refusal?.message).toContain('18.0 supports fresh installs only and does not upgrade a v17 database'); - expect(refusal?.message).not.toMatch(/os migrate organization-ownership|--plan-file|--out PLAN|--backup-taken/); + expect(refusal?.message).toContain("os migrate organization-ownership --apply --plan-file PLAN --backup-taken"); expect(kernel.getState()).not.toBe('running'); }); diff --git a/packages/runtime/src/organization-ownership-boot-guard.test.ts b/packages/runtime/src/organization-ownership-boot-guard.test.ts index 87755e2c693..3034d0ee34f 100644 --- a/packages/runtime/src/organization-ownership-boot-guard.test.ts +++ b/packages/runtime/src/organization-ownership-boot-guard.test.ts @@ -1,37 +1,54 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. /** - * [ADR-0131 D10 item 5; ruling 6163114098, Q3 A] The v18 boot refusal's pins: - * a database still in the v17 shape is refused and the command named; a - * database whose ceremony verified boots; a fresh database boots. Plus the - * fail-closed read, and a driver with no physical columns left inert. + * [ADR-0131 D10 item 5; ruling 6163114098, Q3 A; ruling record 6269955919 on + * the v18 epic, ① A] The v18 boot refusal's pins. The guard decides on ONE + * field, the ownership shape generation the ceremony's `sys_migration` row + * attests: + * + * - a verified marker at the current generation boots — the ceremony's, or + * the BIRTH MARKER the guard writes once on a store created from empty; + * - no marker on any other store (a v17 database, an 18.0 pre-release one) is refused, + * naming the ceremony and the tables still carrying a retired column; + * - a verified marker below the current generation (a shorter ceremony's) is + * refused, naming the ceremony again; + * - an unverified or unreadable marker is refused; a failing column read is + * refused; a store with no physical columns is inert. * * The columns are read from a real SQLite database through the guard's own - * raw-SQL probe; the engine is a minimal double exposing the three seams the - * guard reads (the registry, the driver an object binds to, `findOne`). + * raw-SQL probe; the engine is a minimal double exposing the seams the guard + * reads (the registry, the driver an object binds to and its schema-sync + * stats, `findOne`, `insert`). */ import { afterEach, beforeEach, describe, expect, it } from 'vitest'; import { DatabaseSync } from 'node:sqlite'; +import { assertEngineFindOnePredicate } from '@objectstack/metadata-core'; +import { ORGANIZATION_OWNERSHIP_SHAPE_GENERATION } from '@objectstack/spec/system'; import { AppPlugin } from './app-plugin.js'; import { assertOrganizationOwnershipMigrated, isOrganizationOwnershipBootRefusal, + OrganizationOwnershipBirthMarkerPlugin, OrganizationOwnershipBootGuardPlugin, + recordOrganizationOwnershipBirth, ORGANIZATION_OWNERSHIP_BOOT_GUARD_PLUGIN_NAME, ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE, } from './organization-ownership-boot-guard.js'; let db: DatabaseSync; -/** The objects the registry serves: two no-column objects and one tenant-walled one. */ +/** The objects the registry serves: the three spellings of "no organization column", and one tenant-walled object. */ const OBJECTS = [ { name: 'sys_job', systemFields: { tenant: false }, fields: { status: {} } }, { name: 'sys_metadata', systemFields: { tenant: false }, fields: { type: {}, name: {} } }, + { name: 'sys_api_key', tenancy: { enabled: false }, fields: { key: {} } }, { name: 'crm_deal', fields: { amount: {} } }, ]; -function engine(opts: { client?: unknown; dialectName?: string; failRead?: boolean } = {}) { +type Stats = { created: number; existing: number } | undefined; + +function engine(opts: { client?: unknown; dialectName?: string; failRead?: boolean; stats?: Stats; failInsert?: boolean } = {}) { const driver = { ...(opts.dialectName ? { dialectName: opts.dialectName } : {}), config: { client: opts.client ?? 'better-sqlite3' }, @@ -39,53 +56,67 @@ function engine(opts: { client?: unknown; dialectName?: string; failRead?: boole if (opts.failRead) throw new Error('SQLITE_BUSY: database is locked'); return db.prepare(sql).all(...(params as string[])); }, + getSchemaSyncStats: () => opts.stats, }; return { registry: { getAllObjects: () => OBJECTS }, getDriverForObject: () => driver, async findOne(object: string, query: Record) { + assertEngineFindOnePredicate(object, query); const exists = db.prepare(`SELECT name FROM sqlite_master WHERE type = 'table' AND name = ?`).all(object).length > 0; if (!exists) return null; const id = String((query.where as { id?: unknown } | undefined)?.id); return db.prepare(`SELECT * FROM ${object} WHERE id = ?`).get(id) ?? null; }, + async insert(object: string, data: Record) { + if (opts.failInsert) throw new Error('SQLITE_READONLY: attempt to write a readonly database'); + const keys = Object.keys(data).filter((k) => !['created_at', 'updated_at'].includes(k)); + db.prepare(`INSERT INTO ${object} (${keys.join(', ')}) VALUES (${keys.map(() => '?').join(', ')})`).run( + ...(keys.map((k) => data[k]) as never[]), + ); + return data; + }, }; } +const LEDGER = 'CREATE TABLE sys_migration (id TEXT PRIMARY KEY, last_run_at TEXT, verified_at TEXT, applied_at TEXT, blocking INTEGER, advisory INTEGER, details TEXT);'; + /** The tables a v17 runtime created: every one carries the injected organization column. */ function v17Shape(): void { db.exec(` CREATE TABLE sys_job (id TEXT PRIMARY KEY, status TEXT, organization_id TEXT); CREATE TABLE sys_metadata (id TEXT PRIMARY KEY, type TEXT, name TEXT, organization_id TEXT); + CREATE TABLE sys_api_key (id TEXT PRIMARY KEY, key TEXT, organization_id TEXT); + CREATE TABLE crm_deal (id TEXT PRIMARY KEY, amount REAL, organization_id TEXT); + ${LEDGER} + `); +} + +/** What an 18.0 pre-release install left, or what the ceremony's column-drops step leaves: no table in the v17 shape. */ +function columnlessShape(): void { + db.exec(` + CREATE TABLE sys_job (id TEXT PRIMARY KEY, status TEXT); + CREATE TABLE sys_metadata (id TEXT PRIMARY KEY, type TEXT, name TEXT); + CREATE TABLE sys_api_key (id TEXT PRIMARY KEY, key TEXT); CREATE TABLE crm_deal (id TEXT PRIMARY KEY, amount REAL, organization_id TEXT); - CREATE TABLE sys_migration (id TEXT PRIMARY KEY, last_run_at TEXT, verified_at TEXT, applied_at TEXT, blocking INTEGER, advisory INTEGER); + ${LEDGER} `); } -const marker = (verifiedAt: string | null, blocking = 0): void => { - db.prepare('INSERT INTO sys_migration VALUES (?, ?, ?, ?, ?, ?)').run( - 'adr-0131-organization-ownership', '2026-10-14T00:00:00Z', verifiedAt, '2026-10-14T00:00:00Z', blocking, 0, +/** The ceremony's checkpoint marker, at `generation` (`null`: a marker written before generations existed). */ +const marker = (verifiedAt: string | null, blocking = 0, generation: number | null = ORGANIZATION_OWNERSHIP_SHAPE_GENERATION): void => { + const details = JSON.stringify({ checkpoint: { ceremony: 'ADR-0131 D10', inventoryDigest: 'x', completed: {}, ...(generation === null ? {} : { shapeGeneration: generation }) } }); + db.prepare('INSERT INTO sys_migration VALUES (?, ?, ?, ?, ?, ?, ?)').run( + 'adr-0131-organization-ownership', '2026-10-14T00:00:00Z', verifiedAt, '2026-10-14T00:00:00Z', blocking, 0, details, ); }; -/** - * What 18.0 supports for a v17 database: the sentence every refusal closes on, - * in the words `@objectstack/cli`'s other 18.0 refusals use (the drop gate of - * `os migrate apply`; `os dev`'s way on for a development database). - */ -const FRESH_INSTALLS_ONLY = - '18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release ' + - 'until the 18.x release that carries the upgrade path, and upgrade it there.'; -const FRESH_INSTALLS_ONLY_DEV = - '18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 development database on a ' + - '17.x release until the 18.x release that carries the upgrade path, and upgrade it there.'; -/** `os migrate organization-ownership` refuses on every database in 18.0: no refusal names it or a step of it. */ -const WITHDRAWN_COMMAND = /os migrate organization-ownership|--plan-file|--out PLAN|--backup-taken/; - -/** The message closes on `tail` (compared as a string, so a failure shows the diff). */ -function expectTail(message: string, tail: string): void { - expect(message.slice(-tail.length)).toBe(tail); -} +const markerRow = (): Record | undefined => + db.prepare(`SELECT * FROM sys_migration WHERE id = 'adr-0131-organization-ownership'`).get() as Record | undefined; + +/** The schema-sync stats of a store this boot created from empty, and of one it found. */ +const CREATED_EMPTY: Stats = { created: 40, existing: 0 }; +const FOUND: Stats = { created: 0, existing: 40 }; async function refusalOf(run: () => Promise): Promise { try { @@ -103,80 +134,126 @@ afterEach(() => { db.close(); }); -describe('the v18 boot refusal (ADR-0131 D10 item 5)', () => { - it('⭐ a v17-shape database is refused, naming every table still carrying the column', async () => { +describe('the v18 boot refusal (ADR-0131 D10 item 5; the birth marker, ruling ① A)', () => { + it('⭐ a v17-shape database is refused, naming the command and every table still carrying the column — tenancy.enabled: false included', async () => { v17Shape(); - const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine())); + const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: FOUND }))); expect(isOrganizationOwnershipBootRefusal(err)).toBe(true); expect(err.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); - expect(err.tables).toEqual(['sys_job', 'sys_metadata']); - expect(err.message).toMatch(/still in the v17 organization-ownership shape: sys_job, sys_metadata/); - expect(err.message).toMatch(/the ceremony has not run on it/); - // What it prescribes is pinned per branch below: what 18.0 supports, and no step of the withdrawn command. + expect(err.tables).toEqual(['sys_api_key', 'sys_job', 'sys_metadata']); + expect(err.message).toMatch(/carries no organization-ownership marker: 17\.x or an 18\.0 pre-release \(18\.0\.0-next\.\*\) created it/); + expect(err.message).toContain('sys_api_key, sys_job, sys_metadata still carry a physical organization_id column its object no longer declares.'); + expect(err.message).toMatch(/os migrate organization-ownership --apply --plan-file PLAN --backup-taken/); + // An 18.0 pre-release database takes the same ceremony: the text names no release line to stay on, + // and never says every 18.0 database (18.0.0 itself writes the birth marker). + expect(err.message).not.toMatch(/17\.x release|keep this one|does not upgrade|every 18\.0/); }); - it('a development boot speaks of a development database, as os dev does; a served boot does not', async () => { + it('a development boot names the ceremony with --dev, so it boots the way the refused boot did', async () => { v17Shape(); - const dev = await refusalOf(() => assertOrganizationOwnershipMigrated(engine(), undefined, { dev: true })); - expect(dev.message).toContain(FRESH_INSTALLS_ONLY_DEV); - expect(dev.message).not.toMatch(WITHDRAWN_COMMAND); - expect(dev.message).not.toContain('--dev'); - const served = await refusalOf(() => assertOrganizationOwnershipMigrated(engine())); - expect(served.message).toContain(FRESH_INSTALLS_ONLY); - expect(served.message).not.toContain('development'); + const dev = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: FOUND }), undefined, { dev: true })); + expect(dev.message).toContain( + "Run 'os migrate organization-ownership --dev --out PLAN' to write and review its plan, take a backup, then run " + + "'os migrate organization-ownership --dev --apply --plan-file PLAN --backup-taken'", + ); + const served = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: FOUND }))); + expect(served.message).not.toContain('--dev'); + }); + + it('⭐ R1: a store created from empty gets its birth marker, exactly once, and boots — again and again', async () => { + columnlessShape(); + const fixed = new Date('2026-10-19T12:00:00.000Z'); + await expect(assertOrganizationOwnershipMigrated(engine({ stats: CREATED_EMPTY }), undefined, { now: () => fixed })).resolves.toBeUndefined(); + const row = markerRow()!; + expect(row).toMatchObject({ verified_at: fixed.toISOString(), blocking: 0 }); + const details = JSON.parse(String(row.details)); + expect(Object.keys(details)).toEqual(['birth']); + expect(details.birth).toMatchObject({ shapeGeneration: ORGANIZATION_OWNERSHIP_SHAPE_GENERATION, createdAt: fixed.toISOString() }); + expect(typeof details.birth.platformVersion).toBe('string'); + // The next boot finds its own store and the marker: it boots, and writes nothing. + await expect(assertOrganizationOwnershipMigrated(engine({ stats: FOUND }))).resolves.toBeUndefined(); + expect(markerRow()).toEqual(row); + }); + + it('⭐ an 18.0 pre-release database — no retired column, no marker, a store this boot found — is refused, naming the ceremony', async () => { + columnlessShape(); + const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: FOUND }))); + expect(err.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); + expect(err.tables).toEqual([]); + expect(err.message).toMatch(/^This database carries no organization-ownership marker/); + expect(markerRow()).toBeUndefined(); + }); + + it('"cannot say" is no: a store whose driver reports no stats, on an engine that cannot say either, is refused and no marker written', async () => { + columnlessShape(); + expect(isOrganizationOwnershipBootRefusal(await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: undefined }))))).toBe(true); + expect(markerRow()).toBeUndefined(); + }); + + it('a birth marker that cannot be written refuses the boot, saying the next boot would take the store for an old one', async () => { + columnlessShape(); + const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: CREATED_EMPTY, failInsert: true }))); + expect(err.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); + expect(err.message).toMatch(/birth marker .* could not be written \(SQLITE_READONLY/); + }); + + it('⭐ F5: a verified marker below the current generation — a shorter ceremony, or one written before generations — is refused, naming the ceremony', async () => { + columnlessShape(); + marker('2026-10-14T01:00:00Z', 0, null); + const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: FOUND }))); + expect(err.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); + expect(err.message).toContain( + `certified by an organization-ownership ceremony of shape generation 0, and this release requires generation ${ORGANIZATION_OWNERSHIP_SHAPE_GENERATION}`, + ); + expect(err.message).toMatch(/os migrate organization-ownership --apply --plan-file PLAN --backup-taken/); }); it('a marker that is not verified (an unfinished ceremony, or one whose post-check found discrepancies) is refused', async () => { v17Shape(); marker(null); - expect((await refusalOf(() => assertOrganizationOwnershipMigrated(engine()))).message).toMatch(/not finished on it/); + expect((await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: FOUND })))).message).toMatch(/has not finished on this database/); db.exec(`DELETE FROM sys_migration`); marker('2026-10-14T01:00:00Z', 2); - expect(isOrganizationOwnershipBootRefusal(await refusalOf(() => assertOrganizationOwnershipMigrated(engine())))).toBe(true); + expect(isOrganizationOwnershipBootRefusal(await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: FOUND }))))).toBe(true); }); - it('⭐ a database whose ceremony verified boots', async () => { + it('⭐ a database whose ceremony verified at the current generation boots, whatever its columns', async () => { v17Shape(); marker('2026-10-14T01:00:00Z'); - await expect(assertOrganizationOwnershipMigrated(engine())).resolves.toBeUndefined(); + await expect(assertOrganizationOwnershipMigrated(engine({ stats: FOUND }))).resolves.toBeUndefined(); }); - /** What the ceremony's column-drops step leaves: no table in the v17 shape, the ledger beside it. */ - const migrated = (): void => { - db.exec(` - CREATE TABLE sys_job (id TEXT PRIMARY KEY, status TEXT); - CREATE TABLE sys_metadata (id TEXT PRIMARY KEY, type TEXT, name TEXT); - CREATE TABLE crm_deal (id TEXT PRIMARY KEY, amount REAL, organization_id TEXT); - CREATE TABLE sys_migration (id TEXT PRIMARY KEY, last_run_at TEXT, verified_at TEXT, applied_at TEXT, blocking INTEGER, advisory INTEGER); - `); - }; - it('⭐ a migrated database (the columns dropped) whose ceremony verified boots', async () => { - migrated(); + columnlessShape(); marker('2026-10-14T01:00:00Z'); - await expect(assertOrganizationOwnershipMigrated(engine())).resolves.toBeUndefined(); + await expect(assertOrganizationOwnershipMigrated(engine({ stats: FOUND }))).resolves.toBeUndefined(); }); it('⭐ a migrated database (the columns dropped) whose ceremony did NOT verify is refused: its post-check found rows it has not carried', async () => { - migrated(); + columnlessShape(); marker(null, 602); - const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine())); + const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: FOUND }))); expect(isOrganizationOwnershipBootRefusal(err)).toBe(true); expect(err.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); expect(err.tables).toEqual([]); expect(err.message).toMatch(/^The organization-ownership ceremony has not finished on this database: its marker \(sys_migration 'adr-0131-organization-ownership'\) is not verified/); + expect(err.message).toContain( + "Resolve what the apply report names, then run 'os migrate organization-ownership --out PLAN' again, review it, run " + + "'os migrate organization-ownership --apply --plan-file PLAN --backup-taken', and start again.", + ); // A verified_at with blocking left is not verified either (`isDataMigrationFlagVerified`). db.exec('DELETE FROM sys_migration'); marker('2026-10-14T01:00:00Z', 3); - expect(isOrganizationOwnershipBootRefusal(await refusalOf(() => assertOrganizationOwnershipMigrated(engine())))).toBe(true); + expect(isOrganizationOwnershipBootRefusal(await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: FOUND }))))).toBe(true); }); - it('a migrated database whose marker cannot be read is refused, never passed', async () => { - migrated(); - const failing = { ...engine(), findOne: async () => { throw new Error('SQLITE_BUSY: database is locked'); } }; + it('a marker that cannot be read is refused, never passed — and never overwritten with a birth marker', async () => { + columnlessShape(); + const failing = { ...engine({ stats: CREATED_EMPTY }), findOne: async () => { throw new Error('SQLITE_BUSY: database is locked'); } }; const err = await refusalOf(() => assertOrganizationOwnershipMigrated(failing)); expect(err.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); - expect(err.message).toMatch(/its marker \(sys_migration 'adr-0131-organization-ownership'\) could not be read/); + expect(err.message).toMatch(/marker \(sys_migration 'adr-0131-organization-ownership'\) of this database could not be read/); + expect(markerRow()).toBeUndefined(); }); it('a boot with no storage — no driver serves the ledger, and the engine refuses its read — has no database to judge, and boots', async () => { @@ -188,36 +265,52 @@ describe('the v18 boot refusal (ADR-0131 D10 item 5)', () => { await expect(assertOrganizationOwnershipMigrated(storageless)).resolves.toBeUndefined(); }); - it('a migrated database with no marker (no ceremony ran on it) boots; so does a composition with no ledger object', async () => { - migrated(); - await expect(assertOrganizationOwnershipMigrated(engine())).resolves.toBeUndefined(); - marker(null, 1); - const noLedger = { ...engine(), getObject: () => undefined }; + it('a composition with no ledger object has nothing to judge, and boots', async () => { + columnlessShape(); + const noLedger = { ...engine({ stats: FOUND }), getObject: () => undefined }; await expect(assertOrganizationOwnershipMigrated(noLedger)).resolves.toBeUndefined(); }); - it('⭐ a fresh database boots: no table it creates carries a retired column, and no marker is needed', async () => { - await expect(assertOrganizationOwnershipMigrated(engine())).resolves.toBeUndefined(); - }); - it('a column read that fails is refused, never taken for a migrated database', async () => { v17Shape(); - const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ failRead: true }))); + const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ failRead: true, stats: FOUND }))); expect(isOrganizationOwnershipBootRefusal(err)).toBe(true); - expect(err.message).toMatch(/could not read whether this database is still in the v17/); + expect(err.message).toMatch(/could not read whether this database is still in an earlier organization-ownership shape/); }); it('⭐ a driver whose knex client is a dialect CLASS (SqliteWasmDriver, the default for a served file database) is read by its dialectName, and refused', async () => { v17Shape(); class Client_WasmSqlite {} - const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ client: Client_WasmSqlite, dialectName: 'sqlite' }))); + const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ client: Client_WasmSqlite, dialectName: 'sqlite', stats: FOUND }))); expect(isOrganizationOwnershipBootRefusal(err)).toBe(true); - expect(err.tables).toEqual(['sys_job', 'sys_metadata']); + expect(err.tables).toEqual(['sys_api_key', 'sys_job', 'sys_metadata']); }); - it('a driver with no physical columns (memory, MongoDB) is inert', async () => { + it('a driver with no physical columns (memory, MongoDB) is inert: nothing refused, no marker written', async () => { v17Shape(); - await expect(assertOrganizationOwnershipMigrated(engine({ client: 'memory' }))).resolves.toBeUndefined(); + await expect(assertOrganizationOwnershipMigrated(engine({ client: 'memory', stats: CREATED_EMPTY }))).resolves.toBeUndefined(); + expect(markerRow()).toBeUndefined(); + }); + + it('⭐ the birth half alone (every CLI boot): writes the marker on a store it created from empty, and never refuses', async () => { + columnlessShape(); + // A store this boot found: nothing owed, nothing written, no refusal. + await expect(recordOrganizationOwnershipBirth(engine({ stats: FOUND }))).resolves.toBeNull(); + expect(markerRow()).toBeUndefined(); + // A v17-shape store is never given a birth marker, even one reported created. + db.exec(`DROP TABLE sys_job; CREATE TABLE sys_job (id TEXT PRIMARY KEY, status TEXT, organization_id TEXT);`); + await expect(recordOrganizationOwnershipBirth(engine({ stats: CREATED_EMPTY }))).resolves.toBeNull(); + expect(markerRow()).toBeUndefined(); + db.exec(`DROP TABLE sys_job; CREATE TABLE sys_job (id TEXT PRIMARY KEY, status TEXT);`); + // Created from empty: the marker, once; the plugin over a failing write logs and starts. + const ctx: any = { getService: () => engine({ stats: CREATED_EMPTY }), logger: {} }; + await new OrganizationOwnershipBirthMarkerPlugin().start(ctx); + expect(JSON.parse(String(markerRow()!.details)).birth.shapeGeneration).toBe(ORGANIZATION_OWNERSHIP_SHAPE_GENERATION); + db.exec('DELETE FROM sys_migration'); + const errors: string[] = []; + const failing: any = { getService: () => engine({ stats: CREATED_EMPTY, failInsert: true }), logger: { error: (m: string) => errors.push(m) } }; + await expect(new OrganizationOwnershipBirthMarkerPlugin().start(failing)).resolves.toBeUndefined(); + expect(errors).toEqual([expect.stringMatching(/the first 'os serve' will refuse it/)]); }); it('⭐ the plugin refuses in its own start(), registering no hook: a refused boot stops before any later plugin starts', async () => { @@ -225,7 +318,7 @@ describe('the v18 boot refusal (ADR-0131 D10 item 5)', () => { const hooks = new Map Promise>(); const ctx: any = { hook: (event: string, handler: () => Promise) => hooks.set(event, handler), - getService: (name: string) => (name === 'objectql' ? engine() : undefined), + getService: (name: string) => (name === 'objectql' ? engine({ stats: FOUND }) : undefined), logger: {}, }; const plugin = new OrganizationOwnershipBootGuardPlugin(); @@ -254,80 +347,9 @@ describe('the v18 boot refusal (ADR-0131 D10 item 5)', () => { for (const key of ['OS_ALLOW_UNMIGRATED_ORGANIZATION_OWNERSHIP', 'OS_SKIP_ORGANIZATION_OWNERSHIP_CHECK', 'OS_ALLOW_DEGRADED_TENANCY']) { process.env[key] = '1'; } - expect(isOrganizationOwnershipBootRefusal(await refusalOf(() => assertOrganizationOwnershipMigrated(engine())))).toBe(true); + expect(isOrganizationOwnershipBootRefusal(await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ stats: FOUND }))))).toBe(true); } finally { process.env = saved; } }); - - // 18.0 supports fresh installs only, and its `os migrate organization-ownership` - // refuses on every database, plan and apply alike. One pin per refusal branch: - // each closes on what 18.0 supports for a v17 database and names no step of that - // command. The code and the outcome (refused) are the ones the pins above hold. - describe('18.0: each branch says what 18.0 supports for a v17 database, and names no step of the withdrawn command', () => { - const unreadableMarker = () => ({ ...engine(), findOne: async () => { throw new Error('SQLITE_BUSY: database is locked'); } }); - - it('⭐ the v17 shape (a table still carries the column), whatever the marker says', async () => { - v17Shape(); - const absent = await refusalOf(() => assertOrganizationOwnershipMigrated(engine())); - marker(null); - const unverified = await refusalOf(() => assertOrganizationOwnershipMigrated(engine())); - const unreadable = await refusalOf(() => assertOrganizationOwnershipMigrated(unreadableMarker())); - expect([absent, unverified, unreadable].map((e) => e.tables)).toEqual([ - ['sys_job', 'sys_metadata'], ['sys_job', 'sys_metadata'], ['sys_job', 'sys_metadata'], - ]); - for (const err of [absent, unverified, unreadable]) { - expect(err.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); - expectTail(err.message, `The server refuses to start rather than serve v17 data with v18 semantics. ${FRESH_INSTALLS_ONLY}`); - expect(err.message).not.toMatch(WITHDRAWN_COMMAND); - } - }); - - it('⭐ a ceremony that ran every step and did not verify (no column left): 18.0 cannot re-run it', async () => { - migrated(); - marker(null, 602); - const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine())); - expect(err.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); - expectTail(err.message, `rather than serve rows of an organization-owned object with no organization. ${FRESH_INSTALLS_ONLY}`); - expect(err.message).not.toMatch(WITHDRAWN_COMMAND); - }); - - it('⭐ a ceremony marker that could not be read (no column left): fix the read first', async () => { - migrated(); - const err = await refusalOf(() => assertOrganizationOwnershipMigrated(unreadableMarker())); - expect(err.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); - expect(err.message).toMatch(/its marker \(sys_migration 'adr-0131-organization-ownership'\) could not be read/); - expectTail(err.message, `Fix the read, then start again. ${FRESH_INSTALLS_ONLY}`); - expect(err.message).not.toMatch(WITHDRAWN_COMMAND); - }); - - it('⭐ a column read that failed: fix the read first', async () => { - v17Shape(); - const err = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ failRead: true }))); - expect(err.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); - expect(err.tables).toEqual([]); - expect(err.message).toContain('(SQLITE_BUSY: database is locked)'); - expectTail(err.message, `Fix the read, then start again. ${FRESH_INSTALLS_ONLY}`); - expect(err.message).not.toMatch(WITHDRAWN_COMMAND); - }); - - it('a development boot says the same of a development database in every branch, and names no --dev command', async () => { - const dev = { dev: true }; - v17Shape(); - const shape = await refusalOf(() => assertOrganizationOwnershipMigrated(engine(), undefined, dev)); - const columnRead = await refusalOf(() => assertOrganizationOwnershipMigrated(engine({ failRead: true }), undefined, dev)); - db.close(); - db = new DatabaseSync(':memory:'); - migrated(); - const markerRead = await refusalOf(() => assertOrganizationOwnershipMigrated(unreadableMarker(), undefined, dev)); - marker(null, 602); - const rerun = await refusalOf(() => assertOrganizationOwnershipMigrated(engine(), undefined, dev)); - for (const err of [shape, columnRead, markerRead, rerun]) { - expect(err.code).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); - expectTail(err.message, FRESH_INSTALLS_ONLY_DEV); - expect(err.message).not.toMatch(WITHDRAWN_COMMAND); - expect(err.message).not.toContain('--dev'); - } - }); - }); }); diff --git a/packages/runtime/src/organization-ownership-boot-guard.ts b/packages/runtime/src/organization-ownership-boot-guard.ts index 65f1c9e290c..7f298a87f59 100644 --- a/packages/runtime/src/organization-ownership-boot-guard.ts +++ b/packages/runtime/src/organization-ownership-boot-guard.ts @@ -3,10 +3,14 @@ import type { Plugin, PluginContext } from '@objectstack/core'; import { isDataMigrationFlagVerified, + organizationOwnershipShapeGenerationOf, DATA_MIGRATION_FLAG_OBJECT, ORGANIZATION_OWNERSHIP_MIGRATION_ID, + ORGANIZATION_OWNERSHIP_SHAPE_GENERATION, type DataMigrationFlag, + type OrganizationOwnershipBirthMarker, } from '@objectstack/spec/system'; +import { resolveRuntimeVersion } from './runtime-version.js'; /** * OrganizationOwnershipBootGuardPlugin — the v18 boot refusal (ADR-0131 D10 @@ -17,41 +21,47 @@ import { * objects and the `sys_metadata` family no longer declare still hold their * legacy rows, and nothing reads them. The ceremony that carries those rows, * `os migrate organization-ownership --apply`, is an operator's act, never a - * boot step (D10). So the runtime refuses to start until it has run. In 18.0 - * that command refuses on every database (18.0 supports fresh installs only, - * ruling record 6259608309 on the v18 epic, item 1), so the refusal names no - * step of it: it says what 18.0 supports for a v17 database - * ({@link freshInstallsOnly}). + * boot step (D10). So the runtime refuses to start until it has run, and says + * which command to run. * * ## What is refused * - * Two questions, in order: + * The guard reads the ceremony's `sys_migration` row + * ({@link ORGANIZATION_OWNERSHIP_MIGRATION_ID}) and decides on ONE field of + * it: the ownership shape generation it attests + * (`organizationOwnershipShapeGenerationOf`), against the one this release + * requires ({@link ORGANIZATION_OWNERSHIP_SHAPE_GENERATION}; ruling record + * 6269955919 on the v18 epic, ① A and item 2). It never compares versions. * - * 1. Does a table whose object declares NO organization column - * (`systemFields.tenant: false` and no declared `organization_id`) still - * carry a PHYSICAL `organization_id`? Schema sync only adds, so on a - * database a v17 runtime created the answer is yes until the ceremony's - * column-drops step has run. If it is, the guard reads the ceremony's - * `sys_migration` marker ({@link ORGANIZATION_OWNERSHIP_MIGRATION_ID}); - * unless that marker is verified, it throws - * {@link OrganizationOwnershipBootRefusal}. - * 2. With no such table left: does the marker exist and say the ceremony has - * not finished? A ceremony whose every step ran (the columns are gone) but - * whose post-check found discrepancies records `applied_at` and leaves - * `verified_at` NULL — rows of an organization-owned object still without - * their organization. D10 item 5 refuses a database the ceremony has not - * COMPLETED, and `verified_at` is the ceremony's completion marker, so this - * database is refused too. Without a marker there is nothing to judge: a - * fresh database is never refused. + * 1. A VERIFIED row at the current generation — the ceremony's completion + * marker, or the birth marker below — boots, whatever the columns hold. + * 2. No row, on a datastore this boot observed being CREATED FROM EMPTY: the + * guard writes the BIRTH MARKER (verified, `details.birth` = + * {@link OrganizationOwnershipBirthMarker}) exactly once and boots. The + * observation is the store that holds the ledger — its driver's own schema + * sync stats — never the sum over every driver: a `telemetry` sibling left + * over from an earlier boot does not make a fresh primary look old, and a + * fresh sibling does not make an old primary look new. + * 3. No row on any other store: a database with no birth marker, which 17.x + * and the 18.0 pre-releases (18.0.0-next.*) created — refused until the + * ceremony has run on it. The tables still carrying a retired `organization_id` (an object that + * declares no organization column — `systemFields.tenant: false`, + * `systemFields: false` or `tenancy: { enabled: false }` — whose physical + * table still has one) are named when there are any. + * 4. A verified row BELOW the current generation — a database a shorter + * ceremony certified, or a marker written before generations existed — is + * refused, naming the ceremony again: its new steps have not run there. + * 5. A row the ceremony wrote and did not verify (its post-check found rows it + * has not carried) is refused, naming the re-run; so is a row that could + * not be read. * - * - A FRESH database is never refused: no table it creates carries a retired - * column, and no ceremony wrote a marker on it. - * - A database whose ceremony verified boots, whatever its columns. * - A read that fails is refused, never passed: "I could not check" is not * "there is nothing to find" (the `readDataMigrationFlag` asymmetry). + * - A birth marker that cannot be written refuses the boot: a store left + * without one would be taken, at its next boot, for a database an earlier + * release created. * - ⛔ No escape hatch: no `OS_ALLOW_*` / `OS_SKIP_*` variable and no flag - * skips the check (D10 verbatim). A 17.x runtime is the way to keep 17.x - * semantics on an unmigrated database. + * skips the check (D10 verbatim). * * ## When it judges: in `start()`, before any plugin writes a row * @@ -81,10 +91,14 @@ import { * `MigrationRecoveryPlugin`. ⛔ Never in `createStandaloneStack`'s own list: * every CLI data and schema boot shares that builder, the ceremony's own boot * among them, and a guard there would refuse the one command that clears it. + * That builder composes the birth half alone + * ({@link OrganizationOwnershipBirthMarkerPlugin}), which writes the marker on + * a store it created from empty and never refuses. * - * A driver with no physical columns to read (memory, MongoDB) has no v17 - * shape to find: the guard is inert there, as the recovery plugin is on a - * kernel with no journal. + * A store with no physical columns to read (memory, MongoDB), or a + * composition with no ledger or no store serving it, has nothing to judge: + * the guard is inert there, as the recovery plugin is on a kernel with no + * journal. */ /** The machine code a host discriminates this refusal by, across module instances. */ @@ -103,27 +117,19 @@ export const ORGANIZATION_OWNERSHIP_BOOT_GUARD_PLUGIN_NAME = 'com.objectstack.or const ORG = 'organization_id'; /** - * What 18.0 supports for a v17 database, the sentence every refusal here - * closes on. 18.0 supports fresh installs only, and its `os migrate - * organization-ownership` refuses on every database, plan and apply alike - * (`ORGANIZATION_OWNERSHIP_UPGRADE_UNAVAILABLE` in `@objectstack/cli`; ruling - * record 6259608309 on the v18 epic, item 1). So no refusal here names a step - * of that command: each says this instead, in the words the command's other - * 18.0 refusals use (`os migrate apply`'s drop gate, and `os dev`'s way on for - * a development database). The 18.x release that registers the ceremony names - * its steps here again. + * The ceremony command the refusal names, spelled for the boot that refused: + * a development boot (`os dev`, `os serve --dev`) composes its security + * plugin, `.env*` files and `telemetry` sibling as a development boot, and the + * ceremony has to boot the same way to see the same database, so the command + * it names carries `--dev`. */ -function freshInstallsOnly(dev: boolean | undefined): string { - return ( - '18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 ' + - `${dev ? 'development ' : ''}database on a 17.x release until the 18.x release that carries the upgrade path, ` + - 'and upgrade it there.' - ); +function ceremonyCommand(dev: boolean | undefined): string { + return dev ? 'os migrate organization-ownership --dev' : 'os migrate organization-ownership'; } -/** How the boot that runs the guard was started — what the refusal's words must match. */ +/** How the boot that runs the guard was started — what the refusal's command must match. */ export interface OrganizationOwnershipBootGuardOptions { - /** A development boot (`os dev` / `os serve --dev`): the refusal speaks of a development database. */ + /** A development boot (`os dev` / `os serve --dev`): the named ceremony command carries `--dev`. */ dev?: boolean; } @@ -192,32 +198,57 @@ function columnNameOf(row: Record): string | null { return typeof v === 'string' ? v : null; } +interface GuardDriver { + dialectName?: unknown; + config?: { client?: unknown }; + execute?: (sql: string, params?: unknown[]) => Promise; + getSchemaSyncStats?: () => { created: number; existing: number } | undefined; +} + interface GuardEngine { registry?: { getAllObjects?: () => unknown[] }; getDriverForObject?: (object: string) => unknown; findOne?: (object: string, query: Record) => Promise; + insert?: (object: string, data: Record, options?: Record) => Promise; getObject?: (name: string) => unknown; + wasDatastoreCreatedFromEmpty?: () => boolean; + invalidateDataMigrationFlags?: () => void; +} + +/** + * Whether an object declares that it carries no organization column: every + * spelling the injected-column resolver honours for an object of its own — + * `systemFields: false`, `systemFields.tenant: false`, and + * `tenancy: { enabled: false }` (`injected-system-columns.ts`). + */ +function declaresNoOrganizationColumn(object: { + systemFields?: unknown; + tenancy?: { enabled?: unknown }; +}): boolean { + if (object.systemFields === false) return true; + if ((object.systemFields as { tenant?: unknown } | undefined)?.tenant === false) return true; + return object.tenancy?.enabled === false; } /** * The tables in the v17 shape: an object that declares no organization column - * whose physical table still carries one. Only SQL drivers are read; any other - * driver is skipped. Throws when a column read fails. + * ({@link declaresNoOrganizationColumn}) whose physical table still carries + * one. Only SQL drivers are read; any other driver is skipped. Throws when a + * column read fails. */ export async function findV17ShapeTables(engine: GuardEngine): Promise { const objects = (engine.registry?.getAllObjects?.() ?? []) as Array<{ name?: string; - systemFields?: { tenant?: unknown }; + systemFields?: unknown; + tenancy?: { enabled?: unknown }; fields?: Record; }>; const found: string[] = []; for (const object of objects) { const name = object?.name; - if (typeof name !== 'string' || object.systemFields?.tenant !== false) continue; + if (typeof name !== 'string' || !declaresNoOrganizationColumn(object)) continue; if (object.fields && Object.prototype.hasOwnProperty.call(object.fields, ORG)) continue; - const driver = engine.getDriverForObject?.(name) as - | { dialectName?: unknown; config?: { client?: unknown }; execute?: (sql: string, params?: unknown[]) => Promise } - | undefined; + const driver = engine.getDriverForObject?.(name) as GuardDriver | undefined; const dialect = dialectOfDriver(driver); if (!dialect || typeof driver?.execute !== 'function') continue; const columns = rowsOf(await driver.execute(columnListSql(dialect), [name])) @@ -228,41 +259,56 @@ export async function findV17ShapeTables(engine: GuardEngine): Promise return found.sort(); } +/** The guard's reading of the ceremony's `sys_migration` row (see {@link readMarker}). */ +export type OrganizationOwnershipMarkerState = 'absent' | 'unverified' | 'unreadable' | 'outdated'; + /** - * The refusal text: what was found, that the server refuses to start, and what - * 18.0 supports for a v17 database ({@link freshInstallsOnly}). On a - * v17-shaped database (`tables` not empty) that is the whole way on. With no - * table left in the v17 shape (`tables` empty) a ceremony ran every step and - * its marker is not verified, or the marker could not be read: an unread - * marker says to fix the read first, since a database no ceremony ran on boots - * once the read answers. + * The refusal text: what was found, that the server refuses to start, and the + * command. It names no release line to stay on: a database an 18.0 + * pre-release (18.0.0-next.*) created is refused here exactly as a v17 one is, + * and both take the same ceremony. */ export function organizationOwnershipRefusalMessage( tables: readonly string[], - marker: 'absent' | 'unverified' | 'unreadable', - opts: OrganizationOwnershipBootGuardOptions = {}, + marker: OrganizationOwnershipMarkerState, + opts: OrganizationOwnershipBootGuardOptions & { generation?: number } = {}, ): string { - const answer = freshInstallsOnly(opts.dev); - if (tables.length === 0) { - const state = marker === 'unreadable' - ? `its marker (sys_migration '${ORGANIZATION_OWNERSHIP_MIGRATION_ID}') could not be read` - : `its marker (sys_migration '${ORGANIZATION_OWNERSHIP_MIGRATION_ID}') is not verified: the post-check ` + - 'still found rows the ceremony has not carried'; + const ceremony = ceremonyCommand(opts.dev); + const run = + `Run '${ceremony} --out PLAN' to write and review its plan, take a backup, then run ` + + `'${ceremony} --apply --plan-file PLAN --backup-taken', and start again.`; + const columns = tables.length > 0 + ? ` ${tables.join(', ')} still carr${tables.length === 1 ? 'ies' : 'y'} a physical organization_id column its ` + + 'object no longer declares.' + : ''; + if (marker === 'outdated') { return ( - `The organization-ownership ceremony has not finished on this database: ${state}. The server refuses to start ` + - 'rather than serve rows of an organization-owned object with no organization. ' + - `${marker === 'unreadable' ? 'Fix the read, then start again. ' : ''}${answer}` + `This database was certified by an organization-ownership ceremony of shape generation ${opts.generation ?? 0}, ` + + `and this release requires generation ${ORGANIZATION_OWNERSHIP_SHAPE_GENERATION}: the ceremony steps added since ` + + `have not run on it.${columns} The server refuses to start rather than serve it in a shape it was never ` + + `migrated to. ${run}` + ); + } + if (marker === 'absent') { + return ( + 'This database carries no organization-ownership marker: 17.x or an 18.0 pre-release (18.0.0-next.*) ' + + 'created it, and this release serves only a database it created itself or one its ownership ceremony upgraded.' + + `${columns} The server refuses to start rather than serve it in a shape it was never migrated to. ${run}` + ); + } + if (marker === 'unreadable') { + return ( + `The organization-ownership marker (sys_migration '${ORGANIZATION_OWNERSHIP_MIGRATION_ID}') of this database ` + + `could not be read, and an unread answer is never taken for a migrated database.${columns} The server refuses ` + + `to start. Fix the read, or run the ceremony: ${run}` ); } - const state = marker === 'absent' - ? 'the ceremony has not run on it' - : marker === 'unverified' - ? 'the ceremony has not finished on it (its marker is not verified)' - : 'its marker could not be read'; return ( - `This database is still in the v17 organization-ownership shape: ${tables.join(', ')} still carr` + - `${tables.length === 1 ? 'ies' : 'y'} a physical organization_id column, and ${state}. The server refuses to ` + - `start rather than serve v17 data with v18 semantics. ${answer}` + `The organization-ownership ceremony has not finished on this database: its marker (sys_migration ` + + `'${ORGANIZATION_OWNERSHIP_MIGRATION_ID}') is not verified, so the post-check still found rows the ceremony has ` + + `not carried.${columns} The server refuses to start rather than serve rows of an organization-owned object with ` + + `no organization. Resolve what the apply report names, then run '${ceremony} --out PLAN' again, review it, run ` + + `'${ceremony} --apply --plan-file PLAN --backup-taken', and start again.` ); } @@ -300,68 +346,234 @@ export class OrganizationOwnershipBootGuardPlugin implements Plugin { } } +/** What the guard read of the ceremony's `sys_migration` row. */ +type MarkerReading = + /** No ledger on this composition, or no SQL store serving it: nothing to judge. */ + | { state: 'inert' } + | { state: 'absent' } + | { state: 'unreadable' } + | { state: 'unverified'; generation: number } + | { state: 'verified'; generation: number }; + /** - * The ceremony's marker: `absent` (no ledger on this composition, no store - * serving it, or no row), `verified`, `unverified`, or `unreadable` (a store - * serves the ledger and its read threw). + * [Ruling record 6269955919, ① A] The birth half alone, for a boot that + * composes no guard: every CLI data and schema boot (`createStandaloneStack`), + * which can be the boot that creates a fresh install's store (`os migrate + * apply`, `os seed` before the first `os serve`). It writes the birth marker + * under exactly the guard's conditions — a ledger served by a SQL store this + * boot created from empty, no marker on it, no table in an earlier shape — and + * never refuses: judging the database stays the served boot's. Without it, the + * first `os serve` of an install whose store a CLI boot created would find no + * marker on a store it did not create, and refuse a fresh install. + * + * The ceremony's own boots never trip it: they hold their DDL back + * (`deferSchemaDdl`) or skip schema sync, so their store reports nothing + * created. */ -async function readMarker(engine: GuardEngine): Promise<'absent' | 'verified' | 'unverified' | 'unreadable'> { - if (typeof engine.findOne !== 'function') return 'absent'; - // A composition that registers no ledger object has no marker to hold. - if (typeof engine.getObject === 'function' && !engine.getObject(DATA_MIGRATION_FLAG_OBJECT)) return 'absent'; - // A boot with no storage (no driver serves the ledger — `os serve` with no - // OS_DATABASE_URL and no driver plugin) has no database to judge, exactly as - // the column probe skips an object no driver serves: nothing was ever - // migrated there. Only a store that serves the ledger and fails its read is - // `unreadable`. - if (typeof engine.getDriverForObject === 'function' && !engine.getDriverForObject(DATA_MIGRATION_FLAG_OBJECT)) return 'absent'; +export class OrganizationOwnershipBirthMarkerPlugin implements Plugin { + readonly name = ORGANIZATION_OWNERSHIP_BIRTH_MARKER_PLUGIN_NAME; + readonly type = 'standard'; + readonly version = '1.0.0'; + readonly dependencies: string[] = []; + /** As the guard: after the engine's schema sync and the ledger's registration. */ + readonly optionalDependencies: string[] = ['com.objectstack.engine.objectql', 'com.objectstack.platform-objects']; + + async init(_ctx: PluginContext): Promise {} + + async start(ctx: PluginContext): Promise { + let engine: GuardEngine | undefined; + try { + engine = (ctx as any).getService?.('objectql'); + } catch { + return; + } + if (!engine || typeof engine.getDriverForObject !== 'function') return; + const logger = (ctx as any)?.logger as { info?: (msg: string) => void; error?: (msg: string) => void } | undefined; + try { + await recordOrganizationOwnershipBirth(engine, logger); + } catch (error) { + // Durability: the store keeps looking healthy, and its first served boot + // will refuse it as one an earlier release created. + logger?.error?.( + `[organization-ownership] this store was created from empty and its birth marker (sys_migration ` + + `'${ORGANIZATION_OWNERSHIP_MIGRATION_ID}') could not be written ` + + `(${error instanceof Error ? error.message : String(error)}): the first 'os serve' will refuse it as a ` + + `database an earlier release created. Fix the write and run this command again, or run 'os migrate ` + + `organization-ownership' on it.`, + ); + } + } +} + +/** The birth-marker plugin's kernel name. */ +export const ORGANIZATION_OWNERSHIP_BIRTH_MARKER_PLUGIN_NAME = 'com.objectstack.organization-ownership-birth-marker' as const; + +/** + * Write the birth marker when, and only when, this boot created the ledger's + * store from empty and no marker and no table in an earlier shape is there. + * The marker written, or `null` when none was owed. Throws when the write fails. + */ +export async function recordOrganizationOwnershipBirth( + engine: GuardEngine, + logger?: { info?: (msg: string) => void }, + now: () => Date = () => new Date(), +): Promise { + if (!ledgerStoreCreatedFromEmpty(engine)) return null; + if ((await readMarker(engine)).state !== 'absent') return null; + if ((await findV17ShapeTables(engine)).length > 0) return null; + const birth = await writeBirthMarker(engine, now()); + logger?.info?.( + `[organization-ownership] a datastore created from empty: birth marker written (shape generation ` + + `${birth.shapeGeneration}, platform ${birth.platformVersion}).`, + ); + return birth; +} + +/** + * The ceremony's marker. `inert` when this composition registers no ledger + * object, no store serves it (`os serve` with no OS_DATABASE_URL and no driver + * plugin), or the store that does has no physical columns to read (memory, + * MongoDB) — nothing was ever migrated there. `unreadable` only when a store + * serves the ledger and its read threw. + */ +async function readMarker(engine: GuardEngine): Promise { + if (typeof engine.findOne !== 'function') return { state: 'inert' }; + if (typeof engine.getObject === 'function' && !engine.getObject(DATA_MIGRATION_FLAG_OBJECT)) return { state: 'inert' }; + const store = ledgerStore(engine); + if (!store || !dialectOfDriver(store)) return { state: 'inert' }; try { const row = (await engine.findOne(DATA_MIGRATION_FLAG_OBJECT, { where: { id: ORGANIZATION_OWNERSHIP_MIGRATION_ID }, context: { isSystem: true }, })) as DataMigrationFlag | null; - if (!row) return 'absent'; + if (!row) return { state: 'absent' }; const flag = { ...row, blocking: Number((row as { blocking?: unknown }).blocking) } as DataMigrationFlag; - return isDataMigrationFlagVerified(flag) ? 'verified' : 'unverified'; + const generation = organizationOwnershipShapeGenerationOf(flag.details); + return isDataMigrationFlagVerified(flag) ? { state: 'verified', generation } : { state: 'unverified', generation }; } catch { - return 'unreadable'; + return { state: 'unreadable' }; + } +} + +/** The driver that serves the ledger, or `undefined`. */ +function ledgerStore(engine: GuardEngine): GuardDriver | undefined { + if (typeof engine.getDriverForObject !== 'function') return undefined; + try { + return (engine.getDriverForObject(DATA_MIGRATION_FLAG_OBJECT) as GuardDriver | undefined) ?? undefined; + } catch { + return undefined; } } +/** + * Did THIS boot observe the store that holds the ledger being created from + * empty? Its own driver's schema-sync stats (`getSchemaSyncStats`): every + * table it synced it created, none it found. A driver that cannot report + * answers through the engine's all-driver reading, and "cannot say" is no. + */ +function ledgerStoreCreatedFromEmpty(engine: GuardEngine): boolean { + const stats = ledgerStore(engine)?.getSchemaSyncStats?.(); + if (stats) return (Number(stats.existing) || 0) === 0 && (Number(stats.created) || 0) > 0; + try { + return engine.wasDatastoreCreatedFromEmpty?.() === true; + } catch { + return false; + } +} + +/** + * Write the birth marker: the ceremony's row, verified, with `details.birth` + * naming the generation the store was created in. Only where no row exists. + */ +async function writeBirthMarker(engine: GuardEngine, now: Date): Promise { + if (typeof engine.insert !== 'function') throw new Error('the engine offers no insert'); + const at = now.toISOString(); + // `commit` stays absent: no build of this runtime records its source commit. + const birth: OrganizationOwnershipBirthMarker = { + shapeGeneration: ORGANIZATION_OWNERSHIP_SHAPE_GENERATION, + platformVersion: resolveRuntimeVersion(), + createdAt: at, + }; + await engine.insert( + DATA_MIGRATION_FLAG_OBJECT, + { + id: ORGANIZATION_OWNERSHIP_MIGRATION_ID, + last_run_at: at, + verified_at: at, + applied_at: null, + blocking: 0, + advisory: 0, + details: JSON.stringify({ birth }), + created_at: at, + updated_at: at, + }, + { context: { isSystem: true } }, + ); + engine.invalidateDataMigrationFlags?.(); + return birth; +} + /** * The check itself, for the plugin and for a host that composes its own boot: - * resolves when the database may be served, throws - * {@link OrganizationOwnershipBootRefusal} when it may not. + * resolves when the database may be served (writing the birth marker on a + * store created from empty), throws {@link OrganizationOwnershipBootRefusal} + * when it may not. */ export async function assertOrganizationOwnershipMigrated( engine: GuardEngine, logger?: { info?: (msg: string) => void }, - opts: OrganizationOwnershipBootGuardOptions = {}, + opts: OrganizationOwnershipBootGuardOptions & { now?: () => Date } = {}, ): Promise { + const ceremony = ceremonyCommand(opts.dev); let tables: string[]; try { tables = await findV17ShapeTables(engine); } catch (error) { throw new OrganizationOwnershipBootRefusal( - `The server refuses to start: it could not read whether this database is still in the v17 organization-ownership ` + - `shape (${error instanceof Error ? error.message : String(error)}), and an unread answer is never taken for a ` + - `migrated database. Fix the read, then start again. ${freshInstallsOnly(opts.dev)}`, + `The server refuses to start: it could not read whether this database is still in an earlier organization-` + + `ownership shape (${error instanceof Error ? error.message : String(error)}), and an unread answer is never ` + + `taken for a migrated database. Fix the read, or run the ceremony ('${ceremony} --out PLAN', then ` + + `'${ceremony} --apply --plan-file PLAN --backup-taken'), then start again.`, [], ); } const marker = await readMarker(engine); - if (marker === 'verified') { + if (marker.state === 'inert') return; + if (marker.state === 'verified' && marker.generation >= ORGANIZATION_OWNERSHIP_SHAPE_GENERATION) { if (tables.length > 0) { logger?.info?.( `[organization-ownership] ${tables.join(', ')} still carry a physical organization_id column, and the ` + - `ceremony's marker is verified: the server starts.`, + `marker is verified at shape generation ${marker.generation}: the server starts.`, + ); + } + return; + } + if (marker.state === 'absent' && tables.length === 0 && ledgerStoreCreatedFromEmpty(engine)) { + let birth: OrganizationOwnershipBirthMarker; + try { + birth = await writeBirthMarker(engine, (opts.now ?? (() => new Date()))()); + } catch (error) { + throw new OrganizationOwnershipBootRefusal( + `The server refuses to start: this datastore was created from empty, and its organization-ownership birth ` + + `marker (sys_migration '${ORGANIZATION_OWNERSHIP_MIGRATION_ID}') could not be written ` + + `(${error instanceof Error ? error.message : String(error)}). Without it the next boot would take this ` + + `store for a database an earlier release created. Fix the write and start again, or run the ceremony ` + + `('${ceremony} --out PLAN', then '${ceremony} --apply --plan-file PLAN --backup-taken').`, + [], ); } + logger?.info?.( + `[organization-ownership] a datastore created from empty: birth marker written (shape generation ` + + `${birth.shapeGeneration}, platform ${birth.platformVersion}).`, + ); return; } - // No table left in the v17 shape: only a marker the ceremony wrote and did - // not verify refuses (its post-check found rows it has not carried). No - // marker is a database no ceremony ran on — a fresh one — and it boots. - if (tables.length === 0 && marker === 'absent') return; - throw new OrganizationOwnershipBootRefusal(organizationOwnershipRefusalMessage(tables, marker, opts), tables); + const state: OrganizationOwnershipMarkerState = marker.state === 'verified' ? 'outdated' : marker.state; + throw new OrganizationOwnershipBootRefusal( + organizationOwnershipRefusalMessage(tables, state, { + ...opts, + ...('generation' in marker ? { generation: marker.generation } : {}), + }), + tables, + ); } diff --git a/packages/runtime/src/organization-ownership-boot-guard.turso-remote.kernel.test.ts b/packages/runtime/src/organization-ownership-boot-guard.turso-remote.kernel.test.ts new file mode 100644 index 00000000000..287044bfaed --- /dev/null +++ b/packages/runtime/src/organization-ownership-boot-guard.turso-remote.kernel.test.ts @@ -0,0 +1,118 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [ADR-0131 D10 item 5; ruling record 6269955919, ① A] The boot guard on the + * Turso REMOTE face (`libsql://`), on a real kernel: the birth marker needs the + * ledger's store to read as created from empty, and the remote face answers + * that from what its own schema doors counted + * (`RemoteTransport.schemaSyncStats`) — so + * + * 1. ⭐ R1: a fresh remote database boots, the guard writing its birth marker, + * and the next boot over it boots on that marker; + * 2. ⭐ a remote database an earlier boot created without the guard (an 18.0 + * pre-release install's: tables present, no marker) is refused, naming the ceremony. + * + * The remote transport talks to a SQLite file through `@libsql/client`, the + * double the driver's own remote-face pins use (`turso-remote-*.test.ts`). + */ + +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { DatabaseSync } from 'node:sqlite'; +import { afterEach, describe, expect, it } from 'vitest'; +import { createClient, type Client } from '@libsql/client'; +import { ObjectKernel } from '@objectstack/core'; +import { TursoDriver } from '@objectstack/driver-turso'; +import { ObjectQLPlugin } from '@objectstack/objectql'; +import { PlatformObjectsPlugin } from '@objectstack/platform-objects/plugin'; +import { ORGANIZATION_OWNERSHIP_SHAPE_GENERATION } from '@objectstack/spec/system'; +import { DriverPlugin } from './driver-plugin.js'; +import { + isOrganizationOwnershipBootRefusal, + OrganizationOwnershipBootGuardPlugin, + ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE, +} from './organization-ownership-boot-guard.js'; + +const MARKER = 'adr-0131-organization-ownership'; +const dirs: string[] = []; +const kernels: ObjectKernel[] = []; +const clients: Client[] = []; + +function databaseFile(): string { + const dir = mkdtempSync(join(tmpdir(), 'os-ownership-guard-remote-')); + dirs.push(dir); + return join(dir, 'remote.db'); +} + +function refusalIn(error: unknown): { code?: string; message?: string } | undefined { + for (let e = error, depth = 0; !!e && typeof e === 'object' && depth < 8; e = (e as { cause?: unknown }).cause, depth++) { + if (isOrganizationOwnershipBootRefusal(e)) return e as { code?: string; message?: string }; + } + return undefined; +} + +/** `os serve`'s data spine over a REMOTE Turso database (the transport's file double), the guard composed. */ +async function boot(file: string): Promise<{ kernel: ObjectKernel; error: unknown }> { + const kernel = new ObjectKernel({ logger: { level: 'silent' }, gracefulShutdown: false }); + kernels.push(kernel); + const client = createClient({ url: `file:${file}` }); + clients.push(client); + await kernel.use(new DriverPlugin(new TursoDriver({ url: 'libsql://ownership-guard.example.turso.io', client }))); + await kernel.use(new ObjectQLPlugin()); + await kernel.use(new OrganizationOwnershipBootGuardPlugin()); + await kernel.use(new PlatformObjectsPlugin() as never); + try { + await kernel.bootstrap(); + return { kernel, error: null }; + } catch (error) { + return { kernel, error }; + } +} + +const markerRows = (file: string): Array<{ verified_at: string | null; details: string }> => { + const db = new DatabaseSync(file, { readOnly: true }); + try { + return db.prepare('SELECT verified_at, details FROM sys_migration WHERE id = ?').all(MARKER) as never; + } finally { + db.close(); + } +}; + +afterEach(async () => { + for (const kernel of kernels.splice(0)) { + if (kernel.getState() === 'running') await kernel.shutdown(); + } + for (const client of clients.splice(0)) client.close(); + for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); +}); + +describe('[ADR-0131 D10 item 5] the boot guard on the Turso remote face', () => { + it('⭐ R1: a fresh remote database boots, the guard writing its birth marker; the next boot reads it and boots', async () => { + const file = databaseFile(); + const first = await boot(file); + expect(first.error, String((first.error as Error | null)?.message)).toBeNull(); + expect(first.kernel.getState()).toBe('running'); + const [row] = markerRows(file); + expect(row?.verified_at).toBeTruthy(); + expect(JSON.parse(String(row!.details)).birth).toMatchObject({ shapeGeneration: ORGANIZATION_OWNERSHIP_SHAPE_GENERATION }); + await first.kernel.shutdown(); + + const second = await boot(file); + expect(second.error, String((second.error as Error | null)?.message)).toBeNull(); + expect(markerRows(file)).toHaveLength(1); + }, 60_000); + + it('⭐ a remote database an earlier boot created without the guard (an 18.0 pre-release install: no marker) is refused, naming the ceremony', async () => { + const file = databaseFile(); + const db = new DatabaseSync(file); + db.exec('CREATE TABLE sys_migration (id TEXT PRIMARY KEY, last_run_at TEXT, verified_at TEXT, applied_at TEXT, blocking INTEGER)'); + db.close(); + const { kernel, error } = await boot(file); + const refusal = refusalIn(error); + expect(refusal?.code, String(error)).toBe(ORGANIZATION_OWNERSHIP_BOOT_REFUSAL_CODE); + expect(refusal?.message).toMatch(/^This database carries no organization-ownership marker/); + expect(kernel.getState()).not.toBe('running'); + expect(markerRows(file)).toEqual([]); + }, 60_000); +}); diff --git a/packages/runtime/src/seed-loader-null-organization-holder-real-driver.integration.test.ts b/packages/runtime/src/seed-loader-null-organization-holder-real-driver.integration.test.ts index b8f361b1c77..6dd3470bef2 100644 --- a/packages/runtime/src/seed-loader-null-organization-holder-real-driver.integration.test.ts +++ b/packages/runtime/src/seed-loader-null-organization-holder-real-driver.integration.test.ts @@ -209,12 +209,11 @@ describe('scoped seed load over NULL-organization holders (sqlite-file)', () => expect(kickoff.targetObject).toBe('tw_event'); expect(kickoff.targetField).toBe('subject'); expect(kickoff.attemptedValue).toBe('Kickoff'); - // The message names the key, the unowned holder and what 18.0 supports: - // never the ceremony 18.0 refuses. + // The message names the key, the unowned holder and the remedy's command. expect(kickoff.message).toContain('subject=Kickoff'); expect(kickoff.message).toContain("'ev_kickoff'"); - expect(kickoff.message).toContain('18.0 supports fresh installs only and does not upgrade a v17 database'); - expect(kickoff.message).not.toContain('os migrate organization-ownership'); + expect(kickoff.message).toContain('os migrate organization-ownership --out PLAN'); + expect(kickoff.message).not.toMatch(/fresh installs only|on a 17\.x release|carries the upgrade path/); expect(result.success).toBe(false); // Loud: said at `error`, once per dataset. const lines = logger.error.mock.calls.map((c) => String(c[0])).filter((m) => refusals.some((e) => m.includes(e.message))); diff --git a/packages/runtime/src/standalone-stack.ts b/packages/runtime/src/standalone-stack.ts index 8b74041be49..d22ba210c4d 100644 --- a/packages/runtime/src/standalone-stack.ts +++ b/packages/runtime/src/standalone-stack.ts @@ -79,6 +79,7 @@ import { loadArtifactBundle, isHttpUrl } from './load-artifact-bundle.js'; import { makeDirectorySync } from './make-directory.js'; import { loadTursoDriverFactory } from './turso-driver-factory.js'; import { devAutoMigrateConfig } from './dev-auto-migrate.js'; +import { OrganizationOwnershipBirthMarkerPlugin } from './organization-ownership-boot-guard.js'; import { namesRetiredMemoryEngine, resolveProjectDatabaseUrl, @@ -1143,6 +1144,13 @@ export async function createStandaloneStack(config?: StandaloneStackConfig): Pro // hands the plugin exactly the options it always did. ...(cfg.armLifecycleSweep === false ? { lifecycle: { enabled: false } } : {}), }), + // [ADR-0131 D10; ruling record 6269955919, ① A] A store this boot + // creates from empty is born in the current ownership shape: its + // birth marker is written here too, so a fresh install whose first + // boot is a CLI one (`os migrate apply`, `os seed`) is not refused + // by its first `os serve`. It never refuses — the served boot's guard + // judges (⛔ the guard itself stays out of this list). + new OrganizationOwnershipBirthMarkerPlugin(), ]; if (artifactBundle) { plugins.push(new AppPlugin(artifactBundle, undefined, { diff --git a/packages/runtime/vitest.config.ts b/packages/runtime/vitest.config.ts index 85e81630e13..85e39ce7a1d 100644 --- a/packages/runtime/vitest.config.ts +++ b/packages/runtime/vitest.config.ts @@ -186,6 +186,10 @@ export default defineConfig({ replacement: path.join(path.resolve(__dirname, '..'), 'spec/src/$1/index.ts'), }, { find: /^@objectstack\/spec$/, replacement: path.resolve(__dirname, '../spec/src/index.ts') }, + // [ADR-0131 D10] `organization-ownership-boot-guard.turso-remote.kernel.test.ts` + // boots the guard over the Turso REMOTE face: the face's schema-sync count + // is the pin's subject, so the driver is the checkout's source. + { find: /^@objectstack\/driver-turso$/, replacement: path.resolve(__dirname, '../drivers/driver-turso/src/index.ts') }, // Subpath BEFORE the bare package, same prefix-match reason: `./node` is a // published subpath served by a FILE (`types/src/node.ts` — the node-only slice // the root export deliberately excludes), so the bare entry would resolve it to diff --git a/packages/services/service-automation/src/sys-flow-credential.object.ts b/packages/services/service-automation/src/sys-flow-credential.object.ts index daa0672b0b4..b912b59b1d5 100644 --- a/packages/services/service-automation/src/sys-flow-credential.object.ts +++ b/packages/services/service-automation/src/sys-flow-credential.object.ts @@ -47,6 +47,11 @@ export const SysFlowCredential = ObjectSchema.create({ icon: 'key', isSystem: true, managedBy: 'engine-owned', + // [ADR-0131 D7] Deployment-level: a credential is read by its flow under the + // system context, never by organization, so the store carries no + // organization column. The flip lands with the open upgrade ceremony (18.x + // step 6, line fact F2), whose column-drops step takes an upgraded database's column. + systemFields: { tenant: false }, // ADR-0066 secure-by-default: a credential store is not covered by the // wildcard grant. Everything reads and writes it through the engine under a // system context. diff --git a/packages/services/service-automation/src/walled-organization-less-bridge.test.ts b/packages/services/service-automation/src/walled-flow-credential-write.test.ts similarity index 60% rename from packages/services/service-automation/src/walled-organization-less-bridge.test.ts rename to packages/services/service-automation/src/walled-flow-credential-write.test.ts index ef799d2bbcd..bec4fc8890f 100644 --- a/packages/services/service-automation/src/walled-organization-less-bridge.test.ts +++ b/packages/services/service-automation/src/walled-flow-credential-write.test.ts @@ -1,29 +1,23 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. /** - * [ADR-0131 C8 stage 8b, BRIDGE] `sys_secret` and `sys_flow_credential` keep - * their organization-less system inserts under a walled posture until the - * steps 5+6 landing flips both tables column-less (epic ruling record - * 6270035914 on #15194, item 2; #24582). `WALLED_ORGANIZATION_LESS_BRIDGE` in - * `@objectstack/objectql` is that bridge. + * [ADR-0131 C8 stage 8b; #24582] `sys_secret` and `sys_flow_credential` are + * deployment-level: both declare `systemFields: { tenant: false }`, so neither + * carries an organization column and the engine's walled system-insert rule + * (`Engine.resolveSystemInsertOrganization`) has no tenant field to stamp or + * refuse on. Step 4 kept their organization-less inserts landing under a wall + * through a temporary bridge list; the flips retired it (epic ruling record + * 6270035914 on #15194, item 2). * - * Two things are pinned here: - * - * 1. **The tripwire.** The bridge names exactly those two tables, and each - * still carries its tenant column. The day either declares - * `systemFields: { tenant: false }` this goes red, and the remedy is to - * DELETE the bridge (the list, its reader in - * `Engine.resolveSystemInsertOrganization`) and this tripwire with it, in - * the PR that lands the flip. ⛔ Never extend the list to green it. - * 2. **The flow-credential writer under a wall**, on a real engine over - * SQLite: saving a flow that carries a credential writes its - * `sys_flow_credential` row and, through the engine's secret field, its - * `sys_secret` ciphertext — both organization-less, both landing under - * `isolated` and `group`. Without the bridge both are refused with - * `ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED`. + * Pinned here, on a real engine over SQLite: saving a flow that carries a + * credential writes its `sys_flow_credential` row and, through the engine's + * secret field, its `sys_secret` ciphertext — both organization-less, both + * landing under `isolated` and `group`, with no bridge. If either table took a + * tenant column back, both writes would be refused with + * `ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED`. */ import { afterEach, describe, expect, it } from 'vitest'; -import { ObjectQL, WALLED_ORGANIZATION_LESS_BRIDGE } from '@objectstack/objectql'; +import { ObjectQL } from '@objectstack/objectql'; import { SqlDriver } from '@objectstack/driver-sql'; import { SysSecret } from '@objectstack/platform-objects'; import type { CryptoContext, CryptoHandle, ICryptoProvider } from '@objectstack/spec/contracts'; @@ -89,25 +83,12 @@ function inbound() { }; } -describe('[C8 stage 8b] the walled organization-less bridge — the tripwire', () => { - it('names exactly sys_secret and sys_flow_credential, and each still carries its tenant column', () => { - expect([...WALLED_ORGANIZATION_LESS_BRIDGE].sort()).toEqual(['sys_flow_credential', 'sys_secret']); - for (const object of [SysSecret, SysFlowCredential] as any[]) { - expect( - object.systemFields?.tenant, - `${object.name} now declares systemFields.tenant: false — the steps 5+6 flip has landed, so DELETE ` - + 'WALLED_ORGANIZATION_LESS_BRIDGE, its reader in Engine.resolveSystemInsertOrganization and this tripwire', - ).not.toBe(false); - expect(object.tenancy?.enabled, `${object.name} declares tenancy.enabled: false`).not.toBe(false); - } - }); -}); - -describe('[C8 stage 8b] the walled organization-less bridge — saving a flow credential', () => { +describe('[C8 stage 8b] a flow credential saved under a wall', () => { it.each(['isolated', 'group'] as const)( '%s: the organization-less sys_flow_credential row and its sys_secret ciphertext land', async (posture) => { const ql = await boot(posture); + for (const object of [SysSecret, SysFlowCredential] as any[]) expect(object.systemFields?.tenant, object.name).toBe(false); await new FlowCredentialChannel(() => ql as never).store({ name: 'pin_bridge', state: 'active', body: inbound() }); const credentials = (await ql.find(FLOW_CREDENTIAL_OBJECT, { context: SYSTEM } as never)) as any[]; diff --git a/packages/services/service-datasource/src/datasource-secret-binder.walled.test.ts b/packages/services/service-datasource/src/datasource-secret-binder.walled.test.ts index 07d6c65aaf2..13ec0ebfed3 100644 --- a/packages/services/service-datasource/src/datasource-secret-binder.walled.test.ts +++ b/packages/services/service-datasource/src/datasource-secret-binder.walled.test.ts @@ -1,17 +1,18 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. /** - * [ADR-0131 C8 stage 8b, BRIDGE] Binding a datasource credential under a + * [ADR-0131 C8 stage 8b; #24582] Binding a datasource credential under a * walled posture, through a REAL ObjectQL engine over SQLite. * * The binder persists the ciphertext as a `sys_secret` row with the system * context and no organization: a datasource is deployment-level. `sys_secret` - * keeps its organization column until the steps 5+6 landing flips it (epic - * ruling record 6270035914 on #15194, item 2), so under `group` / `isolated` - * that organization-less system insert must keep landing: the walled - * organization-less bridge in `Engine.resolveSystemInsertOrganization` holds - * it. Without the bridge the admin door's credential bind is a 500 - * `ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED`, and nothing is written. + * declares no organization column (`systemFields: { tenant: false }`, the + * steps 5+6 flip; epic ruling record 6270035914 on #15194, item 2), so under + * `group` / `isolated` that organization-less system insert lands on the + * engine's own rule: with no tenant field, `Engine.resolveSystemInsertOrganization` + * has nothing to stamp or refuse. If the table took an organization column + * back, the admin door's credential bind would be a 500 + * `ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED`, and nothing would be written. */ import { afterEach, describe, expect, it } from 'vitest'; diff --git a/packages/services/service-settings/src/settings-global-secret-walled.test.ts b/packages/services/service-settings/src/settings-global-secret-walled.test.ts index efdaa92a0ea..bb0446bc213 100644 --- a/packages/services/service-settings/src/settings-global-secret-walled.test.ts +++ b/packages/services/service-settings/src/settings-global-secret-walled.test.ts @@ -1,18 +1,19 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. /** - * [ADR-0131 C8 stage 8b, BRIDGE] A global-scope encrypted setting under a + * [ADR-0131 C8 stage 8b; #24582] A global-scope encrypted setting under a * walled posture, through a REAL ObjectQL engine. * * A global value is deployment-level, so its `sys_secret` ciphertext names no * organization (`settings-secret-organization.test.ts` pins the row shape, - * over a fake engine). `sys_secret` keeps its organization column until the - * steps 5+6 landing flips it (epic ruling record 6270035914 on #15194, item - * 2), so under `group` / `isolated` that organization-less system insert must - * keep landing: the walled organization-less bridge in - * `Engine.resolveSystemInsertOrganization` holds it. Without the bridge the - * operator's global encrypted setting is a 500 - * `ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED`, and nothing is written. + * over a fake engine). `sys_secret` declares no organization column + * (`systemFields: { tenant: false }`, the steps 5+6 flip; epic ruling record + * 6270035914 on #15194, item 2), so under `group` / `isolated` that + * organization-less system insert lands on the engine's own rule: with no + * tenant field, `Engine.resolveSystemInsertOrganization` has nothing to stamp + * or refuse. If the table took an organization column back, the operator's + * global encrypted setting would be a 500 `ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED`, + * and nothing would be written. * * The writer is the production pair: `SettingsService` building the row and * the plugin's own secret store (`SettingsServicePlugin` → `buildSecretStore`) diff --git a/packages/services/service-settings/src/settings-secret-organization.test.ts b/packages/services/service-settings/src/settings-secret-organization.test.ts index 0f3f75cf5b2..6e4db84c608 100644 --- a/packages/services/service-settings/src/settings-secret-organization.test.ts +++ b/packages/services/service-settings/src/settings-secret-organization.test.ts @@ -1,15 +1,16 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. /** - * [ADR-0131 D3/D9] The two rows a settings write lands BESIDE its `sys_setting` - * row — the `sys_secret` ciphertext of an encrypted value and the per-key - * `sys_setting_audit` history row — belong to the organization the setting - * row does: the caller's, for a `tenant` or `user` value. A `global` value is - * deployment-level (its row lives in `sys_platform_setting`, which has no - * organization), so neither row names one. Its history row goes to - * `sys_platform_setting_audit`, the audit trail's deployment-level global rung - * (ADR-0131 D7, ruling record 6257510973 · Q4 A); the secret's fate is the - * 18.x ceremony's. + * [ADR-0131 D3/D7/D9] The two rows a settings write lands BESIDE its + * `sys_setting` row. The per-key `sys_setting_audit` history row belongs to the + * organization the setting row does: the caller's, for a `tenant` or `user` + * value. The `sys_secret` ciphertext of an encrypted value names no + * organization under any rung: the cipher store is deployment-level (18.x step + * 6, `systemFields.tenant: false`) and is read by its id, so the owning + * `sys_setting` row alone carries the organization. A `global` value is + * deployment-level throughout (its row lives in `sys_platform_setting`), and its + * history row goes to `sys_platform_setting_audit`, the audit trail's + * deployment-level global rung (ADR-0131 D7, ruling record 6257510973 · Q4 A). */ import { randomBytes } from 'node:crypto'; @@ -57,12 +58,13 @@ function boot() { } describe('[ADR-0131 D3/D9] settings secret and history rows name their setting\'s organization', () => { - it('a tenant value: the ciphertext and the history row carry the caller\'s organization', async () => { + it('a tenant value: the history row carries the caller\'s organization; the ciphertext names none', async () => { const { svc, secrets, history } = boot(); await svc.setMany('tenant_vault', { token: 's3cret' }, { tenantId: 'org-1', userId: 'usr-1' }); expect(secrets).toHaveLength(1); - expect(secrets[0]).toMatchObject({ namespace: 'tenant_vault', key: 'token', organization_id: 'org-1' }); + expect(secrets[0]).toMatchObject({ namespace: 'tenant_vault', key: 'token' }); + expect('organization_id' in secrets[0]).toBe(false); expect(history.filter((e) => e.key === 'token')).toEqual([ expect.objectContaining({ scope: 'tenant', encrypted: true, organizationId: 'org-1' }), ]); diff --git a/packages/services/service-settings/src/settings-service.ts b/packages/services/service-settings/src/settings-service.ts index cfeb5fe9fed..c17c8841450 100644 --- a/packages/services/service-settings/src/settings-service.ts +++ b/packages/services/service-settings/src/settings-service.ts @@ -2118,11 +2118,9 @@ export class SettingsService { alg: handle.alg, version: handle.version, ciphertext: handle.ciphertext, - // [ADR-0131 D3/D9] The ciphertext of a tenant or user value is - // that organization's, as its `sys_setting` row is. A global - // value's is deployment-level and names none (see - // `SettingsSecretStore.insert`). - ...(organizationId ? { organization_id: organizationId } : {}), + // [ADR-0131 D7] No organization: `sys_secret` is deployment-level + // (`systemFields.tenant: false`); the owning `sys_setting` row + // carries the organization, and the cipher is read by its id. }); storedEnc = handle.id; digest = await this.secretAuditDigest(namespace, key, plain); diff --git a/packages/services/service-settings/src/settings-service.types.ts b/packages/services/service-settings/src/settings-service.types.ts index ecb09446119..df76ac8461e 100644 --- a/packages/services/service-settings/src/settings-service.types.ts +++ b/packages/services/service-settings/src/settings-service.types.ts @@ -242,13 +242,6 @@ export interface SettingsSecretStore { alg: string; version: number; ciphertext: string; - /** - * [ADR-0131 D3/D9] The organization that owns the secret — the owning - * `sys_setting` row's, for a tenant or user value. Absent for a global - * value: a deployment-level secret, whose fate in this tenant-column - * store is not settled yet. - */ - organization_id?: string; }): Promise<{ id: string }>; /** Look up the latest ciphertext for a handle id; null when missing. */ get(id: string): Promise<{ diff --git a/packages/spec/api-surface/system.json b/packages/spec/api-surface/system.json index 2d42ed594a4..2540e5f18dd 100644 --- a/packages/spec/api-surface/system.json +++ b/packages/spec/api-surface/system.json @@ -447,6 +447,7 @@ "OPERATION_MESSAGE_FALLBACK_LOCALE (const)", "OPERATION_MESSAGE_KEY_PREFIX (const)", "ORGANIZATION_OWNERSHIP_MIGRATION_ID (const)", + "ORGANIZATION_OWNERSHIP_SHAPE_GENERATION (const)", "ORSet (type)", "ORSetElement (type)", "ORSetElementParsed (type)", @@ -481,6 +482,9 @@ "OpenTelemetryCompatibilitySchema (const)", "OrganizationOwnershipAnchor (type)", "OrganizationOwnershipAnchorSchema (const)", + "OrganizationOwnershipBirthMarker (type)", + "OrganizationOwnershipBirthMarkerParsed (type)", + "OrganizationOwnershipBirthMarkerSchema (const)", "OrganizationOwnershipCategory (type)", "OrganizationOwnershipCategorySchema (const)", "OrganizationOwnershipDeparture (type)", @@ -833,6 +837,7 @@ "objectLabelKey (function)", "objectValidationMessageKey (function)", "operationMessageTranslationKey (function)", + "organizationOwnershipShapeGenerationOf (function)", "preferredLocaleFromHeader (function)", "renderOperationMessage (function)", "renderValidationMessage (function)", diff --git a/packages/spec/authorable-surface/system.json b/packages/spec/authorable-surface/system.json index 3f1b7f05bfb..6569474eebe 100644 --- a/packages/spec/authorable-surface/system.json +++ b/packages/spec/authorable-surface/system.json @@ -865,6 +865,10 @@ "system/OpenTelemetryCompatibility:resource", "system/OpenTelemetryCompatibility:sdkVersion", "system/OpenTelemetryCompatibility:semanticConventionsVersion", + "system/OrganizationOwnershipBirthMarker:commit", + "system/OrganizationOwnershipBirthMarker:createdAt", + "system/OrganizationOwnershipBirthMarker:platformVersion", + "system/OrganizationOwnershipBirthMarker:shapeGeneration", "system/OrganizationOwnershipCategory:citation", "system/OrganizationOwnershipCategory:conflictBy", "system/OrganizationOwnershipCategory:id", diff --git a/packages/spec/declaration-map/system.json b/packages/spec/declaration-map/system.json index 748819da1b7..f1c6736d29c 100644 --- a/packages/spec/declaration-map/system.json +++ b/packages/spec/declaration-map/system.json @@ -319,6 +319,8 @@ "OpenTelemetryCompatibilitySchema": "system/OpenTelemetryCompatibility", "OrganizationOwnershipAnchor": "system/OrganizationOwnershipAnchor", "OrganizationOwnershipAnchorSchema": "system/OrganizationOwnershipAnchor", + "OrganizationOwnershipBirthMarker": "system/OrganizationOwnershipBirthMarker", + "OrganizationOwnershipBirthMarkerSchema": "system/OrganizationOwnershipBirthMarker", "OrganizationOwnershipCategory": "system/OrganizationOwnershipCategory", "OrganizationOwnershipCategorySchema": "system/OrganizationOwnershipCategory", "OrganizationOwnershipDeparture": "system/OrganizationOwnershipDeparture", diff --git a/packages/spec/export-origins/system.json b/packages/spec/export-origins/system.json index bb2cc677f2b..ffbde5c794f 100644 --- a/packages/spec/export-origins/system.json +++ b/packages/spec/export-origins/system.json @@ -428,6 +428,7 @@ "OPERATION_MESSAGE_FALLBACK_LOCALE": "src/system/operation-message.ts#OPERATION_MESSAGE_FALLBACK_LOCALE (const)", "OPERATION_MESSAGE_KEY_PREFIX": "src/system/operation-message.ts#OPERATION_MESSAGE_KEY_PREFIX (const)", "ORGANIZATION_OWNERSHIP_MIGRATION_ID": "src/system/migration.zod.ts#ORGANIZATION_OWNERSHIP_MIGRATION_ID (const)", + "ORGANIZATION_OWNERSHIP_SHAPE_GENERATION": "src/system/migration.zod.ts#ORGANIZATION_OWNERSHIP_SHAPE_GENERATION (const)", "ORSet": "src/system/collaboration.zod.ts#ORSet (type)", "ORSetElement": "src/system/collaboration.zod.ts#ORSetElement (type)", "ORSetElementParsed": "src/system/collaboration.zod.ts#ORSetElementParsed (type)", @@ -461,6 +462,9 @@ "OpenTelemetryCompatibilitySchema": "src/system/tracing.zod.ts#OpenTelemetryCompatibilitySchema (const)", "OrganizationOwnershipAnchor": "src/system/fate-supplement.zod.ts#OrganizationOwnershipAnchor (type)", "OrganizationOwnershipAnchorSchema": "src/system/fate-supplement.zod.ts#OrganizationOwnershipAnchorSchema (const)", + "OrganizationOwnershipBirthMarker": "src/system/migration.zod.ts#OrganizationOwnershipBirthMarker (type)", + "OrganizationOwnershipBirthMarkerParsed": "src/system/migration.zod.ts#OrganizationOwnershipBirthMarkerParsed (type)", + "OrganizationOwnershipBirthMarkerSchema": "src/system/migration.zod.ts#OrganizationOwnershipBirthMarkerSchema (const)", "OrganizationOwnershipCategory": "src/system/fate-supplement.zod.ts#OrganizationOwnershipCategory (type)", "OrganizationOwnershipCategorySchema": "src/system/fate-supplement.zod.ts#OrganizationOwnershipCategorySchema (const)", "OrganizationOwnershipDeparture": "src/system/fate-supplement.zod.ts#OrganizationOwnershipDeparture (type)", @@ -794,6 +798,7 @@ "objectLabelKey": "src/system/i18n-resolver.ts#objectLabelKey (function)", "objectValidationMessageKey": "src/system/i18n-resolver.ts#objectValidationMessageKey (function)", "operationMessageTranslationKey": "src/system/operation-message.ts#operationMessageTranslationKey (function)", + "organizationOwnershipShapeGenerationOf": "src/system/migration.zod.ts#organizationOwnershipShapeGenerationOf (function)", "preferredLocaleFromHeader": "src/system/i18n-resolver.ts#preferredLocaleFromHeader (function)", "renderOperationMessage": "src/system/operation-message.ts#renderOperationMessage (function)", "renderValidationMessage": "src/system/validation-message.ts#renderValidationMessage (function)", diff --git a/packages/spec/json-schema.manifest/system.json b/packages/spec/json-schema.manifest/system.json index 4efde58da7d..c5510b88b0a 100644 --- a/packages/spec/json-schema.manifest/system.json +++ b/packages/spec/json-schema.manifest/system.json @@ -174,6 +174,7 @@ "system/OnceSchedule", "system/OpenTelemetryCompatibility", "system/OrganizationOwnershipAnchor", + "system/OrganizationOwnershipBirthMarker", "system/OrganizationOwnershipCategory", "system/OrganizationOwnershipDeparture", "system/OrganizationOwnershipFate", diff --git a/packages/spec/src/migrations/entries/semantic/18.cli-organization-ownership-apply-plan-file-required.ts b/packages/spec/src/migrations/entries/semantic/18.cli-organization-ownership-apply-plan-file-required.ts index 82efb17de2c..e8731570810 100644 --- a/packages/spec/src/migrations/entries/semantic/18.cli-organization-ownership-apply-plan-file-required.ts +++ b/packages/spec/src/migrations/entries/semantic/18.cli-organization-ownership-apply-plan-file-required.ts @@ -10,29 +10,21 @@ export const entry: SemanticMigration = { 'os migrate organization-ownership --apply, invoked without --plan-file (the bare --apply --backup-taken that ' + '@objectstack/cli 18.0.0-next.1 shipped, with or without --fate-supplement)', replacement: - 'nothing in 18.0: 18.0 refuses this invocation, as it refuses every plan and apply of ' - + '`os migrate organization-ownership` on every database (exit 1, `NOT_IMPLEMENTED`, nothing read or written). ' - + '18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release ' - + 'until the 18.x release that carries the upgrade path. A script or runbook that applies directly does not run ' - + 'against 18.0 at all. In that 18.x release, run the plan first, `os migrate organization-ownership --out plan.json` ' - + '(with the same `--fate-supplement`, under the same tenancy posture, against the same database), review the file ' - + 'it writes, then `os migrate organization-ownership --apply --backup-taken --plan-file plan.json`: the script or ' - + 'runbook gains the plan step and the flag; nothing in a package source or a stored row changes.', + 'Run the plan first, `os migrate organization-ownership --out plan.json` (with the same `--fate-supplement`, ' + + 'under the same tenancy posture, against the same database), review the file it writes, then ' + + '`os migrate organization-ownership --apply --backup-taken --plan-file plan.json`. A script or runbook that ' + + 'applies directly gains the plan step and the flag; nothing in a package source or a stored row changes.', reason: 'ADR-0131 D10 makes the ceremony an operator\'s act on a plan the operator reviewed. The apply computes its ' + 'own plan at apply time, and that plan\'s inputs can move between the review and the run: a different host ' + 'fate supplement, another release\'s inventory, another tenancy posture or another database. Planning with ' + 'one supplement and applying with another ran the second one\'s fates unreviewed, including a mirror deletion ' - + 'the reviewed plan never listed. So the 18.x apply reads the plan file the operator reviewed and refuses, before ' + + 'the reviewed plan never listed. So the apply now reads the plan file the operator reviewed and refuses, before ' + 'anything is written, when its inventory digest, posture or database differs from the run\'s. Without the ' - + 'file there is nothing to compare, so the bare invocation is refused. 18.0 itself withholds the whole ' - + 'ceremony: it is knowingly incomplete in 18.0, and an incomplete ceremony must never certify a v17 database ' - + 'that the boot would then serve, so 18.0 refuses plan and apply alike before it reads anything.', + + 'file there is nothing to compare, so the bare invocation is refused.', acceptanceCriteria: - 'On 18.0, no script or runbook runs `os migrate organization-ownership`, and every v17 database stays on a 17.x ' - + 'release. In the 18.x release that carries the upgrade path, every invocation of ' - + '`os migrate organization-ownership --apply` passes `--plan-file` with the file a plan run wrote, and the run ' - + 'reports `reviewedPlan`. A run that answers `plan-file-required`, `plan-digest-mismatch`, ' + 'Every invocation of `os migrate organization-ownership --apply` passes `--plan-file` with the file a plan run ' + + 'wrote, and the run reports `reviewedPlan`. A run that answers `plan-file-required`, `plan-digest-mismatch`, ' + '`plan-posture-mismatch` or `plan-database-mismatch` has written nothing: plan again, review, and pass the new ' + 'file.', }; diff --git a/packages/spec/src/migrations/entries/semantic/18.sharing-rule-declared-mirror-retired.ts b/packages/spec/src/migrations/entries/semantic/18.sharing-rule-declared-mirror-retired.ts index d505bcbd80e..866a0484c75 100644 --- a/packages/spec/src/migrations/entries/semantic/18.sharing-rule-declared-mirror-retired.ts +++ b/packages/spec/src/migrations/entries/semantic/18.sharing-rule-declared-mirror-retired.ts @@ -9,7 +9,7 @@ export const entry: SemanticMigration = { + 'the declared-rule seeder (`bootstrapDeclaredSharingRules`, @objectstack/plugin-sharing) is ' + 'retired. FROM one `managed_by: package` row per declared rule TO none: the sharing-rule ' + 'service reads each declared rule from the registry, its id is its name, and its grants carry ' - + 'that name as `sys_record_share.source_id`; the 18.x organization-ownership ceremony renames the ' + + 'that name as `sys_record_share.source_id`; the organization-ownership ceremony renames the ' + 'grants of the rows the seeder wrote, then deletes those rows, and Setup\'s Sharing Rules object ' + 'list (which lists rows) no longer shows declared rules. `DELETE /api/v1/sharing/rules/:idOrName` ' + 'on a declared rule: FROM 204 with the seeded row deleted (and written again by the next boot) ' @@ -17,8 +17,8 @@ export const entry: SemanticMigration = { replacement: 'change or remove the rule in its declaration in the package (the stack\'s `sharingRules`); ' + 'read declared rules through `GET /api/v1/sharing/rules`, which lists them beside the rules an ' - + 'organization authors. An organization\'s own rule is deleted as before. Existing databases: 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. ' - + 'In that 18.x release, run `os migrate organization-ownership --apply` on the upgraded database with a fresh plan: its mirror-deletion ' + + 'organization authors. An organization\'s own rule is deleted as before. On an upgraded ' + + 'database run `os migrate organization-ownership --apply` with a fresh plan: its mirror-deletion ' + 'step removes the seeded copies', reason: 'ADR-0131 D2 retires the seeders: declared metadata is code, served from the registry, and a ' @@ -34,6 +34,6 @@ export const entry: SemanticMigration = { acceptanceCriteria: 'A fresh boot in every posture writes no `sys_sharing_rule` row for a declared rule and the rule ' + 'still grants; removing the declaration stops its grant; no consumer deletes a declared rule ' - + 'through the sharing-rule API or expects it as a `sys_sharing_rule` row. 18.0 upgrades no v17 database; in the ' - + '18.x release that carries the upgrade path, an upgraded database certifies only after its ceremony has deleted the seeded copies.', + + 'through the sharing-rule API or expects it as a `sys_sharing_rule` row; an upgraded database ' + + 'certifies only after the ceremony has deleted the seeded copies.', }; diff --git a/packages/spec/src/migrations/entries/semantic/18.sys-metadata-audit-organization-column-retired.ts b/packages/spec/src/migrations/entries/semantic/18.sys-metadata-audit-organization-column-retired.ts index b39ad0129cd..cf4df3480d2 100644 --- a/packages/spec/src/migrations/entries/semantic/18.sys-metadata-audit-organization-column-retired.ts +++ b/packages/spec/src/migrations/entries/semantic/18.sys-metadata-audit-organization-column-retired.ts @@ -18,15 +18,15 @@ export const entry: SemanticMigration = { replacement: 'nothing on this table — metadata is the environment\'s (ADR-0131 D6), and no organization owns ' + 'a row. Delete any authored filter, list-view column, report grouping, formula or seed key that ' - + 'names `organization_id` on `sys_metadata_audit`. Existing databases: 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. ' - + 'In that 18.x release, run its ceremony: `os migrate organization-ownership --out PLAN` to write its plan for review, then ' + + 'names `organization_id` on `sys_metadata_audit`. Existing databases: run the v18 ceremony, ' + + '`os migrate organization-ownership --out PLAN` to write its plan for review, then ' + '`os migrate organization-ownership --apply --plan-file PLAN`, which promotes or reports the legacy ' + 'organization rows and then drops the physical column', reason: 'ADR-0131 D6 retired the per-organization overlay axis: every metadata write now lands ' + 'environment-wide with a NULL organization, every read is environment then code, and a legacy ' - + 'organization row is served by no read, and the v18 runtime refuses to start on a database still holding the column (18.0 on every such database; the 18.x release that carries the upgrade path until its ceremony has run). The column therefore only holds ' - + 'the legacy rows the 17.x organization layer wrote, and the 18.x ceremony owns those: under ' + + 'organization row is served by no read, and the v18 runtime refuses to start on a database still holding the column until the ceremony has run. The column therefore only holds ' + + 'the legacy rows the 17.x organization layer wrote, and the v18 ceremony owns those: under ' + 'single it promotes the presentational customizations to environment rows and carries every ' + 'public-form withdrawal onto the environment, fail-closed (the maintainer rulings on organization customizations and form withdrawals), and ' + 'it reports the rest. Only then does its column-drops step drop the column, through the ' @@ -40,7 +40,6 @@ export const entry: SemanticMigration = { acceptanceCriteria: 'No authored metadata names `organization_id` on `sys_metadata_audit`: the field resolver (lint and the data ' + 'door) answers it as an unknown field. A fresh database creates `sys_metadata_audit` with no organization ' - + 'column. 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. ' - + 'In that 18.x release, after `os migrate organization-ownership --apply --plan-file PLAN` on a legacy database the column is ' + + 'column. On a legacy database, after `os migrate organization-ownership --apply --plan-file PLAN` the column is ' + 'gone and the boot no longer reports it orphaned.', }; diff --git a/packages/spec/src/migrations/entries/semantic/18.sys-metadata-commit-organization-column-retired.ts b/packages/spec/src/migrations/entries/semantic/18.sys-metadata-commit-organization-column-retired.ts index afc82e728a8..ae4d032c7b1 100644 --- a/packages/spec/src/migrations/entries/semantic/18.sys-metadata-commit-organization-column-retired.ts +++ b/packages/spec/src/migrations/entries/semantic/18.sys-metadata-commit-organization-column-retired.ts @@ -18,15 +18,15 @@ export const entry: SemanticMigration = { replacement: 'nothing on this table — metadata is the environment\'s (ADR-0131 D6), and no organization owns ' + 'a row. Delete any authored filter, list-view column, report grouping, formula or seed key that ' - + 'names `organization_id` on `sys_metadata_commit`. Existing databases: 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. ' - + 'In that 18.x release, run its ceremony: `os migrate organization-ownership --out PLAN` to write its plan for review, then ' + + 'names `organization_id` on `sys_metadata_commit`. Existing databases: run the v18 ceremony, ' + + '`os migrate organization-ownership --out PLAN` to write its plan for review, then ' + '`os migrate organization-ownership --apply --plan-file PLAN`, which promotes or reports the legacy ' + 'organization rows and then drops the physical column', reason: 'ADR-0131 D6 retired the per-organization overlay axis: every metadata write now lands ' + 'environment-wide with a NULL organization, every read is environment then code, and a legacy ' - + 'organization row is served by no read, and the v18 runtime refuses to start on a database still holding the column (18.0 on every such database; the 18.x release that carries the upgrade path until its ceremony has run). The column therefore only holds ' - + 'the legacy rows the 17.x organization layer wrote, and the 18.x ceremony owns those: under ' + + 'organization row is served by no read, and the v18 runtime refuses to start on a database still holding the column until the ceremony has run. The column therefore only holds ' + + 'the legacy rows the 17.x organization layer wrote, and the v18 ceremony owns those: under ' + 'single it promotes the presentational customizations to environment rows and carries every ' + 'public-form withdrawal onto the environment, fail-closed (the maintainer rulings on organization customizations and form withdrawals), and ' + 'it reports the rest. Only then does its column-drops step drop the column, through the ' @@ -40,7 +40,6 @@ export const entry: SemanticMigration = { acceptanceCriteria: 'No authored metadata names `organization_id` on `sys_metadata_commit`: the field resolver (lint and the data ' + 'door) answers it as an unknown field. A fresh database creates `sys_metadata_commit` with no organization ' - + 'column. 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. ' - + 'In that 18.x release, after `os migrate organization-ownership --apply --plan-file PLAN` on a legacy database the column is ' + + 'column. On a legacy database, after `os migrate organization-ownership --apply --plan-file PLAN` the column is ' + 'gone and the boot no longer reports it orphaned.', }; diff --git a/packages/spec/src/migrations/entries/semantic/18.sys-metadata-history-organization-column-retired.ts b/packages/spec/src/migrations/entries/semantic/18.sys-metadata-history-organization-column-retired.ts index cc8cc8b3f6e..c0102fcad61 100644 --- a/packages/spec/src/migrations/entries/semantic/18.sys-metadata-history-organization-column-retired.ts +++ b/packages/spec/src/migrations/entries/semantic/18.sys-metadata-history-organization-column-retired.ts @@ -18,15 +18,15 @@ export const entry: SemanticMigration = { replacement: 'nothing on this table — metadata is the environment\'s (ADR-0131 D6), and no organization owns ' + 'a row. Delete any authored filter, list-view column, report grouping, formula or seed key that ' - + 'names `organization_id` on `sys_metadata_history`. Existing databases: 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. ' - + 'In that 18.x release, run its ceremony: `os migrate organization-ownership --out PLAN` to write its plan for review, then ' + + 'names `organization_id` on `sys_metadata_history`. Existing databases: run the v18 ceremony, ' + + '`os migrate organization-ownership --out PLAN` to write its plan for review, then ' + '`os migrate organization-ownership --apply --plan-file PLAN`, which promotes or reports the legacy ' + 'organization rows and then drops the physical column', reason: 'ADR-0131 D6 retired the per-organization overlay axis: every metadata write now lands ' + 'environment-wide with a NULL organization, every read is environment then code, and a legacy ' - + 'organization row is served by no read, and the v18 runtime refuses to start on a database still holding the column (18.0 on every such database; the 18.x release that carries the upgrade path until its ceremony has run). The column therefore only holds ' - + 'the legacy rows the 17.x organization layer wrote, and the 18.x ceremony owns those: under ' + + 'organization row is served by no read, and the v18 runtime refuses to start on a database still holding the column until the ceremony has run. The column therefore only holds ' + + 'the legacy rows the 17.x organization layer wrote, and the v18 ceremony owns those: under ' + 'single it promotes the presentational customizations to environment rows and carries every ' + 'public-form withdrawal onto the environment, fail-closed (the maintainer rulings on organization customizations and form withdrawals), and ' + 'it reports the rest. Only then does its column-drops step drop the column, through the ' @@ -40,7 +40,6 @@ export const entry: SemanticMigration = { acceptanceCriteria: 'No authored metadata names `organization_id` on `sys_metadata_history`: the field resolver (lint and the data ' + 'door) answers it as an unknown field. A fresh database creates `sys_metadata_history` with no organization ' - + 'column. 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. ' - + 'In that 18.x release, after `os migrate organization-ownership --apply --plan-file PLAN` on a legacy database the column is ' + + 'column. On a legacy database, after `os migrate organization-ownership --apply --plan-file PLAN` the column is ' + 'gone and the boot no longer reports it orphaned.', }; diff --git a/packages/spec/src/migrations/entries/semantic/18.sys-metadata-organization-column-retired.ts b/packages/spec/src/migrations/entries/semantic/18.sys-metadata-organization-column-retired.ts index bf086267686..fa3ffbaa83f 100644 --- a/packages/spec/src/migrations/entries/semantic/18.sys-metadata-organization-column-retired.ts +++ b/packages/spec/src/migrations/entries/semantic/18.sys-metadata-organization-column-retired.ts @@ -18,15 +18,15 @@ export const entry: SemanticMigration = { replacement: 'nothing on this table — metadata is the environment\'s (ADR-0131 D6), and no organization owns ' + 'a row. Delete any authored filter, list-view column, report grouping, formula or seed key that ' - + 'names `organization_id` on `sys_metadata`. Existing databases: 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. ' - + 'In that 18.x release, run its ceremony: `os migrate organization-ownership --out PLAN` to write its plan for review, then ' + + 'names `organization_id` on `sys_metadata`. Existing databases: run the v18 ceremony, ' + + '`os migrate organization-ownership --out PLAN` to write its plan for review, then ' + '`os migrate organization-ownership --apply --plan-file PLAN`, which promotes or reports the legacy ' + 'organization rows and then drops the physical column', reason: 'ADR-0131 D6 retired the per-organization overlay axis: every metadata write now lands ' + 'environment-wide with a NULL organization, every read is environment then code, and a legacy ' - + 'organization row is served by no read, and the v18 runtime refuses to start on a database still holding the column (18.0 on every such database; the 18.x release that carries the upgrade path until its ceremony has run). The column therefore only holds ' - + 'the legacy rows the 17.x organization layer wrote, and the 18.x ceremony owns those: under ' + + 'organization row is served by no read, and the v18 runtime refuses to start on a database still holding the column until the ceremony has run. The column therefore only holds ' + + 'the legacy rows the 17.x organization layer wrote, and the v18 ceremony owns those: under ' + 'single it promotes the presentational customizations to environment rows and carries every ' + 'public-form withdrawal onto the environment, fail-closed (the maintainer rulings on organization customizations and form withdrawals), and ' + 'it reports the rest. Only then does its column-drops step drop the column, through the ' @@ -40,7 +40,6 @@ export const entry: SemanticMigration = { acceptanceCriteria: 'No authored metadata names `organization_id` on `sys_metadata`: the field resolver (lint and the data ' + 'door) answers it as an unknown field. A fresh database creates `sys_metadata` with no organization ' - + 'column. 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. ' - + 'In that 18.x release, after `os migrate organization-ownership --apply --plan-file PLAN` on a legacy database the column is ' + + 'column. On a legacy database, after `os migrate organization-ownership --apply --plan-file PLAN` the column is ' + 'gone and the boot no longer reports it orphaned.', }; diff --git a/packages/spec/src/migrations/entries/semantic/18.ups-permission-set-id-column-retired.ts b/packages/spec/src/migrations/entries/semantic/18.ups-permission-set-id-column-retired.ts index 4aa707db32f..cd936872ffe 100644 --- a/packages/spec/src/migrations/entries/semantic/18.ups-permission-set-id-column-retired.ts +++ b/packages/spec/src/migrations/entries/semantic/18.ups-permission-set-id-column-retired.ts @@ -21,8 +21,7 @@ export const entry: SemanticMigration = { replacement: '`permission_set`, the machine name of the permission set the grant holds (ADR-0131 D4). Delete ' + 'the key from any authored `sys_user_permission_set` seed row or data-door write that still ' - + 'carries it, and name the set instead. Existing databases: 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. ' - + 'In that 18.x release, run its ceremony. Plan it first, ' + + 'carries it, and name the set instead. Existing databases: run the v18 ceremony. Plan it first, ' + '`os migrate organization-ownership --out plan.json`, and review the file it writes; then ' + '`os migrate organization-ownership --apply --backup-taken --plan-file plan.json` executes that plan: its verified ' + 'rewrite names every grant from the column and its column-drops step then drops the two retired ' @@ -31,7 +30,7 @@ export const entry: SemanticMigration = { 'ADR-0131 D4 makes every reference to a catalog item a name, and D3 retired the permission-set ' + 'catalog rows the id pointed at. Since then no writer fills the column and no reader confers ' + 'anything through it: a grant holds its set by permission_set. D10 orders the column\'s removal ' - + 'after every grant names its set, which is the 18.x ceremony\'s verified id-to-name rewrite: it ' + + 'after every grant names its set, which is the v18 ceremony\'s verified id-to-name rewrite: it ' + 'reads the physical column by raw SQL, names each grant, verifies every name against the ' + 'security catalog, and only then drops the column through the schema-drift machinery (schema ' + 'sync is additive, so an undeclared column stays until it is dropped). The boot-time name ' @@ -47,7 +46,6 @@ export const entry: SemanticMigration = { 'No authored stack seed or client write names `permission_set_id` on `sys_user_permission_set`: ' + 'the engine refuses the undeclared field with 400 INVALID_FIELD. A second grant of one set to ' + 'one user in one organization is refused by the unique index with 409 UNIQUE_VIOLATION, ' - + 'platform-wide grants (organization NULL) included, on every write path. 18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a 17.x release until the 18.x release that carries the upgrade path. ' - + 'In that 18.x release, after `os migrate organization-ownership --apply --backup-taken --plan-file plan.json` on a legacy database the column ' + + 'platform-wide grants (organization NULL) included, on every write path. On a legacy database, after `os migrate organization-ownership --apply --backup-taken --plan-file plan.json` the column ' + 'is gone and the boot no longer reports it orphaned.', }; diff --git a/packages/spec/src/migrations/organization-ownership-upgrade-entries.test.ts b/packages/spec/src/migrations/organization-ownership-upgrade-entries.test.ts new file mode 100644 index 00000000000..beebf7921b2 --- /dev/null +++ b/packages/spec/src/migrations/organization-ownership-upgrade-entries.test.ts @@ -0,0 +1,46 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The 18.0 upgrade-guide entries that send an operator to the ADR-0131 D10 + * ownership ceremony (#24581). The ceremony is open on this line: `os migrate + * organization-ownership` writes a plan and `--apply --plan-file PLAN + * --backup-taken` applies it, on a v17 database and on one an 18.0 install + * created. So each of the seven entries #24661 re-worded for the closed 18.0 + * command now prescribes the ceremony's steps again, and none tells the + * operator to keep a database on 17.x or that the command refuses. + */ + +import { describe, expect, it } from 'vitest'; + +import { MIGRATIONS_BY_MAJOR } from './registry.js'; + +const CEREMONY_ENTRIES = [ + 'cli-organization-ownership-apply-plan-file-required', + 'sharing-rule-declared-mirror-retired', + 'sys-metadata-audit-organization-column-retired', + 'sys-metadata-commit-organization-column-retired', + 'sys-metadata-history-organization-column-retired', + 'sys-metadata-organization-column-retired', + 'ups-permission-set-id-column-retired', +] as const; + +/** What the closed 18.0 command's wording said; an open ceremony says none of it. */ +const CLOSED_COMMAND = /fresh installs only|NOT_IMPLEMENTED|on a 17\.x release|carries the upgrade path|18\.0 refuses/; + +const semantic18 = MIGRATIONS_BY_MAJOR[18]!.semantic; +const prose = (e: (typeof semantic18)[number]): string => `${e.replacement}\n${e.reason}\n${e.acceptanceCriteria}`; + +describe('[#24581] the 18.0 upgrade-guide entries prescribe the open ceremony', () => { + it.each(CEREMONY_ENTRIES)('%s names the ceremony and none of the closed command\'s wording', (id) => { + const entry = semantic18.find((e) => e.id === id); + expect(entry, id).toBeDefined(); + expect(prose(entry!)).toContain('os migrate organization-ownership'); + expect(prose(entry!)).not.toMatch(CLOSED_COMMAND); + }); + + it('⭐ no 18 entry that names the ceremony says it is closed', () => { + const naming = semantic18.filter((e) => prose(e).includes('os migrate organization-ownership')); + expect(naming.map((e) => e.id)).toEqual(expect.arrayContaining([...CEREMONY_ENTRIES])); + expect(naming.filter((e) => CLOSED_COMMAND.test(prose(e))).map((e) => e.id)).toEqual([]); + }); +}); diff --git a/packages/spec/src/system/migration.zod.ts b/packages/spec/src/system/migration.zod.ts index 5f91853fd18..2812779e774 100644 --- a/packages/spec/src/system/migration.zod.ts +++ b/packages/spec/src/system/migration.zod.ts @@ -178,13 +178,82 @@ export const VALUE_SHAPES_MIGRATION_ID = 'adr-0104-value-shapes'; * to start against a database still in the v17 shape while this marker is not * verified (D10 item 5). * - * ⛔ Not a member of {@link CREATION_ATTESTED_MIGRATION_IDS}: a fresh store is - * not in the v17 shape (no table it creates carries a retired organization - * column), so the boot needs no attestation to start it, and a row claiming a - * ceremony that never ran would read as one that did. + * ⛔ Not a member of {@link CREATION_ATTESTED_MIGRATION_IDS}: those are + * attested by `attestFreshDatastore` as a claim that a backfill's fact holds, + * and a row claiming a ceremony that never ran would read as one that did. + * + * The one exception, and the only one (ruling record 6269955919 on the v18 + * epic, ① A): the organization-ownership boot guard itself writes this row + * ONCE, on a datastore it observed being created from empty, as the store's + * BIRTH MARKER ({@link OrganizationOwnershipBirthMarker}) — verified, with + * `details.birth` naming the {@link ORGANIZATION_OWNERSHIP_SHAPE_GENERATION} + * the store was born in. It claims no ceremony ran: it records that none was + * owed. A database with no row — every database an earlier release created — + * takes the ceremony before an 18.x server serves it. */ export const ORGANIZATION_OWNERSHIP_MIGRATION_ID = 'adr-0131-organization-ownership'; +/** + * The organization-ownership shape generation this release requires (ruling + * record 6269955919 on the v18 epic, item 2). Every ceremony step that changes + * the required shape bumps it, in the change that adds the step. + * + * Both markers carry it — the ceremony's verified checkpoint + * (`details.checkpoint.shapeGeneration`) and the birth marker + * (`details.birth.shapeGeneration`) — and the boot guard decides on it ALONE, + * by one rule: a marker whose generation is below this one refuses the boot + * and names the ceremony. It never compares version strings. A marker written + * before generations existed reads as generation 0, so a database a shorter + * ceremony certified is never taken for one this release's ceremony did. + * + * Generations: + * - `1` — the ceremony that upgrades 17.x and 18.0 pre-release databases: the declared + * webhook settlement, the two sibling moves, the declared sharing-rule + * copies, the `sys_secret` / `sys_flow_credential` column drops and the + * `organization_id` NOT NULL step. + */ +export const ORGANIZATION_OWNERSHIP_SHAPE_GENERATION = 1; + +/** + * The birth marker the organization-ownership boot guard writes into + * `details.birth` of the {@link ORGANIZATION_OWNERSHIP_MIGRATION_ID} row on a + * datastore created from empty (ruling record 6269955919, ① A). Only + * `shapeGeneration` is ever decided on; the rest is provenance. + */ +export const OrganizationOwnershipBirthMarkerSchema = lazySchema(() => z.object({ + shapeGeneration: z.number().int().min(0) + .describe('The organization-ownership shape generation the store was created in; the only field the boot guard decides on'), + platformVersion: z.string() + .describe('The exact platform version that created the store (provenance only, never compared)'), + commit: z.string().optional() + .describe('The source commit of that build, when known (provenance only)'), + createdAt: z.string().datetime() + .describe('When the guard observed the store created from empty'), +}).describe('Birth marker of a datastore created from empty: the ownership shape generation it was born in')); +export type OrganizationOwnershipBirthMarker = z.input; +export type OrganizationOwnershipBirthMarkerParsed = z.infer; + +/** + * The shape generation an {@link ORGANIZATION_OWNERSHIP_MIGRATION_ID} row's + * `details` attests — its birth marker's, else its ceremony checkpoint's — or + * `0` when it names none (a marker written before generations existed, or + * details that do not parse). Pure derivation; the guard compares the answer + * with {@link ORGANIZATION_OWNERSHIP_SHAPE_GENERATION}. + */ +export function organizationOwnershipShapeGenerationOf(details: unknown): number { + let parsed: unknown = details; + if (typeof details === 'string') { + try { + parsed = JSON.parse(details); + } catch { + return 0; + } + } + const d = parsed as { birth?: { shapeGeneration?: unknown }; checkpoint?: { shapeGeneration?: unknown } } | null; + const g = d?.birth?.shapeGeneration ?? d?.checkpoint?.shapeGeneration; + return typeof g === 'number' && Number.isInteger(g) && g >= 0 ? g : 0; +} + /** * ⚰️ RETIRED migration id: ADR-0030 notification convergence. * diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index cdaa6b00f09..be40adef2f0 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -405,7 +405,7 @@ importers: version: 6.0.3 vitest: specifier: ^4.1.11 - version: 4.1.11(@opentelemetry/api@1.9.1)(@types/node@26.6.3)(@vitest/coverage-v8@4.1.11)(happy-dom@20.10.2)(jsdom@30.1.1(@noble/hashes@2.4.0))(msw@2.14.6(@types/node@26.6.3)(typescript@6.0.3))(vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1)) + version: 4.1.11(@opentelemetry/api@1.9.1)(@types/node@26.6.3)(@vitest/coverage-v8@4.1.11)(happy-dom@20.10.2)(jsdom@30.1.1(@noble/hashes@2.4.0))(msw@2.14.6(@types/node@26.6.3)(typescript@6.0.3))(vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1)) packages/apps/setup: dependencies: @@ -427,7 +427,7 @@ importers: version: 6.0.3 vitest: specifier: ^4.1.11 - version: 4.1.11(@opentelemetry/api@1.9.1)(@types/node@26.6.3)(@vitest/coverage-v8@4.1.11)(happy-dom@20.10.2)(jsdom@30.1.1(@noble/hashes@2.4.0))(msw@2.14.6(@types/node@26.6.3)(typescript@6.0.3))(vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1)) + version: 4.1.11(@opentelemetry/api@1.9.1)(@types/node@26.6.3)(@vitest/coverage-v8@4.1.11)(happy-dom@20.10.2)(jsdom@30.1.1(@noble/hashes@2.4.0))(msw@2.14.6(@types/node@26.6.3)(typescript@6.0.3))(vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1)) packages/apps/studio: dependencies: @@ -2412,6 +2412,9 @@ importers: specifier: ^4.6.5 version: 4.6.5 devDependencies: + '@libsql/client': + specifier: ^0.18.0 + version: 0.18.0 '@objectstack/platform-objects': specifier: workspace:* version: link:../platform-objects diff --git a/scripts/check-closed-ceremony-text.mjs b/scripts/check-closed-ceremony-text.mjs new file mode 100644 index 00000000000..cc4324bc451 --- /dev/null +++ b/scripts/check-closed-ceremony-text.mjs @@ -0,0 +1,207 @@ +#!/usr/bin/env node +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * check-closed-ceremony-text (#24581) -- no shipped refusal, upgrade text or + * pending changeset says the organization-ownership ceremony is closed. + * + * node scripts/check-closed-ceremony-text.mjs # the sweep (the gate) + * node scripts/check-closed-ceremony-text.mjs --self-test # the rule itself + * + * ## Why + * + * The 18.0 pre-releases refused `os migrate organization-ownership` on every + * database, and a family of refusals, boot warnings, upgrade-guide entries, + * docs and changesets learned to say so: "18.0 supports fresh installs only", + * "keep a v17 database on a 17.x release until the 18.x release that carries + * the upgrade path". Steps 5 and 6 of the 18.x line opened the ceremony, and a + * sentence like that left anywhere sends an operator away from the one command + * that upgrades their database. Each of those surfaces carries its own wording + * pin; this gate holds the property none of them can hold about the others: + * that the sentence is gone from all of them, including the one written next. + * + * ## The population + * + * What ships or is published: package sources (`packages/**`, never a test + * file, which may name the sentence to assert its absence), the docs site + * (`content/docs/**`) and the changesets still pending (`.changeset/*.md` at + * the root; `.changeset/pre/` holds the ones a pre-release already consumed, + * whose text is published history). Out of scope by rule: `docs/adr/**` and + * `content/docs/releases/**` (records of what was decided and released), and + * every `CHANGELOG.md` (release-owned). + */ + +import { readFileSync } from 'node:fs'; +import { execFileSync } from 'node:child_process'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import process from 'node:process'; +import { isEntrypoint } from './invoked-as.mjs'; + +const ROOT = join(fileURLToPath(new URL('.', import.meta.url)), '..'); + +/** The population, for `scripts/pm/dispatch-gates.mjs` (the `ROOT_DIR_WATCH_HINTS` idiom). */ +export const ROOT_DIR_WATCH_HINTS = ['packages/**', 'content/docs/**', '.changeset/**']; + +/** What the closed ceremony's wording said. Each is a sentence an open ceremony never ships. */ +export const CLOSED_CEREMONY_PHRASES = Object.freeze([ + { id: 'fresh-installs-only', re: /fresh installs? only/i }, + { id: 'keep-on-17x', re: /keep (?:a|this|the|your) (?:v17 )?(?:development )?database on a 17\.x release/i }, + { id: 'carries-the-upgrade-path', re: /the 18\.x release that carries the upgrade path/i }, + { id: 'upgrade-path-arrives-later', re: /upgrade path (?:for an existing database )?arrives in a later 18\.x release/i }, + { id: 'in-a-later-18x-release', re: /\bin a later 18\.x release\b/i }, + { id: 'ceremony-still-refuses', re: /(?:ceremony|command),? (?:which )?(?:still )?refuses(?: in this release| on every database|, plan and apply)/i }, + { id: 'eighteen-refuses-the-command', re: /\b18\.0 refuses (?:it|this invocation|every (?:plan|invocation)|plan and apply)/i }, + { id: 'until-then-it-refuses', re: /until then (?:the ceremony|this command|the command) refuses/i }, +]); + +/** Is `path` (repo-relative, `/`-separated) in the population? */ +export function inPopulation(path) { + if (/(^|\/)(node_modules|dist)\//.test(path)) return false; + if (/(^|\/)CHANGELOG\.md$/.test(path)) return false; + if (path.startsWith('docs/adr/') || path.startsWith('content/docs/releases/')) return false; + if (path.startsWith('.changeset/')) return /^\.changeset\/[^/]+\.md$/.test(path); + if (path.startsWith('content/docs/')) return /\.(mdx?|json)$/.test(path); + if (path.startsWith('packages/')) { + if (/\.(test|spec)\.[cm]?[jt]sx?$/.test(path) || /\/(test|tests|__tests__)\//.test(path)) return false; + return /\.([cm]?[jt]sx?|mdx?|json)$/.test(path); + } + return false; +} + +/** Every line of `text` that says one of the phrases: `{ line, phrase }`. */ +export function findingsIn(text) { + const out = []; + text.split('\n').forEach((line, i) => { + for (const p of CLOSED_CEREMONY_PHRASES) if (p.re.test(line)) out.push({ line: i + 1, phrase: p.id }); + }); + return out; +} + +function trackedFiles() { + return execFileSync('git', ['ls-files', '-z', '--', ...ROOT_DIR_WATCH_HINTS.map((h) => h.replace(/\/\*\*$/, ''))], { cwd: ROOT, encoding: 'utf8' }) + .split('\0') + .filter(Boolean); +} + +export function sweep() { + const findings = []; + let scanned = 0; + for (const path of trackedFiles()) { + if (!inPopulation(path)) continue; + let text; + try { + text = readFileSync(join(ROOT, path), 'utf8'); + } catch { + continue; // deleted in the working tree + } + scanned++; + for (const f of findingsIn(text)) findings.push({ path, ...f }); + } + return { scanned, findings }; +} + +function main() { + const { scanned, findings } = sweep(); + if (scanned === 0) { + console.error('✗ check-closed-ceremony-text: the population is empty — nothing was read, so nothing is proven.'); + return 2; + } + if (findings.length === 0) { + console.log(`✓ check-closed-ceremony-text: ${scanned} shipped file(s) read; none says the ownership ceremony is closed.`); + return 0; + } + console.error(`✗ check-closed-ceremony-text: ${findings.length} line(s) say the organization-ownership ceremony is closed:\n`); + for (const f of findings) console.error(` ${f.path}:${f.line} [${f.phrase}]`); + console.error( + '\n The ceremony is open (#24581): `os migrate organization-ownership --out PLAN`, then\n' + + ' `--apply --plan-file PLAN --backup-taken`, upgrades a database 17.x or an 18.0 pre-release created.\n' + + ' Name that command instead. A test may spell the sentence to assert its absence; shipped text may not.', + ); + return 1; +} + +let selfTestReachedVerdict = false; + +function selfTest() { + let checked = 0; + const failures = []; + const t = (name, ok) => { + checked++; + if (!ok) failures.push(name); + }; + + // Each phrase reds on the sentence the 18.0 pre-releases shipped. + const SHIPPED = '18.0 supports fresh installs only and does not upgrade a v17 database: keep a v17 database on a ' + + '17.x release until the 18.x release that carries the upgrade path, and upgrade it there.'; + t('the shipped 18.0 sentence is found', findingsIn(SHIPPED).length >= 3); + t('…its development spelling too', findingsIn('keep a v17 development database on a 17.x release').some((f) => f.phrase === 'keep-on-17x')); + t('…and the docs callout', findingsIn('The upgrade path for an existing database arrives in a later 18.x release').some((f) => f.phrase === 'upgrade-path-arrives-later')); + t('…a deferral to a later release', findingsIn('the walled half of this change ships in a later 18.x release.').some((f) => f.phrase === 'in-a-later-18x-release')); + t('…a ceremony that still refuses', findingsIn('Upgrading waits for the upgrade ceremony, which still refuses in this release.').some((f) => f.phrase === 'ceremony-still-refuses')); + t('…18.0 refusing the command', findingsIn('18.0 refuses this invocation, as it refuses every plan and apply').some((f) => f.phrase === 'eighteen-refuses-the-command')); + t('…and step 4\'s docs sentence', findingsIn('Until then the ceremony refuses, so `os dev` stops at step 2').some((f) => f.phrase === 'until-then-it-refuses')); + t('a finding names its line', JSON.stringify(findingsIn('a\nfresh install only\nb')) === JSON.stringify([{ line: 2, phrase: 'fresh-installs-only' }])); + + // The open ceremony's own text is not a finding. + for (const open of [ + 'This database carries no organization-ownership marker: 17.x or an 18.0 pre-release (18.0.0-next.*) created it', + "Run 'os migrate organization-ownership --out PLAN' to write and review its plan, take a backup, then run it.", + 'A fresh install holds no legacy organization rows: there, the cause is the one named above.', + 'A database that does hold them was created by 17.x: run `os migrate organization-ownership` on it first.', + 'Fresh installs are supported under every posture: `single`, `group` and `isolated`.', + 'The 18.0 pre-releases refused it with `NOT_IMPLEMENTED`: a bare run writes the plan.', + 'A walled posture boots like `single`, under the same prerequisites as before the 18.0 pre-releases.', + ]) t(`not a finding: ${open.slice(0, 50)}`, findingsIn(open).length === 0); + + // The population. + for (const [path, want] of [ + ['packages/runtime/src/organization-ownership-boot-guard.ts', true], + ['packages/runtime/src/organization-ownership-boot-guard.test.ts', false], + ['packages/qa/dogfood/test/upgrade-18x-harness.dogfood.test.ts', false], + ['packages/cli/test/helpers/config-miss-family.ts', false], + ['packages/spec/src/migrations/entries/semantic/18.sys-metadata-organization-column-retired.ts', true], + ['packages/cli/dist/index.js', false], + ['packages/cli/CHANGELOG.md', false], + ['content/docs/deployment/cli.mdx', true], + ['content/docs/releases/v18.mdx', false], + ['docs/adr/0131-total-organization-ownership-no-null-organization-id.md', false], + ['.changeset/24581-organization-ownership-ceremony-open.md', true], + ['.changeset/pre/15212-180-guards-fresh-single-only.md', false], + ['.changeset/pre.json', false], + ]) t(`population: ${path} → ${want}`, inPopulation(path) === want); + + // The watch hints declare what the sweep walks, and nothing else. + t('every walk root is declared', ['packages', 'content/docs', '.changeset'].every((r) => ROOT_DIR_WATCH_HINTS.includes(`${r}/**`))); + t('and nothing undeclared is walked', ROOT_DIR_WATCH_HINTS.every((h) => ['packages', 'content/docs', '.changeset'].includes(h.replace(/\/\*\*$/, '')))); + + // The tree: a non-empty population. + t('the population on this tree is not empty', sweep().scanned > 0); + + if (failures.length) { + for (const f of failures) console.error(` ✗ ${f}`); + console.error(`check-closed-ceremony-text --self-test: ${failures.length} of ${checked} assertion(s) FAILED.`); + selfTestReachedVerdict = true; + return 1; + } + console.log(`check-closed-ceremony-text --self-test: ${checked} assertion(s) passed.`); + selfTestReachedVerdict = true; + return 0; +} + +if (isEntrypoint(import.meta.url)) { + const argv = process.argv.slice(2); + if (argv.includes('--self-test')) { + const selfTestCode = selfTest(); + if (!selfTestReachedVerdict) { + console.error( + '\n✗ check-closed-ceremony-text self-test: selfTest() returned without reaching its verdict,\n' + + 'so no success line was printed. Exiting 0 here would report a self-test\n' + + 'that never finished as a self-test that passed.\n', + ); + process.exit(1); + } + process.exit(selfTestCode); + } + process.exit(main()); +} diff --git a/scripts/engine-double-contract.pinned.json b/scripts/engine-double-contract.pinned.json index c7bfaee37fc..f38069d31f2 100644 --- a/scripts/engine-double-contract.pinned.json +++ b/scripts/engine-double-contract.pinned.json @@ -4246,6 +4246,11 @@ "verb": "update", "pinned": 1 }, + { + "file": "packages/runtime/src/organization-ownership-boot-guard.test.ts", + "verb": "findOne", + "pinned": 1 + }, { "file": "packages/runtime/src/security/resolve-execution-context.oauth-organization.test.ts", "verb": "findOne", diff --git a/scripts/platform-object-tenancy-census.json b/scripts/platform-object-tenancy-census.json index 177a3745cab..5064e0a3c67 100644 --- a/scripts/platform-object-tenancy-census.json +++ b/scripts/platform-object-tenancy-census.json @@ -25,12 +25,12 @@ "population": "every object registered by a tracked packages/**/*.object.ts module whose name carries a platform prefix (sys_ / cloud_ / ai_). The cloud repository's own cloud_ objects are not in this tree and so not in this census.", "totals": { "registered": 85, - "inReach": 44, - "outOfReach": 41 + "inReach": 42, + "outOfReach": 43 }, "reasonTotals": { "managedBy: 'better-auth'": 25, - "systemFields.tenant: false": 16, + "systemFields.tenant: false": 18, "tenancy.enabled: false": 2 }, "unexplained": [], @@ -187,9 +187,11 @@ { "name": "sys_flow_credential", "file": "packages/services/service-automation/src/sys-flow-credential.object.ts", - "reach": "in", - "tenantField": "organization_id", - "reasons": [] + "reach": "out", + "tenantField": null, + "reasons": [ + "systemFields.tenant: false" + ] }, { "name": "sys_flow_dispatch", @@ -589,9 +591,11 @@ { "name": "sys_secret", "file": "packages/platform-objects/src/system/sys-secret.object.ts", - "reach": "in", - "tenantField": "organization_id", - "reasons": [] + "reach": "out", + "tenantField": null, + "reasons": [ + "systemFields.tenant: false" + ] }, { "name": "sys_session",