Skip to content

Commit 5737f0f

Browse files
committed
Merge remote-tracking branch 'origin/main' into claude/issue-23703-release-text-in-place
2 parents ff753e1 + 6074eaf commit 5737f0f

7 files changed

Lines changed: 512 additions & 15 deletions
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
'@objectstack/objectql': patch
3+
---
4+
5+
fix(objectql): the boot hydration ERROR names the storage remedies when no datasource is declared and no driver is registered, instead of sending the operator to a datasource nobody configured
6+
7+
Clause-②: no
8+
9+
When `sys_metadata` cannot be read at boot, objectql logs one `error` line that says what was not restored and what to fix. The fix half was always "check the datasource behind sys_metadata — connection, credentials, and that the table exists". On a deployment with no datasource declared and no driver registered (a production `os serve` with no `OS_DATABASE_URL`, for one) there is no datasource to check and no credential to fix.
10+
11+
- **Before.** `ERROR sys_metadata could NOT be read at boot — persisted metadata was NOT restored, … Fix: check the datasource behind sys_metadata — connection, credentials, and that the table exists — then restart.`
12+
- **After, when nothing serves `sys_metadata`.** `ERROR sys_metadata could NOT be read at boot — no storage is configured: no datasource is declared and no driver is registered, so nothing serves sys_metadata. Persisted metadata was NOT restored, … Fix: set OS_DATABASE_URL (e.g. file:./data/app.db, or a postgres:// or mysql:// URL); mount a storage driver in the stack's plugins (e.g. new DriverPlugin(driver) from @objectstack/runtime); or run with --dev (an in-memory SQLite store, for development) — then restart.` The three fixes are the ones the CLI's `⚠ No storage:` line names.
13+
- **Unchanged.** The level stays `error`, and the consequence sentence is the same word for word. The datasource remedy stays for every case where a datasource exists: a registered driver whose read failed, `sys_metadata` bound to a named datasource with no driver, and a declared datasource the connection layer could not use. Which line prints is read from the engine's routing, not from the text of the error the read raised.
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
'@objectstack/cli': minor
3+
---
4+
5+
fix(cli)!: `os migrate apply --allow-destructive` holds back the `organization_id` drop on every table the organization-ownership ceremony plans, not only the platform inventory's, until the ceremony has verified (ADR-0131 D7, D10 item 5)
6+
7+
Clause-②: no (narrowing)
8+
9+
<!-- adr-0087: not-required (no-migration-prescription) No metadata moves: no authorable spelling, export or stored shape is removed, renamed or re-shaped, and no stored row is rewritten, converted or deleted, so there is nothing for `objectstack migrate meta` to rewrite. What narrows is one CLI invocation: an `os migrate apply --allow-destructive` run that would drop `organization_id` from a table outside the platform inventory is refused until the organization-ownership ceremony's marker is verified; the remedy is running that ceremony, never a rewrite of a package source or a stored row. The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id is named or touched (not registered or already-registered); and no exported declaration is removed or narrowed (not runtime-interface-only, type-surface-only or runtime-export-retired). -->
10+
11+
**BREAKING** (an accept-set narrowing), shipped as `minor` under the launch-window convention for breaking changes.
12+
13+
**The contract this meets.** The drop gate's own header (`packages/cli/src/utils/organization-ownership-drop-gate.ts`) already states it: the drop of `organization_id` on a table the ceremony owns waits for the ceremony's verified marker, and "⛔ No flag stands it down". ADR-0131 D10 item 5 is the rule behind it: a v18 runtime refuses to boot against a database the ceremony has not completed, and offers no escape hatch that silently skips that check. A column dropped ahead of the ceremony is that escape hatch, because the boot refusal then finds no column to refuse over.
14+
15+
**FROM.** The gate held back only the tables of the platform inventory. Every other `organization_id` drop the schema-drift report offered was applied by `os migrate apply --allow-destructive` with no ceremony marker on the database: on the objects a host declares platform-global (ADR-0131 D7, the `org-scoping` service's `platformGlobalObjects`), and on any host or application object that declares no organization column of its own. Measured on a host composition that declares two objects platform-global: `os migrate apply --database-url file:v17.db --allow-destructive --yes --json` exited 0 and applied `drop_column organization_id` on both.
16+
17+
**TO.** The gate holds back every `organization_id` drop the drift offers. The ceremony's plan meets every physical table that carries the column, with an inventory row, a `--fate-supplement` row, the application-object fate, or a refusal by name; the drift offers only a column that is physically there. So this is the ceremony's own table set, read without a copy of its inventory and without the fate-supplement file `os migrate apply` is never handed. It includes an application object that declares `systemFields: { tenant: false }` over a table that still carries the column: the ceremony plans that table, and the v18 boot refusal refuses over it too. While the `sys_migration` row `adr-0131-organization-ownership` is not verified, the run is refused with the existing refusal. It exits 1 and prints "Refusing this --allow-destructive run: it would drop organization_id from TABLES", naming `os migrate organization-ownership --out PLAN`, then `os migrate organization-ownership --apply --plan-file PLAN`, as the remedy. Under `--json` it answers `message: 'refused_organization_ownership_unmigrated'` with the `tables`, the `detail` and the `marker`. As before, the refusal holds the whole `--allow-destructive` run: nothing in it is created, applied or dropped.
18+
19+
**Unchanged.** Once the ceremony's marker is verified, the drop applies. A destructive change to any other column applies as before. A run without `--allow-destructive` is unchanged. There is no new flag, and no flag skips the refusal.
20+
21+
**Remedy.** Run `os migrate organization-ownership` (with `--fate-supplement FILE` when the host provides objects of its own), review its plan, run `os migrate organization-ownership --apply --plan-file PLAN`, then re-run `os migrate apply --allow-destructive`.

‎packages/cli/src/utils/organization-ownership-drop-gate.ts‎

Lines changed: 25 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,35 +6,50 @@
66
* ceremony verified on this database?
77
*
88
* A v18 runtime declares no organization column on the ceremony's
9-
* column-drop tables (the D7 tables and the `sys_metadata` family), so the
10-
* drift report lists each physical one as an unmapped, destructive column and
9+
* column-drop tables (the D7 tables, the platform's and the ones a deployment
10+
* declares platform-global, and the `sys_metadata` family), so the drift
11+
* report lists each physical one as an unmapped, destructive column and
1112
* names `os migrate apply --allow-destructive` as its remedy. Applied ahead of
1213
* the ceremony, that drop skips its promotion and report steps, turns every
1314
* legacy organization row into an environment row, and leaves no physical
1415
* column for the v18 boot refusal to find — so the server starts with the
1516
* marker never written. D10 item 5 forbids exactly that escape hatch. So the
16-
* drop of `organization_id` on any table in the ceremony's inventory waits for
17-
* the ceremony's verified marker, as `sys_account.issuer` (#17440) and
17+
* drop of `organization_id` on any table the ceremony plans waits for the
18+
* ceremony's verified marker, as `sys_account.issuer` (#17440) and
1819
* `sys_comment.reactions` (#23281) wait for theirs. ⛔ No flag stands it down.
20+
*
21+
* ## Which tables: every `organization_id` drop the drift offers (#23684)
22+
*
23+
* The ceremony's table set is the database's, not a list's: its plan meets
24+
* every physical table that carries `organization_id`, with the platform
25+
* inventory's row, a host fate supplement's row, the application-object fate,
26+
* or a refusal by name (`organization-ownership-plan.ts`). The drift can offer
27+
* to drop only a column that is physically there. So every `organization_id`
28+
* drop it offers is on a table the ceremony plans, and this gate holds back
29+
* every one: the platform inventory's tables, the objects a deployment
30+
* declares platform-global (ADR-0131 D7), and the objects a host's fate
31+
* supplement names, a file `os migrate apply` is never handed. The v18 boot
32+
* refusal judges the same population, every object that declares no
33+
* organization column, not the inventory.
34+
*
35+
* Reading the drift rather than a composed table list keeps one source: no
36+
* second copy of the ceremony's inventory composition, and no part of the
37+
* answer this command could fail to read.
1938
*/
2039

2140
import type { MigrationFlagEngine } from '@objectstack/platform-objects/system';
2241
import type { DataMigrationFlag } from '@objectstack/spec/system';
23-
import { ORGANIZATION_OWNERSHIP_INVENTORY } from './organization-ownership-inventory.js';
2442

2543
/** The column the ceremony owns the drop of. */
2644
export const ORGANIZATION_COLUMN = 'organization_id';
2745

28-
/** Every table the ceremony's inventory names. */
29-
const CEREMONY_TABLES: ReadonlySet<string> = new Set(ORGANIZATION_OWNERSHIP_INVENTORY.map((entry) => entry.object));
30-
31-
/** The tables whose `organization_id` drop is among `intended`, sorted. */
46+
/** The tables whose `organization_id` drop is among `intended`, sorted: every one, since the ceremony plans each table that carries the column. */
3247
export function ceremonyOrganizationColumnDrops(
3348
intended: ReadonlyArray<{ table: string; op?: { type?: string; column?: string } }>,
3449
): string[] {
3550
const tables = new Set<string>();
3651
for (const d of intended) {
37-
if (d.op?.type === 'drop_column' && d.op.column === ORGANIZATION_COLUMN && CEREMONY_TABLES.has(d.table)) tables.add(d.table);
52+
if (d.op?.type === 'drop_column' && d.op.column === ORGANIZATION_COLUMN) tables.add(d.table);
3853
}
3954
return [...tables].sort();
4055
}

‎packages/cli/src/utils/organization-ownership-platform-global-column-drop.integration.test.ts‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,20 @@
4040
*
4141
* Every case runs the real command, plan then apply, over a fresh copy of the
4242
* database; each fate the case does not exercise is `report`. SQLite only.
43+
*
44+
* ## `os migrate apply --allow-destructive` on the same composition
45+
*
46+
* The schema-drift report of this composition offers the two declared
47+
* objects' `organization_id` for the destructive apply. Its drop gate
48+
* (`organization-ownership-drop-gate.ts`, ADR-0131 D10 item 5) holds that drop
49+
* back until the ceremony's marker is verified — on these host-declared
50+
* objects as on the platform's inventory, since the ceremony plans every
51+
* table that carries the column. The real `os migrate apply` command, over a
52+
* fresh copy of the same v17 database:
53+
*
54+
* - no marker: refused, naming the ceremony; nothing dropped;
55+
* - CONTROL: a verified marker: the drop applies;
56+
* - CONTROL: a destructive drop of another column is unchanged.
4357
*/
4458

4559
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
@@ -50,7 +64,9 @@ import { randomBytes } from 'node:crypto';
5064
import { tmpdir } from 'node:os';
5165
import { dirname, join, resolve } from 'node:path';
5266
import { fileURLToPath, pathToFileURL } from 'node:url';
67+
import { ORGANIZATION_OWNERSHIP_MIGRATION_ID } from '@objectstack/spec/system';
5368
import { isExitSignal } from './format.js';
69+
import MigrateApply from '../commands/migrate/apply.js';
5470
import MigrateOrganizationOwnership from '../commands/migrate/organization-ownership.js';
5571

5672
// Paid at module load, not inside a clocked case (`check:test-source-alias`).
@@ -248,6 +264,74 @@ async function ceremony(name: string, drop: string): Promise<Run> {
248264
}
249265
}
250266

267+
/** A fresh copy of the v17 database for one `os migrate apply` case, with `prepare` run on it raw. */
268+
function applyCase(name: string, prepare?: (db: Database.Database) => void): string {
269+
const caseDir = join(dir, 'apply-cases', name);
270+
mkdirSync(caseDir, { recursive: true });
271+
const dbFile = join(caseDir, 'v17.db');
272+
copyFileSync(templateDb, dbFile);
273+
if (prepare) {
274+
const db = new Database(dbFile);
275+
try {
276+
prepare(db);
277+
} finally {
278+
db.close();
279+
}
280+
}
281+
return dbFile;
282+
}
283+
284+
/** The real `os migrate apply --allow-destructive --yes --json` over `dbFile`, from the host's project. */
285+
async function migrateApply(dbFile: string): Promise<{ payload: any; exitCode: number }> {
286+
const savedExit = process.exitCode;
287+
const swallow = ((_chunk: unknown, ...rest: unknown[]) => {
288+
const cb = rest.find((a) => typeof a === 'function') as (() => void) | undefined;
289+
if (cb) cb();
290+
return true;
291+
}) as typeof process.stdout.write;
292+
const stdout = vi.spyOn(process.stdout, 'write').mockImplementation(swallow);
293+
vi.spyOn(process.stderr, 'write').mockImplementation(swallow);
294+
vi.spyOn(console, 'log').mockImplementation(() => {});
295+
vi.spyOn(console, 'warn').mockImplementation(() => {});
296+
vi.spyOn(console, 'error').mockImplementation(() => {});
297+
// `run()` ends the one-shot process itself once the document is written.
298+
let processExit: number | undefined;
299+
vi.spyOn(process, 'exit').mockImplementation(((code?: number) => {
300+
processExit = code ?? 0;
301+
throw new Error(`__PROCESS_EXIT__:${processExit}`);
302+
}) as never);
303+
process.chdir(projectDir);
304+
let thrownExit: number | undefined;
305+
try {
306+
try {
307+
await MigrateApply.run(['--database-url', `file:${dbFile}`, '--allow-destructive', '--yes', '--json'], { root: CLI_ROOT });
308+
} catch (error) {
309+
const trapped = error instanceof Error && error.message.startsWith('__PROCESS_EXIT__');
310+
if (!trapped && !isExitSignal(error)) throw error;
311+
if (!trapped) thrownExit = (error as { oclif?: { exit?: number } }).oclif?.exit;
312+
}
313+
const out = stdout.mock.calls.map((c) => String(c[0])).join('').trim();
314+
let payload: any;
315+
try {
316+
payload = JSON.parse(out);
317+
} catch {
318+
payload = { unparsed: out.slice(-600) };
319+
}
320+
return { payload, exitCode: processExit ?? thrownExit ?? (process.exitCode as number | undefined) ?? 0 };
321+
} finally {
322+
process.chdir(savedCwd);
323+
process.exitCode = savedExit;
324+
vi.restoreAllMocks();
325+
}
326+
}
327+
328+
/** The `organization_id` drops among an apply payload's entries, by table, sorted. */
329+
const organizationDropsIn = (entries: any[] | undefined): string[] =>
330+
(entries ?? [])
331+
.filter((d) => d?.op?.type === 'drop_column' && d.op.column === 'organization_id')
332+
.map((d) => String(d.table))
333+
.sort();
334+
251335
const columnDrops = (report: any): { status: string; summary: Record<string, number>; blockers: string[] } =>
252336
report.steps.find((s: { step: string }) => s.step === 'column-drops');
253337
const blockersFor = (report: any, table: string): string[] =>
@@ -337,3 +421,66 @@ describe('organization-ownership --apply: column-drop on an object the host decl
337421
expect(exitCode).toBe(1);
338422
}, CASE_TIMEOUT_MS);
339423
});
424+
425+
describe('os migrate apply --allow-destructive: the organization_id drop on a host-declared platform-global object waits for the ceremony', () => {
426+
it('holds the drop back until the marker is verified, naming the ceremony, and drops nothing', async () => {
427+
const dbFile = applyCase('no-marker');
428+
const { payload, exitCode } = await migrateApply(dbFile);
429+
430+
const seen = { message: payload.message, error: payload.error, appliedOrganizationDrops: organizationDropsIn(payload.applied) };
431+
expect(exitCode, JSON.stringify(seen)).toBe(1);
432+
expect(payload).toMatchObject({
433+
message: 'refused_organization_ownership_unmigrated',
434+
tables: [...DECLARED].sort(),
435+
applied: [],
436+
created: [],
437+
marker: null,
438+
});
439+
expect(payload.detail).toMatch(/os migrate organization-ownership --apply/);
440+
// The drops the refusal held back are the ones the drift offered.
441+
expect(organizationDropsIn(payload.skipped)).toEqual([...DECLARED].sort());
442+
for (const table of HOST_OBJECTS) expect(hasOrganizationColumn(dbFile, table), table).toBe(true);
443+
}, CASE_TIMEOUT_MS);
444+
445+
it('CONTROL: once the ceremony\'s marker is verified, the drop applies', async () => {
446+
const dbFile = applyCase('verified-marker', (db) => {
447+
const now = new Date().toISOString();
448+
db.prepare('INSERT INTO sys_migration (id, last_run_at, verified_at, blocking) VALUES (?, ?, ?, 0)').run(
449+
ORGANIZATION_OWNERSHIP_MIGRATION_ID,
450+
now,
451+
now,
452+
);
453+
});
454+
const { payload, exitCode } = await migrateApply(dbFile);
455+
456+
expect(exitCode, JSON.stringify(payload).slice(0, 600)).toBe(0);
457+
expect(payload.message).toBeUndefined();
458+
expect(organizationDropsIn(payload.applied)).toEqual([...DECLARED].sort());
459+
for (const table of DECLARED) expect(hasOrganizationColumn(dbFile, table), table).toBe(false);
460+
// The undeclared object's column is the release's, and stays.
461+
expect(hasOrganizationColumn(dbFile, AUDIT)).toBe(true);
462+
}, CASE_TIMEOUT_MS);
463+
464+
it('CONTROL: a destructive drop of another column is unchanged', async () => {
465+
// The declared objects' organization_id already gone, and a column no
466+
// object declares on one of them: the run's one destructive change.
467+
const dbFile = applyCase('other-column', (db) => {
468+
for (const table of DECLARED) db.exec(`ALTER TABLE "${table}" DROP COLUMN organization_id`);
469+
db.exec(`ALTER TABLE "${LEDGER}" ADD COLUMN legacy_23684_note TEXT`);
470+
});
471+
const { payload, exitCode } = await migrateApply(dbFile);
472+
473+
expect(exitCode, JSON.stringify(payload).slice(0, 600)).toBe(0);
474+
expect(payload.message).toBeUndefined();
475+
expect(
476+
(payload.applied as any[]).filter((d) => d?.op?.type === 'drop_column').map((d) => `${d.table}.${d.op.column}`),
477+
).toEqual([`${LEDGER}.legacy_23684_note`]);
478+
const db = new Database(dbFile, { readonly: true });
479+
try {
480+
const columns = (db.prepare(`PRAGMA table_info("${LEDGER}")`).all() as Array<{ name: string }>).map((c) => c.name);
481+
expect(columns).not.toContain('legacy_23684_note');
482+
} finally {
483+
db.close();
484+
}
485+
}, CASE_TIMEOUT_MS);
486+
});

0 commit comments

Comments
 (0)