Skip to content

Commit bb37f9c

Browse files
committed
Merge remote-tracking branch 'origin/main' into claude/issue-24865-template-read-organization
Claude-Session: https://claude.ai/code/session_01UmqzD97e14iws6hmVhgZ5y Co-authored-by: Claude <noreply@anthropic.com>
2 parents 8ed3c06 + 19484af commit bb37f9c

22 files changed

Lines changed: 2977 additions & 27 deletions
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
'@objectstack/cli': minor
3+
---
4+
5+
fix(cli): `os migrate apply` on an empty database writes the organization-ownership birth marker, so the fresh install's first `os serve` starts
6+
7+
Clause-②: yes
8+
9+
`os serve` (and so `os dev` and `os start`) serves a database this release created itself, recognised by the birth marker in `sys_migration` (row `adr-0131-organization-ownership`), or one the ownership ceremony certified. `os migrate apply` holds its schema DDL back until the operator confirms, so its boot saw no table created and wrote no marker, and the tables it then created carried none. A deploy script that ran `os migrate apply` before the first `os serve` was refused as a database "17.x or an 18.0 pre-release created", and sent through the upgrade ceremony.
10+
11+
- **Now.** When `os migrate apply` creates the database from empty, it writes the birth marker right after it creates the tables, and the first `os serve` starts. The log line is the one a serving boot prints: `[organization-ownership] a datastore created from empty: birth marker written (shape generation 1, …)`.
12+
- **Never on a database that was not empty.** The decision is the serving boot's own: the database counts as created from empty only when none of the tables this release declares existed before the apply. A database an earlier release created gains no marker from `os migrate apply`, even when the apply creates a table in it, and is still refused until the ceremony runs: `os migrate organization-ownership --out PLAN`, then `--apply --plan-file PLAN --backup-taken`.
13+
- **A failed write fails the command.** If the tables are created and the marker cannot be written, `os migrate apply` exits `1` and names the ceremony, which certifies such a database.
14+
- **Unchanged.** A database that already carries a marker keeps it as it is. `os migrate plan` writes nothing. On a Turso remote (`libsql://`) primary, `os migrate apply` creates no tables (its schema sync is skipped there), so the first serving boot creates them and writes the marker, as before.
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
'@objectstack/driver-sql': patch
3+
---
4+
5+
fix(driver-sql): `update()` no longer writes its `updated_at` stamp into the caller's `data`, so webhook deliveries for record updates are stored again on PostgreSQL and MySQL
6+
7+
Clause-②: no
8+
9+
- **What was lost.** On PostgreSQL and MySQL, every webhook delivery for a record update was lost. `SqlDriver.update()` assigned its `updated_at` stamp onto the patch object the caller passed in. ObjectQL publishes that same object as the update event's `changes`. On those two dialects the stamp is a knex `Raw` (`CURRENT_TIMESTAMP`) that holds the database client, its pool and a timer. The webhook outbox could not serialise the event (`Converting circular structure to JSON`), so no `sys_http_delivery` row was written. The record write still answered 200, and the webhook still read active. Every other reader of the update event's `changes`, and of the patch after the driver call, saw the same `Raw`. On SQLite the stamp is an ISO string, so the delivery was stored, but the caller's patch and the event's `changes` still carried an `updated_at` the caller never wrote.
10+
- **What changed.** `update()`, including its rotated-table path, stamps `updated_at` into a fresh object, as `updateMany()` already did. The caller's `data` is left exactly as it was passed.
11+
- **Unchanged.** The stored `updated_at` value on every dialect, `preserveAudit` imports, tables without an `updated_at` column, `upsert()` (it already stamps a copy of its input), and every export and option.

‎packages/cli/src/commands/migrate/apply-organization-ownership-gate.integration.test.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,10 @@ beforeAll(async () => {
125125
} finally {
126126
process.chdir(savedCwd);
127127
}
128+
// This apply of THIS tree created the store from empty, so it wrote this
129+
// release's birth marker; the v17 database each case stands in for carries
130+
// none, and the drop gate below reads that absence.
131+
withDb(v18Db, (db) => db.prepare('DELETE FROM sys_migration WHERE id = ?').run(ORGANIZATION_OWNERSHIP_MIGRATION_ID));
128132
expect(columnsOf(v18Db, 'sys_platform_setting'), 'fixture: v18 declares no organization column here').not.toContain('organization_id');
129133
}, CASE_TIMEOUT_MS);
130134

Lines changed: 241 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,241 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [ADR-0131 D10; ruling record 6269955919, ① A] `os migrate apply` is the
5+
* boot that creates a fresh install's tables when an operator's deploy script
6+
* runs it before the first `os serve`, so it writes the organization-ownership
7+
* BIRTH MARKER — and only on a database it created from empty.
8+
*
9+
* Its boot defers schema DDL: each object is stored and skipped before the
10+
* driver counts it, so the birth plugin its boot composes sees nothing created
11+
* and writes nothing. The tables are created at the confirmed flush, and the
12+
* birth half runs there (`SchemaStack.flushSchemaDdl`). The served boot's guard
13+
* then judges the file as `os serve` does — read here through
14+
* `assertOrganizationOwnershipMigrated` on a boot that creates nothing, so the
15+
* verdict is the marker's alone:
16+
*
17+
* (a) an empty file: the apply writes one verified birth row, and the guard
18+
* admits the database;
19+
* (b) a database the apply did not create — no marker, the shape an 18.0
20+
* pre-release leaves, and a 17.x one with a retired organization column —
21+
* gets NO birth marker, though the flush creates a table in it and creates
22+
* nothing else, and the guard still refuses it, naming the ceremony;
23+
* (c, d) the apply run again over the database it created and marked, with a
24+
* table to create: one row, unchanged.
25+
*
26+
* Case (b) is the rule that must not break: a false birth would let a database
27+
* an earlier release created skip the ownership ceremony.
28+
*/
29+
30+
import { describe, it, expect, beforeAll, afterAll, beforeEach, afterEach, vi } from 'vitest';
31+
import { copyFileSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
32+
import { tmpdir } from 'node:os';
33+
import { dirname, join, resolve } from 'node:path';
34+
import { fileURLToPath } from 'node:url';
35+
import Database from 'better-sqlite3';
36+
import { ORGANIZATION_OWNERSHIP_MIGRATION_ID, ORGANIZATION_OWNERSHIP_SHAPE_GENERATION } from '@objectstack/spec/system';
37+
import { assertOrganizationOwnershipMigrated, isOrganizationOwnershipBootRefusal } from '@objectstack/runtime';
38+
import { isExitSignal } from '../../utils/format.js';
39+
import { bootSchemaStack } from '../../utils/schema-migrate.js';
40+
import MigrateApply from './apply.js';
41+
42+
// Pay the first transform of the dist-resolved workspace deps the command
43+
// reaches through dynamic `import()`s at MODULE LOAD, not inside a case.
44+
import '@objectstack/objectql';
45+
46+
const HERE = dirname(fileURLToPath(import.meta.url));
47+
const CLI_ROOT = resolve(HERE, '..', '..', '..');
48+
const CASE_TIMEOUT_MS = 240_000;
49+
50+
/** The ceremony's first step, as the boot refusal prescribes it. */
51+
const CEREMONY_PLAN = 'os migrate organization-ownership --out PLAN';
52+
53+
/** A table the composition declares, dropped from a copy so the apply's flush has one to create. */
54+
const DROPPED_TABLE = 'sys_platform_setting_audit';
55+
56+
const OVERRIDING_ENV = [
57+
'OS_DATABASE_URL', 'DATABASE_URL', 'TURSO_DATABASE_URL', 'OS_DATABASE_DRIVER', 'OS_HOME',
58+
'OS_TELEMETRY_DB', 'OS_AUTH_SECRET', 'AUTH_SECRET', 'BETTER_AUTH_SECRET', 'OS_ARTIFACT_PATH',
59+
] as const;
60+
61+
async function runJson(argv: string[]): Promise<{ payload: any; exitCode: number }> {
62+
const savedExit = process.exitCode;
63+
const swallow = ((_chunk: unknown, ...rest: unknown[]) => {
64+
const cb = rest.find((a) => typeof a === 'function') as (() => void) | undefined;
65+
if (cb) cb();
66+
return true;
67+
}) as typeof process.stdout.write;
68+
const stdout = vi.spyOn(process.stdout, 'write').mockImplementation(swallow);
69+
vi.spyOn(process.stderr, 'write').mockImplementation(swallow);
70+
vi.spyOn(console, 'log').mockImplementation(() => {});
71+
vi.spyOn(console, 'warn').mockImplementation(() => {});
72+
vi.spyOn(console, 'error').mockImplementation(() => {});
73+
let processExit: number | undefined;
74+
vi.spyOn(process, 'exit').mockImplementation(((code?: number) => {
75+
processExit = code ?? 0;
76+
throw new Error(`__PROCESS_EXIT__:${processExit}`);
77+
}) as never);
78+
let thrownExit: number | undefined;
79+
try {
80+
try {
81+
await MigrateApply.run(argv, { root: CLI_ROOT });
82+
} catch (error) {
83+
const trapped = error instanceof Error && error.message.startsWith('__PROCESS_EXIT__');
84+
if (!trapped && !isExitSignal(error)) throw error;
85+
if (!trapped) thrownExit = (error as { oclif?: { exit?: number } }).oclif?.exit;
86+
}
87+
const out = stdout.mock.calls.map((c) => String(c[0])).join('').trim();
88+
let payload: any;
89+
try { payload = JSON.parse(out); } catch { payload = { unparsed: out.slice(-400) }; }
90+
return { payload, exitCode: processExit ?? thrownExit ?? (process.exitCode as number | undefined) ?? 0 };
91+
} finally {
92+
process.exitCode = savedExit;
93+
vi.restoreAllMocks();
94+
}
95+
}
96+
97+
const applyArgv = (file: string) => ['--database-url', `file:${file}`, '--yes', '--json'];
98+
99+
function withDb<T>(file: string, fn: (db: Database.Database) => T): T {
100+
const db = new Database(file);
101+
try {
102+
return fn(db);
103+
} finally {
104+
db.close();
105+
}
106+
}
107+
108+
/** Every `sys_migration` row naming the ceremony: the marker, read straight from the file. */
109+
const markerRows = (file: string): Array<Record<string, unknown>> =>
110+
withDb(file, (db) => db.prepare('SELECT * FROM sys_migration WHERE id = ?').all(ORGANIZATION_OWNERSHIP_MIGRATION_ID) as Array<Record<string, unknown>>);
111+
112+
/**
113+
* The served boot's verdict on `file`, as `os serve`'s guard reaches it, from a
114+
* boot that holds its DDL back: it creates nothing, so the guard cannot read the
115+
* store as created from empty and decides on the marker alone. Resolves when the
116+
* database would be served; rejects with the refusal when it would not.
117+
*
118+
* Composed as the apply composes, so the ledger object is registered: a boot of
119+
* the data stack alone registers no `sys_migration`, and the guard is inert
120+
* there — it would admit every file. Case (b)'s refusal is the control.
121+
*/
122+
async function guardVerdict(file: string): Promise<void> {
123+
const stack = await bootSchemaStack({
124+
jsonOutput: false,
125+
databaseUrl: `file:${file}`,
126+
deferSchemaDdl: true,
127+
readOnlyProbe: true,
128+
composeHostStack: true,
129+
composeServedPlatform: true,
130+
projectRoot: dir,
131+
});
132+
try {
133+
const engine = (stack.kernel as { getService: (name: string) => unknown }).getService('objectql');
134+
await assertOrganizationOwnershipMigrated(engine as never);
135+
} finally {
136+
await stack.shutdown();
137+
}
138+
}
139+
140+
let dir: string;
141+
let freshDb: string;
142+
let freshApply: { payload: any; exitCode: number };
143+
const savedEnv: Record<string, string | undefined> = {};
144+
const savedCwd = process.cwd();
145+
146+
/** A copy of the database the first apply created, edited raw by `edit`. */
147+
let seq = 0;
148+
function copyOfFresh(edit: (db: Database.Database) => void): string {
149+
seq += 1;
150+
const file = join(dir, `case-${seq}.db`);
151+
copyFileSync(freshDb, file);
152+
withDb(file, edit);
153+
return file;
154+
}
155+
156+
beforeAll(async () => {
157+
for (const key of [...OVERRIDING_ENV, 'NODE_ENV'] as const) savedEnv[key] = process.env[key];
158+
for (const key of OVERRIDING_ENV) delete process.env[key];
159+
dir = mkdtempSync(join(tmpdir(), 'os-apply-birth-'));
160+
mkdirSync(join(dir, 'dist'), { recursive: true });
161+
writeFileSync(
162+
join(dir, 'objectstack.config.ts'),
163+
"export default { manifest: { id: 'com.example.applybirth', name: 'birth', version: '0.0.0', type: 'app' } };\n",
164+
);
165+
process.env.OS_ARTIFACT_PATH = join(dir, 'dist', 'objectstack.json');
166+
process.env.NODE_ENV = 'production';
167+
process.env.OS_TELEMETRY_DB = '0';
168+
// (a)'s act: `os migrate apply` on a file that does not exist yet.
169+
freshDb = join(dir, 'fresh.db');
170+
process.chdir(dir);
171+
try {
172+
freshApply = await runJson(applyArgv(freshDb));
173+
} finally {
174+
process.chdir(savedCwd);
175+
}
176+
}, CASE_TIMEOUT_MS);
177+
178+
afterAll(() => {
179+
process.chdir(savedCwd);
180+
for (const [key, value] of Object.entries(savedEnv)) {
181+
if (value === undefined) delete process.env[key];
182+
else process.env[key] = value;
183+
}
184+
try { rmSync(dir, { recursive: true, force: true }); } catch { /* best-effort */ }
185+
});
186+
187+
beforeEach(() => { process.chdir(dir); });
188+
afterEach(() => { process.chdir(savedCwd); });
189+
190+
describe('os migrate apply writes the organization-ownership birth marker on a database it created from empty', () => {
191+
it('(a) an empty file: one verified birth row at the current shape generation, and the guard admits it', async () => {
192+
expect(freshApply.exitCode, JSON.stringify(freshApply.payload).slice(0, 400)).toBe(0);
193+
expect(freshApply.payload.created.length, 'the apply created the tables').toBeGreaterThan(0);
194+
195+
const rows = markerRows(freshDb);
196+
expect(rows).toHaveLength(1);
197+
expect(rows[0].verified_at).not.toBeNull();
198+
expect(Number(rows[0].blocking)).toBe(0);
199+
expect(JSON.parse(String(rows[0].details)).birth).toMatchObject({
200+
shapeGeneration: ORGANIZATION_OWNERSHIP_SHAPE_GENERATION,
201+
});
202+
203+
await expect(guardVerdict(freshDb)).resolves.toBeUndefined();
204+
}, CASE_TIMEOUT_MS);
205+
206+
it.each([
207+
['an 18.0 pre-release shape (no marker, no retired column)', [] as string[]],
208+
['a 17.x shape (no marker, a retired organization column)', ['ALTER TABLE sys_platform_setting ADD COLUMN organization_id TEXT']],
209+
])('(b) a database the apply did not create, %s: no birth marker, and the guard still refuses it', async (_shape, extra) => {
210+
const file = copyOfFresh((db) => {
211+
db.prepare('DELETE FROM sys_migration WHERE id = ?').run(ORGANIZATION_OWNERSHIP_MIGRATION_ID);
212+
db.exec(`DROP TABLE ${DROPPED_TABLE}`);
213+
for (const sql of extra) db.exec(sql);
214+
});
215+
216+
const { payload, exitCode } = await runJson(applyArgv(file));
217+
expect(exitCode, JSON.stringify(payload).slice(0, 400)).toBe(0);
218+
// The trap the birth half must not fall into: the flush's own work names
219+
// only tables it CREATED — a populated database that merely gains one reads
220+
// exactly like an empty one there.
221+
expect(payload.created).toEqual([expect.objectContaining({ table: DROPPED_TABLE, kind: 'create_table' })]);
222+
223+
expect(markerRows(file)).toEqual([]);
224+
const refusal = await guardVerdict(file).then(() => null, (error: unknown) => error);
225+
expect(isOrganizationOwnershipBootRefusal(refusal), String(refusal)).toBe(true);
226+
expect((refusal as Error).message).toContain(CEREMONY_PLAN);
227+
expect((refusal as { tables: string[] }).tables).toEqual(extra.length > 0 ? ['sys_platform_setting'] : []);
228+
}, CASE_TIMEOUT_MS);
229+
230+
it('(c, d) the apply run again over the database it created and marked, with a table to create: one row, unchanged', async () => {
231+
const file = copyOfFresh((db) => db.exec(`DROP TABLE ${DROPPED_TABLE}`));
232+
const before = markerRows(file);
233+
expect(before).toHaveLength(1);
234+
235+
const { payload, exitCode } = await runJson(applyArgv(file));
236+
expect(exitCode, JSON.stringify(payload).slice(0, 400)).toBe(0);
237+
expect(payload.created).toEqual([expect.objectContaining({ table: DROPPED_TABLE, kind: 'create_table' })]);
238+
239+
expect(markerRows(file)).toEqual(before);
240+
}, CASE_TIMEOUT_MS);
241+
});

‎packages/cli/src/commands/migrate/apply.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -629,6 +629,10 @@ export default class MigrateApply extends Command {
629629
// reconciled. Drift was detected against the pre-flush database, and a
630630
// just-created table matches metadata by construction, so the two sets
631631
// never overlap.
632+
// [ADR-0131 D10] The flush also writes the organization-ownership birth
633+
// marker when it created the database from empty — the half this
634+
// deferred boot's birth plugin could not run — so the first `os serve`
635+
// admits it. A database that was not empty gets none.
632636
const created = await stack.flushSchemaDdl();
633637
const { applied, skipped } = await stack.applyMigrationEntries(drift, { allowDestructive });
634638

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [ADR-0131 D10] The birth half `SchemaStack.flushSchemaDdl` runs once a
5+
* deferred boot's DDL is performed (`recordBirthOfFlushedStore`), at the seam
6+
* the real-database pins cannot reach: a birth marker that cannot be written.
7+
*
8+
* The tables exist and the marker does not, so the first `os serve` would
9+
* refuse the database as one an earlier release created. The flush must hand
10+
* that failure to its caller — `os migrate apply` then exits 1 — naming what
11+
* was not written and the ceremony that certifies such a database, never
12+
* report success. The real driver, the real guard and the decision itself
13+
* are `apply.organization-ownership-birth.integration.test.ts`'s.
14+
*/
15+
16+
import { describe, expect, it, vi } from 'vitest';
17+
import { ORGANIZATION_OWNERSHIP_MIGRATION_ID } from '@objectstack/spec/system';
18+
import { recordBirthOfFlushedStore } from './schema-migrate.js';
19+
20+
/** The ceremony's two steps, as the boot refusal prescribes them. */
21+
const CEREMONY_STEPS = [
22+
'os migrate organization-ownership --out PLAN',
23+
'os migrate organization-ownership --apply --plan-file PLAN --backup-taken',
24+
];
25+
26+
const engine = { getDriverForObject: () => ({}) };
27+
const kernelWith = (service: unknown) => ({ getService: (name: string) => (name === 'objectql' ? service : undefined) });
28+
29+
describe('recordBirthOfFlushedStore — the birth half after a deferred flush', () => {
30+
it('hands the runtime its engine and the logger, and resolves when the birth half does', async () => {
31+
const logger = { info: vi.fn() };
32+
const recordBirth = vi.fn(async () => null);
33+
await expect(recordBirthOfFlushedStore(kernelWith(engine), recordBirth, logger)).resolves.toBeUndefined();
34+
expect(recordBirth).toHaveBeenCalledWith(engine, logger);
35+
});
36+
37+
it('a birth marker that cannot be written fails the flush, naming the row, the cause and the ceremony', async () => {
38+
const cause = new Error('SQLITE_READONLY: attempt to write a readonly database');
39+
const recordBirth = vi.fn(async () => { throw cause; });
40+
const failure = await recordBirthOfFlushedStore(kernelWith(engine), recordBirth, undefined).then(
41+
() => null,
42+
(error: unknown) => error as Error,
43+
);
44+
expect(failure).toBeInstanceOf(Error);
45+
expect(failure!.cause).toBe(cause);
46+
expect(failure!.message).toContain(ORGANIZATION_OWNERSHIP_MIGRATION_ID);
47+
expect(failure!.message).toContain(cause.message);
48+
for (const step of CEREMONY_STEPS) expect(failure!.message).toContain(step);
49+
});
50+
51+
it('a kernel with no engine has no ledger to write: nothing is asked', async () => {
52+
const recordBirth = vi.fn(async () => null);
53+
await expect(recordBirthOfFlushedStore(kernelWith(undefined), recordBirth, undefined)).resolves.toBeUndefined();
54+
await expect(
55+
recordBirthOfFlushedStore({ getService: () => { throw new Error('not registered'); } }, recordBirth, undefined),
56+
).resolves.toBeUndefined();
57+
expect(recordBirth).not.toHaveBeenCalled();
58+
});
59+
});

0 commit comments

Comments
 (0)