feat(cli): os migrate plan lists the tenant columns still nullable, naming the voluntary ownership ceremony; the ceremony reports a NOT NULL shard truthfully

.changeset/25134-migrate-plan-nullable-tenant-columns.md

@@ -0,0 +1,17 @@
+---
+'@objectstack/cli': patch
+---
+
+feat(cli): `os migrate plan` lists the tenant columns still nullable and names the voluntary organization-ownership ceremony; the ceremony reports a rotation shard already NOT NULL as such (#25134)
+
+Clause-②: yes (widening) — `os migrate plan --json` gains the `nullableTenantColumns` field, and the ceremony's `not-null` step gains the `already-not-null` outcome and the `alreadyNotNull` summary count. No exit code, drift entry, op, refusal or accepted input moves.
+
+Metadata-protocol: none
+
+A database the published 18.0.0 created is certified at birth with its tenant columns (`organization_id`) nullable, and `os migrate organization-ownership`, which tightens them to `NOT NULL`, is voluntary on it. Nothing told the operator that.
+
+- **`os migrate plan`: a new advisory section.** It lists every tenant column the database still holds nullable (table and column) and names `os migrate organization-ownership` as the remedy. Each column says whether the ceremony tightens it or leaves it, and why. The ceremony leaves a rotation shard created before shards were created `NOT NULL`, `sys_email_template` (its rows are mirrors the ceremony deletes, and the table retires), a table its inventory exempts, and every table on a database whose transport cannot rebuild a table (the Turso remote face). When the ceremony would tighten none of the listed columns, the section says so and does not offer it. A fresh install lists nothing. A column named `organization_id` on an object with tenancy disabled (`sys_sso_provider`) is not a tenant column and is not listed.
+- **Advisory only.** It is not drift: it proposes no change, adds nothing to `total` or `changes`, and the exit code does not change. Under `--json` it is the separate field `nullableTenantColumns`: `{ status: 'read', adr, remedy, columns: [{ table, column, database?, ceremony: 'tightens' | 'leaves', reason? }] }`, or `{ status: 'unreadable', adr, detail }` when the plan could not read it.
+- **The ceremony's `not-null` step.** A rotation shard that the driver created `NOT NULL` used to be reported as `unsupported`, with a note saying each new shard is created nullable. That has not been true since the driver began creating shards `NOT NULL`. Such a shard is now `already-not-null` (counted in `alreadyNotNull`) and gets no note. A shard rotated in before then stays `unsupported`, and its note says it is dropped when its period leaves the base table's retention.
+
+**What to do.** Nothing is required: the server starts either way. To tighten the columns the plan lists, run `os migrate organization-ownership --out PLAN`, review it, then run `os migrate organization-ownership --apply --plan-file PLAN --backup-taken`.

content/docs/deployment/cli.mdx

@@ -986,6 +986,37 @@ without `NODE_ENV=development` describes a production `os serve`, which keeps th
 objects in the primary unless `OS_TELEMETRY_DB` names a file. A `plan` against a
 sibling that does not exist yet lists its tables to create and leaves no file behind.
 
+#### Tenant columns still nullable (advisory)
+
+`os migrate plan` lists every tenant column the database still holds nullable, in a
+section of its own, and names `os migrate organization-ownership` as the remedy. A
+database the published 18.0.0 created is certified at birth with its tenant columns
+nullable, and the [organization-ownership ceremony](#os-migrate-organization-ownership)
+that tightens them is voluntary on it: the server starts either way. A database the
+ceremony upgraded keeps only the few the ceremony leaves, and each of those is listed
+with the reason it stays nullable:
+
+```text
+ℹ 2 tenant column(s) are still nullable (ADR-0131 D1). This is advisory: it is not drift, and nothing in this plan or in "os migrate apply" changes it.
+        • sys_activity__r20261020.organization_id — the ceremony leaves it nullable: a rotation shard of sys_activity, created before shards were created NOT NULL: the ceremony rebuilds no shard, and it is dropped when its period leaves sys_activity's retention
+        • sys_email_template.organization_id — the ceremony leaves it nullable: its rows are mirrors the ceremony deletes, and the table retires (ADR-0131 D10 fate 2, D13), so the ceremony does not tighten it
+      None of them is one the organization-ownership ceremony tightens; each line says why it stays nullable.
+```
+
+When the ceremony would tighten any listed column, the section's last line names its
+two commands instead. On a Turso remote database every listed column is one the
+ceremony leaves, because that transport cannot rebuild a table. A fresh install creates
+every tenant column `NOT NULL` and the section prints nothing. A column named
+`organization_id` on an object with tenancy disabled (`sys_sso_provider`) is not a
+tenant column and is not listed.
+
+The section is advisory only. It is not drift: it proposes no change, adds nothing to
+the change count, and does not change the exit code. Under `--json` it is the separate
+`nullableTenantColumns` field: `{ status: "read", adr, remedy, columns }`, where each
+column is `{ table, column, database?, ceremony: "tightens" | "leaves", reason? }`. It is
+`{ status: "unreadable", detail }` when the plan could not read it; `columns: []` means
+it read and found none.
+
 #### Nothing is applied before you confirm
 
 Both commands boot your app to read its metadata. That boot writes no row and no

content/docs/upgrading.mdx

@@ -354,6 +354,11 @@ ceremony, under any tenancy posture. A v18 server serves a database it created
 itself or one the ceremony certified, and refuses any other at boot, naming the
 command.
 
+**A database the published 18.0.0 created** carries a birth marker and is certified
+with its tenant columns (`organization_id`) still nullable. `os migrate plan` lists
+them, and the ceremony, which such a database does not require, tightens them to
+`NOT NULL` when you run it.
+
 **The sequence:**
 
 1. Before the first 18.0 boot, run `os migrate security-catalog-overlays`. It

packages/cli/src/commands/migrate/organization-ownership.ts

@@ -865,6 +865,14 @@ export default class OrganizationOwnershipCeremony extends Command {
             const { StorageNameMapping } = await import('@objectstack/spec/system');
             const { rotationBaseOf } = await import('../../utils/unmanaged-tables.js');
             const { applyNotNullThroughDrift } = await import('../../utils/organization-ownership-not-null.js');
+            const { readColumnNullability } = await import('../../utils/nullable-tenant-columns.js');
+            const { normalizeRows } = await import('@objectstack/metadata-protocol');
+            const driverFor = (database: string | undefined): unknown =>
+              database === undefined
+                ? liveStack.driver
+                : database === TELEMETRY_DATASOURCE && liveStack.telemetryDatabase
+                  ? liveStack.drivers[1]
+                  : undefined;
             const declared = new Set(
               (liveStack.allObjects() as Array<{ name?: unknown } | null>)
                 .map((o) => o?.name)
@@ -875,12 +883,13 @@ export default class OrganizationOwnershipCeremony extends Command {
               {
                 isDeclared: (table) => declared.has(rotationBaseOf(table) ?? table),
                 shardBaseOf: (table) => rotationBaseOf(table) ?? null,
-                driverFor: (database) =>
-                  (database === undefined
-                    ? liveStack.driver
-                    : database === TELEMETRY_DATASOURCE && liveStack.telemetryDatabase
-                      ? liveStack.drivers[1]
-                      : undefined) as never,
+                // [ADR-0131 D1] A shard the driver created NOT NULL is reported
+                // so, never as one the ceremony could not tighten.
+                isNotNull: async (table, column, database) => {
+                  const read = await readColumnNullability(driverFor(database), [column], normalizeRows);
+                  return 'detail' in read ? null : read.tables.get(table)?.get(column) ?? null;
+                },
+                driverFor: (database) => driverFor(database) as never,
               },
               columns,
             );

packages/cli/src/commands/migrate/plan.apply-limit.test.ts

@@ -24,8 +24,9 @@
  *  2. The footer offers `--allow-destructive` only for a destructive entry the
  *     database can apply, and names how many listed entries it cannot.
  *  3. CONTROL: a driver that declares nothing reads exactly as before.
- *  4. The `--json` document is the report, unchanged: the same keys, and each
- *     entry's message as the differ wrote it.
+ *  4. The `--json` document is the report, unchanged: the same keys (the
+ *     ADR-0131 D1 `nullableTenantColumns` advisory among them, which these
+ *     limits do not move), and each entry's message as the differ wrote it.
  *
  * `bootSchemaStack` is stubbed, so this runs without a database; the same
  * cases run against a real `sqld` in `apply.remote-face.integration.test.ts`.
@@ -245,7 +246,7 @@ describe('a driver that declares what its apply door cannot perform', () => {
     const { payload, exitCode } = await run(['--json']);
     expect(exitCode).toBe(0);
     expect(Object.keys(payload).sort()).toEqual(
-      ['changes', 'database', 'databaseSource', 'duration', 'managedTables', 'pending', 'schemaSync', 'total', 'unmanagedTables'],
+      ['changes', 'database', 'databaseSource', 'duration', 'managedTables', 'nullableTenantColumns', 'pending', 'schemaSync', 'total', 'unmanagedTables'],
     );
     expect(payload.changes.map((c: Entry) => c.message)).toEqual([DROP.message, TIGHTEN.message, RELAX.message]);
   }, CASE_TIMEOUT_MS);
@@ -430,7 +431,7 @@ describe('#24925 — a database the organization-ownership ceremony has not veri
     const { payload, exitCode } = await run(['--json']);
     expect(exitCode).toBe(0);
     expect(Object.keys(payload).sort()).toEqual(
-      ['changes', 'database', 'databaseSource', 'duration', 'managedTables', 'pending', 'total', 'unmanagedTables'],
+      ['changes', 'database', 'databaseSource', 'duration', 'managedTables', 'nullableTenantColumns', 'pending', 'total', 'unmanagedTables'],
     );
     expect(payload.changes.map((c: Entry) => c.message)).toEqual([INDEX.message, ORG_JOB.message, LEGACY.message]);
     expect(getService).not.toHaveBeenCalled();

packages/cli/src/commands/migrate/plan.nullable-tenant-columns.integration.test.ts

@@ -0,0 +1,191 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+import { describe, it, expect, afterAll } from 'vitest';
+import { spawnSync } from 'node:child_process';
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { dirname, join, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import Database from 'better-sqlite3';
+
+/**
+ * [ADR-0131 D1; ruling record 6328195425, letter A, item 2] `os migrate plan`
+ * lists the tenant columns a database still holds nullable, in an advisory
+ * section of its own that names `os migrate organization-ownership`.
+ *
+ * A database the published 18.0.0 created is certified with its tenant columns
+ * nullable; this one is brought to that shape by rebuilding a table a fresh
+ * boot created with its `organization_id` nullable — the 18.0.0 DDL — keeping
+ * its rows and indexes.
+ *
+ * What is pinned, through the public door (`serve`'s provision-and-exit, then
+ * the command, from the source entry):
+ *
+ *  - a fresh store lists nothing (`columns: []`, read — not unreadable);
+ *  - the 18.0.0-shaped store lists the nullable tenant column as one the
+ *    ceremony tightens, and a rotation shard created nullable as one it leaves;
+ *  - ⭐ it is advisory: the plan's `changes` and `total` are the fresh store's,
+ *    byte for byte — no drift entry and no op is added — and the exit code is
+ *    0 in both faces;
+ *  - the human face prints the section and the ceremony's two commands.
+ */
+
+const HERE = dirname(fileURLToPath(import.meta.url));
+const CLI_ROOT = resolve(HERE, '..', '..', '..');
+const RUN_DEV = resolve(CLI_ROOT, 'bin', 'run-dev.js');
+const TSX = resolve(CLI_ROOT, '..', '..', 'node_modules', '.bin', 'tsx');
+const CASE_TIMEOUT_MS = 300_000;
+
+const APP_OBJECT = 'os25134_deal';
+const SHARD = `${APP_OBJECT}__r20261019`;
+
+const CONFIG = [
+  'export default {',
+  "  manifest: { id: 'com.example.os25134advisory', name: 'advisory', version: '0.0.0', type: 'app' },",
+  "  requires: ['auth'],",
+  `  objects: [{ name: '${APP_OBJECT}', fields: { title: { type: 'text' } } }],`,
+  '};',
+  '',
+].join('\n');
+
+const dirs: string[] = [];
+afterAll(() => {
+  for (const d of dirs) {
+    try { rmSync(d, { recursive: true, force: true }); } catch { /* best-effort */ }
+  }
+});
+
+interface Fixture {
+  dir: string;
+  db: string;
+}
+
+function fixture(): Fixture {
+  const dir = mkdtempSync(join(tmpdir(), 'os-25134-advisory-'));
+  dirs.push(dir);
+  mkdirSync(join(dir, 'dist'), { recursive: true });
+  writeFileSync(join(dir, 'objectstack.config.ts'), CONFIG);
+  return { dir, db: join(dir, 'boot.db') };
+}
+
+/** The child's environment: the fixture's database, one file, nothing inherited that retargets it. */
+function childEnv(f: Fixture, extra: NodeJS.ProcessEnv = {}): NodeJS.ProcessEnv {
+  const env: NodeJS.ProcessEnv = { ...process.env };
+  for (const key of [
+    'OS_DATABASE_URL', 'DATABASE_URL', 'TURSO_DATABASE_URL', 'OS_DATABASE_DRIVER', 'OS_HOME',
+    'OS_TELEMETRY_DB', 'AUTH_SECRET', 'BETTER_AUTH_SECRET', 'NO_COLOR',
+  ]) {
+    delete env[key];
+  }
+  return {
+    ...env,
+    OS_DATABASE_URL: `file:${f.db}`,
+    OS_ARTIFACT_PATH: join(f.dir, 'dist', 'objectstack.json'),
+    OS_CLOUD_URL: 'off',
+    OS_TELEMETRY_DISABLED: '1',
+    OS_TELEMETRY_DB: '0',
+    OS_AUTH_SECRET: 'os25134-advisory-secret-at-least-32-characters',
+    OS_LOG_LEVEL: 'warn',
+    NO_COLOR: '1',
+    ...extra,
+  };
+}
+
+/** `os serve`'s own provision-and-exit: a fresh store, the full composition schema-synced into it. */
+function serveBoot(f: Fixture): void {
+  const boot = spawnSync(TSX, [RUN_DEV, 'serve', '--no-server', '--no-ui', '--no-console', '--port', '0'], {
+    cwd: f.dir, env: childEnv(f, { OS_MIGRATE_AND_EXIT: '1' }), encoding: 'utf8', timeout: 240_000,
+  });
+  expect(boot.status, `the serving boot did not provision the database:\n${(boot.stderr ?? '').slice(-6000)}`).toBe(0);
+}
+
+function plan(f: Fixture, json: boolean): { exit: number | null; payload: any; text: string } {
+  const run = spawnSync(TSX, [RUN_DEV, 'migrate', 'plan', ...(json ? ['--json'] : [])], {
+    cwd: f.dir, env: childEnv(f), encoding: 'utf8', timeout: 240_000,
+  });
+  let payload: any;
+  if (json) {
+    try { payload = JSON.parse(run.stdout); } catch { payload = { unparsed: `${run.stdout}\n${run.stderr}`.slice(-800) }; }
+  }
+  return { exit: run.status, payload, text: `${run.stdout}\n${run.stderr}` };
+}
+
+/**
+ * Rebuild `table` with its `organization_id` nullable — the shape the 18.0.0
+ * driver created — keeping its rows and its indexes.
+ */
+function relaxTenantColumn(file: string, table: string): void {
+  const db = new Database(file);
+  try {
+    const { sql } = db.prepare("SELECT sql FROM sqlite_master WHERE type = 'table' AND name = ?").get(table) as { sql: string };
+    const relaxed = sql.replace(/([`"]organization_id[`"][^,]*?)\s+not null/i, '$1');
+    if (relaxed === sql) throw new Error(`${table}.organization_id is not NOT NULL in ${sql}`);
+    const indexes = (db.prepare("SELECT sql FROM sqlite_master WHERE type = 'index' AND tbl_name = ? AND sql IS NOT NULL").all(table) as Array<{ sql: string }>)
+      .map((i) => i.sql);
+    db.exec('PRAGMA foreign_keys = OFF');
+    db.transaction(() => {
+      db.exec(`ALTER TABLE "${table}" RENAME TO "${table}__pre"`);
+      db.exec(relaxed);
+      db.exec(`INSERT INTO "${table}" SELECT * FROM "${table}__pre"`);
+      db.exec(`DROP TABLE "${table}__pre"`);
+      for (const index of indexes) db.exec(index);
+    })();
+  } finally {
+    db.close();
+  }
+}
+
+function notNull(file: string, table: string): boolean {
+  const db = new Database(file, { readonly: true });
+  try {
+    const col = (db.prepare(`SELECT "notnull" AS nn FROM pragma_table_info(?) WHERE name = 'organization_id'`).get(table)) as { nn: number } | undefined;
+    if (!col) throw new Error(`${table} carries no organization_id`);
+    return col.nn === 1;
+  } finally {
+    db.close();
+  }
+}
+
+describe('os migrate plan lists the nullable tenant columns as an advisory (ADR-0131 D1)', () => {
+  it('a fresh store lists nothing; an 18.0.0-shaped one lists them, and neither the drift nor the exit code moves', () => {
+    const f = fixture();
+    serveBoot(f);
+    expect(notNull(f.db, APP_OBJECT), 'a table this release creates carries its tenant column NOT NULL').toBe(true);
+
+    const fresh = plan(f, true);
+    expect(fresh.exit, JSON.stringify(fresh.payload).slice(0, 800)).toBe(0);
+    expect(fresh.payload.nullableTenantColumns).toEqual({
+      status: 'read', adr: 'ADR-0131 D1', remedy: 'os migrate organization-ownership', columns: [],
+    });
+
+    // The 18.0.0 shape: the app's tenant column nullable, and a shard rotated in nullable.
+    relaxTenantColumn(f.db, APP_OBJECT);
+    expect(notNull(f.db, APP_OBJECT)).toBe(false);
+    const db = new Database(f.db);
+    try { db.exec(`CREATE TABLE "${SHARD}" ("id" varchar(255) PRIMARY KEY, "organization_id" varchar(255))`); } finally { db.close(); }
+
+    const shaped = plan(f, true);
+    expect(shaped.exit, JSON.stringify(shaped.payload).slice(0, 800)).toBe(0);
+    expect(shaped.payload.nullableTenantColumns).toEqual({
+      status: 'read',
+      adr: 'ADR-0131 D1',
+      remedy: 'os migrate organization-ownership',
+      columns: [
+        { table: APP_OBJECT, column: 'organization_id', ceremony: 'tightens' },
+        { table: SHARD, column: 'organization_id', ceremony: 'leaves', reason: expect.stringMatching(/^a rotation shard of os25134_deal/) },
+      ],
+    });
+    // ⭐ Advisory, not drift: the plan's changes are the fresh store's, and no
+    // entry names a tenant column.
+    expect(shaped.payload.changes).toEqual(fresh.payload.changes);
+    expect(shaped.payload.total).toBe(fresh.payload.total);
+    expect(shaped.payload.changes.filter((c: any) => c.column === 'organization_id')).toEqual([]);
+
+    const human = plan(f, false);
+    expect(human.exit, human.text.slice(-2000)).toBe(0);
+    expect(human.text).toContain('2 tenant column(s) are still nullable (ADR-0131 D1). This is advisory: it is not drift');
+    expect(human.text).toContain(`• ${APP_OBJECT}.organization_id\n`);
+    expect(human.text).toContain(`• ${SHARD}.organization_id — the ceremony leaves it nullable: a rotation shard of ${APP_OBJECT}`);
+    expect(human.text).toContain('tightens 1 of them: run "os migrate organization-ownership --out PLAN"');
+  }, CASE_TIMEOUT_MS);
+});