Skip to content

Commit 4c7dcb4

Browse files
test(driver-sql): the three live-MySQL NULL-safe shadow suites model a pre-18 store (#25142)
Refs #25131 Clause-②: no (test fixtures only: no response, status, error, outcome or export moves) Metadata-protocol: none `main`'s hourly full run is red in `Temporal Conformance (live PG + MySQL)`, in the step "Run driver-sql suite against both live servers". This PR changes only the fixture setup of the three live-MySQL suites that fail there. No production file moves, and no test or assertion is narrowed, skipped or deleted. ## What is red, measured on `60b42cd0b7` and on today's `abc25a35` **7 failures, not the 6 the card names.** The card's count was read on `fccbd9d0d5`, before #24893 (`d5d8146012`, 18.x step 7b-1 + 7b-2) landed. The hourly run's own log at `60b42cd0b7` (run 39683348463, job 119511306969) lists 7. My local reproduction below lists the same 7, by name. | File | Failing test | What failed | |---|---|---| | `sql-driver-12998-shadow-null-safe-key.test.ts` | collides two NULL-organization rows under an org-scoped shadow unique | the generation expression has no `coalesce` | | 12998 | diagnoses existing NULL-org duplicates instead of failing the boot unexplained | setup insert: `Column 'organization_id' cannot be null` | | 12998 | names a NULL-organization duplicate in declared terms, never as a collision | setup insert: `Field 'organization_id' doesn't have a default value` | | 12998 | counts the sixth conflicting group in the shared "more group(s)" tail | setup insert: `Column 'organization_id' cannot be null` | | `sql-driver-13015-shadow-carried-index-drift.test.ts` | survives "os migrate apply --allow-destructive" over every reported entry | setup insert: `Column 'organization_id' cannot be null` | | 13015 | reports and repairs a shadow that hashes the raw columns | no drift reported for a raw shadow | | `sql-driver-15479-shadow-plain-unique-duplicates.test.ts` | leaves the NULL-safe arm saying the NULL-safe thing | setup insert: `Column 'organization_id' cannot be null` | **There are two causes, not one.** - **7b-0 (#24940):** `initObjects` creates `organization_id` NOT NULL, so the suites' NULL-organization seeds are refused. That accounts for 5 of the 7. - **7b-2 (#24893):** a `unique: 'organization'` declaration now keys the PLAIN tenant column. On a fresh table this is correct and certified. So the shadow no longer carries `COALESCE(organization_id, '__global__')`, and a raw-column shadow is no longer drift. That accounts for the other 2. - Every one of the 7 pins the NULL-safe hash-shadow arm. Since 7b-2, the driver reaches that arm only from an index input that already carries `nullSafeColumns` (`normalizeDeclaredIndex`: "honoured verbatim"), which is the shape a database built before 18.x holds. - So `asPre18TenantColumn` alone, the route the claim names, greens 5 of the 7 at most. The other 2 would still assert a COALESCE form that the declaration no longer produces. ## The change (fixture setup only, the three files) - The org-scoped index these tests use is declared in its **18.0 form**: `fields: ['organization_id', 'v'], unique: true, nullSafeColumns: ['organization_id']`. - This is the same spelling #24893 itself adopted for the retained NULL-safe path in `driver-turso`'s `turso-remote-drift-detection.test.ts` and `turso-remote-migration-entries.test.ts`. - The index names are unchanged. - Where it applies: - 12998: its `orgUniqueOn`, used by all four org tests. - 15479: its control `orgUniqueOn`. - 13015: a new `pre18OrgUniqueOn`, used only by the two failing live tests. The two passing live tests keep the plain declaration. - `asPre18TenantColumn(driver, table)` follows each `initObjects` whose table then receives NULL-organization rows: seven call sites. This is the pattern 7b-0 wired into the other driver-sql suites. - In 15479 the call is conditional on the seed row carrying `organization_id: null`. Only the control does; the plain and global arms stamp an organization or have no tenant column. - Every assertion is untouched. ## Measured before the edit - **The second `initObjects` does not restore NOT NULL.** A scratch probe against the built driver on live MySQL 8.0 read `information_schema.COLUMNS.IS_NULLABLE`: - after the first `initObjects`: `NO`; - after `asPre18TenantColumn`: `YES`; - after a second `initObjects` with the index, on the same driver (12998 dirty/tail and the 15479 shape): `YES`; - after a second `initObjects` on a new driver (the 13015 apply shape): `YES`; - `detectManagedDrift` then reports `[]`, and `applyMigrationEntries(..., { allowDestructive: true })` leaves it `YES`. - So no second relax is needed anywhere. - **Why the PG cell passes:** none of the three suites has a PostgreSQL cell. The hash-shadow route exists only because MySQL refuses a key part over its 768-character ceiling; PostgreSQL and SQLite never take it. The PG-gated suites that seed organization-less rows were wired with `asPre18TenantColumn` by 7b-0. - **Census** of the live-gated `driver-sql` suites: - Static pass: 79 test files use `declareDialectCell`, a dialect cell or an `OS_TEST_*_URL`. Only these three insert organization-less rows without the relax. The one other hit, `sql-driver-tenant-column-not-null.test.ts`, inserts NULL on purpose to pin 7b-0's refusal. - Measured pass: the full live run below. 242/242 files are green, so no fourth file surfaces. ## Verification Live MySQL 8.0 (`mirror.gcr.io/library/mysql:8.0`, `SET GLOBAL time_zone = '+08:00'`) and PostgreSQL 16 (`mirror.gcr.io/library/postgres:16`, timezone `Asia/Shanghai`), both in local containers, with CI's environment: `TZ=America/New_York`, `OS_TEST_MYSQL_URL`, `OS_TEST_POSTGRES_URL` and `OS_EXPECT_LIVE_DIALECT_MATRIX=1`. | Run | Head | Result | |---|---|---| | The three files, live, before the edit | `abc25a35` | `Test Files 3 failed (3)`, `Tests 7 failed / 12 passed (19)`: the 7 above | | The three files, live, after the edit | `e4c6eaf7c` | `Test Files 3 passed (3)`, `Tests 19 passed (19)` | | `pnpm --filter @objectstack/driver-sql test`, live PG + MySQL (CI's step) | `e4c6eaf7c` | `Test Files 242 passed (242)`, `Tests 5688 passed / 2 skipped (5690)`, exit 0 | | `pnpm --filter @objectstack/driver-sql test`, default (no live servers) | `e4c6eaf7c` | `Test Files 230 passed / 12 skipped (242)`, `Tests 3742 passed / 212 skipped`, exit 0 | | `pnpm --filter @objectstack/driver-sql typecheck` | `e4c6eaf7c` | exit 0. `tsc --listFiles` lists all three files. | **Gates on `e4c6eaf7c`.** `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 54 families. - All 54 ran, with each exit code captured before any pipe. - 52 exited 0. `check:dual-build-cjs-loads` and `check:lean-entry-closure` exited 3 (PREREQUISITE NOT MET); after `pnpm exec turbo run build --concurrency=2` (77/77), both exit 0. - `--ran` reconciliation: "54 derived, 54 run, 0 NOT-MEASURED, 0 UNRUN", a DERIVED zero. - `origin/main` was merged before this PR opened: already up to date at `abc25a35`. **Changeset.** `skip-changeset`: `@objectstack/driver-sql` ships `files: [dist, README.md, CHANGELOG.md]`. Built-dist grep: the test fixture symbols hit 0 files, and the control `relax_not_null` hits 4. **The hourly run.** `Temporal Conformance (live PG + MySQL)` runs only on push, schedule and dispatch, never on a PR or in the queue. This PR's CI therefore does not run the live cell. The local live run above is its proof here, and the next hourly full run after merge is the proof on `main`. ## Acceptance notes - **The claim's fence.** The claim named `asPre18TenantColumn` alone. The measurement shows that 2 of the 7 failures come from 7b-2's plain key form, not from 7b-0's NOT NULL. The fix stays in the claim's three files and in fixture setup, and follows #24893's own Turso precedent for the retained NULL-safe path. - **Card arithmetic.** The card body and its triage count 6 failures, read at `fccbd9d0d5`. At `60b42cd0b7` and later it is 7: the extra one is 13015's "reports and repairs a shadow that hashes the raw columns". - **Not in scope:** #25132 remains open, and the NOT NULL itself (ADR-0131 D1) is untouched. --- _Generated by [Claude Code](https://claude.ai/code/session_018gewcgUaNUX9hMkxeP2HE2)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent abc25a3 commit 4c7dcb4

3 files changed

Lines changed: 67 additions & 8 deletions

File tree

‎packages/drivers/driver-sql/src/sql-driver-12998-shadow-null-safe-key.test.ts‎

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,21 +40,36 @@ import { describe, it, expect, afterEach } from 'vitest';
4040
import { SqlDriver } from '../src/index.js';
4141
import { isHashShadowColumn } from './schema-drift.js';
4242
import { MYSQL_CELL, declareDialectCell } from './live-dialect-matrix.testkit.js';
43+
import { asPre18TenantColumn } from './pre-18-tenant-column.testkit.js';
4344

4445
/**
4546
* An object with a tenant column and one long text field carrying an
46-
* ORG-SCOPED unique. `maxLength: 1024` exceeds the 768-char keyable ceiling, so
47-
* MySQL refuses the direct (functional-key-part) index and the sync takes the
48-
* shadow route — the same route the live members
47+
* ORG-SCOPED unique in its NULL-safe (18.0) form. `maxLength: 1024` exceeds the
48+
* 768-char keyable ceiling, so MySQL refuses the direct (functional-key-part)
49+
* index and the sync takes the shadow route — the same route the live members
4950
* (`sys_notification_preference` / `sys_notification_subscription`) take.
51+
*
52+
* ADR-0131 D1: a `unique: 'organization'` declaration now keys the PLAIN
53+
* tenant column, so the NULL-safe key part this file pins reaches the shadow
54+
* only from an input that already carries `nullSafeColumns` (honoured
55+
* verbatim) — the shape a database built before 18.x holds. Every test that
56+
* writes NULL-organization rows also returns the table to that store's
57+
* nullable tenant column (`asPre18TenantColumn`).
5058
*/
5159
const orgUniqueOn = (name: string) => ({
5260
name,
5361
fields: {
5462
organization_id: { type: 'string' },
5563
v: { type: 'text', maxLength: 1024 },
5664
},
57-
indexes: [{ fields: ['v'], unique: 'organization' as const, name: `uniq_${name}_org_v` }],
65+
indexes: [
66+
{
67+
fields: ['organization_id', 'v'],
68+
unique: true as const,
69+
nullSafeColumns: ['organization_id'],
70+
name: `uniq_${name}_org_v`,
71+
},
72+
],
5873
});
5974

6075
declareDialectCell(MYSQL_CELL, 'hash-shadow NULL-safe key (#12998)', (cell) => {
@@ -100,6 +115,7 @@ declareDialectCell(MYSQL_CELL, 'hash-shadow NULL-safe key (#12998)', (cell) => {
100115
it('collides two NULL-organization rows under an org-scoped shadow unique', async () => {
101116
driver = new SqlDriver(cell.config());
102117
await driver.initObjects([orgUniqueOn('os12998_org')]);
118+
await asPre18TenantColumn(driver, 'os12998_org');
103119

104120
const { cols, idx } = await catalog('os12998_org');
105121
const shadow = cols.find((c: any) => isHashShadowColumn(c.COLUMN_NAME));
@@ -184,6 +200,7 @@ declareDialectCell(MYSQL_CELL, 'hash-shadow NULL-safe key (#12998)', (cell) => {
184200
const bare = orgUniqueOn('os12998_dirty');
185201
const withoutIndex = { ...bare, indexes: [] };
186202
await driver.initObjects([withoutIndex]);
203+
await asPre18TenantColumn(driver, 'os12998_dirty');
187204
const knex = (driver as any).knex;
188205
const V = 'd'.repeat(900);
189206
await knex('os12998_dirty').insert([
@@ -220,6 +237,7 @@ declareDialectCell(MYSQL_CELL, 'hash-shadow NULL-safe key (#12998)', (cell) => {
220237
it('names a NULL-organization duplicate in declared terms, never as a collision', async () => {
221238
driver = new SqlDriver(cell.config());
222239
await driver.initObjects([orgUniqueOn('os12998_msg')]);
240+
await asPre18TenantColumn(driver, 'os12998_msg');
223241
const V = 'm'.repeat(900);
224242
await driver.create('os12998_msg', { v: V });
225243
const err: unknown = await driver.create('os12998_msg', { v: V }).then(
@@ -266,6 +284,7 @@ declareDialectCell(MYSQL_CELL, 'hash-shadow NULL-safe key (#12998)', (cell) => {
266284
// rejected by tsc for a key the driver reads regardless (the gap commit b72226f48 closed).
267285
const withoutIndex = { ...bare, indexes: [] };
268286
await driver.initObjects([withoutIndex]);
287+
await asPre18TenantColumn(driver, 'os12998_tail');
269288
const knex = (driver as any).knex;
270289
// Six DISTINCT values, each doubled under a NULL organization: six
271290
// conflicting groups once the declared key folds NULL into the global

‎packages/drivers/driver-sql/src/sql-driver-13015-shadow-carried-index-drift.test.ts‎

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ import {
5656
type PhysicalIndex,
5757
} from './schema-drift.js';
5858
import { MYSQL_CELL, declareDialectCell } from './live-dialect-matrix.testkit.js';
59+
import { asPre18TenantColumn } from './pre-18-tenant-column.testkit.js';
5960

6061
// ─────────────────────────────────────────────────────────────────────────
6162
// Fixtures — the shapes `introspectIndexes` produces on MySQL
@@ -230,6 +231,26 @@ const orgUniqueOn = (name: string) => ({
230231
indexes: [{ fields: ['v'], unique: 'organization' as const, name: `uniq_${name}_org_v` }],
231232
});
232233

234+
/**
235+
* The same unique in its NULL-safe (18.0) form. ADR-0131 D1: a
236+
* `unique: 'organization'` declaration now keys the PLAIN tenant column, so a
237+
* COALESCE shadow — and a raw one reported against it — exists only where an
238+
* input already carries `nullSafeColumns` (honoured verbatim): a database
239+
* built before 18.x, whose tenant column is also nullable
240+
* (`asPre18TenantColumn`).
241+
*/
242+
const pre18OrgUniqueOn = (name: string) => ({
243+
...orgUniqueOn(name),
244+
indexes: [
245+
{
246+
fields: ['organization_id', 'v'],
247+
unique: true as const,
248+
nullSafeColumns: ['organization_id'],
249+
name: `uniq_${name}_org_v`,
250+
},
251+
],
252+
});
253+
233254
declareDialectCell(MYSQL_CELL, 'shadow-carried index drift (#13015)', (cell) => {
234255
describe('shadow-carried UNIQUE against the live MySQL catalog (#13015)', () => {
235256
let driver: SqlDriver;
@@ -304,8 +325,9 @@ declareDialectCell(MYSQL_CELL, 'shadow-carried index drift (#13015)', (cell) =>
304325
*/
305326
it('survives "os migrate apply --allow-destructive" over every reported entry', async () => {
306327
driver = new SqlDriver(cell.config());
307-
const obj = orgUniqueOn('os13015_apply');
328+
const obj = pre18OrgUniqueOn('os13015_apply');
308329
await driver.initObjects([obj]);
330+
await asPre18TenantColumn(driver, 'os13015_apply');
309331
await driver.disconnect();
310332

311333
// A SECOND driver: `runtimeCreatedIndexes` starts empty, so the ledger
@@ -355,8 +377,9 @@ declareDialectCell(MYSQL_CELL, 'shadow-carried index drift (#13015)', (cell) =>
355377
*/
356378
it('reports and repairs a shadow that hashes the raw columns', async () => {
357379
driver = new SqlDriver(cell.config());
358-
const obj = orgUniqueOn('os13015_stale');
380+
const obj = pre18OrgUniqueOn('os13015_stale');
359381
await driver.initObjects([obj]);
382+
await asPre18TenantColumn(driver, 'os13015_stale');
360383
const knex = (driver as any).knex;
361384
const indexName = 'uniq_os13015_stale_org_v';
362385
const shadow = hashShadowColumnFor(indexName);

‎packages/drivers/driver-sql/src/sql-driver-15479-shadow-plain-unique-duplicates.test.ts‎

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ import { describe, it, expect, afterEach } from 'vitest';
4545
import { SqlDriver } from '../src/index.js';
4646
import { isHashShadowColumn } from './schema-drift.js';
4747
import { MYSQL_CELL, declareDialectCell } from './live-dialect-matrix.testkit.js';
48+
import { asPre18TenantColumn } from './pre-18-tenant-column.testkit.js';
4849

4950
/** 900 chars — comfortably over the 768-char keyable ceiling, so `v` is TEXT. */
5051
const LONG = 'p'.repeat(900);
@@ -73,14 +74,27 @@ const globalUniqueOn = (name: string) => ({
7374
indexes: [{ fields: ['v'], unique: 'global' as const, name: `uniq_${name}_v` }],
7475
});
7576

76-
/** The CONTROL shape: an organization-scoped unique, i.e. the NULL-safe arm. */
77+
/**
78+
* The CONTROL shape: an organization-scoped unique in its NULL-safe (18.0)
79+
* form, i.e. the NULL-safe arm. ADR-0131 D1: a `unique: 'organization'`
80+
* declaration now keys the plain tenant column, so only an input that already
81+
* carries `nullSafeColumns` (honoured verbatim) still reaches this arm — the
82+
* shape a database built before 18.x holds, nullable tenant column included.
83+
*/
7784
const orgUniqueOn = (name: string) => ({
7885
name,
7986
fields: {
8087
organization_id: { type: 'string' },
8188
v: { type: 'text', maxLength: 1024 },
8289
},
83-
indexes: [{ fields: ['v'], unique: 'organization' as const, name: `uniq_${name}_v` }],
90+
indexes: [
91+
{
92+
fields: ['organization_id', 'v'],
93+
unique: true as const,
94+
nullSafeColumns: ['organization_id'],
95+
name: `uniq_${name}_v`,
96+
},
97+
],
8498
});
8599

86100
declareDialectCell(MYSQL_CELL, 'hash-shadow plain unique over duplicates (#15479)', (cell) => {
@@ -130,6 +144,9 @@ declareDialectCell(MYSQL_CELL, 'hash-shadow plain unique over duplicates (#15479
130144
// declares it now, so the cast is gone and these two calls are checked
131145
// like any other.
132146
await driver.initObjects([{ ...meta, indexes: [] }]);
147+
// A NULL-organization row exists only on a database built before 18.x
148+
// (ADR-0131 D1 makes a fresh tenant column NOT NULL).
149+
if (row.organization_id === null) await asPre18TenantColumn(driver, meta.name);
133150
const knex = (driver as any).knex;
134151
await knex(meta.name).insert([
135152
{ id: 'a', ...row },

0 commit comments

Comments
 (0)