Repository navigation
Commit d5d8146
feat(driver-sql,driver-turso,driver-memory,objectql,cli,metadata-protocol): plain organization key part in new unique indexes; the differ reads COALESCE(tenant) as tenant on a certified database (18.x step 7b-1 + 7b-2) (#24893)
Part of #24583
Clause-②: yes (widening). New DDL changes form.
- Nothing is renamed, and no declared organization composite is dropped
or rebuilt by the boot, `os migrate plan` or `os migrate apply`. On
SQLite the ownership ceremony's NOT NULL step rebuilds the tables it
tightens, and their composites come back plain, enforcing the same
constraint.
- One index is re-keyed. The `sys_setting` row-identity index, which the
metadata protocol rebuilds on every boot, keys the plain
`organization_id` from the next boot on. This is probe-first and a pure
relaxation, so it cannot fail on existing rows.
- Some tenant columns are still nullable: every table of a database
18.0.0 created, and a table the ceremony reported. On those tables a
newly declared organization unique no longer constrains
NULL-organization rows, and the in-memory driver exempts a record with
no organization.
Metadata-protocol: semantic
A declared `unique: 'organization'` index, and a field-level
organization unique on a tenant-scoped object, change what they enforce
where the tenant column is nullable. Nothing an author writes is added
or refused. The changeset grades all six packages `minor`. There is no
BREAKING banner and no ADR-0087 marker, because the change is a
widening.
<sub>Clause-② text corrected by the epic seat on 2026-10-20 (about
16:15Z) after the contract review's B1 (record 6293077099). It mirrors
`.changeset/24583-plain-organization-key-part.md`. The first bullet's
precision clause (re-review 6295919095, R1) was folded in at
8769400.</sub>
18.x step 7, parts 7b-1 + 7b-2 (claim 6289404450, governing record
6285122667). 7b-0 (#24940) has merged as 5e9b100, and this branch has
merged `main` at that commit, so the diff is this part alone.
## What changes
**7b-2: new DDL keys the plain organization column** (ADR-0131 D1: the
tenant column is NOT NULL).
- `uniqueIndexesFromFields`, `normalizeDeclaredIndex` (`unique:
'organization'`) and `legacyUniqueReplacements` no longer mark the
tenant key part NULL-safe. A field-level organization unique is `UNIQUE
(organization_id, f)`. `buildIndexName` is unchanged, so no index is
renamed.
- The same applies on every emitting path:
- the sql-driver index emitters and the MySQL hash shadow, whose new
shadows hash the plain column;
- Turso, local and remote (it shares the generators);
- `memory-unique-constraint.ts`, where a NULL organization now exempts
the row like any other NULL key part;
- `os generate migration`, which now emits the organization composite it
used to list as NOT EMITTED (`GLOBAL_TENANT_KEY` removed);
- the `sys_setting` row-identity migration: `organization_id` is plain
and `COALESCE(user_id, '')` stays. Its duplicate query lists only rows
the index refuses.
- The duplicate pre-flight (`probeNullSafeUniqueDuplicates`, local and
Turso remote) leaves out rows with a NULL plain key part. The index
never refuses those rows.
- `__global__` as the autonumber counter key is untouched (7b-3's
ground).
**7b-1: the differ reads `COALESCE(tenant)` ≡ `tenant`, in both
directions, where the tenant column holds no NULL.**
- `diffManagedIndexes` takes `tenantKeyCertified`. When it is set, the
tenant key part's form drops out of the comparison identity. The column
set, uniqueness and other key parts still have to match.
- `SqlDriver.detectTableIndexDrift` sets it from two witnesses:
- **The tenant column is physically NOT NULL.** This is self-proving,
and 7b-0 guarantees it on fresh tables. The ceremony's SQLite rebuild
and its PostgreSQL/MySQL `ALTER` produce it too.
- **The database is certified.** The engine hands every driver a
closure, `ObjectQL.isOrganizationOwnershipCertified()`: a verified
`sys_migration` ownership row at the current `shapeGeneration` or above.
This is the same reading the boot guard boots on, and it covers a birth
marker or the ceremony's marker.
- The driver asks for certification lazily, and only for a table that
physically carries the COALESCE tenant key part. A failed read is
reported once and read as "not certified". This is the
`fileColumnsMoved` supply-seam precedent, so the driver never learns the
ledger's vocabulary and there is no layering break.
- The COALESCE form is still read: `enforcedIndexKey`,
`isSyncReproducibleIndex`, `isRuntimeManagedIndex` and the shadow-key
reader are unchanged. An input carrying `nullSafeColumns` is still
honoured verbatim.
## Measured
- **The ceremonied 18.0 fixture** (`ceremoniedFixture`):
- On SQLite, HEAD's own ceremony NOT NULL step rebuilds 62 tables
through the drift apply door, and that rebuild re-keys their indexes
plain. `sys_email_template` stays nullable (with `sys_sso_provider` and
one rotation shard) and keeps its 18.0 COALESCE index, so on the fixture
as-is the **certification** witness is what carries it.
- PostgreSQL/MySQL get NOT NULL by `ALTER` and a Turso remote store not
at all, so there every 18.0 index survives the ceremony. A SQLite store
certified by an earlier 18.x ceremony keeps them too. The pin therefore
also restores all **13** COALESCE indexes exactly as 18.0 wrote them.
- On both shapes, HEAD's differ reports **0 `recreate_index` and 0
destructive drift**. The 12 declared composites survive the boot byte
for byte. `sys_setting`'s index is rebuilt by its own runtime migration,
which rebuilds it every boot anyway.
- **A database born on the published 18.0.0:**
- Measured live: the published 18.0.0 boots a fresh SQLite file, then
HEAD boots it with 0 errors and 0 refusals, and `os migrate plan`
reports 0 changes over 73 tables.
- Pinned as the equivalence file's third leg.
- **Fresh install:** every organization unique index is plain, with 0
COALESCE and no index drift.
-
`packages/qa/dogfood/test/upgrade-18x-index-equivalence.dogfood.test.ts`
pins all of the above: 10/10.
- **Ablation**, via `scripts/ablation-replace.mjs` on `schema-drift.ts`:
- With the equivalence switched off, `ablation-dist-preflight` showed
the marker in 2 built files. The fixture run went red: 2 failed / 5
passed, both drift legs, with 26 `recreate_index` lines in the output.
- With only the certification witness switched off, 3 legs went red, the
18.0.0-born leg among them.
- Restore: the blob equals HEAD, the dist marker is gone, and the tree
is clean.
- Measured before the post-7b-0 merge, which changed no production file
of this part.
- **MySQL hash shadow, live on MySQL 8.0.46** (a local container):
- A fresh shadow's generation is
`unhex(sha2(concat(organization_id,0x1f,v),256))`, which is plain.
- An 18.0 COALESCE shadow is `recreate_index`/destructive when
uncertified and gives no drift when certified.
- A re-boot leaves the generated column's expression and the rows
untouched.
- **No data migration and no column rebuild.** A shadow is regenerated
only when its index has already been dropped (`stale`); the column is
derived, so no data moves.
## Proof
Measured on head 8769400 (`main` at 5e9b100 merged in).
- R1: `pnpm dev:<app> -- --fresh` on a random port for showcase, crm and
todo. Each one:
- booted on an empty database;
- answered health 200 and seeded-admin sign-in 200;
- logged 0 error lines and 0 schema-drift lines.
- R2 + R3: `upgrade-18x-harness`, the equivalence file and
`organization-ownership-boot-refusal-order` gave 26/26.
- R4: built CLI `run-dev.js serve` under `OS_TENANCY_POSTURE=isolated`,
on a throwaway app declaring the real `@objectstack/organizations`
(invite-only membership).
- `Tenancy: isolated`, and Organizations mounted.
- Health 200 and discovery 200.
- 0 error lines and 0 refusals.
- In the store: 13 organization unique indexes, 0 COALESCE, and 0 NULL
rows across 29 tables carrying `organization_id`.
- Step 4's 7 walled e2e cases (`OS_TEST_TIERS=nightly`): 7/7.
- Suites:
| Package | Result |
|---|---|
| driver-sql | 3734 passed, 0 failed |
| driver-turso | 2550 passed |
| driver-memory | 1757 passed |
| objectql | 8641 passed, 1 timeout (see below) |
| plugin-security | 4234 passed, 2 timeouts (see below) |
| metadata-protocol `sys-setting-identity-index` | 34 passed |
| cli and every other package | CI Test Core 1/6 to 6/6, all green |
- The three timeouts happened while five suites ran at once, in
`hook-run-as` (objectql) and `shipped-sets-identity-default-row-scope`
(plugin-security). This part does not touch either file. Run alone, they
pass 25/25 and 32/32.
- `typecheck` is clean on all seven packages touched.
- `node scripts/pm/dispatch-gates.mjs --commands`: 85 commands plus the
4 roster gates whose roster sits under a path this diff touches. That is
89 in all, each exit captured before any pipe, and all 89 exit 0.
- `pnpm --filter @objectstack/spec check:generated`: exit 0, with all 14
artifacts up to date.
- `check-governed-merges --branch
origin/claude/issue-24583-7b12-index-plain-form`: exit 0. 0 of 32 paths
are governed.
## Size
2,741 changed lines over 32 files, under the 3,000-line threshold. Now
that 7b-0 is in `main`, this is no longer Tier H by size, and it touches
no governed surface.
- Production: 528 lines.
- Tests: 2,193 lines.
- Changeset: 20 lines.
- Fixtures: 0 lines; no fixture was regenerated.
## 维护者速读(草稿)
- **改了什么**:新建的组织唯一索引改用普通的 `organization_id` 列,不再用
`COALESCE(organization_id, '__global__')`。旧库里已有的 COALESCE
索引,只要库已认证(出生标记或仪式标记)或租户列已 NOT NULL,差异检测就视为同一个索引,不重建、不改名。唯一例外是
`sys_setting` 的行标识索引:元数据协议每次启动都会重建它,下次启动起改用普通列(先探测再替换,只放宽、不会因已有数据失败)。
- **为什么改**:ADR-0131 D1 规定租户列 NOT NULL,没有组织为空的行,`__global__`
桶已无意义。不加等价判断的话,每个升级上来的库每次启动都会被提示「删掉重建」13 个唯一索引。
- **风险与代价(含回滚)**:在租户列仍可空的表上(用 18.0.0
新建的库的所有表、仪式没收紧的表),新建的组织唯一索引不再约束组织为空的行(与其他可空键一致)。除 `sys_setting`
行标识索引外,已有索引一律不动。回滚即还原本 PR,不涉及数据迁移;MySQL 实测无需重建列。
- **席位意见**:
- **你要做的**:7b-0 已合并,本分支已合入 main。净改动 2,741 行,低于 3,000
行,且不触及受治理面。等席位做完增量复核后,按其结论落地。
## Acceptance notes
- The tightening `recreate_index` of ADR-0120 D4 (bare composite →
NULL-safe) is no longer produced by any generator. Its apply path and
pre-flight remain for an op that already carries `nullSafeColumns`, and
the Turso suites pin them that way.
- On an **uncertified** database with a nullable tenant column, an 18.0
COALESCE index is still reported as a destructive `recreate_index`
toward the plain key.
- The finding's message names `os migrate organization-ownership` as the
remedy, never `os migrate apply --allow-destructive` (B3, pinned in
`sql-driver-tenant-key-equivalence.test.ts`).
- No unattended path applies it: boot reconciles only safe changes, and
the artifact gate refuses.
- Such a database is refused at `os serve` until the ceremony certifies
it.
- `os generate migration` emits the tenant column nullable, while the
driver's fresh table is NOT NULL (7b-0). The generated composite is now
the platform's own constraint, which is stronger than the nothing it
emitted before, but the column's nullability still differs from the
driver's. That is 7b-0's ground and is not changed here.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01NnXVu3vF2TKFLMi5Me4CUd
---
_Generated by [Claude
Code](https://claude.ai/code/session_01NnXVu3vF2TKFLMi5Me4CUd)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 4e2c42f commit d5d8146
32 files changed
Lines changed: 2057 additions & 684 deletions
File tree
- .changeset
- packages
- cli
- src
- commands
- utils
- test
- drivers
- driver-memory/src
- driver-sql/src
- driver-turso/src
- metadata-protocol/src/migrations
- objectql/src
- plugins/plugin-security/src
- qa/dogfood/test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
38 | 49 | | |
39 | | - | |
40 | | - | |
| 50 | + | |
41 | 51 | | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
53 | 57 | | |
54 | | - | |
55 | | - | |
56 | | - | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
57 | 62 | | |
58 | 63 | | |
59 | 64 | | |
| |||
72 | 77 | | |
73 | 78 | | |
74 | 79 | | |
75 | | - | |
76 | | - | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
77 | 83 | | |
78 | 84 | | |
79 | 85 | | |
| |||
82 | 88 | | |
83 | 89 | | |
84 | 90 | | |
85 | | - | |
| 91 | + | |
86 | 92 | | |
87 | 93 | | |
88 | 94 | | |
| |||
109 | 115 | | |
110 | 116 | | |
111 | 117 | | |
112 | | - | |
| 118 | + | |
113 | 119 | | |
114 | 120 | | |
115 | 121 | | |
| |||
210 | 216 | | |
211 | 217 | | |
212 | 218 | | |
213 | | - | |
214 | | - | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
215 | 225 | | |
216 | 226 | | |
217 | 227 | | |
| |||
229 | 239 | | |
230 | 240 | | |
231 | 241 | | |
232 | | - | |
| 242 | + | |
233 | 243 | | |
234 | | - | |
| 244 | + | |
235 | 245 | | |
236 | 246 | | |
237 | 247 | | |
238 | 248 | | |
239 | | - | |
| 249 | + | |
240 | 250 | | |
241 | 251 | | |
242 | 252 | | |
| |||
259 | 269 | | |
260 | 270 | | |
261 | 271 | | |
262 | | - | |
| 272 | + | |
263 | 273 | | |
264 | 274 | | |
265 | 275 | | |
| |||
349 | 359 | | |
350 | 360 | | |
351 | 361 | | |
352 | | - | |
353 | | - | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
354 | 365 | | |
355 | 366 | | |
356 | 367 | | |
| |||
362 | 373 | | |
363 | 374 | | |
364 | 375 | | |
365 | | - | |
366 | | - | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
367 | 382 | | |
368 | 383 | | |
369 | | - | |
370 | | - | |
371 | | - | |
372 | | - | |
373 | | - | |
374 | | - | |
375 | | - | |
376 | | - | |
377 | | - | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
378 | 387 | | |
379 | | - | |
380 | | - | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
381 | 398 | | |
382 | 399 | | |
383 | 400 | | |
| |||
425 | 442 | | |
426 | 443 | | |
427 | 444 | | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
428 | 483 | | |
429 | 484 | | |
430 | 485 | | |
| |||
436 | 491 | | |
437 | 492 | | |
438 | 493 | | |
439 | | - | |
| 494 | + | |
440 | 495 | | |
441 | 496 | | |
442 | 497 | | |
| |||
477 | 532 | | |
478 | 533 | | |
479 | 534 | | |
480 | | - | |
481 | | - | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
482 | 544 | | |
483 | | - | |
484 | | - | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
485 | 558 | | |
486 | 559 | | |
487 | 560 | | |
| |||
495 | 568 | | |
496 | 569 | | |
497 | 570 | | |
498 | | - | |
499 | | - | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
500 | 575 | | |
501 | 576 | | |
502 | | - | |
503 | 577 | | |
504 | 578 | | |
505 | 579 | | |
506 | 580 | | |
507 | 581 | | |
| 582 | + | |
| 583 | + | |
508 | 584 | | |
509 | 585 | | |
510 | 586 | | |
| |||
0 commit comments