Repository navigation
Commit ae4dfaf
fix(driver-turso,cli): the v18 organization-ownership ceremony runs on a Turso remote database (#23843)
Fixes #23816
Clause-②: yes (the remote face's refused drift and migration doors
become accepted ones; triage: not the protocol-first class unless a spec
key moves)
## Summary
The v18 organization-ownership ceremony (`os migrate
organization-ownership`, the plan and `--apply --backup-taken`) now
completes on a Turso database reached over the REMOTE transport, which
is the transport every hosted tenant uses. It was measured end to end on
a real `sqld` (libsql-server 0.24.32), reached as
`libsql://127.0.0.1:PORT?tls=0`.
Two things stopped it, and both are in this repository:
1. **Deferred DDL.** Both ceremony boots armed `deferSchemaDdl`
unconditionally, and the remote face refuses that. `bootSchemaStack`
gains `deferredDdlUnsupported: 'skip-schema-sync'`. When the PRIMARY
datasource resolves to the Turso remote transport (the boot asks
`TursoDriver.detectMode` about the URL it resolved), the boot passes
`skipSchemaSync` to the standalone stack instead of arming the deferral
on that driver.
- No DDL is attempted, so the plan stays read-only.
- Every object reaches the driver through the DDL-free
`registerObjectMetadata`; `measureComposedCoverage` binds a composed
host's objects the same way on such a boot.
- Every OTHER datasource the boot connects is still armed.
- Only the ceremony opts in. Every other `os migrate` command still
refuses on the remote face: their dry runs list held-back schema work,
which a boot that holds nothing back cannot list.
2. **Drift detection and migration entries.** These are the driver-turso
remote face changes.
- `detectManagedDrift()` answers. The verdict is the shared differ's.
Its physical-schema reads go to the remote catalog: `pragma_table_info`,
`sqlite_master`, `pragma_index_list` / `pragma_index_info`, and the
ADR-0120 D4 duplicate probe.
- The declarations it diffs against are recorded by every remote schema
door and by `registerObjectMetadata`.
- `applyMigrationEntries()` applies `drop_index`, `create_index`,
`recreate_index`, `replace_unique_index` and `drop_column`. Each entry
runs as ONE libSQL `write` batch:
- a `drop_column` drops the indexes keyed on its column in the same
transaction;
- a re-create that fails leaves the old index in place;
- an entry is reported `applied` only after a read-back.
- Every other column entry needs the SQLite table rebuild, which this
transport does not run, so it is `skipped` and logged.
- An applied index entry (`create_index`, `recreate_index`,
`replace_unique_index`) is read back by its live unique flag and
key-column order, not by its name alone.
- The remote catalog is read in batched round trips per call: one
`sqlite_master` read, then `pragma_table_info` / `pragma_index_list` in
one read batch, then `pragma_index_info` for the indexes whose DDL does
not parse. Batches are chunked at 200 statements, so it takes 3 round
trips up to 100 tables and one more per further 100.
- The remote face declares `driftReportCaveat`: its drift report cannot
be trusted for a boot verdict yet, because the remote sync creates no
declared `NOT NULL` (#23847) and `os migrate apply` refuses there
(#23848). The local and embedded-replica faces declare `null`.
There is one more change. The remote driver's `config` now carries its
`url`, so the CLI labels a remote database `libsql://host:port`
(credential-free) instead of `better-sqlite3`. The ceremony writes that
label into its plan and refuses an apply whose plan names another
database. While every remote database was labelled `better-sqlite3`,
that refusal could not tell two tenants apart.
The remote face does NOT implement deferred DDL. A remote preview would
also have to preview the remote canonical-temporal and codec-residue
backfills, which have no read-only probe today. The embedded-replica
route was not taken (the card measured it as a dead end).
## Evidence — real `sqld`, `libsql://127.0.0.1:PORT?tls=0`
**The v17 fixture**, built once, snapshotted, and copied fresh for each
run:
- a served `os start` boot against the server: the additive sync on the
remote face, 70 objects;
- then, through a SECOND libSQL connection: `organization_id` plus its
tenant index on `sys_job` and on `sys_metadata_activation`, and two
`sys_job` rows.
**Positive control, unchanged `main` (`1ecb08a1`).** The plan exits 1:
```
{"error":"boot_failed","reason":"boot-failed","detail":"Deferred schema DDL is not supported by the Turso REMOTE transport (this datasource's transport mode is `remote`), so a command that promises a dry run or a confirmation before any schema change cannot keep that promise against it. ..."}
```
**With the fix:** first measured at `39b9abdd`, then re-run at
`cc73b809` after the contract-review round. `cc73b809` is code-identical
to the head `26b6acef`. Both runs read the same:
| step | reading |
|:--|:--|
| `os migrate organization-ownership --json` | exit 0 |
| the plan is read-only | logical dump (every catalog row and every
table's rows, hashed) `c0790dff…` before = `c0790dff…` after, 341
objects / 12 rows |
| the plan's `database` | `libsql://127.0.0.1:38187` (it was
`better-sqlite3` before the label change) |
| `--apply --plan-file … --backup-taken --json` | exit 0 · `complete:
true` · `blocking: 0` · `advisory: 0`; every step `done`; `column-drops`
`{"dropped": 2, "due": 2}` |
| second connection: the columns | `sys_job.organization_id`,
`sys_metadata_activation.organization_id`: absent |
| second connection: their indexes | `idx_sys_job_organization_id`,
`idx_sys_metadata_activation_organization_id`: absent |
| second connection: the marker | `sys_migration` row
`adr-0131-organization-ownership`: `applied_at` and `verified_at` set |
| second connection: the rows | `sys_job`: 2 rows, kept |
An earlier run over a fixture that also put the column on
`sys_audit_log` ended with `blocking 1`. The cause is not the transport:
the composition booted from an empty project declares no object for
`sys_audit_log`, so the step reports `not-declared` there. The same
happens on `file:`. The fixture was corrected to tables that composition
declares.
**Ablation.** Both runs used `scripts/ablation-replace.mjs` (WRAP mode);
each mutation was proven to land on disk, restored to `HEAD`, and the
tree checked clean.
1. **The driver arm removed** (`if (!this.isRemote || this.isRemote)
return super.detectManagedDrift(objects);`). driver-turso was rebuilt,
and `ablation-dist-preflight` found the marker in `dist/index.js` and
`dist/index.mjs`.
- The two remote driver suites: 18 of 18 red.
- The sqld e2e: red, `{"error":"apply_failed","detail":"Unable to
acquire a connection"}`.
- Restore leg: rebuild, then `--absent` (marker gone from all 6 built
files, tree clean); 19/19 and 1/1 green.
2. **The ceremony's posture removed** from the plan boot. The sqld e2e
is red at the plan with the positive-control refusal (`boot_failed`,
`Deferred schema DDL is not supported by the Turso REMOTE transport …`).
Restored blob = `HEAD`.
The contract-review round added three ablations, made the same way, each
restored to `HEAD` with `git diff HEAD` empty:
3. **The apply boot's `deferredDdlUnsupported: 'skip-schema-sync'`
removed** from `organization-ownership.ts`:
`organization-ownership.remote-posture.test.ts` red, 1 failed (the
apply-boot case).
4. **The exemption line in `DeferSchemaDdlPlugin.arm` removed:**
`schema-migrate.remote-posture.test.ts` red, 2 failed. The no-exemption
control stayed green.
5. **The gate's caveat branch disabled** in
`artifact-boot-migration.ts`: `artifact-boot-migration.test.ts` red, 1
failed (the caveat case). The null-caveat control stayed green.
## Tests (head `26b6acef`)
- `@objectstack/driver-turso`: typecheck exit 0; `vitest run` 90 files,
2415 passed / 33 skipped.
- `turso-remote-drift-detection.test.ts` also carries:
- the embedded-replica control: a replica-mode `TursoDriver` over a
drifted file reports the destructive `drop_column`, for the no-argument
and the explicit-objects call. This is the same-layer replacement for
the replica case of the deleted
`turso-remote-drift-detection-refusal.test.ts`;
- the `driftReportCaveat` case: a sentence naming `NOT NULL` on the
remote face, `null` on local and replica;
- the round-trip case: 3 client calls for 1 table and for 5 tables.
- `turso-remote-drift-detection.test.ts` (was
`turso-remote-drift-detection-refusal.test.ts`): the remote face over a
real `@libsql/client` `file:` client, asserted EQUAL to the local Knex
face over the same file. It covers:
- a clean database synced by each remote door;
- four drifted shapes;
- the `registerObjectMetadata` registry;
- an absent table.
- `turso-remote-migration-entries.test.ts`: each outcome is read back
through a second connection. It covers:
- `drop_column` with its indexes, and the rolled-back refusal;
- `drop_index`, `create_index` and `recreate_index`, with the D4
re-probe over duplicates;
- the skipped rebuild-only entry;
- convergence;
- an overlay-index rebuild through the two members;
- the config label.
- `turso-remote-inherited-members.test.ts`: `applyMigrationEntries` is
no longer in the refused set.
- `@objectstack/cli`: typecheck (and the test-layer typecheck) exit 0;
`--project unit` 299 files, 4453 passed. A targeted `--project
integration` run (20 files: every `organization-ownership*` and
`schema-migrate*` integration suite plus `unmanaged-tables`, the sqld
e2e included with `OS_TEST_SQLD_BIN` set) had 182 passed, 0 skipped. It
ran locally because this diff touches the boot path.
- `organization-ownership.remote-posture.test.ts` (new, unit tier, runs
in CI) pins two things: both ceremony boots pass
`deferredDdlUnsupported: 'skip-schema-sync'`, and the apply reads its
ledger from the catalog when the boot skipped schema sync.
- `schema-migrate.remote-posture.test.ts` (new, unit tier, runs in CI):
- `transportRefusesDeferredDdl` over 8 resolved databases;
- the primary exemption, which leaves only the engine's default driver
un-armed, at init and on a later `registerDriver`;
- the no-exemption and other-refusing-driver controls.
- `artifact-boot-migration.test.ts`: the caveat case (warn with the
reason; no detect, no apply; the boot proceeds) and the null-caveat
control (the same report refuses the boot).
- `organization-ownership.remote-face.integration.test.ts` (new) runs
the real command, plan then apply, against a `sqld` it starts itself. It
is skipped unless `OS_TEST_SQLD_BIN` names a sqld binary: it ran green
here, and it reads `1 skipped` without the variable.
- `schema-migration-plugins.test.ts` gains a case for the DDL-free
binding.
- `@objectstack/runtime`: typecheck exit 0; `vitest run` 402 files, 6787
passed / 19 skipped.
- Gates: `node scripts/pm/dispatch-gates.mjs --ran` reports 70 derived
families, 69 run with exit 0, and 1 NOT MEASURED: `pnpm
check:dual-build-cjs-loads` exited 3, PREREQUISITE NOT MET, because it
needs a whole-repo `pnpm build` and only the closure was built here; CI
runs it. Two families first read exit 3 for environment reasons and were
re-run alone with exit 0:
- `check-plugin-teardown-shape --self-test`: the shallow clone lacked
its pinned fixture commit;
- `check:i18n-coverage`: contention.
- eslint over the changed `.ts` files: exit 0.
## Acceptance notes
- **File surface.** The claim named
`packages/cli/src/utils/organization-ownership-*.ts`. The boot posture
lives one file over: `packages/cli/src/utils/schema-migrate.ts`
(`bootSchemaStack`, the option and the primary exemption) and
`packages/cli/src/utils/schema-migration-plugins.ts`
(`measureComposedCoverage`'s DDL-free binding), with that file's test.
- **The `OS_ARTIFACT_URL` boot migration gate is unchanged on a Turso
remote primary.** It still answers "cannot judge": it warns and boots,
with no `detectManagedDrift` call, no apply and no refusal. It decides
that from the driver's declared `driftReportCaveat`, not from the
transport-mode string. The contract review measured why it must not
judge yet: the remote sync creates no declared `NOT NULL`, so every
`storage.notNull` field would read as destructive drift, and `os migrate
apply` refuses on that transport. The gate judges remote databases again
only after the follow-ups #23847 and #23848, which this PR does not
close. The local, embedded-replica and other SQL drivers are unchanged.
- Cloud's overlay-index step (`rebuildTenantOverlayIndex`:
`detectManagedDrift([subject])`, apply the one `recreate_index`, read
again) is the shape the new overlay-index test drives on the remote
face. It has not been run against cloud's code.
- The sqld e2e does not run in CI, because no `sqld` is provisioned
there. One way to provision it is a devDependency on the npm package
that ships the binary (`@sqld/linux-x64`), which is a dependency
decision this PR does not make.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Lr3rg6qTp9r38exA5DHEEH)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 0af559e commit ae4dfaf
17 files changed
Lines changed: 2063 additions & 332 deletions
File tree
- .changeset
- packages
- cli/src
- commands/migrate
- utils
- drivers/driver-turso/src
- runtime/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
0 commit comments