You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 2491729
Browse filesBrowse the repository at this point in the historyBrowse files
fix(driver-turso): remote mode constructs without better-sqlite3 — its Knex base is built with no connection (#20054) (#20073)
Fixes#20054
Clause-②: no (narrowing)
## What this changes
A remote `TursoDriver` (`libsql://`, `https://`, `wss://` and the other
remote schemes) now constructs, connects and runs CRUD with
`better-sqlite3` not installed. That is the install `package.json` (an
optional peer) and the README's "Dependencies by Mode" table give remote
mode, for Vercel and Edge deployments.
The fix is one arm of `TursoDriver.toKnexConfig`. The `SqlDriver` base
constructor always builds a Knex instance. Remote mode used to hand it
`{ client: 'better-sqlite3', connection: { filename: ':memory:' } }`.
knex's `Client` constructor loads the dialect's native driver
(`initializeDriver`, which runs `require('better-sqlite3')`) and builds
a pool only when the config carries a `connection`. Remote mode now
passes `{ client: 'better-sqlite3', useNullAsDefault: true }` with no
`connection`: the SQLite dialect's compiler, with no native module, no
pool and no private `:memory:` database. The client is still spelled
`better-sqlite3`, so `isSqlite` and every dialect-keyed rule the remote
arms borrow from the base answer as before. `SqlDriver` is not touched,
and no export changes.
Files:
- `packages/drivers/driver-turso/src/turso-driver.ts`: the remote arm of
`toKnexConfig` and its comment. Also the text of two remote refusals
this change made false (see Deviations).
-
`packages/drivers/driver-turso/src/turso-remote-no-native-driver.test.ts`:
new pin.
- Two refusal test docblocks now use the past tense for the placeholder
(comments only).
- `.changeset/20054-turso-remote-no-native-driver.md`:
`@objectstack/driver-turso` minor, BREAKING (patch round 1).
## Measurements
All readings were taken in `objectstack-ai/objectstack`. Base:
`3557f85fa5`. Head: this branch. The probes used dist built at each
tree. "Absent" means `better-sqlite3` made unresolvable in a separate
node process (a `Module._resolveFilename` hook). The pin uses a
`Module._load` hook instead.
**H1 (the throw), confirmed at base.** knex 3.3.0 `lib/client.js`: `if
(this.driverName && config.connection) this.initializeDriver();`, which
calls `Client_BetterSQLite3._driver()` (`require('better-sqlite3')`).
| construction, module absent | base | head |
|:--|:--|:--|
| remote `libsql://probe-db.example.turso.io` | throws `Knex: run $ npm
install better-sqlite3 --save` | constructs, `transportMode = 'remote'`
|
| local `:memory:` (control) | throws, same message | throws, same
message |
| local `file:` (control) | throws, same message | throws, same message
|
| all three, module present | construct | construct |
**H2: what the remote face uses `this.knex` for.** I wrapped
`driver.knex` in a Proxy that records every property read and call after
construction. The remote driver ran over a real `@libsql/client` `file:`
client.
- Construction: the `SqlDriver` constructor's `knex(config)` call, and
`installQueryTiming` (`this.knex.on` for three events). Read from the
source, not recorded.
- 22 remote-armed doors: `connect`, `checkHealth`, `initObjects`,
`syncSchema`, `syncSchemasBatch`, `create`, `bulkCreate`, `find`,
`findOne`, `count`, `aggregate`, `update`, `upsert`, `bulkUpdate`,
`updateMany`, `execute`, `paginationTieBreaker`, `bulkDelete`,
`deleteMany`, `delete`, `dropTable` and `disconnect`. Plus the 4
refusals: `beginTransaction`, `detectManagedDrift`,
`planMediaColumnMove`, `setDeferredDdl(true)`. **Zero reads**, at base
and at head.
- The inherited helpers those arms call have no `this.knex` reference in
their bodies (dist, 14 names): `temporalFilterValue`,
`temporalFilterColumnSql`, the three `sqlite*Sql` rules,
`isNonTextColumn`, `formatInput`, `formatOutput`, `computeTenantField`,
`calendarDayUpperBoundRewrite`, `orderKeysFor`,
`registerExternalObject`, `toDateOnly` and `rawStatementFault`.
- Only the `SqlDriver` methods remote mode does not override reach it:
`introspectSchema` (`.raw`), `distinct` (builder),
`findWithWindowFunctions` (builder, `.raw`, `.client`), `analyzeQuery`
and `explain` (builder, `.raw`), and `reclaimSpace` (`.raw`).
- On this state, `previewDeferredSchemaWork`, `flushDeferredSchemaDdl`,
`applyMigrationEntries([])` and `getSchemaSyncStats` made zero reads.
`rotateShards` threw before it reached Knex.
- Remote `disconnect()` never calls `super.disconnect()`, so knex is
never destroyed on the remote face.
So nothing on the remote face needs a live Knex, and nothing needs a
compiling one either. A Knex with no connection was the smallest seam:
it lives in `toKnexConfig`, and `SqlDriver`'s constructor contract stays
as it is.
**H3: the inherited methods, before and after.** A remote face over a
libsql `file:` client holding rows in `probe_t`.
| method | base, module present | head, present | head, absent |
|:--|:--|:--|:--|
| `introspectSchema()` | **resolves `{ tables: {} }`** (silent) |
rejects `Unable to acquire a connection` | same |
| `distinct('probe_t', 'name')` | rejects `DATABASE_ERROR` / 500 | same
| same |
| `findWithWindowFunctions(…)` | rejects raw `SqliteError` `no such
table: probe_t` | rejects `Unable to acquire a connection` | same |
| `analyzeQuery` / `explain` | resolves `{ sql, bindings, client, error:
'EXPLAIN QUERY PLAN … no such table …' }` | resolves, same shape,
`error: 'Unable to acquire a connection'` | same |
| `reclaimSpace()` | **resolves** (vacuumed the private database) |
rejects `Unable to acquire a connection` | same |
| `previewDeferredSchemaWork`, `flushDeferredSchemaDdl`,
`applyMigrationEntries([])`, `getSchemaSyncStats` | resolve, no Knex
read | same | same |
| `rotateShards('probe_t')` | throws, no rotation policy | same | same |
- Loud to silent: **0**. Silent to loud: **2** (`introspectSchema`,
`reclaimSpace`).
- The head readings with the module present and absent are
byte-identical (diffed).
- A side reading: at base, a remote driver that had run
`introspectSchema()` kept the process alive after `disconnect()`.
`timeout 20` killed it (exit 124), because a pooled better-sqlite3
connection was never destroyed. At head, the process exits (no pool is
built).
**H4: in-repo constructions.** `git grep 'new TursoDriver('` on non-test
`.ts` files: 9 hits, 3 of them real calls:
- `createTursoDriver` and the plugin's `onEnable` in
`driver-turso/src/index.ts`;
- the turso arm of
`service-datasource/src/default-datasource-driver-factory.ts`.
The runtime loader adds `new TursoDriverCtor(` in
`runtime/src/turso-driver-factory.ts`. None of the four reads `.knex`.
The datasource factory's `sqlServerVersion` goes through
`driver.execute`, which is the remote transport.
Repo-wide reads of a driver's Knex outside `driver-sql`, non-test:
- `getKnex()`: 0 calls (1 comment). Control: `sql-driver.ts` has 1 hit.
- `.knex` reads: 2 files. `metadata-protocol`'s
`resolveDriverClientName` reads `driver.config.client` first, and that
answer is unchanged (`better-sqlite3`).
`runtime/src/raw-foreign-key-fixture.ts` is used by two SqlDriver
integration tests and not by any Turso test.
## Tests
- New pin `turso-remote-no-native-driver.test.ts`: 10 tests.
- Control: local `:memory:` and `file:` still throw without the module.
The hook counted at least one load attempt, which proves it is live.
- Remote construction without the module: 0 load attempts.
- Remote CRUD through a real `@libsql/client` `file:` client without the
module: 0 load attempts, and the rows read back from that file.
- `introspectSchema`, `distinct` and `findWithWindowFunctions`, each
with the module present and absent. Each call must either fail or answer
from the remote database, never "no tables". The envelope is left to
#20055.
- **Reverse verification.** The fix was committed first. `node
scripts/ablation-replace.mjs` restored `connection: { filename:
':memory:' }` in the remote arm: anchor 1 to 0, blob `227df64b3046` to
`a5b36d46ed8f`. The pin went **6 failed, 4 passed**, the direction
predicted before the run:
- construction and CRUD failed on knex's install error;
- `introspectSchema`, module present, failed on `expected [] to include
'probe_t'`;
- all three inherited cases with the module absent failed at
construction;
- the two controls stayed green, and so did `distinct` and
`findWithWindowFunctions` with the module present, which were already
loud at base.
After the restore, the blob equals HEAD and `git diff HEAD` is empty.
The test imports `./index.js` (source), so no rebuild was part of either
leg.
- `pnpm --filter @objectstack/driver-turso test`: 67 files, 1510 passed.
`typecheck`: exit 0. `tsc --listFiles` includes all 4 changed `.ts`
files.
- `pnpm --filter @objectstack/driver-sql test`: 184 files passed, 11
skipped. 2845 tests passed, 170 skipped.
- Consumers that construct a `TursoDriver` (targeted files):
- `service-datasource`: `default-datasource-driver-factory`,
`datasource-pool-support`, `turso-bound-secret-authoring` and
`turso-driver-config`. 4 files, 147 passed.
- `runtime`: `turso-driver-factory.convergence` and
`standalone-stack.libsql`. 2 files, 38 passed.
- `dogfood`: `date-bucket-parity-turso`, which builds a real remote
driver. 1 file, 5 passed.
- The CLI's `storage-driver.test.ts` builds a fake `TursoDriver` class,
not this one. It is declared to CI, not run here.
- `pnpm check:driver-conformance`, before (base, in a detached worktree)
and after: `OK — 50 covered cell(s), 0 in the DEBT ledger, 0 exempt`,
both.
**Gates at head `014b58a689`**, derived from the real diff with `node
scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` (merge base `3557f85fa`): 61 commands. The
dispatch's list (derived at `2c1011b`) had 54, and all 54 are included.
The 7 new ones are `check-adr-0087-registration` x2,
`check-empty-changeset` x2, `release-rehearsal-clone --self-test`,
`check:objectui-changeset` and `check:pm-changeset-deadline-census`.
- **59 exit 0.**
- **2 NOT MEASURED** (exit 3, `PREREQUISITE NOT MET`):
`check:dual-build-cjs-loads` and `check:type-check-debt`. Both read the
whole workspace's built output, and this worktree built only the
closures listed above. lint.yml builds the whole closure first, so CI
measures them.
- `check:dts-closure` first exited 1. It named 26 packages whose `dist/`
I had built locally with `OS_SKIP_DTS=1` to run the consumer suites. I
removed those `dist/` directories and re-ran it: exit 0, `46/46 declared
declaration file(s) present across 8 package(s)`, `driver-turso` among
them.
- The `--ran` reconciliation with exit codes recorded: `61 derived, 59
run, 2 NOT-MEASURED, 0 UNRUN`.
- `node scripts/check-issue-citations.mjs --base 3557f85`: exit 0, 3
citations resolve.
- Out-of-gate control-byte scan of the 5 changed files: no match.
- Not run locally, and owned by CI: the repo-wide `pnpm lint`, the CLI
unit layer, and the path-scheduled CI jobs `dispatch-gates` names.
## Deviations
- **Two runtime strings in `turso-driver.ts` outside the claimed arm.**
The `NOT_IMPLEMENTED` / 501 messages of the remote
`detectManagedDrift()` and `planMediaColumnMove()` said "in remote mode
that connection is a placeholder in-memory database holding none of this
datasource's tables". This change made that false. They now say remote
mode has no Knex connection. Each still gives a reason that holds at
head, measured by calling the inherited methods on a head remote driver:
- the no-argument drift call answers `[]` from the empty
`managedObjectFields`;
- the media planner answers an empty scan;
- an explicit-objects drift call now fails on `hasTable`.
First sentences, codes and statuses are unchanged, so both existing pins
stay green. Their docblocks and the two comments on the overrides were
edited to match.
## Acceptance notes
- **`introspectSchema()` on a remote driver now fails with knex's
`Unable to acquire a connection`.** That message points at connectivity.
The real answer belongs to #20055, which will either route the call to
the transport or refuse it with `NOT_IMPLEMENTED`. #20055 remains open
for all three methods. The callers I traced (not run):
- `ExternalDatasourceService.testConnection` now reports `ok: false`
with that message, where it reported `ok: true, tableCount: 0`;
- the boot validation sweep now rows a federated object on a
remote-Turso datasource as `unreachable` (logged at warn, boot
continues), where it rowed `missing_table` against the empty answer.
The datasource admin's `testConnection` prefers `checkHealth`, so it is
unaffected.
- The pending changesets `19845-turso-remote-drift-detection-refusal.md`
and `19894-turso-remote-media-column-move-refusal.md` describe the
placeholder in the present tense. They narrate what those refusals
replaced. I left them alone, and this PR's changeset says the
placeholder is gone.
- **Banner.** `os serve`'s banner reads a registered driver's `config`.
For a remote Turso driver, `describeDriverConnection` returned
`:memory:` at base (a false address) and returns `undefined` at head.
`describeRegisteredDriver` then prints `(unknown)`. The renderer was
measured and the fallback traced.
- The CLI's migrate `describeDb` falls back to the client name. At head
it would print `better-sqlite3` for a remote Turso datasource instead of
`:memory:`. Traced only.
## Semver
`@objectstack/driver-turso: minor`, `fix(driver-turso)!:`, `Clause-②: no
(narrowing)`, with an ADR-0087 `not-required
(no-migration-prescription)` disposition in the changeset. This was
ruled in patch round 1, below.
## Patch round 1
The PM seat answered the open question with **B**, for consistency with
this lane's precedents on the same driver face: #19893 (PR #19971) and
#19894 (PR #20014). On a remote driver, `introspectSchema()` and
`reclaimSpace()` resolved before and reject now. That is a call the
published driver answered and no longer answers, so it is declared,
whatever the old answer's quality.
What changed, in `.changeset/20054-turso-remote-no-native-driver.md`
only (commit `93d49b886e`, no code change):
- the bump is now `minor` and the summary reads `fix(driver-turso)!:`;
- the declaration line is now `Clause-②: no (narrowing)`;
- a **BREAKING** paragraph names `introspectSchema()` (used to resolve
`{ tables: {} }`) and `reclaimSpace()` (used to resolve). On a remote
driver both now reject with knex's `Unable to acquire a connection`;
- `findWithWindowFunctions()`, `analyzeQuery()` / `explain()` and
`distinct()` are listed apart, as changing their error wording only or
not at all;
- an ADR-0087 `not-required (no-migration-prescription)` disposition in
the #19893 shape. No key, schema, object definition or stored
representation moves, only which calls a remote driver answers. #20055
carries the per-method answer or refusal.
The version axis is unchanged:
`.changeset/19894-turso-remote-media-column-move-refusal.md` already
bumps `@objectstack/driver-turso` `minor` in the pending release.
Gate verdicts at `93d49b886e`, against merge base `3557f85fa5`:
- `check-changeset-no-major`: ``✓ This diff introduces no `major`
bump.`` With this body as the `--event` payload: ``✓ LEVEL AXIS: this PR
declares clause-② `no (narrowing)`, and no package whose
`packages/**/src/**` it moves is graded `patch`.``
- `check-adr-0087-registration`: ``✓ check-adr-0087-registration: 1
declared-breaking changeset(s), each carrying an ADR-0087 disposition.``
The changeset is tagged ``[BREAKING+bang+clause-②-narrowing]
not-required (no-migration-prescription)``.
- `check-empty-changeset`: `✓ No empty-frontmatter changeset introduced
by this diff (1 declaring changeset(s) added).`
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Bvd69VPa6puiNzzPUroDBx)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
fix(driver-turso)!: a REMOTE `TursoDriver` no longer needs `better-sqlite3` installed, and the two inherited calls that answered from its private in-memory database now reject (#20054)
6
+
7
+
Clause-②: no (narrowing)
8
+
9
+
`package.json` declares `better-sqlite3` an optional peer, and the README tells a remote-only deployment (Vercel, an Edge runtime) that it does not need it. The code did not keep that promise. With `better-sqlite3` absent, `new TursoDriver({ url: 'libsql://…' })` threw knex's `Knex: run $ npm install better-sqlite3 --save` error at construction, before any remote call.
10
+
11
+
The cause: remote mode handed the `SqlDriver` base a `better-sqlite3` Knex config on `:memory:`, and knex loads a dialect's native driver whenever the config carries a `connection`. Remote mode now builds that Knex instance with no `connection`. It loads no native module, opens no pool and holds no private in-memory database. With `better-sqlite3` absent, a remote driver constructs, connects and runs CRUD through `@libsql/client`.
12
+
13
+
**BREAKING** — two calls on a remote driver that resolved before now reject. This is an accept-set narrowing on a published driver, shipped as `minor` under the repo's launch-window convention for breaking changes (`scripts/check-changeset-no-major.mjs`). The calls are `SqlDriver` methods that remote mode does not override, and they now reject with knex's `Unable to acquire a connection` error:
14
+
15
+
-`introspectSchema()` used to resolve `{ tables: {} }`, "no tables", whatever the remote database held;
16
+
-`reclaimSpace()` used to resolve.
17
+
18
+
Both old answers came from the private in-memory database, not from the remote one. The per-method answer or refusal for these inherited calls is carried by #20055.
19
+
20
+
**Error wording only, not the narrowing.** On a remote driver:
21
+
22
+
-`findWithWindowFunctions()` still rejects. The error is now knex's `Unable to acquire a connection` instead of a missing-table error from the in-memory database.
23
+
-`analyzeQuery()` and `explain()` still resolve the compiled SQL with an `error` field. That field now carries knex's message instead of a missing-table error.
24
+
-`distinct()` answers the same `DATABASE_ERROR` / 500 as before.
25
+
26
+
**Unchanged:**
27
+
28
+
-**Local and embedded-replica modes.** Their `toKnexConfig` arms are untouched and still run on `better-sqlite3`, so a local driver (`:memory:` or a `file:` url) still fails at construction without it.
29
+
-**Every call remote mode sends to `RemoteTransport` behaves as before**, including raw SQL through `execute()`. None of them used the Knex instance.
30
+
-**The `NOT_IMPLEMENTED` / 501 refusals of `detectManagedDrift()` and `planMediaColumnMove()` on a remote driver** still refuse, with the same code and status. Their messages no longer say that remote mode's Knex connection is a placeholder in-memory database; they say that remote mode has no Knex connection.
31
+
32
+
<!-- adr-0087: not-required (no-migration-prescription) A narrowing of which calls a remote `TursoDriver` answers: no key, spec symbol, Zod schema, object definition or stored representation is added, removed or renamed — `TursoDriverConfig` and both `TursoConfigSchema` copies are untouched, and `introspectSchema` / `reclaimSpace` keep their names and signatures. What moves is only that a remote driver no longer answers those two calls from a private in-memory database, so `objectstack migrate meta` has nothing to visit and there is no tombstone to mint. The per-method answer or refusal on the remote face is carried by #20055. -->
0 commit comments