Skip to content

Commit 5d27d32

Browse files
fix(service-datasource): in development, a SQLite file that cannot be opened or created fails its connect instead of stepping down to an in-memory engine (#24955)
Fixes #24950 Clause-②: no ## What was wrong `resolveSqliteDriver` (`packages/services/service-datasource/src/sqlite-driver-fallback.ts`) probes a native `better-sqlite3` connection in development and steps down to wasm SQLite, then to an in-memory engine. It stepped down on ANY probe failure. Its own module note says the step-down exists for one cause: the native addon loads lazily, so an ABI or load failure stays invisible until the first query. So under `bin/run-dev.js` (`NODE_ENV=development`), a declared SQLite datasource whose FILE could not be opened was handed to the in-memory engine and read `connected`. Measured at the public door before this change (below): `os migrate apply --yes` with a bound `archive` datasource on a path under a regular file printed "neither native nor wasm SQLite available — dev falling back to InMemoryDriver", then `datasource 'archive': connected`, `Applied 0 change(s)`, and exit 0. ## What changed - In development, a probe failure that is the database FILE failing to open or be created is re-thrown, as it is in production. The datasource's connect fails, and `listConnectionStates()` reports it `failed`. This applies to every SQLite datasource, the primary included. - The classification reads the thrown value's `code` only, never its message (`isSqliteFileOpenFailure`, module-private). A file failure is an operating-system code (`E` plus capitals and digits, no underscore: `EEXIST`, `ENOTDIR`, `ENOENT`, `EACCES`, …) or `SQLITE_CANTOPEN`. `SqlDriver.connect()` raises these as its `SqliteDatabaseFileError`. - A native addon that cannot load still steps down to wasm SQLite, with the same warning. Its failures carry `ERR_DLOPEN_FAILED`, `MODULE_NOT_FOUND`, or no code at all, and at the probe's `SELECT 1` they arrive wrapped as `DATABASE_ERROR`. None of these matches. Production is unchanged. - The module note now says which failures step down and why. - One changeset: `@objectstack/service-datasource` `patch`. Not touched: `packages/cli/**` (`bin/run-dev.js` keeps `NODE_ENV=development`), `connect-failure-remedy.ts`, `datasource-connection-service.test.ts`, `packages/drivers/**`. No new env var, flag or gate. ## Premises, measured before the change was committed All readings were taken on this branch's base `feb09db14a` with real `better-sqlite3` 13.0.3 on Node v22.22.0, running the base copy of `sqlite-driver-fallback.ts` byte for byte (`git show BASE:path`) against the real drivers. - **P1 holds, on codes.** Raw `better-sqlite3`: under a regular file, `SqliteError` `SQLITE_CANTOPEN`. A directory as the file: `SQLITE_CANTOPEN`. A missing parent: a code-less `TypeError` ("directory does not exist"). Through `SqlDriver.connect()`, which makes the directory first, the probe's thrown value is `SqliteDatabaseFileError` with code `EEXIST` (under a regular file), `ENOTDIR` (deeper), `ENOENT` (`/proc/missing/`), `EACCES` (mkdir as an unprivileged user in a 0555 directory) or `SQLITE_CANTOPEN` (a directory, or a new file in a read-only directory). Addon-load shapes: a real `process.dlopen` of a non-addon gives `ERR_DLOPEN_FAILED`. A missing `.node` path gives `MODULE_NOT_FOUND`. `bindings`' "Could not locate the bindings file" has no code. A missing `better-sqlite3` package gives knex's install hint, which has no code either. No message text was needed to separate them. - **P2 holds.** An absent file in an existing directory opens native, and the file is created. An absent file under two missing directory levels also opens native: `connect()` makes the directories. Neither hits the re-throw. The standalone host already refuses a primary whose directory it cannot make (`createStandaloneStack`, #23737), and no test or flow in the tree relies on the in-memory rung for an unopenable file. - **P3 holds for the shapes the ruling names, with one refinement.** On base, under a regular file, `/proc/missing/` and an EACCES mkdir, wasm failed on the same path and the in-memory rung answered (`engine: memory`). For a directory as the file, and for a new file in a read-only directory, wasm's `connect()` RESOLVED, so the base answer was `engine: sqlite-wasm` over a file it could not read. For the directory, the first query then failed with `DATABASE_ERROR` (cause `EISDIR`). The native re-throw catches both shapes before wasm. - **Why the re-throw sits at the native rung only.** I first put the same re-throw on the wasm rung too. The ablation showed that no measured path reaches it. `connect()` makes the file's directory before the first statement loads the addon, so even with an addon that cannot load, the native rung refuses every path the wasm rung would. I removed the wasm-rung re-throw (`4da3d4bf0c`). The step-down for an addon that cannot load behaves exactly as before. The wasm rung's catch only moved its best-effort `disconnect()` into the shared `closeQuietly`. ## Pins (new file `src/__tests__/sqlite-driver-fallback-file-open-failure.test.ts`) These use real `better-sqlite3`, real wasm SQLite and real files in a scratch directory. The one thing faked is an addon that cannot load. The fake sits where `better-sqlite3` loads the addon, on the knex better-sqlite3 client's own `acquireRawConnection`, and throws a REAL load error: Node's `ERR_DLOPEN_FAILED`, or `bindings`' own "not built" error. The existing `src/sqlite-driver-fallback.test.ts` mocks all three driver packages and is unchanged; its 11 cases pass. - development: under a regular file, the probe rejects with `SqliteDatabaseFileError`: `EEXIST`, and `ENOTDIR` one level deeper. No warning, never wasm or memory. - development: a directory as the file rejects with `SQLITE_CANTOPEN`. - development: an absent file in an existing directory opens native, and the file is created. - development: an absent file under missing directories opens native. - development: `ERR_DLOPEN_FAILED` steps down to wasm SQLite with `NATIVE_SQLITE_WASM_FALLBACK_WARNING`, and `SELECT 1` answers. - development: the code-less "not built" error steps down to wasm SQLite. - development: an addon that cannot load plus a path under a regular file still rejects, never memory. - production: under a regular file, the resolver returns native unprobed with no warning, and its `connect()` refuses the file (`EEXIST`), as before. - `DatasourceConnectionService` + `createDefaultDatasourceDriverFactory({ dev: true })`: a declared `archive` (`autoConnect: true`) under a regular file reads `availability: 'failed'`, its reason names the file, and no driver is registered. **Ablation**, one leg for the one behaviour changed. The native re-throw was replaced through `scripts/ablation-replace.mjs`: anchor 1 → 0, marker 0 → 1, blob `ec4203705cea` → `fc817e7389d4`. The subject is imported relatively from `src`, so no dist leg applies. Predicted 4 red / 5 green, observed 4 red / 5 green: regular file, directory, addon+regular file and connection-state turned red; both absent-file pins, both addon-load pins and production stayed green. The restore was proved by blob `ec4203705cea` equal to HEAD and an empty `git diff HEAD`. ## Public door, before and after Each run is `bin/run-dev.js migrate apply --yes --database-url file:main.db` in a throwaway project whose `dist/objectstack.json` declares `archive` (`driver: 'sqlite'`) on a path under a regular file. | fixture | before (base behaviour planted in `dist/`) | after (`05e9a8e09a`) | |---|---|---| | an object binds to `archive` | "neither native nor wasm SQLite available — dev falling back to InMemoryDriver", `datasource 'archive': connected`, `Applied 0 change(s)`, exit 0 | `datasource 'archive': connect failed — SQLite database file '…/plain/archive.db' cannot be created: its directory '…/plain' could not be made: EEXIST …` (fail-fast per ADR-0062 D5), exit 1 | | `autoConnect: true`, nothing bound | `connected` on the in-memory engine, exit 0 | WARN `datasource 'archive': connect failed — SQLite database file …` (degraded, as D5 says), exit 0 | For the before leg, the native re-throw was mutated away in `src` through `ablation-replace` (WRAP). Then `service-datasource` was rebuilt and `ablation-dist-preflight` confirmed the marker in 2 built files. After the restore, the package was rebuilt again and `--absent` confirmed the marker gone from all 24 built files, with a clean tree. **The card's own path, `os migrate audit-mail-bodies` over an archive datasource: NOT MEASURED.** The archive examination exists only on PR #24934, which is not merged at this head. The `--apply` boot of that command builds its datasources through this same factory, so the after row above is that boot's mechanism. ## Local verification, on head `d353b3979b` (after merging `origin/main` `ea08ce584c`, which landed #24933) - `pnpm exec turbo run build --filter='@objectstack/service-datasource...' --concurrency=2`: 19/19 successful. - `pnpm --filter @objectstack/service-datasource exec vitest run --maxWorkers=2`: 50 files, 892 tests passed. - `pnpm --filter @objectstack/service-datasource typecheck` (`tsc --noEmit`): exit 0. `--listFiles` includes the new test file. - Gates: `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` (no paths) derived 66 commands at `d353b3979b`. All 66 ran one at a time, each exit captured before any pipe: 66 × exit 0. `--ran` printed: 66 derived, 66 run, 0 NOT-MEASURED, 0 UNRUN. Before the merge, the same 66 at `05e9a8e09a` gave 65 × exit 0, and `check:dual-build-cjs-loads` exited 3 (PREREQUISITE NOT MET: unbuilt packages elsewhere in the tree). Every family that CI runs is read from this PR's check-runs, not re-run here. ## Acceptance notes - **Boundary, left as is: a file that is not a SQLite database still steps down, and wasm then moves it aside.** The native probe answers `SQLITE_NOTADB`, which is not an open failure, so it keeps today's path. Wasm SQLite then renames the file to `archive.db.corrupt-TIMESTAMP`, creates an empty database in its place, and the datasource reads `connected`. The warning says "native better-sqlite3 unavailable (ABI mismatch or not built)", which is false. Measured at the same public door on this head: exit 0, the operator's file moved aside. This is outside the ruling's named route; the report puts it to the seat as an open question. - **Boundary, left as is: the read-only boot treats an unopenable path as an ABSENT file.** `os migrate plan`, and the dry runs that boot with `readOnlyProbe`, pass `sqliteAbsentFile: 'empty-in-memory'`. driver-sql's `resolveSqliteAbsentFileTarget` sees `existsSync(path) === false` for a path under a regular file and opens an empty in-memory SQLite in its place. So `bin/run-dev.js migrate plan` with the bound fixture above prints `datasource 'archive': connected` and plans `+ os24950_note [create_table, 8 column(s)]` into a file that cannot exist, exit 0. This PR does not change that: no probe failure occurs, and the fix belongs in `packages/drivers/**`. The report hands it to the seat. - **Not reachable from here: with an addon that cannot load, wasm's `connect()` resolves over a file it cannot read** (a directory as the file, a new file in a read-only directory). Its first query fails instead. `applyAutoVacuumIncremental` swallows the open error, and the wasm rung's probe is `connect()` alone. This is the addon-cannot-load path, which this card does not change. Carrier: none. - `os db clean` on a target it cannot open now prints the driver's `SqliteDatabaseFileError` text, which names the file. Before, it printed its own "native and wasm both failed to load". `provisionTelemetryDatasource` already swallows any throw, and returns `undefined` exactly as it did for the memory engine. --- _Generated by [Claude Code](https://claude.ai/code/session_011TdZ8GjixNZFLz1xJD9zzK)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 2941a19 commit 5d27d32

3 files changed

Lines changed: 333 additions & 21 deletions

File tree

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
'@objectstack/service-datasource': patch
3+
---
4+
5+
fix(service-datasource): in development, a SQLite datasource whose file cannot be opened or created fails its connect, as in production, instead of stepping down to an in-memory engine
6+
7+
Clause-②: no
8+
9+
`resolveSqliteDriver` probes a native `better-sqlite3` connection in development and steps down to wasm SQLite, then to an in-memory engine, when the probe fails. It used to step down on any probe failure. So a declared SQLite datasource whose file could not be opened read `connected` on a development boot (`NODE_ENV=development`, which the source-tree `bin/run-dev.js` sets). Examples: a path under a regular file, a parent directory the process cannot make, a permission refusal, a directory where the file should be. Every query on it then went to an empty in-memory store, so a maintenance command could report a store it never opened as examined and clean.
10+
11+
- **Now:** a probe failure that is the database file failing to open or be created is re-thrown in development, as in production. The datasource's connect fails with the driver's `SqliteDatabaseFileError`, which names the file and the operating system's answer (`EEXIST`, `ENOTDIR`, `ENOENT`, `EACCES`, … or `SQLITE_CANTOPEN`). `listConnectionStates()` reports it as `failed`. This holds for every SQLite datasource, the primary included, so a development boot whose primary database file cannot be opened now stops and names the file.
12+
- **Unchanged:** a native addon that cannot load (an ABI mismatch, an addon that was never built, a missing `better-sqlite3`) still steps down to wasm SQLite, with the same warning. An absent file in a directory that exists, or that the driver can make, still opens natively and is created. Production behaviour is unchanged.
13+
14+
What to do: if a development boot now refuses a SQLite datasource, fix its `filename`. The boot names the path it could not open.
Lines changed: 230 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,230 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* In development, a SQLite database FILE that cannot be opened or created fails
5+
* the probe, as it does in production. Only a native addon that cannot LOAD
6+
* steps down to wasm SQLite (#24950).
7+
*
8+
* Before: `resolveSqliteDriver` stepped down on ANY probe failure in
9+
* development. A file under a regular file failed native, failed wasm on the
10+
* same path, and was answered by the in-memory rung. So a declared datasource
11+
* that could not answer one query read `connected`, and
12+
* `os migrate audit-mail-bodies` reported a store it never opened as examined
13+
* and clean.
14+
*
15+
* Real `better-sqlite3`, real wasm SQLite and real files in a scratch
16+
* directory, as opposed to `../sqlite-driver-fallback.test.ts`, which mocks
17+
* all three driver packages. The one thing faked is an addon that cannot load.
18+
* It is faked at the seam where `better-sqlite3` loads its addon: the knex
19+
* better-sqlite3 client's `acquireRawConnection` (`new Database(file)`).
20+
* The fake throws a REAL load error: Node's own `ERR_DLOPEN_FAILED`, or the
21+
* `bindings` package's own "not built" error.
22+
*/
23+
24+
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
25+
import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
26+
import { createRequire } from 'node:module';
27+
import { tmpdir } from 'node:os';
28+
import { dirname, join } from 'node:path';
29+
import { SqlDriver } from '@objectstack/driver-sql';
30+
import {
31+
NATIVE_SQLITE_WASM_FALLBACK_WARNING,
32+
resolveSqliteDriver,
33+
type ResolvedSqliteDriver,
34+
} from '../sqlite-driver-fallback.js';
35+
import { DatasourceConnectionService } from '../datasource-connection-service.js';
36+
import { createDefaultDatasourceDriverFactory } from '../default-datasource-driver-factory.js';
37+
38+
let dir: string;
39+
const opened: ResolvedSqliteDriver[] = [];
40+
41+
beforeEach(() => {
42+
dir = mkdtempSync(join(tmpdir(), 'os-24950-'));
43+
writeFileSync(join(dir, 'plain'), '');
44+
});
45+
46+
afterEach(async () => {
47+
vi.restoreAllMocks();
48+
await Promise.all(opened.splice(0).map((r) => r.driver?.disconnect?.().catch(() => {})));
49+
rmSync(dir, { recursive: true, force: true });
50+
});
51+
52+
async function resolveDev(filename: string, warn = vi.fn()): Promise<ResolvedSqliteDriver> {
53+
const resolved = await resolveSqliteDriver({ filename, dev: true, warn });
54+
opened.push(resolved);
55+
return resolved;
56+
}
57+
58+
/** `promise`'s rejection; a failure naming the engine when it resolves instead. */
59+
async function rejectionOf(promise: Promise<ResolvedSqliteDriver>): Promise<any> {
60+
let resolved: ResolvedSqliteDriver;
61+
try {
62+
resolved = await promise;
63+
} catch (error) {
64+
return error;
65+
}
66+
opened.push(resolved);
67+
throw new Error(`resolved to engine '${resolved.engine}' for a file that cannot be opened or created`);
68+
}
69+
70+
/** The driver's refusal of the file: its own error name, the system's code, and the file. */
71+
function expectFileRefusal(error: any, filename: string, code: string): void {
72+
expect(error?.name).toBe('SqliteDatabaseFileError');
73+
expect(error?.code).toBe(code);
74+
expect(error?.filename).toBe(filename);
75+
}
76+
77+
/** A real `ERR_DLOPEN_FAILED`: Node asked to load, as an addon, a file that is not one. */
78+
function realDlopenFailure(): Error {
79+
const notAnAddon = join(dir, 'not-an-addon.node');
80+
writeFileSync(notAnAddon, 'x');
81+
try {
82+
process.dlopen({ exports: {} }, notAnAddon);
83+
} catch (error) {
84+
return error as Error;
85+
}
86+
throw new Error('process.dlopen loaded a file that is not an addon');
87+
}
88+
89+
/**
90+
* The real error `better-sqlite3` throws when its addon was never built: the
91+
* `bindings` package it loads the addon through, asked for an addon that is
92+
* not there. It carries no code.
93+
*/
94+
function realNotBuiltFailure(): Error {
95+
const fromHere = createRequire(import.meta.url);
96+
const fromDriver = createRequire(fromHere.resolve('@objectstack/driver-sql'));
97+
const betterSqlite3 = fromDriver.resolve('better-sqlite3/package.json');
98+
const bindings = createRequire(betterSqlite3)('bindings');
99+
try {
100+
bindings({ bindings: 'os_24950_never_built.node', module_root: dirname(betterSqlite3) });
101+
} catch (error) {
102+
return error as Error;
103+
}
104+
throw new Error('bindings located an addon that was never built');
105+
}
106+
107+
/**
108+
* Make the native addon fail to load with `error`, where `better-sqlite3`
109+
* loads it: the knex better-sqlite3 client's `acquireRawConnection`, which runs
110+
* `new Database(file)`. The method is the client class's own, and wasm SQLite's
111+
* client is another class, so the wasm rung is untouched.
112+
*/
113+
async function failNativeAddonLoad(error: Error): Promise<void> {
114+
const probe = new SqlDriver({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true } as any);
115+
const clientProto = Object.getPrototypeOf((probe as any).knex.client);
116+
await probe.disconnect();
117+
expect(Object.prototype.hasOwnProperty.call(clientProto, 'acquireRawConnection')).toBe(true);
118+
vi.spyOn(clientProto, 'acquireRawConnection').mockRejectedValue(error);
119+
}
120+
121+
describe('resolveSqliteDriver in development: a file that cannot be opened or created is not stepped down (#24950)', () => {
122+
it('a path under a regular file rejects with the driver\'s refusal, never wasm or memory', async () => {
123+
const warn = vi.fn();
124+
const at = join(dir, 'plain', 'archive.db');
125+
expectFileRefusal(await rejectionOf(resolveDev(at, warn)), at, 'EEXIST');
126+
127+
const through = join(dir, 'plain', 'sub', 'archive.db');
128+
expectFileRefusal(await rejectionOf(resolveDev(through, warn)), through, 'ENOTDIR');
129+
130+
expect(warn).not.toHaveBeenCalled();
131+
});
132+
133+
it('a directory where the file should be rejects with SQLITE_CANTOPEN', async () => {
134+
const filename = join(dir, 'is-a-directory.db');
135+
mkdirSync(filename);
136+
const warn = vi.fn();
137+
expectFileRefusal(await rejectionOf(resolveDev(filename, warn)), filename, 'SQLITE_CANTOPEN');
138+
expect(warn).not.toHaveBeenCalled();
139+
});
140+
141+
it('an absent file in a directory that exists still opens native, and is created', async () => {
142+
const filename = join(dir, 'fresh.db');
143+
const warn = vi.fn();
144+
const resolved = await resolveDev(filename, warn);
145+
expect(resolved.engine).toBe('better-sqlite3');
146+
expect(await resolved.driver.execute('SELECT 1 AS one')).toEqual([{ one: 1 }]);
147+
expect(existsSync(filename)).toBe(true);
148+
expect(warn).not.toHaveBeenCalled();
149+
});
150+
151+
it('an absent file whose parent directories are missing still opens native: connect() makes them', async () => {
152+
const filename = join(dir, 'new', 'deeper', 'fresh.db');
153+
const resolved = await resolveDev(filename);
154+
expect(resolved.engine).toBe('better-sqlite3');
155+
expect(existsSync(filename)).toBe(true);
156+
});
157+
});
158+
159+
describe('resolveSqliteDriver in development: a native addon that cannot load still steps down to wasm (#2229)', () => {
160+
it('an ABI mismatch (ERR_DLOPEN_FAILED) steps down to wasm SQLite, with the warning', async () => {
161+
const loadError = realDlopenFailure();
162+
expect((loadError as any).code).toBe('ERR_DLOPEN_FAILED');
163+
await failNativeAddonLoad(loadError);
164+
165+
const warn = vi.fn();
166+
const filename = join(dir, 'dev.db');
167+
const resolved = await resolveDev(filename, warn);
168+
expect(resolved.engine).toBe('sqlite-wasm');
169+
expect(warn).toHaveBeenCalledWith(NATIVE_SQLITE_WASM_FALLBACK_WARNING);
170+
expect(await resolved.driver.execute('SELECT 1 AS one')).toEqual([{ one: 1 }]);
171+
});
172+
173+
it('an addon that was never built (no code) steps down to wasm SQLite', async () => {
174+
const loadError = realNotBuiltFailure();
175+
expect((loadError as any).code).toBeUndefined();
176+
await failNativeAddonLoad(loadError);
177+
178+
const resolved = await resolveDev(join(dir, 'dev.db'));
179+
expect(resolved.engine).toBe('sqlite-wasm');
180+
});
181+
182+
it('an addon that cannot load AND a path under a regular file still rejects, never memory', async () => {
183+
// connect() makes the file's directory before the first statement loads
184+
// the addon, so the native rung refuses the path before the load fails.
185+
await failNativeAddonLoad(realDlopenFailure());
186+
187+
const warn = vi.fn();
188+
const at = join(dir, 'plain', 'archive.db');
189+
expectFileRefusal(await rejectionOf(resolveDev(at, warn)), at, 'EEXIST');
190+
expect(warn).not.toHaveBeenCalled();
191+
});
192+
});
193+
194+
describe('resolveSqliteDriver in production: unchanged', () => {
195+
it('a path under a regular file returns native unprobed; its connect() refuses the file', async () => {
196+
const warn = vi.fn();
197+
const at = join(dir, 'plain', 'archive.db');
198+
const resolved = await resolveSqliteDriver({ filename: at, dev: false, warn });
199+
opened.push(resolved);
200+
expect(resolved.engine).toBe('better-sqlite3');
201+
expect(warn).not.toHaveBeenCalled();
202+
let refusal: unknown;
203+
await resolved.driver.connect().catch((e: unknown) => { refusal = e; });
204+
expectFileRefusal(refusal, at, 'EEXIST');
205+
});
206+
});
207+
208+
describe('DatasourceConnectionService in development: a declared SQLite datasource on an unopenable path (#24950)', () => {
209+
it('reports a failed connection state, and registers no driver', async () => {
210+
const registered: string[] = [];
211+
const engine = {
212+
registerDriver: (driver: { name?: string }) => { registered.push(String(driver?.name)); },
213+
getDriverByName: (name: string) => (registered.includes(name) ? {} : undefined),
214+
};
215+
const service = new DatasourceConnectionService({
216+
factory: () => createDefaultDatasourceDriverFactory({ dev: true }),
217+
engine: () => engine as any,
218+
});
219+
const filename = join(dir, 'plain', 'archive.db');
220+
221+
await service.connectDeclared({
222+
datasources: [{ name: 'archive', driver: 'sqlite', config: { filename }, autoConnect: true }],
223+
});
224+
225+
const state = service.getConnectionState('archive');
226+
expect(state?.availability).toBe('failed');
227+
expect(state?.reason).toContain(filename);
228+
expect(registered).toEqual([]);
229+
});
230+
});

‎packages/services/service-datasource/src/sqlite-driver-fallback.ts‎

Lines changed: 89 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,39 @@
3131
* so it surfaces loudly (fail-closed) instead of an operator unknowingly running
3232
* on wasm/mingo. This mirrors the existing `serve.ts` default-dev fallback and
3333
* hoists it into one place shared by every sqlite construction site.
34+
*
35+
* ## Which probe failures step down, and which do not (#24950)
36+
*
37+
* The step-down exists for ONE cause: the native addon failing to LOAD. That is
38+
* the only failure a different engine can cure. In development, then:
39+
*
40+
* - **The addon cannot load — steps down.** An ABI mismatch (Node's
41+
* `ERR_DLOPEN_FAILED`, a `NODE_MODULE_VERSION` message), an addon that was
42+
* never built (`bindings`' "Could not locate the bindings file", which has
43+
* no code), or a missing `better-sqlite3` package. Today's path,
44+
* unchanged.
45+
* - **The database FILE cannot be opened or created — re-thrown, as in
46+
* production.** A path under a regular file, a parent directory the driver
47+
* cannot make, a permission refusal, a directory where the file should be.
48+
* `SqlDriver.connect()` names these with the operating system's code
49+
* (`EEXIST`, `ENOTDIR`, `ENOENT`, `EACCES`, …) or SQLite's
50+
* `SQLITE_CANTOPEN`; see {@link isSqliteFileOpenFailure}. Every rung opens
51+
* the same file, so no rung can cure this. Before, the step-down answered
52+
* in its place: the in-memory rung when wasm failed on the same path, or a
53+
* wasm driver whose connect resolved over a file it could not read. So a
54+
* declared datasource that could not answer one query read `connected`,
55+
* and a maintenance command reported a store it never opened as examined
56+
* and clean. Now the datasource's connect fails and its connection state
57+
* says so. That holds for every datasource, the primary included.
58+
* `connect()` makes the directory before the addon loads, so a path the
59+
* driver cannot make is refused at the native rung even when the addon
60+
* could not have loaded.
61+
*
62+
* An ABSENT file in a directory that exists, or can be made, is not an open
63+
* failure: the driver makes the directory and SQLite creates the file. A dev
64+
* boot on a fresh checkout keeps working.
65+
*
66+
* Every other probe failure keeps today's path.
3467
*/
3568

3669
import type { SqliteAbsentFileMode } from '@objectstack/driver-sql';
@@ -100,9 +133,50 @@ function isEphemeralFilename(filename: string): boolean {
100133
return filename === ':memory:' || filename.startsWith(':');
101134
}
102135

136+
/**
137+
* Is this probe failure the database FILE failing to open or be created, as
138+
* opposed to the native addon failing to load (#24950)? Judged on the thrown
139+
* value's own `code` alone, never on its message. `SqlDriver.connect()` is what
140+
* refuses such a file, with a `SqliteDatabaseFileError` whose `code` is the
141+
* answer it got:
142+
*
143+
* - an operating-system error code (`EEXIST`, `ENOTDIR`, `ENOENT`, `EACCES`,
144+
* `EPERM`, `EROFS`, …), from making the file's directory. These codes are
145+
* `E` plus capital letters and digits, with no underscore;
146+
* - SQLite's `SQLITE_CANTOPEN`: the directory is there, and SQLite still
147+
* cannot open the file. Only a LOADED addon can answer with a SQLite code.
148+
*
149+
* An addon that cannot load never carries either. It surfaces at the probe's
150+
* `SELECT 1`, whose failure `SqlDriver.execute()` hands back as
151+
* `DATABASE_ERROR`, with the load error as its `cause`: Node's
152+
* `ERR_DLOPEN_FAILED`, `MODULE_NOT_FOUND`, or no code at all (`bindings`'
153+
* "Could not locate the bindings file"). A missing `better-sqlite3` package
154+
* fails at construction with knex's install hint, which has no code. None of
155+
* these matches.
156+
*
157+
* ⛔ No message text is read. A failure this misses keeps today's path.
158+
*/
159+
function isSqliteFileOpenFailure(err: unknown): boolean {
160+
const code = (err as { code?: unknown } | null | undefined)?.code;
161+
if (typeof code !== 'string') return false;
162+
return /^E[A-Z0-9]+$/.test(code) || code === 'SQLITE_CANTOPEN';
163+
}
164+
165+
/** Best-effort close of a probe driver that failed; its own error is not the answer. */
166+
async function closeQuietly(driver: any): Promise<void> {
167+
if (typeof driver?.disconnect !== 'function') return;
168+
try {
169+
await driver.disconnect();
170+
} catch {
171+
/* ignore */
172+
}
173+
}
174+
103175
/**
104176
* Probe a `better-sqlite3` SQLite construction and, in dev, step down to wasm
105-
* SQLite (then in-memory) when the native addon cannot load.
177+
* SQLite (then in-memory) when the native addon cannot load. A database file
178+
* that cannot be opened or created is re-thrown instead, in dev as in
179+
* production (#24950).
106180
*
107181
* @see {@link ResolveSqliteDriverOptions}
108182
*/
@@ -149,29 +223,29 @@ export async function resolveSqliteDriver(
149223
return { driver: buildNative(), engine: 'better-sqlite3', label: 'SqlDriver(sqlite)' };
150224
}
151225

152-
// ── Dev: probe-by-connect, step down on native ABI/load failure. ──────────
226+
// ── Dev: probe-by-connect, step down on native ABI/load failure only. ─────
153227

154228
// 1. Native better-sqlite3.
155229
let nativeDriver: any;
156230
let nativeOk = false;
157231
try {
158232
nativeDriver = buildNative();
159-
// connect() runs mkdir (so a SELECT on a file DB whose dir is missing does
160-
// not false-positive as an ABI failure) + a PRAGMA whose error it swallows;
161-
// the raw SELECT 1 below is what reliably forces the native addon to load
162-
// and PROPAGATES an ABI mismatch.
233+
// connect() makes the file's directory (so a missing directory is not an
234+
// open failure) and refuses a file it cannot open or create, naming it; a
235+
// PRAGMA whose load error it swallows follows. The raw SELECT 1 below is
236+
// what reliably forces the native addon to load and PROPAGATES an ABI
237+
// mismatch.
163238
await nativeDriver.connect();
164239
await nativeDriver.execute('SELECT 1');
165240
nativeOk = true;
166-
} catch {
241+
} catch (err) {
167242
nativeOk = false;
168-
if (typeof nativeDriver?.disconnect === 'function') {
169-
try {
170-
await nativeDriver.disconnect();
171-
} catch {
172-
/* ignore */
173-
}
174-
}
243+
await closeQuietly(nativeDriver);
244+
// The FILE cannot be opened or created: no other engine can cure that, so
245+
// the datasource's connect fails here, as it does in production (#24950).
246+
// Every other failure, a native addon that cannot load first among them,
247+
// goes on down the chain as before.
248+
if (isSqliteFileOpenFailure(err)) throw err;
175249
}
176250
if (nativeOk) {
177251
return { driver: nativeDriver, engine: 'better-sqlite3', label: 'SqlDriver(sqlite)' };
@@ -193,13 +267,7 @@ export async function resolveSqliteDriver(
193267
wasmOk = true;
194268
} catch {
195269
wasmOk = false;
196-
if (typeof wasmDriver?.disconnect === 'function') {
197-
try {
198-
await wasmDriver.disconnect();
199-
} catch {
200-
/* ignore */
201-
}
202-
}
270+
await closeQuietly(wasmDriver);
203271
}
204272
if (wasmOk) {
205273
warn(NATIVE_SQLITE_WASM_FALLBACK_WARNING);

0 commit comments

Comments
 (0)