Repository navigation
Commit 53646fc
Fixes #23696
Clause-②: no
## What was wrong (measured, root-caused)
A declared SQLite datasource whose `config.filename` lay under a
directory the OS will not make (`/proc/os-23696/ds.db`) never finished
its `connect()`. The boot was refused only by a plugin start timeout,
and that message named neither the datasource, the path nor the error.
**Root cause: Node's own recursive `mkdir`, not a knex pool retry
(dispatch hypothesis H2 falsified).** `SqlDriver.ensureDatabaseExists()`
made the file's directory with `mkdir(dir, { recursive: true })`. On
Node v22.22.0 that call never settles when one level answers `ENOENT`
although its parent exists. Under `strace`, one second of
`fs.mkdir('/proc/os-23696', { recursive: true })` shows this loop:
```
mkdir("/proc/os-23696") = -1 ENOENT
mkdir("/proc") = -1 EEXIST
statx("/proc") = 0 (a directory, so it retries the child)
... repeat ...
```
That is 1,640 failing `mkdir` and 819 `statx` per second. They are
syscalls, which is the system CPU #23649's dev read. The sync form
(`mkdirSync`) loops the same way. A bare `node -e` of the async call,
with no ObjectStack code, ran at 90% CPU for over 2 minutes before I
killed it.
The knex pool does not loop. When better-sqlite3 cannot open the file,
the acquire rejects at once. But the driver's first statement (`PRAGMA
auto_vacuum`) caught `SQLITE_CANTOPEN`, logged it as a warning and
returned. So `connect()` **resolved** in 11 ms on `/proc/ds-23696.db`
(the directory exists, the file cannot be opened), and the datasource
counted as connected while it could answer no query. The issue's Wanted
covers this arm too: "cannot be opened or created fails its connect at
once, with the open error".
## The change (producer: `packages/drivers/driver-sql`)
- `src/sqlite-database-file.ts` (new):
- `makeDatabaseDirectory(dir)` walks up to the nearest existing level,
then creates each missing level with a non-recursive `mkdir`, top-down.
The walk is bounded by the path's depth. An `EEXIST` is accepted only as
a concurrent creator winning the race. Any other refusal is the answer,
and it is the OS's own: `ENOENT`, `EACCES`, `ENOTDIR`, or `EEXIST` when
a file holds the directory's own name.
- `SqliteDatabaseFileError` names the file (and its resolved path when
relative), the directory, and the cause's message. It carries `code`
(the underlying one) and `cause`.
- `src/sql-driver.ts`:
- `ensureDatabaseExists()` uses the bounded walk and throws
`SqliteDatabaseFileError` (`cannot be created: its directory 'X' could
not be made: …`).
- In the connect's first PRAGMA, `SQLITE_CANTOPEN` now fails `connect()`
with `SqliteDatabaseFileError` (`cannot be opened or created: …`). A
native-addon load failure keeps its warning and its step-down to wasm.
Any other PRAGMA failure keeps its warning, as before.
- `AppPlugin` is unchanged. The connection service's existing ADR-0062
D5 refusal (`datasource 'NAME': connect failed — REASON`) carries the
driver's message verbatim, so it already names the datasource, the path
and the error. **No `packages/runtime` edit and no cross-lane line were
needed.** The `objectql:drivers-ready` start-order region is untouched,
and there is no timeout change.
Every caller of the driver gets this once. Measured: the mapped
datasource (`AppPlugin`'s pre-sync connect), the host's default
datasource from `OS_DATABASE_URL=file:…` under `serve`, and
`SqliteWasmDriver`, whose `super.connect()` runs the walk before its own
mkdir and rejects in 3 ms.
## Verification
All boot readings use the built CLI (`packages/cli/bin/run.js serve
objectstack.config.ts`, `NODE_ENV` unset = production,
`OS_DATABASE_URL=file:./ctl.db`). The host config declares `primary`
(`driver: 'sqlite'`, `config.filename` as listed), one app object and
one host plugin. CPU is the child's `utime` and `stime` from its last
`/proc/PID/stat` sample.
**H1, before (`f40d877f8`, unmodified):**
| shape | wall | outcome | CPU (user / sys) |
|:--|:--|:--|:--|
| `datasourceMapping: [{ default: true, datasource: 'primary' }]`,
filename `/proc/os-23696/ds.db` | 122.7 s | `Plugin
com.objectstack.engine.objectql start timeout after 120000ms`. Names no
datasource, path or error | 28.6 s / 83.2 s |
| H3a: no mapping, `autoConnect: true`, same filename | 33.2 s | `Plugin
plugin.app.com.example.fx start timeout after 30000ms` | 10.2 s / 21.0 s
|
| H3b: no mapping, `OS_DATABASE_URL=file:/proc/os-23696/ctl.db` | 32.5 s
| `Plugin com.objectstack.runtime.default-datasource init timeout after
30000ms` | 9.8 s / 20.5 s |
| CONTROL: mapping, filename `./ds-primary.db` | 4.2 s | ready banner |
4.9 s / 0.4 s |
**After (final head `9f89d446`):**
| shape | wall (whole process, CLI load included) | outcome | CPU (user
/ sys) |
|:--|:--|:--|:--|
| H1: mapping, `/proc/os-23696/ds.db` | 2.6 s | `datasource 'primary':
connect failed — SQLite database file '/proc/os-23696/ds.db' cannot be
created: its directory '/proc/os-23696' could not be made: ENOENT: no
such file or directory, mkdir '/proc/os-23696'. (71 object(s) are routed
to it by a datasourceMapping rule … ⇒ fail-fast per ADR-0062 D5)` | 3.0
s / 0.3 s |
| mapping, `/proc/ds-23696.db` (`SQLITE_CANTOPEN`) | 2.8 s | `datasource
'primary': connect failed — SQLite database file '/proc/ds-23696.db'
cannot be opened or created: unable to open database file. (…fail-fast
per ADR-0062 D5)` | 3.0 s / 0.2 s |
| H3b: `OS_DATABASE_URL=file:/proc/os-23696/ctl.db` | 2.7 s |
`datasource 'default': connect failed — SQLite database file
'/proc/os-23696/ctl.db' cannot be created: … ENOENT … (declared
boot-critical by the host … ⇒ fail-fast per ADR-0062 D5)` | 3.0 s / 0.3
s |
| H3a: `autoConnect`, unrouted | 3.9 s | ready banner. Not addressed by
a mapping, a binding or `bootCritical`, so it degrades per D5 (see the
acceptance notes) | 4.3 s / 0.4 s |
| CONTROL: mapping, `./ds-primary.db` | 4.4 s | ready banner | 5.0 s /
0.3 s |
**At the driver** (the built `dist`, `new SqlDriver({ client:
'better-sqlite3', connection: { filename } }).connect()`):
| filename | before | after |
|:--|:--|:--|
| `/proc/os-23696/ds.db` | unsettled after 20 s | rejected in 1 ms,
`ENOENT` |
| `/proc/os-23696/a/b/ds.db` | (same loop) | rejected in 2 ms, `ENOENT`,
names `mkdir '/proc/os-23696'` |
| `/proc/ds-23696.db` | **resolved** in 11 ms | rejected in 7 ms,
`SQLITE_CANTOPEN` |
| a fresh path three levels deep | connects | connects, creates the
file, `select 1` answers |
**Pins:**
`packages/drivers/driver-sql/src/sql-driver-23696-sqlite-uncreatable-file.test.ts`,
4 tests. Each refusal pin asserts `SqliteDatabaseFileError`, its `code`,
its `filename`, a message holding both the file and the cause's message,
and arrival within 1,000 ms. A 5 s race turns a regression into a red,
not a hang.
- `/proc/MISSING/` (`ENOENT`). Runs where `/proc/self` exists, so on
Linux CI. This is the exact looping shape.
- a path through a regular file (`ENOTDIR`) and a file holding the
directory's own name (`EEXIST`). Portable.
- a filename that is a directory (`SQLITE_CANTOPEN`). Portable.
- CONTROL: a creatable path three levels deep is made and opened, and
answers `SELECT 1`.
**Ablation** (`scripts/ablation-replace.mjs`, mutation proven on disk,
then restored, with blob = HEAD and an empty `git status --porcelain`).
The test imports `./sql-driver.js` from `src`, so no `dist` is involved.
- Leg A: `await makeDatabaseDirectory(dir)` replaced by `mkdir(dir, {
recursive: true })`. 2 failed / 2 passed. The `/proc` pin failed with
`connect() had not settled after 5000 ms` (5,007 ms). The regular-file
pin failed with `expected 'EEXIST' to be 'ENOTDIR'`.
- Leg B: the `SQLITE_CANTOPEN` throw replaced by a no-op. 1 failed / 3
passed: `connect() resolved for a file that cannot be created or
opened`.
- Both legs ran on `fa0158df`. The two mutated lines are byte-identical
on the final head. The later commits only declare `cause` as a field and
replace the synthesized `ENOTDIR` with the OS's own answer.
**Package:**
- `pnpm --filter @objectstack/driver-sql test` on `9f89d446`: 221 files
passed, 11 skipped (live dialects). 3,663 tests passed, 208 skipped.
Exit 0.
- `pnpm --filter @objectstack/driver-sql typecheck` and `build`: exit 0,
against a rebuilt `spec/dist` with declarations. `check-dts-emitted`:
1/1.
**Gate families:** `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack`, re-derived on `9f89d446` with no paths. The
change set is 4 paths against merge base `f40d877f8`, and it derives 67
commands, the same 67 as at dispatch. Each was run with its exit code
captured before any pipe and reconciled with `--ran` (`COMMAND :: exit
N`): `✓ dispatch-gates --ran: 67 derived famil(ies) accounted for — 66
run, 1 NOT-MEASURED (1 DERIVED from a recorded exit 3).`
- 66 exited 0.
- NOT MEASURED: `pnpm check:dual-build-cjs-loads`, exit 3 `PREREQUISITE
NOT MET` (it needs every package built: `studio`, `client-react`, … have
no `dist/`). CI's Lint & Repo Gates runs it.
- Three families needed a fix or a rebuild before they went green, and
all three are recorded:
- `check:dispatcher-error-vocabulary` reddened on a code literal that an
earlier commit had synthesized (`{ code: 'ENOTDIR' }`). Fixed by letting
the OS answer.
- `check:dts-closure` first read a CLI closure I had built with
`OS_SKIP_DTS=1`. After a full declaration build it reads `161/161
declared declaration file(s) present`.
- `check-plugin-teardown-shape --self-test` needed its pinned fixture
commit fetched in this shallow clone. It is green after that.
ADRs grepped for the surface (Prime Directive #13): ADR-0062 D5 and its
#4462 amendment ("a `declared-auto` failure is **fatal**") are what the
boot now reaches instead of a timeout. ADR-0057's
`auto_vacuum=INCREMENTAL` default is unchanged. No recorded decision is
reversed.
## Acceptance notes
- **Not fixed here, same defect class, different producer:** `os start`
with `OS_DATABASE_URL=file:/proc/os-23696/ctl.db` and no config prints
`→ Starting server...` and then hangs. It was killed at 25 s. Under
`strace`, the respawned server's main thread made 21,443 failing
`mkdir("/proc/os-23696")` calls in 12 s. That is the synchronous
`mkdirSync(…, { recursive: true })` in
`packages/runtime/src/standalone-stack.ts` (`createStandaloneStack`, the
sqlite arm, and the same line in the `sqlite-wasm` arm). Because it
blocks the event loop, no plugin timeout can fire. ADR-0062's #3826
resolution names that `mkdir` as a host concern. It is outside this
card's file surface and gate family, so it is reported to the seat for
filing, not fixed here. `packages/cli/src/commands/dev.ts` (`os dev`'s
database directory) uses the same call shape and was not measured.
- H3a (`autoConnect: true`, no mapping, unaddressed) now degrades
instead of hanging, as ADR-0062 D5 prescribes for an unaddressed
datasource. Its `degrading (datasource left unconnected)` warning did
not appear on `serve`'s stream, even at `OS_LOG_LEVEL=warn`. That
visibility question predates this change and is noted only.
- `packages/services/service-datasource/src/sqlite-driver-fallback.ts`'s
dev probe comment says `connect()` runs "a PRAGMA whose error it
swallows". It still swallows everything except `SQLITE_CANTOPEN`. The
probe's `catch` treats a throwing `connect()` exactly as it treated the
throwing `SELECT 1` that used to follow, so its branch is unchanged.
Related: #23649 and PR #23694 (which moved the mapped connect into the
engine plugin's start; that is where the 120 s figure comes from). This
PR does not change that region.
Dispatched by the `domain:engine` seat 1 PM
(`session_01GP3comALHRcRBixS2TEnLF`). Developed in
`session_01CxHmGdRW1sJtL25b6ysocg`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
_Body refreshed 2026-10-15T23:03Z by `domain:engine` seat 1: the one
wording edit the dev's report 6203616771 lists._
---
_Generated by [Claude
Code](https://claude.ai/code/session_01CxHmGdRW1sJtL25b6ysocg)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent 3b46855 commit 53646fc
4 files changed
Lines changed: 223 additions & 3 deletions
File tree
- .changeset
- packages/drivers/driver-sql/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
| 158 | + | |
158 | 159 | | |
159 | 160 | | |
160 | 161 | | |
| |||
7001 | 7002 | | |
7002 | 7003 | | |
7003 | 7004 | | |
| 7005 | + | |
| 7006 | + | |
| 7007 | + | |
| 7008 | + | |
| 7009 | + | |
| 7010 | + | |
| 7011 | + | |
| 7012 | + | |
| 7013 | + | |
| 7014 | + | |
7004 | 7015 | | |
7005 | 7016 | | |
7006 | 7017 | | |
| |||
20105 | 20116 | | |
20106 | 20117 | | |
20107 | 20118 | | |
20108 | | - | |
| 20119 | + | |
| 20120 | + | |
| 20121 | + | |
20109 | 20122 | | |
20110 | 20123 | | |
20111 | 20124 | | |
20112 | 20125 | | |
20113 | | - | |
20114 | 20126 | | |
20115 | 20127 | | |
20116 | | - | |
| 20128 | + | |
| 20129 | + | |
| 20130 | + | |
| 20131 | + | |
| 20132 | + | |
20117 | 20133 | | |
20118 | 20134 | | |
20119 | 20135 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
0 commit comments