Repository navigation
Commit 6e6881e
test(driver-turso): the cross-process autonumber pin starts its writers once and stops waiting on fsync (#23218)
Part of #23182 (row 2)
Clause-②: no
**Row 2 of #23182 only:** `@objectstack/driver-turso`,
`src/turso-remote-autonumber-concurrency.test.ts`, read at 29.3 s in run
`38228297391`. Refresh PR #23141 lists it over 10 s in 6 of the 7 hourly
full-battery runs that executed it. #23182 remains open for rows 1 and 3
to 6.
## What changed (one file, test-only)
The file pins cross-process autonumber atomicity on the remote face: two
writer processes and one libSQL database file must give N distinct
numbers, monotonic per writer, with evidence that the writers
overlapped. Most of its time went to work it does not assert on.
Measured at `d4aaeaf0e` on this 4-vCPU container, one round took about
4.4 s:
| phase | per round, before | what it was |
|:--|--:|:--|
| writer start-up | ~2.2 s | `node --import tsx`, then the driver's
dependencies (`@objectstack/driver-sql` and what it imports,
`@objectstack/spec`, `@libsql/client`) loaded through tsx's hook: ~2 s,
against ~1.2 s under plain Node. The package's own source compiled in
~40 ms. |
| writes (2 x 200 creates) | ~2.2 s | fsync-bound. A `create` is 3
statements (reserve, insert, read back). A single writer took 5.9 ms per
create with `synchronous = FULL` and 1.2 ms with `synchronous = OFF`. |
A round whose writers did not overlap also re-spawned both writers.
Three changes, none to what is asserted:
1. **Writers start once.** They start once per test and serve every
round over stdin/stdout, one JSON line per command and per answer. A
retry round costs only its writes.
2. **Dependencies load before tsx.** A writer requires the driver's
dependencies with plain Node first, then calls
`require('tsx/cjs/api').register()` and requires `src/index.ts`. The
source is still compiled by tsx, and the dependencies are the same
modules, already in the require cache. `tsx` is already a declared
devDependency, and `tsx/cjs/api` is one of its public exports.
3. **No fsync.** Each writer runs `PRAGMA synchronous = OFF` on its own
connection. That drops only the disk flush at commit, on a throwaway
temp file. The journal mode, the locks and the busy timeout are
unchanged, so the cross-process serialisation the pin depends on is
unchanged.
The round's ready and write times now join the existing measurement
line, so a slow CI run can be attributed from its log.
Kept as they were: N = 200 per writer, at most 3 rounds, every pin, and
the NOT MEASURED skip. N was not reduced: writes are no longer the cost,
and a smaller N would only lower how often a broken counter is caught
(see the reverse verification below).
## Readings
The CI per-file number is vitest's module line (`✓ path (1 test) Nms`).
All readings below are from this container under
`scripts/pm/os-verify-lock.sh`, on a shared box (other seats' unlocked
work was running), with vitest's default reporter (agent-detection
variables unset, as in CI).
| density | BASE `d4aaeaf0e` | this PR |
|:--|--:|--:|
| the file alone | 4575, 5256, 4406 ms | 1836, 1840, 1853 ms
(`83ece05da`) |
| whole `driver-turso` package (88 files) | 5735, 5914 ms | 1912 ms
(`83ece05da`) |
| package plus `driver-sql`'s suite at once | 10658, 8741, 8973 ms |
4126 ms (`794b74409`, same run as BASE's 8973) |
Writer changes per round: 2 to 40 before, 141 to 221 after (unmutated
runs).
**CI reading for the after-state: NOT MEASURED here.** Job logs and
artefacts redirect to an Azure blob host that this container's egress
proxy refuses (403). This PR's own Test Core shard carries the file's
module line and the measurement line in its log.
## Reverse verification (ablation)
**Mutation:** the warm reservation (`UPDATE … SET last_value =
last_value + 1 … RETURNING`) split into a `SELECT last_value` and then
an `UPDATE … SET last_value = ?` (read-then-write), once with no gap and
once with a 1 ms gap.
- **Applied** through `scripts/ablation-replace.mjs`: the anchor hit
went 1 → 0 and the blob changed. The restore was proven after every leg:
blob equal to HEAD and `git diff HEAD` empty.
- **Reach:** a tsx child spawned from the package root sees the mutated
method, because `nextRemoteSequenceValue.toString()` carries the marker.
No `dist` is involved, since both the parent and the writers run `src`.
- **Predicted:** red on the "no write failed" pin, with `UNIQUE
constraint failed` in a writer's error list, as the file's header
recorded. **Observed:** that direction in every red run.
| shape | no gap | 1 ms gap |
|:--|--:|--:|
| this PR (`794b74409` and `83ece05da`) | red 8 of 11 | red 7 of 7 |
| this PR's first cut `7e0c6cc50` (before the dependency preload) | red
7 of 10 | red 5 of 5 |
| BASE, same box and hour, interleaved with the first cut | red 1 of 5 |
not run |
A non-atomic counter is caught more often than before, not less: without
the fsync the writers interleave far more finely. A zero-gap
read-then-write remains a probabilistic catch, as it was at BASE.
## Gates and tests (at `83ece05da` unless named)
- **Build:** `pnpm --filter '@objectstack/driver-turso...' build` green,
after merging `origin/main`.
- **Typecheck and tests:** `pnpm --filter @objectstack/driver-turso
typecheck` (`tsc --noEmit`) is green, and its tsconfig includes
`src/**/*`, tests included. The package suite ran 88/88 files: 2373
passed, 33 skipped, the same counts as BASE.
- **Driver conformance:** `pnpm check:driver-conformance` before the
first edit and after the last commit read `50 covered cell(s), 0 in the
DEBT ledger, 0 exempt` both times.
- **Derived gates:** `node scripts/pm/dispatch-gates.mjs --commands
--repo objectstack-ai/objectstack` gave 51 commands, identical to the
dispatch-time set, and all ran. The `--ran` verdict reads `51 derived,
51 run, 0 NOT-MEASURED, 0 UNRUN`. Three first answered exit 3 (unmet
prerequisites) and were re-run green after their remedy:
`check-plugin-teardown-shape --self-test` needed `git fetch
--unshallow`, and `check:lean-entry-closure` and
`check:dual-build-cjs-loads` needed package `dist`s, restored from the
turbo cache.
- **Lint, narrowed and proven:** eslint with `--no-inline-config` on the
one changed file linted 1 file with 0 errors and 0 warnings. The file is
in eslint's population (`--print-config` returns a config for it). The
config enables no type-aware linting (the resolved `parserOptions` are
`ecmaVersion` and `sourceType`), so this diff cannot move any other
file's verdict.
- **Bytes:** `check:nul-bytes` is green, and a control-byte self-scan of
the file found 0 lines.
## Changeset
None. The diff is test-only: the changed file is not in `driver-turso`'s
`files[]` (`dist`, `README.md`, `CHANGELOG.md`). Measured on a `dist`
built after the merge, four symbols unique to the test
(`PRELOADED_DEPENDENCIES`, `WORKER_SOURCE`, `REPLY_DEADLINE_MS`,
`turso-autonumber-concurrency-`) have 0 hits, and the positive control
`nextRemoteSequenceValue` has 6. Label: `skip-changeset`.
## Acceptance notes
- **Import time is shared, not this file's.** The whole vitest process
for this one file still takes about 16 to 19 s of wall here, and about
14 to 15 s of that is vitest's import of the driver's source graph.
Every file in this package pays that import, and the per-file reading
does not count it.
- **`@objectstack/core` import cost.** It alone takes about 0.9 s to
import under plain Node and dominates a writer's remaining start-up of
about 1.2 s. That is an observation about another package's import cost,
not a defect; it is not filed.
- **The old ablation reading was optimistic.** The file's header
recorded the zero-gap ablation as red in 2 of 3 runs at BASE. On this
box the same BASE shape was red in 1 of 5. The new shape records its own
reading beside the old one.
- **The card's pin can only be read after landing:** "the next refresh
PR's list no longer names the file".
---
_Generated by [Claude
Code](https://claude.ai/code/session_01P88ehrZ6hM19hBzdpsQTTo)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 38a2d3c commit 6e6881e
1 file changed
Lines changed: 198 additions & 84 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
26 | 27 | | |
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
32 | 33 | | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
33 | 64 | | |
34 | 65 | | |
35 | 66 | | |
| |||
76 | 107 | | |
77 | 108 | | |
78 | 109 | | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
79 | 118 | | |
80 | 119 | | |
81 | 120 | | |
82 | 121 | | |
83 | | - | |
| 122 | + | |
84 | 123 | | |
85 | 124 | | |
86 | 125 | | |
| |||
90 | 129 | | |
91 | 130 | | |
92 | 131 | | |
93 | | - | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
94 | 141 | | |
95 | 142 | | |
96 | 143 | | |
97 | 144 | | |
98 | 145 | | |
| 146 | + | |
| 147 | + | |
99 | 148 | | |
100 | 149 | | |
101 | 150 | | |
| |||
119 | 168 | | |
120 | 169 | | |
121 | 170 | | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
127 | 185 | | |
128 | 186 | | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
141 | | - | |
142 | | - | |
143 | | - | |
144 | | - | |
145 | | - | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
151 | | - | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
152 | 221 | | |
153 | 222 | | |
154 | | - | |
155 | | - | |
156 | 223 | | |
157 | 224 | | |
158 | | - | |
159 | | - | |
160 | | - | |
161 | | - | |
162 | | - | |
163 | | - | |
164 | | - | |
165 | | - | |
166 | | - | |
167 | | - | |
168 | | - | |
169 | | - | |
170 | | - | |
171 | | - | |
172 | | - | |
173 | | - | |
174 | | - | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
175 | 256 | | |
176 | | - | |
| 257 | + | |
177 | 258 | | |
178 | | - | |
| 259 | + | |
| 260 | + | |
179 | 261 | | |
180 | | - | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
181 | 266 | | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
182 | 296 | | |
183 | 297 | | |
184 | | - | |
185 | | - | |
186 | 298 | | |
187 | 299 | | |
188 | 300 | | |
189 | 301 | | |
190 | | - | |
| 302 | + | |
191 | 303 | | |
192 | 304 | | |
193 | 305 | | |
| |||
199 | 311 | | |
200 | 312 | | |
201 | 313 | | |
202 | | - | |
203 | | - | |
204 | | - | |
205 | | - | |
206 | | - | |
207 | | - | |
208 | | - | |
209 | 314 | | |
210 | | - | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | | - | |
215 | | - | |
216 | | - | |
217 | | - | |
218 | | - | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
219 | 323 | | |
220 | 324 | | |
221 | 325 | | |
| |||
260 | 364 | | |
261 | 365 | | |
262 | 366 | | |
263 | | - | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
264 | 373 | | |
265 | 374 | | |
266 | 375 | | |
267 | 376 | | |
268 | 377 | | |
| 378 | + | |
269 | 379 | | |
270 | | - | |
271 | | - | |
272 | | - | |
273 | | - | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
274 | 388 | | |
275 | 389 | | |
276 | 390 | | |
| |||
0 commit comments