Skip to content

Commit 4d3a077

Browse files
test(driver-sqlite-wasm): run the persistence suites' temp files on /dev/shm, so a flush's fsync stops waiting on the shard's other suites (#27245)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"} --> _Requested by **Leehom** · [project thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1X2kei2ECbgeH2p5LqGYSsPn)_ Fixes #27076 Clause-②: no Metadata-protocol: none ## 维护者速读 - **改了什么**:`driver-sqlite-wasm` 这个包的 vitest 配置把测试进程的 `TMPDIR` 指到可写的 `/dev/shm`(内存盘);机器上没有 `/dev/shm`(macOS、Windows)就保持原样。只动了 `packages/drivers/driver-sqlite-wasm/vitest.config.ts` 一个文件,产品代码、用例、断言和 5 s 预算都没动。 - **为什么改**:全量测试两次红(10-22 的 39853791549、今天 08:05Z 的 40654333991)都是这个包里写真实数据库文件的落盘用例超时。每次落盘最后都要 `fsync`,而 `fsync` 要排在同一台 CI 机器上其他测试套件的磁盘队列后面,所以本地 30 ms 的用例在分片上能拖到 5 s 以上。用例测的是驱动的落盘语义,不是邻居的磁盘有多忙。 - **风险与回滚**:低。代码路径完全一样(`sync()` 照样调用),只是临时文件落在内存盘上;不发布任何东西(`vitest.config.ts` 不在包的 `files[]` 里)。回滚即 revert。 - **先看**:`packages/drivers/driver-sqlite-wasm/vitest.config.ts`(唯一改动,头部注释写了测量)。 - **两分钟验证**:`cd packages/drivers/driver-sqlite-wasm && pnpm exec vitest run`,预期 39 个文件 746 条全过;在用例里打印 `os.tmpdir()` 会看到 `/dev/shm`。 - **需要你做的**:无,席位落地。下一次 4 小时全量测试转绿即关卡。 ## What changes `packages/drivers/driver-sqlite-wasm/vitest.config.ts` used to be the bare re-export of `vitest.preset.mts`. It is now a `defineConfig` with two keys: - `disableConsoleIntercept: true`, restated from the preset: `check-console-intercept-disarm.mjs` judges the preset in a config's place only for the bare re-export, so a config with its own key carries the disarm itself (`node scripts/check-console-intercept-disarm.mjs` → `OK: 77 vitest-running package(s)`). - `env: { TMPDIR: '/dev/shm' }` when `/dev/shm` is writable, `{}` otherwise. Every test file in the package reaches its temp directory through `os.tmpdir()` (`mkdtempSync(join(tmpdir(), …))`, 9 files), and `os.tmpdir()` reads `TMPDIR` on each call, so no test file changes. Every other key stays on vitest's defaults, as under the re-export. ### Why the cases were slow (the cause) The failing case, read from the full job log by the maintainer (card comment 6468136240): `SqliteWasmDriver (file persistence) > should persist data on-write`, 5,026 ms, `Test timed out in 5000ms`, on `Test Core (6/8)`; its sibling `on-disconnect` took 4,236 ms. Run 39853791549 timed out five cases, all on on-write or flush paths (`background-flush-failure` ×3, `returning-persist` `UPDATE … RETURNING`, `durability` atomic write). Each of those flushes ends in `WasmSqliteConnection.atomicWriteFile`: write a sibling temp file, `handle.sync()`, rename. The case itself is about 30 ms on an idle disk (step probe: create 3, insert 1, flush 4, reopen + find 3, raw 8 KiB `sync()` 1 ms). The time goes to `sync()` waiting behind other processes' writes on the same disk: | Disk state (4-core box; the 4 persistence files, 29 cases) | `on-write` case | Slowest case | Cases over 1 s | Timeouts | |:--|--:|--:|--:|--:| | idle, `TMPDIR` on the disk (2 runs, base 24351cf) | 15–39 ms | 0.26 s | 0 | 0 | | 4 `dd … conv=fsync` writers, `TMPDIR` on the disk (3 runs) | 1,388–3,630 ms | 3.6 s | 7–11 per run | 0 | | 4 writers, `TMPDIR=/dev/shm` (3 runs) | 12–28 ms | 0.54 s | 0 | 0 | The same set of cases that went red on CI is the set that inflates under disk load, and only those. CPU load alone does not do it: 8 runs of the whole suite beside 3 busy-loop processes kept every case under 0.6 s. ### Why this is the test's correctness, not a budget A case whose wall time is set by unrelated suites' disk I/O measures the runner, not the driver. On `/dev/shm` the same code runs, `sync()` and `rename()` included, and every assertion (bytes on disk, atomic rename with no leftover temp file, the serialized flush chain, the failure report and retry) is unchanged. The triage ruling holds: no budget raised, no case skipped, no fixture shrunk, no retry wrapper. ## Tests <details><summary>Red on the base config, green on this head: 8 fsync writers, alternated, 3 rounds (head 2c4bfce)</summary> Base leg: `node scripts/ablation-replace.mjs` replaced the `env:` line with `env: {}` (the pre-change behaviour: anchor 1 → 0, blob `08fbf987a6e2` → `ea1e118ddf13`; a probe case printed `TMPDIR_SEEN=/tmp` under it and `TMPDIR_SEEN=/dev/shm` on head), ran the four persistence files, then restored (`ok restored: blob == HEAD (08fbf98) and git diff HEAD is empty`, checked after every round). Head leg: the committed config. All legs under 8 `dd if=/dev/zero bs=1M count=512 conv=fsync` writers on the same disk, inside one `os-verify-lock` hold (VERDICT command-exit 0, 236 s). ``` base-1 Tests 2 failed | 27 passed (29) returning-persist UPDATE … RETURNING 6061 ms timeout; returning-persist transaction 7227 ms timeout head-1 Tests 29 passed (29) slowest 178 ms base-2 Tests 2 failed | 27 passed (29) returning-persist INSERT … RETURNING 6641 ms timeout; driver.test.ts "should persist data on-write" 5358 ms timeout head-2 Tests 29 passed (29) slowest 223 ms base-3 Tests 1 failed | 28 passed (29) returning-persist UPDATE … RETURNING 6223 ms timeout head-3 Tests 29 passed (29) slowest 217 ms ``` Base round 2 reproduces the CI failure exactly (`should persist data on-write`, `Test timed out in 5000ms`). </details> <details><summary>Whole package, head 2c4bfce</summary> `pnpm --filter @objectstack/driver-sqlite-wasm test` under `os-verify-lock`: VERDICT command-exit 0, `Test Files 39 passed (39)`, `Tests 746 passed (746)`. `/dev/shm` holds no leftover after the run (the suites clean their `mkdtemp` directories as before). </details> <details><summary>What was tried before the case was named (base 24351cf)</summary> Alone ×2, beside 3 busy-loop CPU burners ×8, and the shard's real whole-package leg (the 10 packages of `Test Core (6/8)`, `turbo run test test:repo --concurrency=4`, `OS_TEST_TIERS=queue`) ×3: all 746/746 green. Those loads were CPU-heavy, not fsync-heavy, which is why none reproduced it. </details> ## Acceptance notes - `/dev/shm` is writable in the cloud container these readings come from; I did not read a GitHub runner's mounts, so the shard's next run is the proof that it is used there. A host without a writable `/dev/shm` keeps its own temp directory, so the config never errors and never changes behaviour there. - Other packages that fsync real files in tests were not surveyed; this PR is scoped to the package the card names. ## Verification Local, head `2c4bfcec`, change set `packages/drivers/driver-sqlite-wasm/vitest.config.ts` only. - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 42 commands; each run with its exit code captured before any pipe; `--ran` reconciliation: `42 derived famil(ies) accounted for — 40 run, 2 NOT-MEASURED`. - NOT MEASURED, reason: `pnpm check:dual-build-cjs-loads` and `pnpm check:lean-entry-closure` exit 3 (PREREQUISITE NOT MET: they need a full workspace build, which this card thread did not run). CI runs both. - `node scripts/check-plugin-teardown-shape.mjs --self-test` exited 3 on the shallow clone, then 0 after `git fetch --depth=1` of its positive-control commit (48 cases pass). - Outside the derivation: `node scripts/check-console-intercept-disarm.mjs` → exit 0 (`OK: 77 vitest-running package(s)`); `pnpm exec eslint --no-inline-config packages/drivers/driver-sqlite-wasm` → exit 0; `pnpm --filter @objectstack/driver-sqlite-wasm typecheck` → exit 0; `node scripts/check-commit-card-trailers.mjs --range origin/main..HEAD` → exit 0. - Changeset: none. `vitest.config.ts` is not in the package's `files[]` (`dist`, `README.md`, `CHANGELOG.md`), so nothing publishes: `skip-changeset`. ## Human Minimum - [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic. (not touched) - [x] 2. Every name I added uses the casing for its kind. - [x] 3. No formatter ran; `pnpm lint` passes. (eslint scoped to the package, in Verification above) - [x] 4. One issue, one branch off an up-to-date `main`, one pull request. - [x] 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. (publishes nothing: `skip-changeset`) - [x] 6. `packages/spec` touched: its generated artifacts are regenerated and committed. (not touched) - [x] 7. No hand edits to `content/docs/references/`, `packages/*/CHANGELOG.md` or `content/docs/releases/`. - [x] 8. Title, body and comments are in English. (the 维护者速读 block is the dispatch protocol's Chinese summary) - [x] The *Before Submitting* list in CONTRIBUTING.md is done. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01FEkQ5oKUikM4XcQpd1HJ6U --- _Generated by [Claude Code](https://claude.ai/code/session_01FEkQ5oKUikM4XcQpd1HJ6U)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 9ff8154 commit 4d3a077

1 file changed

Lines changed: 54 additions & 2 deletions

File tree

Lines changed: 54 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,56 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
22

3-
// This package needs nothing beyond the shared preset; see its header.
4-
export { default } from '../../../vitest.preset.mts';
3+
import { accessSync, constants } from 'node:fs';
4+
import { defineConfig } from 'vitest/config';
5+
6+
// This package re-exported the shared preset until #27076. It now carries one
7+
// key of its own, `test.env.TMPDIR`, so the preset's one setting is restated
8+
// below (a merged or wrapped preset is not what
9+
// scripts/check-console-intercept-disarm.mjs judges in a config's place).
10+
// Every other key stays on vitest's defaults (`globals: false`,
11+
// `environment: 'node'`), exactly as under the bare re-export.
12+
//
13+
// ## Why TMPDIR points at a memory-backed directory
14+
//
15+
// The persistence tests write a real database file under `os.tmpdir()`, and
16+
// every flush ends in `atomicWriteFile`'s `handle.sync()` (fsync before the
17+
// rename; see wasm-connection.ts). An fsync waits for the disk's queue, and on
18+
// a Test Core shard that queue belongs to the other suites sharing the runner.
19+
// Measured (#27076): `should persist data on-write` takes about 30 ms on an
20+
// idle disk and 5,026 ms on schedule run 40654333991 (Test Core 6/8), where it
21+
// timed out; run 39853791549 timed out five on-write and flush cases the same
22+
// way. Locally, with four `dd … conv=fsync` writers on the same disk, those
23+
// cases took 1.4–3.6 s; with TMPDIR on /dev/shm under the same writers, every
24+
// case in the four persistence files stayed under 0.6 s.
25+
//
26+
// So the cases' wall time measured the neighbours' disk, not the driver. On a
27+
// tmpfs the same code still runs, `sync()` included: what each case asserts
28+
// (bytes on disk, atomic rename, the flush chain, the failure report) is
29+
// unchanged; only the wait for unrelated I/O is gone. Budgets, fixtures and
30+
// cases are untouched.
31+
//
32+
// A host without a writable /dev/shm (macOS, Windows) keeps its own temp
33+
// directory: the setting is absent there, not an error.
34+
const MEMORY_BACKED_TMP = '/dev/shm';
35+
36+
function writableDir(dir: string): boolean {
37+
try {
38+
accessSync(dir, constants.W_OK);
39+
return true;
40+
} catch {
41+
return false;
42+
}
43+
}
44+
45+
export default defineConfig({
46+
test: {
47+
// A late console.* must not redden a green suite (#10374): vitest's worker
48+
// forwards console output over RPC and discards the promise, and a write
49+
// landing after teardown's rpcDone() snapshot is rejected into an unhandled
50+
// error — a fully green run that exits 1. Disarming removes the mechanism.
51+
// Mechanism + measured costs: examples/app-showcase/vitest.config.ts.
52+
// Enforced repo-wide by scripts/check-console-intercept-disarm.mjs.
53+
disableConsoleIntercept: true,
54+
env: writableDir(MEMORY_BACKED_TMP) ? { TMPDIR: MEMORY_BACKED_TMP } : {},
55+
},
56+
});

0 commit comments

Comments
 (0)