Skip to content

Commit c1bb4b5

Browse files
test(plugin-pinyin-search): seed the boot-reconcile history in batches and give the kernel suite a 30 s budget (#25775)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"} --> _Requested by **Leehom** · [project thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1XWP2L6oD6QXq7uAFXPnHfYb)_ Fixes #25749 Fixes #25750 The anchor #25750 closes with this fix (triage 6394128781); the seat claim on it, 6396273722, names this branch. Clause-②: no Metadata-protocol: none ## What changed One file, test-only: `packages/plugins/plugin-pinyin-search/src/boot-reconcile.integration.test.ts`. Production source and `vitest.config.ts` are untouched. 1. **Route (a), the seed.** `FixturePlugin.directInsert` wrote the 2,500-row history with one awaited `driver.create` per row. It now calls `driver.bulkCreate` in slices of 500 rows, five statements in all. - A single 2,500-row `bulkCreate` does not work on SQLite. Measured: all 7 seeded cases fail in boot with `SQLITE_ERROR: too many terms in compound SELECT`. `driver-sql` sends a batch as one `insert … select … union all select …` statement, and SQLite's default compound-SELECT ceiling is 500 terms. Hence the slices (`INSERT_BATCH = 500`). - `bulkCreate` keeps the given `created_at` / `updated_at`. `stampInsertTimestamps` only fills a null or missing column, the same function `create()` calls, so the `HISTORIC` stamps the watermark cases rely on are unchanged. Every assertion is unchanged too, and they still pass. 2. **Route (b), the budget, added on measured evidence.** With the seed batched, 2 of 8 cases still went past 5 s under load. A timing probe (one-off, not committed) showed the seed costs only 0.2 to 0.6 s under 20 busy loops. The rest of the time is the real-kernel boots and the 2,500-row reconcile walks the cases exist to measure: 6.8 s for the first-boot/second-boot case and 10.1 s for the forced rebuild, seed excluded. So the kernel-booting `describe` gets `{ timeout: 30_000 }` (the `packages/runtime/*integration.test.ts` precedent), with a comment that names the boots and walks it pays for. The fake-engine `describe` stays on the default. ## Measurements 4-core container, `pnpm --filter @objectstack/plugin-pinyin-search exec vitest run --maxWorkers=2 --reporter=verbose src/boot-reconcile.integration.test.ts`. "Load" means 20 busy loops (`while :; do :; done`), the load used in reproduction 6394530515. Each run held `scripts/pm/os-verify-lock.sh`. Times are per test in ms; ✗ means `Test timed out in 5000ms`. | case | main idle (2 runs) | main load (2 runs) | head idle | head load (2 runs) | |---|---|---|---|---| | first boot walks every row; unchanged second boot reads none | 1474 / 1578 | 8125 ✗ / 8450 ✗ | 1335 | 6744 / 7884 | | second boot fills a row a hook-bypassing writer added | 985 / 974 | 6752 ✗ / 5720 ✗ | 585 | 3834 / 4142 | | control: hook write stamps `__search`, reconcile off the boot path | 789 / 865 | 6270 ✗ / 6720 ✗ | 446 | 3868 / 3502 | | control: forced rebuild walks every row | 1473 / 1502 | 10363 ✗ / 11576 ✗ | 1139 | 8373 / 8543 | | readiness does not wait for the reconcile | 897 / 880 | 7207 ✗ / 4676 | 454 | 3078 / 3638 | | shutdown stops the reconcile between pages | 901 / 864 | 5745 ✗ / 5420 ✗ | 577 | 3164 / 3661 | | without the ledger every boot walks every row | 874 / 793 | 5201 ✗ / 6187 ✗ | 454 | 3008 / 2816 | | a failing reconcile is logged (fake engine) | 4 / 5 | 53 / 24 | 5 | 30 / 34 | | **result** | 8/8, 8/8 | **1/8, 2/8** | 8/8 | **8/8, 8/8** | The intermediate commit, with the seed batched and no budget (`11d10c22`), ran 8/8 idle twice (slowest 1094 ms). Under load it ran 6/8: first-boot 6899 ✗ and forced rebuild 9482 ✗. That is the measurement behind route (b). The secondary error `Cannot open database because the directory does not exist` shows up 10 and 6 times in the two `main` load logs and 0 times in the three head logs. It was fallout from `afterEach` removing the temp directory under a timed-out body, and it goes away with the fix, as the order expected. It was not patched separately. ## Verification at head `d1f610e4d1` - `pnpm --filter @objectstack/plugin-pinyin-search typecheck`: exit 0 (`check:test-typecheck: OK`, 0 errors). - `pnpm --filter @objectstack/plugin-pinyin-search test`: Test Files 4 passed (4), Tests 39 passed (39). Lock verdict `command-exit 0`. - `eslint packages/plugins/plugin-pinyin-search --no-inline-config`: exit 0. - Derived gate families (`node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack`, 57 commands): 56 exit 0. `pnpm check:dual-build-cjs-loads` exited 3 (PREREQUISITE NOT MET: no `dist/` for about 53 packages without a full workspace build), so it is NOT MEASURED locally and CI's run is the measurement. `--ran` reconciliation, with exit codes recorded: `57 derived, 56 run, 1 NOT-MEASURED, 0 UNRUN`. `node scripts/check-plugin-teardown-shape.mjs --self-test` exited 3 on the first pass from a shallow clone, then 0 after `git fetch --unshallow`. The final pass at this head recorded 0. - No ablation. The diff changes a fixture's write path and a timeout. It changes no behaviour the tests pin, and every assertion is byte-identical. ## Acceptance notes - `SqlDriver.bulkCreate` on SQLite refuses a batch of more than 500 rows (`too many terms in compound SELECT`). The in-tree batch callers stay under that limit: `seed-loader.ts` `BULK_BATCH_SIZE = 200`, `core/src/utils/bulk-write.ts` `DEFAULT_BATCH_SIZE = 200`, `import-runner.ts` `MAX_CREATE_BATCH_SIZE = 200`. I measured no public door that reaches it, so this is an observation, not a filed card. Carrier: none. - The second commit's message says the forced-rebuild walk took 9.9 s. The probe measured 10,288 ms with a 222 ms seed, which is 10.1 s. The third commit corrects the code comment. The pushed message is left as it is, with no history rewrite. - `skip-changeset`: test-only. A `*.test.ts` file is outside the package's `files[]` (`dist` only), so nothing published changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01T5digj2EfFsbXihqJEo1ix --- _Generated by [Claude Code](https://claude.ai/code/session_01T5digj2EfFsbXihqJEo1ix)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 5612dd9 commit c1bb4b5

1 file changed

Lines changed: 20 additions & 3 deletions

File tree

‎packages/plugins/plugin-pinyin-search/src/boot-reconcile.integration.test.ts‎

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,8 @@ const SYSTEM = { isSystem: true } as const;
3737
const HISTORIC = '2026-01-01T00:00:00.000Z';
3838
const ROWS = 2_500;
3939
const PAGE = 1_000;
40+
/** SQLite's compound-SELECT ceiling: the most rows one `bulkCreate` can carry. */
41+
const INSERT_BATCH = 500;
4042

4143
const PROBE_OBJECT = {
4244
name: PROBE,
@@ -98,9 +100,19 @@ class FixturePlugin implements Plugin {
98100
};
99101
}
100102

101-
/** A write that bypasses every hook and the audit stamp: an import, a migration. */
103+
/**
104+
* A write that bypasses every hook and the audit stamp: an import, a migration.
105+
* Batched `bulkCreate`, not one awaited `create` per row (#25749): 2,500
106+
* single-row autocommit inserts timed these cases out under CI load. A batch
107+
* is one `… UNION ALL SELECT …` statement on SQLite, which refuses more than
108+
* 500 terms ("too many terms in compound SELECT"), hence the slices. The
109+
* driver keeps the given `created_at`/`updated_at`, so the history stays older
110+
* than any watermark.
111+
*/
102112
async directInsert(rows: Array<Record<string, unknown>>): Promise<void> {
103-
for (const row of rows) await this.driver.create(PROBE, row);
113+
for (let i = 0; i < rows.length; i += INSERT_BATCH) {
114+
await this.driver.bulkCreate(PROBE, rows.slice(i, i + INSERT_BATCH));
115+
}
104116
}
105117

106118
async row(id: string): Promise<Record<string, any> | null> {
@@ -167,7 +179,12 @@ function history(): Array<Record<string, unknown>> {
167179
}));
168180
}
169181

170-
describe('#25634 the boot reconcile reads what changed since its last completed walk', () => {
182+
// [#25749] Each case pays for real-kernel boots over a SQLite file (two in
183+
// most) and at least one full 2,500-row reconcile walk; the forced-rebuild
184+
// case walks twice. Idle, the slowest case takes about 1 s. On a 4-core box
185+
// with 20 busy loops the boots and walks alone took 6.8 s and 10.1 s (seed
186+
// excluded), past vitest's 5 s default, so the suite carries its own budget.
187+
describe('#25634 the boot reconcile reads what changed since its last completed walk', { timeout: 30_000 }, () => {
171188
it('a first boot walks every row; an unchanged second boot reads none — and a receipt sits in sys_migration', async () => {
172189
const first = await boot({ before: (f) => f.directInsert(history()) });
173190
await first.plugin.whenReconciled();

0 commit comments

Comments
 (0)