Skip to content

Commit df8aac4

Browse files
test(cli): cut per-spawn cost in five migrate integration suites (#23181 slice 12) (#26207)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"} --> _Requested by **Leehom** · [project thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1XAViUhQDMdVeNR9YVhJW1y5)_ Part of #23181 (slice 12, claim 6417640032). The card stays open: this PR cuts real work in five of the eight `packages/cli/src/commands/migrate/*.integration.test.ts` rows, and closes none of the 10 s rows outright. The routes that could close them are in "Not taken" below. Clause-②: no Metadata-protocol: none `skip-changeset`: a test-only diff. The five files are `*.integration.test.ts` under `packages/cli/src/`, and `@objectstack/cli` publishes `files: [dist, README.md, CHANGELOG.md]`; the build program does not compile test files. ## The reading used The newest refresh with a per-file list is PR #25960 (14 hourly full-battery runs, `measure-test-shard-timings --diff`); #26089 landed from the same list. This family's rows in it, CI wall per file: | file | CI s (run) | spawns of the CLI per file | |:--|--:|:--| | `data-commands.absent-database` | 394.05 (`40200279306`) | 43 CLI + 2 seed boots | | `object-scope` | 180.37 (`40204927516`) | 18 CLI + 1 seed + 17 reader/writer children | | `plan.boot-parity` | 118.02 (`40204927516`) | 6, through tsx | | `plan.telemetry-sibling` | 113.36 (`40200279306`) | 5, through tsx | | `unmapped-columns` | 98.92 (`40204927516`) | 10 CLI + 1 seed + 2 reader children | | `plan.declared-default-datasource` | 95.65 (declared 80) | 4, through tsx | | `plan.nullable-tenant-columns` | 62.72 (`40204927516`) | 4 (1 serve boot, 3 plans), through tsx | | `resume.recorded-by` | 43.98 (`40200279306`) | 4 crash children + 2 serve boots, resume in-process | The family's ninth row, `plan.deferred-reads` (18.22 s), is left for a later slice: the claim caps a slice at about 8 files. ## Mechanism: the order's hypothesis does not hold here The order's usual shape, a per-case boot, is not present. Every file already builds its fixture once, in a `beforeAll` it owns or in a case that owns its fixture alone. Each file's cost is the number of CLI spawns times the per-spawn floor, run one after another. On this box (4 cores, idle, inside the verify lock): - `node bin/run-dev.js --version`: 4.93 to 4.99 s; `migrate resume --json` on an absent database: 5.25 to 6.06 s. - `tsx bin/run-dev.js --version`: 8.28 to 8.44 s; the same `migrate resume`: 9.35 to 10.09 s. - One `node -e` child that opens the database with `driver-sql` and reads it: 0.45 to 0.54 s. Most of the 5 s floor is oclif importing every command from `src/`, because no `oclif.manifest.json` ships for the source entry. That is product code and outside this claim. ## What changed 1. **Four files spawn the source entry with this process's `node`, not the `tsx` binary** (`plan.boot-parity`, `plan.telemetry-sibling`, `plan.declared-default-datasource`, `plan.nullable-tenant-columns`). The entry (`bin/run-dev.js`), its `src/` and the `NODE_ENV=development` it pins are unchanged. This is how the family's other spawners already spawn it (`data-commands.absent-database`, `object-scope`, `unmapped-columns`, and `resume.recorded-by` for `serve`). The `tsx` wrapper added a second process and its own transform, about 3.4 s per spawn. Fixture configs still load through the CLI's own `bundle-require`. `vitest-tiers.ts` still files all four as integration (they value-import `node:child_process` and name `run-dev`). 2. **`object-scope` reads and writes its fixture database in-process** through `better-sqlite3`, as `plan.boot-parity` and `plan.telemetry-sibling` already do. It used one `node` child per read: 16 `readState` calls and 1 off-shape write. Each query is the same column list and `ORDER BY id`, and the connection is opened read-write as knex opened it. Every assertion is unchanged. The diff touches no `expect`, no case and no timeout. ## Measured Same box, before (`a335f1de`) and after (`54d51cc3`), N=3 per file, one file per `vitest run --project integration --maxWorkers=1` under `os-verify-lock`. These are process wall and user CPU, which include the hook (the JSON reporter's per-file time leaves out `beforeAll`). | file | before wall s (3 runs) | after wall s (3 runs) | median change | user CPU s, medians | |:--|:--|:--|--:|:--| | `plan.boot-parity` | 63.16 · 67.17 · 63.06 | 40.54 · 37.84 · 41.89 | −36% | 87.5 → 56.2 | | `plan.telemetry-sibling` | 64.47 · 67.21 · 62.03 | 53.08 · 38.21 · 42.63 | −34% | 89.6 → 58.4 | | `plan.declared-default-datasource` | 63.37 · 63.40 · 63.33 | 48.20 · 47.17 · 51.16 | −24% | 89.4 → 69.3 | | `plan.nullable-tenant-columns` | 42.54 · 42.74 · 42.72 | 28.34 · 29.28 · 27.87 | −34% | 59.2 → 39.1 | | `object-scope` | 108.79 · 117.71 · 112.76 | 106.64 · 100.00 · 105.03 | −7% | 157.6 → 149.4 | CPU falls with wall, so this is work removed, not work packed. **Projection, not proof.** Scaling by each file's own CI-to-local ratio (CI reading ÷ local before-median): `plan.declared-default-datasource` ≈ 73 s, under its declared 80 s budget, so the next refresh should stop listing it. That is the one row this PR is expected to close. The others project to `plan.boot-parity` ≈ 76 s, `plan.telemetry-sibling` ≈ 75 s, `plan.nullable-tenant-columns` ≈ 42 s and `object-scope` ≈ 168 s, all still over 10 s. **Isolation.** Two shuffled runs of the five files, all green: - `--sequence.shuffle --sequence.seed=23181`: 5/5 files, 27/27 tests. - `--sequence.seed=7` with an execution-order reporter: file order nullable → boot-parity → object-scope → declared-default → telemetry, and `object-scope`'s 20 cases ran out of definition order; 27/27 passed. The hook files' cases only read what a hook recorded. `plan.boot-parity` and `plan.telemetry-sibling` build a fresh `mkdtemp` fixture per case. ## Not taken: the routes that would close the remaining rows None of these is inside the claim. They are open questions on the card for the seat. - **A. Spawn the built entry (`bin/run.js`).** Its floor was measured at 2.46 to 2.66 s against 5.45 to 6.07 s (`vitest.config.ts`); slice 9 reads 2.7 to 3.0 against 10.0 to 10.3 s. It changes the `NODE_ENV` posture and makes `dist/` a prerequisite, so it needs a ruling. Slice 9 raises the same question. - **B. Run a file's independent spawns concurrently.** Measured on `data-commands.absent-database` (a local experiment, not committed): pool 1 → 263.1 s wall, 431.9 s CPU; pool 2 → 152.4 s, 422.5 s; pool 4 → 120.7 s, 433.4 s. Wall falls while CPU stays flat, so this packs the work rather than removing it. `vitest.config.ts` rejects the same trade for `maxWorkers` because it spends cores that belong to the shard. - **C. A declared budget** (ruling record 6181973363 letter B, class "spawns the real producer once per single-cause fixture"). `plan.declared-default-datasource` and `audit-mail-bodies.archive` in this directory already use it. It conflicts with the card's "runs under 10 s". ## Gates All at head `54d51cc3d`. - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 58 commands over the 5-path change set. Each one's exit code was captured by redirecting, then reading `$?`, with no pipe. **All 58 exit 0.** `--ran`: "Run reconciliation — 58 derived, 58 run, 0 NOT-MEASURED, 0 UNRUN", exit 0. - Six gates first exited 3 (PREREQUISITE NOT MET), and all six exit 0 on re-run: `check-plugin-teardown-shape --self-test` (shallow clone; fixed by fetching its positive control `621a4876`), and `check:dual-build-cjs-loads`, `check:i18n`, `check:i18n-coverage`, `check:i18n-walk-parity`, `check:type-check-debt` (no built `dist/`; fixed by a full `turbo run build`, 121/121 tasks). - `eslint --no-inline-config packages/cli`: exit 0. - `pnpm --filter @objectstack/cli typecheck` (`tsc --noEmit` + `check:test-typecheck`): `VERDICT command-exit 0`. - The five files, `vitest run --project integration` (the integration tier is this diff's own layer): 5/5 files and 27/27 tests in every run above (3 ordered runs and 2 shuffled). - `packages/cli/dist` after the build contains no test file (`dist/commands/migrate` has 0 `*.test.*`). ## Acceptance notes - `unmapped-columns` has the same reader-child shape (2 children, about 1 s of its about 99 s). It is left as it is: its readers compare BLOB-carrying rows byte for byte, which is not worth the risk for one second. - `data-commands.absent-database`, `unmapped-columns` and `resume.recorded-by` already spawn through `node`. Inside this claim their only lever is B, which is not taken. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01XYjtBuXuGmMz385DUttmzN --- _Generated by [Claude Code](https://claude.ai/code/session_01XYjtBuXuGmMz385DUttmzN)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 4f48d2b commit df8aac4

5 files changed

Lines changed: 61 additions & 37 deletions

‎packages/cli/src/commands/migrate/object-scope.integration.test.ts‎

Lines changed: 25 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ import { copyFileSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'nod
4747
import { tmpdir } from 'node:os';
4848
import { dirname, join, resolve } from 'node:path';
4949
import { fileURLToPath } from 'node:url';
50+
import Database from 'better-sqlite3';
5051

5152
const HERE = dirname(fileURLToPath(import.meta.url));
5253
const CLI_ROOT = resolve(HERE, '..', '..', '..');
@@ -125,26 +126,14 @@ process.exit(0);
125126
`;
126127

127128
/** The repro's off-shape value, written past the write path: a location keyed the retired way. */
128-
const OFF_SHAPE_CHILD = `
129-
const { SqlDriver } = await import('@objectstack/driver-sql');
130-
const raw = new SqlDriver({ client: 'better-sqlite3', connection: { filename: process.env.FIXTURE_DB }, useNullAsDefault: true });
131-
await raw.knex('os21644_site').where({ id: 's1' }).update({ geo: JSON.stringify({ latitude: 1, longitude: 2 }) });
132-
await raw.disconnect();
133-
process.exit(0);
134-
`;
135-
136-
/** The flag rows and the app rows, read on a connection of our own. */
137-
const READ_STATE_CHILD = `
138-
const { SqlDriver } = await import('@objectstack/driver-sql');
139-
const raw = new SqlDriver({ client: 'better-sqlite3', connection: { filename: process.env.FIXTURE_DB }, useNullAsDefault: true });
140-
const k = raw.knex;
141-
const flags = await k('sys_migration').select('id', 'last_run_at', 'verified_at', 'blocking').orderBy('id');
142-
const site = await k('os21644_site').select('id', 'geo').orderBy('id');
143-
const product = await k('os21644_product').select('id', 'image').orderBy('id');
144-
await raw.disconnect();
145-
process.stdout.write(JSON.stringify({ flags, site, product }));
146-
process.exit(0);
147-
`;
129+
function writeOffShape(db: string): void {
130+
const conn = new Database(db);
131+
try {
132+
conn.prepare('UPDATE os21644_site SET geo = ? WHERE id = ?').run(JSON.stringify({ latitude: 1, longitude: 2 }), 's1');
133+
} finally {
134+
conn.close();
135+
}
136+
}
148137

149138
interface Run {
150139
code: number | null;
@@ -186,10 +175,22 @@ function freshDb(): string {
186175
return db;
187176
}
188177

178+
/**
179+
* The flag rows and the app rows, read on a connection of our own, in this
180+
* process: a child `node` per read cost ~0.5 s each, sixteen times a file.
181+
*/
189182
function readState(db: string): State {
190-
const out = childNode(READ_STATE_CHILD, { FIXTURE_DB: db });
191-
if (out.status !== 0) throw new Error(`could not read the fixture database (status ${out.status})\n${out.stderr}`);
192-
return JSON.parse(out.stdout) as State;
183+
// Opened as the child it replaces opened it (read-write, knex's default).
184+
const conn = new Database(db);
185+
try {
186+
return {
187+
flags: conn.prepare('SELECT id, last_run_at, verified_at, blocking FROM sys_migration ORDER BY id').all() as FlagRow[],
188+
site: conn.prepare('SELECT id, geo FROM os21644_site ORDER BY id').all() as State['site'],
189+
product: conn.prepare('SELECT id, image FROM os21644_product ORDER BY id').all() as State['product'],
190+
};
191+
} finally {
192+
conn.close();
193+
}
193194
}
194195

195196
function flagRow(state: State, id: string): FlagRow | undefined {
@@ -296,8 +297,7 @@ beforeAll(async () => {
296297

297298
// The measured repro: one off-shape value stored, then a misspelled --object --apply.
298299
const reproDb = freshDb();
299-
const offShape = childNode(OFF_SHAPE_CHILD, { FIXTURE_DB: reproDb });
300-
if (offShape.status !== 0) throw new Error(`the off-shape value was not written\n${offShape.stderr}`);
300+
writeOffShape(reproDb);
301301
const reproRightDb = join(dir, 'data', 'repro-right.db');
302302
copyFileSync(reproDb, reproRightDb);
303303
reproFullDry = await runCommand('value-shapes', ['--json'], reproDb);

‎packages/cli/src/commands/migrate/plan.boot-parity.integration.test.ts‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -57,8 +57,14 @@ import Database from 'better-sqlite3';
5757

5858
const HERE = dirname(fileURLToPath(import.meta.url));
5959
const CLI_ROOT = resolve(HERE, '..', '..', '..');
60+
/**
61+
* The source entry, spawned by this process's own `node`, as the family's other spawners
62+
* (`data-commands.absent-database`, `object-scope`, `unmapped-columns`) spawn it. Not through
63+
* the `tsx` binary: the entry, its `src/` and its pinned `NODE_ENV=development` are the same
64+
* either way, and the wrapper only adds a second process and its own transform, ~3.4 s of
65+
* every spawn (#23181 slice 12: `tsx bin/run-dev.js --version` 8.28-8.44 s, `node` 4.93-4.99 s).
66+
*/
6067
const RUN_DEV = resolve(CLI_ROOT, 'bin', 'run-dev.js');
61-
const TSX = resolve(CLI_ROOT, '..', '..', 'node_modules', '.bin', 'tsx');
6268
const CASE_TIMEOUT_MS = 300_000;
6369
const NOT_AN_OBJECT = ['sys_packages'];
6470

@@ -98,10 +104,10 @@ interface Reading {
98104
function bootThenPlan(dir: string): Reading {
99105
const dbFile = join(dir, 'boot.db');
100106
const env = childEnv(dir, dbFile);
101-
const boot = spawnSync(TSX, [RUN_DEV, 'serve', '--no-server', '--no-ui', '--no-console', '--port', '0'], {
107+
const boot = spawnSync(process.execPath, [RUN_DEV, 'serve', '--no-server', '--no-ui', '--no-console', '--port', '0'], {
102108
cwd: dir, env: { ...env, OS_MIGRATE_AND_EXIT: '1' }, encoding: 'utf8', timeout: 240_000,
103109
});
104-
const plan = spawnSync(TSX, [RUN_DEV, 'migrate', 'plan', '--json'], {
110+
const plan = spawnSync(process.execPath, [RUN_DEV, 'migrate', 'plan', '--json'], {
105111
cwd: dir, env, encoding: 'utf8', timeout: 240_000,
106112
});
107113
let payload: any;

‎packages/cli/src/commands/migrate/plan.declared-default-datasource.integration.test.ts‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,8 +55,14 @@ import Database from 'better-sqlite3';
5555

5656
const HERE = dirname(fileURLToPath(import.meta.url));
5757
const CLI_ROOT = resolve(HERE, '..', '..', '..');
58+
/**
59+
* The source entry, spawned by this process's own `node`, as the family's other spawners
60+
* (`data-commands.absent-database`, `object-scope`, `unmapped-columns`) spawn it. Not through
61+
* the `tsx` binary: the entry, its `src/` and its pinned `NODE_ENV=development` are the same
62+
* either way, and the wrapper only adds a second process and its own transform, ~3.4 s of
63+
* every spawn (#23181 slice 12: `tsx bin/run-dev.js --version` 8.28-8.44 s, `node` 4.93-4.99 s).
64+
*/
5865
const RUN_DEV = resolve(CLI_ROOT, 'bin', 'run-dev.js');
59-
const TSX = resolve(CLI_ROOT, '..', '..', 'node_modules', '.bin', 'tsx');
6066
const CASE_TIMEOUT_MS = 600_000;
6167
const SPAWN_TIMEOUT_MS = 240_000;
6268

@@ -134,7 +140,7 @@ async function freePort(): Promise<number> {
134140
*/
135141
function devBootOnce(): string {
136142
const r = spawnSync(
137-
TSX,
143+
process.execPath,
138144
[RUN_DEV, 'dev', '--port', String(port), '--compile', '--no-watch', '--no-restart', '--no-seed-admin'],
139145
{ cwd: dir, env: env({ OS_MIGRATE_AND_EXIT: '1' }), encoding: 'utf8', timeout: SPAWN_TIMEOUT_MS },
140146
);
@@ -145,7 +151,7 @@ function devBootOnce(): string {
145151

146152
/** One read-only `os migrate` door's `--json` document. */
147153
function migrateJson(args: string[]): any {
148-
const r = spawnSync(TSX, [RUN_DEV, 'migrate', ...args, '--json'], {
154+
const r = spawnSync(process.execPath, [RUN_DEV, 'migrate', ...args, '--json'], {
149155
cwd: dir, env: env(), encoding: 'utf8', timeout: SPAWN_TIMEOUT_MS,
150156
});
151157
expect(r.status, `os migrate ${args.join(' ')} failed:\n${`${r.stdout}\n${r.stderr}`.slice(-1500)}`).toBe(0);

‎packages/cli/src/commands/migrate/plan.nullable-tenant-columns.integration.test.ts‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,14 @@ import Database from 'better-sqlite3';
3232

3333
const HERE = dirname(fileURLToPath(import.meta.url));
3434
const CLI_ROOT = resolve(HERE, '..', '..', '..');
35+
/**
36+
* The source entry, spawned by this process's own `node`, as the family's other spawners
37+
* (`data-commands.absent-database`, `object-scope`, `unmapped-columns`) spawn it. Not through
38+
* the `tsx` binary: the entry, its `src/` and its pinned `NODE_ENV=development` are the same
39+
* either way, and the wrapper only adds a second process and its own transform, ~3.4 s of
40+
* every spawn (#23181 slice 12: `tsx bin/run-dev.js --version` 8.28-8.44 s, `node` 4.93-4.99 s).
41+
*/
3542
const RUN_DEV = resolve(CLI_ROOT, 'bin', 'run-dev.js');
36-
const TSX = resolve(CLI_ROOT, '..', '..', 'node_modules', '.bin', 'tsx');
3743
const CASE_TIMEOUT_MS = 300_000;
3844

3945
const APP_OBJECT = 'os25134_deal';
@@ -93,14 +99,14 @@ function childEnv(f: Fixture, extra: NodeJS.ProcessEnv = {}): NodeJS.ProcessEnv
9399

94100
/** `os serve`'s own provision-and-exit: a fresh store, the full composition schema-synced into it. */
95101
function serveBoot(f: Fixture): void {
96-
const boot = spawnSync(TSX, [RUN_DEV, 'serve', '--no-server', '--no-ui', '--no-console', '--port', '0'], {
102+
const boot = spawnSync(process.execPath, [RUN_DEV, 'serve', '--no-server', '--no-ui', '--no-console', '--port', '0'], {
97103
cwd: f.dir, env: childEnv(f, { OS_MIGRATE_AND_EXIT: '1' }), encoding: 'utf8', timeout: 240_000,
98104
});
99105
expect(boot.status, `the serving boot did not provision the database:\n${(boot.stderr ?? '').slice(-6000)}`).toBe(0);
100106
}
101107

102108
function plan(f: Fixture, json: boolean): { exit: number | null; payload: any; text: string } {
103-
const run = spawnSync(TSX, [RUN_DEV, 'migrate', 'plan', ...(json ? ['--json'] : [])], {
109+
const run = spawnSync(process.execPath, [RUN_DEV, 'migrate', 'plan', ...(json ? ['--json'] : [])], {
104110
cwd: f.dir, env: childEnv(f), encoding: 'utf8', timeout: 240_000,
105111
});
106112
let payload: any;

‎packages/cli/src/commands/migrate/plan.telemetry-sibling.integration.test.ts‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,8 +46,14 @@ import Database from 'better-sqlite3';
4646

4747
const HERE = dirname(fileURLToPath(import.meta.url));
4848
const CLI_ROOT = resolve(HERE, '..', '..', '..');
49+
/**
50+
* The source entry, spawned by this process's own `node`, as the family's other spawners
51+
* (`data-commands.absent-database`, `object-scope`, `unmapped-columns`) spawn it. Not through
52+
* the `tsx` binary: the entry, its `src/` and its pinned `NODE_ENV=development` are the same
53+
* either way, and the wrapper only adds a second process and its own transform, ~3.4 s of
54+
* every spawn (#23181 slice 12: `tsx bin/run-dev.js --version` 8.28-8.44 s, `node` 4.93-4.99 s).
55+
*/
4956
const RUN_DEV = resolve(CLI_ROOT, 'bin', 'run-dev.js');
50-
const TSX = resolve(CLI_ROOT, '..', '..', 'node_modules', '.bin', 'tsx');
5157
const CASE_TIMEOUT_MS = 300_000;
5258

5359
/** The one table outside the object set (`PackageServicePlugin.start()`'s raw DDL; see the boot-parity pin). */
@@ -116,7 +122,7 @@ const BOOT_STREAM_TAIL = 8_000;
116122
* there at `error`), and an assertion that failed without it left the CI log unable to confirm or refute one.
117123
*/
118124
function serveBoot(f: Fixture, extra: NodeJS.ProcessEnv = {}): string {
119-
const boot = spawnSync(TSX, [RUN_DEV, 'serve', '--no-server', '--no-ui', '--no-console', '--port', '0'], {
125+
const boot = spawnSync(process.execPath, [RUN_DEV, 'serve', '--no-server', '--no-ui', '--no-console', '--port', '0'], {
120126
cwd: f.dir, env: childEnv(f, { ...extra, OS_MIGRATE_AND_EXIT: '1' }), encoding: 'utf8', timeout: 240_000,
121127
});
122128
const output =
@@ -130,7 +136,7 @@ function serveBoot(f: Fixture, extra: NodeJS.ProcessEnv = {}): string {
130136
const afterBoot = (bootOutput: string, what: string): string => `${what}\n${bootOutput}`;
131137

132138
function command(f: Fixture, argv: string[], extra: NodeJS.ProcessEnv = {}): { exit: number | null; payload: any } {
133-
const run = spawnSync(TSX, [RUN_DEV, 'migrate', ...argv], { cwd: f.dir, env: childEnv(f, extra), encoding: 'utf8', timeout: 240_000 });
139+
const run = spawnSync(process.execPath, [RUN_DEV, 'migrate', ...argv], { cwd: f.dir, env: childEnv(f, extra), encoding: 'utf8', timeout: 240_000 });
134140
let payload: any;
135141
try { payload = JSON.parse(run.stdout); } catch { payload = { unparsed: `${run.stdout}\n${run.stderr}`.slice(-800) }; }
136142
return { exit: run.status, payload };

0 commit comments

Comments
 (0)