Skip to content

Commit e6dfeb0

Browse files
test(verify): pay the first boot load at collection and share one boot per configuration in two harness suites (#26175)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"} --> _Requested by **Leehom** · [project thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1XVdKSKTiX7AWsYagk2P8mJT)_ Part of #23181 (slice 9). The card stays open for its other rows. Clause-②: no Metadata-protocol: none Test files only, so this PR carries `skip-changeset`: both files are `*.test.ts` under `packages/verify/src`, which the package's build does not ship. ## What changed Two `@objectstack/verify` harness suites booted a fresh in-process stack in every case and paid the boot's first load inside the first case's clock. - `src/harness.app-default-profile.test.ts`: `await warmUpBoot()` at module scope (`boot-warm-up.testkit.ts`, the convention slices 3 and 4 used). The first two cases now read ONE boot of the declared-default configuration from a `beforeAll`. The first reads the profile the boot wired; the second signs a member up and reads as them. Neither reads what the other writes. Boots: 4 → 3. - `src/harness.org-context.test.ts`: `await warmUpBoot()` at module scope. One `beforeAll` boots `app` with the flag off, records what the cases read, and stops. Then it boots with `orgContext: true`, records, and stops, in the same order the ruling case always booted them. The ruling case compares the two recorded postures, the bind case reads the `orgContext: true` boot, and the default-boot case reads the default boot. Boots: 4 → 2. Every assertion reads the same value of the same boot shape (same configuration, same options) as before. The test full names are identical before and after (sorted `fullName` lists compared). ## The reading, and the rows this PR brings under 10 s The reading is refresh PR #25960's per-file list (runs `40200279306` and others in its window). The newest LANDED refresh, #25409, lists no per-file rows; slice 8's report records why. Of this family, #25960 still lists five rows: | file | #25960 slowest (runs over / executed) | local before → after, median of 3 | this PR | |:--|--:|--:|:--| | `src/harness.app-default-profile.test.ts` | 18.90 s (1 / 2) | 6608 → 3126 ms (×0.47) | **under 10 s**: projected 8.9 s in CI | | `src/harness.org-context.test.ts` | 16.60 s (1 / 2) | 6741 → 2431 ms (×0.36) | **under 10 s**: projected 6.0 s in CI | | `src/harness.key-custody.test.ts` | 22.41 s (2 / 2) | 6852–7338 ms, untouched | open: see Acceptance notes | | `src/harness.one-composition.test.ts` | 17.89 s (2 / 2) | 9206–10354 ms, untouched | open: see Acceptance notes | | `src/handle.test.ts` | 17.30 s (1 / 2) | 3857–4253 ms, untouched | open: already warmed, one shared boot | The pin is the next landed refresh: it should no longer name the two rows marked under 10 s. ## Measurement On one 4-vCPU box, under `os-verify-lock`. It is a shared box, so the ratios carry the result rather than the absolute times. Each run is one `vitest run --maxWorkers=2` process per file. The number is the default reporter's file line (the module's duration: hooks included, collection excluded), which is the figure the timing refresh reads. The json reporter's `endTime - startTime` leaves out a `beforeAll` that runs before the first case, so it is not used. BASE `a6e8f95d6` copies and HEAD files were interleaved A/B in the same lock hold. | run | app-default BASE | app-default HEAD | org-context BASE | org-context HEAD | |--:|--:|--:|--:|--:| | 1 | 6488 | 3126 | 6523 | 2431 | | 2 | 7591 | 3523 | 6741 | 2702 | | 3 | 6608 | 3023 | 7223 | 2263 | **Cases still isolated.** `vitest run --sequence.shuffle --sequence.seed=N` over both files, seeds 1 to 5: 11 / 11 passed each time. With seed 1, the member sign-up case ran before the profile read on the shared boot. With seed 2, the default-boot case ran before the ruling case. **Where a boot's time goes.** A CPU profile of six warm `bootStack` calls (0.82–1.09 s each) puts the largest share in the default datasource's schema sync on sql.js (`knex` `ensureConnection` / `query` → `driver-sqlite-wasm` `_query`, `driver-sql` `initObjects`). A test file can only boot fewer times; it cannot make a boot cheaper. ## Acceptance notes - `harness.key-custody.test.ts` (22.41 s, 2 / 2) is untouched. Each of its 5 boots runs under its own key posture, so no two boots can be shared. A warm-up is excluded there by slice 4's seat ruling (option A on `6221027193`): it would build the harness data key outside the keyless posture the file inspects. - `harness.one-composition.test.ts` (17.89 s, 2 / 2) is untouched. It is already warmed. Each of its 12 boots is the subject of its case (the instance rule and composition), so no two can be shared. It reads 9.2–10.4 s locally. - `handle.test.ts` (17.30 s, 1 of 2 runs over) is untouched. It is already warmed and boots once per file. The one reading over 10 s came from the window's most loaded run (`40200279306`, where all five rows took their slowest). - `#23181` remains open for those rows and for the other packages' rows. ## Tests - `pnpm --filter @objectstack/verify typecheck`: VERDICT command-exit 0 (`check:test-typecheck: OK`, 0 debt). - `eslint --no-inline-config packages/verify`: exit 0. - Both files: 11 / 11 passed in 6 A/B runs and 5 shuffled runs. - `@objectstack/verify` whole suite (`vitest run --maxWorkers=2`): Test Files 46 passed (46), Tests 393 passed (393), VERDICT command-exit 0. - Gates at `cb29c5b4d`: `dispatch-gates --commands` derived 55 commands, and all 55 exit 0. `--ran` reports "55 derived, 55 run, 0 NOT-MEASURED, 0 UNRUN" and exits 0. On the first pass two gates exited 3 with PREREQUISITE NOT MET, which is not a verdict: `check-plugin-teardown-shape --self-test` (shallow clone; fixed by fetching `621a4876`) and `check:dual-build-cjs-loads` (eight packages had no `dist/`; fixed by building them). Both exit 0 on the re-run. --- _Generated by [Claude Code](https://claude.ai/code/session_01Bmz18vLaqY8KsiVaCXL5zS)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 3f970b9 commit e6dfeb0

2 files changed

Lines changed: 99 additions & 63 deletions

File tree

‎packages/verify/src/harness.app-default-profile.test.ts‎

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
// fixture object is denied unless the app's declared profile is actually wired.
2828
// A name-only assertion could not tell "wired" from "wired but inert".
2929

30-
import { describe, it, expect, afterAll } from 'vitest';
30+
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
3131
import { defineStack } from '@objectstack/spec';
3232
import { ObjectSchema, Field } from '@objectstack/spec/data';
3333
import { PermissionSetSchema } from '@objectstack/spec/security';
@@ -48,10 +48,16 @@ import {
4848
// in `DEBT`, `TEST_DEBT` or `EXEMPT`. The layer is at zero with no ledger, so
4949
// an error a new file introduces is red on arrival.
5050
import { bootStack, type VerifyStack } from './harness.js';
51+
import { warmUpBoot } from './boot-warm-up.testkit.js';
5152

5253
// Booting the full in-process stack runs well past vitest's 5s default.
5354
const BOOT_TIMEOUT = 120_000;
5455

56+
// The boot's first load, paid during collection rather than in the first
57+
// boot's clock (see `boot-warm-up.testkit.ts`). Every boot below that is not
58+
// the opt-out case uses the default options.
59+
await warmUpBoot();
60+
5561
const APP_DEFAULT_SET = 'appdefault_member_default';
5662

5763
// `defineStack` enforces `${manifest.namespace}_${shortName}` on every object
@@ -142,10 +148,20 @@ const wiredBaseline = (stack: VerifyStack): Promise<string | null> =>
142148
stack.kernel.getServiceAsync<string | null>('security.fallbackPermissionSet');
143149

144150
describe('bootStack honours the app-declared default permission set (#7001)', () => {
151+
// The first two cases read ONE boot of the declared-default configuration
152+
// with the default options: the first reads the profile the boot wired, a
153+
// fixed answer of the boot, and the second signs a member up and reads as
154+
// them, which changes no wiring. Neither reads what the other writes, so
155+
// they hold in either order.
156+
let declaredDefault: VerifyStack;
157+
beforeAll(async () => {
158+
declaredDefault = await boot(buildAppWithDeclaredDefault());
159+
}, BOOT_TIMEOUT);
160+
145161
it(
146162
'wires the SAME profile `serve` wires for the same config',
147163
async () => {
148-
const stack = await boot(buildAppWithDeclaredDefault());
164+
const stack = declaredDefault;
149165

150166
// The exact expression `serve.ts` evaluates for its `fallbackPermissionSet`.
151167
const servesChoice = appDefaultPermissionSetName(appWithDeclaredDefault.permissions);
@@ -162,7 +178,7 @@ describe('bootStack honours the app-declared default permission set (#7001)', ()
162178
it(
163179
'and that profile is load-bearing: a fresh member holds the declared grants',
164180
async () => {
165-
const stack = await boot(buildAppWithDeclaredDefault());
181+
const stack = declaredDefault;
166182
await stack.signIn(); // first user is the seeded dev admin
167183
const memberToken = await stack.signUp('appdefault-member@verify.test');
168184

‎packages/verify/src/harness.org-context.test.ts‎

Lines changed: 80 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,13 @@
2525
// lives in the dogfood suite, against a real app:
2626
// `packages/qa/dogfood/test/org-scoped-sharing-rule-listing.dogfood.test.ts`.
2727

28-
import { describe, it, expect, afterEach } from 'vitest';
28+
import { describe, it, expect, beforeAll, afterEach } from 'vitest';
2929
// `.js` extension deliberately: this package resolves NodeNext, so an
3030
// extensionless relative import is a TS2835 the type-check-debt ratchet counts
3131
// (the sibling test files predate the gate and carry theirs in the ledger — a
3232
// NEW one would raise it, which is not something a test file gets to do).
33-
import { bootStack, type VerifyStack } from './harness.js';
33+
import { bootStack, type BootOptions, type VerifyStack } from './harness.js';
34+
import { warmUpBoot } from './boot-warm-up.testkit.js';
3435

3536
const app = {
3637
manifest: {
@@ -56,6 +57,11 @@ const SYS = { isSystem: true } as const;
5657
// Booting the full in-process stack runs well past vitest's 5s default.
5758
const BOOT_TIMEOUT = 120_000;
5859

60+
// The boot's first load, paid during collection rather than in the first
61+
// boot's clock (see `boot-warm-up.testkit.ts`). The first boot below uses the
62+
// default options.
63+
await warmUpBoot();
64+
5965
afterEach(() => {
6066
delete process.env.OS_TENANCY_POSTURE;
6167
});
@@ -96,7 +102,58 @@ async function sessionOrgId(stack: VerifyStack): Promise<string | null> {
96102
return null;
97103
}
98104

105+
/** What one boot of `app` answered, read while it was live. */
106+
interface BootReading {
107+
posture: TenancyShape;
108+
// eslint-disable-next-line @typescript-eslint/no-explicit-any
109+
orgs: any[];
110+
sessionOrgId: string | null;
111+
/** The roles of the admin's `sys_member` rows in the first organization. */
112+
adminRoles: unknown[];
113+
}
114+
115+
/**
116+
* Boot `app` with `opts`, read the posture, the organizations, the org id the
117+
* admin's session carries and the admin's membership roles, and stop. Each
118+
* read is one the cases below used to make on a boot of their own with the
119+
* same options.
120+
*/
121+
async function bootAndRead(opts: BootOptions): Promise<BootReading> {
122+
const stack = await bootStack(app, opts);
123+
try {
124+
const shape = await posture(stack);
125+
// eslint-disable-next-line @typescript-eslint/no-explicit-any
126+
const ql = await stack.kernel.getServiceAsync<any>('objectql');
127+
const orgs = rowsOf(await ql.find('sys_organization', { limit: 10, context: SYS }));
128+
const orgId = await sessionOrgId(stack);
129+
const users = rowsOf(
130+
await ql.find('sys_user', { where: { email: 'admin@objectos.ai' }, limit: 1, context: SYS }),
131+
);
132+
const memberships = rowsOf(
133+
await ql.find('sys_member', {
134+
where: { user_id: users[0]?.id, organization_id: orgs[0]?.id },
135+
limit: 10,
136+
context: SYS,
137+
}),
138+
);
139+
return { posture: shape, orgs, sessionOrgId: orgId, adminRoles: memberships.map((m) => m.role) };
140+
} finally {
141+
await stack.stop();
142+
}
143+
}
144+
99145
describe('bootStack orgContext (#7762)', () => {
146+
// Two boots for the three cases that read one: the flag off (the default
147+
// boot) and on, in that order, each stopped before the next starts — the
148+
// order the ruling case always booted them in. Every case only reads what a
149+
// boot answered, so no case depends on another's run or on their order.
150+
let off: BootReading;
151+
let on: BootReading;
152+
beforeAll(async () => {
153+
off = await bootAndRead({});
154+
on = await bootAndRead({ orgContext: true });
155+
}, BOOT_TIMEOUT * 2);
156+
100157
it(
101158
'THE RULING: the tenancy posture and `degraded` are IDENTICAL with the flag off vs on',
102159
async () => {
@@ -106,21 +163,8 @@ describe('bootStack orgContext (#7762)', () => {
106163
// so binding the admin to an organization must not move it. If this ever
107164
// fails, `orgContext` is making the open core claim a wall it does not
108165
// have and the option must not ship in that shape.
109-
const off = await bootStack(app, {});
110-
let baseline: TenancyShape;
111-
try {
112-
baseline = await posture(off);
113-
} finally {
114-
await off.stop();
115-
}
116-
117-
const on = await bootStack(app, { orgContext: true });
118-
let withOrg: TenancyShape;
119-
try {
120-
withOrg = await posture(on);
121-
} finally {
122-
await on.stop();
123-
}
166+
const baseline = off.posture;
167+
const withOrg = on.posture;
124168

125169
expect(withOrg).toEqual(baseline);
126170
// Spelled out too, so a future refactor of `posture()` cannot make the
@@ -137,23 +181,16 @@ describe('bootStack orgContext (#7762)', () => {
137181
it(
138182
'binds the admin to a real organization, and their session carries it',
139183
async () => {
140-
const stack = await bootStack(app, { orgContext: true });
141-
try {
142-
// eslint-disable-next-line @typescript-eslint/no-explicit-any
143-
const ql = await stack.kernel.getServiceAsync<any>('objectql');
144-
const orgs = rowsOf(await ql.find('sys_organization', { limit: 10, context: SYS }));
145-
expect(orgs.length, 'the default-org bootstrap minted an organization').toBeGreaterThan(0);
146-
147-
// `session.activeOrganizationId` is the ONE field `resolveAuthzContext`
148-
// reads into `tenantId` → `ExecutionContext`. Without it on the session
149-
// row, every org-scoped read downstream is org-less no matter how many
150-
// organizations exist in the table.
151-
const orgId = await sessionOrgId(stack);
152-
expect(orgId, "the admin's session carries an active organization").toBeTruthy();
153-
expect(orgs.map((o) => o.id)).toContain(orgId);
154-
} finally {
155-
await stack.stop();
156-
}
184+
const { orgs } = on;
185+
expect(orgs.length, 'the default-org bootstrap minted an organization').toBeGreaterThan(0);
186+
187+
// `session.activeOrganizationId` is the ONE field `resolveAuthzContext`
188+
// reads into `tenantId` → `ExecutionContext`. Without it on the session
189+
// row, every org-scoped read downstream is org-less no matter how many
190+
// organizations exist in the table.
191+
const orgId = on.sessionOrgId;
192+
expect(orgId, "the admin's session carries an active organization").toBeTruthy();
193+
expect(orgs.map((o) => o.id)).toContain(orgId);
157194
},
158195
BOOT_TIMEOUT,
159196
);
@@ -165,32 +202,15 @@ describe('bootStack orgContext (#7762)', () => {
165202
// Default Organization (a boot invariant), and the admin is bound to it
166203
// without asking — the shape `objectstack dev` / `serve` boot. There is
167204
// no org-less `single` boot left to keep a fixture on (ADR-0131 D11).
168-
const stack = await bootStack(app, {});
169-
try {
170-
// eslint-disable-next-line @typescript-eslint/no-explicit-any
171-
const ql = await stack.kernel.getServiceAsync<any>('objectql');
172-
const orgs = rowsOf(await ql.find('sys_organization', { limit: 10, context: SYS }));
173-
expect(orgs.map((o) => o.slug)).toEqual(['default']);
174-
expect(await sessionOrgId(stack)).toBe(orgs[0].id);
175-
// The harness line this case pins is the owner bind (ADR-0093 D7), on
176-
// for every boot: the admin OWNS the Default Organization. The two
177-
// assertions above hold without it (the boot invariant creates the
178-
// organization and the membership reconciler binds the admin), but as
179-
// `member` — which is what an `objectstack dev` admin never is.
180-
const users = rowsOf(
181-
await ql.find('sys_user', { where: { email: 'admin@objectos.ai' }, limit: 1, context: SYS }),
182-
);
183-
const memberships = rowsOf(
184-
await ql.find('sys_member', {
185-
where: { user_id: users[0]?.id, organization_id: orgs[0].id },
186-
limit: 10,
187-
context: SYS,
188-
}),
189-
);
190-
expect(memberships.map((m) => m.role)).toEqual(['owner']);
191-
} finally {
192-
await stack.stop();
193-
}
205+
const { orgs } = off;
206+
expect(orgs.map((o) => o.slug)).toEqual(['default']);
207+
expect(off.sessionOrgId).toBe(orgs[0].id);
208+
// The harness line this case pins is the owner bind (ADR-0093 D7), on
209+
// for every boot: the admin OWNS the Default Organization. The two
210+
// assertions above hold without it (the boot invariant creates the
211+
// organization and the membership reconciler binds the admin), but as
212+
// `member` — which is what an `objectstack dev` admin never is.
213+
expect(off.adminRoles).toEqual(['owner']);
194214
},
195215
BOOT_TIMEOUT,
196216
);

0 commit comments

Comments
 (0)