Repository navigation
Commit 2b27460
test(dogfood): activity-text-predicate pays its boot's first load at module scope and splits its slow control case (#24026)
Part of #23977
Clause-②: no
The card has two halves. Here is where each stands at this head
(`0c7592a22`):
- **The `control` case under load: met.** At base it timed out at 5000
ms on 2 of 3 loaded runs here, which reproduces the nightly's red. At
head it is four cases, and the slowest of them stays under 2.5 s (half
the timeout) on all 3 loaded runs.
- **The file under the 10 s line: not reachable from the test file.**
Head is 12.2 s unloaded, against 16.0 s at base (medians). Between 62 %
and 72 % of each data request's time is one per-request platform cost in
`@objectstack/rest` and `@objectstack/metadata-protocol` (see "Out of
scope" below). The claim's hold-off keeps platform code out of this PR,
so #23977 remains open for that half.
## What changed
One file,
`packages/qa/dogfood/test/activity-text-predicate.dogfood.test.ts`: +66
/ -26, or +41 / -1 when whitespace is ignored.
1. **The boot's first load moves to module scope.**
`payTheBootsFirstLoad()` boots a throwaway app (no objects, the
`requires` this file's stack declares, `automation: true`). It signs in
and signs up once, then stops, and the file awaits it at module scope.
Vitest runs that during collection, which no clock covers (AGENTS.md,
Build & Test). It has the shape of `@objectstack/verify`'s own
`boot-warm-up.testkit.ts`, but uses `bootStack`, which is exported
today. Nothing new is exported from `@objectstack/verify`. A fuller
warm-up was also measured: the same three extra plugins as the file's
boot made no difference (1217 and 1303 ms against 1456 ms for the hook's
boot), so it was dropped.
2. **The unmasking reader's control case is now four cases.** A
`describe` keeps the original title. Three `it`s, one per class, carry
the per-class loop body verbatim. A fourth carries the tail, the
snapshot miss and the grouping. Every probe and every `expect(` moved
intact, re-indented only.
Nothing else moved: no timeout, no budget line, no roster entry, no
project. There is no `// test-budget:` line. One `beforeAll` boot is
neither of the two classes ruling `6181973363` names, and the remaining
wait is a per-request cost, which is not a named class either.
## Where the time went at base (assumption 1)
These numbers come from an instrumented copy, run isolated on the shared
4-vCPU box, 2 to 4 runs.
| cost | measured |
|:--|:--|
| 141 sequential requests across the 22 cases | 9.3 to 11.3 s in total,
a median of 60 to 74 ms per request. The control case alone took 2.4 to
2.8 s. |
| the hook's `bootStack` | 4.17 s cold, and 1.15 to 1.46 s after a
module-scope warm-up |
| the hook's 5 sign-ups with their grants, then the REST seeding | 0.90
to 1.07 s, then 0.69 to 0.88 s |
## The PM's mechanism assumptions, measured
1. **Measured; the top three costs are in the table above.**
2. **Falsified as a speed-up.** The probes are independent reads: none
writes, and their answers were equal in sequence and concurrently.
Running them concurrently does not shorten them, though. 33 control
probes took 2016 ms in sequence and 2137 ms under `Promise.all`. The
requests run in-process (`app.request`) on one thread over an in-memory
sql.js database, so they are CPU-bound. No concurrency was added; the
case is cut instead.
3. **Confirmed: load reproduces the red.** With seven CPU hogs, the base
control case took 5028, 4892 and 5031 ms, two of those runs ending in
`Test timed out in 5000ms.` At head, its slowest part took 2488, 1906
and 1629 ms.
4. **Confirmed.** The first `bootStack` pays about 2.8 s of first load:
4.17 s cold against 1.15 to 1.46 s warm.
## A/B: isolated, interleaved, 3 rounds each
- **Base** is an untracked copy, blob-identical to `61a680956`
(`bb7cbee56a89`). **Head** is `0c7592a22`.
- Each run is one vitest process with `--maxWorkers=2`, and every run
went through `os-verify-lock`.
- The number read is the default reporter's file line: hooks plus tests,
collection excluded. The agent-detection variables were unset so that
vitest picks that reporter.
- The order was base then head, head then base, base then head.
- **Loaded** means seven `node -e 'for(;;){}'` hogs on the 4-vCPU box
for the whole round. That made runs about 2.2 to 2.7 times slower than
unloaded, which is harsher than the card's 1.5 times.
- The box is shared, so other agents' unlocked work ran alongside. Read
the ratios over the absolutes.
| run | base file ms | head file ms | base control ms | head control,
slowest part ms |
|:--|--:|--:|--:|--:|
| unloaded r1 | 17680 | 12059 | 2572 | 779 |
| unloaded r2 | 16022 | 12731 | 2493 | 804 |
| unloaded r3 | 15338 | 12247 | 2398 | 832 |
| **unloaded median** | **16022** | **12247** (-24 %) | **2493** |
**804** |
| loaded r1 | 42696 (1 failed) | 37072 | 5028 (timed out) | 2488 |
| loaded r2 | 34224 | 29319 | 4892 | 1906 |
| loaded r3 | 32896 (1 failed) | 26863 | 5031 (timed out) | 1629 |
| **loaded median** | **34224** | **29319** | **5028** | **1906** |
The hook's time (file line minus the tests' sum, unloaded) went from
5779 to 6778 ms at base to 3079 to 3415 ms at head. The four control
parts together still take what the one case took, 2321 to 2413 ms
unloaded: the split moves no work.
## Assertions kept
- The 12 `it` / `describe` title literals at base hash to
`887e5a912c4fe675` (sha256 prefix, in file order).
- Head's 14, minus the two part titles this PR adds, hash to the same
value.
- The added titles are `` `every column of a pinned parent, for the
stored values of the ${c} class` `` and `'a non-matching approval
snapshot filter, and a grouping of the activity stream'`.
- `expect(` counts 40 at base and 40 at head.
- No test was removed, skipped or moved to another project.
## Ablation: the split cannot swallow a failure
This was a one-time run; no test was left behind.
- **Prediction, written before the run:** the three per-class parts go
red on `expect(snapHit.rows).toBe(1)`, and everything else stays green,
so 3 failed / 22 passed.
- **The mutation:** `node scripts/ablation-replace.mjs` (wrap mode)
changed the per-class `snapHit` probe's filter from the class's stored
snapshot value to `NOMATCH`. It landed: anchor x1 → x0, blob
`8b5eb87856f9` → `2ac1463e3d0a`.
- **The result:** 3 failed / 22 passed, each failure `expected +0 to be
1`, as predicted.
- **The restore:** the blob is back to HEAD's `8b5eb87856f9`, and `git
diff HEAD` is empty.
## Tests and gates, at `0c7592a22`
- `pnpm --filter @objectstack/dogfood exec vitest run --maxWorkers=2
test/activity-text-predicate.dogfood.test.ts
test/activity-parent-read-gate.dogfood.test.ts
test/audit-log-parent-read-gate.dogfood.test.ts`: 3 files passed, 39
tests passed, exit 0.
- `pnpm --filter @objectstack/dogfood typecheck` (`tsc --noEmit`): exit
0. `tsc --listFilesOnly` lists the changed file.
- `pnpm exec eslint --no-inline-config` over the changed file: exit 0.
The repo-wide lint run is this PR's `Lint & Repo Gates` check.
- `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack`, taken from the merge base, derived 53
commands:
- 52 exited 0.
- `pnpm check:dual-build-cjs-loads` exited 3 with PREREQUISITE NOT MET,
because it reads every package's `dist/`. That is NOT MEASURED locally;
CI's run is the reading.
- `--ran`, with an exit code recorded per command, accounted for all 53:
52 run, 1 NOT-MEASURED.
- The whole dogfood suite was not run locally. It runs on `main`'s full
battery, not on PR CI.
## No changeset
The only file changed is a test of `@objectstack/dogfood`, which is
private, and no published package's `files[]` reaches it. The seat
applies `skip-changeset`.
## Out of scope: reported to the seat, not fixed here
**The REST API-exposure gate re-validates the metadata of every object
on every data request.**
- **The path.** Every `/data/OBJECT` read and query request enters
`enforceApiAccess` (`packages/rest/src/rest-server.ts`) to read one
object's `enable` flags. From there it calls `loadObjectItems`
(`packages/rest/src/meta-item-read-gate.ts`), then the protocol's
`getMetaItems({ type: 'object' })`, then `readFlattenedMetaItems(...,
'served')` (`packages/metadata-protocol/src/protocol.ts`). That ends in
`decorateMetadataItems` → `computeMetadataDiagnostics`
(`packages/metadata-protocol/src/metadata-diagnostics.ts`), which runs a
full Zod `safeParse` of each registered object's schema.
- **The profile.** A CPU profile of the control case's 33 list calls
(inspector, 200 µs sampling) put 1311 ms of 2841 ms sampled inside
`readFlattenedMetaItems`, 1244 ms of that inside
`computeMetadataDiagnostics`.
- **The memoization probe.** It was measurement-only and not committed.
Memoizing that one object-list read for the same 33 calls cut them from
2376 and 2255 ms to 865 and 668 ms.
- **What it would close.** By that ratio, it is roughly 5 to 7 s of this
file's 9 to 11 s of tests. That is the 10 s half of #23977.
## Acceptance notes
- Under the seven-hog load (head r1), one per-class case, the masked
reader's query-door case, reached 2604 ms, over half its timeout.
Unloaded it takes 0.5 to 0.7 s. It is not the card's control case and is
left as it is; the platform finding above would shorten it too.
- The warm-up moves about 2.8 s into collection rather than removing it,
so each run's `import` time grows by that much. Collection is not
clocked.
---
_Generated by [Claude
Code](https://claude.ai/code/session_018t6a5rvqPRvEDMCVVW9kYT)_
Co-authored-by: Claude <noreply@anthropic.com>1 parent 75ced06 commit 2b27460
1 file changed
Lines changed: 66 additions & 26 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
62 | 93 | | |
63 | 94 | | |
64 | 95 | | |
| |||
467 | 498 | | |
468 | 499 | | |
469 | 500 | | |
470 | | - | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
471 | 507 | | |
472 | | - | |
473 | | - | |
474 | | - | |
475 | | - | |
476 | | - | |
477 | | - | |
478 | | - | |
479 | | - | |
480 | | - | |
481 | | - | |
482 | | - | |
483 | | - | |
484 | | - | |
485 | | - | |
486 | | - | |
487 | | - | |
488 | | - | |
489 | | - | |
490 | | - | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
491 | 529 | | |
492 | | - | |
493 | | - | |
494 | | - | |
495 | | - | |
496 | | - | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
497 | 538 | | |
498 | | - | |
499 | 539 | | |
500 | 540 | | |
0 commit comments