Repository navigation
Commit ec16dbe
test(dogfood): the 18.x upgrade harness — an 18.0 database boots on HEAD with its data (step 0) (#24552)
Part of #24522
Clause-②: no — tests, a fixture and scripts only; nothing a package
publishes changes (`@objectstack/dogfood` is private, `scripts/` ships
nowhere)
Step 0 of the 18.x line (ruling record 6266750393 on #15194): **the
upgrade harness**, built, green on this branch and ablated. The card's
second half, the measured plan for steps 1–7, is in the `os-dev-report`
on #24522 under `step_plan`; the seat files the child cards from it, so
this PR does not close the card.
## What it adds
- **The fixture** —
`packages/qa/dogfood/test/upgrade-harness/fixture-18.0/`.
- `database.sql` is an 18.0 showcase database after real use, dumped as
SQL text: 920 lines, 431 KB, 107 tables.
- `manifest.json` records the ref and commit it came from, every table's
row count, and every row (key columns only) of the 14 tables steps 1–7
reshape.
- It was generated from `5b14b7f3ad8e`, this branch's base on `main`,
which contains `f8ef14487` (#24513). The 18.0 release tag does not exist
yet.
- **The populator** — `upgrade-harness/populate-fixture.ts` and
`run-populate.ts`. It boots the showcase over a SQLite file, composed as
`os serve` composes it (the app's default profile, `AuditPlugin`,
`OrganizationOwnershipBootGuardPlugin`). It then uses the install
through the real HTTP doors:
- two sign-ups;
- the Default Organization made active;
- an invitation to a person with no account;
- a password reset (identity mail);
- a branding change and an encrypted `ai` setting (audit and secret);
- an inbound flow with a hook secret (flow credential);
- the declared webhook activated (a customized mirror) and a task
created and edited (its two deliveries, activity, audit);
- a runtime sharing rule;
- a share link.
It then reboots once. Every door's answer is checked, and the census
refuses to write a fixture in which any reshaped table is empty.
- **The generator** — `node scripts/gen-upgrade-harness-fixture.mjs
--ref REF [--checkout DIR]`.
- Without `--checkout`, it makes a temporary worktree of REF, installs
it, builds the dogfood package's closure, and runs HEAD's populator
there.
- With `--checkout`, it first proves the checkout equals REF everywhere
outside `packages/qa/dogfood` and `scripts/`.
- The output is not byte-reproducible (ids, timestamps and ciphertexts
are minted per run). Its coverage and provenance are reproducible.
- **The harness** —
`packages/qa/dogfood/test/upgrade-18x-harness.dogfood.test.ts`. On a
fresh copy of the fixture:
1. HEAD boots, guard included, with no ownership refusal.
2. The fixture's people sign in with their 18.0 passwords, and the
fixture's task is served.
3. After HEAD stops, every reshaped table still holds every fixture row
with identical key columns (`organization_id` among them). Counts are
checked as "at least", so rows a boot adds pass.
4. **R3:** the same fixture with one v17 trait (a physical
`organization_id` on `sys_metadata`) is refused, with the guard's code.
## Measured
| reading | value |
|:--|:--|
| harness, on this branch | `Test Files 1 passed (1)` · `Tests 5 passed
(5)` · `Duration 53.89s (… import 42.80s, tests 10.70s)` |
| generator, `--checkout` over a built tree | 29–30 s under the verify
lock |
| dogfood build closure, cold | 66 tasks, 10m51s at `--concurrency=2` on
the shared 4-vCPU box |
| reshaped rows in the fixture | `sys_webhook`=1 (customized, active) ·
`sys_http_delivery`=2 · `sys_secret`=3 · `sys_setting`=2 ·
`sys_setting_audit`=4 (2 global, 2 tenant) · `sys_flow_credential`=2 ·
`sys_email`=2 (an invitation, a reset) · `sys_invitation`=3 ·
`sys_sharing_rule`=1 · `sys_share_link`=1 · `sys_activity`=164 ·
`sys_user_permission_set`=2 · `sys_member`=3 · `sys_user`=3 |
| NULL `organization_id` in those tables | 0 everywhere: 8b stamped the
Default Organization. Step 5's moves must therefore select by shape,
never by NULL. |
## Ablation (one-time; no permanent mutation left)
Each leg ran through `node scripts/ablation-replace.mjs` on the
fixture's SQL (a data file, so there is no build or `dist/` leg). Each
restore was proven by the tool as blob `e43efe830f55` == HEAD and `git
diff HEAD` empty.
- **Leg 0 was a no-op, reported.** The first replacement contained its
own anchor, so the tool refused it (`anchor count moved 1 -> 1`) and
restored the file. Nothing was measured.
- **Leg A: a reshaped table loses a row.** The `sys_webhook` INSERT was
commented out (anchor 1 → 0, blob → `5a8d318e7b45`).
- Result: `× 3. every reshaped table keeps every 18.0 row …`, with
`AssertionError: sys_webhook row whk_8a24917e-…: expected undefined to
deeply equal { …(4) }`, so `Tests 1 failed | 4 passed (5)`.
- The boot-time seeder re-created *a* webhook row, so a count check
alone would have stayed green. The by-id check is what caught it.
- **Leg B: a forced refusal.** `sys_metadata` was created with a
physical `organization_id` (blob → `a5b65a723f9b`).
- Result: `× 1` with `the ownership boot guard refused the 18.0
database: This database is still in the v17 organization-ownership
shape: sys_metadata still carries …`, plus `× 2` and `× 4`, so `Tests 3
failed | 2 passed (5)`.
## Where it runs before merge
Dogfood skips `pull_request` and `merge_group` today. The cheapest
wiring measured is a label-gated opt-in lane, the `pack-smoke-optin.yml`
shape: one file, its build closure, and the label
`needs:upgrade-harness`. It touches `.github/workflows/**`, so it is a
separate draft PR for the maintainer on
`claude/issue-24522-harness-lane` (Tier H). Until that lands, this file
runs in `main`'s full-battery dogfood like any other dogfood test.
## Acceptance notes
- The fixture covers no `sys_sequences` or `__global__` counter rows,
because the showcase writes none. Step 7's 8d PR regenerates the fixture
with an autonumber write first (recorded in the step plan).
- `sys_invitation` holds two `inv_verify_*` rows that
`@objectstack/verify`'s `signUp` files against its audience-gate
organization. They are harness residue, not product data, and kept as
the populator produced them.
- The populator unwraps the showcase's CJS-interop default under `tsx`
(measured: without it, `requires` read `undefined` and no provider
mounted). Under vitest the export is the stack itself, so the harness
boots the identical configuration.
- The fixture carries what the platform stores: password hashes,
ciphertexts sealed under the populating process's throwaway key, and
four session rows of that run. No clear-text secret: the two placeholder
secrets the populator types (`…not-a-real-…`) have 0 hits in
`database.sql`, and `PRIVATE KEY` has 0 hits.
## Gates
Gate readings are cited at the final head in the `os-dev-report` on
#24522.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01SH2ojNSqz93pYiNSmUyQnf)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent c46b5f1 commit ec16dbe
8 files changed
Lines changed: 2769 additions & 0 deletions
File tree
- packages/qa/dogfood/test
- upgrade-harness
- fixture-18.0
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
0 commit comments