Repository navigation
Commit c7168ac
fix(service-package,cloud-connection): convert stored package bodies forward at the sys_packages and marketplace doors (#24028)
Fixes #23974
Clause-②: yes (widening)
Three package doors called the registry without going through the
ADR-0087 versioned window that PR #23962 opened at the registration
door. This PR routes each door through that window, or records why the
door stays strict. The conversion entry point
`ObjectQL.convertPackageBodyForward` is reused as it stands. Nothing
changes in `packages/objectql/**`, `packages/metadata-core/**` or
`packages/spec/**`, and `SchemaRegistry.installPackage` gets no second
judge.
## Premise check, on `origin/main` (base `92fd343`, merged up to
`fd9664ffa`)
- **Door 1 holds.** `service-package/src/index.ts:715` called
`registry.installPackage(rec.manifest)`, and one `try/catch` wrapped the
whole loop. That catch logged anything other than a seam failure at
**`debug`**. So the first record the registry refused also dropped every
record after it, and nothing above `debug` said so.
- **Door 2 holds** (`protocol.ts:31393`). Its premise as a conversion
door does not, see door 2 below.
- **Door 3 holds, on both paths.** Rehydrate
(`:780`/`:790`/`:795`/`:827`) and the install path
(`:1554`/`:1618`/`:1625`) both hand `register` the raw body and hand the
same raw body to `bindArtifactHandlers` / `applySideEffects`.
- **Correction to the card's reach.** HotCRM 4.0.1 exactly as published
declares `engines.protocol: ^17.7.0`. This runtime is protocol 18, so
the ADR-0087 **D1 handshake** at doors 1 and 3 (and
`assertProtocolCompat` at door 2) refuses that body by name **before**
the registry or any conversion sees it. Door 1 `continue`s past it, so a
HotCRM-4.0.1 row alone does not stop hydration on v18 (pinned below).
The window at these doors opens for a stored body whose range **admits**
major 18 and whose floor predates a retirement, for example `>=17.7.0`.
The converting pins restamp the fixture's body that way: same 17.7.0
floor, same pages. The loop defect is real for any registry refusal and
is fixed below.
## Door 1: `sys_packages` boot hydration
(`@objectstack/service-package`)
- Each stored body goes through `objectql.convertPackageBodyForward`
(reached through the engine the plugin already holds, as `registry` is)
before `registry.installPackage` judges it. The stored row is not
rewritten, so it converts again at every boot. This follows the ADR-0087
addendum: metadata at rest replays the chain at every load.
- The loop survives one bad record. A record the registry still refuses
after conversion is logged at **`error`**, naming the package id,
carrying the refusal's message, and with the refusal attached. The line
also states the consequence (absent from the registry for this process
while its row remains) and the fix. The records after it keep hydrating.
- The `isSeamUnreadable` WARN is untouched.
- `Hydrated N package(s) from sys_packages into registry` now counts
exactly the records the registry **accepted** in that pass. It excludes
records the D1 handshake refused, records already registered, and
records the registry refused (each of those is reported by name).
## Door 2: `protocol.installPackage` / `POST /api/v1/packages` stays
STRICT (authoring-side)
The decision comes from what each caller actually sends. It is recorded
at the call site in `protocol.ts`, and the code is unchanged.
- **`POST /api/v1/packages`**
(`packages/runtime/src/domains/packages.ts`) parses the whole body
through `PackageInstallBodySchema` / the closed `ManifestSchema` before
calling the primitive. Measured on this tree: the HotCRM body is refused
there with `Unrecognized key(s) on this package manifest: pages` in all
three cases: as published, with only the convertible shapes, and with
the conversion applied by hand. No package body that carries metadata
collections reaches the primitive over HTTP, so a conversion inside it
would never run for this door.
- **`duplicatePackage`** (`:27496`) copies `srcPkg.manifest`, the
registry's own record. The registry holds only what it accepted (already
converted at whichever door installed it), so the duplicate sees the
converted body. It behaves the same either way.
- **The unmanaged install's base** (`:27870`): `plan.baseManifest` is
identity keys plus `id` / `name` / `namespace`
(`package-unmanaged-install.ts:473`), with no collections. The window
has nothing to act on.
- **AI Studio's blueprint tools** (triage's cloud reading, `6219354007`)
call the primitive in-process with a manifest they are authoring now. A
retired declaration there is the author's to fix, refused loudly at the
moment of writing.
- **What `sys_packages` stores:** unchanged. The primitive refuses
before it persists, so a refused body writes no row. A body that lands
in `sys_packages` by any road is data at rest from then on, and door 1
converts it at every boot.
- Pinned over the real registry: the same converting body through
`protocol.installPackage` is refused with `MisplacedDefaultPageError`
(`INVALID_METADATA`, 422), with no registry record, no `sys_packages`
row and no conversion warning.
## Door 3: the marketplace install's side readers
(`@objectstack/cloud-connection`)
- Install and rehydrate each convert the body **once** (a new
`convertPackageBodyForward(ctx, manifest)` that delegates to the
engine's entry point). That one object is what `manifest.register`,
`bindArtifactHandlers`, `applySideEffects` and the rehydrate's seed read
(`maybeHealSampleData`, which now takes the converted body) receive. The
ledger keeps the body as installed.
- `register` meets the entry point a second time. The already-converted
body comes back with no second warning, and the pin asserts exactly one
warning per conversion on both paths.
## Pins (real `SchemaRegistry` throughout; a stub cannot refuse)
- `service-package/src/hydration-convert-forward.test.ts`: **real
stack** (`ObjectKernel` + `ObjectQLPlugin` + `SqlDriver` better-sqlite3
on disk + `PackageServicePlugin`). One process publishes through the
`package` service; a second process on the same file hydrates, as a
restart does. I chose the real stack because the defect sits between the
store, the engine's converter and the registry's refusal, and a unit
double can show none of the three. Cases:
- the body hydrates converted with the named warning, and the row stays
as published;
- a refused record (named at `error`, with `code`/`status` asserted on
the attached refusal) followed by a record that still hydrates;
- CONTROL: a range-less body is refused exactly as before, with no
conversion;
- HotCRM 4.0.1 as published is refused by the D1 handshake and the next
record hydrates;
- door 2's strict refusal.
-
`cloud-connection/src/marketplace-install-local-convert-forward.test.ts`:
a real `ObjectKernel` + `ObjectQLPlugin` supplies the converter and the
`manifest` service. On rehydrate and install, all readers receive the
**same** converted object. CONTROL on both paths: a range-less body is
refused at `register` (install: `422 PLUGIN_REGISTER_FAILED`; rehydrate:
the refusal is logged for that entry with `code`/`status`), and no side
reader runs.
### Ablations (`scripts/ablation-replace.mjs`; each restore proven blob
== HEAD and `git diff HEAD` empty)
| # | mutation | result |
|---|---|---|
| A1 | door 1 hands `rec.manifest` unconverted | 2 red (`hydrates …
converted`, `keeps hydrating`) |
| A2 | door 1's per-record catch rethrows | 2 red (`keeps hydrating`,
CONTROL: the refusal falls to the old `debug` line) |
| A3 | rehydrate binder reads `entry.manifest` | 1 red (`rehydrate — …
same converted body`) |
| A4 | install `applySideEffects` reads the raw `manifest` | 1 red
(`install — … same converted body`) |
The tests import the touched files from `src/` (and
`@objectstack/objectql` / `core` / `driver-sql` / `metadata-protocol`
through anchored vitest aliases to `src/`), so no `dist/` leg applies.
## Verification, on head `fe7b944bb`
The later heads' readings are in the addenda and the dev's hand-backs:
`f1443d9bca` (the gate fix) and `cbec0eac83` (patch round 1, addendum
`6224822850`). At `cbec0eac83`, `dispatch-gates --commands` derives 102,
101 exit 0, and 1 is NOT MEASURED (`check:dual-build-cjs-loads`, exit
3). The name-fold gate's `--self-test` passes 30/30 with a
matcher-battery floor of 20. Full `pnpm lint` exits 0. The readings
below are the first report's.
- `@objectstack/service-package`: test 118/118 (8 files), typecheck exit
0. `@objectstack/cloud-connection`: test 559/559 (47 files), typecheck
exit 0. `@objectstack/metadata-protocol` (a comment-only change):
typecheck exit 0, plus `durable-package.test.ts` 10/10. Its full suite
is left to CI, a declared narrowing. `--listFiles` confirms that both
new test files are in their package's tsc program.
- `pnpm lint` (full, `eslint . --no-inline-config`): exit 0.
- `dispatch-gates --commands`: 98 derived, 97 run with exit 0. One is
NOT MEASURED: `pnpm check:dual-build-cjs-loads` exit 3 (`PREREQUISITE
NOT MET`, it needs every package's `dist/`), and CI runs it.
`check-plugin-teardown-shape.mjs --self-test` first exited 3 on the
shallow clone; after fetching its pinned fixture commit it exited 0.
`--ran` reconciliation: 98 accounted, 97 run, 1 NOT-MEASURED derived
from the recorded exit 3.
## Wiring the pins needed
- `service-package` devDependencies `@objectstack/objectql`,
`@objectstack/driver-sql`, `@objectstack/metadata-protocol` (no cycle;
`pnpm-lock.yaml` +9). Its `vitest.config.ts` gains anchored aliases to
those packages' `src/` (`check:test-source-alias` green, no ledger
widening).
- Both new tests read the one HotCRM fixture under
`packages/objectql/src/__fixtures__/`, so it is declared in
`scripts/cross-package-test-inputs.mjs` and `turbo.json`
(`check:cross-package-test-inputs`, `check:ci-filter-parity` green;
`core` already covers the path).
- **A surface supplement outside the claim:
`scripts/check-position-name-fold-loaders.mjs`.** Its glob-shape rule 3
read the new
`packages/objectql/src/__fixtures__/hotcrm-4.0.1-published-pages.artifact.json`
line in `scripts/cross-package-test-inputs.mjs` as a reference to the
metadata permissions artifact, a false positive. Rule 3 now judges whole
path tokens. A token is set aside only when all of these hold:
- it ends at a quote, backtick, whitespace, `,`, `]`, `)` or end of
line;
- it has no `..` segment;
- its `path.posix.normalize`d form is outside
`packages/metadata/src/__fixtures__/`;
- it is one literal file path, with no glob character and no `${`.
Everything else on the line is still judged. The self-test passes 30/30,
with the matcher-battery floor at 20. Ablating the normalisation reds
one case, and so does ablating the token boundary. The first carve-out
(`f1443d9bca`) leaked `./` and partial-glob spellings (contract record
`6224173098`, R1), and patch round 1 (`cbec0eac83`) closed that (record
`6225074650`).
## Acceptance notes
- **Other raw-ledger readers in `marketplace-install-local-plugin.ts`,
outside the claimed region.** These belong to the same family as door 3.
Conversions DO rewrite translations, jobs and hooks today
(`packages/spec/src/conversions/registry.ts`), though none rewrites seed
`data`.
- The sample-data probes (`seedDatasetsOf(entry.manifest)`) and the
install-time `unrunnableCode(ctx, manifest, …)` read nothing a
conversion touches, so neither gives a wrong answer now.
- The reseed door (`applySideEffects(ctx, entry.manifest, … door:
'reseed')`) reloads RAW translations, including a per-app `settings`
group that `translation-per-app-settings-removed` strips. Carrier:
#24040.
- **`duplicatePackage`'s `catch {}`** around its `installPackage` call
swallows a refusal (pre-existing, unchanged here). Carrier: #24041.
- **`checkProtocolCompat` reads `^17.7.0 || ^18.0.0` as `incompatible`**
on this runtime (measured with a direct call), although the range admits
major 18 and the handshake's own comment says an unparsed range never
causes a false rejection. Reach through a public door is not measured.
Carrier: #24029.
- The runtime's no-protocol fallback in `POST /api/v1/packages`
(`registry.installPackage` after `assertProtocolCompat`) is the same
authoring door as door 2 and stays strict too.
_Seat edit · `domain:services` seat 2 ·
`session_018VqJPGA1NDNSCZsnecsChP` · 2026-10-16T21:32Z: the gate edit is
declared under "Wiring the pins needed" (N2), the raw-ledger-reader note
is corrected and its carriers named (N1, contract record `6224173098`),
and a pointer to the later heads' readings is added under Verification._
---
_Generated by [Claude
Code](https://claude.ai/code/session_013JuCowY7RrGXYbGEv3cWsQ)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent b6805ac commit c7168ac
12 files changed
Lines changed: 800 additions & 19 deletions
File tree
- .changeset
- packages
- cloud-connection/src
- metadata-protocol/src
- services/service-package
- src
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
0 commit comments