Repository navigation
Commit bc7747c
fix(service-datasource)!: on objectstack start, read the metadata service when it is used, so validate and the boot gate compare every federated object (#21887)
Fixes #21876
Clause-②: no (narrowing)
## What this changes
`ExternalDatasourceServicePlugin.init()` read the `metadata` service
once and kept the answer. `objectstack start` composes no metadata
plugin. Its `metadata` service is the kernel's in-memory fallback, which
the kernel pre-injects after every plugin's `init()`, just before the
start phase. The `start` log shows the order: `Service
'external-datasource' registered`, then `Service 'metadata' registered`,
then `Phase 2: Start plugins`. So on `start`, every reader of the kept
value saw no service for the life of the process.
The plugin now looks up the `metadata` service when each reader runs. It
uses a resolver function called at each use, the same pattern
`metadataSaveDoor` already follows in this `init()` (AGENTS.md, "Startup
registry reads", cure 1). That covers `getDatasource`, `getObject`,
`listObjects`, `getNamespace` and the catalog write. The catalog write
was a conditional spread, so `init()` decided whether the
`persistCatalog` slot existed at all (H2). It is now a getter, as
`persistObject` already is. With no metadata service at all, each reader
returns the same fallback it always did.
Not changed: what validation judges, what each `onMismatch` value does,
and the boot gate's code (`packages/runtime`). The import's save call
(`persistObject`, landed with #21874) is not changed either. Objects are
still read from the metadata service's copy. #21842's PR #21875 moves
them to the engine registry.
**The object reads, called out.** `getObject` and `listObjects` read the
same kept value, so replacing it with a resolver also changes how those
two readers are spelled: each calls the resolver instead of using the
constant. They read the same members with the same fallback as before.
Only the moment the service is looked up moves. Without this, the card's
"Done when" cannot hold: on `start`, validate would still list no
objects. These are the lines PR #21875 replaces. When #21875 merges
`main`, its `getObject` and `listObjects` should replace these, and it
should keep this PR's `getDatasource` line.
Files: `packages/services/service-datasource/src/plugin.ts`, one new
unit file beside it, one new dogfood file, and a `minor` changeset with
the `!` banner. No `packages/spec` edit, no new export, and no edit to
an existing `packages/qa` file.
## Measured on the real composition (showcase, `objectstack start` and
`objectstack dev`)
Remote fixture for the probe: `customers.lifetime_value` is TEXT on the
remote, while `showcase_ext_customer` declares a currency field. That is
a `type_mismatch` at severity `error`. A missing column would not
survive the boot, because the showcase's own `onEnable` adds missing
columns. A column type it leaves alone. The base build of
`service-datasource` is `plugin.ts` at `2e780467`. The branch build is
`plugin.ts` at `10dd86129b`, which is this head's `plugin.ts` minus
#21874's one-line `writeFace` change from the merge.
| `objectstack start` | base | branch |
| --- | --- | --- |
| boot gate, drift, showcase's `onMismatch: 'warn'` | `all federated
objects match their remote schema {"objects":0}` | `external schema
drift` (warn) on `showcase_ext_customer`: `type_mismatch`
`lifetime_value`, expected `currency`, actual `text` |
| boot gate, no drift | (`objects: 0` whatever the remote holds) | `all
federated objects match their remote schema {"objects":2}` |
| boot gate, drift, datasource set to `onMismatch: 'fail'` for the probe
(restored after; blob equals HEAD) | boots, `objects: 0` | **refuses to
boot**: "Object 'showcase_ext_customer' does not match its remote table
on datasource 'showcase_external': type_mismatch:
customers.lifetime_value (expected currency, actual text)" |
| `POST /datasources/showcase_external/external/validate` | `{ ok: true,
results: [] }` | 2 rows: `showcase_ext_customer` `ok: false` with the
`type_mismatch`, `showcase_ext_order` `ok: true` |
| `POST …/external/refresh-catalog`, then `GET
/meta/external_catalog/showcase_external_catalog` | snapshot answered;
the read answers `RESOURCE_NOT_FOUND` (never stored) | snapshot
answered; the read returns the stored record |
| `GET …/external/tables` | 2 tables | same (the showcase sets no
`allowedSchemas`) |
| `POST …/tables/customers/draft` | `customers`, with the
`TODO(namespace)` note | same (see the namespace note below) |
| import `orders` as `probe_ext_orders_21876`, then as
`showcase_probe_orders_21876` | 201, 201 | 201, 201 (same note) |
| validate after both imports | `results: []` | the 2 code-defined rows
only; the imported objects are not listed until restart (H5, #21842's) |
**`objectstack dev`, the control (H4):** base and branch gave
byte-identical answers from validate, validate-after-import, tables, the
catalog read and both imports, after stripping `snapshotAt`. Both boot
gates log the same drift warning. `dev` answers exactly what the branch
now answers on `start`.
**The namespace note.** The showcase's code-defined datasource carries
no `_packageId` on either composition, `dev` included, so its namespace
never resolves there. The namespace half (the draft's prefix, the
import's name check) is pinned in the unit file, with a datasource that
carries package provenance. On `start` it was blind on every deployment,
and it now answers as on `dev`.
**Introspection (`data` service).** It is still read at `init()`. On
`start` it is already registered by then: base `start` answered tables,
draft and refresh. See Acceptance notes.
## Tests
- New
`packages/services/service-datasource/src/__tests__/external-metadata-read-at-use.test.ts`:
10 cases, relative import, so it measures `src/`. The `metadata` service
is registered AFTER `init()`, as on `start`. Under that ordering:
validate compares each federated object and reports the drifted column,
and the boot gate's sweep (`validateAll`) lists every federated object.
The draft takes the namespace of the datasource's package. An import's
explicit name that breaks the prefix is refused, asserting `code:
'EXTERNAL_IMPORT_ERROR'` and `status: 400`, and the save door is never
called. The refreshed catalog is persisted. `allowedSchemas` is
honoured. The service is looked up again at each use, never remembered.
Control: a service registered BEFORE `init()` (the `dev` ordering) gives
the same answers. With no metadata service, every reader keeps its
fallback.
- New
`packages/qa/dogfood/test/external-validate-start-ordering.dogfood.test.ts`:
3 cases, booted showcase. The remote's `customers.email` is renamed away
after provisioning; the harness does not run `onEnable` at boot, so the
drift survives. A precondition asserts that the harness's `metadata`
service is the kernel's in-memory fallback, the `start` shape. Validate
compares both federated objects and reports `missing_column email`.
`validateAll()` lists both.
- **Red at base** (`ce28b73809`: the two test files on the base
`plugin.ts`): unit 7 failed, 3 passed, for example `expected [] to
deeply equal [ 'wh_customer', 'wh_order' ]` and `expected "vi.fn()" to
be called 1 times, but got 0 times`. Dogfood 2 failed, 1 passed (the
precondition), `expected [] to deeply equal [ 'showcase_ext_customer', …
]`.
- **Ablation** at `10dd86129b` (fix committed first), through
`scripts/ablation-replace.mjs` in WRAP mode. The resolver line was
replaced by a capture taken at `init()` and a resolver returning that
capture (anchor hits 1, blob `3f9f1968` to `2dbc3ade`, marker
`ABLATION-21876` count 1 on disk, anchor count 0). Unit: 7 of 10 failed,
the same 7 as at base. Dogfood: 2 of 3 failed. Restore: blob after
restore `3f9f1968` equals HEAD, `git diff HEAD` is empty and `git
status` is clean. No `dist/` is on either path: the unit file imports
`src/` relatively, and the dogfood config aliases
`@objectstack/service-datasource` to `src/`.
- **At this head `c6f237478c`** (`origin/main` `e864db56df` merged,
which carries #21874): closure build (`@objectstack/dogfood`
dependencies plus `service-datasource`) 63 of 63 tasks.
`@objectstack/service-datasource`: 38 files, 728 tests passed. `tsc
--noEmit` passes, and `--listFiles` includes the new unit file (count
1). Dogfood: the new file plus `external-import-saves-like-meta` and
`external-import-destructive-remedy` (the other two files that mount
this plugin), 3 files, 10 tests passed. Dogfood `tsc --noEmit` passes,
with the new file in `--listFiles` (count 1).
**The gap, named.** The dogfood file boots the verify harness, not the
`objectstack start` CLI. The harness composes no metadata plugin, so its
`metadata` service is the same kernel fallback, injected at the same
moment as on `start`. The precondition case holds that. The harness does
not mount the boot gate. Mounting it from this file would need
`@objectstack/runtime` as a value import, which the dogfood package does
not alias. That would mean a new pair in `check:test-source-alias`'s
shrink-only ledger or an edit to the dogfood vitest config, and both are
outside this dispatch. So the file pins the gate's input
(`validateAll()`), and the gate's own verdict on `start` is the probe
table above.
## Gates
At `c6f237478c`: `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` derives 67 commands: the dispatch's 59 plus
8 that the changeset brings. All 67 were run and all exit 0. `--ran`
reconciles 67 run, 0 NOT-MEASURED, 0 UNRUN. `check:dual-build-cjs-loads`
first answered PREREQUISITE NOT MET (exit 3), because 8 unrelated
packages had no `dist/`. `check:type-check-debt`'s re-measure later in
the same battery built them, and the gate was re-run on the same head to
exit 0. Also run: `pnpm check:startup-registry-verdict` exit 0 ("43
startup/open-registry seam(s) … none recording a verdict the boot can
contradict").
Narrowed eslint (`--no-inline-config`, `--format json`) on the 3 touched
`.ts` files: 3 files, 0 errors, 0 warnings. The changeset `.md` is
outside eslint's population: eslint answers "File ignored because no
matching configuration was supplied". `eslint.config.mjs` enables no
type-aware linting (no `parserOptions.project`), so this diff cannot
move the verdict on any untouched file.
## Docs
No `content/docs` sentence is made false.
`data-modeling/external-datasources.mdx` promises a federated datasource
is "validated at boot" and that a mismatch "fails boot". That was false
on `start` and is now true.
## Acceptance notes
- **`engine` (the `data` service) is still read at `init()`.** It is not
part of this card. On both `start` and `dev`, `ObjectQLPlugin` registers
it in its own `init()`, which runs before this plugin. Base `start`
introspected (tables, draft, refresh answered), so nothing was measured
wrong. Carrier: none.
- **The showcase's code-defined datasources carry no package
provenance**, so the federation draft and import never resolve a
namespace for them, on `dev` as on `start`. It is reported to the seat
with its evidence, not filed from here.
- **A federated object saved at runtime** is still listed by the
validate door only after the next restart. That is #21842's, fixed by PR
#21875 after this lands.
---
_Generated by [Claude
Code](https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent e085a8c commit bc7747c
4 files changed
Lines changed: 543 additions & 17 deletions
File tree
- .changeset
- packages
- qa/dogfood/test
- services/service-datasource/src
- __tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
0 commit comments