Repository navigation
Commit 07e933b
fix(service-datasource)!: Import as Object saves through the metadata door's save (#21837)
Fixes #21788
Clause-②: no (narrowing)
## What this changes
`ExternalDatasourceServicePlugin` wired the import's `persistObject` to
`metadata.register('object', name, definition)`. That held the generated
object in the metadata service's memory and did nothing else: no
`sys_metadata` row, no storage sync, and the SQL driver was never told
the object's `external.remoteName`. `persistObject` now calls
`saveMetaItem` on the `protocol` service, the save `PUT
/api/v1/meta/object/:name` makes, with the request that door sends for
an `object` (`{ type: 'object', name, item }`: no organization because
`object` is not org-overridable, and no package, mode or force because
the import route takes none). That save persists the row, writes it
through to the engine registry and runs `syncObjectSchema`, which maps a
federated object onto its remote table. There is no second registration
path beside it.
The save door is looked up when an import runs (a getter on the service
config), not at `init()`. The `protocol` service registers in another
plugin's `init()`, and a verdict drawn at this plugin's `init()` would
be kept for the life of the process. With no save door registered, the
service's existing "requires a writable metadata store" refusal still
fires before any remote introspection.
The import route's request and response shapes are unchanged. No
`packages/spec` edit, and no new export.
## Measured on the real composition (`pnpm dev`, showcase, at this
branch)
| step | before (base `3237b4a2`) | after (`2beb1441`) |
|:--|:--|:--|
| `POST …/tables/customers/import {"name":"ext_cust"}` | 201 | 201 |
| `GET /data/ext_cust` | **500 DATABASE_ERROR**, log `no such table:
ext_cust` | 200, the 3 remote rows |
| `POST …/tables/orders/import {"name":"orders"}`, then `GET
/data/orders` | 201, 200 | 201, 200 |
| `sys_metadata` rows for `ext_cust` / `orders` | none | both, env-wide,
`active` |
| restart on the same DB, `GET /data/ext_cust` and `/data/orders` |
**404 OBJECT_NOT_FOUND** both | 200 both, with rows |
| control: `PUT /meta/object/ext_cust_meta` (same binding) | 200, rows,
survives restart | unchanged |
## Tests
- New
`packages/services/service-datasource/src/__tests__/external-import-saves-through-metadata-door.test.ts`
(4 cases, relative import, measures `src/`): the import reaches
`saveMetaItem` with the door's request and never calls
`metadata.register`; a `protocol` registered after `init()` still
receives the save; a refused save refuses the import with the door's own
error (`code` and `status` asserted); with no save door, the import is
refused before introspection.
- `@objectstack/service-datasource`: 36 files, 713 tests pass; `tsc
--noEmit` passes, and its program includes the new test file
(`--listFiles`).
- Gates at `9f5279c8` (this branch after merging `origin/main`
`088428fb`): all 63 commands `dispatch-gates --commands` derives exit 0,
and `--ran` reconciles 63 of 63 with 0 NOT-MEASURED. Narrowed eslint
(`--no-inline-config`) on the two touched `.ts` files: 2 files, 0
findings. The config enables no type-aware linting (no
`parserOptions.project`), so this diff cannot move the verdict on any
untouched file.
- Ablation, committed fix first, through `scripts/ablation-replace.mjs`
with `persistObject` put back to register-only: the unit file went 3
failed and 1 passed (the no-save-door case is untouched by that
mutation, as expected). The booted-stack pin described below went 3
failed and 1 passed, with `404 OBJECT_NOT_FOUND` on both reads and after
the restart. The restore leg proved the blob equal to HEAD, rebuilt
`dist/`, and `ablation-dist-preflight --absent` was clean on the whole
tree. Both files were 4 of 4 green again.
## Not in this PR, and why
- **The card's dogfood pin is written and measured, but not committed.**
It imports under a different name, reads the rows, restarts on the same
DB file, and reads again, with a same-name import as the control. The
verify harness does not mount `ExternalDatasourceServicePlugin`, so the
pin has to import it. Committing that needs
`@objectstack/service-datasource` declared in
`packages/qa/dogfood/package.json`, plus a source alias in
`packages/qa/dogfood/vitest.config.ts`, because
`check:test-source-alias` refuses a new unaliased dogfood import and its
ledger only shrinks. Both are existing `packages/qa` files outside this
dispatch's file surface, so the pin waits for the seat. It was run
locally through an untracked `node_modules` link: green on the fix, and
red under the ablation above.
- **A durable federated object can abort the next boot.** This is
measured, and it already happens through `PUT /meta/object`, whose code
this PR does not touch. After a restart, Gate 2
(`ExternalValidationPlugin`) reads a stored federated object with the
platform-injected anchors (`organization_id`, `created_by`,
`updated_by`, `owner_id`, `owning_business_unit_id`) and reports each
one as a `missing_column` on the remote table. On a datasource with the
default `external.validation.onMismatch: 'fail'`, the boot then aborts
with "Object 'NAME' does not match its remote table". This PR makes
imports durable, so an import on such a datasource reaches that same
abort at the next restart, where it used to vanish. A measured remedy,
kept out of this PR's diff: have `validateObject` skip
`unprovisionedInjectedColumns(obj)` (`@objectstack/spec/data`). With
that, the same database booted and both datasources validated `ok:
true`. The landing order goes back to the seat.
- **A re-import that drops a column is refused with a remedy this route
cannot take.** The door's destructive-change check now applies: `409
DESTRUCTIVE_CHANGE`, answered as `400 EXTERNAL_IMPORT_ERROR`, says
"re-submit with ?force=true to proceed", and the import route reads no
`force`. The text comes from `destructiveChangeRemedy` in
`@objectstack/metadata-protocol`, which has no face for this door.
## Acceptance notes
- The history row of an imported object records `recorded_by: null`. The
door records the caller's user id. The import route does not pass its
caller to `IExternalDatasourceService.importObject`, and that contract
has no actor parameter.
- Until a restart, `POST …/external/validate` does not list an object
saved through the door, and that now includes an import. The federation
service reads objects from the metadata service, which picks up
`sys_metadata` objects only at boot. Before this change, an import sat
in that service's memory, so it was presumably listed until the restart
and then disappeared. That half was not measured on the base.
- Under the verify harness, `ExternalDatasourceServicePlugin` mounted as
an extra plugin reads the `metadata` service at `init()` and finds none.
So `persistCatalog` (`external_catalog`) is never wired there. That
decision is recorded at `init()`, the same shape this PR removes from
`persistObject`. `persistCatalog` is untouched here.
## Patch round 1 (head 8d640fa)
_Appended by the seat (`domain:services#1`,
`session_011K3zqE8Pv1Evw5hc8tZCnN`) from the dev's patch-round report
`5991725015`, after seat verdict `5990234116`. Line 2 was corrected in
the same act (the claim amendment is in that verdict)._
`#21841` (the import route's `?force` prescription) and `#21842` (the
validate door) are not addressed here.
**Q1: the card's dogfood pin is committed.**
`packages/qa/dogfood/test/external-import-saves-like-meta.dogfood.test.ts`
has three cases:
- an import under a name that differs from its remote table serves the
remote rows;
- the same-name control serves its rows;
- after a cold boot on the same database file, both still serve their
rows.
To make that possible:
- `@objectstack/service-datasource` is declared in
`packages/qa/dogfood/package.json`;
- its lockfile importer entry was regenerated by `pnpm install`. The
same run also wrote `deprecated:` registry metadata onto twelve
`@yuku-analyzer/binding-*` entries; none of it was hand-edited;
- `packages/qa/dogfood/vitest.config.ts` aliases the package to source,
in the shape `check:test-source-alias` reads.
No other `packages/qa` file is edited. This is declared on #6024
(`5990241427`).
**Q2: validation skips the platform's unprovisioned injected anchors.**
- `validateObjectUsing` (`external-datasource-service.ts`) skips the
columns `unprovisionedInjectedColumns(obj)` names. It reuses the spec's
own provenance predicate and builds no second list.
- This also closes the boot abort for federated objects saved through
`PUT /api/v1/meta/object/:name`, which `main` has today.
- A field an author declares under an anchor's name is still compared.
- **Negative control:** a declared column the remote lacks is still
`missing_column` at `error`. On `pnpm dev` a default-policy datasource
still aborts the boot on that real drift (measured, not a committed
booted pin).
**Q3: declaration.**
- `Clause-②: no (narrowing)`, BREAKING: some re-imports that answered
`201` now answer `400`.
- The changeset is `minor`, with the `!` title, the banner and one
measured handling line.
- ADR-0087: `not-required (no-migration-prescription)`, accepted by the
gate. No authorable key, export, config field or stored shape moves, so
there is nothing for `migrate meta` to rewrite.
**Measured at `8d640fa2`:**
- `service-datasource`: 37 files, 717 tests pass; `tsc` exit 0.
- The new validation pins pass, 4 of 4, including the negative control.
The dogfood pin passes, 3 of 3. `@objectstack/dogfood` typecheck exit 0.
- **Ablation 1** (`persistObject` back to register-only): the dogfood
pin went 3 of 3 red (`404 OBJECT_NOT_FOUND`). The restore was proven.
- **Ablation 2** (the anchor skip removed): 3 of 4 validation pins red.
The restore was proven.
- `pnpm dev`, a default-policy datasource with one imported and one
door-saved object: the restart boots, and validate answers `ok: true`.
- `dispatch-gates`: 78 derived, all exit 0. `--ran`: 78 of 78, 0
NOT-MEASURED.
**Acceptance notes (added by this round):**
- `persistCatalog` (the `external_catalog` snapshot) still goes through
`metadata.register` only, decided at `init()`. Its durability on `pnpm
dev` is NOT MEASURED.
- An imported object's history row records no actor (`recorded_by`
null), measured. The import route does not pass its caller to
`IExternalDatasourceService.importObject`, and that contract has no
actor parameter.
- An import named after a packaged object whose fields do not conflict
would save an env-wide overlay of it, exactly as `PUT /meta/object` with
the same body does. It was not measured on a compatible body; the same
door semantics apply.
---
_Generated by [Claude
Code](https://claude.ai/code/session_011K3zqE8Pv1Evw5hc8tZCnN)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 0fe0a59 commit 07e933b
9 files changed
Lines changed: 512 additions & 6 deletions
File tree
- .changeset
- packages
- qa/dogfood
- test
- services/service-datasource/src
- __tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
264 | 264 | | |
265 | 265 | | |
266 | 266 | | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
267 | 276 | | |
268 | 277 | | |
269 | 278 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
0 commit comments