Repository navigation
Commit ef7067f
<!-- claude-projects-attribution: {"github_login":"os-project-manager"}
-->
_Requested by **Leehom** · [project
thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1XMNo82yTJxw28uqj2PYAVvH)_
Fixes #26684
Clause-②: no (narrowing)
Metadata-protocol: none
Skills-docs: none — no `skills/**` text describes the import dry run or
the insert preview (`git grep -i -e 'dry run' -e dryRun skills/` hits
only the i18n CLI `--dry-run` flag and `os migrate plan`)
## 维护者速读
- **改了什么**:导入的试运行(dryRun)不再逐行单独问引擎,而是把新建行按正式导入写入时的同一批次(每批 200
行)一次性交给引擎预览;引擎的批量预览也在规则之后跑一遍写入时的引用检查,通过才计入后面行看到的父汇总。
- **为什么改**:#25579
之后正式导入会把同批前面已通过的行计入父汇总("一个供应商只能有一个主联系人"),试运行却仍逐行判断,两行主联系人都说"有效",正式导入却拒掉第二行,操作员被告知文件没问题。
- **风险与回滚**:试运行现在会多报出正式导入本来就会拒的行(收窄,BREAKING 标注);单行的插入预览会报出悬空引用。回滚即
revert 本 PR。已知未覆盖:试运行不写库,跨 200 行批次边界的两行冲突仍都显示有效(已写进文档,待你决定是否要补)。
- **先看**:`packages/core/src/utils/import-runner.ts`(`previewCreateRun` /
`flushPendingPreviews`);`packages/objectql/src/engine.ts` 的 `validate()`
新增引用检查那一段;`packages/rest/src/import-dry-run-batch-parent-summary.test.ts`。
- **两分钟验证**:`pnpm --filter @objectstack/rest exec vitest run
src/import-dry-run-batch-parent-summary.test.ts`,预期 9
个用例全绿,其中"两行主联系人"一例里试运行第二行失败、信息与正式导入相同。
- **需要你做的**:无,席位落地;跨批次边界那一条另作决策。
## What changes
**`@objectstack/core` — `packages/core/src/utils/import-runner.ts`.** A
dry run's CREATE rows are no longer asked about one at a time. They are
buffered as the commit buffers its creates (`pendingPreviews`, beside
`pendingCreates`) and flushed at the commit's checkpoints (every
`progressEvery` rows, and the end) by `flushPendingPreviews`, which
splits them into the commit's runs (`Math.min(progressEvery,
MAX_CREATE_BATCH_SIZE)`, the `bulkWrite` batch size) and asks the engine
about each run in ONE `validateData` call with `data` as an array, in
file order (`previewCreateRun`). The engine's multi-row `validate()`
(#26680) does the accumulation: ⛔ the runner neither reads nor judges a
rule. Each row still gets its own report, built by the same code as
before (`reportPreview`, lifted out of the row loop unchanged): same
`row`, `ok`, `action`, `field`, `code`, `error`, `warnings`,
`droppedFields`, at the row's own position. The run's call carries the
union of its rows' kept option values, as the commit's batch call does
(`writeCtxFor`). UPDATE rows keep their per-row preview
(`previewVerdict`, one row object, `mode: 'update'`), unchanged.
A run whose call throws, or answers other than one verdict per row, is
degraded as the commit degrades a failed batch: each row is asked alone
(so a row the engine refuses outright, such as an undeclared column,
carries its own refusal, as `insertManyData` culls it per row), then the
rows that survived alone are asked together again so they still see each
other.
**`@objectstack/objectql` — `ObjectQL.validate()` (the seat's addendum
6442401477).** The insert preview's admit step now matches `insert()`'s:
after the rules, the write's reference check (`assertReferencesResolve`,
the same method, with the caller's pre-default row `rawRows[i]` as
`supplied`, the write's `suppliedPerRow`). A row it refuses is answered
`valid: false` with the write's own `reference_not_found` finding, and
is not counted toward a later row's parent summary. ⛔ It runs only
behind the same write-gate probe (`mayWrite`) as the preview's related
read: the probe on a `lookup` target is elevated, so a caller the probe
refuses learns nothing about which ids exist. The rows are now judged in
a sequential loop (`judgeRow`) instead of `rows.map`, because the admit
step is awaited before the next row reads what the earlier rows added.
Update mode is untouched.
**Docs.** `content/docs/data-modeling/fields.mdx` (the 批量等于逐条 paragraph
#26680 added): the dry run previews create rows in the import's batches,
and the one case it cannot see (rows in different batches, below).
## Why
#25579 made a bulk create count the earlier admitted rows of the same
batch when a child rule reads a parent `summary`. The import commit
does; the dry run asked the engine about each row alone, so on #25579's
reproduction (two primary contacts for one new supplier) it answered
both rows valid while the commit created one and refused the other. The
engine's multi-row preview already accumulates (#26680); the dry run
just never handed it more than one row.
## Tests
- **`packages/rest/src/import-dry-run-batch-parent-summary.test.ts`**
(new, 9 cases) — through the real `POST /api/v1/data/:object/import`, a
real ObjectQL engine and a sqlite `:memory:` driver, #25579's schema.
Each agreement case runs the same file twice on two fresh stacks,
`dryRun: true` and committed, and pins the per-row verdicts equal
(`row`, `ok`, `action`, `error`) plus `created` / `updated` / `errors`:
- two primary contacts for a new supplier: the dry run refuses row 2
with the commit's message, `A supplier has at most one primary
contact.`;
- the dry run writes nothing (no contact, summary still `0`);
- two non-primary rows: both valid in the dry run, as both land;
- a row the rules refuse is not counted (placeholder name, then a
primary): both halves admit the second;
- a file larger than one batch (203 rows): two primary rows inside the
second batch, the dry run refuses row 202 as the commit does;
- an update-mode dry run keeps its per-row preview (an update against a
stored primary is refused, a rename is not) and agrees with its commit;
- the seat's addendum: a multi-row insert preview refuses the
dangling-reference row with `region` / `reference_not_found` and admits
the next primary row, as `insertMany` does; a single-row insert preview
refuses a dangling reference as `insert()` does; a caller the write-gate
probe refuses gets no `reference_not_found` (no existence oracle).
- **`packages/core/src/utils/import-runner-dry-run-batch.test.ts`**
(new, 6 cases) — against a protocol double that answers one verdict per
submitted row: the create rows of a small file are one insert-mode call
in file order with each row's own report; 450 rows are previewed as 200
/ 200 / 50; a caller `progressEvery` of 3 sets the checkpoints and run
length (`[r1..r3] [r4..r6] [r7]`, progress `3, 6, 7`); an upsert's
matched row is asked alone in `update` mode with its id while the
creates around it keep one batch; a batch the engine refuses outright is
asked row by row, then the survivors together again; no `validateData`
falls back to the coercion verdict.
- **Updated, stale doubles** — four core tests modelled `validateData`
as answering one verdict for any call, so a batch of rows read as one
row: `import-runner-dropped-fields.test.ts` (two cases) and
`import-runner-kept-options.test.ts` now answer one verdict per
submitted row (the `ValidateDataResponse` contract, "one entry per
submitted row"); the kept-options dry-run case now pins the ONE batch
call and its union context, the context the commit's batch write carries
(plus a case: a batch that kept nothing carries the import's context
unchanged); `import-runner-update-preview-address.test.ts`'s create
control now reads `data` as a one-row array.
- **Updated, `packages/objectql/src/validate-preview-parent.test.ts`
(c)** — "insert and update previews of a plain line issue no read of the
header object" split in two: the update preview still issues none; the
insert preview now issues exactly the reads the insert itself makes (its
reference check asks the header's read door whether `h_sent` exists),
and the #22474 header binding adds none.
<details><summary>Red first on main (production sources restored to
1685a06, core and objectql rebuilt)</summary>
Restore proved by `git hash-object` equal to the base blobs
(`import-runner.ts` 4983d0e, `engine.ts` 1f7cd15714), then `turbo run
build --filter=@objectstack/objectql --filter=@objectstack/core` exit 0.
- rest `import-dry-run-batch-parent-summary.test.ts`: `Tests 4 failed |
5 passed (9)` — red: the two-primary case, the second-batch case, both
dangling-reference cases. The two-primary case's diff: row 2 expected
`action: "failed"`, `error: "A supplier has at most one primary
contact."`, received `ok: true`, `action: "created"`.
- core `import-runner-dry-run-batch.test.ts`: `Tests 5 failed | 1 passed
(6)` (only the no-`validateData` fallback is green on `main`).
- Restore leg: `git checkout HEAD --`, `git diff --quiet HEAD` clean,
both packages rebuilt; the same files then `Tests 9 passed (9)` and
`Tests 6 passed (6)`.
</details>
<details><summary>Ablations (both through scripts/ablation-replace.mjs,
package rebuilt, ablation-dist-preflight proving the marker in dist/,
restore proved by blob == HEAD)</summary>
1. **Send the create rows one by one again** — `import-runner.ts` `const
runLength = Math.min(progressEvery, MAX_CREATE_BATCH_SIZE);` → `const
runLength = 1;`. Predicted: the two-primary and second-batch agreement
cases red, everything else in the rest file green; the core batching
cases red. Observed: rest `Tests 2 failed | 7 passed (9)` (exactly those
two), core `Tests 5 failed | 1 passed (6)`. A first run of this ablation
read a stale objectql `dist/` left by the red-first leg (the two
reference cases were also red); that reading is void and is not counted:
objectql was rebuilt at HEAD and the ablation re-run.
2. **Drop the preview's reference check** — `engine.ts` the `await
this.assertReferencesResolve(schemaForValidation, row, rawRows[i],
options?.context, messages);` line deleted. Predicted: the two
dangling-reference cases red. Observed: rest `Tests 2 failed | 7 passed
(9)`, exactly those two.
After both: tree == HEAD, core and objectql rebuilt, preflight
`--absent` / present confirms the HEAD build, rest `9 passed (9)`, core
`6 passed (6)`.
</details>
## Acceptance notes
- **Not covered: two rows that conflict across a batch boundary.**
Measured on this head with a throwaway probe (not committed): 199
non-primary rows, then primary rows at 200 and 201. The commit refuses
row 201 (batch 1's recompute writes the stored count before batch 2
reads it); the dry run answers both valid, because a dry run writes
nothing between batches and batch 2's preview reads the stored `0`.
Inside one batch (the card's reproduction, and every file of up to 200
create rows) the two agree. The docs paragraph and the changeset's "Not
covered" bullet say so. Closing it needs one of: one `validate()` call
over all of a file's create rows (up to 5,000 sync / 50,000 job rows in
one engine call), or an accumulation carried across preview calls (a new
`validateData` request key in `packages/spec`, outside this lane). Left
to the seat as an open question in the os-dev-report.
- **Update rows keep their per-row preview** (the ruling). Two update
rows that each move a parent summary still preview against the stored
value; their commit writes them one by one, so the second can be
refused. Unchanged by this PR.
- **The reference check in the preview is insert-only.** `update()` also
runs it on the write; the update-mode preview still does not. Out of the
addendum's scope (the admit step exists only on insert).
- **The insert preview now reads one more thing**: the reference check's
probe of each id the caller supplied, behind the write-gate probe. A
caller the probe refuses gets no such read and no `reference_not_found`.
- **Degraded run**: when a run's call throws, each row is asked alone
and the survivors together again. This mirrors the commit's
`insertManyData`, which culls a refused row and accumulates the rest; it
costs extra calls only on a failing run.
- `packages/spec` untouched (lane red line). The import door's `dryRun`
description in `export.zod.ts` ("The verdict is the engine's own
write-path validation") stays true and is not edited.
- Open PRs touching these files: #26337 and #26755 (`engine.ts`), #26598
(`fields.mdx`); `git merge-tree` of each against this head is clean.
## Verification
All on head `80b0a99e`, in this card thread's container.
- **Package suites** (`pnpm --filter PKG test`, the `local` project):
`@objectstack/core` 2803 passed (2803); `@objectstack/objectql` 8807
passed (8807); `@objectstack/rest` exit 0. **`repo` projects** (`vitest
run --project repo`): core 83 passed, objectql 5 passed, rest 280 passed
/ 18 skipped. **Type check** (`pnpm --filter PKG typecheck`): core,
objectql, rest exit 0.
- **Downstream readers of the dry run and of `validate()`**: plugin-auth
`admin-import-users*.test.ts` 62 passed; runtime
`external-validation-*.test.ts` 25 passed;
`@objectstack/service-datasource` test 975 passed; service-automation
`connector-pull` 22 passed.
- **Derived gates**: `node scripts/pm/dispatch-gates.mjs --commands
--repo objectstack-ai/objectstack` on this diff: 96 families, every one
run with its exit code captured before any pipe, all exit 0; `--ran`
reconciliation: `96 derived famil(ies) accounted for — 96 run, 0
NOT-MEASURED`. Three first answered exit 3 (prerequisite not met: the
teardown self-test's positive control commit not fetched, no full
`dist/` for `check:skill-examples` and `check:dual-build-cjs-loads`) and
were re-run green after `git fetch origin 621a487` and the full build
`check:type-check-debt` makes.
- **Scoped lint**: `pnpm exec eslint --no-inline-config` on the 8
touched source and test files: exit 0.
<details><summary>Dogfood battery (narrowing arm)</summary>
On `80b0a99e`, `pnpm --filter @objectstack/dogfood test` under the
verify lock, no shard, after `turbo run build
--filter='@objectstack/dogfood^...'`: `Test Files 302 passed | 3 skipped
(305)`, `Tests 2505 passed | 20 skipped (2525)`, `os-verify-lock:
VERDICT command-exit 0`, held 799s. No failures; the skips are declared
skips.
</details>
## Human Minimum
- [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic.
(No `packages/spec` change.)
- [x] 2. Every name I added uses the casing for its kind.
- [x] 3. No formatter ran; `pnpm lint` passes (scoped to the touched
files locally; the repo-wide run is CI's).
- [x] 4. One issue, one branch off an up-to-date `main`, one pull
request.
- [x] 5. A change that publishes carries a changeset, and both
declaration lines are filled in here and in the changeset.
- [x] 6. `packages/spec` touched: not touched.
- [x] 7. No hand edits to `content/docs/references/`,
`packages/*/CHANGELOG.md` or `content/docs/releases/`.
- [x] 8. Title, body and comments are in English (the 维护者速读 block is the
lane's maintainer summary).
- [x] The *Before Submitting* list in CONTRIBUTING.md is done.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01FEkQ5krDpBpMjEHKqjJ6u8)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent 665059b commit ef7067f
10 files changed
Lines changed: 647 additions & 46 deletions
File tree
- .changeset
- content/docs/data-modeling
- packages
- core/src/utils
- objectql/src
- rest/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
322 | 322 | | |
323 | 323 | | |
324 | 324 | | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
325 | 330 | | |
326 | 331 | | |
327 | 332 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
74 | 78 | | |
75 | 79 | | |
76 | 80 | | |
| |||
85 | 89 | | |
86 | 90 | | |
87 | 91 | | |
88 | | - | |
| 92 | + | |
89 | 93 | | |
90 | 94 | | |
91 | 95 | | |
| |||
116 | 120 | | |
117 | 121 | | |
118 | 122 | | |
119 | | - | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
120 | 126 | | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
| 127 | + | |
| 128 | + | |
125 | 129 | | |
126 | 130 | | |
127 | 131 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
77 | 77 | | |
78 | 78 | | |
79 | 79 | | |
80 | | - | |
81 | | - | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
82 | 83 | | |
83 | 84 | | |
84 | 85 | | |
| |||
135 | 136 | | |
136 | 137 | | |
137 | 138 | | |
138 | | - | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
139 | 142 | | |
140 | 143 | | |
141 | 144 | | |
142 | | - | |
143 | | - | |
144 | | - | |
145 | | - | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
146 | 155 | | |
147 | 156 | | |
148 | 157 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
81 | | - | |
| 81 | + | |
| 82 | + | |
82 | 83 | | |
83 | 84 | | |
0 commit comments