Skip to content

Commit 95555e7

Browse files
feat(spec): declare the per-row write-drop signal on the import and validate answers, and the served import warnings (#20930)
Closes #20921 Clause-②: yes (widening) - `ImportRowResultSchema` and each `ValidateDataResponseSchema.results[]` row gain an optional `droppedFields`: `z.array(DroppedFieldsEventSchema)`, the reference `BatchOperationResultSchema.droppedFields` already uses. No second reason enum: the set is the engine's, `computed` included. - `ImportRowResult.warnings` is declared as served: `z.array(ValidateDataIssueSchema)`, set on ok dry-run rows only. Producer `packages/rest/src/import-runner.ts:852`; it reaches the sync route and the async job results; zero readers in objectui at `741864f7b9`. Undeclared, `parse` stripped it. - `ImportJobResultsSchema.results` describe: a capped sample, failures first; an ok row's report reaches the async reader only inside it. The cap is unchanged. - Pins in `export.test.ts` and `validate-data.test.ts`. Ablation at `3e9b7ece06` (each key renamed out of its schema, restore proven by blob hash): 4 / 3 / 1 tests red; control 53/53 green. - Local at `3e9b7ece06`: spec test 583 files, 17177 passed; spec typecheck exit 0; `check:generated` 15 of 15 current; 104 of 108 derived gates exit 0, 4 NOT MEASURED (they need whole-workspace builds; CI runs them). - Changeset: `@objectstack/spec` minor. It names objectui's `STRIPPED_LINE` toast table as the exhaustive `reason` consumer. ## Acceptance notes - The key is declared ahead of its producer: #20922 (engine) and the REST item of #20701 set it. Until both land no server emits `droppedFields`, and both describes say an absent key does not prove nothing was dropped. --- _Generated by [Claude Code](https://claude.ai/code/session_01Sfe5YjBLwB9J3y8fvm2xq1)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent bee75ce commit 95555e7

8 files changed

Lines changed: 247 additions & 4 deletions

File tree

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
'@objectstack/spec': minor
3+
---
4+
5+
The import row report and the validate-only answer can now say which fields a write drops. `ImportRowResultSchema` and each `ValidateDataResponseSchema.results[]` row gain an optional `droppedFields`: an array of `DroppedFieldsEventSchema`, the engine's own strip event. So the reason vocabulary is the engine's (`readonly`, `readonly_when`, `primary_key`, `computed`), and there is no second enum. The row still succeeds: `ok`, `action` and `valid` are unchanged. The key is declared ahead of its producer. A server that does not produce the report omits the key, so an absent key alone does not prove nothing was dropped.
6+
7+
`ImportRowResultSchema` also declares `warnings`, which the REST import dry run already serves: the findings the validate verdict admits, in the `ValidateDataIssue` shape, on an ok dry-run row. Until now `ImportRowResultSchema.parse` stripped the key, and readers typed by the spec could not see it.
8+
9+
`ImportJobResultsSchema.results` now says what an async reader gets: a capped sample, failures first. An ok row's `droppedFields` or `warnings` reaches that reader only if the row falls inside the sample. The cap is unchanged.
10+
11+
A consumer that branches on `reason` must stay exhaustive over `DroppedFieldsEvent['reason']`. The known exhaustive consumer is objectui's write-warning toast table, `STRIPPED_LINE` in `packages/app-shell/src/providers/writeWarningToast.ts`, which covers all four reasons today. A reader that renders the import or preview report should word `reason` through that table rather than a second one. When the union widens again, the table keyed by it fails type-check on the missing reason, while a second table would fall behind without a sound.

‎content/docs/references/api/export.mdx‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -215,7 +215,7 @@ const result = CreateImportJobRequestSchema.parse(data);
215215
| **startedAt** | `string` | optional | Processing start timestamp (ISO 8601) |
216216
| **completedAt** | `string` | optional | Completion timestamp (ISO 8601) |
217217
| **createdAt** | `string` | ✅ | Job creation timestamp (ISO 8601) |
218-
| **results** | `{ row: integer; ok: boolean; action: Enum<'created' \| 'updated' \| 'skipped' \| 'failed'>; id?: string; … }[]` | ✅ | Capped sample of per-row outcomes (failures first) |
218+
| **results** | `{ row: integer; ok: boolean; action: Enum<'created' \| 'updated' \| 'skipped' \| 'failed'>; id?: string; … }[]` | ✅ | Capped sample of per-row outcomes, failures first. An ok row, and any `droppedFields` or `warnings` it carries, reaches this reader only if it falls inside the sample; `resultsTruncated` says whether the sample is partial, and the job counters count every row. |
219219
| **resultsTruncated** | `boolean` | ✅ | Whether `results` is a capped sample of a larger set |
220220

221221
### Nested Shape: `ImportJobResults.results[number]`
@@ -229,6 +229,8 @@ const result = CreateImportJobRequestSchema.parse(data);
229229
| **field** | `string` | optional | Field that caused a coercion/validation error |
230230
| **code** | `string` | optional | Error code (failed rows) |
231231
| **error** | `string` | optional | Human-readable error message (failed rows) |
232+
| **warnings** | `{ field: string; code: string; message: string }[]` | optional | Findings this deployment ADMITS rather than rejects (ADR-0104 value shapes under a warn-first posture), in the same `{ field, code, message }` shape the validate verdict carries. Set only on a dry-run row the verdict accepted: the row is `ok` because the write would store it and log the same complaint. A committed row never carries it, since the write path has no channel to report admitted findings on. |
233+
| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed'> }[]` | optional | Write-observability: caller-supplied fields the engine LEGALLY strips from THIS row, one event per reason, in the engine's own `droppedFields` shape and reason vocabulary. A dry-run row reports the strips the write would make; a committed row reports the strips it made. The row still succeeds: `ok` and `action` are unchanged. Present only when at least one field was dropped. A server that does not produce this report omits the key too, so an absent key alone does not prove nothing was dropped. In an async job's `results`, an ok row's drops reach the reader only if the row falls inside that capped, failures-first sample. |
232234

233235

234236
---
@@ -355,6 +357,8 @@ Type: `{ sourceField: string; targetField: string; targetLabel?: string; transfo
355357
| **field** | `string` | optional | Field that caused a coercion/validation error |
356358
| **code** | `string` | optional | Error code (failed rows) |
357359
| **error** | `string` | optional | Human-readable error message (failed rows) |
360+
| **warnings** | `{ field: string; code: string; message: string }[]` | optional | Findings this deployment ADMITS rather than rejects (ADR-0104 value shapes under a warn-first posture), in the same `{ field, code, message }` shape the validate verdict carries. Set only on a dry-run row the verdict accepted: the row is `ok` because the write would store it and log the same complaint. A committed row never carries it, since the write path has no channel to report admitted findings on. |
361+
| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed'> }[]` | optional | Write-observability: caller-supplied fields the engine LEGALLY strips from THIS row, one event per reason, in the engine's own `droppedFields` shape and reason vocabulary. A dry-run row reports the strips the write would make; a committed row reports the strips it made. The row still succeeds: `ok` and `action` are unchanged. Present only when at least one field was dropped. A server that does not produce this report omits the key too, so an absent key alone does not prove nothing was dropped. In an async job's `results`, an ok row's drops reach the reader only if the row falls inside that capped, failures-first sample. |
358362

359363

360364
---
@@ -372,6 +376,26 @@ Type: `{ sourceField: string; targetField: string; targetLabel?: string; transfo
372376
| **field** | `string` | optional | Field that caused a coercion/validation error |
373377
| **code** | `string` | optional | Error code (failed rows) |
374378
| **error** | `string` | optional | Human-readable error message (failed rows) |
379+
| **warnings** | `{ field: string; code: string; message: string }[]` | optional | Findings this deployment ADMITS rather than rejects (ADR-0104 value shapes under a warn-first posture), in the same `{ field, code, message }` shape the validate verdict carries. Set only on a dry-run row the verdict accepted: the row is `ok` because the write would store it and log the same complaint. A committed row never carries it, since the write path has no channel to report admitted findings on. |
380+
| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed'> }[]` | optional | Write-observability: caller-supplied fields the engine LEGALLY strips from THIS row, one event per reason, in the engine's own `droppedFields` shape and reason vocabulary. A dry-run row reports the strips the write would make; a committed row reports the strips it made. The row still succeeds: `ok` and `action` are unchanged. Present only when at least one field was dropped. A server that does not produce this report omits the key too, so an absent key alone does not prove nothing was dropped. In an async job's `results`, an ok row's drops reach the reader only if the row falls inside that capped, failures-first sample. |
381+
382+
### Nested Shape: `ImportRowResult.warnings[number]`
383+
384+
| Property | Type | Required | Description |
385+
| :--- | :--- | :--- | :--- |
386+
| **field** | `string` | ✅ | The field the finding is about (`_record` for an object-level rule). |
387+
| **code** | `string` | ✅ | Machine-readable finding code, e.g. `required`, `invalid_type`, `rule_violation`. |
388+
| **message** | `string` | ✅ | Human-readable message — a validation rule's author-written text where one exists. |
389+
390+
### Nested Shape: `ImportRowResult.droppedFields[number]`
391+
392+
A write-path strip event: caller-supplied fields legally dropped from the payload
393+
394+
| Property | Type | Required | Description |
395+
| :--- | :--- | :--- | :--- |
396+
| **object** | `string` | ✅ | Object the write targeted (resolved object name) |
397+
| **fields** | `string[]` | ✅ | Caller-supplied field names the engine removed from the write payload |
398+
| **reason** | `Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed'>` | ✅ | Why the fields were dropped: static readonly, a TRUE readonlyWhen predicate, the primary-key strip of a payload id the engine ruled is not an identifier, or a computed (formula) field no driver has a column for |
375399

376400

377401
---

‎content/docs/references/api/protocol.mdx‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2977,7 +2977,7 @@ A write-path strip event: caller-supplied fields legally dropped from the payloa
29772977
| **object** | `string` | ✅ | The object name. |
29782978
| **mode** | `Enum<'insert' \| 'update'>` | ✅ | The write mode the verdict was reached for. |
29792979
| **valid** | `boolean` | ✅ | True when EVERY row is valid — the whole-set answer. |
2980-
| **results** | `{ valid: boolean; errors: object[]; warnings: object[] }[]` | ✅ | Per-row verdicts, in submission order. |
2980+
| **results** | `{ valid: boolean; errors: object[]; warnings: object[]; droppedFields?: object[] }[]` | ✅ | Per-row verdicts, in submission order. |
29812981
| **posture** | `{ valueShapeStrict: boolean; mediaValueShapeStrict: boolean }` | ✅ | The ADR-0104 posture the verdict was reached under — reported because it is the difference between "this row is fine" and "this row is fine HERE". The same row can be an error on a self-certified deployment and an admitted warning on an un-migrated one, and a caller explaining a verdict needs to know which it got. An unconditionally-strict preview was considered and rejected (option B): it would fail rows on every un-migrated deployment that the write would have accepted. |
29822982

29832983
### Nested Shape: `ValidateDataResponse.results[number]`
@@ -2987,6 +2987,7 @@ A write-path strip event: caller-supplied fields legally dropped from the payloa
29872987
| **valid** | `boolean` | ✅ | True when this row would be accepted by the write path. |
29882988
| **errors** | `{ field: string; code: string; message: string }[]` | ✅ | Findings that would REJECT this row. Empty when valid. |
29892989
| **warnings** | `{ field: string; code: string; message: string }[]` | ✅ | Findings the target deployment ADMITS rather than rejects — today, ADR-0104 value shapes under a warn-first posture. The row is valid; the write would store it and log the same complaint. |
2990+
| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed'> }[]` | optional | Write-observability for the preview: caller-supplied fields the write would LEGALLY strip from THIS row, one event per reason, in the shape and reason vocabulary the write itself reports. A strip is not a finding: `valid` is unaffected, and the fields appear in neither `errors` nor `warnings`. Present only when at least one field would be dropped. A server that does not produce this report omits the key too, so an absent key alone does not prove nothing would be dropped. |
29902991

29912992
### Nested Shape: `ValidateDataResponse.posture`
29922993

‎packages/spec/authorable-surface/api.json‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -941,11 +941,13 @@
941941
"api/ImportResponse:writeMode",
942942
"api/ImportRowResult:action",
943943
"api/ImportRowResult:code",
944+
"api/ImportRowResult:droppedFields",
944945
"api/ImportRowResult:error",
945946
"api/ImportRowResult:field",
946947
"api/ImportRowResult:id",
947948
"api/ImportRowResult:ok",
948949
"api/ImportRowResult:row",
950+
"api/ImportRowResult:warnings",
949951
"api/ImportValidationConfig:dateFormat",
950952
"api/ImportValidationConfig:deduplication",
951953
"api/ImportValidationConfig:maxErrors",

‎packages/spec/src/api/export.test.ts‎

Lines changed: 91 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { describe, it, expect, expectTypeOf } from 'vitest';
2-
import type { ImportRequest } from './export.zod';
2+
import type { ImportRequest, ImportRowResult } from './export.zod';
33
import {
44
ExportFormat,
55
ImportValidationMode,
@@ -10,7 +10,12 @@ import {
1010
ExportImportTemplateSchema,
1111
ImportRequestSchema,
1212
CreateImportJobRequestSchema,
13+
ImportRowResultSchema,
14+
ImportJobResultsSchema,
1315
} from './export.zod';
16+
import type { ValidateDataIssue } from './protocol.zod';
17+
import { DroppedFieldsEventSchema } from '../data/data-engine.zod';
18+
import type { DroppedFieldsEvent } from '../data/data-engine.zod';
1419

1520
// ==========================================
1621
// Export Format
@@ -386,3 +391,88 @@ describe('ImportRequestSchema — mappingName declared (#10330)', () => {
386391
expect(described).toMatch(/CONFLICTING_MAPPING/);
387392
});
388393
});
394+
395+
// ==========================================
396+
// Import Row Result — the write-drop signal and the served dry-run warnings
397+
// ==========================================
398+
399+
/**
400+
* The row report carries two optional keys that ride on an `ok` row:
401+
* `droppedFields` (the engine's strip report, declared ahead of its producer)
402+
* and `warnings` (served by the REST dry run before it was declared). The
403+
* schema is a plain, non-strict `z.object`, so an undeclared key is not
404+
* refused — it is STRIPPED by `parse`. Each pin therefore asserts the key
405+
* SURVIVES a parse, never bare `success`: a schema without the key would
406+
* still answer `success: true` and hand back a row with the report gone.
407+
*/
408+
describe('ImportRowResultSchema — droppedFields and warnings', () => {
409+
const okRow = { row: 2, ok: true, action: 'created' as const, id: 'rec_1' };
410+
const drops = [
411+
{ object: 'project', fields: ['doubled'], reason: 'computed' as const },
412+
{ object: 'project', fields: ['code'], reason: 'readonly' as const },
413+
];
414+
415+
it('parses a row without droppedFields — the key is optional', () => {
416+
const parsed = ImportRowResultSchema.parse(okRow);
417+
expect(parsed).toEqual(okRow);
418+
expect('droppedFields' in parsed).toBe(false);
419+
});
420+
421+
it('parses a row with droppedFields and keeps every event', () => {
422+
const parsed = ImportRowResultSchema.parse({ ...okRow, droppedFields: drops });
423+
expect(parsed.droppedFields).toEqual(drops);
424+
expect(parsed.ok).toBe(true);
425+
expect(parsed.action).toBe('created');
426+
});
427+
428+
it('the element is the engine\'s DroppedFieldsEventSchema itself — no second vocabulary', () => {
429+
const element = ImportRowResultSchema.shape.droppedFields.unwrap().element;
430+
expect(element).toBe(DroppedFieldsEventSchema);
431+
});
432+
433+
it('accepts exactly the engine\'s reason set, `computed` included', () => {
434+
const engineReasons = DroppedFieldsEventSchema.shape.reason.options;
435+
expect(engineReasons).toContain('computed');
436+
for (const reason of engineReasons) {
437+
const parsed = ImportRowResultSchema.parse({
438+
...okRow, droppedFields: [{ object: 'project', fields: ['f'], reason }],
439+
});
440+
expect(parsed.droppedFields?.[0]?.reason, `reason ${reason}`).toBe(reason);
441+
}
442+
const outside = ImportRowResultSchema.safeParse({
443+
...okRow, droppedFields: [{ object: 'project', fields: ['f'], reason: 'not_a_reason' }],
444+
});
445+
expect(outside.success).toBe(false);
446+
const issue = outside.success ? undefined : outside.error.issues[0];
447+
expect(issue?.code).toBe('invalid_value');
448+
expect(issue?.path).toEqual(['droppedFields', 0, 'reason']);
449+
});
450+
451+
it('keeps the served dry-run warnings, in the validate verdict\'s issue shape', () => {
452+
const warnings = [{ field: 'billing_address', code: 'invalid_type', message: 'billing_address has an invalid value' }];
453+
const parsed = ImportRowResultSchema.parse({ ...okRow, warnings });
454+
expect(parsed.warnings).toEqual(warnings);
455+
const partial = ImportRowResultSchema.safeParse({ ...okRow, warnings: [{ field: 'x', code: 'y' }] });
456+
expect(partial.success).toBe(false);
457+
const issue = partial.success ? undefined : partial.error.issues[0];
458+
expect(issue?.path).toEqual(['warnings', 0, 'message']);
459+
});
460+
461+
it('the async job sample carries the same row keys', () => {
462+
const parsed = ImportJobResultsSchema.parse({
463+
jobId: 'imp_1', object: 'project', status: 'succeeded', dryRun: false, writeMode: 'insert',
464+
total: 1, processed: 1, created: 1, updated: 0, skipped: 0, errors: 0, percentComplete: 100,
465+
undoable: true, createdAt: '2026-09-30T00:00:00.000Z',
466+
results: [{ ...okRow, droppedFields: drops }], resultsTruncated: false,
467+
});
468+
expect(parsed.results[0]?.droppedFields).toEqual(drops);
469+
const described = (ImportJobResultsSchema.shape.results as { description?: string }).description ?? '';
470+
expect(described).toMatch(/failures first/);
471+
expect(described).toMatch(/inside the sample/);
472+
});
473+
474+
it('the typed reader sees both keys, in the engine\'s and the verdict\'s types', () => {
475+
expectTypeOf<NonNullable<ImportRowResult['droppedFields']>[number]>().toEqualTypeOf<DroppedFieldsEvent>();
476+
expectTypeOf<NonNullable<ImportRowResult['warnings']>[number]>().toEqualTypeOf<ValidateDataIssue>();
477+
});
478+
});

0 commit comments

Comments
 (0)