Skip to content

Commit 48e84a9

Browse files
fix(objectql,types,rest)!: a database NOT NULL refusal on a declared field is the record validator's ValidationError at every door (#26026)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"} --> _Requested by **Leehom** · [project thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1XCDufXsPatdpw4EF3iFqDwB)_ Fixes #24768 Fixes #26065 Clause-②: yes (narrowing: `@objectstack/types` adds the export `notNullViolationSite` and retires `notNullRefusalSite` and `relabelNotNullRefusal`; the HTTP doors' answer for this refusal moves to the record validator's envelope) Metadata-protocol: none **BREAKING** (an export retirement and a wire-answer change), shipped as `patch`: no metadata protocol kind moves. The changeset carries the ADR-0087 disposition (`runtime-export-retired`) and the FROM → TO lines for both retired exports. ## What was wrong A field declared `storage: { notNull: true }` without `required` is ADR-0113's engine-populated column. The record validator judges `required` only (D1 / D2), so a write with no value for that field reaches the driver and the database refuses it. The engine let the driver's error out, and each door classified it from the driver's text its own way. Measured on `origin/main` `0c37228826` with a real better-sqlite3 store, the real `ObjectQL` engine and protocol, and `HttpDispatcher.dispatch()` over a Fetch `Request` (the rig is `packages/runtime/src/dispatcher-not-null-refusal-24768.test.ts`): | Door | Before | After | |:--|:--|:--| | dispatcher `POST /data/:object` without the field | raw `SqliteError` (`SQLITE_CONSTRAINT_NOTNULL`) out of `dispatch()`; `declaredEnvelopeForThrow` and `validationFailureDetails` both decline it, so `500 INTERNAL_ERROR` | `400 VALIDATION_FAILED`, `Sales Price is required`, `details.fields` = the validator's envelope with `label` | | dispatcher `PATCH /data/:object/:id` writing null | same `500 INTERNAL_ERROR` | `400 VALIDATION_FAILED`, the clear-out sentence (`required_cleared`) | | actions door, declarative `operation: 'update'` writing null | `400 VALIDATION_ERROR`, the driver's text as message, `declaredCode: 'SQLITE_CONSTRAINT_NOTNULL'` | `400 VALIDATION_FAILED` with the field envelope, no `declaredCode`, no driver text | | `RestServer` `POST` / `PATCH` | `400 VALIDATION_FAILED`, relabelled by a second object read (#24715) plus a `hint` | the validator's `required` body exactly: `fields[0]` carries `label`, no `hint`, PATCH reads `required_cleared` | | `RestServer` `POST /data/:object/batch` row | the column key (the card's measurement; not re-measured here) | row error `{ code: 'VALIDATION_FAILED', message: 'Sales Price is required' }` | Import rows, the public form submit and MCP write through the same engine doors and answer `ValidationError` through the same arms; they were not driven separately. ## The change - **`@objectstack/objectql`** (`not-null-refusal-error.ts`, new). `envelopeNotNullViolation` turns a driver NOT NULL refusal on a field the object's field map holds into `NotNullRefusalError extends ValidationError`, built by the validator's own `buildFieldError` with the validator's message context: one field error, `code: 'required'`, named by its label (translation bundle, then declared `label`, then API name) in the caller's locale, with the sentence the validator gives the same verb (`required` on insert, `required_cleared` on update). The driver's error rides `cause`. `envelopeDriverWriteRefusal` composes it after the existing unique translation (#14095) and replaces `envelopeUniqueViolation` at all five driver exits in `engine.ts` (both autonumber-retry sites, the `bulkCreate` exit, by-id update, `updateMany`). The two write-failure log lines log `cause` for this class too, so the log still carries the database's diagnosis. Where the error leaves the engine, `redactPropagatedDriverFault` cuts its `cause` like any driver error. - Left as the driver threw it: a column no field declares (drift), a refusal the driver attributes to another table, and a NOT NULL code whose column the driver did not name. ⛔ No pre-write check reads `storage.notNull`: `required` and `storage.notNull` stay apart (ADR-0113 D1 / D2). - **`@objectstack/types`.** New `notNullViolationSite(error)`: the column (and table, where the dialect names it) of a driver's NOT NULL refusal, read from the error's codes first (better-sqlite3 `SQLITE_CONSTRAINT_NOTNULL`; libsql `extendedCode` / `rawCode: 1299`; node-postgres `23502` with `column` / `table`; mysql2 `ER_BAD_NULL_ERROR` / `errno: 1048`), then its text (all sql.js gives), then down its `cause` chain. An error whose code names another refusal is not read for NOT NULL text. `notNullRefusalSite` and `relabelNotNullRefusal` retire. Before removing them, `ui/**`, `examples/**` and hotcrm were grepped: zero consumers (the control legs, the export names in this repo's own sources, matched). `mapDataError`'s NOT NULL branch is unchanged and answers the undeclared column. - **`@objectstack/rest`.** PR #24759's door-side relabel (`not-null-refusal-label.ts` and its test) retires with its `CrudRouteHost.labelNotNullRefusal` member and call sites: every path it served is an engine write. `crud-routes.ts` goes back to `const context` inside the try. `objectItemReader` stays (the exposure gate reads it); its doc no longer mentions the relabel. ## Interpretations for review - **`Clause-②` departs from the claim's `no`.** Measured: two exports of `@objectstack/types`' published entry retire, and the dispatcher's `500` becomes a `400` with a different body, so this declares `yes (narrowing)`. The seat's contract review applies. - **`required_cleared` on update.** Same sentence the validator gives a PATCH that clears a `required` field, so the two refusals read alike. - **No `hint` for a declared field.** The validator's `required` envelope has none. The undeclared-column answer keeps the classification's column sentence and drift hint. - **A drifted column that a field still declares is now labelled** (the 24715 drift control now reads `Drift Column` with `required_cleared`). - **The table check.** A nested write's refusal on another object is left for that object's door to name. - **The tenant column is a declared field.** `sys_user_permission_set` declares `organization_id`, so plugin-auth's anchor-parity case "fresh tenant column, no organization object, single" now gets `VALIDATION_FAILED` naming `organization_id`, with `SQLITE_CONSTRAINT_NOTNULL` on `cause`. The pin moved (claim amendment 6411384588) and still proves the column refused the row. Whether platform-filled columns (tenant, audit, owner, readonly) should be excluded is left as an open question in the report. ## Tests - `packages/runtime/src/dispatcher-not-null-refusal-24768.test.ts` (new): red on `main` (3 failed), green here (3 passed). - `packages/objectql/src/engine-not-null-refusal.test.ts` (new, 18): every driver exit, the log line, five dialects, zh-CN / zh parity with the validator's own `required`, and the controls (undeclared column, other table, `23502` with no column, unique still `DuplicateRecordError`, `required` still pre-driver). - `packages/types/src/data-error-classification.not-null-label.test.ts` (rewritten, 14): the reader per dialect, the cause chain, the other-code refusals, the undefined cases. - `packages/rest/src/data-not-null-field-label-24715.test.ts` (8) and `data-not-null-hint-20963.test.ts`: the RestServer doors and batch row by label; the undeclared column keeps its sentence and hint. - `driver-fault-boundary-redaction.test.ts` (168): the NOT NULL shapes × write doors pinned as `NotNullRefusalError` with a cut `cause` of the same prototype and code. - The duplicate-record controls' NOT NULL fixture now names an undeclared column; plugin-auth's anchor-parity pin as above. - Full `--no-bail` run of types (781), objectql (8756), rest (6469), runtime (7214), plugin-audit (993), plugin-auth (3720 + the moved pin) and service-package (125): green. ## Ablation `envelopeDriverWriteRefusal` mutated to return the unique translation only (`scripts/ablation-replace.mjs --hold`), objectql rebuilt and `ablation-dist-preflight` confirmed the mutation in `dist/`. Red: runtime 3/3, rest 24715 7/8, objectql 12/18, plugin-auth anchor-parity 1/13. Restored: `git diff HEAD` empty, file hash equals `HEAD`'s blob, preflight `--absent` 0, rebuilt, all four green. ## Gates On head `50c3929ba3`: `dispatch-gates --commands` derived 77; all 77 ran and exit 0 (`check:dual-build-cjs-loads` re-run after a full `turbo run build`, its first run exit 3 for missing `dist/`). `dispatch-gates --ran`: 77 derived, 77 run, 0 NOT-MEASURED. `eslint --no-inline-config` over the touched packages' `src`: 0. `typecheck` for types, objectql, rest, runtime, plugin-auth: 0. `check-platform-checklist --changed-from origin/main`: OK. No open PR touches `packages/objectql/**` or `packages/rest/**`. ## Dogfood `pnpm --filter @objectstack/dogfood test` under the verify lock, after a full `turbo run build` on head `50c3929ba3`: `VERDICT command-exit 1`, Test Files 1 failed | 295 passed | 3 skipped (299), Tests 1 failed | 2522 passed | 18 skipped (2541). The one failure is not this change's: `test/authz-probe-blind-spot.test.ts` > `packages/runtime/src/domains/i18n.ts — every positive control is still present in THAT file` (`control "handleI18nRequest" moved: expected 4 to be 3`). The census reads source text; this PR touches neither the census nor `i18n.ts`, and `origin/main`'s `i18n.ts` holds `handleI18nRequest` 4 times since `69fa23d60b` (#25956), against 3 at its parent. It is red on `main` by construction and is reported, not fixed here. ## Acceptance notes - Every HTTP door answers a database NOT NULL refusal on a declared field as `400 VALIDATION_FAILED`, the field named by its label, in the shape the record validator gives `required`. - A column no field declares keeps today's answer (column sentence, drift hint). - No pre-write check reads `storage.notNull`. - The REST relabel from #24759 is gone; the two `@objectstack/types` exports it used retire with a declared migration. --- _Generated by [Claude Code](https://claude.ai/code/session_01E1oXN1ZJ8NnZKKhzWeCZ6q)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 95294be commit 48e84a9

18 files changed

Lines changed: 1007 additions & 573 deletions
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
'@objectstack/objectql': patch
3+
'@objectstack/types': patch
4+
'@objectstack/rest': patch
5+
---
6+
7+
fix(objectql,types,rest)!: a database NOT NULL refusal on a declared field is the record validator's `ValidationError` at every door, named by the field's label
8+
9+
A field declared `storage: { notNull: true }` without `required` is ADR-0113's engine-populated column: the record validator judges `required` only, so a write with no value for it reaches the driver and the database refuses it. The engine let the driver's error out, and each HTTP door classified it from the driver's text its own way. For the same refused write, the runtime dispatcher's `/data` POST and PATCH answered `500 INTERNAL_ERROR`, the actions door answered `400 VALIDATION_ERROR` with the driver's text and `declaredCode: 'SQLITE_CONSTRAINT_NOTNULL'`, and `RestServer` answered `400 VALIDATION_FAILED` relabelled by a second object read on its own `POST` / `PATCH` only.
10+
11+
Clause-②: yes (narrowing: `@objectstack/types` adds the export `notNullViolationSite` and retires `notNullRefusalSite` and `relabelNotNullRefusal`; the HTTP doors' answer for this refusal moves to the record validator's envelope)
12+
Metadata-protocol: none
13+
14+
<!-- adr-0087: not-required (runtime-export-retired @objectstack/types#notNullRefusalSite, @objectstack/types#relabelNotNullRefusal) No metadata moves: no spec key, authorable spelling, object definition or stored shape is added, removed, renamed or re-shaped, so there is nothing for `objectstack migrate meta` to rewrite. What retires is two function exports of `@objectstack/types`' published entry, whose only consumer was `@objectstack/rest`'s door-side relabel, retired in this same change; no other consumer was found in this repository (its `ui/` and `examples/` included) or in hotcrm. The affected party is a TypeScript caller, and the compiler at that call site delivers the break. -->
15+
16+
**BREAKING** (an export retirement and a wire-answer change), shipped as `patch`: no metadata protocol kind moves.
17+
18+
- **`@objectstack/objectql`.** The write doors' driver exits (single-row and batch insert, the autonumber retry, by-id and predicate update) translate a driver NOT NULL refusal on a field the object's field map holds into the record validator's own `ValidationError` (`code: 'VALIDATION_FAILED'`), built by the validator's own builder with the validator's message context: one field error, `code: 'required'`, the field named by its label (translation bundle, then declared `label`, then API name) in the caller's locale, with the sentence the validator gives the same verb (`required` on insert, `required_cleared` on update). The driver's error rides `cause`, and the doors' log line still carries the database's own diagnosis. Left as the driver threw it: a column no field declares, a refusal the driver attributes to another table, and a NOT NULL code whose column the driver did not name. `required` and `storage.notNull` stay apart (ADR-0113 D1 / D2): no pre-write check reads `storage.notNull`.
19+
- **Every door answers it alike**, because every door already answers that class alike: the dispatcher `/data` POST and PATCH now answer `400 VALIDATION_FAILED` (was `500 INTERNAL_ERROR`), the actions door `400 VALIDATION_FAILED` with the field envelope (was `400 VALIDATION_ERROR` with the driver's text), and `RestServer`'s `POST` / `PATCH` and its `/data/:object/batch` rows answer the validator's envelope and sentence. Import rows, the public form submit and MCP write through the same engine doors and answer the same class through the same arms; they were not driven separately. On `RestServer` the body is now exactly the validator's `required` shape: `fields[0]` carries `label`, there is no `hint`, and a `PATCH` writing null reads the clear-out sentence.
20+
- **`@objectstack/types`.** New export `notNullViolationSite(error)`: the column (and table, where the dialect names it) of a driver's NOT NULL refusal, read from the error's codes (better-sqlite3 `SQLITE_CONSTRAINT_NOTNULL`; libsql `extendedCode` / `rawCode: 1299`; node-postgres `23502` with its `column` / `table` fields; mysql2 `ER_BAD_NULL_ERROR` / `errno: 1048`), then its text (all sql.js gives), then one step down its `cause` chain. An error whose code names another refusal is not read for NOT NULL text. `mapDataError`'s NOT NULL branch is unchanged, and answers the column no field declares.
21+
- **`@objectstack/rest`.** The `/data` create and update doors' NOT NULL relabel (`not-null-refusal-label.ts`) retires: every path it served is an engine write.
22+
23+
Migration, for a TypeScript caller of the retired exports:
24+
25+
- FROM `notNullRefusalSite(answer)` TO `notNullViolationSite(error)`: ask the driver's error, not `mapDataError`'s answer.
26+
- FROM `relabelNotNullRefusal(answer, { message, declaresNotNull })` TO nothing (no replacement): write through the engine, which answers a declared field's NOT NULL refusal as a labelled `ValidationError` itself.

‎packages/objectql/src/driver-fault-boundary-redaction.test.ts‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ import {
4545
} from '@objectstack/types';
4646
import { ObjectQL } from './engine.js';
4747
import { DuplicateRecordError } from './duplicate-record-error.js';
48+
import { NotNullRefusalError } from './not-null-refusal-error.js';
4849
import { SummaryRecomputeError } from './summary-errors.js';
4950
import { redactBoundStatement, redactPropagatedDriverFault } from '@objectstack/types';
5051

@@ -489,9 +490,20 @@ const DOORS: ReadonlyArray<readonly [string, Failing, (e: ObjectQL) => Promise<u
489490
/** One representative per dialect for the door sweep. */
490491
const DOOR_SHAPES = [SHAPES[1], SHAPES[6], SHAPES[11]];
491492

493+
/**
494+
* [#24768] The NOT NULL shapes, one per dialect. On a field the object declares
495+
* (`title` here) the insert and update doors refuse them as the record
496+
* validator's `ValidationError` (`NotNullRefusalError`) with the driver's error
497+
* on `cause`, so those door × shape pairs are pinned in their own block below
498+
* rather than by the sweep's same-prototype assertion.
499+
*/
500+
const NOT_NULL_SHAPES = [SHAPES[2], SHAPES[6], SHAPES[14]];
501+
const NOT_NULL_DOORS = new Set(['insert', 'insert (batch)', 'update by id', 'update by predicate']);
502+
492503
describe('[#21274] every engine door that reaches a driver exits through the cut', () => {
493504
for (const shape of DOOR_SHAPES) {
494505
for (const [door, failing, run] of DOORS) {
506+
if (NOT_NULL_SHAPES.includes(shape) && NOT_NULL_DOORS.has(door)) continue;
495507
it(`${door} — ${shape.id}`, async () => {
496508
const { engine, state } = await bootEngine(shape.make, failing);
497509
const error = await caught(run(engine));
@@ -527,6 +539,23 @@ describe('[#21274] every engine door that reaches a driver exits through the cut
527539
}
528540
}
529541

542+
for (const shape of NOT_NULL_SHAPES) {
543+
for (const [door, failing, run] of DOORS.filter(([name]) => NOT_NULL_DOORS.has(name))) {
544+
it(`${door}: a NOT NULL refusal on a declared field leaves as the validator's ValidationError, and a cut cause — ${shape.id}`, async () => {
545+
const { engine, state } = await bootEngine(shape.make, failing);
546+
const error = (await caught(run(engine))) as NotNullRefusalError;
547+
expectNoSentinel(error);
548+
expect(error).toBeInstanceOf(NotNullRefusalError);
549+
expect(error.code).toBe('VALIDATION_FAILED');
550+
expect(error.fields.map((f) => [f.field, f.code])).toEqual([['title', 'required']]);
551+
const cause = error.cause as Error & { code?: unknown };
552+
expect(Object.getPrototypeOf(cause)).toBe(Object.getPrototypeOf(state.lastRaw));
553+
expect(cause.code).toBe((state.lastRaw as Error & { code?: unknown }).code);
554+
expect(cause.message).toContain(shape.diagnostic);
555+
});
556+
}
557+
}
558+
530559
it('a middleware that logs what it caught sees the cut error too', async () => {
531560
const { engine } = await bootEngine(SHAPES[1].make, 'write');
532561
const seen: unknown[] = [];

‎packages/objectql/src/engine-insert-duplicate-record.test.ts‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,8 @@
3030
* via the per-row fallback loop, `insertMany`'s partial mode, and the
3131
* last-chance create the autonumber resync issues when the field vanished
3232
* mid-flight; and
33-
* - **nothing else moves**: a NOT NULL violation, a deadlock, a missing table
33+
* - **nothing else moves**: a NOT NULL violation on a column no field declares
34+
* ([#24768]: on a declared field it is the validator's `ValidationError`), a deadlock, a missing table
3435
* and an unreachable store leave the door as the very object the driver
3536
* threw — asserted on IDENTITY, not on a message match, so a future
3637
* "helpful" re-wrap cannot pass this file.
@@ -125,8 +126,14 @@ const memoryDuplicate = () =>
125126

126127
/* -------- the negative side: failures that must NOT change shape --------- */
127128

129+
/**
130+
* [#24768] On a column NO field of `doc` declares: a NOT NULL refusal on a
131+
* declared field is the engine's `ValidationError` now
132+
* (`engine-not-null-refusal.test.ts`), and this one is what still leaves as the
133+
* driver threw it.
134+
*/
128135
const notNullViolation = () =>
129-
Object.assign(new Error('NOT NULL constraint failed: doc.title'), {
136+
Object.assign(new Error('NOT NULL constraint failed: doc.legacy_col'), {
130137
code: 'SQLITE_CONSTRAINT_NOTNULL',
131138
});
132139

@@ -408,7 +415,7 @@ describe('engine.insert — a driver unique violation is a DUPLICATE_RECORD enve
408415

409416
describe('nothing that is not a unique violation changes shape', () => {
410417
const controls: Array<[string, () => unknown]> = [
411-
['a NOT NULL violation', notNullViolation],
418+
['a NOT NULL violation on an undeclared column', notNullViolation],
412419
['a missing table', missingTable],
413420
['a deadlock', deadlock],
414421
['an unreachable store', unreachableStore],

0 commit comments

Comments
 (0)