Skip to content

Commit f7eb860

Browse files
fix(verify): read a refused create's whole answer before calling it a hard failure (#24516)
Fixes #24498 Clause-②: no Basis: the CRUD stage reads the same answer correctly. No accept set, export or published shape moves. ## What changed `packages/verify/src/verify.ts`, `runCrudVerification`, the refused-create branch: - The body is read once, in full. The verdict comes from a new private helper, `isValidationAnswer(text)`. - When the body is JSON with a string top-level `code`, a 400 is `needs-fixture` exactly when that code is `VALIDATION_FAILED`. - When the body is not JSON, or has no string `code`, the stage's original pattern (`VALIDATION_FAILED|Validation failed`, case-insensitive) is matched against the full text. - The stored `detail` keeps its 280-character cut (`text.slice(0, 280)`), taken after the verdict. - The rule is unchanged: a 400 validation answer is `needs-fixture`, and anything else is `create-failed`. No other status value, exit-code rule or verify stage is touched. **The shape was measured, not guessed.** A real in-process stack (`bootStack`, this branch, after the dependency closure was built) answered a create refused by a script rule with `400` and a 755-character body whose keys are `["error","code","fields","object"]`. The `"code":"VALIDATION_FAILED"` key starts at character 324. That is the flat envelope `packages/types/src/data-error-classification.ts` builds for the validation arm (`error` = the app's message, then `code`, `fields`, `object`). It is also the top-level `code` that the sibling stages already read off this same door (`reference-shape.ts`, `delete-behavior.ts`, `unique-scope.ts`). The test fixture uses that measured key order and `fields` entry shape. ## Before and after: the stored row for a long validation answer, measured The probe is a throwaway script, not committed. It boots one real stack with one object whose script rule (CEL `true`, events insert and update) refuses every create with a three-sentence message, like hotcrm's forecast rule. It then runs `runCrudVerification` over that app. "Before" imports `verify.ts` exactly as it is at base `6b42b6bc8`. "After" imports it at `3024fa2b7`. | run | `verify.ts` | status | code | `detail` length | summary | |---|---|---|---|---|---| | before | base `6b42b6bc8` | `create-failed` | 400 | 280 | `createFailed: 1, needsFixture: 0` | | after | `3024fa2b7` | `needs-fixture` | 400 | 280 | `createFailed: 0, needsFixture: 1` | The stored `detail` is byte-identical in both rows. It ends `... reconcile the three amounts, th`, before the `code` key at character 324. Only the verdict moves. The optional hotcrm re-run was **not done**. The hotcrm checkout has no `node_modules`. Running `os verify` there with this branch's build means installing its 18.0.0-next.3 dependency tree and building and wiring `@objectstack/cli` from this branch, which is more than linking the built package. The in-process probe above drives the same door and the same stage. ## Ablation The ablation restores today's cut-before-classify order with one mutation in `verify.ts`. It goes through `node scripts/ablation-replace.mjs` (WRAP mode, its own exit, INT and TERM trap, plus a shell trap that runs `git checkout HEAD --` on the absolute path): - Before the mutation, the line read `const text = await created.text();`. The mutation made it `const text = (await created.text()).slice(0, 280);`. - The anchor was hit 1 time and dropped to 0. The replacement count went from 0 to 1. The blob moved from `d8fe2c05c833` to `d9528c3cad95`. - With the mutation, `vitest run src/verify.create-refusal-verdict.test.ts` gave **1 failed, 4 passed**. The one red is the long-message case, `expected status "needs-fixture", received "create-failed"`. All four controls stayed green. - After the restore, the blob read `d8fe2c05c833`, which equals HEAD. `git diff HEAD` was empty and `git status --porcelain` was empty. The test imports `./verify.js` from source (a relative import, so there is no `dist/` resolution), so no rebuild leg was owed. ## Controls The new test file is `packages/verify/src/verify.create-refusal-verdict.test.ts`. It uses a stack double (`apiAs` only, the same double pattern `reference-shape.test.ts` uses) and boots nothing. It has 5 cases, and all pass on HEAD: 1. A 400 `VALIDATION_FAILED` whose message runs past 280 characters is `needs-fixture`, and `detail` equals the first 280 characters of the body. The fixture asserts its own precondition: the code key starts after character 280. 2. Control: a short 400 `VALIDATION_FAILED` is `needs-fixture`. 3. Control: a 500 `INTERNAL_ERROR` is `create-failed`. 4. Control: a 400 `INVALID_FIELD` is `create-failed`. 5. Control: a 400 non-JSON `Validation failed: ...` body is `needs-fixture`. This exercises the full-text fallback. ## Local verification (HEAD `3bcc4ae59`) - `pnpm --filter @objectstack/verify exec vitest run --maxWorkers=2` (the whole package suite, under the shared lock) gave 44 test files and 376 tests passed, `VERDICT command-exit 0`. - `pnpm --filter @objectstack/verify typecheck` exited 0. It runs `tsc --noEmit` and then `check:test-typecheck`, which reported "0 file(s) / 0 error(s)". `tsc -p tsconfig.test.json --listFiles` lists the new test file (1 hit). For comparison it lists 44 verify test files and 100 `packages/core/src` files. - eslint (`--no-inline-config`) over `verify.ts` and the new test exited 0, with 0 errors and 0 warnings. The JSON output confirms both files were linted. - `node scripts/pm/dispatch-gates.mjs --commands` derived 63 commands. All 63 were run with each exit code captured before any pipe, and all exited 0. `pnpm check:dual-build-cjs-loads` first answered `PREREQUISITE NOT MET` (exit 3, unbuilt packages elsewhere in the tree). It was re-run after a full `turbo run build` (75 of 76 tasks cached) and exited 0. - The `--ran` reconciliation read: `63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN` (a derived zero, all 63 with recorded exit codes). - `pnpm lint` (`eslint . --no-inline-config`), the full union, was run at `3bcc4ae59` (the final commit) and exited 0 with no findings printed. ## Acceptance notes - A JSON 400 whose top-level `code` is something other than `VALIDATION_FAILED`, but whose prose contains "Validation failed", now reads `create-failed`. The old pattern over the cut made it `needs-fixture`. On the `/data` create door, the only producer that writes that phrase into the flat envelope is the `VALIDATION_FAILED` arm itself (`data-error-classification.ts`). The other producers of the phrase are other doors: plugin-auth's adapter, the import runner, and the CLI's own printers. So no answer the door gives today changes its verdict except the long-message case. - There are no out-of-scope findings. No other verify stage cuts a body before it classifies it: the sibling stages parse the full text, and their `clip` is applied to stored detail only. --- _Generated by [Claude Code](https://claude.ai/code/session_01SYEJqDL3NXxrJRBVqaMu9T)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent cbe6d87 commit f7eb860

3 files changed

Lines changed: 134 additions & 3 deletions

File tree

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
'@objectstack/verify': patch
3+
---
4+
5+
fix(verify): the CRUD stage reads a refused create's whole answer before it calls the create a hard failure
6+
7+
Clause-②: no
8+
9+
- **A long validation message is `needs-fixture` again.** The stage used to cut a refused create's body to 280 characters and then look for the validation code in that cut. The `/data` door answers a validation refusal as `{ error, code, fields, object }`, with the app-authored message first, so a message long enough pushed `"code":"VALIDATION_FAILED"` past the cut. The object was then reported `create-failed`, a hard failure that makes `os verify` exit 1, although the app's own rule had refused the auto-derived record.
10+
- **The verdict reads the whole body.** When the body is JSON, the stage reads the envelope's `code`: `VALIDATION_FAILED` on a 400 is `needs-fixture`. A body that is not JSON, or has no string `code`, is matched against the full text with the stage's original pattern.
11+
- **Unchanged:** the stored `detail` still holds at most 280 characters, and the rule is the same: a 400 validation answer is `needs-fixture`, and anything else is `create-failed`.
12+
13+
A patch: the stage reads the same answer correctly. No export, accepted value or published shape changes.
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
//
3+
// The CRUD stage's verdict on a refused create, read from the WHOLE answer.
4+
//
5+
// `runCrudVerification` sorts a refused create into two rows: a 400 validation
6+
// answer is `needs-fixture` (the auto-derived record did not satisfy the app's
7+
// own rules), anything else is `create-failed` (a hard failure that makes
8+
// `os verify` exit 1). The row stores at most 280 characters of the body as its
9+
// `detail`. The `/data` door's validation envelope puts the app-authored message
10+
// FIRST and `code` after it, so a long message pushes `"code":"VALIDATION_FAILED"`
11+
// past character 280: a verdict taken from the stored cut reads a validation
12+
// answer as a hard failure. These cases pin the verdict to the whole body and
13+
// the cut to the stored `detail` only.
14+
//
15+
// A stack double answers the create, so nothing here boots a stack: the unit
16+
// under test is the classification, and the refusal bodies are the `/data`
17+
// door's flat envelope as it is written on the wire.
18+
19+
import { describe, it, expect } from 'vitest';
20+
21+
import type { VerifyStack } from './harness.js';
22+
import { runCrudVerification, type ObjectVerifyResult } from './verify.js';
23+
24+
const OBJECT = 'vcr_forecast';
25+
const config = {
26+
manifest: { id: 'com.example.verify-create-refusal', namespace: 'vcr', version: '0.0.1', type: 'app', name: 'Create Refusal Fixture' },
27+
objects: [{ name: OBJECT, label: 'Forecast', fields: { name: { type: 'text', label: 'Name' } } }],
28+
};
29+
30+
/** A stack double whose only answer is the given response to the create. */
31+
function answeringCreateWith(response: () => Response): VerifyStack {
32+
return {
33+
async apiAs(_token: string, method: string, path: string) {
34+
if (method === 'POST' && path === `/data/${OBJECT}`) return response();
35+
throw new Error(`unexpected ${method} ${path}: the stage must stop at a refused create`);
36+
},
37+
} as unknown as VerifyStack;
38+
}
39+
40+
async function verdictFor(response: () => Response): Promise<ObjectVerifyResult> {
41+
const report = await runCrudVerification(answeringCreateWith(response), 'token', config);
42+
expect(report.results).toHaveLength(1);
43+
return report.results[0];
44+
}
45+
46+
/** The `/data` door's validation envelope, in its wire key order. */
47+
const validationEnvelope = (message: string) => ({
48+
error: message,
49+
code: 'VALIDATION_FAILED',
50+
fields: [{ field: '_record', code: 'rule_violation', message }],
51+
object: OBJECT,
52+
});
53+
54+
// Three sentences, as an app's forecast rule writes them: long enough that the
55+
// envelope's `code` lands past the stored cut.
56+
const LONG_MESSAGE =
57+
'A forecast must name the fiscal period it commits to before it can be saved. ' +
58+
'The committed amount cannot exceed the best-case amount, and the best-case amount cannot exceed the pipeline amount. ' +
59+
'Ask the forecast owner to set the period and reconcile the three amounts, then submit the forecast again for approval.';
60+
61+
describe('runCrudVerification — the verdict on a refused create reads the whole answer', () => {
62+
it('a 400 VALIDATION_FAILED whose message runs past 280 characters is needs-fixture, and the stored detail keeps its cut', async () => {
63+
const text = JSON.stringify(validationEnvelope(LONG_MESSAGE));
64+
// The fixture exercises the defect: the code is NOT inside the stored cut.
65+
expect(text.indexOf('"code":"VALIDATION_FAILED"')).toBeGreaterThan(280);
66+
67+
const r = await verdictFor(() => new Response(text, { status: 400, headers: { 'content-type': 'application/json' } }));
68+
expect(r).toMatchObject({ object: OBJECT, status: 'needs-fixture', code: 400 });
69+
expect(r.detail).toBe(text.slice(0, 280));
70+
expect(r.detail!.length).toBeLessThanOrEqual(280);
71+
});
72+
73+
it('control: a short 400 VALIDATION_FAILED is needs-fixture', async () => {
74+
const r = await verdictFor(() => Response.json(validationEnvelope('A forecast must name its fiscal period.'), { status: 400 }));
75+
expect(r).toMatchObject({ object: OBJECT, status: 'needs-fixture', code: 400 });
76+
});
77+
78+
it('control: a 5xx is create-failed', async () => {
79+
const r = await verdictFor(() => Response.json({ error: 'Internal server error', code: 'INTERNAL_ERROR' }, { status: 500 }));
80+
expect(r).toMatchObject({ object: OBJECT, status: 'create-failed', code: 500 });
81+
});
82+
83+
it('control: a 400 with another code is create-failed', async () => {
84+
const r = await verdictFor(() =>
85+
Response.json({ error: `Unknown field 'region' on object '${OBJECT}'`, code: 'INVALID_FIELD', object: OBJECT }, { status: 400 }),
86+
);
87+
expect(r).toMatchObject({ object: OBJECT, status: 'create-failed', code: 400 });
88+
});
89+
90+
it('control: a 400 that is not JSON is read as text with the original pattern', async () => {
91+
const r = await verdictFor(() => new Response('Validation failed: name is required', { status: 400 }));
92+
expect(r).toMatchObject({ object: OBJECT, status: 'needs-fixture', code: 400 });
93+
});
94+
});

‎packages/verify/src/verify.ts‎

Lines changed: 27 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,29 @@ function deepEqual(a: unknown, b: unknown): boolean {
4444
return JSON.stringify(a) === JSON.stringify(b);
4545
}
4646

47+
/**
48+
* Whether a refused create's body is the app's own validation answer, read from
49+
* the WHOLE body. The `/data` door answers a validation refusal with the flat
50+
* ADR-0112 envelope `{ error: <message>, code: 'VALIDATION_FAILED', fields, object }`
51+
* (`@objectstack/types` data-error-classification), the same top-level `code`
52+
* the sibling stages read off this door. The message comes first, so a long
53+
* app-authored message pushes `code` past any fixed-length cut of the text:
54+
* the verdict is never taken from the 280-character `detail` the row stores.
55+
* A body that is not JSON, or carries no string `code`, is read as text with
56+
* the stage's original pattern.
57+
*/
58+
function isValidationAnswer(text: string): boolean {
59+
let body: unknown;
60+
try {
61+
body = JSON.parse(text);
62+
} catch {
63+
body = undefined;
64+
}
65+
const code = (body as { code?: unknown } | null | undefined)?.code;
66+
if (typeof code === 'string') return code === 'VALIDATION_FAILED';
67+
return /VALIDATION_FAILED|Validation failed/i.test(text);
68+
}
69+
4770
/**
4871
* Run auto-derived CRUD round-trip proofs for every object in `config`, as the
4972
* authenticated `token`, against the live `stack`. Returns a structured report;
@@ -79,17 +102,18 @@ export async function runCrudVerification(
79102
continue;
80103
}
81104
if (created.status >= 300) {
82-
const text = (await created.text()).slice(0, 280);
105+
const text = await created.text();
83106
// A 400 validation failure means our auto-derived record didn't satisfy the
84107
// app's own validation rules (record-level format/json-schema/CEL) — that's
85108
// a fixture gap, not a platform finding. A 5xx (or crypto/driver error) is a
86109
// real runtime failure the app's author needs to see.
87-
const isValidation = created.status === 400 && /VALIDATION_FAILED|Validation failed/i.test(text);
110+
const isValidation = created.status === 400 && isValidationAnswer(text);
88111
results.push({
89112
object: c.object,
90113
status: isValidation ? 'needs-fixture' : 'create-failed',
91114
code: created.status,
92-
detail: text,
115+
// The cut is for the stored row only; the verdict above read the whole answer.
116+
detail: text.slice(0, 280),
93117
});
94118
continue;
95119
}

0 commit comments

Comments
 (0)