Skip to content

Commit 682873d

Browse files
fix(core): the JSON-column refusal reads true on every face and reaches a REST caller whole (#21213)
Fixes #21067 Clause-②: no ## What this changes `jsonColumnOperatorRefusalText` (`packages/core/src/utils/json-column-operator-refusal.ts`) is the one builder of the `INVALID_FILTER` / 400 refusal a filter gets for a scalar comparison or text operator on a multi-value or JSON field. It is rewritten once, for both of the card's reasons together (seat 2's carried note folded in: one rewrite, not two passes): 1. **Under the REST bound.** The withheld message was 748 characters. The `/data` door cuts a 4xx message of 500 or more to 499 plus an ellipsis, so the wire ended `…Refused rather than compiled because the answ…`, and no caller read the sentence saying the field and the operator were withheld. It is now **486** characters, one constant text. 2. **True on every face.** The old reason named `driver-sql`'s storage ("a field this driver stores as a JSON TEXT column") and the two wrong answers SQL used to give. Both are untrue on the engine's per-aggregation `filter` and on `driver-memory`, which print the same text. The reason is now the field's declaration: `it aims a scalar comparison or text operator at a multi-value or JSON field, which it cannot test for one member.` 3. **The null comparand's repair (added at the PM's note on this card).** The refused set also catches `{ f: null }`, `$eq: null` and `$ne: null`, which ask whether the field has a value; `$contains` cannot express that. One constant clause, with no branch on the comparand, names the presence spellings: `For no value, use "$null" or "$empty".` Both answer on a multi-value or JSON field on every face (they are outside the refused set), and the REST pin now proves `$null: true`, `$null: false` and `$empty: true` answer on both the `where` and the per-aggregation faces. The new message, in full: ```text A constraint in this filter WAS NOT APPLIED: it aims a scalar comparison or text operator at a multi-value or JSON field, which it cannot test for one member. Use "$contains" for membership ({ "FIELD": { "$contains": "a" } }), or an $or of "$contains" for any-of ({ "$or": [{ "FIELD": { "$contains": "a" } }, { "FIELD": { "$contains": "b" } }] }). For no value, use "$null" or "$empty". The field and the operator are withheld from the message; the full diagnostic is in the server log. ``` The diagnostic (server log, and the author-disclosed wire text, see below) shares that reason, names the operator in it (`it aims "$in", a scalar comparison or text operator, at …`; the bare spelling's operator as `=`, as before), names the field, and spells the remedy with the field's name, presence clause included. It drops the same storage and SQL history. One `refusalReason()` and one `containsRemedy()` (which ends in `PRESENCE_REMEDY`) serve both texts, so the two cannot drift. The SQL mechanism and the measured wrong answers stay in the builder's docblock. **Unchanged:** `code`, `status`, the refused operator set (`JSON_COLUMN_INCOMPATIBLE_OPERATORS`, 27 spellings), the `$contains` remedy, no new export, no new code. **Faces.** Re-derived at `cb45469e` by name and by text: every face that prints this sentence calls the builder, and none builds its own copy. They are `driver-sql` `jsonColumnOperatorError` (`sql-driver.ts`), `objectql` `having-filter.ts` (the judgment and the per-row backstop, three call sites), and `driver-memory` `jsonStoredFieldOperatorError` (`filter-refusal.ts`). No face file changes except one stale docblock in `sql-driver.ts`. `driver-turso`'s `remote-transport.ts` and `driver-mongodb` share phrases with other refusals, not this sentence. No docs page or skill quotes it (`content/docs/**`, `skills/**`, `docs/**`, `apps/docs/**`, by builder name and by seven distinctive phrases: zero hits). ## Measured, before and after | reading | `BASE` `cb45469e` | this PR | |:--|:--|:--| | withheld message, constant | 748 chars | 486 chars | | its wire `error`, SQLite and live PostgreSQL 16.14, `mapDataError` | 499 plus an ellipsis, ending `Refused rather than compiled because the answ…` | the whole message | | author-disclosed diagnostic, field `owners`, `$in` | 643 chars | 402 chars | | that, at `POST /api/v1/data/:object/query` through the real `SecurityPlugin` | 499 plus an ellipsis, ending `Refused rather than compiled beca…` | the whole diagnostic (measured at 377 before the presence clause; the 402 text is pinned whole through `mapDataError`) | | field-name length from which the diagnostic is cut on the wire | every length | 27 characters (the presence clause goes first) | | field-name length from which the cut takes the any-of example | 10 characters | 37 characters | ### A dispatch hypothesis this falsified (H5) The dispatch read the diagnostic as server-log text the envelope never bounds. Measured, it is also a wire text. `driver-sql`'s `'author'` provenance arm (`resolveWithheldFilterRefusal`) swaps the refusal for the diagnostic when the predicate is the caller's own, and `plugin-security` marks the caller's verbatim `where` `'author'` (`security-plugin.ts`, the `markFilterSubtreeProvenance(callerWhere, 'author')` call). Through `POST /api/v1/data/:object/query` with a real `SecurityPlugin`, a member caller sending `{ owners: { $in: ['u1'] } }` received the diagnostic: cut to 500 at `BASE`, whole (377) at `caf0e3c7`, before the presence clause raised it to 402. That was a one-off measurement file, run twice and not committed; the `BASE` leg rebuilt `core`'s `dist/` with the old text and restored it, with dist preflight proofs both ways. The diagnostic was in the rewrite already, because its reason clause was the same untrue `driver-sql` mechanism seat 2's note names. All four conditions of the bounded in-place fix hold: same defect class (a refusal cut at the envelope), a mechanical fix in the shape triage pinned, the claimed file, and the same gate families. So it is fixed here, not filed. Its content is kept: field and operator named, remedy with the field's name. ## Pins New, compared with the builder's output and the bound's own function (`truncateClientMessage` / `mapDataError` from `@objectstack/types`), never with a copied sentence or a retyped 500. `CLIENT_MESSAGE_MAX` itself is module-private in `packages/types/src/data-error-classification.ts`, so it is not imported; `truncateClientMessage` is already exported there for `rest`, and nothing new is exported. A sibling that rewords nothing and only calls the builder cannot flip these. - `packages/core/src/utils/json-column-operator-refusal.test.ts`: re-captured hashes and lengths. For every refused spelling, `truncateClientMessage(message)` returns it unchanged. The message carries the one-member, any-of and no-value remedy and ends with the withheld sentence. Neither text names a storage form or a backend's wrong answer. The diagnostic gives the same reason with the operator named. - `packages/drivers/driver-sql/src/sql-driver-json-column-refusal-wire-bound.test.ts` (new, `DIALECT_CELLS`): for 14 operators plus bare equality, on a multi-value lookup, a `tags` field and a `json` field, `mapDataError(err).body.error` equals the shared message (unmarked, carrying the presence clause) and the shared diagnostic (author-marked), with the remedy spelling. SQLite always; **PostgreSQL and MySQL in CI's `Temporal Conformance (live PG + MySQL)` job**, which runs this package's whole suite with both URLs set. - `packages/rest/src/aggregation-filter-json-column-refusal.test.ts`: through `POST /api/v1/data/:object/query`, the `where` twin's body equals the builder's message whole, not just the per-aggregation face's (`toBe`, plus the any-of remedy, the presence clause and the withheld sentence), on the null-comparand rows too; three new controls show `$null: true`, `$null: false` and `$empty: true` answer with equal counts on both faces. Flipped, each to the new substance: - `sql-driver-json-column-operator-refusal.test.ts`: `'JSON TEXT column'` becomes the operator-named reason, and the remedy is asserted with the field's own name. - `sql-driver-target-field-provenance.test.ts`: the class fragment that survives redaction, `'JSON TEXT column'`, becomes `'at a multi-value or JSON field'`. - `sql-driver-json-column-refusal-shared-text.test.ts`: the docblock's "did not change by one byte" claim. ADR-0112 `code` plus `status` assertions are untouched everywhere. ### Reverse verification At `fefb6e1f`, `BASE`'s builder was written to disk (tree only, never staged) and its landing checked by grep: old text 1, new 0. Then: - `sql-driver-json-column-refusal-wire-bound.test.ts` (SQLite plus live PostgreSQL 16.14): **90 failed** (45 per cell), 1 skipped (MySQL). The red direction, as expected. - The core pin: **8 failed**, 4 passed. Restored with `git checkout HEAD -- PATH` under an `EXIT INT TERM` trap. The blob is `2f17d7f3`, equal to HEAD's, and `git diff HEAD` is empty. ## Verification (on `d352319a`, the final head: the branch, one merge of `origin/main` `0d421041`, which touches none of these packages, and the presence-clause commit) The targeted files below ran on `d352319a`, with `OS_TEST_POSTGRES_URL` set to a private PostgreSQL 16.14: - core: 12 passed - driver-sql, the five JSON-column files: 360 passed, 2 skipped (the MySQL cells) - rest `aggregation-filter-json-column-refusal`: 136 passed, 68 skipped (MySQL) - driver-memory `memory-20444-*` and `memory-21066-*`: 125 passed - objectql `engine-aggregate-filter-json-column-refusal`, `engine-aggregate-filter`, `engine-aggregate-filter-array-membership` and `engine-cascade-delete-multivalue-probe`: 263 passed Full suites, on the pre-merge commits (before the presence clause; that commit touches only the builder, its unit pin and the two wire pins above, which were re-run): - `@objectstack/core` local: 74 files, 2107 passed - `@objectstack/driver-sql`, SQLite: 208 files passed, 11 skipped; 3469 passed, 192 skipped - `@objectstack/rest` local: 255 files, 4834 passed, 301 skipped Typecheck: `core`, `driver-sql` and `rest` all pass, and `tsc --listFiles` confirms the four touched driver-sql test files are in its program. **Gates** (`node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands`, re-derived and re-run on `d352319a`): 65 derived, 65 exit 0, 0 NOT MEASURED. `--ran` with exit codes reconciles to 65 accounted, 0 unrun. - `check:dual-build-cjs-loads` measured this time: 105 require entries across 66 packages load. The first run on `8450f66b` was exit 3 `PREREQUISITE NOT MET`, since a whole-repo build was missing then. - `check:driver-conformance`, before the first edit and after the last commit: 50 covered, 0 DEBT, 0 exempt; dialect axis 8 suites, 0 in the DIALECT ledger. The ledger did not move. **Lint**, narrowed: 1. The population comes from eslint's own config: `--format json` reports every listed file with 0 warnings, so none was ignored. 2. On `d352319a`, the 8 `.ts` files of this diff were linted, with 0 errors and 0 warnings. 3. `eslint.config.mjs` enables no type-aware linting (no `parserOptions.project`), so this diff cannot move the verdict on any untouched file. The repo-wide `pnpm lint` is CI's to run. **Patch round 1 (`89209290`).** `Lint & Repo Gates` was red at `d352319a` on `pnpm check:live-db-isolation`: `packages/rest/src/aggregation-filter-json-column-refusal.test.ts:258 use names the literal "$null"`. The fix is test-only: the presence clause is asserted in two pieces. On `89209290`: - `pnpm check:live-db-isolation`: exit 0, 34 live-server files scanned. - That test file: 136 passed, 68 skipped, with SQLite and live PostgreSQL 16.14 running and MySQL a named skip. - The 65 derived families re-run: 65 exit 0. `check:dual-build-cjs-loads` gave exit 3 PREREQUISITE NOT MET until another family built the missing `dist/`, then exit 0 on re-run. `--ran` reconciles to 65 run, 0 NOT MEASURED. - `check:live-db-isolation` is a declared WIDE-population family, so per-card derivation never names it. ## Siblings that print this sentence (both landed before this PR; seat edit) Both siblings this PR named as later landers merged first, so they are earlier landers now: - #21178 → `862f12c0b` (PR #21208): the `driver-turso` remote face. It prints the builder's output and pins it by equality with the builder, so this rewrite does not flip it. - #20822 group 3b → `e18fea6dc` (PR #21196): new `having-filter.ts` callers of the builder. This branch is behind both. The merge queue rebuilds it onto `main`, so their pins run against this text in the merge group. ## Acceptance notes - **The diagnostic's boundary.** It names the field four times, so its length grows with the name. It is whole on the wire up to 26-character field names (measured over `$in`, `$startsWith` and bare equality). From 27 characters the cut takes the presence clause first, and from 37 the any-of example; the one-member remedy and both names come before both. Field names declare no maximum length, so no text that repeats the name can be bounded. The withheld message, the card's subject, is constant and bounded. - **A gate false positive, on the test's spelling.** `check:live-db-isolation`'s statement-head needle matches the verb USE followed by a quoted operand (`STATEMENT` in `scripts/check-live-db-isolation.mjs`). It read the assertion string `'For no value, use "$null" or "$empty".'` at `packages/rest/src/aggregation-filter-json-column-refusal.test.ts:258` as a MySQL USE naming a database `"$null"`. Fixed in `89209290` by asserting the clause in two pieces; the gate and the product sentence are unchanged. - `packages/objectql/src/engine-cascade-delete-multivalue-probe.test.ts` keeps a test double whose refusal paraphrases the old wording ("is stored as a JSON TEXT column"). It asserts only `code` and `status`, so it is not a pin of this text. Left as is. - `rest`'s own PostgreSQL and MySQL cells of `aggregation-filter-json-column-refusal.test.ts` are still provisioned by no CI job (its header says so). The PostgreSQL wire pin that CI does run is the new `driver-sql` file. - The local PostgreSQL leg ran against a private PostgreSQL 16.14 started for this run on a random port (UTC server, so `driver-sql`'s temporal files were not run against it). It was stopped afterwards. --- _Generated by [Claude Code](https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 1ecb871 commit 682873d

9 files changed

Lines changed: 389 additions & 55 deletions
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
"@objectstack/core": patch
3+
"@objectstack/driver-sql": patch
4+
"@objectstack/driver-memory": patch
5+
"@objectstack/objectql": patch
6+
---
7+
8+
fix(core): the refusal a filter gets for a scalar comparison or text operator on a multi-value or JSON field reads true on every backend that prints it, and reaches a REST caller whole
9+
10+
Clause-②: no
11+
12+
The `INVALID_FILTER` / 400 refusal `driver-sql`'s `where`, the engine's per-aggregation `filter` and `driver-memory` all print (`jsonColumnOperatorRefusalText`) explained itself with `driver-sql`'s storage ("a field this driver stores as a JSON TEXT column") and the two wrong answers SQL used to give. That is untrue on the engine and on `driver-memory`. The message was also 748 characters, and the REST envelope cuts a 4xx message at 499 plus an ellipsis, so callers on SQLite and PostgreSQL read `…Refused rather than compiled because the answ…` and never reached the sentence saying the field and the operator were withheld.
13+
14+
The message now reads, on every backend, in 486 characters: `A constraint in this filter WAS NOT APPLIED: it aims a scalar comparison or text operator at a multi-value or JSON field, which it cannot test for one member.`, then the same `$contains` / `$or` of `$contains` remedy, then `For no value, use "$null" or "$empty".` (a `null` comparand such as `{ f: null }`, `$eq: null` or `$ne: null` is refused too, and `$contains` could not express it), then `The field and the operator are withheld from the message; the full diagnostic is in the server log.` The diagnostic (the server-log text, and what a filter's own author is shown) gives the same reason with the operator named, names the field, and spells the remedy with the field's name. It drops the storage and the SQL history too, and is now whole on the wire for field names up to 26 characters (it was 643 characters or more and always cut).
15+
16+
Code, status, the refused operator set and the `$contains` remedy are unchanged. A client that matched on the old words `JSON TEXT column` or `Refused rather than compiled` should match on `code: "INVALID_FILTER"` instead.

‎packages/core/src/utils/json-column-operator-refusal.test.ts‎

Lines changed: 83 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -4,26 +4,35 @@
44
* [#21007] The JSON-column refusal's operator set and words, moved here from
55
* `driver-sql` so the engine's per-aggregation `filter` refuses with them too.
66
*
7-
* Two pins, both against what `driver-sql` answered BEFORE the move:
7+
* Two pins:
88
*
99
* - **The set** — the 22 spellings `driver-sql`'s module-private
1010
* `JSON_COLUMN_INCOMPATIBLE_OPERATORS` held, member for member, and
1111
* [#21009] the five text operators that joined them since.
12-
* - **The words** — the SHA-256 of each text, captured from `driver-sql`'s
13-
* built `jsonColumnOperatorError` at the commit before the move (`8f784959c`)
14-
* through a real `SqlDriver` over SQLite: the withheld message (one text for
15-
* every operator), and the diagnostic for an operator, for `$between`, and
16-
* for the bare equality spelling, on a column named `members`. A hash rather
17-
* than a second literal copy, so this file is not a third place the sentence
18-
* lives; the length beside each hash says how far a failure moved it.
12+
* - **The words** — the SHA-256 of each text: the withheld message (one text
13+
* for every operator), and the diagnostic for an operator, for `$between`,
14+
* and for the bare equality spelling, on a column named `members`. A hash
15+
* rather than a second literal copy, so this file is not a third place the
16+
* sentence lives; the length beside each hash says how far a failure moved
17+
* it. The hashes were captured from `driver-sql`'s built
18+
* `jsonColumnOperatorError` at the commit before the move (`8f784959c`), and
19+
* [#21067] re-captured from this builder when the words were rewritten.
20+
*
21+
* [#21067] And what the words must SAY, whatever they are: the message reaches
22+
* the REST caller WHOLE — run through the envelope's own bound
23+
* (`truncateClientMessage`, `@objectstack/types`; the 500 itself is
24+
* module-private there, so the function that applies it is what is imported)
25+
* it comes back unchanged — and it carries the remedy and the "withheld"
26+
* sentence. It names no face's storage or wrong answer, since three faces
27+
* print it.
1928
*
2029
* A deliberate change of wording updates the hashes in the PR that makes it —
21-
* and then reaches `where` and the per-aggregation `filter` alike, which is the
22-
* point of the move.
30+
* and then reaches every face alike, which is the point of the move.
2331
*/
2432

2533
import { describe, it, expect } from 'vitest';
2634
import { createHash } from 'node:crypto';
35+
import { truncateClientMessage } from '@objectstack/types';
2736
import { JSON_COLUMN_INCOMPATIBLE_OPERATORS, jsonColumnOperatorRefusalText } from './json-column-operator-refusal.js';
2837

2938
const sha256 = (text: string): string => createHash('sha256').update(text, 'utf8').digest('hex');
@@ -51,13 +60,13 @@ describe('[#21007] JSON_COLUMN_INCOMPATIBLE_OPERATORS', () => {
5160
});
5261
});
5362

54-
describe('[#21007] jsonColumnOperatorRefusalText — byte for byte what driver-sql printed before the move', () => {
55-
const MESSAGE = { sha: 'c6103dd665625ab3a779822fecbe650bd7f67fc6ea38d17d5fd57cf6ae9193ff', length: 748 };
63+
describe('[#21067] jsonColumnOperatorRefusalText — the words, by hash', () => {
64+
const MESSAGE = { sha: 'f0a54a98fb30ae8e201c7be5e9f0d1715649fd34303e176abea131981dfd36f7', length: 486 };
5665

5766
it.each([
58-
['an operator', '$in', false, { sha: '358d5aae39170ab3da198beb39368a3f3476dd057dbd5da5f4295cf244eab67e', length: 648 }],
59-
['$between', '$between', false, { sha: '505c094ac5212ac121cdeb4803787ba036f46177450d542fbccf60a9386924fe', length: 658 }],
60-
['the bare equality spelling', '=', true, { sha: '92bb1728ca012c4cde6deb05be3cb0320e113c8c5ac250ecad5b091f3bb87094', length: 660 }],
67+
['an operator', '$in', false, { sha: 'da62717ac99e4e7470c977fd13ec7e36c32bcecb5c7d45b95d5ba09ded5d81cd', length: 406 }],
68+
['$between', '$between', false, { sha: '4ff8d528fb53e31a73b8a190424fa7e2446bd0bd64ac984bf1f70e4ebe1e83d0', length: 416 }],
69+
['the bare equality spelling', '=', true, { sha: '35e2fd3a5e287ac76ef3fb09922d537935283b930143035b69a07267a762cb5c', length: 418 }],
6170
] as const)('%s', (_name, op, bare, diagnostic) => {
6271
const text = jsonColumnOperatorRefusalText('members', op, bare);
6372
expect({ sha: sha256(text.message), length: text.message.length }).toEqual(MESSAGE);
@@ -83,3 +92,62 @@ describe('[#21007] jsonColumnOperatorRefusalText — byte for byte what driver-s
8392
expect(diagnostic).toContain('{ "secret_col": { "$contains": "a" } }');
8493
});
8594
});
95+
96+
describe('[#21067] jsonColumnOperatorRefusalText — whole on the wire, and true on every face', () => {
97+
/** Every spelling that gets this refusal: each member of the set, and the bare one. */
98+
const SPELLINGS: ReadonlyArray<readonly [op: string, bare: boolean]> = [
99+
...[...JSON_COLUMN_INCOMPATIBLE_OPERATORS].map((op) => [op, false] as const),
100+
['=', true],
101+
];
102+
103+
it('the message passes the REST envelope\'s bound unchanged, for every refused spelling', () => {
104+
for (const [op, bare] of SPELLINGS) {
105+
const { message } = jsonColumnOperatorRefusalText('members', op, bare);
106+
// The bound's own function: a message at or past it comes back cut to 499
107+
// characters plus an ellipsis, so equality here means every word arrives.
108+
expect(truncateClientMessage(message), op).toBe(message);
109+
}
110+
});
111+
112+
it('inside the bound: the remedy for one member, for any-of and for no value, then where the field and the operator went', () => {
113+
const { message } = jsonColumnOperatorRefusalText('members', '$in', false);
114+
expect(message.startsWith('A constraint in this filter WAS NOT APPLIED: ')).toBe(true);
115+
expect(message).toContain(
116+
'Use "$contains" for membership ({ "FIELD": { "$contains": "a" } }), or an $or of "$contains" for any-of ' +
117+
'({ "$or": [{ "FIELD": { "$contains": "a" } }, { "FIELD": { "$contains": "b" } }] }).',
118+
);
119+
// A `null` comparand (`{ f: null }`, `$eq: null`, `$ne: null`) is refused too, and asks
120+
// about presence, which `$contains` cannot spell: the presence operators are named.
121+
expect(message).toContain('For no value, use "$null" or "$empty".');
122+
expect(message.endsWith(
123+
'The field and the operator are withheld from the message; the full diagnostic is in the server log.',
124+
)).toBe(true);
125+
});
126+
127+
it('names the rule, not a face: the field declaration, never a storage form or one backend\'s wrong answer', () => {
128+
for (const [op, bare] of SPELLINGS) {
129+
const { message, diagnostic } = jsonColumnOperatorRefusalText('members', op, bare);
130+
for (const text of [message, diagnostic]) {
131+
expect(text, op).toContain('a scalar comparison or text operator');
132+
expect(text, op).toContain('at a multi-value or JSON field, which it cannot test for one member.');
133+
expect(text, op).toContain('For no value, use "$null" or "$empty".');
134+
expect(text, op).not.toMatch(/this driver|JSON TEXT|serializ|matched nothing|asked to exclude/);
135+
}
136+
// The diagnostic names the operator in the reason too — the bare spelling's as `=`.
137+
expect(diagnostic, op).toContain(`it aims "${op}", a scalar comparison or text operator, at`);
138+
}
139+
});
140+
141+
it('the diagnostic is the message with the names put back: same reason, same remedy with the field in it', () => {
142+
const { message, diagnostic } = jsonColumnOperatorRefusalText('members', '$nin', false);
143+
const reason = message.slice(message.indexOf(': ') + 2, message.indexOf(' Use "$contains"'));
144+
expect(diagnostic.startsWith('Operator "$nin" on field "members" WAS NOT APPLIED: ')).toBe(true);
145+
expect(diagnostic).toContain(reason.replace('a scalar comparison or text operator at', '"$nin", a scalar comparison or text operator, at'));
146+
expect(diagnostic.endsWith(
147+
'{ "$or": [{ "members": { "$contains": "a" } }, { "members": { "$contains": "b" } }] }). For no value, use "$null" or "$empty".',
148+
)).toBe(true);
149+
// An author-marked refusal puts this text on the wire (driver-sql's #8220
150+
// arm): for a field name of an ordinary length it is whole there too.
151+
expect(truncateClientMessage(diagnostic)).toBe(diagnostic);
152+
});
153+
});

‎packages/core/src/utils/json-column-operator-refusal.ts‎

Lines changed: 85 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,30 @@
2626
* carries the #8220 provenance seam, the engine's the ADR-0112 envelope); what
2727
* they share is what the caller reads.
2828
*
29+
* ## The words: true on every face, whole on the wire
30+
*
31+
* [#21067] Three faces print the sentence below — `driver-sql`'s `where`, the
32+
* engine's per-aggregation `filter`, and `driver-memory`'s filter gate (#21066)
33+
* — and each reached its wrong answer by a different route: SQL compared the
34+
* serialization, the engine compared an array in JS, and mingo compared each
35+
* member. So the reason the sentence gives is the one that holds on all three:
36+
* a scalar comparison or text operator met a multi-value or JSON field;
37+
* membership is `$contains`, and no value is `$null` / `$empty`. It names no
38+
* storage form and no backend's wrong answer; those stay in these docblocks,
39+
* out of what a caller reads.
40+
*
41+
* It is also sized for the door it is read through. The REST envelope cuts a
42+
* 4xx message of 500 characters or more to 499 plus an ellipsis, keeping the
43+
* head (`truncateClientMessage`, `@objectstack/types`). The withheld message is
44+
* one constant text, so it is held WHOLE under that bound, the remedy and the
45+
* "withheld" sentence included, by a pin that runs it through that function
46+
* (`json-column-operator-refusal.test.ts`). The diagnostic names the field four
47+
* times, so its length grows with the name. It is whole on the wire for a field
48+
* name of up to 26 characters when an author-marked refusal discloses it. Its
49+
* order (what was refused, why, then the remedy) leaves the presence clause
50+
* and then the any-of example last, so a longer name pushes those out first;
51+
* the any-of example survives up to 36 characters.
52+
*
2953
* ## The other half of the JSON column's contract
3054
*
3155
* `$contains` is the membership spelling on such a column (`FILTER_OPERATORS`'
@@ -108,12 +132,53 @@ export interface JsonColumnOperatorRefusalText {
108132
* What the caller is told. It names neither the field nor the operator: on a
109133
* read scope the predicate is an administrator's, so both are withheld
110134
* (#7929 / #8197), and the sentence says where they went.
135+
*
136+
* [#21067] One constant text, shorter than the REST envelope's 4xx bound, so
137+
* every word of it reaches the caller.
111138
*/
112139
readonly message: string;
113-
/** The full diagnostic — the field and the operator named — for the server log. */
140+
/**
141+
* The full diagnostic — the field and the operator named — for the server log,
142+
* and the wire text when a face discloses it to the filter's own author
143+
* (`driver-sql`'s `'author'` provenance arm, #8220).
144+
*/
114145
readonly diagnostic: string;
115146
}
116147

148+
/**
149+
* [#21067] Why the operator was refused, in words true on every face that
150+
* prints them — see the module docblock. Shared by both texts, so the message
151+
* and the diagnostic cannot come to give two reasons; the diagnostic passes
152+
* `op` and so names the operator, the bare spelling's as `=`.
153+
*/
154+
function refusalReason(op?: string): string {
155+
const operator = op === undefined
156+
? 'a scalar comparison or text operator'
157+
: `"${op}", a scalar comparison or text operator,`;
158+
return `it aims ${operator} at a multi-value or JSON field, which it cannot test for one member.`;
159+
}
160+
161+
/**
162+
* [#21067] The other half of the prescription. The refused set also catches a
163+
* `null` comparand — `{ f: null }`, `$eq: null`, `$ne: null` — whose caller
164+
* asked whether the field has a value, not which member it holds, so
165+
* `$contains` cannot express it. The presence spellings can, and they answer on
166+
* a multi-value or JSON field on every face (they are outside the set):
167+
* `$null` is the literal `= null`, and `$empty` also counts an empty list.
168+
* One constant clause, never a branch on the comparand, so the withheld message
169+
* stays one text.
170+
*/
171+
const PRESENCE_REMEDY = 'For no value, use "$null" or "$empty".';
172+
173+
/** The prescription, spelled with `name` in the field position. */
174+
function containsRemedy(name: string): string {
175+
return (
176+
`Use "$contains" for membership ({ "${name}": { "$contains": "a" } }), or an $or of ` +
177+
`"$contains" for any-of ({ "$or": [{ "${name}": { "$contains": "a" } }, ` +
178+
`{ "${name}": { "$contains": "b" } }] }). ${PRESENCE_REMEDY}`
179+
);
180+
}
181+
117182
/**
118183
* [#7398] The words of the refusal: a scalar-comparison operator met a field
119184
* stored as JSON TEXT, so the comparison can never mean what the caller wrote.
@@ -147,39 +212,33 @@ export interface JsonColumnOperatorRefusalText {
147212
* byte, so `where` and the per-aggregation `filter` print one sentence. The
148213
* caller builds the error: this returns only the text.
149214
*
150-
* [#21009] The text family that joined the set reads these same words,
151-
* unchanged. Its prescription holds as written — membership is `$contains` —
152-
* while the "scalar comparison" wording and the two directions the closing
153-
* sentence names are the equality family's.
215+
* [#21009] The text family that joined the set reads these same words. Its
216+
* prescription holds as written — membership is `$contains`.
217+
*
218+
* [#21067] Reworded once, for two reasons. The words named `driver-sql`'s
219+
* mechanism ("a field this driver stores as a JSON TEXT column", and the two
220+
* wrong answers SQL gave), which is untrue where the engine's per-aggregation
221+
* `filter` and `driver-memory` print them; the reason is now
222+
* {@link refusalReason}'s. And the message ran to 748 characters, so the REST
223+
* envelope cut it at 499, partway through the sentence explaining the refusal,
224+
* and no caller read the sentence saying the field and the operator were
225+
* withheld; it is now 486, with the presence spellings a `null` comparand
226+
* needs (see `PRESENCE_REMEDY`). The mechanism above stays here, where the
227+
* next author reads it.
154228
*/
155229
export function jsonColumnOperatorRefusalText(
156230
field: string,
157231
op: string,
158232
bare: boolean,
159233
): JsonColumnOperatorRefusalText {
160-
const spelling = bare
234+
const subject = bare
161235
? `The bare equality spelling { "${field}": value }`
162-
: `Operator "${op}"`;
163-
const on = bare ? '' : ` on field "${field}"`;
236+
: `Operator "${op}" on field "${field}"`;
164237
return {
165238
message:
166-
`A constraint in this filter WAS NOT APPLIED: it aims a scalar comparison operator at a ` +
167-
`field this driver stores as a JSON TEXT column (e.g. ["a","b"]), and such an operator ` +
168-
`compares that whole serialized text against a single value — it can never equal one ` +
169-
`member. Use "$contains" for membership ({ "FIELD": { "$contains": "a" } }), or an $or of ` +
170-
`"$contains" for any-of ({ "$or": [{ "FIELD": { "$contains": "a" } }, ` +
171-
`{ "FIELD": { "$contains": "b" } }] }). Refused rather than compiled because the answer ` +
172-
`was silently wrong in BOTH directions: $in/$eq matched nothing, while $nin/$ne returned ` +
173-
`the very rows they were asked to exclude. The field and the operator this filter ` +
174-
`used are withheld from the message; the full diagnostic is in the server log.`,
175-
diagnostic:
176-
`${spelling}${on} WAS NOT APPLIED: "${field}" is a multi-value (or otherwise JSON-valued) ` +
177-
`field, stored by this driver as a JSON TEXT column (e.g. ["a","b"]), and "${op}" compares ` +
178-
`that whole serialized text against a single value — it can never equal one member. ` +
179-
`Use "$contains" for membership ({ "${field}": { "$contains": "a" } }), or an $or of ` +
180-
`"$contains" for any-of ({ "$or": [{ "${field}": { "$contains": "a" } }, ` +
181-
`{ "${field}": { "$contains": "b" } }] }). Refused rather than compiled because the answer ` +
182-
`was silently wrong in BOTH directions: $in/$eq matched nothing, while $nin/$ne returned ` +
183-
`the very rows they were asked to exclude.`,
239+
`A constraint in this filter WAS NOT APPLIED: ${refusalReason()} ${containsRemedy('FIELD')} ` +
240+
`The field and the operator are withheld from the message; the full diagnostic is in the ` +
241+
`server log.`,
242+
diagnostic: `${subject} WAS NOT APPLIED: ${refusalReason(op)} ${containsRemedy(field)}`,
184243
};
185244
}

‎packages/drivers/driver-sql/src/sql-driver-json-column-operator-refusal.test.ts‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -120,10 +120,12 @@ function expectJsonColumnRefusal(err: WireBearingError, op: string, field: strin
120120
expect(err.message).toContain(op);
121121
expect(err.message).toContain(field);
122122
// The three things the message owes a caller: that the filter did NOT run,
123-
// why (the column is JSON text), and what to write instead.
123+
// why, and what to write instead. [#21067] The why is the field's
124+
// declaration, not this driver's storage form: the same words are printed by
125+
// faces that store no JSON text.
124126
expect(err.message).toContain('WAS NOT APPLIED');
125-
expect(err.message).toContain('JSON TEXT column');
126-
expect(err.message).toContain('$contains');
127+
expect(err.message).toContain(`it aims "${op}", a scalar comparison or text operator, at a multi-value or JSON field`);
128+
expect(err.message).toContain(`{ "${field}": { "$contains": "a" } }`);
127129
}
128130

129131
/**

‎packages/drivers/driver-sql/src/sql-driver-json-column-refusal-shared-text.test.ts‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,11 @@
77
* move: what it throws IS the shared text, and what it refuses IS the shared
88
* set, on a real `SqlDriver` over SQLite.
99
*
10-
* That the shared text is byte for byte what this driver printed before the move
11-
* is pinned beside the text itself (`@objectstack/core`'s
12-
* `json-column-operator-refusal.test.ts`, hashes captured from this driver at the
13-
* commit before). Together: this driver's refusal did not change by one byte.
10+
* The words themselves are pinned beside the text (`@objectstack/core`'s
11+
* `json-column-operator-refusal.test.ts`, by hash): captured from this driver at
12+
* the commit before the move, and re-captured when [#21067] rewrote them for
13+
* every face and under the REST bound. That they reach a caller whole is
14+
* `sql-driver-json-column-refusal-wire-bound.test.ts`.
1415
*
1516
* The per-operator content of the refusal (code, status, the prescription, every
1617
* lowering face) stays pinned in `sql-driver-json-column-operator-refusal.test.ts`.

0 commit comments

Comments
 (0)