Skip to content

Commit 7ddf396

Browse files
fix(service-analytics)!: the analytics where door runs the shared comparand-TYPE face on the object spelling (#20035) (#20058)
Fixes #20035 Clause-②: no (narrowing) ## What this changes The analytics `where` door (`lowerAnalyticsWhere` in `packages/services/service-analytics/src/strategies/filter-normalizer.ts`) now runs the shared comparand-TYPE face, `normalizeFilterComparandTypes` (`@objectstack/spec/data`), on the object spelling. It runs after the comparand-shape face and before any node is built, the order `parseFilterAST` and the engine seam use. The condition the door lowers is the face's RETURN value, so a bigint within 2^53 is narrowed to its number, copy-on-write, as the engine does it. The governing record is the #7872 ruling (2026-08-12): the accepted comparand types are `string | number | bigint | boolean | null | Date`, and the face 「refuses everything else loudly at the compile face」. The `FilterArray` spelling of this door (inside `parseFilterAST`) and the ObjectQL engine seam already ran it. The object spelling did not. How: - `normalizeWhereComparands` (new, exported) is the door's comparand gate: the shape passes (unchanged, now module-private as `assertWhereComparandShapes`), then the type pass over the whole condition. - The type pass hands each field entry to the face as a one-entry node with the path of the node that holds it, the same hand-over the shape pass makes. The face therefore reports exactly the path it reports on the whole condition, so the object spelling gets the `FilterArray` spelling's bytes. - Nested-relation entries are handed over too. The face leaves a nested-relation object alone as filter structure; this compiler flattens it to a dotted member, so its entries are comparands. `{ acct: { amt: … } }` is judged at `where.acct.amt`, the path the dotted spelling gets. - The shared traversal (`forEachWhereFieldEntry`, now a copy-on-write `mapWhereFieldEntries`) descends a nested relation only when it is a PLAIN object (prototype `Object.prototype` or `null`). That is the type face's own structure test. Before, a `Uint8Array`, `Map` or class instance in the implicit slot was descended as a relation: `{ stage: Uint8Array }` compiled to `stage.0 = 1 AND stage.1 = 2`. Now it is visited as the comparand it is. - The draft preview (`preview-evaluator.ts`) calls the same gate and evaluates the narrowed condition it returns. This closes the card's class-a leg: an `undefined` comparand is refused instead of answering no row. ## Measured first (recorded on this branch as `4e1cd13aac`, before any source change) Base `origin/main` `246314dffe`. The harness was a real sql.js engine (driver-sqlite-wasm) over rows d1 (amt 1, 'won'), d2 (5, 'lost'), d3 (10, 'open'), d4 (NULL, NULL) and d5 (3, ''). The faces were: the native SQL execute; the `/analytics/sql` echo; the ObjectQL engine path over a real `ObjectQL` `engine.aggregate`; `engine.find` on the same object `where` (the engine seam); and the draft preview. | cell | object spelling at base | `FilterArray` spelling | engine seam (object) | |:--|:--|:--|:--| | plain object `$ne` `{stage:{$ne:{a:1}}}` | native bound the JSON text `'{"a":1}'`: EVERY row; echo `DATABASE_ERROR` / 500; engine path 400; preview EVERY row | 400, type face at `where.stage.$ne` | 400 | | plain object `$gt` | native: no row; echo 500; engine path 400; preview: no row | 400 | 400 | | plain object `$eq` | native: no row; echo 500; the engine path received `{stage:{a:1}}` and driver-sql refused it in its own words | no `$eq` spelling (`=` lowers to a nested relation) | 400 | | plain-object `$between` endpoint | the lower bound compared `amt` against the JSON text `'{"a":1}'`: no row; echo 500; the engine path refused it as a `$gte` the author never wrote | 400 at `where.amt.$between[0]` | 400 | | plain object as an `$in` member / `$contains` | refused 400 in this door's #5234 wording | 400, type face | 400 | | `{ $field: 5 }` under `$gt` (not a reference) | native: no row; echo 500; engine path 400 | 400 (a plain object) | 400 | | binary `$eq` / `$in` member | native bound JSON TEXT `'{"0":1,"1":2}'`, not a blob: no row; echo bound the raw buffer: no row; engine path 400 | 400 | 400 | | binary `$ne` | native, echo and preview: EVERY row; engine path 400 | 400 | 400 | | binary implicit `{stage: Uint8Array}` | flattened as a nested relation: native `DATABASE_ERROR` / 500; echo and engine path `INVALID_FIELD` / 400; preview refused operator "0" | 400 at `where.stage` | 400 | | `Map` `$eq` | native bound `'{}'`: no row; echo 500; engine path 400 | no `$eq` spelling | 400 | | `Map` implicit | refused as #5240's zero-operator wrapper; preview EVERY row | 400 (a Map instance) | 400 | | bigint within 2^53 `{amt:{$gt:2n}}` | native, echo and engine path d2, d3, d5 (bound as a bigint); preview d2, d5 (ordered as text: '10' before '2') | narrowed to 2: d2, d3, d5 | d2, d3, d5 | | bigint within 2^53, implicit `5n` / `$in [5n,10n]` | the same rows as the narrowed number on every face | narrowed | the same rows | | bigint beyond 2^53 `$gt` / `$in` member | native and echo bound it: no row; engine path 400; preview d2, d5 / no row | 400 | 400 | | `undefined`, implicit / `$gt` / `$in` member | refused 400 in #6386's wording (`[analytics] comparand at "amt".$gt is undefined`); the preview answered NO row | 400, the type face's `undefined` sentence | 400 | | `{$null: undefined}` / `{$null: {a:1}}` | lowered to `set` (IS NOT NULL): d1, d2, d3, d5 on native, echo and engine path; the preview refused `$null` as unevaluable | no spelling | 400 | | nested relation `{acct:{amt:{$gt:{a:1}}}}` | native: no row; echo 500; engine path 400 | dotted `['acct.amt','>',{a:1}]` 400 | 400 | | under `$or` `{$or:[{id:'d3'},{stage:{$ne:{a:1}}}]}` | native and preview: EVERY row; echo 500 | 400 at `where.$or[1].stage.$ne` | 400 | | controls `{amt:{$gt:2}}`, `{stage:null}` | d2, d3, d5 / d4 on every face and both spellings | the same | the same | After the change, every refusal row is refused on all four faces, before any statement or `engine.aggregate` call. The object message equals the `FilterArray` message and the face's own message, byte for byte. The bigint rows serve the same rows on all four faces, the preview included, and no face binds or receives a bigint. ## Binary: reconciled, not kept as a declared local extra The type face's docblock lets a door keep "its recorded driver-local extras — binary bindables … declared at the use site", and #8186 asked for an explicit keep-or-reconcile call. **The call is reconcile.** The evidence: - **This door never delivered the extra.** `isBindableComparand` (`comparand-shape.ts`) admits binary, but the native path binds every object through `toSqlBindValue`, which JSON-stringifies it. Measured: `{stage:{$in:[Uint8Array]}}` bound `'{"0":1,"1":2}'`, a value no blob column holds. `$ne` then served every row, and the implicit spelling compiled to a dotted member no object has. The engine path and the `FilterArray` spelling refused binary outright. - **No producer relies on it.** JSON has no binary type, so no REST caller can send one. A census of the 258 non-test source files under `packages/**` and `examples/**` that mention analytics found no binary value built into a `where`. The six files that mention a binary type at all are the client's stream and upload code, `rest-server.ts`'s xlsx streaming, the two drivers' own extras, and this package's two filter files. Positive control: 21 of the 258 carry a `where:` object literal, which the scan read. - **Scope.** This PR does not touch the read-scope door (`read-scope-sql.ts`). Since #20018 landed (PR #20046), that door also refuses a binary comparand, with the same face after its own gates, so a binary is now refused at both analytics doors. The `isBindableComparand` predicate still admits it, and the `#8186` predicate-level pins stay green. ## Pins re-judged: predicted 39 red in 9 files, measured 39 red in the same 9 files Before wiring the gate I read every candidate pin and predicted the red set. I then wired the gate and ran the package suite (`86907678c0`): `Tests 39 failed | 2617 passed (2656)`, the identical set. Each pin keeps its verdict, or flips with a note quoting the #7872 ruling. ⛔ None is deleted. - `filter-normalizer-undefined-comparand.test.ts` (21). #6386's twelve positions, its five `$not` rewrite paths, "says ONE thing" and "names the repairs" keep the refusal and now read in the face's sentence and path (`where.d.$gt`). The face runs BEFORE the #5146 rewrite, so the rewrite block now pins that the author's own `$not` is judged. `{$null: undefined}` / `{$exists: undefined}` flip from `set` to refused, because the face judges those comparands as literals. The boolean-domain question (#5347 / #5369 / #6387) stays untouched for accepted values: `{$null: 'false'}` still lowers as before, and a pin now says so. - `comparand-shape-refusal.test.ts` (8). #5234's `$in` / `$nin` object-member and LIKE-family object sentences now read in the face's sentence. The `{ $eq: {…} }` account #5234 "left open" flips to refused, `{ $eq: { $field: 5 } }` included. - `cross-field-reference-refusal.test.ts` (3). A non-string `$field` and a plain object under `$eq` flip to refused; the routing detector half is unchanged. #7693's `$icontains {foo: 1}` keeps its refusal and its sibling-equality check, in the face's words. - `comparand-door-single-source.test.ts` (2). The #8186 matrix: `binary` `whereIn` / `whereEq` and `plain object` `whereEq` move from accept to refused. The predicate and read-scope cells are unchanged. - `filter-normalizer-mixed-wrapper.test.ts` (1). #6444's order: `{d:{$eq: undefined, nested:'x'}}` is still diagnosed as the comparand first, now by the face. - `filter-value-type-fidelity.test.ts`, `filter-refusal-envelope.test.ts`, `filter-normalizer-not-null-safe.test.ts` (1 each). The #6386 wording only. - `where-face-arms-refusal.test.ts` (1). The #20010 CONTROL that pinned "the TYPE face is not run here" now pins the type face's sentence. **Census of consumer pins outside the package.** I searched all 3904 tracked test files under `packages/**` and `examples/**` (outside this package) for #6386's sentence, #5234's two sentences, #6444's wording and the face's sentence. The hits were driver-sql's and driver-turso's own refusals and spec's own tests, none through this door. A second pass covered the 127 test files that reach analytics (`@objectstack/rest` 15, `runtime` 17, `qa/dogfood` 11 and the rest). It looked for an `undefined`, plain-object, binary, `Map` or bigint comparand in a `where`, and found none. So there are **zero hits and no edit outside the package**. `@objectstack/rest` ran in full anyway (below). ## Compile surfaces | surface | verdict | |:--|:--| | `lowerAnalyticsWhere` / `normalizeAnalyticsFilterTree`: caller `where`, dataset scope `filter`, measure `filter`; native execute, `/analytics/sql` echo, ObjectQL engine path | **changed.** Refused before any statement or `engine.aggregate`; a bigint within 2^53 is narrowed. Pinned per face over a real engine. | | `evaluateAnalyticsQueryOverRows` (draft preview) | **changed.** The same gate; it evaluates the narrowed condition. | | `normalizeFilterComparandTypes` (the shared face) | **already compliant.** This PR calls it and does not change it. | | `parseFilterAST` (this door's `FilterArray` spelling) | **already compliant.** Measured at base: every cell refused or narrowed. | | ObjectQL engine seam (`lowerWhereFilterArray`) | **already compliant.** Measured at base with `engine.find`. | | `compileScopedFilterToSql` (service-analytics read scope) | **out of scope.** A separate door and envelope (500); #20018 is in flight in `read-scope-sql.ts`. Not touched. | | driver-sql, driver-turso RemoteTransport, formula, driver-memory, driver-mongodb | **already compliant at the platform doors.** They sit behind `parseFilterAST` or the engine seam, which run the face. Not re-measured here. | | objectql HAVING (`applyHaving` / `matchesHaving`) | **out of scope.** A different door, over aggregated rows. | ## Tests and evidence (head `f4ba18ba9a`) - **New `src/__tests__/where-type-face-refusal.test.ts`, 106 tests.** Every refusal asserts `code` + `status` + the face's opening sentence. - Byte identity for 24 cells: object = `FilterArray` = `normalizeFilterComparandTypes`. Another 9 object-only cells (`$eq`, the `$null` / `$exists` flags, `$not`) are held to the face. The nested relation is held to the dotted spelling. - What is diagnosed first: the shape face before the type face, over the whole condition (`parseFilterAST`'s order); the #19888 equality list before either; the type face before #6444's mixed wrapper and the unsupported-operator refusal. - What the face does not judge keeps the door's sentences: an `undefined` inside an array comparand or under an unknown operator (#6386), and an array or `{ $field }` member / LIKE comparand (#5234 / #7598). - Narrowing: the tree carries the number, the same tree the `FilterArray` spelling compiles. The caller's condition is never edited, and the same reference comes back when nothing narrowed. CONTROL: every accepted comparand compiles as before. - Four faces over a REAL `ObjectQL` engine on sql.js, with 14 refused cells × 4 faces. Each asserts 0 statements and 0 `engine.aggregate` calls. CONTROLS: scalars and `null` serve the same rows on every face. A bigint within 2^53 serves its number's rows on every face, and no face binds or receives a bigint. - Stored datasets through the service doors: the dashboard door and the draft preview, for a scope filter and a measure filter; the registered cube on the ObjectQL door (0 aggregates); and a CONTROL. - **Package run at `f4ba18ba9a`** (`pnpm --filter @objectstack/service-analytics test && … typecheck`, under the verify lock): `Test Files 120 passed (120)`, `Tests 2762 passed (2762)`, `tsc --noEmit` exit 0. `tsc --listFiles` includes all 12 touched TS files. Base: 119 files / 2656 tests. - **`@objectstack/rest` full suite** (its closure built, service-analytics `dist/` carrying `normalizeWhereComparands`): `Test Files 194 passed (194)`, `Tests 3265 passed | 1 skipped (3266)`. - **Ablations.** Each ran from the committed fix through `scripts/ablation-replace.mjs`: anchor x1 to x0, marker `grep -c` 1, blob changed. The restore was proven as blob == HEAD with `git diff HEAD` empty, and each wrapper carried its own `trap` restore. The tests import the source relatively, so no build is on the path. Every count was predicted before the run: - **A, the type walk removed** (`return normalizeWhereComparandTypes(node, path);` became `return node;`). Predicted 138; measured `Tests 138 failed | 2624 passed (2762)`: 99 in the new file and the 39 re-judged pins. Samples: `native execute: expected a refusal, got rows: expected [ 'd1', 'd2', 'd3', 'd4', 'd5' ] to be undefined`, `expected 'accept' to be 'INVALID_FILTER/400'`, `native execute: a bigint was bound: expected true to be false`. - **B, the preview evaluates `query.where` instead of the gate's return.** Predicted 1; measured `1 failed | 2761 passed`: `draft preview: $gt 2n: expected [ 'd2', 'd5' ] to deeply equal [ 'd2', 'd3', 'd5' ]`. - **C, the door discards the gate's return.** Predicted 3; measured `3 failed | 2759 passed`: the tree-narrowing, copy-on-write and bound-bigint cases. - **D, the plain-object check removed from the nested-relation test.** Predicted 12; measured `12 failed | 2750 passed`: the binary / `Map` / class-instance implicit cells. Samples: `expected 'DATABASE_ERROR' to be 'INVALID_FILTER'`, `expected '[analytics] "stage" carries a field c…' to be 'Filter comparand at where.stage is a …'`. - **Gates.** `dispatch-gates --repo objectstack-ai/objectstack --commands` at `f4ba18ba9a` derived 60 families. The union run was 61: those 60 plus `check:dispatcher-error-vocabulary` from the dispatch list, which exited 0. 59 exited 0. **NOT MEASURED (2):** `check:dual-build-cjs-loads` and `check:type-check-debt` exited 3 (PREREQUISITE NOT MET: they need the whole workspace built, which CI does). `--ran`: `60 derived famil(ies) accounted for — 58 run, 2 NOT-MEASURED (2 DERIVED from a recorded exit 3)`, 0 UNRUN. The passes include `check-adr-0087-registration --base origin/main` (it accepted the changeset's disposition), `check-changeset-no-major`, `check:changeset-gate-self-tests`, `check:nul-bytes`, `check:where-matcher`, `check:test-source-alias`, `check:cross-package-test-inputs` and `check:issue-citations`. `check-issue-citations` also ran in its board-probing mode with `GITHUB_TOKEN`: exit 0, 43 citations resolve. - **Lint, narrowed to the change.** `eslint --no-inline-config --format json` over the 12 touched TS files: 12 files, 0 errors, 0 warnings. `eslint --print-config` resolves a config for each, which is the population read from eslint's own config. `eslint.config.mjs` never enables type-aware linting (its note at line 327), so this diff cannot move the verdict on an untouched file. ## Changeset and the ADR-0087 disposition `.changeset/20035-analytics-where-type-face.md`: `@objectstack/service-analytics` minor, a `!` headline, `Clause-②: no (narrowing)`, and a **BREAKING** paragraph with a before / now table for every accept-cell that now refuses. The marker is `not-required (no-migration-prescription)`, and it is the honest one. **#7872's transition is not on the ledger.** No entry in `packages/spec/src/migrations/registry.ts` names the type face or its accepted set, and #7872's own changeset declared no breaking change. So `already-registered` would name an id that does not cover this change, which ADR-0087's own addendum calls out. `registered` needs a ledger entry in `packages/spec`, which this card excludes. And no ledger entry could carry this change: a stored plain-object comparand has no accepted comparand to be rewritten to, and the other refused values cannot be stored as JSON at all. The table records each cell's verdict before and now and prescribes no rewrite. ## Deviations from the dispatch, stated 1. **"FROM → TO table" is spelled "before / now".** The gate reads a literal `FROM` / `TO` label as a migration prescription, which would refuse the only honest disposition above. The table is the transition in substance (the verdict each accept-cell had and has), and its columns say so. The seat may prefer registering a ledger entry in a `packages/spec` follow-up instead; that is out of this card's surface. 2. **The traversal's nested-relation test changed** (plain objects only). It was needed for byte identity: without it, `{ stage: Uint8Array }` and `{ stage: new Map() }` never reach the type face. Ablation D pins it. 3. **`assertWhereComparandShapes` is now module-private.** Its only other caller, the preview, now calls `normalizeWhereComparands`. The package entry never re-exported it. 4. **`origin/main` merged in (round 2), as a merge commit (`0164eba2b7`), after #20046 (#20018) landed as `980bc05e5b`.** Two test files conflicted; both sets of re-judgements hold on the merged tree (see "Round 2" below). `origin/main` has since moved one commit, to `3557f85fa5` (a driver-turso README). It touches none of these files, so it was not merged again. ## Round 2: merged with `origin/main` after #20046 (head `0164eba2b7`) - **Conflict 1, `comparand-door-single-source.test.ts`.** The matrix keeps both sets of cells: - #20018's scope cells: `null` `scopeIn`, and binary and plain object `scopeIn` / `scopeEq`, all `READ_SCOPE_COMPILE_FAILED/500`; - this PR's `where` cells: binary `whereIn` / `whereEq` and plain object `whereEq`, all `INVALID_FILTER/400`. Each moved cell's comment names the change that moved it. Every other cell stays `accept`, and #20046's per-cell `moved` map for the six accepted types is kept as is. The header's binary bullet now says the extra is reachable at neither door. #20046's "the `where` door keeps the extra" is gone, because this PR makes it false. - **Conflict 2, `cross-field-reference-refusal.test.ts`.** The non-string `$field` case now holds on both doors, each in its own envelope. The `where` door refuses it as a plain object at `where.amount.$gt` with `INVALID_FILTER` / 400 (this PR). The read-scope lowering refuses it with `READ_SCOPE_COMPILE_FAILED` / 500, in the type face's words and not the field-reference gate's (#20018). The routing detector still returns `null` for it. - **Sentences one PR made false, re-read on the merged tree.** Each is now scoped: - `comparand-shape-refusal.test.ts`'s "one sentence, two envelopes" note: for a plain object, `Map` or binary, the `where` door now answers in the type face's sentence, while the read scope keeps this package's sentence because its own gates run first; - the binary predicate note in the matrix file; - the `comparand-shape.ts` binary-extra docblock (above); - this PR's changeset and normalizer lines about the read-scope binary admission. The file list against `main` is 14 files; `comparand-shape.ts` (docblock only) is the one added. - **Evidence on the merged head `0164eba2b7`.** - Dependency closure rebuilt (14 tasks, 0 cached). - The service-analytics suite: `Test Files 121 passed (121)`, `Tests 2795 passed (2795)`. `tsc --noEmit` exit 0. - Ablation A again (the type walk removed): predicted 138, measured `Tests 138 failed | 2657 passed (2795)`, with the same per-file split (99 new-file, 21 / 8 / 3 / 2 / 1×5 re-judged). Restore: blob == HEAD (`6de15db882ed`), `git diff HEAD` empty. - Gates re-derived: 60 families, identical to round 1. The union of 61 gave 59 exit 0 and 2 exit 3, the same two NOT MEASURED (`check:dual-build-cjs-loads`, `check:type-check-debt`). `--ran`: `60 derived famil(ies) accounted for — 58 run, 2 NOT-MEASURED`. - `check-adr-0087-registration --base origin/main` exit 0 (same disposition). `check-issue-citations` board-probing exit 0 (48 resolve). - eslint over the 13 touched TS files: 0 errors, 0 warnings. ## Round 3: stale docblocks carried (head `8c78550a75`) - One commit, `comparand-shape.ts` docblocks only: 53 comment lines added, 6 removed, with no code or test line changed. So there is no ablation this round: it moves no behaviour and no assertion. - Evidence at `8c78550a75`: - the service-analytics suite: `Test Files 121 passed (121)`, `Tests 2795 passed (2795)`; `tsc --noEmit` exit 0; eslint on the file: 0 errors, 0 warnings; - `dispatch-gates --commands`: the same 60 families; the union of 61 gave 59 exit 0 and 2 exit 3 (the same NOT MEASURED pair); `--ran`: `60 derived famil(ies) accounted for — 58 run, 2 NOT-MEASURED`; - `check-issue-citations` board-probing: exit 0, 72 resolve. - `origin/main` is at `b76aad5f6f` at the time of writing. None of its commits since the merge base `980bc05e5b` touches `service-analytics` or the shared filter faces, so it was not merged. ## Round 4: the unbindable list-member refusal no longer offers "(or a binary value)" (head `cf1563d6f8`) - **What changed.** This is the seat's round-3 answer (option A, on this PR). It is one commit, `cf1563d6f8`: 3 files, +28 / −10. - `comparand-shape.ts`: `unbindableListMemberMessage`'s runtime text no longer offers "(or a binary value)" as a repair. It now names only the accepted set. Its code, status and verdict do not change. - Two things are unchanged. `isBindableComparand`'s binary arm stays as driver-sql's mirror. driver-sql's own copy of the sentence (`sql-driver.ts`) also stays, because that driver does bind a binary. - `comparand-door-single-source.test.ts`: the #8186 pin that asserted `toContain('(or a binary value)')` is re-judged, flipped, with a note. The message must not contain `binary`, and the accepted-set sentence must be followed directly by "Refusing rather than binding it". - The changeset gains one line that says so. - **Census of pins on the old wording.** This is a `git grep -F` over all 9482 tracked files at `cf1563d6f8`, so every package is covered, `packages/rest` included. The tree has no snapshot files. - "or a binary value": 4 hits. One is `sql-driver.ts:3007`, driver-sql's own copy, which is expected and stays. The other 3 are prose that names the removed words: the changeset line, the re-judged pin's comment and the `comparand-shape.ts` docblock. None of them is an assertion. - The wider "binary value" gives 7 hits. The 3 extra hits are another driver-sql message, a driver-sql comment and the docblock's quote of the old hand copy. None of them is an assertion. - "in its own right — use": 2 hits, both runtime strings: this message (`comparand-shape.ts:700`) and driver-sql's copy (`sql-driver.ts:3006`). - "Every member of an $in/$nin/$between list is a comparand": 1 hit, this message itself. - "Refusing rather than binding it": 5 hits. They are this message, the new pin (`comparand-door-single-source.test.ts:325`), driver-sql's copy, and two other messages (`comparand-shape.ts:595`, driver-turso `remote-transport.ts:4141`). - `unbindableListMemberMessage`: 9 hits. They are the definition, its two door callers (`filter-normalizer.ts:680`, `read-scope-sql.ts:1009`), their imports and the matrix test's three calls. The other two calls assert only the accepted-set sentence and `bigint`, and neither moved. - Control: "cannot be bound as a SQL", a string known to be present, gives 10 hits. They include this message (`comparand-shape.ts:698`) and three door pins that still hold on it (`where-type-face-refusal.test.ts:229-230`, `cross-field-reference-refusal.test.ts:288`). - **Result: the push missed no pin on the old wording.** - **Ablation of the new negative pin.** It ran from the committed head through `scripts/ablation-replace.mjs`, which put "(or a binary value)" back into the runtime string. - On disk: anchor x1 → x0, injected text x0 → x1, blob `4cc83d7bf44c` → `77c17ca505c8`. - Predicted: 1 red in the whole package suite. Measured: `Test Files 1 failed | 120 passed (121)`, `Tests 1 failed | 2794 passed (2795)`. - The red was `comparand-door-single-source.test.ts` › "binary stays a package-local extra the door does not admit": `AssertionError: expected '"$in" on "status" has a value at inde…' not to contain 'binary'`. - Restore: blob == HEAD (`4cc83d7bf44c`), `git diff HEAD` empty, `git status --porcelain` empty. The wrapper carried its own `trap` restore. - Vitest stops a case at its first failed assertion, so the pin's second assertion (the accepted-set sentence followed directly by "Refusing rather than binding it") was not measured on its own here. - **Evidence at `cf1563d6f8`:** - the dependency closure: 14 of 14 turbo tasks cached; - the service-analytics suite: `Test Files 121 passed (121)`, `Tests 2795 passed (2795)`; `tsc --noEmit` exit 0; - eslint `--no-inline-config --format json` on the two touched TS files: 2 files, 0 errors, 0 warnings; - `dispatch-gates --commands`: the same 60 families as rounds 1 to 3. 58 exited 0. 2 exited 3 and are NOT MEASURED (`check:dual-build-cjs-loads`, `check:type-check-debt`), because they need the whole workspace built. `--ran`: `60 derived famil(ies) accounted for — 58 run, 2 NOT-MEASURED (2 DERIVED from a recorded exit 3)`; - `check-issue-citations` in board-probing mode: exit 0, 74 citations resolve. - `origin/main` is at `66960564d9`, 5 commits past the merge base `980bc05e5b`. None of those commits touches `service-analytics` or the shared comparand-type face. This round changes no code, so `origin/main` was not merged. ## Acceptance notes - **Stale `comparand-shape.ts` docblocks: fixed.** Round 2 corrected #20046's binary-extra note so that it says NEITHER door serves the extra. Round 3 (`8c78550a75`, docblocks only) fixes the rest of the file's pre-#20035 `where`-door account: - the `undefined` table's `where` row now names the shared comparand-TYPE face as the first refusal (same verdict and envelope), with #6386's gate kept for the two positions the face steps around; - the header's envelope clause, the non-string `$field` note, the two shared refusal sentences (`unrenderableTextComparandMessage`, `unbindableListMemberMessage`) and the two `$between`-endpoint notes are each scoped to the door they are true of. - **`unbindableListMemberMessage`'s "(or a binary value)": fixed in round 4 (`cf1563d6f8`).** Round 3 left it because that round changed docblocks only. Neither door accepts a binary any more (#20018 at the read scope, this PR at the `where` door), so the runtime text now names only the accepted set. The pin in `comparand-door-single-source.test.ts` is re-judged. driver-sql's own copy keeps the words, because that driver binds a binary. - **`$ne` with a list** is still the face's to judge (#19886 stage 2). The type face steps around arrays, so this change neither moves nor pins it. - **Residual door-local sentences.** An `undefined` inside an array comparand (`{d:{$contains:['a', undefined]}}`) or under an unknown operator still reads in #6386's sentence, and an array or `{ $field }` member / LIKE comparand in #5234's. Those are positions the face does not judge, and both spellings agree on them. - #20010 remains open for its `$ne` arm, which is not this card's. --- _Generated by [Claude Code](https://claude.ai/code/session_01Evb5jFDZGKQE9KG4jbMfMF)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 1df29df commit 7ddf396

14 files changed

Lines changed: 1262 additions & 210 deletions
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
---
2+
"@objectstack/service-analytics": minor
3+
---
4+
5+
fix(service-analytics)!: the analytics `where` door runs the shared comparand-TYPE face on the object spelling, so a plain-object, binary, `Map`, class-instance, oversized-bigint or `undefined` comparand is refused like the `FilterArray` spelling and the engine refuse it (#20035)
6+
7+
Clause-②: no (narrowing)
8+
9+
<!-- adr-0087: not-required (no-migration-prescription) the #7872 comparand-type transition is not on the ADR-0087 ledger: no entry names normalizeFilterComparandTypes or its accepted set, and #7872's own changeset declared no breaking change. This change adds no new transition either; it brings the analytics object-form `where`, its dataset and measure filters and the draft-data preview under the refusal that face already makes at every other door. No ledger entry could carry it: a plain object in a comparand slot has no accepted comparand it can be mechanically rewritten to, and a binary, a Map, a class instance, a bigint beyond 2^53 and undefined cannot be stored as JSON metadata at all. Which accepted comparand the author meant is an authoring decision. A bigint within 2^53 is narrowed to its number and answers the same rows. The table below records the verdict each cell had and has; it prescribes no rewrite. -->
10+
11+
**BREAKING**: this narrows what the analytics faces of `@objectstack/service-analytics` accept. A caller `where`, a dataset `filter` or a measure `filter` in the object spelling that carries one of the comparands below compiled before this change, at any depth under `$and` / `$or` / `$not` or inside a nested relation. It is now refused with `INVALID_FILTER` / 400, in the shared face's own message, path and prescription, before any SQL statement runs or any `engine.aggregate` call is made. It ships as `minor` under the launch-window convention for accept-set narrowings.
12+
13+
| comparand in an object-form `where` | before (native SQL execute) | now, on every face |
14+
|:--|:--|:--|
15+
| a plain object under a scalar operator: `{ stage: { $ne: { a: 1 } } }`, `$gt`, `$eq`, the LIKE family | bound as the JSON text `'{"a":1}'`; `$ne` served EVERY row, the others matched nothing; the `/analytics/sql` echo answered `DATABASE_ERROR` / 500 | refused 400: "Filter comparand at where.stage.$ne is a plain object …" |
16+
| a plain object as a `$between` endpoint or an `$in` / `$nin` member | the endpoint was compared as JSON text; the member was refused in this package's own sentence | refused 400, in the face's sentence |
17+
| a `{ $field: 5 }` object (a non-string `$field`) under an ordering operator | bound as JSON text (it is not a field reference) | refused 400 as a plain object |
18+
| a binary (`Uint8Array` / `Buffer`) under `$eq` / `$ne` or as an `$in` member | bound as the JSON text `'{"0":1,…}'`, never as a blob; `$ne` served every row | refused 400 |
19+
| a binary, a `Map` or a class instance as the implicit comparand `{ stage: VALUE }` | read as a NESTED RELATION (`stage.0 = 1 AND …`, a 500 on the native path) or refused as a zero-operator wrapper | refused 400 as the value it is |
20+
| a `bigint` beyond ±2^53 | bound as-is, answering no row | refused 400 |
21+
| `undefined`, implicit or under any operator, including `$null` / `$exists` | refused 400 in this package's own sentence, except `{ $null: undefined }`, which compiled to IS NOT NULL; the draft preview answered no row | refused 400, in the face's sentence |
22+
23+
The shared comparand-type face in `@objectstack/spec` (`normalizeFilterComparandTypes`) is the #7872 door: its accepted comparand types are `string | number | bigint | boolean | null | Date`, and it refuses everything else loudly at the compile face (maintainer ruling, 2026-08-12). `parseFilterAST` runs it on everything it returns and the ObjectQL engine's seam on every object-form `where`, so the `FilterArray` spelling of this door and the engine path already refused each row above. The object spelling now meets it too, after the comparand-shape face and before any node is built, the order `parseFilterAST` uses. Both spellings of one condition get the same refusal, byte for byte, on the native SQL execute path, the `/analytics/sql` echo, the ObjectQL engine path and the draft-data preview.
24+
25+
A `bigint` within ±2^53 is not refused: the face narrows it to its number, and the condition every face lowers is the narrowed one. The rows do not change on the published faces. The draft-data preview used to order a bigint as text (`{ amt: { $gt: 2n } }` lost the row holding 10); it now evaluates the narrowed number and charts the published rows.
26+
27+
Binary comparands are reconciled with the face rather than kept as a declared local extra of this package. Measured before this change, the `where` door never compared a binary as a blob on any face: the native path bound it as JSON text, the engine path and the `FilterArray` spelling refused it, and no producer can send one over JSON. This change does not touch the read-scope door, which refuses a binary comparand too since the read-scope lowering began running the same face (#20018), so a binary is now refused at both analytics doors.
28+
29+
Refusals this door already gave in its own words now read in the face's words, the same words the `FilterArray` spelling gets: an `undefined` comparand, and a plain-object `$in` / `$nin` member or LIKE-family comparand. Their verdict, code and status are unchanged. The positions the face does not judge keep this package's sentences: an array or a `{ $field }` reference as a list member or a LIKE comparand, and an `undefined` inside an array comparand or under an operator outside the vocabulary.
30+
31+
Who is affected: nothing in this repository's examples, seeds, docs or package sources authors any of the refused comparands. A text scan over 4013 non-test files found none, and it did find the shape in a code comment written for this change. Stored datasets, dashboard widget filters and report runtime filters in a deployment were NOT measured. Of the refused values, only the plain object can be stored as JSON.
32+
33+
The refusal both analytics doors give for an unbindable `$in` / `$nin` / `$between` member no longer offers "(or a binary value)" as a repair, because neither door accepts a binary any more. It now names only the accepted set; its code, status and verdict are unchanged.
34+
35+
Not changed: every string, number, boolean, `null` and `Date` comparand; a `{ $field }` reference in an ordering slot (served on the engine path); nested relations and dotted members; `$ne` with a list, which the shared face does not judge yet.

‎packages/services/service-analytics/src/__tests__/comparand-door-single-source.test.ts‎

Lines changed: 59 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -38,16 +38,23 @@
3838
* - **`undefined` is REFUSED by both doors** — `assertDefinedComparands`
3939
* (#6386, on #6050's ruling B) at the `where` door and #6125's upstream
4040
* refusal at the read scope, both of which fire before a predicate is
41-
* consulted. The `undefined` arms inside the predicates, and `comparand()`'s
41+
* consulted. [#20035] At the `where` door the shared comparand-type face
42+
* now fires first, in its own words; same verdict, same envelope. The `undefined` arms inside the predicates, and `comparand()`'s
4243
* normalise-to-`null`, survive as deliberately-kept dead arms (#5526's call
4344
* to reopen, not this card's). The `undefined` row below is what makes that
4445
* checkable instead of asserted — it pins REFUSAL, at both doors.
4546
* - **binary** binds but has no faithful text rendering, so it is accepted in
4647
* a bind position and refused by the LIKE family. That asymmetry is the
4748
* reason the package carries two predicates rather than one with a flag.
48-
* [#20018] On the READ-SCOPE door it is now refused in every position: the
49+
* At the PREDICATE level it still is, and at neither DOOR is it reachable
50+
* as an accepted comparand any more: both now run the shared comparand-type
51+
* face, which does not admit binary.
52+
* [#20018] On the READ-SCOPE door it is refused in every position: the
4953
* lowering runs the shared comparand-type face after its own gates, as the
50-
* ObjectQL execute face does. The predicate's own answer is unchanged.
54+
* ObjectQL execute face does.
55+
* [#20035] On the `where` DOOR it is refused in every position too: the
56+
* door runs the same face before any predicate is asked (see the binary
57+
* row's note for the evidence the keep-or-reconcile call rests on).
5158
*
5259
* @see comparand-shape.ts — the predicates and the messages this pins
5360
* @see https://github.com/objectstack-ai/objectstack/issues/8186
@@ -150,31 +157,50 @@ const MATRIX: readonly Row[] = [
150157
// ── the two package-local predicate admissions, neither reachable ──────────
151158
// `undefined`: the predicates admit it; BOTH doors refuse it upstream (#6386
152159
// at the `where` door, #6125 at the read scope) before a predicate is asked.
160+
// [#20035] At the `where` door the shared comparand-type face now answers
161+
// first, in its own words; the verdict and envelope below are unchanged.
153162
{ label: 'undefined', value: undefined,
154163
bindable: true, renderable: true,
155164
whereLike: REFUSED_WHERE, whereIn: REFUSED_WHERE, whereEq: REFUSED_WHERE,
156165
scopeLike: REFUSED_SCOPE, scopeIn: REFUSED_SCOPE, scopeEq: REFUSED_SCOPE },
157166
// binary: binds, does not render — accepted where it binds, refused by LIKE.
158-
// [#20018] Two cells moved AFTER the #8186 measurement, on purpose: the
159-
// read-scope lowering now runs the shared comparand-type face after its own
160-
// gates, and that face does not admit binary (the predicate above it still
161-
// does — the `where` door keeps the extra, the read scope no longer reaches
162-
// it). The ObjectQL execute face already refused a binary read-scope
167+
// Four cells of this row moved AFTER the #8186 measurement, each named with
168+
// the change that moved it; `whereLike` / `scopeLike` were refused from the
169+
// start, and the predicate columns never moved.
170+
// [#20018] `scopeIn` / `scopeEq`: the read-scope lowering now runs the shared
171+
// comparand-type face after its own gates, and that face does not admit
172+
// binary. The ObjectQL execute face already refused a binary read-scope
163173
// comparand.
174+
// [#20035] `whereIn` / `whereEq`: RE-JUDGED from accept to refused, on
175+
// purpose and with the evidence stated, because the type face's docblock
176+
// lets a door keep binary only as a DECLARED local extra and #8186 asked for
177+
// an explicit keep-or-reconcile call. Reconciled: the maintainer's ruling on
178+
// #7872 (2026-08-12) 「refuses everything else loudly at the compile face」,
179+
// and this door never delivered the extra — measured on a real engine before
180+
// the change, the native path bound the buffer as the JSON TEXT
181+
// '{"0":1,"1":2}' (no row; `$ne` served every row), the engine path and the
182+
// FilterArray spelling refused it, and no producer can send one over JSON.
183+
// So the extra is now reachable from neither door; the predicate above still
184+
// admits it.
164185
{ label: 'binary', value: new Uint8Array([1, 2]),
165186
bindable: true, renderable: false,
166-
whereLike: REFUSED_WHERE, whereIn: OK, whereEq: OK,
187+
whereLike: REFUSED_WHERE, whereIn: REFUSED_WHERE, whereEq: REFUSED_WHERE,
167188
scopeLike: REFUSED_SCOPE, scopeIn: REFUSED_SCOPE, scopeEq: REFUSED_SCOPE },
168189

169190
// ── shapes outside the fence ──────────────────────────────────────────────
170-
// `$eq` accepts them on purpose: #5234 left the `{$eq: {…}}` account alone.
171-
// [#20018] …on the `where` door. On the read-scope lowering the shared
172-
// comparand-type face now closes that account (#7872's set), the answer the
173-
// ObjectQL execute face already gave: a plain object bound as a scalar was
174-
// refused by the database at execution, not by this package.
191+
// `$eq` accepted them on purpose: #5234 left the `{$eq: {…}}` account alone.
192+
// Both `$eq` cells have moved since, each by the #7872 set (「refuses
193+
// everything else loudly at the compile face」) reaching one more door:
194+
// [#20018] `scopeEq`: the read-scope lowering now runs the shared
195+
// comparand-type face, the answer the ObjectQL execute face already gave (a
196+
// plain object bound as a scalar was refused by the database at execution,
197+
// not by this package).
198+
// [#20035] `whereEq`: RE-JUDGED to refused — the `where` door now runs the
199+
// same face on the object spelling, as `parseFilterAST` and the engine seam
200+
// always did.
175201
{ label: 'plain object', value: { foo: 1 },
176202
bindable: false, renderable: false,
177-
whereLike: REFUSED_WHERE, whereIn: REFUSED_WHERE, whereEq: OK,
203+
whereLike: REFUSED_WHERE, whereIn: REFUSED_WHERE, whereEq: REFUSED_WHERE,
178204
scopeLike: REFUSED_SCOPE, scopeIn: REFUSED_SCOPE, scopeEq: REFUSED_SCOPE },
179205
// [#19975] One cell of this row moved AFTER the #8186 measurement, on
180206
// purpose: ruling 乙 (#19757) refuses a list in the equality slot, and the
@@ -276,11 +302,27 @@ describe('[#8186] the comparand matrix is unchanged by the door reconciliation',
276302
});
277303

278304
it('binary stays a package-local extra the door does not admit', () => {
305+
// [#20035] At the predicate level, which is what this case reads. The
306+
// `where` door refuses binary before this predicate is asked (the matrix
307+
// row above); the read-scope door still asks it and then refuses the
308+
// value with the shared type face after its own gates (#20018), so the
309+
// extra is admitted here and served at neither door.
279310
const buf = new Uint8Array([1, 2]);
280311
expect(isAcceptedFilterComparand(buf)).toBe(false);
281312
expect(isBindableComparand(buf)).toBe(true);
282-
expect(unbindableListMemberMessage('$in', 'status', { foo: 1 }, 0))
283-
.toContain('(or a binary value)');
313+
// [#20035] RE-JUDGED — FLIPPED. This asserted the refusal sentence offered
314+
// "(or a binary value)" as a repair, the parenthetical #8186 kept beside
315+
// the door's sentence. Neither door accepts a binary any more: the
316+
// read-scope lowering refuses it with the shared comparand-type face
317+
// (#20018) and so does the `where` door (#20035, the #7872 ruling:
318+
// 「refuses everything else loudly at the compile face」). A refusal that
319+
// still prescribed it sent the author to a value the same door refuses,
320+
// so the sentence now names the accepted set and nothing more. The
321+
// predicate's binary arm above is unchanged — it is `driver-sql`'s
322+
// mirror, and `driver-sql`, which does bind a binary, keeps its own copy.
323+
const message = unbindableListMemberMessage('$in', 'status', { foo: 1 }, 0);
324+
expect(message).not.toContain('binary');
325+
expect(message).toContain(`use ${ACCEPTED_FILTER_COMPARAND_TYPES_SENTENCE}. Refusing rather than binding it`);
284326
});
285327
});
286328

0 commit comments

Comments
 (0)