Skip to content

Commit c35436c

Browse files
fix(objectql)!: a per-aggregation filter and having refuse a non-boolean $exists / $null with INVALID_FILTER / 400, in the drivers' words (#20981) (#21157)
Fixes #20981 Clause-②: no (narrowing) The engine's in-process evaluator for a per-aggregation `filter` and a `having` now refuses a non-boolean `$exists` / `$null` with `INVALID_FILTER` / 400, in the words every driver's `where` refuses it in. It is judged once, before any driver is asked for a row, and per row as the floor, beside the existing `$empty` gate. `true` / `false` answer exactly as before. ## What was wrong Measured on `origin/main` `7a606a9a3` through `engine.aggregate` on driver-memory AND driver-sql (better-sqlite3). The object has a text field `name`: row `a` holds `"won"` (amount 10), `b` holds null (amount 1), and `c` has no value (amount 100). The two drivers answered identically, because the engine evaluates both clauses itself: | clause · comparand | `$exists` | `$null` | |:--|:--|:--| | filter · `"yes"` / `1` / `"false"` | sum 10 (the valued row) | sum 111 (every row) | | filter · `0` / `null` | sum 101 (the no-value rows) | sum 111 (every row) | | having · `"yes"` / `1` / `"false"` | the `won` group | both groups | | having · `0` / `null` | the null group | both groups | | filter / having · `true` | 10 / `won` | 101 / null group | | filter / having · `false` | 101 / null group | 10 / `won` | `checkCondition` read `$exists` as `!!target`, and its `$null` arm tested only `=== true` / `=== false`, so any other value constrained nothing. After this PR, every non-boolean cell above is `INVALID_FILTER` / 400 on both drivers, and the `true` / `false` rows are unchanged (re-measured with the same scratch harness at the head). **Reach.** `POST /api/v1/data/:object/query` never reached the defect. The route parses the request against the spec's query schema first, and that parse already refuses a non-boolean flag in `where`, in `aggregations[i].filter` and in `having` with 400 `VALIDATION_FAILED`. This was measured on the same base and is unchanged here. The engine refusal is the floor for every caller that reaches `engine.aggregate` without that parse: server-side code, a flow or hook, the analytics bridge that lowers a measure's `filter` into an aggregation filter, and a host calling `applyInMemoryAggregation` directly. ## The change (`packages/objectql/src/having-filter.ts`) - `nonBooleanFlagComparandError(op, field, value, path)` sits beside `emptyFlagComparandError`. `assertConditionIsEvaluable` calls it next to the `$empty` gate, so `assertHavingIsEvaluable` and `assertAggregationFilterIsEvaluable` (the gates `engine.aggregate` calls before any driver read) refuse at every depth, a branch a `$or` would short-circuit included. `checkCondition` raises it above the no-value exit as the per-row floor, as `$empty` does. - The two arms now read the boolean itself: `$exists` is "has a value" and `$null` is its mirror. No truthiness reading and no drop. - `engine.ts` and `in-memory-aggregation.ts` are not touched: both already route through the walker. - Precedence: a `{ $field }` in a flag's slot is now refused as a non-boolean first, as `$empty`'s gate already does, rather than by the reference-position refusal. The code and status are the same (`INVALID_FILTER` / 400), and the two existing reference rows that assert only the path stay green, with a note added. A plain object or `undefined` is still refused first by the comparand-type door, in its own words. An array in the flag's slot reached the old `!!target` read and is now refused. ## The words, and where they live The text is `driver-sql`'s own `nonBooleanExistsComparandError` / `nonBooleanNullComparandError` diagnostic, which is also the text `driver-memory` and `driver-mongodb` put on the wire. Its "this driver" clause is re-aimed at `driver-sql`, as `driver-memory`'s copy re-aims it. No new wording. The text has **no importable home**. Measured: `@objectstack/core` exports nothing for it. The spec's save door spells its own message (`nonBooleanFlagComparandMessage`, not exported, with different words). Every face spells its own copy: driver-sql, driver-memory, driver-mongodb, Turso's remote transport, service-analytics twice, and the spec save door. This package cannot depend on a driver. So this is a **declared verbatim copy**, not a silent one. The drivers' `describeFilterOperand` / `safeShapePreview` rendering is copied too (`describeFlagOperand`), and the copy is held to its source by `packages/rest`'s new cell. That cell reads the `where` twin's withheld diagnostic off the error `driver-sql` throws (`withheldFilterDiagnosticOf`) and requires the engine's message to equal it under exactly two edits: the location re-rooted at the clause, and "this driver" re-aimed at driver-sql. A change to either side breaks that equality. Whether the sentence should get a shared `@objectstack/core` home (the shape #21007 gave its JSON-column sentence) is raised to the PM as an open question in the report on #20981, and is not decided here. Disclosure: `driver-sql` withholds the field and the value from a `where` refusal, because a `where` can carry a merged read scope. A per-aggregation `filter` and a `having` never do, so this message names both, as this face's `$empty` and `$icontains` refusals already do. ## Pins - `packages/objectql/src/engine-aggregate-flag-comparand-refusal.test.ts` (new, 39 tests). Through `engine.aggregate`, over a driver of each `having` path's shape: `rows`, with no `aggregate()`, the face that evaluates `aggregations[i].filter`; and `native`, where the driver aggregates and the engine applies `having`. Each of `"yes"`, `1`, `"false"`, `0`, `null` (plus a list) under both flags is refused in the filter and in `having`, on an empty and a populated table, with no driver read. Each case asserts `code` `INVALID_FILTER`, `status` 400, the drivers' first three sentences (operator, field, the rendered comparand, the position, the declaration) and the reason. Four more positions are covered: `$and`, a `$or` branch after one that holds, `$not`, and beside a boolean sibling flag. A `{ $field }` in the slot is refused as a non-boolean. The `true` / `false` control gives the sums and groups in the table above, on both paths. The published `applyInMemoryAggregation(rows, ast, tz, fields)`, plus `applyHaving` and `matchesAggregationFilter`, refuse a row that reaches the flag, with and without a value. - `packages/rest/src/aggregation-flag-comparand-refusal.test.ts` (new, 15 tests). This is the real `SqlDriver` (SQLite) cell, the declared `domain:cli` test-only touch. The refusal cases run on `engine.aggregate`, beside the `where` twin's diagnostic (the word parity above). One case records that the route refuses all three positions in its own parse first: 400 `VALIDATION_FAILED`, asserted by status, code and field path only. The `true` / `false` control runs through `POST /api/v1/data/:object/query`, and the filter's sum and the kept group equal the `where` twin's. - driver-memory: this package does not depend on it, and the in-memory driver's test consumers are a closed census (`check:driver-memory-census`). Its cell is therefore the engine-level `rows` shape above, as #20873 and #21007 did, and the measurement table above was taken on the real `InMemoryDriver`. **Pin sweep.** I ran a repo-wide grep for a non-boolean `$exists` / `$null` literal in tests and for both refusal sentences. Every other hit pins a driver's, the analytics door's or the save door's own face, and none asserts the engine's old answer. Two `REFERENCE_REFUSED` rows (`engine-aggregate-filter.test.ts`, `engine-aggregate-having-comparand-shape.test.ts`) now meet the flag gate first. They assert the path only and stay green, and each has a comment pointing at the new file. `rest-4xx-message-truncation.test.ts` carries a copy of driver-sql's `$null` text for a truncation fixture. It is not a pin on this face and is not touched. ## Verification (final head `704a58ab2`, after merging `origin/main` twice) - `pnpm --filter @objectstack/objectql test`: **358 files, 7051 tests passed** (post-merge, `a876bc2af`; the second merge brought no objectql, rest, core, formula, driver or spec `data/` change). `pnpm --filter @objectstack/objectql typecheck`: exit 0. The new test file is in `tsconfig.test.json`'s program (`--listFiles`: 1 hit; 0 in `tsconfig.json`'s, which excludes tests), and `check:test-typecheck` is OK. - At `704a58ab2`: the four objectql files around this change (302 tests) and the rest cell (15 tests) passed, after rebuilding the closure. `packages/rest`: `tsc --noEmit` and `check:test-typecheck` OK. The 12 aggregation and flag-related rest files (`aggregation-*`, `rest-aggregate-*`, `data-query-having-temporal-door`, and the three other files naming a flag operator) passed at `faafb51d3`, before the merges: 200 passed, 158 skipped (the env-gated PostgreSQL / MySQL cells). The whole rest suite is declared to CI. - **Reverse verification**, from the committed fix, through `scripts/ablation-replace.mjs`. Each mutation was proven on disk (anchor count and blob change), and each restore was proven by blob equal to HEAD with `git diff HEAD` empty: - leg A, both gates neutralised: objectql file **31 failed / 8 passed** (the 8 controls stay green); - leg B, the one-time gate only: **29 failed / 10 passed** (the engine cases go red; the per-row and control cases stay green); - leg C, the per-row floor only: **2 failed / 37 passed** (only the published row-evaluator cases go red); - leg R, for the rest cell, which reads objectql through `dist/`: a string marker was planted, objectql rebuilt, and `ablation-dist-preflight` found the marker in 4 built files. Result: **10 failed / 5 passed** (the 10 refusal cases). The leg was then restored and rebuilt; preflight `--absent` reported the marker absent from all 14 built files and a clean tree, and the cell passed **15/15**. - Driver conformance ledger (`node scripts/check-driver-conformance.mjs`): **50 covered, 0 DEBT, 0 exempt**, before and after. - Gates: `node scripts/pm/dispatch-gates.mjs --commands` after the final commit derived 63 families from the 6 changed paths. `--ran` reconciles them as **63 accounted for: 62 run and exit 0, 1 NOT MEASURED**. The NOT MEASURED one is `check:dual-build-cjs-loads` (exit 3, prerequisite: it reads every package's `dist/`, and this tree builds only the closures). `check:dts-closure` first read red on two partially built packages, `organizations` and `plugin-webhooks`. Their partial `dist/` files were written inside the two windows in which my own 560-second timeout killed `check:type-check-debt`. Both were rebuilt, and the gate then swept 61 built packages with 153/153 declaration files present. `check:type-check-debt` then ran to completion: exit 0, in 232 s. - Lint, narrowed: `eslint --no-inline-config --format json` over the five touched TypeScript files reported 5 files, 0 errors and 0 warnings. These files are in the config's own population (`files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']` minus `NEVER_LINTED` and `packages/spec/**`). The config never enables type-aware linting (no `parserOptions.project`), so this diff cannot move any untouched file's verdict. The full `pnpm lint` is CI's. ## Acceptance notes - **The spec's three sentences now read true for every query face.** These are `filter.zod.ts` ("Every query face refuses a non-boolean `$null` / `$exists`"), the save-door refusal text in `filter-save-door-refusals.ts`, and migration entry `18.filter-query-face-comparands-refused-at-save`. No spec edit. `@objectstack/formula`'s `matchesFilterCondition`, the RLS write-`check` evaluator, still answers a non-boolean flag (`v === true ? … : …`). By the repo's own vocabulary it is not a query face: its `$empty` docblock contrasts itself with "the spec's save door and every query face". Observation only, not filed: no public door was measured for it. - **Changeset**: `@objectstack/objectql` `minor`, BREAKING, with the ADR-0087 disposition `not-required (already-registered filter-query-face-comparands-refused-at-save)`. That registered entry's surface names a query `having`, the aggregate call's `having` and an aggregation filter, and its replacement is this change's whole migration. #20979 used the same disposition for the drivers' half. - **Tooling, observed while verifying.** The `turbo` bump to 2.11.5 that `origin/main` brought in this window writes a managed "agent guidance" block into the root `AGENTS.md` before repo-scoped commands when it detects an agent. That happened here on every `turbo run` and on `check:type-check-debt`. It dirties every agent worktree (+11 lines), reddens `check:pm-skill-ratchet` locally (1119 lines against a ceiling of 1116), and puts a governed-surface edit one `git add -A` away. Here it was restored from `HEAD` each time and is not in this diff. Raised in the report on #20981 for the seat to route. - Not touched: `engine.ts`, `in-memory-aggregation.ts`, every driver, the spec and formula. --- _Generated by [Claude Code](https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent fde553c commit c35436c

6 files changed

Lines changed: 657 additions & 10 deletions
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
'@objectstack/objectql': minor
3+
---
4+
5+
fix(objectql)!: a per-aggregation `filter` and a `having` refuse a non-boolean `$exists` / `$null` with `INVALID_FILTER` / 400, in the words every driver's `where` refuses it in, instead of reading `$exists` by truthiness and dropping `$null` (#20981)
6+
7+
Clause-②: no (narrowing)
8+
9+
<!-- adr-0087: not-required (already-registered filter-query-face-comparands-refused-at-save) this narrows the engine's own in-process evaluator to the rule that registered entry already records: its reason states that every query face refuses a non-boolean $null / $exists flag, its surface names a query having, the data-engine aggregate call's having and an aggregation filter among the carriers, and its replacement is this change's whole migration (a flag is the boolean itself; $null true is "has no value", $exists true is "has a value"). This change makes that statement true on the per-aggregation filter and having positions, which the engine evaluates itself after the driver. No authorable key, spelling, export or published type moves, and no stored row is read or rewritten; a stored filter carrying such a flag is already refused when it is saved, by that entry. -->
10+
11+
**BREAKING**: this narrows what `aggregate` accepts in two positions, `aggregations[i].filter` and `having`, on every driver. A `$exists` or `$null` comparand that is not a boolean (a string such as `"false"`, a number, `null`, an array) is now refused with `INVALID_FILTER` / 400, before any driver is asked for a row, so an empty table refuses it too, at any depth under `$and` / `$or` / `$not`. A plain object or `undefined` there is refused first by the comparand-type check, in its own words, as before; a `{ $field }` reference there, already refused as a reference outside a scalar comparison, is now refused in this entry's words. The published `applyInMemoryAggregation(rows, ast, timezone, fields)` narrows the same way, per row: it throws the same refusal for a row its per-aggregation filter judges on the flag, with or without a `fields` map (an empty `rows` array, or a row a `$or` branch settles first, is not judged there; `engine.aggregate` judges the whole filter once before any row). It ships as `minor` under the launch-window convention for accept-set narrowings. No export or published type changes.
12+
13+
**Why a refusal.** `FieldOperatorsSchema` declares both flags as booleans, and every driver's `where` refuses any other comparand. The engine evaluates a per-aggregation `filter` and a `having` itself, and it read one anyway. Measured through `engine.aggregate` on the in-memory driver and on `SqlDriver` (SQLite), with identical answers: `$exists` was read by truthiness, so `"yes"`, `1` and the string `"false"` selected the rows and groups WITH a value, and `0` / `null` the ones without; and `$null` tested only `true` / `false`, so any other value constrained nothing, and every row and every group came back.
14+
15+
**What an author sees now.** The message `driver-sql` gives the same flag, beginning `Operator "$exists" on field "FIELD" requires a boolean comparand (true or false).`, naming what arrived and the position (`aggregations[1].filter.stage.$exists`, `having.stage.$null`). Unlike a `where` on `SqlDriver`, the field and the value are not withheld: a per-aggregation `filter` and a `having` never carry a merged read scope.
16+
17+
**What to write instead.** Write the boolean itself. `"$exists": true` and `"$null": false` match a field that has a value; `"$exists": false` and `"$null": true` match one that has none.
18+
19+
**Who is affected.** A caller that reaches `engine.aggregate` without the REST query door's schema parse (server-side code, a flow or hook, the analytics bridge that lowers a dataset measure's filter into an aggregation filter, a host calling `applyInMemoryAggregation` directly) and read the count as a real answer. `POST /api/v1/data/:object/query` already refused all three positions with 400 `VALIDATION_FAILED` before the request reached the engine, and still does.
20+
21+
**Unchanged.** `$exists: true` / `false` and `$null: true` / `false` answer exactly as before, on both positions. `$empty` and every other operator, and `where`.

‎packages/objectql/src/engine-aggregate-filter.test.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -439,6 +439,8 @@ describe('[#20122] per-aggregation filter — the walker\'s refusals belong to t
439439
['a reference as an $in member', () => ({ amount: { $in: [{ $field: 'amount' }] } }), `${AT}.amount.$in`],
440440
['a reference as a $nin member', () => ({ amount: { $nin: [1, { $field: 'amount' }] } }), `${AT}.amount.$nin`],
441441
['a reference as a $contains pattern', () => ({ stage: { $contains: { $field: 'region' } } }), `${AT}.stage.$contains`],
442+
// [#20981] Refused at the same path, as a non-boolean flag first now — the
443+
// words are pinned in engine-aggregate-flag-comparand-refusal.test.ts.
442444
['a reference under $exists', () => ({ amount: { $exists: { $field: 'amount' } } }), `${AT}.amount.$exists`],
443445
['a fractional addDays', () => ({ amount: { $lte: { $field: 'amount', addDays: 1.5 } } }), `${AT}.amount.$lte`],
444446
['a string addDays', () => ({ amount: { $lte: { $field: 'amount', addDays: '7' } } }), `${AT}.amount.$lte`],
Lines changed: 284 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,284 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
//
3+
// [#20981] A per-aggregation `filter` and a `having` REFUSE a non-boolean
4+
// `$exists` / `$null` with `INVALID_FILTER` / 400, in the words every driver's
5+
// `where` refuses it in, before any driver is asked. `true` / `false` answer
6+
// exactly as before.
7+
//
8+
// Measured before this change (`origin/main` `7a606a9a3`) through
9+
// `engine.aggregate` on driver-memory AND driver-sql (better-sqlite3), over a
10+
// text column `name` holding `'won'` on one row (amount 10) and no value on two
11+
// (amounts 1 and 100). The engine evaluates both clauses itself, so the two
12+
// drivers answered identically:
13+
//
14+
// | clause · comparand | `$exists` | `$null` |
15+
// |:--|:--|:--|
16+
// | filter · `"yes"` / `1` / `"false"` | sum 10 (the valued row) | sum 111 (every row) |
17+
// | filter · `0` / `null` | sum 101 (the no-value rows) | sum 111 (every row) |
18+
// | having · `"yes"` / `1` / `"false"` | the `won` group | both groups |
19+
// | having · `0` / `null` | the null group | both groups |
20+
//
21+
// while the same flag in a `where` is refused 400 on both drivers. After it,
22+
// every cell above is that 400, and the `true` / `false` control keeps the
23+
// numbers below.
24+
//
25+
// This file is the engine-level cell, over a driver of each `having` path's
26+
// shape — `rows` (no `aggregate()`: the engine reads rows and lowers in memory,
27+
// the face that evaluates `aggregations[i].filter`) and `native` (the driver
28+
// aggregates, the engine applies `having`). Every refusal precedes the driver,
29+
// so the backend does not enter into it; the same answers were measured on
30+
// driver-memory and driver-sql above. This package does not depend on either
31+
// driver, and the in-memory driver's test consumers are a closed census
32+
// (`check:driver-memory-census`), so the SQLite cell — beside its live `where`
33+
// twin, through `POST /data/:object/query` — is `packages/rest`'s
34+
// `aggregation-flag-comparand-refusal.test.ts`.
35+
36+
import { describe, it, expect } from 'vitest';
37+
import type { EngineAggregateOptions, FilterCondition } from '@objectstack/spec/data';
38+
import { ObjectQL } from './engine.js';
39+
import { applyHaving, matchesAggregationFilter } from './having-filter.js';
40+
import { applyInMemoryAggregation } from './in-memory-aggregation.js';
41+
42+
const OBJECT = 'os20981_deal';
43+
44+
const FIELDS = {
45+
name: { type: 'text' },
46+
amount: { type: 'number' },
47+
};
48+
49+
// `a` holds a value; `b` holds null and `c` does not carry the column — both
50+
// have NO value, the reading `$exists` and `$null` share (#5298).
51+
const ROWS = [
52+
{ id: 'a', name: 'won', amount: 10 },
53+
{ id: 'b', name: null, amount: 1 },
54+
{ id: 'c', amount: 100 },
55+
];
56+
57+
type Path = 'native' | 'rows';
58+
59+
/** A driver of one `having` path's shape, counting every read of the object. */
60+
function makeDriver(path: Path, rows: ReadonlyArray<Record<string, unknown>>) {
61+
const reads = { aggregate: 0, find: 0 };
62+
const driver: any = {
63+
name: `${path}-recorder`,
64+
version: '0.0.0',
65+
supports: {},
66+
async connect() {}, async disconnect() {}, async checkHealth() { return true; }, async execute() { return null; },
67+
async find() { reads.find += 1; return rows.map((r) => ({ ...r })); },
68+
async findOne() { return null; },
69+
async create(_o: string, d: any) { return d; },
70+
async update(_o: string, _id: string, d: any) { return d; },
71+
async delete() { return true; },
72+
async count() { return rows.length; },
73+
async bulkCreate(_o: string, r: any[]) { return r; },
74+
async bulkUpdate() { return []; }, async bulkDelete() {},
75+
async beginTransaction() { return { commit: async () => {}, rollback: async () => {} }; },
76+
async commit() {}, async rollback() {},
77+
};
78+
if (path === 'native') {
79+
driver.aggregate = async (_o: string, ast: any) => { reads.aggregate += 1; return applyInMemoryAggregation([...rows], ast); };
80+
}
81+
return { driver, reads };
82+
}
83+
84+
async function makeEngine(path: Path, rows: ReadonlyArray<Record<string, unknown>> = ROWS) {
85+
const { driver, reads } = makeDriver(path, rows);
86+
const engine = new ObjectQL();
87+
engine.registerDriver(driver, true);
88+
await engine.init();
89+
(engine.registry as any).registerObject({ name: OBJECT, fields: FIELDS });
90+
return { engine, reads };
91+
}
92+
93+
/** `s` sums `amount` over the rows `filter` selects; `n` counts every row. Always the rows path. */
94+
const filterQuery = (filter: unknown): EngineAggregateOptions => ({
95+
aggregations: [
96+
{ function: 'count', alias: 'n' },
97+
{ function: 'sum', field: 'amount', alias: 's', filter: filter as FilterCondition },
98+
],
99+
});
100+
101+
/** Groups by `name`; on the rows path a filtered aggregation forces the engine's own lowering. */
102+
const havingQuery = (path: Path, having: unknown): EngineAggregateOptions => ({
103+
groupBy: ['name'],
104+
aggregations: path === 'native'
105+
? [{ function: 'count', alias: 'n' }]
106+
: [{ function: 'count', alias: 'n' }, { function: 'count', alias: 'all', filter: { amount: { $gte: 0 } } }],
107+
having: having as FilterCondition,
108+
});
109+
110+
interface Refusal extends Error { code?: unknown; status?: unknown }
111+
112+
async function refusalOf(run: () => Promise<unknown>): Promise<Refusal> {
113+
try {
114+
await run();
115+
} catch (e) {
116+
return e as Refusal;
117+
}
118+
throw new Error('expected engine.aggregate to refuse this clause, but it answered');
119+
}
120+
121+
function syncRefusalOf(run: () => unknown): Refusal {
122+
try {
123+
run();
124+
} catch (e) {
125+
return e as Refusal;
126+
}
127+
throw new Error('expected a refusal, but the clause was answered');
128+
}
129+
130+
/**
131+
* The comparand doors' refusal: the ADR-0112 identity, then the drivers' first
132+
* three sentences — the operator, the field, what arrived (in the drivers'
133+
* `describeFilterOperand (safeShapePreview)` rendering) and where, and the
134+
* declaration — then the reason that names the backends.
135+
*/
136+
function expectFlagRefusal(err: Refusal, op: '$exists' | '$null', received: string, at: string): void {
137+
expect(err.code, err.message).toBe('INVALID_FILTER');
138+
expect(err.status, err.message).toBe(400);
139+
expect(err.message.startsWith(
140+
`Operator "${op}" on field "name" requires a boolean comparand (true or false). `
141+
+ `Received ${received} at ${at}. `
142+
+ `@objectstack/spec FieldOperatorsSchema declares ${op} as a boolean. It is refused rather than coerced `,
143+
), err.message).toBe(true);
144+
expect(err.message).toContain('OPPOSITE directions');
145+
expect(err.message).toContain('Note "false" the STRING is truthy');
146+
}
147+
148+
const OPS = ['$exists', '$null'] as const;
149+
150+
/** The card's five non-booleans, each with the rendering the drivers give it. */
151+
const NON_BOOLEANS: ReadonlyArray<readonly [string, unknown, string]> = [
152+
['"yes"', 'yes', 'string ("yes")'],
153+
['1', 1, 'number (1)'],
154+
['"false" (truthy)', 'false', 'string ("false")'],
155+
['0', 0, 'number (0)'],
156+
['null', null, 'null (null)'],
157+
// Not one of the card's five, but reaching the same gate: no earlier door
158+
// refuses a list here, so it met the old `!!target` read like any other value.
159+
['[true] (a list)', [true], 'array ([true])'],
160+
];
161+
162+
describe('[#20981] a non-boolean $exists / $null — refused before any driver read, on both positions', () => {
163+
for (const op of OPS) {
164+
for (const [label, comparand, received] of NON_BOOLEANS) {
165+
it(`aggregations[1].filter ${op}: ${label} — 400, the drivers' words, on an empty and a populated table`, async () => {
166+
for (const rows of [[], ROWS]) {
167+
const { engine, reads } = await makeEngine('rows', rows);
168+
const err = await refusalOf(() => engine.aggregate(OBJECT, filterQuery({ name: { [op]: comparand } })));
169+
expectFlagRefusal(err, op, received, `aggregations[1].filter.name.${op}`);
170+
expect(reads, 'the verdict is the filter\'s: no row was read').toEqual({ aggregate: 0, find: 0 });
171+
}
172+
});
173+
174+
it(`having ${op}: ${label} — 400, the drivers' words, on both having paths`, async () => {
175+
for (const path of ['native', 'rows'] as const) {
176+
for (const rows of [[], ROWS]) {
177+
const { engine, reads } = await makeEngine(path, rows);
178+
const err = await refusalOf(() => engine.aggregate(OBJECT, havingQuery(path, { name: { [op]: comparand } })));
179+
expectFlagRefusal(err, op, received, `having.name.${op}`);
180+
expect(reads, `${path}: no row was read`).toEqual({ aggregate: 0, find: 0 });
181+
}
182+
}
183+
});
184+
}
185+
}
186+
187+
// Where the flag sits must not change the verdict: the walk is row-independent,
188+
// so a branch the per-row walk would short-circuit past is judged too.
189+
const POSITIONS: ReadonlyArray<readonly [string, (flag: Record<string, unknown>) => Record<string, unknown>, string]> = [
190+
['nested in $and', (flag) => ({ $and: [{ amount: { $gte: 0 } }, { name: flag }] }), '.$and[1].name'],
191+
['behind a $or branch that already holds', (flag) => ({ $or: [{ amount: { $gte: 0 } }, { name: flag }] }), '.$or[1].name'],
192+
['under $not', (flag) => ({ $not: { name: flag } }), '.$not.name'],
193+
['beside a boolean sibling flag', (flag) => ({ name: { $empty: false, ...flag } }), '.name'],
194+
];
195+
196+
for (const [name, wrap, tail] of POSITIONS) {
197+
it(`${name}: refused at that position, in the per-aggregation filter and in having`, async () => {
198+
const filterEngine = await makeEngine('rows');
199+
expectFlagRefusal(
200+
await refusalOf(() => filterEngine.engine.aggregate(OBJECT, filterQuery(wrap({ $exists: 'yes' })))),
201+
'$exists', 'string ("yes")', `aggregations[1].filter${tail}.$exists`,
202+
);
203+
// `having` reads the aggregated row, which carries `name` (the groupBy)
204+
// but not `amount` — so its wrapper's other branch tests `n` instead.
205+
const having = JSON.parse(JSON.stringify(wrap({ $null: 1 })).replaceAll('"amount"', '"n"'));
206+
const havingEngine = await makeEngine('native');
207+
expectFlagRefusal(
208+
await refusalOf(() => havingEngine.engine.aggregate(OBJECT, havingQuery('native', having))),
209+
'$null', 'number (1)', `having${tail}.$null`,
210+
);
211+
expect(filterEngine.reads.find + havingEngine.reads.aggregate).toBe(0);
212+
});
213+
}
214+
215+
it('a { $field } reference in a flag\'s slot is a non-boolean first, as in $empty\'s', async () => {
216+
const { engine } = await makeEngine('rows');
217+
expectFlagRefusal(
218+
await refusalOf(() => engine.aggregate(OBJECT, filterQuery({ name: { $exists: { $field: 'amount' } } }))),
219+
'$exists', 'object ({"$field":"amount"})', 'aggregations[1].filter.name.$exists',
220+
);
221+
const native = await makeEngine('native');
222+
expectFlagRefusal(
223+
await refusalOf(() => native.engine.aggregate(OBJECT, havingQuery('native', { name: { $null: { $field: 'n' } } }))),
224+
'$null', 'object ({"$field":"n"})', 'having.name.$null',
225+
);
226+
});
227+
});
228+
229+
describe('[#20981] the control — true / false answer exactly as before, on both positions and both paths', () => {
230+
// `a` (10) has a value; `b` (1) and `c` (100) have none.
231+
const SUMS: ReadonlyArray<readonly [string, Record<string, unknown>, number]> = [
232+
['$exists: true', { $exists: true }, 10],
233+
['$exists: false', { $exists: false }, 101],
234+
['$null: true', { $null: true }, 101],
235+
['$null: false', { $null: false }, 10],
236+
];
237+
238+
for (const [name, flag, sum] of SUMS) {
239+
it(`aggregations[1].filter ${name} sums ${sum}`, async () => {
240+
const { engine, reads } = await makeEngine('rows');
241+
expect(await engine.aggregate(OBJECT, filterQuery({ name: flag }))).toEqual([{ n: 3, s: sum }]);
242+
expect(reads.find).toBe(1);
243+
});
244+
245+
const kept = sum === 10 ? ['won'] : ['null'];
246+
it(`having ${name} keeps the ${kept[0]} group on both paths`, async () => {
247+
for (const path of ['native', 'rows'] as const) {
248+
const { engine } = await makeEngine(path);
249+
const groups = await engine.aggregate(OBJECT, havingQuery(path, { name: flag }));
250+
expect(groups.map((g: any) => String(g.name)), path).toEqual(kept);
251+
}
252+
});
253+
}
254+
});
255+
256+
describe('[#20981] the published row evaluators refuse per row — the floor for a caller that evaluates rows itself', () => {
257+
it('applyInMemoryAggregation with fields: $exists "yes" and $null 1 are refused, true / false answer', () => {
258+
for (const [op, comparand, received] of [['$exists', 'yes', 'string ("yes")'], ['$null', 1, 'number (1)']] as const) {
259+
const err = syncRefusalOf(() => applyInMemoryAggregation(
260+
ROWS.map((r) => ({ ...r })),
261+
filterQuery({ name: { [op]: comparand } }),
262+
undefined,
263+
FIELDS,
264+
));
265+
expectFlagRefusal(err, op, received, `aggregations[1].filter.name.${op}`);
266+
}
267+
expect(applyInMemoryAggregation(ROWS.map((r) => ({ ...r })), filterQuery({ name: { $exists: true } }), undefined, FIELDS))
268+
.toEqual([{ n: 3, s: 10 }]);
269+
expect(applyInMemoryAggregation(ROWS.map((r) => ({ ...r })), filterQuery({ name: { $null: true } }), undefined, FIELDS))
270+
.toEqual([{ n: 3, s: 101 }]);
271+
});
272+
273+
it('applyHaving and matchesAggregationFilter refuse a row that reaches the flag, with or without a value', () => {
274+
const groups = [{ name: 'won', n: 1 }, { name: null, n: 2 }];
275+
for (const row of groups) {
276+
expectFlagRefusal(syncRefusalOf(() => applyHaving([row], { name: { $exists: 'false' } } as never)),
277+
'$exists', 'string ("false")', 'having.name.$exists');
278+
expectFlagRefusal(syncRefusalOf(() => matchesAggregationFilter(row, { name: { $null: 0 } } as never, 2)),
279+
'$null', 'number (0)', 'aggregations[2].filter.name.$null');
280+
}
281+
expect(applyHaving(groups, { name: { $null: false } })).toEqual([{ name: 'won', n: 1 }]);
282+
expect(applyHaving(groups, { name: { $exists: false } })).toEqual([{ name: null, n: 2 }]);
283+
});
284+
});

‎packages/objectql/src/engine-aggregate-having-comparand-shape.test.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -646,6 +646,8 @@ describe('[#20099] having — a { $field } reference resolves against the aggreg
646646
['a reference as a $nin member', () => ({ total: { $nin: [1, { $field: 'max_cap' }] } }), 'having.total.$nin'],
647647
['a reference as a $contains pattern', () => ({ customer_id: { $contains: { $field: 'customer_id' } } }), 'having.customer_id.$contains'],
648648
['a reference as a $startsWith pattern', () => ({ customer_id: { $startsWith: { $field: 'customer_id' } } }), 'having.customer_id.$startsWith'],
649+
// [#20981] These two are refused at the same path as a non-boolean flag
650+
// first now — the words are pinned in engine-aggregate-flag-comparand-refusal.test.ts.
649651
['a reference under $exists', () => ({ total: { $exists: { $field: 'max_cap' } } }), 'having.total.$exists'],
650652
['a reference under $null', () => ({ total: { $null: { $field: 'max_cap' } } }), 'having.total.$null'],
651653
['a reference naming no column', () => ({ total: { $gt: { $field: 'nope' } } }), 'having.total.$gt'],

0 commit comments

Comments
 (0)