Skip to content

Commit 793fb83

Browse files
feat(service-analytics,driver-memory): the shared filter lowering at the analytics seams and the cube face's new door (#5930 step 3) (#20857)
Fixes #20810 Clause-②: yes (narrowing) #5930 step 3, under ADR-0053 D-D1 as amended by #20754: the one shared `FilterCondition → FilterCondition` lowering (`lowerFilterCondition`, `@objectstack/spec/data`, landed in step 2 as #20794) now runs at the three analytics seams, after the comparand doors and after filter-token resolution, and the two faces that could not compile its output now can. `packages/spec/src/**`, the interim window arms and #20807's position are untouched. ## Named gap: three pins outside the claim's file surface `packages/drivers/driver-memory/src/memory-driver-filter-logic-conformance.test.ts` holds three pins of the cube face's PRE-change vocabulary and shape, and it is outside the claim's `memory-analytics*.test.ts` surface, so this branch does not edit it. They are red here, and the `driver-memory` Test Core shard is red until they move: - "every combinator case is refused rather than dropped" asserts every `$or` case refuses. The face compiles `$or` now; the same file's case-by-case invariant (agree with `find()` or refuse) stays green over every `$or` case. - "every operator this face DECLARES compiles to a predicate that agrees with find()" needs a probe for `$null`, which joined the face's table. - the `$notContains` pipeline-dump row pins the un-lowered `$match`; the lowering's NULL escape now wraps it. The three-row patch (15 changed lines) is prepared and was verified on a copy of the file: 125 of 125 tests pass. It lands once the file is added to the surface. ## Per seam | Seam | Position at `bbe03f406` | Pin | Ablation (lowering removed at that seam) | |---|---|---|---| | analytics `where` door, F10 (`where` → tree, both strategies) | `normalizeWhereComparands` `filter-normalizer.ts:2218`, reached through `lowerAnalyticsWhere` `:2244`; the lowering sits in `normalizeAnalyticsFilterTree` `:2356`, the one compile entry both spellings reach | `where-door-shared-lowering-seam.test.ts`, 18 rows | 6 of 18 red: the tree rows, the ObjectQL hand-off and the echo | | draft-preview door, F11 | `preview-evaluator.ts:675`, right after its `normalizeWhereComparands` call | same file (preview block) | 3 of 18 red: the three NULL-escape rows | | read scope | entry of `compileScopedFilterToSql` `read-scope-sql.ts:747`, after placeholder resolution `:764`, before `compileNode` `:769` | `read-scope-shared-lowering-seam.test.ts`, 12 rows | 5 of 12 red: the caller-filter rows, the RLS bound as written, the SQL row and the date-macro row | | cube face F5, the new door | `normalizeFilters` `memory-analytics.ts:1585`, ahead of its gate `:1592` | `memory-analytics-shared-lowering-door.test.ts`, 13 rows | doors removed: 4 of 13 red (#20734's table); lowering removed: 2 of 13 red | | the where door's nested-relation spelling (below) | `normalizeAnalyticsFilterTree` | the nested-relation rows in two files | 2 red | Every ablation mutated the committed file through `scripts/ablation-replace.mjs` (anchor hit once, blob moved), ran the pins, and restored: blob equal to HEAD and `git diff HEAD` empty, each time. The pins import their subjects by relative path, so vitest reads `src/` and no build sits between the mutation and the reading. ## What each seam does - **Read scope.** `compileScopedFilterToSql` lowers the scope right after its placeholders resolve. Column-type scope (item 7): the `declaredValueShape` option both consumers already pass, so a declared `datetime` column is rewritten and a `date`, `time` or text column compiles byte-identical; no declarations handed in reads no column as `datetime`. The shared comparand doors still judge the scope as written after compilation (#20018's order); the lowering never refuses, so no verdict moves. - **Analytics `where` door (F10).** `normalizeAnalyticsFilterTree` lowers what the door admitted before `buildNode` reads it. Its column-type reader is now a REQUIRED argument: both strategies pass `declaredDatetimeLowering` (a member is `datetime` when its column is declared so, through the `declaredFieldType` hook, asked of the same target each strategy compiles against); a context without the hook, and the member-only readers (`assertWhereFields`, the cross-object view), pass `NO_DATETIME_COLUMNS`. `lowerAnalyticsWhere` stays un-lowered: its other readers (ad-hoc cube dimension minting, the routing detectors) read the authored condition. - **Nested relations at the `where` door.** The shared lowering has no reading of the nested-relation spelling (`{ account: { region: 'NA' } }`: accepted by the schema, refused by the engine, flattened only by this door). Inside a `$not` it read `account` as a column and guarded it. The door now spells nested relations as the dotted members `fieldLeaves` has always compiled them to, before the lowering reads the condition, so the guard lands on `account.region`. - **Draft preview (F11).** Lowered after its `normalizeWhereComparands` with `NO_DATETIME_COLUMNS`: drafted rows carry no schema, and a type-blind rewrite would move one cell away from the typed drivers (`$lte` on the last supported day over a non-temporal value that sorts above it); its own `lteBound` copy keeps the whole-day rule until its deletion card. It now evaluates `$null`. - **Cube face (F5).** `normalizeFilters` runs `assertListComparandShapes`, then `normalizeFilterComparandTypes` (its return is the condition read from there on), then the lowering (type-blind: the face reaches declared types only as a storage-form conversion, and its own copy and `find()` are type-blind already), then its own gate on the lowered condition. It compiles `$or` (a disjunction entry both exits render, with the #5322 identities) and `$null`; `$not`, `$startsWith`, `$endsWith` and `$empty` stay refused. ## Every corrected answer Read scope (`compileScopedFilterToSql`: the NativeSQL statement's scope and the `/analytics/sql` echo's), each now `SqlDriver.find`'s rows on the same filter: - `{ signed_at: { $lte: '2026-07-28' } }` on a declared `datetime` over rows at 10:00Z on 07-27, 07-28, 07-29 and a row with no value: was 07-27, now 07-27 and 07-28 (`< '2026-07-29'`). #20733's caller-filter half. - `{ signed_at: { $between: ['2026-07-28', '2026-07-28'] } }`: was no row, now 07-28. - a `{today}` upper bound is widened as the day it resolves to. Analytics `where`, ObjectQL path: a bare-day `$lte` on a `datetime` member reaches the engine as `$lt` the next day, and the `/analytics/sql` echo prints that half-open bound where it printed `<=` the named day. Rows unchanged. Draft preview: `$null` answered (was refused 400). A row with no value now satisfies `$ne`, `$nin` and the negation of an equality when the comparand is the text `"null"` or `"undefined"`; the face compared those as text against the missing value. Cube face, measured on the published `MemoryAnalyticsService.query` at the base blob and at this head: | `where` (fixture: `d` is `v1`, `v2`, null, absent) | base | this head | `find()` | |---|---|---|---| | `{d: undefined}` / `{d: {$eq: undefined}}` | 3, 4 | refused 400 | 3, 4 | | `{d: {$ne: undefined}}` | 1, 2 | refused 400 | 1, 2 | | `{d: {$in: ['v1', undefined]}}` | 1 | refused 400 | 1 | | `{d: {$in: ['v1', null]}}` | 1, 3, 4 | refused 400 | 1, 3, 4 | | `{d: {$nin: ['v1', null]}}` | 2 | refused 400 | 2 | | `{d: {$gt: null}}` | none | refused 400 | none | | `{d: {$in: 'v1'}}` | 1 | refused 400 | uncoded throw | | `{d: {$eq: {a: 1}}}` | none | refused 400 | none | | `{d: {$ne: {a: 1}}}` | all four | refused 400 | all four | | `{d: new Map()}` | all four | refused 400 | none | | `{n: {$gt: 2n ** 60n}}` | uncoded throw | refused 400 | uncoded throw | | `{n: {$gt: 2n}}` | uncoded throw | 3, 4 | uncoded throw | | `{d: {$between: ['v1', 'v2']}}` | refused 400 | 1, 2 | 1, 2 | | `{d: {$null: true}}` | refused 400 | 3, 4 | 3, 4 | | `{$or: [{d: 'v1'}, {n: 4}]}` | refused 400 | 1, 4 | 1, 4 | | `{d: {$ne: 'v1'}}`, `{$not: {d: 'v1'}}` | unchanged | unchanged | — | (`find()` here is the driver called directly, without the engine seam whose doors it relies on.) Every "refused 400" is the ADR-0112 `INVALID_FILTER` envelope every other analytics face already answers. ## Clause-②, measured - **Widening:** the cube face accepts `$or`, `$null` and `$between` (each refused at the base, each now `find()`'s rows), and the draft preview accepts `$null`. `ANALYTICS_FILTER_CAPABILITIES` names `$null` and `$or`. - **Narrowing:** the cube face refuses the comparand shapes in the table above that it used to answer. Hence `yes (narrowing)`. `@objectstack/driver-memory` ships `minor` with the BREAKING banner, the migration and an ADR-0087 `not-required (no-migration-prescription)` disposition; `@objectstack/service-analytics` ships `minor` (`Clause-②: yes`, widening only: the read scope and the `where` door refuse nothing new). ## Mechanism assumptions, measured - **A1 held**, with one refinement: the F10 call sits in `normalizeAnalyticsFilterTree` rather than inside `normalizeWhereComparands`, because the array spelling reaches F10 through `parseFilterAST` and never through `normalizeWhereComparands`, and because `lowerAnalyticsWhere`'s other readers want the authored condition. F11 lowers right after its own `normalizeWhereComparands` call. - **A2 measured.** `where` door: tokens resolve upstream on every path that reaches it (`AnalyticsService.resolveQueryTokens` from `query()` and `generateSql()`, the per-request dataset-scope getter, `DatasetExecutor.resolveSelectionTokens` ahead of the preview). Read scope: placeholder resolution is the compiler's first act, and the doors run after `compileNode` on the scope as written (#20018), so the lowering sits between resolution and compilation. F5: no token resolution exists on that face. - **A3 proved, not assumed.** With the real `SecurityPlugin` and the real read-scope compiler on SQLite (a one-off run, not committed): `getReadFilter` returned `{"$and":[{"signed_at":{"$lt":"2026-07-29"}},{"due_on":{"$lte":"2026-07-28"}}]}`, the read scope compiled `("t"."signed_at" < ? AND "t"."due_on" <= ?)` and answered c27 and c28, equal to `SqlDriver.find`. The committed pin holds the RLS half in both spellings the read scope can be handed (lowered by the RLS seam, and as written), so its answer no longer depends on which arrives. - **A4 measured.** F5 compiled 12 operators with `$and` alone; F11 10 operators with `$and`/`$or`/`$not`. The lowering emits `$and`, `$or`, `$lt`, `$gte`, `$lte`, `$null`. Widened: F5 `$or` + `$null`; F11 `$null`. Nothing wider. - **A5 held.** No window arm is touched (NativeSQL, the preview's window loop, F5's `timeDimensions`). - **A6** as the table above: #20733's rows (both halves, the last day, the declared `date` control) and #20734's table, plus a pin and an ablation per seam. - **A7** above. - **A8, coupling, re-measured:** #20280 landed on `main` as `05a7547c9` while this branch was open. Its datetime year floor lives in the engine's temporal comparand door (`objectql` `temporal-comparand-door.ts`) and in `@objectstack/core`'s temporal storage form; in `packages/spec/src/data` it changed one comment. The two spec doors the cube face now runs are not changed by it. What the two share is the storage-form conversion both of the face's exits apply to a comparand (the driver's `filterComparandStorageForm`, which reads `@objectstack/core`): #20280 changed that conversion, and this branch neither touches nor pre-empts it. Nothing in this PR's file surface changed on `main`, so the branch is not re-merged; this PR's CI runs on the merge ref with #20280 in it, and the merge queue adjudicates the rest. ## Evidence (head `92a449c2d`, `main` merged at `c90f9fb6e`) - `@objectstack/service-analytics`: 143 files, 3297 tests passed; `typecheck` green (baseline at `bbe03f406`: 141 files, 3267). - `@objectstack/driver-memory`: 66 files, 1482 passed, 3 failed (the named gap above); `typecheck` green (baseline 65 files, 1470). - Existing shape pins moved to the lowered structure; no asserted row count moved. The row-level conformance suites (native-SQL filter-logic and temporal, read-scope conformance, preview temporal) are unchanged and green. - Gates: `node scripts/pm/dispatch-gates.mjs --commands` derived 63 families at this head; all 63 were run with their exit codes recorded, and `--ran` reports 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN (a derived zero). `check:dual-build-cjs-loads` and `check:type-check-debt` first answered `PREREQUISITE NOT MET` and were re-run green after `turbo run build --filter='./packages/*' --filter='./packages/*/*'`. The six roster families under this card's directories (`check-changeset-fixed`, `check:authz-resolver`, `check:error-code-casing`, `check:filter-alias-parity`, `check:object-def-param-keys`, `check:tenant-chokepoint`) are green too. - Downstream analytics suites on the rebuilt dists: `@objectstack/rest` `analytics-*` (10 files, 150 tests), `@objectstack/runtime` `analytics-*` (3 files, 23), `@objectstack/dogfood` `analytics-adhoc-query-isolation` (24): green. - Lint, a narrowed run: the population is the `eslint.config.mjs` block `files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']`; `eslint --no-inline-config --format json` over the 38 changed source files reports 38 files, 0 errors, 0 warnings at `92a449c2d`; the config enables no type-aware linting (every `parserOptions` is `ecmaVersion` / `sourceType`), so this diff cannot move a verdict on an untouched file. - NOT MEASURED: live PostgreSQL / MySQL (CI's temporal job), a real mongod. ## Acceptance notes - **Double guards.** Each face's own interim NULL-polarity copy still guards what the lowering already guarded, so the SQL, the trees and the ObjectQL hand-off for `$ne`, `$nin`, `$notContains` and a `$not` operand carry the guard twice. The same rows; the deletion cards remove the inner copy and update these pins. - **For the step-4 deletion cards.** The draft preview reads no member as `datetime`; the `where` door with no `declaredFieldType` hook, and the read scope with no `declaredValueShape`, read none. Once a face's own bound copy is deleted, that path gets no whole-day bound unless its card gives it a typed reader. - **The read scope's temporal coercion (ADR-0053 D-A1, #20733's other half).** The read scope binds the lowered calendar string as written, as it bound the authored one. Measured correct on SQLite ISO text; PostgreSQL / MySQL NOT MEASURED. Carrier: none. - **The shared lowering and the nested-relation spelling.** The spec module reads a nested-relation spec under a `$not` as a column constraint. No face meets that after this change (the analytics door spells it dotted; the engine, the read scope and the preview refuse the spelling; the cube face refuses `$not`). Noted for the module's owner; carrier: none. ## Patch round 1 (appended by the `domain:services` seat) **The named gap is closed.** The claim's surface gained `packages/drivers/driver-memory/src/memory-driver-filter-logic-conformance.test.ts` (amendment `5911719808`), and `edb3e2de4` applies the prepared 15-line patch to it and to nothing else: - every `$not` case must still refuse; the `$or` cases moved to the answered column, which the file's case-by-case invariant holds to `find()`'s rows; - `$null` joins the declared-operator probe roster; - the `$notContains` pipeline dump shows the lowering's NULL escape. Evidence on head `edb3e2de4`: - `@objectstack/driver-memory`: 66 files, 1485 passed (the three pins green, nothing else moved); `typecheck` green. - `@objectstack/service-analytics`: 143 files, 3297 passed. - Gates: the same 63 derived families, all exit 0; `--ran` reports 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN; the six roster families green. - `main` was not re-merged: none of the five commits after `c90f9fb6e` touches this PR's files. --- _Generated by [Claude Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 856321f commit 793fb83

41 files changed

Lines changed: 1490 additions & 234 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
'@objectstack/service-analytics': minor
3+
---
4+
5+
The analytics seams now run the one shared filter lowering (`lowerFilterCondition`, `@objectstack/spec/data`) that ADR-0053 D-D1, as amended, places at every seam that runs the shared comparand doors: after the doors and after filter-token resolution, so each face compiles one lowered condition — the `$between` split, the whole-day upper bound on a bare `YYYY-MM-DD`, the last supported day, and the NULL-polarity guards.
6+
7+
Clause-②: yes
8+
9+
**The read scope (`compileScopedFilterToSql`).** The scope is lowered at the compiler's entry, right after its placeholders resolve. It reads each column's declared type from the `declaredValueShape` option both of its consumers already pass, so the whole-day rule rewrites a declared `datetime` column and nothing else; with no declarations handed in, no column is read as `datetime`. Corrected answers, each now the rows `SqlDriver.find` returns for the same filter:
10+
11+
- a bare-day `$lte` on a declared `datetime` column kept only the rows before that day and dropped the day itself. Rows at 10:00Z on 07-27, 07-28 and 07-29 under `{ signed_at: { $lte: '2026-07-28' } }` answered 07-27 alone; they now answer 07-27 and 07-28, compiled as `< '2026-07-29'`. This is the NativeSQL statement's read scope and the `/analytics/sql` echo's.
12+
- a bare-day `$between` on a declared `datetime` column answered no row for a one-day range, and now answers that day's rows.
13+
- a `{today}` (or any date-macro) upper bound is widened as the day it resolves to.
14+
15+
An RLS `using` bound already reached the read scope lowered by the RLS compile seam, and answers as before. A declared `date` column compiles byte-identical to before.
16+
17+
**The analytics `where` and draft-preview door.** The condition the door admits is lowered before either face reads it. The `where` → tree face (both strategies) reads each member's declared column type through the host's declared-type hook: a bare-day `$lte` on a `datetime` member now reaches the engine as `$lt` the next day, and the `/analytics/sql` echo prints that half-open bound — the statement the engine runs, where it used to print `<=` the named day. Rows are unchanged on every strategy. A nested-relation filter (`{ account: { region: 'NA' } }`) is spelled as the dotted member it has always compiled to before the lowering reads it, so a guard it adds under `$not` names that member.
18+
19+
The draft preview (`queryDataset` with `previewDrafts`) now evaluates `$null` — the one operator the lowering emits that it did not — so a drafted chart filtered on `{ field: { $null: true } }` is answered instead of refused `INVALID_FILTER` / 400; `$exists` and `$empty` stay refused. Corrected answers: a row with no value now satisfies `$ne`, `$nin` and the negation of an equality even when the comparand is the text `"null"` or `"undefined"`, which this face used to compare as text against the missing value — the answer every data driver gives. Its bare-day bounds answer as before.
20+
21+
Compiled SQL for `$ne`, `$nin`, `$notContains` and a `$not` operand now carries the lowering's NULL guard around each face's own copy of it: the same rows, a longer statement, until those copies are deleted.
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
'@objectstack/driver-memory': minor
3+
---
4+
5+
`MemoryAnalyticsService` (the in-memory analytics cube face) now runs the two shared filter comparand doors every other analytics face runs, then the shared filter lowering of ADR-0053 D-D1 (as amended), before it compiles a query's `where`. It also compiles `$or` and `$null`, the two parts of the lowering's output it could not.
6+
7+
Clause-②: yes (narrowing)
8+
9+
<!-- adr-0087: not-required (no-migration-prescription) Nothing authorable changes spelling or type: no spec key, export name or stored row moves. What changes is which filter comparand values one runtime face of this package accepts, to the set every other face already accepts; which value the caller meant by a refused comparand is not something a ledger entry can decide. The package publishes (not `unpublished`); no ADR-0087 id covers a comparand check (not `registered` / `already-registered`); and the change is runtime behaviour, not a declaration (not `runtime-interface-only` / `type-surface-only`). -->
10+
11+
**BREAKING**: `query()` and `generateSql()` now refuse, with `INVALID_FILTER` / 400, filter comparands they used to answer. Each is refused the same way by every other analytics face and by the query engine, so a filter that worked here worked nowhere else. Measured on a fixture where `d` is `'v1'`, `'v2'`, `null` and absent:
12+
13+
- `undefined` in any comparand position — `{ d: undefined }` and `{ d: { $eq: undefined } }` answered the no-value rows, `{ d: { $ne: undefined } }` the valued ones, `{ d: { $in: ['v1', undefined] } }` row `v1`;
14+
- a `null` member of `$in` / `$nin` (`{ d: { $in: ['v1', null] } }` answered `v1` and both no-value rows), and a `null` under an ordering operator (`{ d: { $gt: null } }` answered no row);
15+
- a scalar where `$in` / `$nin` takes a list (`{ d: { $in: 'v1' } }`);
16+
- a plain object, a `Map` or a binary value as a comparand — `{ d: { $ne: { a: 1 } } }` and `{ d: new Map() }` answered EVERY row.
17+
18+
The fix is to write the comparand you mean: `null` or `{ $null: true }` for "has no value", a list for `$in` / `$nin` (and `{ $null: true }` in a `$or` for "one of these, or no value"), a scalar for an ordering operator.
19+
20+
Corrected answers, each now what the live query path (`find()`) returns:
21+
22+
- a bigint comparand within 2^53 (`{ n: { $gt: 2n } }`) is read as its number and answered; beyond 2^53 it is refused `INVALID_FILTER` / 400. Both used to fail with an uncoded error.
23+
- `$between` is answered as its two bounds, with a bare-day maximum widened to the whole day before it is converted to the field's storage form; it was refused.
24+
- `$null` (true: no value; false: has a value) and `$or` (a `{}` branch is TRUE, `$or: []` is FALSE) are compiled on both exits; they were refused. `$not`, `$startsWith`, `$endsWith` and `$empty` stay refused. `ANALYTICS_FILTER_CAPABILITIES` names `$null` and the `$or` combinator accordingly.
25+
26+
The `generateSql()` echo and the `query()` pipeline dump for `$ne`, `$nin` and `$notContains` now show the lowering's NULL escape around this face's own guard — `(d IS NULL OR (d IS NULL OR d != 'v1'))` — the same rows as before.

‎packages/drivers/driver-memory/src/memory-analytics-20661-lte-whole-day-first.test.ts‎

Lines changed: 14 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -148,18 +148,20 @@ describe('[#20661] the siblings on this face', () => {
148148
expect(ids(result.rows)).toEqual(['r27', 'r28']);
149149
});
150150

151-
it('`$between` is refused on both exits, so its maximum never reaches a bound here', async () => {
152-
// Not a lowering this face owns: widening `MONGO_TO_CUBE_OPERATOR` to take
153-
// `$between` turns this red, and its maximum then owes the same whole-day
154-
// order as the `lte` rows (`lteUpperBound`).
155-
const { service } = await setup('declared');
156-
const between = cubeQuery({ created_at: { $between: ['2026-07-01', '2026-07-28'] } });
157-
for (const exit of [() => service.query(between), () => service.generateSql(between)]) {
158-
const err = await exit().then(
159-
() => undefined,
160-
(e: unknown) => e as { code?: unknown; status?: unknown },
161-
);
162-
expect(err).toMatchObject({ code: 'INVALID_FILTER', status: 400 });
151+
it('`$between` reaches the face as its two bounds, and its maximum is widened BEFORE it is converted', async () => {
152+
// [ADR-0053 D-D1, amended — #5930 step 3] This row used to pin `$between`
153+
// refused on both exits, and said what widening it would owe: the same
154+
// whole-day order as the `lte` rows. The face's door now runs the shared
155+
// lowering first, which splits the range and widens the AUTHORED maximum
156+
// in the calendar-string domain (`$lt: '2026-07-29'`); the exit converts
157+
// that bound to the storage form like any comparand. D-E3's order holds by
158+
// construction, declared or not.
159+
for (const declaration of ['undeclared', 'declared'] as const) {
160+
const got = await answer(declaration, { created_at: { $between: ['2026-07-01', '2026-07-28'] } });
161+
expect(got.cube, declaration).toEqual(got.find);
162+
expect(got.cube, declaration).toEqual(['r27', 'r28']);
163163
}
164+
expect((await answer('declared', { created_at: { $between: ['2026-07-01', '2026-07-28'] } })).echo)
165+
.toBe("created_at >= '2026-07-01T00:00:00.000Z' AND created_at < '2026-07-29T00:00:00.000Z'");
164166
});
165167
});

‎packages/drivers/driver-memory/src/memory-analytics-echo-operator-coverage.test.ts‎

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -177,14 +177,27 @@ const ACCEPTED_CASES: Record<string, FilterCondition> = {
177177
$icontains: { name: { $icontains: 'industries' } },
178178
$notContains: { name: { $notContains: 'Industries' } },
179179
$exists: { name: { $exists: true } },
180+
// [ADR-0053 D-D1, amended — #5930 step 3] `$null` joined the face's table:
181+
// the shared lowering emits it. Row 6's NULL `name` makes it a real predicate.
182+
$null: { name: { $null: true } },
180183
};
181184

182185
/**
183-
* The spellings this face REFUSES — the complement, so the two sets are total.
184-
* [#20446] `$empty` joined `FILTER_OPERATORS`, and this cube face refuses it as
185-
* it refuses `$null`: a declared operator it has no lowering for.
186+
* [ADR-0053 D-D1, amended — #5930 step 3] The spellings the face's door LOWERS
187+
* before its vocabulary is asked: a `$between` reaches the face as the two
188+
* bounds the shared lowering splits it into. Not in the face's table and not
189+
* refused — so a third set, held to the same executed-echo invariant.
186190
*/
187-
const REFUSED_OPERATORS = ['$between', '$startsWith', '$endsWith', '$null', '$empty'] as const;
191+
const LOWERED_CASES: Record<string, FilterCondition> = {
192+
$between: { amount: { $between: [20, 40] } },
193+
};
194+
195+
/**
196+
* The spellings this face REFUSES — the complement, so the three sets are total.
197+
* [#20446] `$empty` joined `FILTER_OPERATORS`, and this cube face refuses it: a
198+
* declared operator it has no lowering for.
199+
*/
200+
const REFUSED_OPERATORS = ['$startsWith', '$endsWith', '$empty'] as const;
188201

189202
describe('[#7117] the analytics echo renders the query it describes', () => {
190203
let db: any;
@@ -441,8 +454,8 @@ describe('[#7117] the analytics echo renders the query it describes', () => {
441454
// ── The vocabulary, enumerated so the two tables cannot drift apart ────────
442455

443456
describe('the closed vocabulary, enumerated', () => {
444-
it('the accepted and refused sets together are the whole Filter Protocol', () => {
445-
expect(sortIds([...Object.keys(ACCEPTED_CASES), ...REFUSED_OPERATORS]))
457+
it('the accepted, lowered and refused sets together are the whole Filter Protocol', () => {
458+
expect(sortIds([...Object.keys(ACCEPTED_CASES), ...Object.keys(LOWERED_CASES), ...REFUSED_OPERATORS]))
446459
.toEqual(sortIds([...FILTER_OPERATORS]));
447460
});
448461

@@ -451,7 +464,7 @@ describe('[#7117] the analytics echo renders the query it describes', () => {
451464
.toEqual(sortIds([...ANALYTICS_FILTER_CAPABILITIES.fieldOperators]));
452465
});
453466

454-
for (const [op, where] of Object.entries(ACCEPTED_CASES)) {
467+
for (const [op, where] of Object.entries({ ...ACCEPTED_CASES, ...LOWERED_CASES })) {
455468
it(`${op}: running the echo returns exactly the rows the query returns`, async () => {
456469
const executed = await executedIds(where);
457470
const echoed = await echoIds(where);

‎packages/drivers/driver-memory/src/memory-analytics-filter-refusal.test.ts‎

Lines changed: 45 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,14 @@
2424
* Both public entry points are covered here — `query()` and `generateSql()` —
2525
* because both call `normalizeFilters` and a refusal on one only would leave the
2626
* other silently answering the old way.
27+
*
28+
* [ADR-0053 D-D1, amended 2026-09-30 — #5930 step 3] Three of those shapes are
29+
* no longer refusals: the face now COMPILES `$or` and `$null` (the shared
30+
* lowering emits both), and a `$between` reaches its vocabulary gate as the two
31+
* bounds the lowering splits it into. Each is pinned below as answered with the
32+
* rows the live query path gives it — the invariant this file has always held,
33+
* refuse-or-agree, never a different row set. `$not`, `$startsWith` and
34+
* `$endsWith` stay refused.
2735
*/
2836

2937
import { describe, it, expect, beforeEach } from 'vitest';
@@ -104,23 +112,30 @@ describe('[#5345] MemoryAnalyticsService — filters it cannot compile are refus
104112

105113
// ── The combinators (`continue` #1) ────────────────────────────────────────
106114

107-
it('refuses $or instead of aggregating the whole table', async () => {
115+
// [#5930 step 3] `$or` is compiled now, so the regression this section was
116+
// written against — three rows asked for, five returned — is pinned as its
117+
// answer: the rows the live query path returns, never the whole table.
118+
const findCount = async (where: FilterCondition): Promise<number> => (await driver.find('deals', { where })).length;
119+
120+
it('compiles $or instead of aggregating the whole table', async () => {
108121
const where: FilterCondition = { $or: [{ stage: 'won' }, { stage: 'lost' }] };
109-
const err = await expectRefusal(() => count(where), '$or', 'where.$or');
110-
// The regression in one line: three rows were asked for, five were returned.
111-
expect(err.message).toContain('WIDENS');
122+
expect(await count(where)).toBe(3);
123+
expect(await count(where)).toBe(await findCount(where));
112124
});
113125

114-
it('refuses a $or nested inside a $and it CAN compile', async () => {
126+
it('compiles a $or nested inside a $and', async () => {
115127
const where: FilterCondition = { $and: [{ owner: 'u1' }, { $or: [{ stage: 'won' }, { stage: 'open' }] }] };
116-
await expectRefusal(() => count(where), '$or', 'where.$and[1].$or');
128+
expect(await count(where)).toBe(2);
129+
expect(await count(where)).toBe(await findCount(where));
117130
});
118131

119-
it('refuses a $or that sits beside a compilable sibling key', async () => {
120-
// The sharpest shape: the sibling `owner` lowered fine, so the query ran and
121-
// returned a plausible-looking number computed without the $or at all.
132+
it('compiles a $or that sits beside a sibling key, ANDed with it', async () => {
133+
// The sharpest shape when `$or` was dropped: the sibling `owner` lowered
134+
// fine, so the query ran and returned a plausible-looking number computed
135+
// without the $or at all. Compiled, it narrows.
122136
const where: FilterCondition = { owner: 'u1', $or: [{ stage: 'won' }, { stage: 'lost' }] };
123-
await expectRefusal(() => count(where), '$or');
137+
expect(await count(where)).toBe(2);
138+
expect(await count(where)).toBe(await findCount(where));
124139
});
125140

126141
it('refuses $not — the shape an RLS read scope compiles to', async () => {
@@ -130,19 +145,31 @@ describe('[#5345] MemoryAnalyticsService — filters it cannot compile are refus
130145
});
131146

132147
it('names the combinators it CAN compile, so the refusal is actionable', async () => {
133-
const err = await expectRefusal(() => count({ $or: [{ stage: 'won' }] }));
134-
expect(err.message).toContain('Supported combinators on this surface: $and');
148+
const err = await expectRefusal(() => count({ $not: { stage: 'won' } }));
149+
expect(err.message).toContain('Supported combinators on this surface: $and, $or');
135150
});
136151

137152
// ── The unmapped operators (`continue` #2) ─────────────────────────────────
138153

139154
const UNCOMPILABLE: Array<{ op: string; where: FilterCondition }> = [
140-
{ op: '$between', where: { amount: { $between: [100, 200] } } },
141155
{ op: '$startsWith', where: { name: { $startsWith: 'al' } } },
142156
{ op: '$endsWith', where: { name: { $endsWith: 'ta' } } },
143-
{ op: '$null', where: { closed_at: { $null: true } } },
144157
];
145158

159+
// [#5930 step 3] Answered, not refused: `$null` joined this face's table, and
160+
// a `$between` reaches it as its two bounds. Each agrees with the live path.
161+
const NOW_COMPILED: Array<{ op: string; where: FilterCondition; rows: number }> = [
162+
{ op: '$between', where: { amount: { $between: [100, 200] } }, rows: 2 },
163+
{ op: '$null', where: { closed_at: { $null: true } }, rows: 2 },
164+
];
165+
166+
for (const { op, where, rows } of NOW_COMPILED) {
167+
it(`compiles ${op}, with the rows the live query path returns`, async () => {
168+
expect(await count(where)).toBe(rows);
169+
expect(await findCount(where)).toBe(rows);
170+
});
171+
}
172+
146173
for (const { op, where } of UNCOMPILABLE) {
147174
it(`refuses ${op} — declared by the Filter Protocol, not compilable by this face`, async () => {
148175
const err = await expectRefusal(() => count(where), op);
@@ -221,17 +248,17 @@ describe('[#5345] MemoryAnalyticsService — filters it cannot compile are refus
221248
() => service.generateSql(asQuery({
222249
cube: 'deals',
223250
measures: ['deals.count'],
224-
where: { $or: [{ stage: 'won' }, { stage: 'lost' }] },
251+
where: { $not: { stage: 'lost' } },
225252
})),
226-
'$or',
253+
'$not',
227254
);
228255
await expectRefusal(
229256
() => service.generateSql(asQuery({
230257
cube: 'deals',
231258
measures: ['deals.count'],
232-
where: { amount: { $between: [1, 2] } },
259+
where: { name: { $startsWith: 'al' } },
233260
})),
234-
'$between',
261+
'$startsWith',
235262
);
236263
const ok = await service.generateSql(asQuery({
237264
cube: 'deals',

0 commit comments

Comments
 (0)