Repository navigation
Commit 793fb83
Fixes #20810
Clause-②: yes (narrowing)
#5930 step 3, under ADR-0053 D-D1 as amended by #20754: the one shared
`FilterCondition → FilterCondition` lowering (`lowerFilterCondition`,
`@objectstack/spec/data`, landed in step 2 as #20794) now runs at the
three analytics seams, after the comparand doors and after filter-token
resolution, and the two faces that could not compile its output now can.
`packages/spec/src/**`, the interim window arms and #20807's position
are untouched.
## Named gap: three pins outside the claim's file surface
`packages/drivers/driver-memory/src/memory-driver-filter-logic-conformance.test.ts`
holds three pins of the cube face's PRE-change vocabulary and shape, and
it is outside the claim's `memory-analytics*.test.ts` surface, so this
branch does not edit it. They are red here, and the `driver-memory` Test
Core shard is red until they move:
- "every combinator case is refused rather than dropped" asserts every
`$or` case refuses. The face compiles `$or` now; the same file's
case-by-case invariant (agree with `find()` or refuse) stays green over
every `$or` case.
- "every operator this face DECLARES compiles to a predicate that agrees
with find()" needs a probe for `$null`, which joined the face's table.
- the `$notContains` pipeline-dump row pins the un-lowered `$match`; the
lowering's NULL escape now wraps it.
The three-row patch (15 changed lines) is prepared and was verified on a
copy of the file: 125 of 125 tests pass. It lands once the file is added
to the surface.
## Per seam
| Seam | Position at `bbe03f406` | Pin | Ablation (lowering removed at
that seam) |
|---|---|---|---|
| analytics `where` door, F10 (`where` → tree, both strategies) |
`normalizeWhereComparands` `filter-normalizer.ts:2218`, reached through
`lowerAnalyticsWhere` `:2244`; the lowering sits in
`normalizeAnalyticsFilterTree` `:2356`, the one compile entry both
spellings reach | `where-door-shared-lowering-seam.test.ts`, 18 rows | 6
of 18 red: the tree rows, the ObjectQL hand-off and the echo |
| draft-preview door, F11 | `preview-evaluator.ts:675`, right after its
`normalizeWhereComparands` call | same file (preview block) | 3 of 18
red: the three NULL-escape rows |
| read scope | entry of `compileScopedFilterToSql`
`read-scope-sql.ts:747`, after placeholder resolution `:764`, before
`compileNode` `:769` | `read-scope-shared-lowering-seam.test.ts`, 12
rows | 5 of 12 red: the caller-filter rows, the RLS bound as written,
the SQL row and the date-macro row |
| cube face F5, the new door | `normalizeFilters`
`memory-analytics.ts:1585`, ahead of its gate `:1592` |
`memory-analytics-shared-lowering-door.test.ts`, 13 rows | doors
removed: 4 of 13 red (#20734's table); lowering removed: 2 of 13 red |
| the where door's nested-relation spelling (below) |
`normalizeAnalyticsFilterTree` | the nested-relation rows in two files |
2 red |
Every ablation mutated the committed file through
`scripts/ablation-replace.mjs` (anchor hit once, blob moved), ran the
pins, and restored: blob equal to HEAD and `git diff HEAD` empty, each
time. The pins import their subjects by relative path, so vitest reads
`src/` and no build sits between the mutation and the reading.
## What each seam does
- **Read scope.** `compileScopedFilterToSql` lowers the scope right
after its placeholders resolve. Column-type scope (item 7): the
`declaredValueShape` option both consumers already pass, so a declared
`datetime` column is rewritten and a `date`, `time` or text column
compiles byte-identical; no declarations handed in reads no column as
`datetime`. The shared comparand doors still judge the scope as written
after compilation (#20018's order); the lowering never refuses, so no
verdict moves.
- **Analytics `where` door (F10).** `normalizeAnalyticsFilterTree`
lowers what the door admitted before `buildNode` reads it. Its
column-type reader is now a REQUIRED argument: both strategies pass
`declaredDatetimeLowering` (a member is `datetime` when its column is
declared so, through the `declaredFieldType` hook, asked of the same
target each strategy compiles against); a context without the hook, and
the member-only readers (`assertWhereFields`, the cross-object view),
pass `NO_DATETIME_COLUMNS`. `lowerAnalyticsWhere` stays un-lowered: its
other readers (ad-hoc cube dimension minting, the routing detectors)
read the authored condition.
- **Nested relations at the `where` door.** The shared lowering has no
reading of the nested-relation spelling (`{ account: { region: 'NA' }
}`: accepted by the schema, refused by the engine, flattened only by
this door). Inside a `$not` it read `account` as a column and guarded
it. The door now spells nested relations as the dotted members
`fieldLeaves` has always compiled them to, before the lowering reads the
condition, so the guard lands on `account.region`.
- **Draft preview (F11).** Lowered after its `normalizeWhereComparands`
with `NO_DATETIME_COLUMNS`: drafted rows carry no schema, and a
type-blind rewrite would move one cell away from the typed drivers
(`$lte` on the last supported day over a non-temporal value that sorts
above it); its own `lteBound` copy keeps the whole-day rule until its
deletion card. It now evaluates `$null`.
- **Cube face (F5).** `normalizeFilters` runs
`assertListComparandShapes`, then `normalizeFilterComparandTypes` (its
return is the condition read from there on), then the lowering
(type-blind: the face reaches declared types only as a storage-form
conversion, and its own copy and `find()` are type-blind already), then
its own gate on the lowered condition. It compiles `$or` (a disjunction
entry both exits render, with the #5322 identities) and `$null`; `$not`,
`$startsWith`, `$endsWith` and `$empty` stay refused.
## Every corrected answer
Read scope (`compileScopedFilterToSql`: the NativeSQL statement's scope
and the `/analytics/sql` echo's), each now `SqlDriver.find`'s rows on
the same filter:
- `{ signed_at: { $lte: '2026-07-28' } }` on a declared `datetime` over
rows at 10:00Z on 07-27, 07-28, 07-29 and a row with no value: was
07-27, now 07-27 and 07-28 (`< '2026-07-29'`). #20733's caller-filter
half.
- `{ signed_at: { $between: ['2026-07-28', '2026-07-28'] } }`: was no
row, now 07-28.
- a `{today}` upper bound is widened as the day it resolves to.
Analytics `where`, ObjectQL path: a bare-day `$lte` on a `datetime`
member reaches the engine as `$lt` the next day, and the
`/analytics/sql` echo prints that half-open bound where it printed `<=`
the named day. Rows unchanged.
Draft preview: `$null` answered (was refused 400). A row with no value
now satisfies `$ne`, `$nin` and the negation of an equality when the
comparand is the text `"null"` or `"undefined"`; the face compared those
as text against the missing value.
Cube face, measured on the published `MemoryAnalyticsService.query` at
the base blob and at this head:
| `where` (fixture: `d` is `v1`, `v2`, null, absent) | base | this head
| `find()` |
|---|---|---|---|
| `{d: undefined}` / `{d: {$eq: undefined}}` | 3, 4 | refused 400 | 3, 4
|
| `{d: {$ne: undefined}}` | 1, 2 | refused 400 | 1, 2 |
| `{d: {$in: ['v1', undefined]}}` | 1 | refused 400 | 1 |
| `{d: {$in: ['v1', null]}}` | 1, 3, 4 | refused 400 | 1, 3, 4 |
| `{d: {$nin: ['v1', null]}}` | 2 | refused 400 | 2 |
| `{d: {$gt: null}}` | none | refused 400 | none |
| `{d: {$in: 'v1'}}` | 1 | refused 400 | uncoded throw |
| `{d: {$eq: {a: 1}}}` | none | refused 400 | none |
| `{d: {$ne: {a: 1}}}` | all four | refused 400 | all four |
| `{d: new Map()}` | all four | refused 400 | none |
| `{n: {$gt: 2n ** 60n}}` | uncoded throw | refused 400 | uncoded throw
|
| `{n: {$gt: 2n}}` | uncoded throw | 3, 4 | uncoded throw |
| `{d: {$between: ['v1', 'v2']}}` | refused 400 | 1, 2 | 1, 2 |
| `{d: {$null: true}}` | refused 400 | 3, 4 | 3, 4 |
| `{$or: [{d: 'v1'}, {n: 4}]}` | refused 400 | 1, 4 | 1, 4 |
| `{d: {$ne: 'v1'}}`, `{$not: {d: 'v1'}}` | unchanged | unchanged | — |
(`find()` here is the driver called directly, without the engine seam
whose doors it relies on.) Every "refused 400" is the ADR-0112
`INVALID_FILTER` envelope every other analytics face already answers.
## Clause-②, measured
- **Widening:** the cube face accepts `$or`, `$null` and `$between`
(each refused at the base, each now `find()`'s rows), and the draft
preview accepts `$null`. `ANALYTICS_FILTER_CAPABILITIES` names `$null`
and `$or`.
- **Narrowing:** the cube face refuses the comparand shapes in the table
above that it used to answer.
Hence `yes (narrowing)`. `@objectstack/driver-memory` ships `minor` with
the BREAKING banner, the migration and an ADR-0087 `not-required
(no-migration-prescription)` disposition;
`@objectstack/service-analytics` ships `minor` (`Clause-②: yes`,
widening only: the read scope and the `where` door refuse nothing new).
## Mechanism assumptions, measured
- **A1 held**, with one refinement: the F10 call sits in
`normalizeAnalyticsFilterTree` rather than inside
`normalizeWhereComparands`, because the array spelling reaches F10
through `parseFilterAST` and never through `normalizeWhereComparands`,
and because `lowerAnalyticsWhere`'s other readers want the authored
condition. F11 lowers right after its own `normalizeWhereComparands`
call.
- **A2 measured.** `where` door: tokens resolve upstream on every path
that reaches it (`AnalyticsService.resolveQueryTokens` from `query()`
and `generateSql()`, the per-request dataset-scope getter,
`DatasetExecutor.resolveSelectionTokens` ahead of the preview). Read
scope: placeholder resolution is the compiler's first act, and the doors
run after `compileNode` on the scope as written (#20018), so the
lowering sits between resolution and compilation. F5: no token
resolution exists on that face.
- **A3 proved, not assumed.** With the real `SecurityPlugin` and the
real read-scope compiler on SQLite (a one-off run, not committed):
`getReadFilter` returned
`{"$and":[{"signed_at":{"$lt":"2026-07-29"}},{"due_on":{"$lte":"2026-07-28"}}]}`,
the read scope compiled `("t"."signed_at" < ? AND "t"."due_on" <= ?)`
and answered c27 and c28, equal to `SqlDriver.find`. The committed pin
holds the RLS half in both spellings the read scope can be handed
(lowered by the RLS seam, and as written), so its answer no longer
depends on which arrives.
- **A4 measured.** F5 compiled 12 operators with `$and` alone; F11 10
operators with `$and`/`$or`/`$not`. The lowering emits `$and`, `$or`,
`$lt`, `$gte`, `$lte`, `$null`. Widened: F5 `$or` + `$null`; F11
`$null`. Nothing wider.
- **A5 held.** No window arm is touched (NativeSQL, the preview's window
loop, F5's `timeDimensions`).
- **A6** as the table above: #20733's rows (both halves, the last day,
the declared `date` control) and #20734's table, plus a pin and an
ablation per seam.
- **A7** above.
- **A8, coupling, re-measured:** #20280 landed on `main` as `05a7547c9`
while this branch was open. Its datetime year floor lives in the
engine's temporal comparand door (`objectql`
`temporal-comparand-door.ts`) and in `@objectstack/core`'s temporal
storage form; in `packages/spec/src/data` it changed one comment. The
two spec doors the cube face now runs are not changed by it. What the
two share is the storage-form conversion both of the face's exits apply
to a comparand (the driver's `filterComparandStorageForm`, which reads
`@objectstack/core`): #20280 changed that conversion, and this branch
neither touches nor pre-empts it. Nothing in this PR's file surface
changed on `main`, so the branch is not re-merged; this PR's CI runs on
the merge ref with #20280 in it, and the merge queue adjudicates the
rest.
## Evidence (head `92a449c2d`, `main` merged at `c90f9fb6e`)
- `@objectstack/service-analytics`: 143 files, 3297 tests passed;
`typecheck` green (baseline at `bbe03f406`: 141 files, 3267).
- `@objectstack/driver-memory`: 66 files, 1482 passed, 3 failed (the
named gap above); `typecheck` green (baseline 65 files, 1470).
- Existing shape pins moved to the lowered structure; no asserted row
count moved. The row-level conformance suites (native-SQL filter-logic
and temporal, read-scope conformance, preview temporal) are unchanged
and green.
- Gates: `node scripts/pm/dispatch-gates.mjs --commands` derived 63
families at this head; all 63 were run with their exit codes recorded,
and `--ran` reports 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN (a
derived zero). `check:dual-build-cjs-loads` and `check:type-check-debt`
first answered `PREREQUISITE NOT MET` and were re-run green after `turbo
run build --filter='./packages/*' --filter='./packages/*/*'`. The six
roster families under this card's directories (`check-changeset-fixed`,
`check:authz-resolver`, `check:error-code-casing`,
`check:filter-alias-parity`, `check:object-def-param-keys`,
`check:tenant-chokepoint`) are green too.
- Downstream analytics suites on the rebuilt dists: `@objectstack/rest`
`analytics-*` (10 files, 150 tests), `@objectstack/runtime`
`analytics-*` (3 files, 23), `@objectstack/dogfood`
`analytics-adhoc-query-isolation` (24): green.
- Lint, a narrowed run: the population is the `eslint.config.mjs` block
`files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']`; `eslint
--no-inline-config --format json` over the 38 changed source files
reports 38 files, 0 errors, 0 warnings at `92a449c2d`; the config
enables no type-aware linting (every `parserOptions` is `ecmaVersion` /
`sourceType`), so this diff cannot move a verdict on an untouched file.
- NOT MEASURED: live PostgreSQL / MySQL (CI's temporal job), a real
mongod.
## Acceptance notes
- **Double guards.** Each face's own interim NULL-polarity copy still
guards what the lowering already guarded, so the SQL, the trees and the
ObjectQL hand-off for `$ne`, `$nin`, `$notContains` and a `$not` operand
carry the guard twice. The same rows; the deletion cards remove the
inner copy and update these pins.
- **For the step-4 deletion cards.** The draft preview reads no member
as `datetime`; the `where` door with no `declaredFieldType` hook, and
the read scope with no `declaredValueShape`, read none. Once a face's
own bound copy is deleted, that path gets no whole-day bound unless its
card gives it a typed reader.
- **The read scope's temporal coercion (ADR-0053 D-A1, #20733's other
half).** The read scope binds the lowered calendar string as written, as
it bound the authored one. Measured correct on SQLite ISO text;
PostgreSQL / MySQL NOT MEASURED. Carrier: none.
- **The shared lowering and the nested-relation spelling.** The spec
module reads a nested-relation spec under a `$not` as a column
constraint. No face meets that after this change (the analytics door
spells it dotted; the engine, the read scope and the preview refuse the
spelling; the cube face refuses `$not`). Noted for the module's owner;
carrier: none.
## Patch round 1 (appended by the `domain:services` seat)
**The named gap is closed.** The claim's surface gained
`packages/drivers/driver-memory/src/memory-driver-filter-logic-conformance.test.ts`
(amendment `5911719808`), and `edb3e2de4` applies the prepared 15-line
patch to it and to nothing else:
- every `$not` case must still refuse; the `$or` cases moved to the
answered column, which the file's case-by-case invariant holds to
`find()`'s rows;
- `$null` joins the declared-operator probe roster;
- the `$notContains` pipeline dump shows the lowering's NULL escape.
Evidence on head `edb3e2de4`:
- `@objectstack/driver-memory`: 66 files, 1485 passed (the three pins
green, nothing else moved); `typecheck` green.
- `@objectstack/service-analytics`: 143 files, 3297 passed.
- Gates: the same 63 derived families, all exit 0; `--ran` reports 63
derived, 63 run, 0 NOT-MEASURED, 0 UNRUN; the six roster families green.
- `main` was not re-merged: none of the five commits after `c90f9fb6e`
touches this PR's files.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent 856321f commit 793fb83
41 files changed
Lines changed: 1490 additions & 234 deletions
File tree
- .changeset
- packages
- drivers/driver-memory/src
- services/service-analytics/src
- __tests__
- strategies
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
151 | | - | |
152 | | - | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | | - | |
157 | | - | |
158 | | - | |
159 | | - | |
160 | | - | |
161 | | - | |
162 | | - | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
163 | 163 | | |
| 164 | + | |
| 165 | + | |
164 | 166 | | |
165 | 167 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
177 | 177 | | |
178 | 178 | | |
179 | 179 | | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
180 | 183 | | |
181 | 184 | | |
182 | 185 | | |
183 | | - | |
184 | | - | |
185 | | - | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
186 | 190 | | |
187 | | - | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
188 | 201 | | |
189 | 202 | | |
190 | 203 | | |
| |||
441 | 454 | | |
442 | 455 | | |
443 | 456 | | |
444 | | - | |
445 | | - | |
| 457 | + | |
| 458 | + | |
446 | 459 | | |
447 | 460 | | |
448 | 461 | | |
| |||
451 | 464 | | |
452 | 465 | | |
453 | 466 | | |
454 | | - | |
| 467 | + | |
455 | 468 | | |
456 | 469 | | |
457 | 470 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
27 | 35 | | |
28 | 36 | | |
29 | 37 | | |
| |||
104 | 112 | | |
105 | 113 | | |
106 | 114 | | |
107 | | - | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
108 | 121 | | |
109 | | - | |
110 | | - | |
111 | | - | |
| 122 | + | |
| 123 | + | |
112 | 124 | | |
113 | 125 | | |
114 | | - | |
| 126 | + | |
115 | 127 | | |
116 | | - | |
| 128 | + | |
| 129 | + | |
117 | 130 | | |
118 | 131 | | |
119 | | - | |
120 | | - | |
121 | | - | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
122 | 136 | | |
123 | | - | |
| 137 | + | |
| 138 | + | |
124 | 139 | | |
125 | 140 | | |
126 | 141 | | |
| |||
130 | 145 | | |
131 | 146 | | |
132 | 147 | | |
133 | | - | |
134 | | - | |
| 148 | + | |
| 149 | + | |
135 | 150 | | |
136 | 151 | | |
137 | 152 | | |
138 | 153 | | |
139 | 154 | | |
140 | | - | |
141 | 155 | | |
142 | 156 | | |
143 | | - | |
144 | 157 | | |
145 | 158 | | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
146 | 173 | | |
147 | 174 | | |
148 | 175 | | |
| |||
221 | 248 | | |
222 | 249 | | |
223 | 250 | | |
224 | | - | |
| 251 | + | |
225 | 252 | | |
226 | | - | |
| 253 | + | |
227 | 254 | | |
228 | 255 | | |
229 | 256 | | |
230 | 257 | | |
231 | 258 | | |
232 | | - | |
| 259 | + | |
233 | 260 | | |
234 | | - | |
| 261 | + | |
235 | 262 | | |
236 | 263 | | |
237 | 264 | | |
| |||
0 commit comments