Skip to content

Commit e8fcf55

Browse files
fix(spec)!: a dataset or measure filter with a list inside a nested relation is refused on save (#20080) (#20207)
Fixes #20080 Clause-②: no ## What this changes A dataset `filter` and a measure `filter` now refuse, when the dataset is saved, a list in the equality slot of a field inside a nested-relation condition: `{ account: { region: ['a'] } }` and `{ account: { region: { $eq: ['a'] } } }`, the empty list included, at any relation depth and under `$and` / `$or` / `$not`. Before, at `origin/main` `9e7824a445`, `DatasetSchema.safeParse` answered `success: true` for both shapes. The analytics `where` door charts both carriers on every path (the native-SQL and ObjectQL strategies and the draft preview). It flattens the relation to the dotted member `account.region` and hands the list to the shared comparand-shape face, which refuses it with `INVALID_FILTER` / 400. So the dataset saved clean and every chart on it failed. This is remedy A as triage routed it (record 5825670610). The shared `FilterConditionSchema` is not touched: ruling A (record 5805248669) keeps its reach at the face's, because the engine reads a no-`$` field spec as a deep-equality comparand. Remedy B (the analytics door stops descending) is not taken. ## How - `packages/spec/src/ui/dataset.zod.ts`: `refuseNestedRelationEqualityLists` walks a filter the way the analytics door's `mapWhereFieldEntries` does. It descends `$and` / `$or` arrays and `$not`, skips every other `$` key, and descends a plain object with no `$` key as a nested relation, at any depth. The nested-relation test is the door's own `isNestedRelationSpec`: prototype `Object.prototype` or `null`, no `$` key. Inside a nested relation it refuses exactly what `assertNoListInEqualitySlot` refuses there: an array spec (implicit form), or an object spec whose `$eq` is an array. It reports ONLY inside a nested relation. Everything outside one is `FilterConditionSchema`'s own refusal, so nothing is reported twice. - The words come from `arrayEqualityComparandMessage`, the one builder the face and the schema door both print from. The analytics door appends ` at where.account.region`. The carrier leaves that out, as the schema door does, and the issue `path` carries it instead (`filter.account.region`, `measures.0.filter.account.region.$eq`). - The check sits on each carrier's `.optional()` wrapper, not on `FilterConditionSchema` itself. A first cut refined the recursive schema directly. That clones it, and the published `ui/Dataset.json` then inlined a second copy of the condition beside its `$ref`. On the wrapper, the published bodies of `ui/Dataset` and `ui/DatasetMeasure` are byte-identical to base, apart from `x-dropped-refinements`. That was measured by generating both trees and comparing the JSON with that key removed. - `packages/spec/dropped-refinements.baseline.json`: the new rule is declared where `z.toJSONSchema()` drops it. That is 11 new sites across 6 existing entries (`filter` on both carriers, `measures.element.filter`, and the four manifest embeddings), exactly as `build-schemas.ts` printed them. The header total moves 574 to 585. ## ADR-0087 registration A new semantic entry, `dataset-filter-nested-relation-equality-array-refused-at-save` (protocol major 18), plus its regenerated `registry.ts` region. The projections (`spec-changes.json`, `docs/protocol-upgrade-guide.md`) cover up to the current protocol major 17, so they do not move. `check:spec-changes` and `check:upgrade-guide` are green. Two unreleased major-18 sibling entries each had one sentence this change makes false, so each gets a minimal correction: - `filter-equality-array-comparand-refused-at-save`: its position (1) said the nested-relation list is refused only when charted. It now also names the carriers' refusal on save. Its acceptance criteria get the matching exception. - `filter-equality-array-comparand-refused`: "that door alone also refuses a list inside a nested-relation condition" now reads "among the runtime doors", and the save-side list names the two carriers. ## Deliberate correction of two pending changesets `Check Changeset` (`check-empty-changeset`) is red on purpose. It is the DELIBERATE CORRECTION class that gate's own text names, ⛔ not a collision, and no `skip-changeset` label is applied. Both pending notes described an exception this PR removes. Triage item 2 (record 5825670610) requires the correction to land in the same PR if this card lands before the next `changeset version`. - `.changeset/19889-filter-schema-door-array-equality.md`: the "reach is the face's" bullet said the nested-relation list "is not judged" and is refused when a dataset or measure filter is charted. It now says `FilterConditionSchema` does not judge it, and that the two carriers the analytics door charts refuse it on save. - `.changeset/19888-analytics-implicit-array.md`: "A document carrying that still publishes, and it is refused when it is charted" now says a dataset or measure filter is refused on save, and that any other `where` reaching the door, such as a caller `where` or a dataset selection `runtimeFilter`, is still refused only when charted. Please confirm both corrections. Restoring either note from base would republish a false sentence. This PR's own changeset is `.changeset/20080-dataset-filter-nested-relation-list.md` (`@objectstack/spec`: `minor`, BREAKING narrowing, `Clause-②: no (narrowing)`, ADR-0087 `registered`). `check-adr-0087-registration` and `check-changeset-no-major` are green. ## Evidence All runs below are at head `3ed391abf9` unless another sha is named. They went through `scripts/pm/os-verify-lock.sh`, and each exit code was captured before any pipe. - **New pins**, `packages/spec/src/ui/dataset-filter-nested-relation-list.test.ts`: 68 passed. - §1: 12 nested shapes are refused at the list's own path on `DatasetSchema.filter`, on a measure filter through `DatasetSchema`, and on `DatasetMeasureSchema` alone. One more case checks that a single document reports every such list. - §2: the carrier's message EQUALS the analytics door's refusal with the location clause removed. The door's refusal is rebuilt from the one-entry node it hands the face, carries envelope `INVALID_FILTER` / 400, and the clause is proven present exactly once. - §3: a list outside a nested relation is reported exactly ONCE. - §4: 13 controls are accepted and kept on both carriers. The shared `FilterConditionSchema` still accepts both nested shapes, so ruling A is untouched. - **Ablation 1** (via `scripts/ablation-replace.mjs`, anchor hit 1 to 0, restore proven): with the refinement removed, 50 failed and 18 passed. §1 and §2 went red; the §3 and §4 controls stayed green. Restore check: blob == HEAD `781a5b6df01f`, and `git diff HEAD` is empty. - **Ablation 2**: with the outside-a-relation guard removed, exactly the 4 §3 rows went red (4 failed, 64 passed). Restore proven the same way. - **Cross-door conformance** (one-off, not committed): 3000 seeded random filters went through the REAL analytics door (`normalizeAnalyticsFilterTree`, service-analytics source) and the rebuilt `DatasetSchema`. The door refused 2240 with the equality-slot sentence, and 1304 of those sit where the shared schema alone would miss them. The carrier agreed on all 3000, with 0 mismatches: the same sentence whenever the door refused, and no equality issue whenever it accepted. - `pnpm --filter @objectstack/spec test` at `43df1403dd`: 15902 passed, 1 failed, 2 todo. The failure was the ledger header total, fixed in `3ed391abf9`. That file (`scripts/dropped-refinements.test.ts`) plus the new pins, rerun at `3ed391abf9`: 95 passed. - `pnpm --filter @objectstack/spec typecheck`: exit 0. `check:test-typecheck` reports OK (53 files / 255 errors held in the ledger). `tsc -p tsconfig.test.json --listFiles` lists the new test file once, and 0 of the 255 errors sit in any touched file. - `pnpm --filter @objectstack/spec build && check:generated` at `43df1403dd`: all 15 generated artifacts up to date. `gen:schema` was rerun at `3ed391abf9` for the ledger: exit 0, 585 refinement sites across 207 published schemas, all declared in the ledger, 1549 schemas generated, and the tree left clean. - **Published JSON Schema**: base and head `ui/Dataset.json` / `ui/DatasetMeasure.json` were generated side by side (a compare worktree at `9e7824a445`). The bodies are byte-identical with `x-dropped-refinements` removed. - **Consumers.** Test-filter direction is the upstream prefix form, `--filter '@objectstack/service-analytics^...'`, which built its 14-package closure. - `pnpm --filter @objectstack/service-analytics test`: 128 files and 3017 tests passed, all 128 tracked test files. That includes the read-only control `where-equality-slot-list-refusal.test.ts`, so the analytics door's refusal is unchanged. - `packages/metadata` `typescript-serializer-annotation.test.ts`: 7 passed. - `packages/platform-objects` `report-form-echo-decisions.test.ts`: 35 passed. - `packages/rest` (3 files) and `packages/cli` (1 file) are declared to CI; their closures are 25 and 57 packages. Static sweep: none of the 80 consumer test files that reference `DatasetSchema` passes a list under a nested relation through it. The only such literals go straight to the analytics door or to read-scope compilers. - **Gates.** Derived with `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands` at `3ed391abf9` (identical to the derivation at `43df1403dd`). Reconciled with `--ran`: 87 derived, 85 run, 2 NOT MEASURED, 0 unrun. - 84 exit 0. - `check-empty-changeset` exits 1 BY DESIGN (see the deliberate-correction section). - NOT MEASURED: `check:type-check-debt` and `check:dual-build-cjs-loads`, both exit 3 (PREREQUISITE NOT MET: they need the whole-repo build, which CI does). - **Lint**, a declared narrowing: `eslint --no-inline-config --format json` over the 6 touched TS files reported 6 files, 0 errors, 0 warnings, none ignored. The repo's one `eslint.config.mjs` never enables type-aware linting (its own note, about line 326), and no touched module's exports changed, so no untouched file's verdict can move. The repo-wide `pnpm lint` is CI's. ## Census In-repo, at `9e7824a445`: 0 dataset or measure filters carry a list under a nested relation. The scan covered the dataset definitions of `examples/app-crm`, `examples/app-showcase` and `examples/app-todo`, the platform objects' `system.datasets.ts`, the dataset examples in `content/docs` and `skills`, and a repo-wide literal scan of non-test `filter` / `where` carriers. Deployed datasets: NOT MEASURED. ## Acceptance notes - **Same family, measured, not fixed here.** The shared face's OTHER arms are accepted by `DatasetSchema.filter` at the top level AND inside a nested relation: a `null` ordering comparand (`$gt: null`), a non-list `$in`, a `null` member of `$in`, and a `null` `$between` endpoint. The shared face refuses each with `INVALID_FILTER` / 400, and so does the analytics door when it charts the filter. Probe at `3ed391abf9`, with a control dataset that parses: `{ amount: { $gt: null } }`, `{ stage: { $in: 'won' } }`, `{ amount: { $between: [null, 5] } }` and their `{ acct: … }` nested forms all answer `success: true`. This is the #19889 / #20116 save-door family, and it is reported for the PM to fold into that family rather than filed singly. Dedupe words: `dataset filter $gt null accepted at save` · `FilterConditionSchema other comparand arms not judged at save` · `$in non-list dataset filter saves clean`. - **Other routes to the analytics door.** A caller `where` and a dataset selection `runtimeFilter` still reach the door with a nested list and are refused when charted. Both are requests, so the refusal is loud to the caller. Whether a stored dashboard widget `filter` or report `runtimeFilter` reaches the door that way is decided on the objectui side and was NOT MEASURED here. - **Stale header number, noted and not filed.** `dropped-refinements.baseline.json`'s `measured.refinementSitesThatDidProject` reads 369, while `build-schemas.ts` prints 409 at base `9e7824a445` and at head. No test reads that field. Nobody is carrying it. - The claim's `Clause-②: no` is copied above verbatim. This PR's changeset carries `Clause-②: no (narrowing)`, the spelling the ADR-0087 gate reads. --- _Generated by [Claude Code](https://claude.ai/code/session_01QcAS3qiYYZNezaxZxaUdMV)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 03d6cb0 commit e8fcf55

10 files changed

Lines changed: 610 additions & 24 deletions

‎.changeset/19888-analytics-implicit-array.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,4 +27,4 @@ Ruling 乙 of #19757 refuses a list in the equality slot at the shared comparand
2727

2828
Each list is now handed to the shared face's equality arm before any node is built. Both spellings of one condition therefore get the same refusal, with the same wording, path and `$in` prescription. The draft-data preview runs the same gate, so a drafted chart refuses what the published chart refuses. Before, it compared each row against the list's string form.
2929

30-
Who is affected: nothing in this repository's examples, seeds or docs authors the shape (measured over `examples/**`, `packages/**` and the fenced code in the docs). Stored datasets, dashboard widget filters, report runtime filters and measure filters in a deployment were NOT measured. In this release the authoring schema refuses the shape too, when such a document is saved (a separate change in `@objectstack/spec`, ADR-0087 entry `filter-equality-array-comparand-refused-at-save`). One position is judged only here: a list inside a nested relation, which this door flattens to a dotted member. A document carrying that still publishes, and it is refused when it is charted. The refusal names the field and the path. `$ne` with a list is not part of the ruling and is not judged here. The list operators (`$in`, `$nin`, `$between`) keep their lists, and every scalar, `null` included, compiles as before.
30+
Who is affected: nothing in this repository's examples, seeds or docs authors the shape (measured over `examples/**`, `packages/**` and the fenced code in the docs). Stored datasets, dashboard widget filters, report runtime filters and measure filters in a deployment were NOT measured. In this release the authoring schema refuses the shape too, when such a document is saved (a separate change in `@objectstack/spec`, ADR-0087 entry `filter-equality-array-comparand-refused-at-save`). One position is judged here and not by the shared authoring schema: a list inside a nested relation, which this door flattens to a dotted member. A dataset `filter` or measure `filter` carrying one is refused when it is saved, in the same words (a separate change in `@objectstack/spec`, ADR-0087 entry `dataset-filter-nested-relation-equality-array-refused-at-save`). Any other `where` that reaches this door with one, such as a caller `where` or a dataset selection's `runtimeFilter`, is refused only when it is charted. The refusal names the field and the path. `$ne` with a list is not part of the ruling and is not judged here. The list operators (`$in`, `$nin`, `$between`) keep their lists, and every scalar, `null` included, compiles as before.

‎.changeset/19889-filter-schema-door-array-equality.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ Two request doors parse these carriers, and they now answer before the analytics
2121
## What does NOT change
2222

2323
- **Nothing stored is rewritten, and nothing is dropped.** The parse fails and strips nothing. The read path does not re-validate stored rows, so a stored document keeps loading, and its next save is refused. Such a filter has failed every query since the equality-slot change, so the refusal is a repair.
24-
- **The reach is the face's, and no wider.** A field spec with no `$` key, such as the nested-relation condition `{ account: { region: ['a'] } }`, is not judged, because the face does not judge it either. The analytics `where` door does refuse that shape when a dataset or measure filter is charted.
24+
- **The reach is the face's, and no wider.** A field spec with no `$` key, such as the nested-relation condition `{ account: { region: ['a'] } }`, is not judged by `FilterConditionSchema`, because the face does not judge it either. The analytics `where` door does refuse that shape, because it flattens the relation to a dotted member. So the two carriers that door charts, the dataset `filter` and the measure `filter`, refuse it on save as well, in the same words. That is a separate change in this release, ADR-0087 entry `dataset-filter-nested-relation-equality-array-refused-at-save`.
2525
- **The data-engine calls' `where` option still parses.** Its type is a union whose first arm is an open record. The face refuses the shape when the call runs.
2626
- **`$ne` carrying an array is not judged.**
2727
- The list operators keep their arrays, `$in: []` and `$nin: []` included. Every scalar, `null`, a `Date` and a `{ $field }` reference pass as before.
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
---
2+
"@objectstack/spec": minor
3+
---
4+
5+
fix(spec)!: a dataset or measure filter with a list inside a nested relation is refused when it is saved, not when it is charted (#20080)
6+
7+
**BREAKING**: an accept-set narrowing of a published authoring schema, shipped as `minor` under the repo's launch-window convention for accept-set narrowings. The hand-migration prescription is registered under protocol major 18 as `dataset-filter-nested-relation-equality-array-refused-at-save`.
8+
9+
## What changes
10+
11+
`DatasetSchema.filter` and `DatasetMeasureSchema.filter` now refuse an ARRAY in the equality slot of a field inside a nested-relation condition. That covers the implicit form `{ account: { region: ['a'] } }` and the explicit form `{ account: { region: { $eq: ['a'] } } }`, the empty array included, at any relation depth and under `$and` / `$or` / `$not`. So `defineStack`, `os validate` and a save through the metadata protocol (`422 INVALID_METADATA`) refuse such a dataset at the filter's path, for example `filter.account.region` or `measures.0.filter.account.region.$eq`.
12+
13+
Before this change the dataset saved clean. The analytics `where` door, which charts both filters on every path, flattens the relation to the dotted member `account.region` and refuses the list with `INVALID_FILTER` / 400. So every chart built on the dataset failed. Measured on `origin/main` `9e7824a445`: `DatasetSchema.safeParse` answered `success: true` for both forms.
14+
15+
The refusal is the analytics door's sentence: the field, the received list, and the two operators a list in that slot stood in for. The door adds `at where.account.region`. The schema leaves that out, because the issue's `path` already says where it is.
16+
17+
## What does NOT change
18+
19+
- **The shared `FilterConditionSchema` keeps its reach.** Every other schema that carries a `FilterCondition` still accepts a list inside a nested relation, because the engine reads that spec as a deep-equality comparand.
20+
- **Nothing stored is rewritten, and nothing is dropped.** The parse fails and strips nothing. The read path does not re-validate stored rows, so a stored dataset keeps loading, and its next save is refused. Such a filter has failed every chart, so the refusal is a repair.
21+
- A list outside a nested relation is refused as before, once, by `FilterConditionSchema`.
22+
- `$ne` carrying an array is not judged. The list operators keep their arrays, `$in: []` and `$nin: []` included. Every scalar, `null`, a `Date` and a `{ $field }` reference inside a nested relation pass as before.
23+
24+
## FROM → TO
25+
26+
| you wrote | write instead |
27+
|:--|:--|
28+
| `{ account: { region: ['a', 'b'] } }`, meaning "one of these values" | `{ account: { region: { $in: ['a', 'b'] } } }` |
29+
| `{ account: { tags: ['a'] } }`, meaning "the stored list holds `a`" on a multi-value field | `{ account: { tags: { $contains: 'a' } } }` |
30+
| `{ account: { region: ['a'] } }`, meaning one value | `{ account: { region: 'a' } }` |
31+
| `{ account: { region: { $eq: [...] } } }` | any of the rows above |
32+
33+
## Who is affected, measured
34+
35+
Nothing shipped in this repository carries the shape. At `9e7824a445`, the dataset and measure filters of `examples/app-crm`, `examples/app-showcase`, `examples/app-todo` and the platform objects, plus the dataset examples in `content/docs` and `skills`, carry no list inside a nested relation. Deployed datasets were NOT measured. Validating each stack, or re-saving each dataset, finds every instance.
36+
37+
Clause-②: no (narrowing) — nothing is widened. No key is added, removed or renamed, no exported symbol moves, and the operator vocabulary is unchanged. One comparand shape in one position of two carriers, which the analytics door already refused on every chart, is now refused on save as well.
38+
39+
<!-- adr-0087: registered dataset-filter-nested-relation-equality-array-refused-at-save -->

‎packages/spec/dropped-refinements.baseline.json‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
"measured": {
44
"zod": "4.4.3",
55
"publishedSchemasWithDroppedRefinements": 207,
6-
"droppedRefinementSites": 577,
6+
"droppedRefinementSites": 588,
77
"refinementSitesThatDidProject": 369,
88
"refinementSitesWithNoJsonFormToCompare": 0
99
},
@@ -56,7 +56,9 @@
5656
"manifest.dashboards.element.globalFilters.element",
5757
"manifest.dashboards.element.widgets.element",
5858
"manifest.datasets.element",
59+
"manifest.datasets.element.filter",
5960
"manifest.datasets.element.include.element",
61+
"manifest.datasets.element.measures.element.filter",
6062
"manifest.datasources.element",
6163
"manifest.flows.element",
6264
"manifest.flows.element.errorHandling",
@@ -154,7 +156,9 @@
154156
"data.options[1].manifest.dashboards.element.globalFilters.element",
155157
"data.options[1].manifest.dashboards.element.widgets.element",
156158
"data.options[1].manifest.datasets.element",
159+
"data.options[1].manifest.datasets.element.filter",
157160
"data.options[1].manifest.datasets.element.include.element",
161+
"data.options[1].manifest.datasets.element.measures.element.filter",
158162
"data.options[1].manifest.datasources.element",
159163
"data.options[1].manifest.flows.element",
160164
"data.options[1].manifest.flows.element.errorHandling",
@@ -236,7 +240,9 @@
236240
"options[1].manifest.dashboards.element.globalFilters.element",
237241
"options[1].manifest.dashboards.element.widgets.element",
238242
"options[1].manifest.datasets.element",
243+
"options[1].manifest.datasets.element.filter",
239244
"options[1].manifest.datasets.element.include.element",
245+
"options[1].manifest.datasets.element.measures.element.filter",
240246
"options[1].manifest.datasources.element",
241247
"options[1].manifest.flows.element",
242248
"options[1].manifest.flows.element.errorHandling",
@@ -279,7 +285,9 @@
279285
"data.packages.element.options[1].manifest.dashboards.element.globalFilters.element",
280286
"data.packages.element.options[1].manifest.dashboards.element.widgets.element",
281287
"data.packages.element.options[1].manifest.datasets.element",
288+
"data.packages.element.options[1].manifest.datasets.element.filter",
282289
"data.packages.element.options[1].manifest.datasets.element.include.element",
290+
"data.packages.element.options[1].manifest.datasets.element.measures.element.filter",
283291
"data.packages.element.options[1].manifest.datasources.element",
284292
"data.packages.element.options[1].manifest.flows.element",
285293
"data.packages.element.options[1].manifest.flows.element.errorHandling",
@@ -1134,12 +1142,15 @@
11341142
"ui/Dataset": {
11351143
"sites": [
11361144
"",
1145+
"filter",
11371146
"filter.lazy",
1138-
"include.element"
1147+
"include.element",
1148+
"measures.element.filter"
11391149
]
11401150
},
11411151
"ui/DatasetMeasure": {
11421152
"sites": [
1153+
"filter",
11431154
"filter.lazy"
11441155
]
11451156
},
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import type { SemanticMigration } from '../../types.js';
4+
5+
// The analytics carriers' half of filter-equality-array-comparand-refused-at-save.
6+
// That entry refuses an equality-slot list on save wherever the shared face
7+
// refuses it, and names a list inside a nested-relation condition as a position
8+
// it deliberately leaves to execution. This one closes that position on the two
9+
// carriers the analytics where door charts — a dataset filter and a measure
10+
// filter — because that door flattens a nested relation to a dotted member and
11+
// refuses the list there. Recorded as its own entry because the surface is
12+
// narrower (two carriers, one position) and the shared schema does not move.
13+
export const entry: SemanticMigration = {
14+
id: 'dataset-filter-nested-relation-equality-array-refused-at-save',
15+
// No backticks in `surface` — build-upgrade-guide renders it inside a code
16+
// span already, and a nested backtick would close it.
17+
surface:
18+
'ui.Dataset filter and ui.DatasetMeasure filter — an ARRAY as an EQUALITY comparand on a '
19+
+ 'field INSIDE a nested-relation condition, now refused when the dataset is PARSED: the '
20+
+ 'implicit form { account: { region: [...] } } and the explicit form '
21+
+ '{ account: { region: { $eq: [...] } } }, the empty array included, at any relation depth '
22+
+ 'and under $and / $or / $not. Every other schema that carries a FilterCondition keeps the '
23+
+ 'shared schema\'s reach',
24+
replacement:
25+
'the operator the list was standing in for, on the same field inside the same relation, '
26+
+ 'exactly as in filter-equality-array-comparand-refused-at-save. "One of these values" is '
27+
+ '$in: { account: { region: { $in: ["a", "b"] } } } (authoring spelling "in"). "The stored '
28+
+ 'multi-value field holds this value" is $contains with ONE member (authoring spelling '
29+
+ '"contains"), and an $or of those for any-of. A filter that meant a single value writes that '
30+
+ 'value: { account: { region: "a" } }. The list operators keep their arrays, empty lists '
31+
+ 'included; every scalar, null above all, is untouched; and $ne is NOT judged by this entry',
32+
reason:
33+
'Measured on origin/main 9e7824a445 before the change: DatasetSchema parsed a dataset whose '
34+
+ 'filter was { account: { region: ["a"] } }, and one whose measure filter was '
35+
+ '{ account: { region: { $eq: ["a"] } } }, GREEN — while the analytics where door, which '
36+
+ 'charts both carriers on every path (the native-SQL and ObjectQL strategies and the draft '
37+
+ 'preview), flattens the relation to the dotted member account.region and hands the list to '
38+
+ 'the shared comparand-shape face, which refuses it with INVALID_FILTER / 400. So such a '
39+
+ 'dataset saved clean and every chart built on it failed, for a different person, later. '
40+
+ 'The shared FilterConditionSchema does not descend a field spec with no $ key, because the '
41+
+ 'engine reads one as a deep-equality comparand; ruling A on #19889 (record 5805248669) put '
42+
+ 'it there and it stays there. Triage on #20080 (record 5825670610) routed the fix to the two '
43+
+ 'analytics carriers instead: they refine their filter with the analytics door\'s own walk '
44+
+ '($and / $or arrays and $not descended, other $ keys skipped, a plain object with no $ key '
45+
+ 'descended as a nested relation at any depth) and refuse, inside a nested relation only, '
46+
+ 'exactly what that door refuses there, in the face\'s words from the one builder both doors '
47+
+ 'import. The one difference is that the door appends the location (at '
48+
+ 'where.account.region) and the carrier does not, because its issue carries the location as '
49+
+ 'its path (filter.account.region, measures.0.filter.account.region.$eq). A list outside a '
50+
+ 'nested relation is the shared schema\'s refusal and is reported once. No filter changes '
51+
+ 'meaning: the refusal moves from chart time to save. Metadata AT REST is not rewritten and '
52+
+ 'this entry adds no D2 conversion, for the reason the runtime entry gives: an array on '
53+
+ 'equality has no single honest value. The read path does not re-validate stored rows, so a '
54+
+ 'stored dataset keeps loading; what changes is that re-saving it through the metadata '
55+
+ 'protocol (422 INVALID_METADATA), defineStack or os validate is refused at the filter\'s '
56+
+ 'path. Such a filter has failed every chart since the analytics door began refusing it, so '
57+
+ 'the refusal is a repair and not a loss. In-repo census at 9e7824a445: no dataset or '
58+
+ 'measure filter in examples, platform objects, docs or skills carries the shape; deployed '
59+
+ 'datasets were NOT measured. ADR-0021 / ADR-0087.',
60+
acceptanceCriteria:
61+
'Validate every stack and re-save every stored dataset: os validate or defineStack, and a '
62+
+ 'save through the metadata protocol, report each list in an equality slot inside a nested '
63+
+ 'relation by path, with the field, the received list and both remedies, so the sweep of the '
64+
+ 'two carriers is mechanical. Decide per filter what it meant — one of these values ($in), '
65+
+ 'the stored list holds a value ($contains, an $or of them for several), or one value — and '
66+
+ 're-check what each chart is supposed to show: the filter had been failing every chart. A '
67+
+ 'filter that reaches the analytics door by any other route, such as a caller where or a '
68+
+ 'dataset selection runtimeFilter, is still refused only when it is charted, with '
69+
+ 'INVALID_FILTER / 400 naming the field and the path.',
70+
};

‎packages/spec/src/migrations/entries/semantic/18.filter-equality-array-comparand-refused-at-save.ts‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,10 @@ export const entry: SemanticMigration = {
5353
+ 'with no $ key (a nested-relation or deep-equality condition), which the face never '
5454
+ 'descends either. ⚠️ Three positions therefore still refuse only at execution. (1) A list '
5555
+ 'inside a nested-relation condition, { account: { region: ["a"] } }: the analytics where '
56-
+ 'door flattens that to the dotted member account.region and refuses it when a dataset or '
57-
+ 'measure filter is charted. (2) The where option of the data-engine calls (find, count, '
56+
+ 'door flattens that to the dotted member account.region and refuses it when the filter is '
57+
+ 'charted; a dataset filter and a measure filter refuse it on save as well, which is the '
58+
+ 'sibling entry dataset-filter-nested-relation-equality-array-refused-at-save. (2) The where '
59+
+ 'option of the data-engine calls (find, count, '
5860
+ 'update, delete, aggregate, vector find): its type is a union whose first arm is an open '
5961
+ 'record, so it parses and the face refuses it when the call runs. (3) $ne carrying a '
6062
+ 'list, which no ruling has decided. Two request doors parse these carriers and now answer '
@@ -79,5 +81,7 @@ export const entry: SemanticMigration = {
7981
+ 'query. ⛔ A clean re-save is NOT a complete sweep for the three positions the reason '
8082
+ 'names: grep nested-relation conditions and data-engine where options for a field whose '
8183
+ 'value is a list, and exercise them, where the runtime doors refuse with INVALID_FILTER '
82-
+ '/ 400 naming the field and the path.',
84+
+ '/ 400 naming the field and the path. A dataset filter or a measure filter is the '
85+
+ 'exception: its nested-relation lists are refused on save too '
86+
+ '(dataset-filter-nested-relation-equality-array-refused-at-save).',
8387
};

0 commit comments

Comments
 (0)