Repository navigation
Commit e8fcf55
Fixes #20080
Clause-②: no
## What this changes
A dataset `filter` and a measure `filter` now refuse, when the dataset
is saved, a list in the equality slot of a field inside a
nested-relation condition: `{ account: { region: ['a'] } }` and `{
account: { region: { $eq: ['a'] } } }`, the empty list included, at any
relation depth and under `$and` / `$or` / `$not`.
Before, at `origin/main` `9e7824a445`, `DatasetSchema.safeParse`
answered `success: true` for both shapes. The analytics `where` door
charts both carriers on every path (the native-SQL and ObjectQL
strategies and the draft preview). It flattens the relation to the
dotted member `account.region` and hands the list to the shared
comparand-shape face, which refuses it with `INVALID_FILTER` / 400. So
the dataset saved clean and every chart on it failed.
This is remedy A as triage routed it (record 5825670610). The shared
`FilterConditionSchema` is not touched: ruling A (record 5805248669)
keeps its reach at the face's, because the engine reads a no-`$` field
spec as a deep-equality comparand. Remedy B (the analytics door stops
descending) is not taken.
## How
- `packages/spec/src/ui/dataset.zod.ts`:
`refuseNestedRelationEqualityLists` walks a filter the way the analytics
door's `mapWhereFieldEntries` does. It descends `$and` / `$or` arrays
and `$not`, skips every other `$` key, and descends a plain object with
no `$` key as a nested relation, at any depth. The nested-relation test
is the door's own `isNestedRelationSpec`: prototype `Object.prototype`
or `null`, no `$` key. Inside a nested relation it refuses exactly what
`assertNoListInEqualitySlot` refuses there: an array spec (implicit
form), or an object spec whose `$eq` is an array. It reports ONLY inside
a nested relation. Everything outside one is `FilterConditionSchema`'s
own refusal, so nothing is reported twice.
- The words come from `arrayEqualityComparandMessage`, the one builder
the face and the schema door both print from. The analytics door appends
` at where.account.region`. The carrier leaves that out, as the schema
door does, and the issue `path` carries it instead
(`filter.account.region`, `measures.0.filter.account.region.$eq`).
- The check sits on each carrier's `.optional()` wrapper, not on
`FilterConditionSchema` itself. A first cut refined the recursive schema
directly. That clones it, and the published `ui/Dataset.json` then
inlined a second copy of the condition beside its `$ref`. On the
wrapper, the published bodies of `ui/Dataset` and `ui/DatasetMeasure`
are byte-identical to base, apart from `x-dropped-refinements`. That was
measured by generating both trees and comparing the JSON with that key
removed.
- `packages/spec/dropped-refinements.baseline.json`: the new rule is
declared where `z.toJSONSchema()` drops it. That is 11 new sites across
6 existing entries (`filter` on both carriers,
`measures.element.filter`, and the four manifest embeddings), exactly as
`build-schemas.ts` printed them. The header total moves 574 to 585.
## ADR-0087 registration
A new semantic entry,
`dataset-filter-nested-relation-equality-array-refused-at-save`
(protocol major 18), plus its regenerated `registry.ts` region. The
projections (`spec-changes.json`, `docs/protocol-upgrade-guide.md`)
cover up to the current protocol major 17, so they do not move.
`check:spec-changes` and `check:upgrade-guide` are green.
Two unreleased major-18 sibling entries each had one sentence this
change makes false, so each gets a minimal correction:
- `filter-equality-array-comparand-refused-at-save`: its position (1)
said the nested-relation list is refused only when charted. It now also
names the carriers' refusal on save. Its acceptance criteria get the
matching exception.
- `filter-equality-array-comparand-refused`: "that door alone also
refuses a list inside a nested-relation condition" now reads "among the
runtime doors", and the save-side list names the two carriers.
## Deliberate correction of two pending changesets
`Check Changeset` (`check-empty-changeset`) is red on purpose. It is the
DELIBERATE CORRECTION class that gate's own text names, ⛔ not a
collision, and no `skip-changeset` label is applied. Both pending notes
described an exception this PR removes. Triage item 2 (record
5825670610) requires the correction to land in the same PR if this card
lands before the next `changeset version`.
- `.changeset/19889-filter-schema-door-array-equality.md`: the "reach is
the face's" bullet said the nested-relation list "is not judged" and is
refused when a dataset or measure filter is charted. It now says
`FilterConditionSchema` does not judge it, and that the two carriers the
analytics door charts refuse it on save.
- `.changeset/19888-analytics-implicit-array.md`: "A document carrying
that still publishes, and it is refused when it is charted" now says a
dataset or measure filter is refused on save, and that any other `where`
reaching the door, such as a caller `where` or a dataset selection
`runtimeFilter`, is still refused only when charted.
Please confirm both corrections. Restoring either note from base would
republish a false sentence.
This PR's own changeset is
`.changeset/20080-dataset-filter-nested-relation-list.md`
(`@objectstack/spec`: `minor`, BREAKING narrowing, `Clause-②: no
(narrowing)`, ADR-0087 `registered`). `check-adr-0087-registration` and
`check-changeset-no-major` are green.
## Evidence
All runs below are at head `3ed391abf9` unless another sha is named.
They went through `scripts/pm/os-verify-lock.sh`, and each exit code was
captured before any pipe.
- **New pins**,
`packages/spec/src/ui/dataset-filter-nested-relation-list.test.ts`: 68
passed.
- §1: 12 nested shapes are refused at the list's own path on
`DatasetSchema.filter`, on a measure filter through `DatasetSchema`, and
on `DatasetMeasureSchema` alone. One more case checks that a single
document reports every such list.
- §2: the carrier's message EQUALS the analytics door's refusal with the
location clause removed. The door's refusal is rebuilt from the
one-entry node it hands the face, carries envelope `INVALID_FILTER` /
400, and the clause is proven present exactly once.
- §3: a list outside a nested relation is reported exactly ONCE.
- §4: 13 controls are accepted and kept on both carriers. The shared
`FilterConditionSchema` still accepts both nested shapes, so ruling A is
untouched.
- **Ablation 1** (via `scripts/ablation-replace.mjs`, anchor hit 1 to 0,
restore proven): with the refinement removed, 50 failed and 18 passed.
§1 and §2 went red; the §3 and §4 controls stayed green. Restore check:
blob == HEAD `781a5b6df01f`, and `git diff HEAD` is empty.
- **Ablation 2**: with the outside-a-relation guard removed, exactly the
4 §3 rows went red (4 failed, 64 passed). Restore proven the same way.
- **Cross-door conformance** (one-off, not committed): 3000 seeded
random filters went through the REAL analytics door
(`normalizeAnalyticsFilterTree`, service-analytics source) and the
rebuilt `DatasetSchema`. The door refused 2240 with the equality-slot
sentence, and 1304 of those sit where the shared schema alone would miss
them. The carrier agreed on all 3000, with 0 mismatches: the same
sentence whenever the door refused, and no equality issue whenever it
accepted.
- `pnpm --filter @objectstack/spec test` at `43df1403dd`: 15902 passed,
1 failed, 2 todo. The failure was the ledger header total, fixed in
`3ed391abf9`. That file (`scripts/dropped-refinements.test.ts`) plus the
new pins, rerun at `3ed391abf9`: 95 passed.
- `pnpm --filter @objectstack/spec typecheck`: exit 0.
`check:test-typecheck` reports OK (53 files / 255 errors held in the
ledger). `tsc -p tsconfig.test.json --listFiles` lists the new test file
once, and 0 of the 255 errors sit in any touched file.
- `pnpm --filter @objectstack/spec build && check:generated` at
`43df1403dd`: all 15 generated artifacts up to date. `gen:schema` was
rerun at `3ed391abf9` for the ledger: exit 0, 585 refinement sites
across 207 published schemas, all declared in the ledger, 1549 schemas
generated, and the tree left clean.
- **Published JSON Schema**: base and head `ui/Dataset.json` /
`ui/DatasetMeasure.json` were generated side by side (a compare worktree
at `9e7824a445`). The bodies are byte-identical with
`x-dropped-refinements` removed.
- **Consumers.** Test-filter direction is the upstream prefix form,
`--filter '@objectstack/service-analytics^...'`, which built its
14-package closure.
- `pnpm --filter @objectstack/service-analytics test`: 128 files and
3017 tests passed, all 128 tracked test files. That includes the
read-only control `where-equality-slot-list-refusal.test.ts`, so the
analytics door's refusal is unchanged.
- `packages/metadata` `typescript-serializer-annotation.test.ts`: 7
passed.
- `packages/platform-objects` `report-form-echo-decisions.test.ts`: 35
passed.
- `packages/rest` (3 files) and `packages/cli` (1 file) are declared to
CI; their closures are 25 and 57 packages. Static sweep: none of the 80
consumer test files that reference `DatasetSchema` passes a list under a
nested relation through it. The only such literals go straight to the
analytics door or to read-scope compilers.
- **Gates.** Derived with `node scripts/pm/dispatch-gates.mjs --repo
objectstack-ai/objectstack --commands` at `3ed391abf9` (identical to the
derivation at `43df1403dd`). Reconciled with `--ran`: 87 derived, 85
run, 2 NOT MEASURED, 0 unrun.
- 84 exit 0.
- `check-empty-changeset` exits 1 BY DESIGN (see the
deliberate-correction section).
- NOT MEASURED: `check:type-check-debt` and
`check:dual-build-cjs-loads`, both exit 3 (PREREQUISITE NOT MET: they
need the whole-repo build, which CI does).
- **Lint**, a declared narrowing: `eslint --no-inline-config --format
json` over the 6 touched TS files reported 6 files, 0 errors, 0
warnings, none ignored. The repo's one `eslint.config.mjs` never enables
type-aware linting (its own note, about line 326), and no touched
module's exports changed, so no untouched file's verdict can move. The
repo-wide `pnpm lint` is CI's.
## Census
In-repo, at `9e7824a445`: 0 dataset or measure filters carry a list
under a nested relation. The scan covered the dataset definitions of
`examples/app-crm`, `examples/app-showcase` and `examples/app-todo`, the
platform objects' `system.datasets.ts`, the dataset examples in
`content/docs` and `skills`, and a repo-wide literal scan of non-test
`filter` / `where` carriers. Deployed datasets: NOT MEASURED.
## Acceptance notes
- **Same family, measured, not fixed here.** The shared face's OTHER
arms are accepted by `DatasetSchema.filter` at the top level AND inside
a nested relation: a `null` ordering comparand (`$gt: null`), a non-list
`$in`, a `null` member of `$in`, and a `null` `$between` endpoint. The
shared face refuses each with `INVALID_FILTER` / 400, and so does the
analytics door when it charts the filter. Probe at `3ed391abf9`, with a
control dataset that parses: `{ amount: { $gt: null } }`, `{ stage: {
$in: 'won' } }`, `{ amount: { $between: [null, 5] } }` and their `{
acct: … }` nested forms all answer `success: true`. This is the #19889 /
#20116 save-door family, and it is reported for the PM to fold into that
family rather than filed singly. Dedupe words: `dataset filter $gt null
accepted at save` · `FilterConditionSchema other comparand arms not
judged at save` · `$in non-list dataset filter saves clean`.
- **Other routes to the analytics door.** A caller `where` and a dataset
selection `runtimeFilter` still reach the door with a nested list and
are refused when charted. Both are requests, so the refusal is loud to
the caller. Whether a stored dashboard widget `filter` or report
`runtimeFilter` reaches the door that way is decided on the objectui
side and was NOT MEASURED here.
- **Stale header number, noted and not filed.**
`dropped-refinements.baseline.json`'s
`measured.refinementSitesThatDidProject` reads 369, while
`build-schemas.ts` prints 409 at base `9e7824a445` and at head. No test
reads that field. Nobody is carrying it.
- The claim's `Clause-②: no` is copied above verbatim. This PR's
changeset carries `Clause-②: no (narrowing)`, the spelling the ADR-0087
gate reads.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01QcAS3qiYYZNezaxZxaUdMV)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent 03d6cb0 commit e8fcf55
10 files changed
Lines changed: 610 additions & 24 deletions
File tree
- .changeset
- packages/spec
- src
- migrations
- entries/semantic
- ui
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
| 30 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
| 24 | + | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| |||
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
| 59 | + | |
59 | 60 | | |
| 61 | + | |
60 | 62 | | |
61 | 63 | | |
62 | 64 | | |
| |||
154 | 156 | | |
155 | 157 | | |
156 | 158 | | |
| 159 | + | |
157 | 160 | | |
| 161 | + | |
158 | 162 | | |
159 | 163 | | |
160 | 164 | | |
| |||
236 | 240 | | |
237 | 241 | | |
238 | 242 | | |
| 243 | + | |
239 | 244 | | |
| 245 | + | |
240 | 246 | | |
241 | 247 | | |
242 | 248 | | |
| |||
279 | 285 | | |
280 | 286 | | |
281 | 287 | | |
| 288 | + | |
282 | 289 | | |
| 290 | + | |
283 | 291 | | |
284 | 292 | | |
285 | 293 | | |
| |||
1134 | 1142 | | |
1135 | 1143 | | |
1136 | 1144 | | |
| 1145 | + | |
1137 | 1146 | | |
1138 | | - | |
| 1147 | + | |
| 1148 | + | |
1139 | 1149 | | |
1140 | 1150 | | |
1141 | 1151 | | |
1142 | 1152 | | |
| 1153 | + | |
1143 | 1154 | | |
1144 | 1155 | | |
1145 | 1156 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
56 | | - | |
57 | | - | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
58 | 60 | | |
59 | 61 | | |
60 | 62 | | |
| |||
79 | 81 | | |
80 | 82 | | |
81 | 83 | | |
82 | | - | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
83 | 87 | | |
0 commit comments