Repository navigation
Commit 02b1392
fix(spec)!: the filter save and validate doors refuse a dotted addDays offset (#26655)
<!-- claude-projects-attribution: {"github_login":"os-support-ai"} -->
_Requested by **Yibo Wang** · [project
thread](https://claude.ai/code/project/chan_01BVUyyPie8gHwoPM18iABwB?thread=cmsg_01BVUyyPie8gHwoPM18iABwBTLty4kcd1RfdV2tNc1HXpk)_
Fixes #26601
Clause-②: no (narrowing) — the save and validate doors refuse a dotted
`addDays` `$field` that every SQL read already refused. Nothing is
widened.
Metadata-protocol: narrowed (a filter, a dataset filter or a dataset
measure filter may no longer carry an `addDays` offset whose `$field` is
a dotted path)
Skills-docs: none — `skills/objectstack-query/rules/filters.md` already
teaches the offset as a number field of the same record and says SQL
push-down refuses a dotted offset; this moves that refusal to save time
and adds no new authoring rule. Moving "dotted" into that page's
refused-at-400 list is a one-word follow-up for a skills PR (governed
surface, kept out of this one).
Ruling-ref: 6415710193 (letter A on #25808), seat ruling 6439436943,
claim 6439488124. Scope is the author-time half only; B′ (an offset
through a to-one path) is not in scope.
## 维护者速读
- **改了什么:** 过滤条件里 `addDays` 偏移量若写成带点的跨表路径(如
`contract_type.review_sla_days`),现在在保存与 `os validate`
时就被拒(`INVALID_FILTER` / 400),措辞与 SQL 读取时的拒绝一致;同时把数据集度量过滤器读取时被隐去的原因一并带出。
- **为什么改:** 以前 `os validate` 通过、看板每次读取都 400;内存求值器沿路径读到的是关联记录的 id,偏移按 0
天算,静默算错。裁决 A 已把偏移限定为同一记录的数值列。
- **风险与回滚:** 收窄(BREAKING):带点偏移的已存过滤器在保存时会被拒,但这些过滤器在任何 SQL 部署上本来就读不出来。回滚
= revert 本 PR。
- **先看:**
`packages/spec/src/data/filter-comparand-type.ts`(`judgeAddDaysOffset`、`dottedAddDaysOffsetRefusal`)、`packages/spec/src/data/filter.zod.ts`(`AddDaysReferenceSchema`
的 `regex` 与
`addDaysOffsetMessage`)、`.changeset/26601-dotted-add-days-offset-refused.md`。
- **两分钟验证:** `pnpm --filter @objectstack/spec exec vitest run
src/data/filter.test.ts -t 26601`,预期 7 个用例全绿,其中数据集度量过滤器与
`ObjectStackDefinitionSchema` 均在
`measures.1.filter.review_started_at.$lt` 处被拒。
- **需要你做的:** 无,席位落地。
## What changes
- **The shared comparand walk** (`filter-comparand-type.ts`,
`normalizeFilterComparandTypes`) still steps around a `{ $field }`
reference and a well-formed `$now` anchor, except for one part: an
`addDays` offset whose `$field` is a dotted path is refused at every
scalar comparison with `INVALID_FILTER` / 400. The reason is
`driver-sql`'s own sentence for the same refusal
(`sql-driver-cross-field-comparison.ts:352` and `:517`): `the addDays
offset "contract_type.review_sla_days" is a dotted path, and SQL
push-down compiles same-table column references only (no relation
traversal, no alias-qualified columns).` The walk runs at the
`FilterConditionSchema` save door (data filters, dataset `filter`,
dataset measure `filter`), in `parseFilterAST`, and at the engine's
`where`, `aggregations[i].filter` and `having` doors.
- **The offset schema** (`filter.zod.ts`, `AddDaysReferenceSchema`)
refuses a dotted `$field` with the same reason. That covers
`FieldReferenceSchema.addDays`, `NowAnchorSchema.addDays` and the
ordering slots of `FieldOperatorsSchema` / `ComparisonOperatorSchema`.
It is a `regex`, not a `refine`, so the published JSON Schema carries
the rule as a `pattern` and the dropped-refinement ledger does not grow.
- **The withheld 400 on a dataset measure filter.** Measured: the
measure filter reaches the engine's `aggregations[i].filter` door, where
`assertAggregationFilterIsEvaluable` (`having-filter.ts`) refuses
cross-field rules in `where`'s withheld words, with no author-provenance
restore. The walk runs before it (`engine.ts` near line 20463), so a
dotted offset there is now refused by the walk with the path and reason
named. No executor file changed. The other cross-field refusals at that
door stay withheld (acceptance note below).
- **Review 6438749334's two should-fixes** (the now-anchor extra-key
refusal): the "there is no subDays" clause prints only when `subDays` is
among the undeclared keys, so a one-key refusal of any other key keeps
its last sentence inside the 500-character client bound; the stale
`isNowAnchor` docblock sentence ("refused … by each executor at query
time") is trimmed to name the walk.
- **Superseded readings**, named per the seat: the #14104 ruling-record
correction 5550346132 ("answered by the memory evaluator, refused by SQL
push-down") and the spec pin at `filter.test.ts:1972` (the dotted nested
reference accepted "exactly as `$field` itself allows"). Reason: ruling
A on #25808 bounds the offset to a same-record column, and the only
executor that walked the path, the formula evaluator, reads a lookup's
id on a stored row and counts the offset as zero days. The pin now
asserts the refusal.
- **Docs**: `content/docs/protocol/objectql/query-syntax.mdx` loses "A
dotted offset … validates" in both places it said so;
`content/docs/references/data/filter.mdx` regenerated for the new
describe.
- **Not changed**: the reference's own `$field` (a dotted base reference
keeps its asymmetry: memory walks it, SQL refuses it), the bare implicit
form `{ field: { $field, addDays } }` (an operator spec named `$field`,
whose fate is unchanged), and every executor.
## Executors measured before the change (origin/main c8ed4c9)
- `driver-sql` (and `driver-sqlite-wasm`, which inherits it): refuses a
dotted offset on the pair and the anchor (conformance cells `:858`,
`:1118`).
- `driver-mongodb`: refuses every `$field` and `$now` anchor by shape,
dotted or not (`mongodb-filter.ts`, `carriesFieldReference` /
`carriesNowAnchor`).
- `@objectstack/formula` `matchesFilterCondition`: walks the path with
`getPath`. On a flat row (the lookup holds an id) the offset reads
undefined, counts as 0 days, and answers silently: an SLA of -10 days
read as 0 counted a 4-day-old review as overdue. It answers correctly
only on a record with the relation expanded.
- `driver-memory` `find`: answers `[]` for any cross-field or anchor
comparand handed to it directly (the known #15104 gap).
- Every authoring door accepted it: `FieldReferenceSchema`,
`NowAnchorSchema`, `FieldOperatorsSchema`, `FilterConditionSchema`,
`parseFilterAST`, the walk, `DatasetSchema` measure `filter` and
`filter`.
## Tests
- `packages/spec/src/data/filter.test.ts`: a `[#26601]` block. The
schemas refuse at `addDays.$field`; the four ordering slots on both
operator copies; `FilterConditionSchema` at all six scalar comparisons
and under `$and` / `$or` / `$not`; `DatasetSchema` measure `filter` and
`filter` at the slot; `ObjectStackDefinitionSchema` (the schema `os
validate` parses) at
`datasets.0.measures.1.filter.review_started_at.$lt`, with a same-record
control through the same door; controls for a same-record offset and a
dotted base reference. The `:1972` pin flipped; the ordering-slot pin at
`:1986` now uses a same-record offset.
- `packages/spec/src/data/filter-comparand-type.test.ts`: the walk
refuses on the pair, the anchor and the resolved anchor at all six
comparisons, with code, status and the `driver-sql` sentence; under
combinators and both `parseFilterAST` forms; the measure-filter position
stays under 500 characters with its last sentence intact; controls kept
by reference. The subDays clause is pinned both ways, plus a one-key
refusal under `$and` with the engine prefix inside 500 characters.
- `service-analytics` (tests only): `now-anchor-dataset.test.ts`,
`cross-field-offset-dataset.test.ts` and
`include-relation-cross-field-boundary.test.ts` asserted that the dotted
offset validates at the schema door. They now assert the refusal, and
they keep their read-time pins by putting the dotted filter back past
the save door. The bare dotted arm still validates and is still refused
by the driver.
## Acceptance notes
- **Other withheld refusals at `aggregations[i].filter`** (out of scope,
no card): an undeclared, non-numeric or tenant offset column in a
dataset measure filter is still answered in `where`'s withheld words by
`assertAggregationFilterIsEvaluable`, with no author-provenance restore.
A dataset measure filter is the author's own, so a provenance mark there
would restore those reasons; that is a design question for the analytics
lane, not a pass-through in a file this PR touches.
- **Disclosure at the engine doors**: the walk's refusal names the field
path and the offset, as the walk's other refusals at the same doors
already do (malformed anchor, comparand type). A row-level read scope
cannot carry an `addDays` (the CEL compiler emits none), so no policy
text reaches this sentence.
- **Base `$field` asymmetry kept**: `{ $lte: { $field: 'duty.due_on' }
}` still validates; memory walks it and SQL refuses it. Same reasoning
would apply there, but it is outside this card's order.
- **Formula evaluator called directly** still walks a dotted offset
(`matches-filter-now-anchor.test.ts:69`). Every in-repo caller sits
behind a door that runs the walk.
## Verification
Package suites ran on `b9c396e87`; the merge of `origin/main` that made
`c7f0010ef` brought no file in spec, service-analytics, objectql,
formula, plugin-security, lint or driver-memory. Dogfood and the derived
gates ran on `c7f0010ef`. Every command ran under
`scripts/pm/os-verify-lock.sh`, with the exit code captured before any
pipe.
<details><summary>Package suites and typecheck</summary>
- `@objectstack/spec` test: Test Files 730 passed, Tests 22073 passed, 1
todo. Typecheck exit 0.
- `@objectstack/service-analytics` test: 191 files, 4647 passed, 282
skipped. Typecheck exit 0.
- `@objectstack/objectql` test: 472 files, 8795 passed. Typecheck exit
0.
- `@objectstack/formula` test: 54 files, 1452 passed.
- `@objectstack/plugin-security` test: 229 files, 4962 passed, 45
skipped.
- `@objectstack/lint` test: 166 files, 7389 passed.
- `@objectstack/driver-memory` test: 72 files, 1758 passed.
- `driver-sql` / `driver-sqlite-wasm` / `driver-mongodb` not re-run:
none of them imports the walk or the offset schema, and no file of
theirs changed.
- `pnpm --filter @objectstack/spec check:generated --fix`, then plain:
"All 14 generated artifacts are up to date" (only
`references/data/filter.mdx` moved).
</details>
<details><summary>Dogfood (owed for the narrowing arm) at
c7f0010</summary>
`pnpm --filter @objectstack/dogfood test` after the
`@objectstack/dogfood^...` closure build: VERDICT command-exit 0, Test
Files 301 passed and 3 skipped (304), Tests 2501 passed and 18 skipped
(2519).
</details>
<details><summary>Derived gates at c7f0010</summary>
`node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` gave 106 commands, run serially with exit
codes recorded. 105 exit 0. `check:skill-examples` first exited 3
(client SDK not built); after building `@objectstack/client-react...` it
exited 0 with "266 prose examples type-check".
`check:dual-build-cjs-loads` exits 3 (PREREQUISITE NOT MET, it needs the
whole workspace built) and is left to CI. `--ran` reconciliation: "106
derived famil(ies) accounted for — 105 run, 1 NOT-MEASURED".
</details>
<details><summary>Reverse verification, both legs restored by blob
hash</summary>
- Walk leg: `dottedAddDaysOffset` made to answer `undefined`
(`ablation-replace.mjs`, anchor 1 to 0, blob `48b975f2` to `edc0bfb4`).
6 failed / 231 passed across the two spec files. Every red is a walk or
save-door pin: the scalar comparisons, the combinators, the length
bound, the FilterCondition door, the dataset doors and the validate
door. The schema-level pins stayed green. Restored: blob equals HEAD,
`git diff HEAD` empty.
- Schema leg: the `AddDaysReferenceSchema` pattern made vacuous (blob
`2403f93a` to `34ee3e03`). 3 failed / 234 passed: the flipped `:1972`
pin, the two schemas, and the ordering slots. The walk pins stayed
green. Restored the same way.
</details>
<details><summary>Engine probe at the measure-filter door (objectql
dist, after the change)</summary>
`engine.aggregate('clm_contract', { aggregations: [{ function: 'count',
filter: { review_started_at: { $lt: { $now: true, addDays: { $field:
'contract_type.review_sla_days' } } } } }] })` answers `INVALID_FILTER`
/ 400: `aggregate('clm_contract'): Filter comparand at
aggregations[0].filter.review_started_at.$lt is refused: the addDays
offset "contract_type.review_sla_days" is a dotted path, and SQL
push-down compiles same-table column references only …`. The same
measure with `addDays: { $field: 'review_sla_days' }` answers
`[{"over":1}]`.
</details>
---
_Generated by [Claude
Code](https://claude.ai/code/session_01WGufzeyqyEtWpnqFTkcC9r)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent fe53769 commit 02b1392
10 files changed
Lines changed: 414 additions & 39 deletions
File tree
- .changeset
- content/docs
- protocol/objectql
- references/data
- packages
- services/service-analytics/src/__tests__
- spec/src/data
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
509 | 509 | | |
510 | 510 | | |
511 | 511 | | |
512 | | - | |
513 | | - | |
514 | | - | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
515 | 517 | | |
516 | 518 | | |
517 | 519 | | |
| |||
521 | 523 | | |
522 | 524 | | |
523 | 525 | | |
524 | | - | |
| 526 | + | |
| 527 | + | |
525 | 528 | | |
526 | 529 | | |
527 | 530 | | |
| |||
583 | 586 | | |
584 | 587 | | |
585 | 588 | | |
586 | | - | |
587 | | - | |
588 | | - | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
589 | 594 | | |
590 | 595 | | |
591 | 596 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
240 | 240 | | |
241 | 241 | | |
242 | 242 | | |
243 | | - | |
| 243 | + | |
244 | 244 | | |
245 | 245 | | |
246 | 246 | | |
| |||
389 | 389 | | |
390 | 390 | | |
391 | 391 | | |
392 | | - | |
| 392 | + | |
393 | 393 | | |
394 | 394 | | |
395 | 395 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
339 | 339 | | |
340 | 340 | | |
341 | 341 | | |
342 | | - | |
343 | | - | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
344 | 346 | | |
345 | | - | |
| 347 | + | |
346 | 348 | | |
347 | 349 | | |
348 | | - | |
349 | | - | |
350 | | - | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
351 | 356 | | |
352 | 357 | | |
353 | 358 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
109 | | - | |
110 | | - | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
111 | 119 | | |
112 | 120 | | |
113 | 121 | | |
| |||
119 | 127 | | |
120 | 128 | | |
121 | 129 | | |
122 | | - | |
| 130 | + | |
123 | 131 | | |
124 | 132 | | |
125 | 133 | | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
126 | 138 | | |
127 | 139 | | |
128 | 140 | | |
| |||
212 | 224 | | |
213 | 225 | | |
214 | 226 | | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
215 | 238 | | |
216 | 239 | | |
217 | 240 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
332 | 332 | | |
333 | 333 | | |
334 | 334 | | |
335 | | - | |
| 335 | + | |
336 | 336 | | |
337 | | - | |
| 337 | + | |
338 | 338 | | |
339 | 339 | | |
340 | | - | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
341 | 343 | | |
342 | 344 | | |
343 | 345 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
350 | 350 | | |
351 | 351 | | |
352 | 352 | | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
0 commit comments