Skip to content

Commit be531f6

Browse files
feat(spec,core,objectql,driver-sql,formula,driver-mongodb,service-analytics): now-anchored filter comparand { $now: true, addDays } shifted by days held on the record (#26261)
<!-- claude-projects-attribution: {"github_login":"zhuangjianguo"} --> _Requested by **Steedos** · [project thread](https://claude.ai/code/project/chan_018ETKaNwMZCNRYD7D8Q9Ubb?thread=cmsg_018ETKaNwMZCNRYD7D8Q9UbbFpVLroZXBfHvGDxkKmgWwk)_ Fixes #25808 Clause-②: yes (widening) Metadata-protocol: added Ruling-ref: 6415710193 ## What A filter comparand anchored at the current instant and shifted by whole days read off the record — the maintainer's letter A on #25808, comment 6415710193: ```ts // opened more than review_sla_days_offset days ago — the column holds NEGATIVE days { opened_at: { $lt: { $now: true, addDays: { $field: 'review_sla_days_offset' } } } } // closed on or after thirty days ago { closed_on: { $gte: { $now: true, addDays: -30 } } } ``` `{ $now: true, addDays: <integer | { $field }> }` is legal exactly where a `{ $field }` reference is: the whole comparand of `$eq` / `$ne` / `$gt` / `$gte` / `$lt` / `$lte`, on a `date` or `datetime` column. `addDays` is REQUIRED (the unshifted instant stays the `{now}` placeholder); its grammar is `FieldReferenceSchema.addDays`' — an integer of any sign or a `{ $field }` to a numeric column of the same record. There is no `subDays`: **"now minus the SLA" needs a column holding negative days**, the sign rule the ruling names ("sign rules as `addDays`"). ## Work items 1–5 of the ruling 1. **Spec shape + authorable surface** — `NowAnchorSchema` in `data/filter.zod.ts`, in the ordering and equality comparand unions beside `FieldReferenceSchema`; refused by name as an `$in` / `$nin` member and a `$between` endpoint; an equality triple with an anchored comparand lowers to the explicit `$eq` (as #7597 did for a reference). The `FilterCondition` door (comparand-TYPE walk, `filter-comparand-type.ts`) steps around a well-formed anchor and refuses a malformed one in the anchor's words; the lowering face (`filter-lowering.ts`) treats an anchored comparison as NULL-total. `authorable-surface/data.json`, `json-schema.manifest/data.json`, the api surface and `content/docs/references/data/filter.mdx` regenerated. 2. **Engine resolution + class rule** — `resolveFilterTokens` (`@objectstack/core`, where the resolver lives) pins `$now` to the tree's one instant as an ISO string, so every executor sees `{ $now: '<ISO>', addDays }`; an anchor without `addDays` or with a malformed offset is `INVALID_FILTER` / 400 there. The temporal-comparand door steps around the anchor as it steps around a reference. `having-filter.ts` judges the anchor on `aggregations[i].filter` (operands withheld, diagnostic to the log — `where`'s posture) and on `having` (named — the author's own query): no base column, the target `date` / `datetime`, the offset declared and numeric, never the tenant column (on SQL), and compares it per row through the formula evaluator. No `engine.ts` edit was needed: the judges it calls are the ones that grew. 3. **driver-sql on all three dialects** — `applyNowAnchorComparison`: the instant bound in the target column's storage form (`datetime` keeps the time of day — canonical ISO, `mysqlDatetimeLiteral` on MySQL, `cast(? as timestamptz)` / `cast(? as datetime(3))`; `date` meets the instant's UTC calendar day, `cast(? as date)`; SQLite reads the canonical text directly), shifted through the existing `crossFieldOffsetExpr` — so **NULL offset = 0 days through the same `coalesce`**, pinned and documented. Predicate total: `(A is not null and A op Bx)` for the orderings and `$eq`, `(A is null or A <> Bx)` for `$ne`, so `$not` takes exactly the complement. Offset column under the 2026-08-06 rules; non-temporal target, malformed anchor and every other position refused with the operands withheld (#7929) and the diagnostic in the server log. 4. **Formula + one driver-mongodb cell** — `matchesFilterCondition` resolves the anchor in the form of the stored value it meets (a bare calendar day stays a calendar day), `$now: true` reads the evaluator's clock for a caller with no resolver seam; MongoDB refuses the anchor in every position (`INVALID_FILTER` / 400), one pinned cell per position. 5. **Conformance, lint, docs** — `CROSS_FIELD_NOW_CASES` (12 expectations × both temporal pairs + 4 per-class cells with a NOW off the fixture's time of day — 28 cases) and `CROSS_FIELD_NOW_REFUSALS` (14) on the offset fixture, driven on both paths by driver-sql (sqlite + live PostgreSQL locally; MySQL is the `Temporal Conformance` battery on `main`), driver-sqlite-wasm, service-analytics (where + RLS read scope + dataset measure filter + dataset filter, authored `$now: true`), plus formula, objectql engine (where / aggregation filter / having), spec, core and lint cells. `content/docs/protocol/objectql/query-syntax.mdx` gains "Comparing Against Now, Shifted by Days Held on the Record" after "Comparing Two Fields". Item 6 (hotcrm's dataset authoring) is a hotcrm card, not this PR. ## NativeSQLStrategy Declines a query carrying an anchor as it declines a `{ $field }` comparand (`findCrossFieldComparand` detects `$now` too), so a dataset filter or measure filter routes to the engine path where the driver compiles it under the 2026-08-12 ruling; the ObjectQL strategy's equality lowering emits the explicit `$eq` for an anchored comparand. A dataset's authored `$now: true` is pinned by the service's own token seam (`resolvedDatasetScopeGetter`). ## Acceptance notes - **Sign caveat.** The ruling's title says "now minus an SLA"; the ruled shape has no subtraction. A column that means "days before now" must hold negative values (`review_sla_days_offset: -14`). The docs section and the schema description say so; a positive SLA column needs a negated sibling column or a formula field. - **Related record.** The ruling names the SLA "held on a related record"; SQL push-down compiles same-table columns only (2026-08-06), so a dotted offset validates, resolves in memory and is refused by SQL push-down by name — the same asymmetry the `addDays` pair has, pinned on both faces. The shape that answers everywhere is the SLA propagated onto the row. - **driver-memory** (not on the claim's surface): measured with a probe — it answers `[]` for a `{ $field }` comparand today and answers the same `[]` for an anchor; that silent hole is pre-existing and reported as out of scope. ## Verification See the `os-dev-report` comment on #25808 for the test and gate ledger. _Generated by [Claude Code](https://claude.ai/code/session_019YLF9PyrRqUX4N7C2pzSoH)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 2409c9c commit be531f6

35 files changed

Lines changed: 2587 additions & 95 deletions
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
'@objectstack/spec': minor
3+
'@objectstack/core': patch
4+
'@objectstack/objectql': patch
5+
'@objectstack/driver-sql': patch
6+
'@objectstack/driver-mongodb': patch
7+
'@objectstack/formula': patch
8+
'@objectstack/service-analytics': patch
9+
---
10+
11+
feat(spec,core,objectql,driver-sql,formula,driver-mongodb,service-analytics): a filter compares a datetime column against now shifted by whole days held on the record (`{ $now: true, addDays }`)
12+
13+
Clause-②: yes (widening)
14+
Metadata-protocol: added
15+
Ruling-ref: 6415710193
16+
17+
An analytics dataset could not say "opened longer ago than this ticket's own SLA": the comparand grammar had a `{ $field, addDays }` pair for two columns of the record and `{N_days_ago}` placeholders for a fixed number of days, but nothing that anchored at the current instant and read the number of days off the row. Authors fell back to a literal day count, which drifts from every record whose SLA differs.
18+
19+
`FilterCondition` gains a now-anchored comparand, legal exactly where a `{ $field }` reference is — as the whole comparand of `$eq` / `$ne` / `$gt` / `$gte` / `$lt` / `$lte`, on a `date` or `datetime` column:
20+
21+
```ts
22+
// opened more than review_sla_days_offset days ago — the column holds NEGATIVE days
23+
{ opened_at: { $lt: { $now: true, addDays: { $field: 'review_sla_days_offset' } } } }
24+
// on or after thirty days ago
25+
{ closed_on: { $gte: { $now: true, addDays: -30 } } }
26+
```
27+
28+
- **`@objectstack/spec`.** `NowAnchorSchema` (`data/filter.zod.ts`): `$now` is the literal `true`, `addDays` is REQUIRED — an integer of any sign or a `{ $field }` reference to a numeric column of the same record, the `addDays` grammar `FieldReferenceSchema` already carries; there is no `subDays`, and the unshifted instant stays the `{now}` placeholder. The `FilterCondition` door refuses a malformed anchor in the anchor's own words, and an anchor as an `$in` / `$nin` member or a `$between` endpoint by name, as it refuses a reference there. `filter.mdx` and the authorable surface document the shape.
29+
- **`@objectstack/core`.** `resolveFilterTokens` pins `$now` to the tree's one instant (the `now` every `{N_days_ago}` in the tree resolves from), as an ISO string, so every executor sees `{ $now: '<ISO>', addDays }`; an anchor without `addDays` is `INVALID_FILTER` / 400 there.
30+
- **`@objectstack/driver-sql`.** SQL push-down compiles the anchor on SQLite, PostgreSQL and MySQL through the `addDays` arithmetic (`crossFieldOffsetExpr`), with the instant bound in the target column's storage form: a `datetime` column keeps the time of day, a `date` column meets the instant's UTC calendar day. A NULL offset column is zero days (`coalesce`). The predicate is total: an ordering and `$eq` need the target to have a value, `$ne` holds when it has none or differs, so `$not` takes exactly the complement. The offset column is held to the same-table rules the `addDays` pair's is (declared, numeric, same table, never the tenant-isolation column); a non-temporal target, a malformed anchor and every other position are refused with the operands withheld from the message and the diagnostic in the server log.
31+
- **`@objectstack/formula`.** `matchesFilterCondition` resolves the anchor in the form of the stored value it meets (a calendar day stays a calendar day), so the memory path answers the SQL path's rows; `$now: true` reads the evaluator's clock for a caller with no resolver seam.
32+
- **`@objectstack/objectql`.** `aggregations[i].filter` and `having` judge the anchor by `where`'s rules (temporal target, declared numeric offset) before any row is read, and evaluate it per row through the formula evaluator.
33+
- **`@objectstack/service-analytics`.** `NativeSQLStrategy` declines a query carrying an anchor, as it declines a `{ $field }` comparand, so a dataset filter or measure filter routes to the engine path where the driver compiles it; a dataset's authored `$now: true` is pinned by the service's own token seam.
34+
- **`@objectstack/driver-mongodb`.** Refuses the anchor in every position (`INVALID_FILTER` / 400) rather than translating it as a literal sub-document.
35+
36+
Pinned on the drivers' cross-field conformance corpus (`CROSS_FIELD_NOW_CASES` / `CROSS_FIELD_NOW_REFUSALS`, exported from `@objectstack/driver-sql`) on both execution paths, and on the analytics dataset shape the ruling was made for.

‎content/docs/protocol/objectql/query-syntax.mdx‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -526,6 +526,67 @@ is in the dataset's `include`; measured and pinned in
526526
(#15103). The spelling that answers on both paths is the same-table one — every column the
527527
reference names, offset included, is a column of the object the filter runs over.
528528

529+
### Comparing Against Now, Shifted by Days Held on the Record
530+
531+
A comparand can be **anchored at the current instant** — `{ $now: true, addDays: <offset> }` —
532+
as the *whole* comparand of one of the six scalar comparison operators, on a `date` or
533+
`datetime` column. `addDays` is required and takes the same two spellings the field
534+
reference's does: an integer literal of any sign, or a nested `{ $field }` reference to a
535+
numeric column of the **same record**. There is no `subDays`: a column that means "days
536+
before now" holds negative values. For the unshifted instant, write the `{now}` placeholder.
537+
538+
{/* os:check */}
539+
```typescript
540+
import type { FilterCondition } from '@objectstack/spec/data';
541+
542+
// opened more than review_sla_days_offset days ago — the column holds NEGATIVE days
543+
const pastSla: FilterCondition = {
544+
opened_at: { $lt: { $now: true, addDays: { $field: 'review_sla_days_offset' } } },
545+
};
546+
547+
// closed on or after thirty days ago
548+
const recentlyClosed: FilterCondition = {
549+
closed_on: { $gte: { $now: true, addDays: -30 } },
550+
};
551+
552+
// due within the next seven days (and not yet past)
553+
const dueSoon: FilterCondition = {
554+
$and: [
555+
{ due_on: { $gte: { $now: true, addDays: 0 } } },
556+
{ due_on: { $lte: { $now: true, addDays: 7 } } },
557+
],
558+
};
559+
```
560+
561+
The engine's token resolver pins `$now` to the one instant every `{N_days_ago}` placeholder
562+
in the same query resolves from, so a driver sees `{ $now: '<ISO instant>', addDays }`; a
563+
caller that reaches a driver or the memory evaluator directly, with no resolver in front of
564+
it, gets that executor's own clock. The target column's class decides the form the instant
565+
is compared in, exactly as a `{now}` literal is: a `datetime` column keeps the time of day;
566+
a `date` column meets the instant's **UTC calendar day** (the time of day is dropped, never
567+
rounded). The anchor itself is never NULL, so the only NULL on the table is the target's:
568+
569+
| Case | Reads as |
570+
|:-----|:---------|
571+
| The offset column is NULL | zero days — `now + NULL` is `now` |
572+
| The target column is NULL | fails the orderings and `$eq`, satisfies `$ne` — its ordinary reading against a literal, so `$not` takes exactly the complement |
573+
| The target column is not `date` / `datetime` | refused with `INVALID_FILTER` — an instant has no meaning on a number, a wall-clock `time` or text |
574+
575+
Both execution paths answer it: SQL push-down (`driver-sql`, `driver-sqlite-wasm`) binds the
576+
instant in the column's storage form and shifts it with the same per-dialect day arithmetic
577+
the `addDays` pair uses (SQLite, PostgreSQL, MySQL), and the in-memory evaluator shifts it in
578+
the form of the stored value it meets, so the two return the same rows. The offset column is
579+
held to every rule the `addDays` pair's offset is held to — a declared, numeric column of the
580+
same table, never the tenant-isolation column, no dotted path — and the analytics native-SQL
581+
strategy declines a query carrying an anchor so that the driver compiles it, as it does for a
582+
`{ $field }` comparand. The MongoDB driver refuses the anchor rather than translating it.
583+
584+
**The SLA held on a related record.** The shape that compiles is the same-table one: the SLA
585+
(or its negative) propagated onto the row, or the dataset's base object carrying the column.
586+
A dotted offset (`addDays: { $field: 'queue.sla_days' }`) validates, is resolved by the
587+
memory evaluator, and is refused by SQL push-down with `INVALID_FILTER`, exactly as a dotted
588+
`$field` is — a dataset's `include` does not widen it (see above).
589+
529590
### Date, Datetime, and Time Filters
530591

531592
Before a comparison is built, the driver puts the comparand into the **same canonical

0 commit comments

Comments
 (0)