From 045c2b2c9b1b48c478c7fbe9bbf40261c12b914d Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 27 Oct 2026 10:40:13 +0000 Subject: [PATCH 1/3] fix(spec)!: refuse a dotted addDays offset at the filter save and validate doors The comparand-type walk and the addDays offset schema now refuse an addDays { $field } naming a dotted path, on the { $field, addDays } pair and on the $now anchor, in driver-sql's own same-table words. A data filter, a dataset filter and a dataset measure filter fail at save and at os validate instead of on the first SQL read. Also takes the two should-fixes from the contract review of the now-anchor extra-key refusal: the subDays clause prints only when subDays is among the undeclared keys, and the isNowAnchor docblock no longer claims each executor refuses a malformed anchor. Claude-Session: https://claude.ai/code/session_01WGufzeyqyEtWpnqFTkcC9r Co-Authored-By: Claude --- .../26601-dotted-add-days-offset-refused.md | 24 +++ .../docs/protocol/objectql/query-syntax.mdx | 19 ++- .../cross-field-offset-dataset.test.ts | 17 ++- ...lude-relation-cross-field-boundary.test.ts | 29 +++- .../src/__tests__/now-anchor-dataset.test.ts | 8 +- .../src/data/filter-comparand-type.test.ts | 92 +++++++++++ .../spec/src/data/filter-comparand-type.ts | 69 ++++++++- packages/spec/src/data/filter.test.ts | 144 +++++++++++++++++- packages/spec/src/data/filter.zod.ts | 47 +++++- 9 files changed, 412 insertions(+), 37 deletions(-) create mode 100644 .changeset/26601-dotted-add-days-offset-refused.md diff --git a/.changeset/26601-dotted-add-days-offset-refused.md b/.changeset/26601-dotted-add-days-offset-refused.md new file mode 100644 index 00000000000..54182526d8f --- /dev/null +++ b/.changeset/26601-dotted-add-days-offset-refused.md @@ -0,0 +1,24 @@ +--- +'@objectstack/spec': minor +--- + +fix(spec)!: the filter save and validate doors refuse an `addDays` offset whose `{ $field }` is a dotted path, in the words SQL push-down already refuses it with on every read + +An `addDays` offset reads its days from a column of the same record: `{ $field: 'due_on', addDays: { $field: 'grace_days' } }`, or `{ $now: true, addDays: { $field: 'review_sla_days' } }`. A dotted offset such as `addDays: { $field: 'contract_type.review_sla_days' }` passed `FieldReferenceSchema`, `NowAnchorSchema`, `FilterConditionSchema`, the dataset `filter` and measure `filter`, and `os validate`, and then failed every SQL read with `INVALID_FILTER` / 400. The one executor that walks the path, the memory evaluator, reads a lookup's id there on a stored row and counts the offset as zero days. Now every authoring door refuses it, naming the path: `the addDays offset "contract_type.review_sla_days" is a dotted path, and SQL push-down compiles same-table column references only (no relation traversal, no alias-qualified columns).` + +This retires the earlier reading that the dotted offset validates and is answered in memory (the #14104 ruling-record correction 5550346132, and the spec pin that accepted `addDays: { $field: 'duty.grace_days' }`). Ruling A on #25808 bounds the offset to a same-record column. + +Clause-②: no (narrowing) — the save and validate doors refuse a dotted `addDays` `$field` that every SQL read already refused. Nothing is widened. + +Metadata-protocol: narrowed (a filter, a dataset filter or a dataset measure filter may no longer carry an `addDays` offset whose `$field` is a dotted path) + + + +**BREAKING** accept-set narrowing on the `addDays` offset (`@objectstack/spec`). + +- **Before.** `FieldReferenceSchema`, `NowAnchorSchema`, the ordering slots of `FieldOperatorsSchema`, `FilterConditionSchema` (data filters, dataset `filter`, dataset measure `filter`) and `os validate` accepted `addDays: { $field: 'a.b' }`. SQL reads answered `INVALID_FILTER` / 400; a dataset measure filter answered it with the reason withheld. +- **Now.** Each of those doors refuses it. `parseFilterAST` and the engine's filter doors (`where`, `aggregations[i].filter`, `having`) refuse it with `INVALID_FILTER` / 400 and the path named, so a dataset measure filter that never met the save door now gets the reason too. +- **Unchanged.** A same-record offset column, a literal offset, and the reference's own `$field` (a dotted base reference keeps its asymmetry: resolved in memory, refused by SQL push-down). +- **Who sees it.** An author who read the offset through a relation. Every SQL deployment already refused that filter on read. + +**The remedy.** Copy the related value onto a numeric column of the record (on create and whenever the relation changes) and name that column: `addDays: { $field: 'review_sla_days' }`. diff --git a/content/docs/protocol/objectql/query-syntax.mdx b/content/docs/protocol/objectql/query-syntax.mdx index 3f85feed77c..ca8f98a9e91 100644 --- a/content/docs/protocol/objectql/query-syntax.mdx +++ b/content/docs/protocol/objectql/query-syntax.mdx @@ -509,9 +509,11 @@ On SQL push-down the offset compiles only between two temporal columns of the sa class (`date` with `date`, `datetime` with `datetime`) against a numeric offset column, on every dialect the `$field` compiler covers (SQLite, PostgreSQL, MySQL); the memory evaluator matches, and a fractional offset *value* is truncated toward zero on -both. The same-table rule applies to the offset column too: `addDays: { $field: -'duty.grace_days' }` (a relation path) is resolved by the memory evaluator and refused -by SQL push-down with `INVALID_FILTER`, exactly as a dotted `$field` is. +both. The same-table rule applies to the offset column too, and the offset is held to it +at save time: `addDays: { $field: 'duty.grace_days' }` (a relation path) is refused with +`INVALID_FILTER` by `os validate`, by the filter and dataset save doors and by every read, +in the SQL push-down's own words. Only the referenced `$field` itself keeps the older +asymmetry — resolved by the memory evaluator, refused by SQL push-down. **A dataset's `include` does not widen this.** A dataset compiles a `LEFT JOIN` for every relation it `include`s (ADR-0071, up to three hops), and on the native-SQL path those joins @@ -521,7 +523,8 @@ declines it so that the driver, which alone holds the declared field set, the co and the tenant column the cross-field rules check, compiles it (#7598) — and the driver compiles single-table statements, with no join in scope. So `completed_on <= due_on + duty.grace_days` is refused with `INVALID_FILTER` even when `duty` -is in the dataset's `include`; measured and pinned in +is in the dataset's `include` — at save time now, and by the driver for a dataset that +never met the save door; measured and pinned in `packages/services/service-analytics/src/__tests__/include-relation-cross-field-boundary.test.ts` (#15103). The spelling that answers on both paths is the same-table one — every column the reference names, offset included, is a column of the object the filter runs over. @@ -583,9 +586,11 @@ strategy declines a query carrying an anchor so that the driver compiles it, as **The SLA held on a related record.** The shape that compiles is the same-table one: the SLA (or its negative) propagated onto the row, or the dataset's base object carrying the column. -A dotted offset (`addDays: { $field: 'queue.sla_days' }`) validates, is resolved by the -memory evaluator, and is refused by SQL push-down with `INVALID_FILTER`, exactly as a dotted -`$field` is — a dataset's `include` does not widen it (see above). +A dotted offset (`addDays: { $field: 'queue.sla_days' }`) is refused with `INVALID_FILTER` +when the filter, dataset or stack is saved or validated, in the SQL push-down's words — a +dataset's `include` does not widen it (see above). The memory evaluator would walk the path, +but on a stored row the relation holds the related record's id, so the offset would read as +zero days: copy the related value onto a numeric column of the row instead. ### Date, Datetime, and Time Filters diff --git a/packages/services/service-analytics/src/__tests__/cross-field-offset-dataset.test.ts b/packages/services/service-analytics/src/__tests__/cross-field-offset-dataset.test.ts index 01d4352b6c1..71d7a1975aa 100644 --- a/packages/services/service-analytics/src/__tests__/cross-field-offset-dataset.test.ts +++ b/packages/services/service-analytics/src/__tests__/cross-field-offset-dataset.test.ts @@ -339,15 +339,20 @@ describe('[#14104] the dataset, valued end to end on the engine path', () => { expect(Number(byTitle.get('f')?.late ?? 0)).toBe(0); }); - it('the dotted offset spelling of the ruling validates, routes, and is refused by SQL push-down by name', async () => { - // Validates at the schema door (the memory evaluator walks the path). + it('[#26601] the dotted offset spelling of the ruling is refused at the schema door, and past it by SQL push-down', async () => { + // Refused at the schema door: the offset column is a column of the same + // record (ruling A on #25808), and the memory evaluator, which walks the + // path, reads a lookup's id there on a stored row — zero days, silently. const dotted = { completed_on: { $lte: { $field: 'due_on', addDays: { $field: 'duty.grace_days' } } } }; - expect(DatasetSchema.safeParse({ + const saved = DatasetSchema.safeParse({ ...TASK_HEALTH, measures: [{ name: 'done_on_time', aggregate: 'count', filter: dotted }], - }).success).toBe(true); - // …and on a SQL deployment it is a loud INVALID_FILTER, never a wrong count: - // the 2026-08-06 same-table ruling on #5222, inherited by the offset. + }); + expect(saved.success).toBe(false); + expect(saved.error?.issues[0]?.message).toContain('"duty.grace_days" is a dotted path'); + // …and a dataset that never met that door is a loud INVALID_FILTER on a SQL + // deployment, never a wrong count: the 2026-08-06 same-table ruling on + // #5222, inherited by the offset. let err: WireBearingError | null = null; try { await service.queryDataset( diff --git a/packages/services/service-analytics/src/__tests__/include-relation-cross-field-boundary.test.ts b/packages/services/service-analytics/src/__tests__/include-relation-cross-field-boundary.test.ts index a5d920cfa91..8bbf3f64839 100644 --- a/packages/services/service-analytics/src/__tests__/include-relation-cross-field-boundary.test.ts +++ b/packages/services/service-analytics/src/__tests__/include-relation-cross-field-boundary.test.ts @@ -106,8 +106,16 @@ const RELATION_BARE: FilterCondition = { completed_on: { $lte: { $field: 'duty.grace_days' } }, }; -/** The dataset: `duty` IS in `include`, and a dimension already reads through it. */ -const DATASET = DatasetSchema.parse({ +/** + * The dataset: `duty` IS in `include`, and a dimension already reads through it. + * + * [#26601] The offset arm is a DOTTED `addDays` offset, which the dataset write + * door now refuses (ruling A on #25808 bounds the offset to a same-record + * column). So the dataset is parsed with the same-table filter in that slot + * and the dotted one put back PAST the save door — the read-time half of the + * boundary is still this file's to pin, for a row that never met that door. + */ +const PARSED = DatasetSchema.parse({ name: 'task_health_by_duty', label: 'Task health by duty', object: TASK, @@ -119,10 +127,14 @@ const DATASET = DatasetSchema.parse({ measures: [ { name: 'total', aggregate: 'count' }, { name: 'done_on_time', aggregate: 'count', filter: SAME_TABLE_ON_TIME }, - { name: 'done_on_time_by_duty', aggregate: 'count', filter: RELATION_OFFSET_ON_TIME }, + { name: 'done_on_time_by_duty', aggregate: 'count', filter: SAME_TABLE_ON_TIME }, { name: 'done_by_duty_bare', aggregate: 'count', filter: RELATION_BARE }, ], }); +const DATASET = { + ...PARSED, + measures: PARSED.measures.map((m) => (m.name === 'done_on_time_by_duty' ? { ...m, filter: RELATION_OFFSET_ON_TIME } : m)), +} as typeof PARSED; interface WireBearingError extends Error { code?: string; @@ -212,6 +224,17 @@ describe('[#15103] a cross-field comparand on an `include`d relation — the bou }); }); + describe('[#26601] the save door refuses the dotted OFFSET; the dotted bare reference still validates', () => { + it('DatasetSchema refuses the offset arm at its slot, in the SQL read\'s words, and keeps the bare arm', () => { + const offset = DatasetSchema.safeParse({ ...PARSED, measures: [{ name: 'x', aggregate: 'count', filter: RELATION_OFFSET_ON_TIME }] }); + expect(offset.success).toBe(false); + expect(offset.error?.issues[0]?.path).toEqual(['measures', 0, 'filter', 'completed_on', '$lte']); + expect(offset.error?.issues[0]?.message).toContain('"duty.grace_days" is a dotted path'); + expect(offset.error?.issues[0]?.message).toContain('same-table column references only'); + expect(DatasetSchema.safeParse({ ...PARSED, measures: [{ name: 'x', aggregate: 'count', filter: RELATION_BARE }] }).success).toBe(true); + }); + }); + describe('the `$field` COMPARAND never reaches that chain — refused even with `duty` in `include`', () => { for (const [arm, measure, ref] of [ ['offset', 'done_on_time_by_duty', 'duty.grace_days'], diff --git a/packages/services/service-analytics/src/__tests__/now-anchor-dataset.test.ts b/packages/services/service-analytics/src/__tests__/now-anchor-dataset.test.ts index 497aedc2f47..efa4ba84809 100644 --- a/packages/services/service-analytics/src/__tests__/now-anchor-dataset.test.ts +++ b/packages/services/service-analytics/src/__tests__/now-anchor-dataset.test.ts @@ -332,12 +332,14 @@ describe('[#25808] the dataset, valued end to end on the engine path', () => { expect(Number(byTitle.get('e')?.past_grace ?? 0)).toBe(0); }); - it('the dotted offset spelling validates at the schema door and is refused by SQL push-down by name', async () => { + it('[#26601] the dotted offset spelling is refused at the schema door, and past it by the read, by name', async () => { const dotted = { completed_on: { $lte: { $now: true, addDays: { $field: 'duty.sla_days' } } } }; - expect(DatasetSchema.safeParse({ + const saved = DatasetSchema.safeParse({ ...TASK_HEALTH, measures: [{ name: 'within_grace', aggregate: 'count', filter: dotted }], - }).success).toBe(true); + }); + expect(saved.success).toBe(false); + expect(saved.error?.issues[0]?.message).toContain('"duty.sla_days" is a dotted path'); const err = await (async () => { try { await service.queryDataset( diff --git a/packages/spec/src/data/filter-comparand-type.test.ts b/packages/spec/src/data/filter-comparand-type.test.ts index 8da5b5250bc..5970b8e7134 100644 --- a/packages/spec/src/data/filter-comparand-type.test.ts +++ b/packages/spec/src/data/filter-comparand-type.test.ts @@ -350,3 +350,95 @@ describe('[#26413] the now-anchor is closed at the door, as NowAnchorSchema decl } }); }); + +describe('[#26413 review] the subDays clause answers subDays alone, so a one-key refusal stays inside the bound', () => { + const NOT_APPLIED_TAIL = 'an unapplied filter would have returned the UNFILTERED result set.'; + + it('names the subDays rule when subDays is among the keys, and only then', () => { + const subDays = refusalOf(() => normalizeFilterComparandTypes({ opened_at: { $lt: { $now: true, addDays: -30, subDays: 7 } } })); + expect(subDays?.message).toContain('there is no subDays: a negative addDays subtracts'); + const note = refusalOf(() => normalizeFilterComparandTypes({ opened_at: { $lt: { $now: true, addDays: -30, note: 'x' } } })); + expect(note?.code).toBe('INVALID_FILTER'); + expect(note?.message).toContain('undeclared key "note" — the anchor takes $now and addDays only.'); + expect(note?.message).not.toContain('subDays'); + }); + + it('a one-key refusal under the engine prefix, under $and, keeps its last sentence under 500 characters', () => { + // The contract review's measured case was 496 characters with the clause + // printed for `note`; a longer field and a path under `$and` pushed it over. + const err = refusalOf(() => normalizeFilterComparandTypes( + { $and: [{ status: 'open' }, { review_opened_at: { $lte: { $now: true, addDays: -30, note: 'x' } } }] }, + "find('support_ticket')", + )); + expect(err?.code).toBe('INVALID_FILTER'); + expect(err!.message.length).toBeLessThanOrEqual(500); + expect(err!.message.endsWith(NOT_APPLIED_TAIL)).toBe(true); + }); +}); + +describe('[#26601] a dotted addDays offset is refused at the door, on a reference and on an anchor', () => { + const REF = 'contract_type.review_sla_days'; + /** `driver-sql`'s reason for the same refusal on every SQL read, verbatim. */ + const SQL_REASON = `the addDays offset "${REF}" is a dotted path, and SQL push-down compiles same-table ` + + 'column references only (no relation traversal, no alias-qualified columns).'; + const COMPARANDS: Array<[string, Record]> = [ + ['the { $field, addDays } pair', { $field: 'review_due_at', addDays: { $field: REF } }], + ['the now-anchor', { $now: true, addDays: { $field: REF } }], + ['the now-anchor in its resolved form', { $now: '2026-03-03T12:00:00.000Z', addDays: { $field: REF } }], + ]; + + it('refuses it at every scalar comparison, in the SQL read\'s own words, with the envelope', () => { + for (const [label, comparand] of COMPARANDS) { + for (const op of ['$eq', '$ne', '$gt', '$gte', '$lt', '$lte']) { + const err = refusalOf(() => normalizeFilterComparandTypes({ review_started_at: { [op]: comparand } })); + expect(err?.code, `${label} ${op}`).toBe(StandardErrorCode.enum.INVALID_FILTER); + expect(err?.status, `${label} ${op}`).toBe(400); + expect(err?.message, `${label} ${op}`).toContain(`at where.review_started_at.${op} is refused: ${SQL_REASON}`); + } + } + }); + + it('refuses it under $and / $or / $not and through parseFilterAST\'s two forms', () => { + for (const [label, comparand] of COMPARANDS) { + for (const filter of [ + { $and: [{ status: 'open' }, { review_started_at: { $lt: comparand } }] }, + { $or: [{ review_started_at: { $gte: comparand } }] }, + { $not: { review_started_at: { $lt: comparand } } }, + ]) { + expect(refusalOf(() => normalizeFilterComparandTypes(filter))?.code, `${label} ${JSON.stringify(filter)}`).toBe('INVALID_FILTER'); + } + expect(refusalOf(() => parseFilterAST({ review_started_at: { $lt: comparand } }))?.code, label).toBe('INVALID_FILTER'); + expect(refusalOf(() => parseFilterAST(['review_started_at', '<', comparand]))?.code, label).toBe('INVALID_FILTER'); + } + }); + + it('the refusal at the measure-filter position keeps its last sentence under 500 characters', () => { + const err = refusalOf(() => normalizeFilterComparandTypes( + { $and: [{ status: 'open' }, { contract_review_started_at: { $lte: { $now: true, addDays: { $field: 'contract_type_of_the_agreement.review_sla_days_offset' } } } }] }, + "aggregate('clm_contract_review')", + 'aggregations[0].filter', + )); + expect(err?.code).toBe('INVALID_FILTER'); + expect(err!.message.length).toBeLessThanOrEqual(500); + expect(err!.message.endsWith('an unapplied filter would have returned the UNFILTERED result set.')).toBe(true); + }); + + it('CONTROL: a same-record offset column, a literal offset and a dotted BASE reference are kept by reference', () => { + for (const comparand of [ + { $field: 'review_due_at', addDays: { $field: 'review_sla_days' } }, + { $now: true, addDays: { $field: 'review_sla_days' } }, + { $now: true, addDays: -3 }, + // The reference's own `$field` is not this rule's: memory walks it, SQL refuses it. + { $field: 'contract_type.review_due_at' }, + { $field: 'contract_type.review_due_at', addDays: 2 }, + ]) { + const filter = { review_started_at: { $lt: comparand } }; + expect(normalizeFilterComparandTypes(filter), JSON.stringify(comparand)).toBe(filter); + } + }); + + it('CONTROL: the bare implicit form keeps its own fate — an operator spec named $field, not judged here', () => { + const filter = { review_started_at: { $field: 'review_due_at', addDays: { $field: REF } } }; + expect(normalizeFilterComparandTypes(filter)).toBe(filter); + }); +}); diff --git a/packages/spec/src/data/filter-comparand-type.ts b/packages/spec/src/data/filter-comparand-type.ts index a7e880598f9..48b975f2017 100644 --- a/packages/spec/src/data/filter-comparand-type.ts +++ b/packages/spec/src/data/filter-comparand-type.ts @@ -84,7 +84,12 @@ * #7597 pins the hand-authored implicit form's fate). The door steps around * every one of those decisions. [#25808] A `$now` anchor * (`{ $now: true, addDays }`, `NowAnchorSchema`) the same way, and for the - * same reason: it is legal exactly where a reference is. + * same reason: it is legal exactly where a reference is. [#26601] One part + * of either IS judged here, at a scalar operator: an `addDays` offset + * `{ $field }` naming a dotted path, refused in `driver-sql`'s same-table + * words — ruling A on #25808 + * bounds the offset to a column of the same record, so the authoring doors + * that run this walk refuse it on save instead of on the first read. * - **Arrays outside `$in`/`$nin`/`$between`.** An array in an implicit or * scalar-operator position is answered per driver today (`driver-sql` and * `driver-memory` refuse it, each with its own message; `driver-mongodb` @@ -216,9 +221,9 @@ function isFieldReference(value: unknown): boolean { * resolved form the engine hands the executors (`$now` holding the pinned ISO * instant). Not a literal either: its legality per position is the anchor's * own ruling (the six scalar comparisons, as a `{ $field }` reference), and a - * malformed one is refused by its own schema sentence at the authoring door - * and by each executor at query time. Matches `driver-sql`'s `nowAnchorOf` on - * what an anchor IS: a non-array object whose `$now` is `true` or a string. + * malformed one is refused by this walk ({@link isWellFormedNowAnchor}). + * Matches `driver-sql`'s `nowAnchorOf` on what an anchor IS: a non-array + * object whose `$now` is `true` or a string. */ function isNowAnchor(value: unknown): boolean { return isFilterNode(value) && (value.$now === true || typeof value.$now === 'string'); @@ -260,9 +265,12 @@ function isWellFormedNowAnchor(value: Record): boolean { function malformedNowAnchorRefusal(context: string | undefined, path: string, value: Record): Error { const undeclared = undeclaredNowAnchorKeys(value); + // The subDays clause answers the one extra key it explains, so a refusal of + // any other key stays inside the 500-char client bound with its last + // sentence intact. const detail = undeclared.length > 0 ? `it carries an undeclared key ${undeclared.map((key) => preview(key)).join(', ')} — the anchor takes ` - + '$now and addDays only (there is no subDays: a negative addDays subtracts)' + + `$now and addDays only${undeclared.includes('subDays') ? ' (there is no subDays: a negative addDays subtracts)' : ''}` : value.addDays === undefined ? 'it carries no addDays — for the unshifted instant write the "{now}" placeholder' : `its addDays (${preview(value.addDays)}) is neither an integer nor a { "$field": "numeric_column" } reference`; @@ -274,6 +282,37 @@ function malformedNowAnchorRefusal(context: string | undefined, path: string, va ); } +/** + * [#26601] The column an `addDays` offset reads from — the `$field` of a + * `{ $field }` offset on a reference or on a now-anchor — when it is a DOTTED + * path, else `undefined`. Ruling A on #25808 bounds the offset to a column of + * the SAME record, and SQL push-down refuses a dotted one on every read + * (`driver-sql`, the 2026-08-06 same-table ruling); the one executor that + * walks the path, the formula evaluator, reads a lookup's id there and counts + * the offset as zero days. So the walk refuses it, and the authoring doors + * that run the walk refuse it on save rather than on the first read. + */ +function dottedAddDaysOffset(value: Record): string | undefined { + const offset = value.addDays; + if (!isFilterNode(offset)) return undefined; + const ref = offset.$field; + return typeof ref === 'string' && ref.includes('.') ? ref : undefined; +} + +/** + * The refusal of a dotted offset, its reason in `driver-sql`'s own words for + * the same refusal, so the save door and the first read say one sentence. + */ +function dottedAddDaysOffsetRefusal(context: string | undefined, path: string, ref: string): Error { + return invalidComparandError( + context, + `Filter comparand at ${path} is refused: the addDays offset ${preview(ref)} is a dotted path, and SQL ` + + 'push-down compiles same-table column references only (no relation traversal, no ' + + 'alias-qualified columns). Name a numeric column of the same record instead, holding the ' + + `related value. ${NOT_APPLIED}`, + ); +} + /** The word the refusal uses for the offending value's type. */ function describeComparandType(value: unknown): string { if (value === null) return 'null'; @@ -369,6 +408,17 @@ type ComparandVerdict = | { kind: 'narrow'; value: number } | { kind: 'refuse'; error: Error }; +/** + * [#26601] A reference or a well-formed anchor is stepped around — unless its + * `addDays` offset names a dotted path ({@link dottedAddDaysOffset}). + */ +function judgeAddDaysOffset(context: string | undefined, path: string, value: Record): ComparandVerdict { + const dotted = dottedAddDaysOffset(value); + return dotted === undefined + ? { kind: 'keep' } + : { kind: 'refuse', error: dottedAddDaysOffsetRefusal(context, path, dotted) }; +} + /** * Judge one LITERAL comparand. `skip` answers (as `keep`) the shapes the door * deliberately does not judge — see the module note's list. @@ -390,10 +440,11 @@ function judgeLiteralComparand( // field references, and plain-object filter STRUCTURE — the caller routes // those before asking. What reaches this line is a non-plain object // (Map / Set / class instance), a Symbol or a function. - if (Array.isArray(value) || isFieldReference(value)) return { kind: 'keep' }; + if (Array.isArray(value)) return { kind: 'keep' }; + if (isFieldReference(value)) return judgeAddDaysOffset(context, path, value as Record); if (isNowAnchor(value)) { return isWellFormedNowAnchor(value as Record) - ? { kind: 'keep' } + ? judgeAddDaysOffset(context, path, value as Record) : { kind: 'refuse', error: malformedNowAnchorRefusal(context, path, value as Record) }; } return { kind: 'refuse', error: comparandTypeRefusal(context, path, value) }; @@ -448,7 +499,9 @@ export function normalizeFilterComparandTypes(node: T, context?: string, path function normalizeFieldSpec(context: string | undefined, path: string, spec: unknown): unknown { // A plain object is filter STRUCTURE: an operator spec, a nested-relation / // deep-equality condition (not descended into — see the module note), or a - // field reference (stepped around — #7597 owns its fate). + // field reference (stepped around whole — #7597 owns its fate, a dotted + // `addDays` beside it included: that bare form is an operator spec named + // `$field`, which no executor implements). if (isFilterNode(spec)) { if (isFieldReference(spec)) return spec; const keys = Object.keys(spec); diff --git a/packages/spec/src/data/filter.test.ts b/packages/spec/src/data/filter.test.ts index 9895c0c3cbb..a4ae7666ac6 100644 --- a/packages/spec/src/data/filter.test.ts +++ b/packages/spec/src/data/filter.test.ts @@ -25,6 +25,8 @@ import { type QueryFilter, type FieldOperators, } from './filter.zod'; +import { DatasetSchema } from '../ui/dataset.zod'; +import { ObjectStackDefinitionSchema } from '../stack.zod'; // ============================================================================ // 3.1 Comparison Operators Tests @@ -1969,11 +1971,17 @@ describe('FieldReferenceSchema.addDays — a whole-day offset on a reference, an }).success).toBe(true); }); - it('a DOT-PATH nested reference — exactly as `$field` itself allows', () => { - // The ruling's driving shape: `completed_at <= due_date + duty.grace_days`. - expect(FieldReferenceSchema.safeParse({ + it('[#26601] NOT a dot-path nested reference — the offset column is a column of the same record', () => { + // Retires the reading that the dotted spelling validates and is answered + // in memory (the #14104 ruling-record correction): SQL push-down refuses + // it on every read, and the memory evaluator reads a lookup's id there on + // a stored row, counting the offset as zero days. + const result = FieldReferenceSchema.safeParse({ $field: 'due_date', addDays: { $field: 'duty.grace_days' }, - }).success).toBe(true); + }); + expect(result.success).toBe(false); + expect(result.error?.issues[0]?.path).toEqual(['addDays', '$field']); + expect(result.error?.issues[0]?.message.startsWith(DOTTED_OFFSET_SENTENCE('duty.grace_days'))).toBe(true); }); it('positive control — a bare reference is unchanged', () => { @@ -1983,7 +1991,7 @@ describe('FieldReferenceSchema.addDays — a whole-day offset on a reference, an }); it('rides in every ordering slot, on both the documentation and the ENFORCED copy', () => { - const ref = { $field: 'due_date', addDays: { $field: 'duty.grace_days' } }; + const ref = { $field: 'due_date', addDays: { $field: 'grace_days' } }; for (const op of ['$gt', '$gte', '$lt', '$lte'] as const) { expect(ComparisonOperatorSchema.safeParse({ [op]: ref }).success, op).toBe(true); expect(FieldOperatorsSchema.safeParse({ [op]: ref }).success, op).toBe(true); @@ -2069,6 +2077,132 @@ describe('FieldReferenceSchema.addDays — a whole-day offset on a reference, an }); }); +// ============================================================================ +// [#26601] A DOTTED `addDays` offset — refused at every authoring door, in the +// words SQL push-down already refuses it with on every read +// ============================================================================ + +/** + * `driver-sql`'s reason for the same refusal at read time + * (`sql-driver-cross-field-comparison.ts`), capitalised as a first sentence, + * plus the prescription. The save door and the first read say one sentence. + */ +function DOTTED_OFFSET_SENTENCE(ref: string): string { + return `The addDays offset "${ref}" is a dotted path, and SQL push-down compiles same-table column ` + + 'references only (no relation traversal, no alias-qualified columns)'; +} + +describe('[#26601] a dotted addDays offset is refused at the save and validate doors', () => { + const SLA = 'contract_type.review_sla_days'; + const ANCHOR = { $now: true, addDays: { $field: SLA } }; + const PAIR = { $field: 'review_started_at', addDays: { $field: SLA } }; + /** The walk's own wording, as the `FilterCondition` door reports it. */ + const WALK_REASON = `the addDays offset "${SLA}" is a dotted path, and SQL push-down compiles same-table ` + + 'column references only (no relation traversal, no alias-qualified columns).'; + const MEASURED_SHAPE = { review_started_at: { $lt: ANCHOR } }; + const DATASET = { + name: 'contract_review', + label: 'Contract review', + object: 'clm_contract', + include: ['contract_type'], + dimensions: [{ name: 'contract_type', field: 'contract_type', type: 'string' }], + measures: [{ name: 'total', aggregate: 'count' }], + }; + const issuesOf = (result: { success: boolean; error?: { issues: Array<{ path: PropertyKey[]; message: string }> } }) => + result.error?.issues ?? []; + + it('the schemas themselves: FieldReferenceSchema.addDays and NowAnchorSchema.addDays', () => { + for (const [label, result] of [ + ['reference', FieldReferenceSchema.safeParse(PAIR)], + ['anchor', NowAnchorSchema.safeParse(ANCHOR)], + ] as const) { + expect(result.success, label).toBe(false); + expect(issuesOf(result)[0]?.path, label).toEqual(['addDays', '$field']); + expect(issuesOf(result)[0]?.message.startsWith(DOTTED_OFFSET_SENTENCE(SLA)), label).toBe(true); + expect(issuesOf(result)[0]?.message, label).toContain('name a numeric column of the same record instead'); + expect(issuesOf(result)[0]?.message, label).not.toMatch(/#\d{3,}/); + } + }); + + it('every ordering slot, on the documentation copy and the enforced copy', () => { + // `$eq` / `$ne` admit any comparand at the operator schema; the + // `FilterCondition` door below judges them through the comparand walk. + for (const comparand of [PAIR, ANCHOR]) { + for (const op of ['$gt', '$gte', '$lt', '$lte'] as const) { + const enforced = FieldOperatorsSchema.safeParse({ [op]: comparand }); + expect(enforced.success, op).toBe(false); + expect(issuesOf(enforced)[0]?.path, op).toEqual([op, 'addDays', '$field']); + expect(issuesOf(enforced)[0]?.message.startsWith(DOTTED_OFFSET_SENTENCE(SLA)), op).toBe(true); + } + expect(ComparisonOperatorSchema.safeParse({ $lt: comparand }).success).toBe(false); + } + }); + + it('a data filter: the FilterCondition save door, at the slot, under $and / $or / $not too', () => { + for (const comparand of [PAIR, ANCHOR]) { + for (const op of ['$eq', '$ne', '$gt', '$gte', '$lt', '$lte'] as const) { + const result = FilterConditionSchema.safeParse({ review_started_at: { [op]: comparand } }); + expect(result.success, op).toBe(false); + expect(issuesOf(result)[0]?.path, op).toEqual(['review_started_at', op]); + expect(issuesOf(result)[0]?.message, op).toContain(WALK_REASON); + } + for (const wrapped of [ + { $and: [{ status: 'open' }, { review_started_at: { $lt: comparand } }] }, + { $or: [{ review_started_at: { $gte: comparand } }] }, + { $not: { review_started_at: { $lt: comparand } } }, + ]) { + expect(FilterConditionSchema.safeParse(wrapped).success, JSON.stringify(wrapped)).toBe(false); + } + } + }); + + it('a dataset measure filter and a dataset filter: the dataset write door', () => { + const measure = DatasetSchema.safeParse({ + ...DATASET, + measures: [...DATASET.measures, { name: 'over_sla', aggregate: 'count', filter: MEASURED_SHAPE }], + }); + expect(measure.success).toBe(false); + expect(issuesOf(measure)[0]?.path).toEqual(['measures', 1, 'filter', 'review_started_at', '$lt']); + expect(issuesOf(measure)[0]?.message).toContain(WALK_REASON); + const scope = DatasetSchema.safeParse({ ...DATASET, filter: MEASURED_SHAPE }); + expect(scope.success).toBe(false); + expect(issuesOf(scope)[0]?.path).toEqual(['filter', 'review_started_at', '$lt']); + expect(issuesOf(scope)[0]?.message).toContain(WALK_REASON); + }); + + it('the validate door: a stack carrying the dataset fails ObjectStackDefinitionSchema', () => { + const stack = (dataset: Record) => ({ + manifest: { id: 'com.example.dotted-offset', name: 'dotted_offset', version: '1.0.0', type: 'app' }, + datasets: [dataset], + }); + const result = ObjectStackDefinitionSchema.safeParse(stack({ + ...DATASET, + measures: [...DATASET.measures, { name: 'over_sla', aggregate: 'count', filter: MEASURED_SHAPE }], + })); + expect(result.success).toBe(false); + expect(issuesOf(result).map((i) => i.path)).toEqual([['datasets', 0, 'measures', 1, 'filter', 'review_started_at', '$lt']]); + // CONTROL: the same stack with a same-record offset column passes the same door. + const sameRecord = { review_started_at: { $lt: { $now: true, addDays: { $field: 'review_sla_days' } } } }; + expect(ObjectStackDefinitionSchema.safeParse(stack({ + ...DATASET, + measures: [...DATASET.measures, { name: 'over_sla', aggregate: 'count', filter: sameRecord }], + })).success).toBe(true); + }); + + it('CONTROL: a same-record offset passes every door; a dotted BASE reference keeps its asymmetry', () => { + for (const comparand of [ + { $now: true, addDays: { $field: 'review_sla_days' } }, + { $field: 'review_started_at', addDays: { $field: 'review_sla_days' } }, + ]) { + expect(FilterConditionSchema.safeParse({ review_started_at: { $lt: comparand } }).success).toBe(true); + expect(FieldOperatorsSchema.safeParse({ $lt: comparand }).success).toBe(true); + } + // Out of scope here: the reference's own `$field` — memory walks it, SQL + // refuses it — is unchanged by this rule. + expect(FilterConditionSchema.safeParse({ completed_on: { $lte: { $field: 'duty.due_on' } } }).success).toBe(true); + }); +}); + // ============================================================================ // [#16923] The docblock @examples are RUNNABLE, and same-table // ============================================================================ diff --git a/packages/spec/src/data/filter.zod.ts b/packages/spec/src/data/filter.zod.ts index 233f69314ff..2403f93ae36 100644 --- a/packages/spec/src/data/filter.zod.ts +++ b/packages/spec/src/data/filter.zod.ts @@ -98,9 +98,15 @@ import { OPERATOR_PREFIX_KEY_PATTERN, bannedKeyPattern } from '../shared/refinem * never to a column. The ruling put the offset ON the reference: `addDays` is * either an INTEGER literal of any sign (a negative value subtracts; there is no * `subDays`, and no other unit — whole days only) or a nested `{ $field }` - * reference to a numeric column (dot-path allowed exactly as `$field` allows it; - * SQL push-down applies the same same-table rule to it). Options B (a derived + * reference to a numeric column of the SAME record. Options B (a derived * date-difference measure) and C (a computed dimension) were not taken. + * [#26601] A DOTTED offset path is refused at every door, the authoring doors + * included: SQL push-down refuses it on every read (the 2026-08-06 same-table + * rule), and the one executor that walks the path, the memory evaluator, reads + * a lookup's id there on a stored row and counts the offset as zero days. This + * retires the earlier reading that the dotted spelling validates and is + * answered in memory; the base `$field` itself keeps the asymmetry stated + * above. * * The NULL semantics are stated here, and pinned, rather than inherited from * SQL three-valued logic — the shape #5146 used for `$not`: @@ -183,7 +189,8 @@ import { OPERATOR_PREFIX_KEY_PATTERN, bannedKeyPattern } from '../shared/refinem * * Both standing rulings hold unchanged: the offset column is a column of the * SAME record (letter A; a path-valued offset — B′ — was not taken), and no - * JOIN is planned for it. + * JOIN is planned for it. [#26601] So a dotted offset is refused at the + * authoring doors too, in the driver's words, not first on a read. * * @see https://github.com/objectstack-ai/objectstack/issues/25808 (the `$now` anchor) * @@ -238,6 +245,8 @@ function addDaysOffsetMessage(input: unknown): string { 'addDays is an integer literal of any sign (a negative value subtracts; whole days only, no ' + 'other unit) or a nested { "$field": "numeric_column" } reference. A NULL offset column ' + 'contributes zero days; a NULL referenced column makes the comparison false.'; + const dotted = dottedAddDaysReference(input); + if (dotted !== undefined) return `${dottedAddDaysOffsetSentence(dotted)} ${prescription}`; if (typeof input === 'number') { return `addDays must be a whole number of days, and ${String(input)} is not an integer. ${prescription}`; } @@ -254,13 +263,41 @@ function addDaysOffsetMessage(input: unknown): string { return `addDays must be an integer or a { "$field" } reference. ${prescription}`; } +/** + * [#26601] The `$field` of an `addDays` offset object when it names a DOTTED + * path, else `undefined`. The offset column is a column of the same record + * (ruling A on #25808), and SQL push-down refuses a dotted one on every read. + */ +function dottedAddDaysReference(input: unknown): string | undefined { + if (!input || typeof input !== 'object' || Array.isArray(input)) return undefined; + const ref = (input as Record).$field; + return typeof ref === 'string' && ref.includes('.') ? ref : undefined; +} + +/** + * [#26601] The dotted-offset refusal, in `driver-sql`'s own words for the same + * refusal at read time, so the save door and the first read say one sentence. + * The comparand-type walk (`filter-comparand-type.ts`) spells it too, for the + * `FilterCondition` door; `filter.test.ts` pins both to this wording. + */ +function dottedAddDaysOffsetSentence(ref: string): string { + return `The addDays offset ${JSON.stringify(ref)} is a dotted path, and SQL push-down compiles ` + + 'same-table column references only (no relation traversal, no alias-qualified columns): name a ' + + 'numeric column of the same record instead, holding the related value.'; +} + /** * [#14104] The offset's nested reference. Deliberately its OWN object rather * than `FieldReferenceSchema` again: an offset on an offset has no ruled - * meaning, so the nested shape carries `$field` and nothing else. + * meaning, so the nested shape carries `$field` and nothing else. [#26601] A + * dotted `$field` is refused here: see {@link dottedAddDaysReference}. */ const AddDaysReferenceSchema = z.object({ - $field: z.string().describe('Numeric column whose value is the number of days to add'), + // A pattern rather than a refinement, so the published JSON Schema carries + // the rule too (`pattern`) instead of joining the dropped-refinement ledger. + $field: z.string() + .regex(/^[^.]*$/, { error: (issue) => dottedAddDaysOffsetSentence(String(issue.input)) }) + .describe('Numeric column of the same record whose value is the number of days to add (no dotted path)'), }).describe('A { $field } reference to the numeric column holding the day offset'); export const FieldReferenceSchema = lazySchema(() => z.object({ From 2cf756c89ebc86aaf2eb89c20747e99c98b7c577 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 27 Oct 2026 10:45:13 +0000 Subject: [PATCH 2/3] chore(spec): regenerate the filter reference for the addDays offset describe Claude-Session: https://claude.ai/code/session_01WGufzeyqyEtWpnqFTkcC9r Co-Authored-By: Claude --- content/docs/references/data/filter.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/references/data/filter.mdx b/content/docs/references/data/filter.mdx index 6b2eb2e3606..08a3efbf10c 100644 --- a/content/docs/references/data/filter.mdx +++ b/content/docs/references/data/filter.mdx @@ -240,7 +240,7 @@ A `{ $field }` reference to the numeric column holding the day offset | Property | Type | Required | Description | | :--- | :--- | :--- | :--- | -| **$field** | `string` | ✅ | Numeric column whose value is the number of days to add | +| **$field** | `string` | ✅ | Numeric column of the same record whose value is the number of days to add (no dotted path) | --- @@ -389,7 +389,7 @@ A `{ $field }` reference to the numeric column holding the day offset | Property | Type | Required | Description | | :--- | :--- | :--- | :--- | -| **$field** | `string` | ✅ | Numeric column whose value is the number of days to add | +| **$field** | `string` | ✅ | Numeric column of the same record whose value is the number of days to add (no dotted path) | --- From b9c396e8768774563c3341348bcc5b29998a3d73 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 27 Oct 2026 10:54:07 +0000 Subject: [PATCH 3/3] test(service-analytics): a dataset measure needs an identifier of two characters or more Claude-Session: https://claude.ai/code/session_01WGufzeyqyEtWpnqFTkcC9r Co-Authored-By: Claude --- .../__tests__/include-relation-cross-field-boundary.test.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/services/service-analytics/src/__tests__/include-relation-cross-field-boundary.test.ts b/packages/services/service-analytics/src/__tests__/include-relation-cross-field-boundary.test.ts index 8bbf3f64839..301bfa646a5 100644 --- a/packages/services/service-analytics/src/__tests__/include-relation-cross-field-boundary.test.ts +++ b/packages/services/service-analytics/src/__tests__/include-relation-cross-field-boundary.test.ts @@ -226,12 +226,12 @@ describe('[#15103] a cross-field comparand on an `include`d relation — the bou describe('[#26601] the save door refuses the dotted OFFSET; the dotted bare reference still validates', () => { it('DatasetSchema refuses the offset arm at its slot, in the SQL read\'s words, and keeps the bare arm', () => { - const offset = DatasetSchema.safeParse({ ...PARSED, measures: [{ name: 'x', aggregate: 'count', filter: RELATION_OFFSET_ON_TIME }] }); + const offset = DatasetSchema.safeParse({ ...PARSED, measures: [{ name: 'by_duty', aggregate: 'count', filter: RELATION_OFFSET_ON_TIME }] }); expect(offset.success).toBe(false); expect(offset.error?.issues[0]?.path).toEqual(['measures', 0, 'filter', 'completed_on', '$lte']); expect(offset.error?.issues[0]?.message).toContain('"duty.grace_days" is a dotted path'); expect(offset.error?.issues[0]?.message).toContain('same-table column references only'); - expect(DatasetSchema.safeParse({ ...PARSED, measures: [{ name: 'x', aggregate: 'count', filter: RELATION_BARE }] }).success).toBe(true); + expect(DatasetSchema.safeParse({ ...PARSED, measures: [{ name: 'by_duty', aggregate: 'count', filter: RELATION_BARE }] }).success).toBe(true); }); });