Repository navigation
Commit d1663ba
feat(spec,analytics,lint,platform-objects,app-shell): a dataset measure can aggregate the time between two datetime fields of each record (#25903)
<!-- claude-projects-attribution: {"github_login":"marchtian"} -->
_Requested by **田騏** · [project
thread](https://claude.ai/code/project/chan_01VmuUhXU9SSUzqJYyXSmYSi?thread=cmsg_01VmuUhXU9SSUzqJYyXSmYSi9FC1KtovKr42Nv6HsVMzxW)_
Fixes #25778
Clause-②: yes (widening)
Metadata-protocol: added
## Summary
A dataset measure can now aggregate the time between two `datetime`
fields of each record. This is item 1 of the card. Item 2 (comparing
against "now minus N days" where N lives on a related record) needs its
own key, so it is split to #25808 as the order directed.
```ts
{ name: 'avg_hours_to_resolve', aggregate: 'avg', duration: { from: 'created_at', to: 'resolved_at', unit: 'hours' } }
```
- **Spec** (`packages/spec/src/ui/dataset.zod.ts`):
`DatasetMeasureSchema` gains an optional `duration: { from, to, unit }`
in place of `field`.
- `from` and `to` are two bare column names of the dataset's own object.
- `unit` is `seconds | minutes | hours | days`.
- `aggregate` is `avg | min | max | sum`.
- Refused at parse: a path, an expression or `*` in either column, the
same column twice, any other unit, a counting or missing aggregate, and
`field` / `derived` / `currency` beside it.
- No expression language and no SQL enter the metadata (ADR-0021).
- **Analytics**:
- The compile refuses a column not declared `datetime` with `400
DATASET_INVALID`.
- `NativeSQLStrategy` computes `to - from` per row in double precision,
per dialect: SQLite `julianday`, PostgreSQL `EXTRACT(EPOCH …)`, MySQL
`TIMESTAMPDIFF(MICROSECOND …) / 1.0E6`. It divides by the unit and wraps
the result in the aggregate, or in the conditional aggregate when the
measure has a scoped filter.
- The draft preview computes the same value from the rows.
- Nulls in either column are excluded. A `to` before its `from` counts
as negative. Over a group with no record carrying both, `sum` answers
`0` and the others `null`. The `0` is answered in the statement
(`withEmptyGroupIdentity`, as for every other measure since #25953), so
it sorts and pages as `0`.
- **Fail-closed**: the measure is deliberately absent from
`cube.measures`. Every other path refuses it with `400 INVALID_FIELD`,
naming the measure: a `granularity` bucket, a field-to-field filter, an
object served through middleware or an external datasource, an unknown
dialect, or a non-SQL driver. The field-level read gate judges both
columns. `getMeta` lists the measure, and the `CUBE.MEASURE` spelling it
lists is served too.
- **`os validate`**: new gating rule
`measure-duration-field-type-refused`, at
`datasets[N].measures[M].duration.from` / `.to`, with an `os explain`
entry. `dataset-field-unknown` now covers both columns.
- **Console** (`ui/packages/app-shell`, in this repo since the objectui
merge): the dataset inspector's base-object change now clears a duration
measure with the other measures that name the old object's fields.
Before this it was kept as if it were a plain row count, because it has
no `field`.
- **Docs**: `data-modeling/analytics.mdx` gets a "Duration measures"
section, and `deployment/validating-metadata.mdx` explains the rule. The
liveness ledger drills `duration.{from,to,unit}`.
## Tests
-
`packages/services/service-analytics/src/__tests__/dataset-duration-measure.test.ts`:
live on **SQLite and PostgreSQL**, with 7 cases on each driver (14/14 in
total).
- Covers values per unit and aggregate, nulls, a negative duration, an
empty group, a measure filter, a derived measure, ordering with limit
and offset (the all-NULL group pages as `0` on both drivers), and the
cube door.
- Rejections assert `code` + `status`: granularity, objectql-only, and
date / number / text columns.
- `native-sql-duration-measure.test.ts` (23):
- exact SQL per dialect, joins, temporal repair, the unknown-dialect
decline, presentation, and the empty-group identity in the statement;
- the preview evaluator;
- service gates: no mint, the read gate (including a dimension sharing
the measure's name), dry-run, preview, the unparsed-dataset door,
compile guards, `getMeta`, the `CUBE.MEASURE` spelling, and the `$field`
refusal.
- `packages/spec/src/ui/dataset-measure-duration.test.ts` (18),
`packages/lint/src/validate-dataset-measure-duration.test.ts` (12), and
the ADR-0122 isomorphism pins.
-
`ui/packages/app-shell/.../DatasetDefaultInspector.firstPickKeepsCount-12848.test.tsx`:
the move fixture gains a duration measure, and the move clears it. The
test fails with the predicate reverted.
- Full suites: service-analytics 4842 pass at `8821a8d4`, with 1
pre-existing failure ("the server is not on UTC", because the local PG
runs on UTC). Lint 7310 pass. `tsc --noEmit` is clean for
service-analytics, and the typecheck of lint, platform-objects and spec
exits 0.
- Not measured locally: the `@object-ui/app-shell` type-check.
`@object-ui/components` fails its own declaration build in this
container at the main tip, in files this diff does not touch, so CI's
`Type Check UI` is the reading for it. It is green on this head.
## Review
An independent full review ran on Fable. Round 1 was CHANGES REQUESTED,
with 1 must-fix, 2 should-fix and 4 nits:
- the read-gate escape when a dimension shares the name;
- MySQL DECIMAL arithmetic;
- `getMeta` not listing the measure;
- the `CUBE.MEASURE` spelling;
- docs and comment nits.
All of them are applied except F6 (refusing a dimension/measure name
collision at parse), which is a narrowing and so is out of scope. Three
delta reviews followed, the last one after the ui/ merge, and each
answered ACCEPT. The verdict record is in a comment on this PR.
## Acceptance notes
- **Item 2 → #25808.** It needs a now-anchored comparand with a `$field`
offset read across a relationship, and the #5222 rulings keep SQL
`$field` to the same table.
- **Not served off the native-SQL strategy.** The engine/ObjectQL
aggregate has no node for a two-column operand, so a duration with a
`granularity` bucket (a monthly trend) is refused loudly rather than
answered. Follow-up candidate: an engine aggregation operand for
durations.
- **The console inspector has no editor for `duration`.** A duration
measure keeps its key through edits (`patchMeasure` spreads the row),
and picking a field beside it is refused by the spec at save. Follow-up
candidate: a from / to / unit editor in `DatasetDefaultInspector`.
- **No wire `unit` on result fields.** The unit lives on the dataset
definition, and adding it to the result is a further protocol addition.
Deliberately not done here.
- **MySQL is pinned as text only.** This package runs no live MySQL, so
the docs claim live parity for SQLite and PostgreSQL only.
- **Pre-existing, measured: a filtered ordinary measure asked as
`CUBE.MEASURE` drops its filter.** `query({ cube: 'sales', measures:
['sales.won_amount'] })` runs `SUM(amount)` while `['won_amount']` runs
`SUM(CASE WHEN stage = $1 THEN amount END)`. That spelling is the one
`getMeta` lists. The duration measure resolves its filter by the bare
key, but ordinary measures do not. Filed as #25906.
- **An all-NULL sum group now sorts and pages as its `0` (#25907, fixed
on main by #25953).** It used to fold to `0` only after the statement
had ordered and paged it as NULL, at opposite ends on SQLite and
PostgreSQL. Since the merge of main at `38050ced`, the duration measure
takes the same in-statement identity, and the live ordering test
includes that group: ascending, it pages b (-3599.25), c (0), a (91800)
on both drivers.
- **Unmeasured: a dimension and a measure may share a name**
(`DatasetSchema` checks uniqueness per list). Refusing that is a
narrowing (review F6).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01AKPixnkg91qCkQCVFDJuqW
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 0479cd6 commit d1663ba
40 files changed
Lines changed: 2079 additions & 73 deletions
File tree
- .changeset
- content/docs
- data-modeling
- deployment
- references
- ui
- docs/audits/2026-07-unknown-key-strictness-ledger.counts
- packages
- lint/src
- platform-objects/src/apps/translations
- services/service-analytics/src
- __tests__
- strategies
- spec
- api-surface
- authorable-surface
- declaration-map
- export-origins
- json-schema.manifest
- liveness
- state-counts
- src
- data
- ui
- ui/packages/app-shell/src/views/metadata-admin/inspectors
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
| 67 | + | |
| 68 | + | |
67 | 69 | | |
68 | 70 | | |
69 | 71 | | |
| |||
94 | 96 | | |
95 | 97 | | |
96 | 98 | | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
97 | 131 | | |
98 | 132 | | |
99 | 133 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
249 | 249 | | |
250 | 250 | | |
251 | 251 | | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
252 | 262 | | |
253 | 263 | | |
254 | 264 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
36 | | - | |
37 | | - | |
| 36 | + | |
| 37 | + | |
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| |||
366 | 366 | | |
367 | 367 | | |
368 | 368 | | |
369 | | - | |
| 369 | + | |
370 | 370 | | |
371 | 371 | | |
372 | 372 | | |
| |||
379 | 379 | | |
380 | 380 | | |
381 | 381 | | |
382 | | - | |
| 382 | + | |
383 | 383 | | |
384 | 384 | | |
385 | 385 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
38 | | - | |
| 37 | + | |
| 38 | + | |
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| |||
89 | 89 | | |
90 | 90 | | |
91 | 91 | | |
| 92 | + | |
92 | 93 | | |
93 | 94 | | |
94 | 95 | | |
| |||
115 | 116 | | |
116 | 117 | | |
117 | 118 | | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
118 | 131 | | |
119 | 132 | | |
120 | 133 | | |
| |||
131 | 144 | | |
132 | 145 | | |
133 | 146 | | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
134 | 169 | | |
135 | 170 | | |
136 | 171 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
| 24 | + | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
| 41 | + | |
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
57 | | - | |
| 57 | + | |
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
65 | | - | |
| 65 | + | |
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
| 63 | + | |
63 | 64 | | |
64 | 65 | | |
65 | 66 | | |
| |||
0 commit comments