Repository navigation
Commit d1633f3
Fixes #20889
Clause-②: yes (widening)
## What changed
On PostgreSQL, the analytics native-SQL path answered every measure its
response declares `number` as a string: `count: "2"`, `sum:
"500.000000000000000000000000000000"`. SQLite answered numbers.
`driver-sql`'s own `aggregate()` has presented these answers since
#20335, but through a private table and presenter that
`service-analytics` could not reach. The seat's route ruling
(`5922780640`) moves that one rule into `@objectstack/core`, and both
faces now call it.
- **`@objectstack/core`** (new `src/utils/aggregate-answer.ts`, one
export line in `index.ts` beside `compensatedSum`):
- `AGGREGATE_ANSWER_KIND`, with its precision-policy docblock, moved
from `sql-driver.ts`. The text is byte-identical apart from the `export`
keyword; the proof is below.
- `presentAsNumber(value)`, the body of `presentReadValue`'s `'number'`
arm as a named function. A string `Number()` reads as a number becomes
that number. Any other value is returned as given.
- **`@objectstack/driver-sql`**, `sql-driver.ts`, the two ruled regions
only:
- the `AGGREGATE_ANSWER_KIND` definition becomes an import, and a
pointer comment marks where it was;
- `presentReadValue`'s `'number'` arm returns `presentAsNumber(value)`.
The import is a separate statement beside the `AggregationFunction`
import (inserted after base line 23; line 27 at HEAD). It does NOT join
the `@objectstack/core` import block at base lines 94-100, which PR
#20988 rewrites. That hunk starts at base line 91, 67 lines below the
insertion. No behaviour change: the full driver-sql suite is green on
SQLite and PostgreSQL (below).
- **`@objectstack/service-analytics`**, `NativeSQLStrategy.execute`
only. After the raw statement runs, each measure column is presented by
the measure's DECLARED aggregate function (`lookupMember(cube, m,
'measure').type`), never by whether a value looks numeric:
- `count`, `count_distinct`, `sum` and `avg` (the table's `'number'`
kind) always take `presentAsNumber`;
- `min` / `max` (the `'column'` kind) take it only when
`declaredFieldType(object, column)` is in `NUMERIC_VALUE_TYPES`. That is
the rule `driver-sql`'s `readPresentationKind` applies to a declared
numeric column. A host that cannot answer, or a relationship-path
column, leaves the value as given;
- expression metric types (`number` / `string` / `boolean`) are left as
they are (ruling Q3).
One point covers both doors: the cube read (`POST
/api/v1/analytics/query`) and the dataset door (`POST
/api/v1/analytics/dataset/query`, through `DatasetExecutor`).
- **`.changeset/20889-analytics-native-measure-number.md`**:
`@objectstack/core` minor, `@objectstack/driver-sql` and
`@objectstack/service-analytics` patch, `Clause-②: yes (widening)` for
the two new root exports.
## Per-measure answers, before and after
Fixture: `text` category; `rating` stars (an integer column); `number`
amount and frac (the exact-decimal column); `currency` price. Group `a`
holds 2 rows.
- **Before, PostgreSQL:** phase 0's reading at base `7fa67dada3`,
through `AnalyticsService.query` (what the route relays verbatim). The
red pins at `719b98795e` read the same strings.
- **After:** read at `e827c237d1` through the real `dispatcher-plugin`
mount of `POST /api/v1/analytics/query`, by a scratch probe that was
deleted after the run.
| function | column (declared type) | SQLite before | SQLite after |
PostgreSQL 16.13 before | PostgreSQL 16.13 after |
|:--|:--|:--|:--|:--|:--|
| `count` | `*` | `2` | `2` | `"2"` | `2` |
| `count_distinct` | note (`text`) | `2` | `2` | `"2"` | `2` |
| `sum` | stars (`rating`, integer) | `7` | `7` | `"7"` | `7` |
| `avg` | stars (`rating`, integer) | `3.5` | `3.5` |
`"3.5000000000000000"` | `3.5` |
| `sum` | amount (`number`, decimal) | `500` | `500` |
`"500.000000000000000000000000000000"` | `500` |
| `avg` | amount (`number`, decimal) | `250` | `250` |
`"250.000000000000000000000000000000"` | `250` |
| `min` | amount (`number`) | `100` | `100` |
`"100.000000000000000000000000000000"` | `100` |
| `max` | price (`currency`) | `20.5` | `20.5` |
`"20.500000000000000000000000000000"` | `20.5` |
| `max` | note (`text`), the control | not read at base | `"y"` | not
read at base | `"y"` (left as the client gave it) |
| `sum` | amount over `9007199254740993` | `9007199254740992` |
`9007199254740992` | `"9007199254740993.000000000000000000000000000000"`
| `9007199254740992` |
| dataset `row_count` (`count`) | `*` | `2` | `2` | `"2"` | `2` |
| dataset `filtered_count`, a group the statement answered | `*` | `1` |
`1` | `"1"` | `1` |
| expression measure, type `number` | `SUM(amount) / 2` | `250` | `250`
| `"250.000000000000000000000000000000"` | unchanged, left as is (ruling
Q3) |
`fields[]` declared `number` for every measure on both dialects, before
and after. On the ObjectQL face of the same route, every answer was
already a number on both dialects, and it is unchanged.
## Move proof
`move-proof.py` (scratch) reads the block from BASE `2f2fa11d75`'s
`sql-driver.ts` and from HEAD's `aggregate-answer.ts`:
```text
table+docblock: base lines 57, core lines 57; differing lines 1: ["export const AGGREGATE_ANSWER_KIND: Readonly(...) = {"] (type arguments elided in this copy only)
sha1 base e899be861c6e5e13cab24558923e41b0ca7b740c
sha1 core (export keyword removed) e899be861c6e5e13cab24558923e41b0ca7b740c
presenter: base arm body lines 8, core function body lines 8; identical after de-indent: True
MOVE PROOF: PASS
```
The table and its docblock are byte-identical apart from `export`. The
presenter's eight body lines, comment included, are identical once
de-indented. In core, a new file header says why the rule lives there.
The moved docblock still speaks of `SqlDriver.aggregate` and
`formatOutput` in the driver's voice, and the header says so.
## Pins (red first), the ablation, and the local PostgreSQL run
- **Red, at `719b98795e`** (pins only, no fix), on a private PostgreSQL
16.13 (`initdb`, 127.0.0.1, trust):
- `@objectstack/core` `aggregate-answer.test.ts`: 1 file failed, `Cannot
find module './aggregate-answer'`.
- `service-analytics` `native-sql-measure-number-presentation.test.ts`:
5 failed, all PostgreSQL; 5 passed, all SQLite. First failure: `a
row_count is a number, never "2": expected 'string' to be 'number'`.
- `rest`, the new `analytics-dataset-measure-number-door.test.ts` and
the tightened `analytics-dataset-json-dimension-door.test.ts`: 2 failed,
both PostgreSQL; 6 passed. Failures: `a row_count is a number, never
"2"` and `expected [ [ 'x', '2' ], [ 'y', '1' ] ] to deeply equal [ [
'x', 2 ], [ 'y', 1 ] ]`.
- **Green, at `e827c237d1`:** core 5/5; service-analytics 10/10 (SQLite
and PostgreSQL); rest 8/8 (SQLite and PostgreSQL).
- **Ablation** (the strategy stops calling the presenter). Predicted
before running: exactly the PostgreSQL cells go red, service-analytics 5
and rest 2; every SQLite test and the core pin stay green.
- The mutation went through `scripts/ablation-replace.mjs` (WRAP mode,
with its restore trap) on the absolute path. The guard `if
(numberMeasures.length > 0 && Array.isArray(rows))` gained `&&
String(numberMeasures) === 'ABLATION-20889'`, so it never holds. Anchor
count went 1 to 0, blob `3bce7bb073` to `8f36f62dfe`.
- `service-analytics` was rebuilt, and `ablation-dist-preflight.mjs`
found the marker in 2 built files.
- Observed: service-analytics 5 failed (all PostgreSQL), 5 passed; rest
2 failed (both PostgreSQL), 6 passed; core 5/5. That is exactly as
predicted.
- Restore: the blob after restore equals HEAD (`3bce7bb073`), `git diff
HEAD` is 0 bytes, and porcelain is 0 lines. Then a rebuild,
`ablation-dist-preflight.mjs --absent` (marker absent from all 6 built
files, tree clean), and a re-run: all green.
- **`driver-sql`'s full suite at `e827c237d1`**, SQLite and live
PostgreSQL, `TZ=America/New_York`, with the server at `Asia/Shanghai` as
CI runs it: 210 files passed, 3 skipped; 4061 tests passed, 95 skipped.
`sql-driver-20335-aggregate-numeric-presentation.test.ts` 15 tests (1
skipped, the MySQL cell), and
`sql-driver-20387-aggregate-double-accumulation.test.ts` 11 (1 skipped).
A first run on a UTC server failed only the four suites whose
precondition asserts a non-UTC server.
- **CI:** no CI job provisions `OS_TEST_POSTGRES_URL` for
`service-analytics` or `rest`, so their PostgreSQL cells are named skips
there (`skipped: set OS_TEST_POSTGRES_URL to run this cell`). The core
pin (string to number) and every SQLite cell run in CI. The private
cluster was stopped and its data directory removed after the runs.
## Gates
All at HEAD `e827c237d1`, as ONE sequential script under the shared
verify lock, each exit code captured before any pipe.
- **Derived families:** `node scripts/pm/dispatch-gates.mjs --commands
--repo objectstack-ai/objectstack` names 67. `--ran` reconciles: `67
derived famil(ies) accounted for — 65 run, 2 NOT-MEASURED`, 0 unrun. All
65 exit 0.
- **NOT MEASURED: `check:dual-build-cjs-loads` and
`check:type-check-debt`.** Reason: both exit 3, `PREREQUISITE NOT MET`.
They read the built output of every package, and this run built only the
dependency closure it tests. CI's `lint.yml` builds the full closure
before them.
- **The four roster families the derivation marks ⛔ (roster in a
directory this diff touches):** `check-changeset-fixed.mjs`,
`check:authz-resolver`, `check:error-code-casing`,
`check:filter-alias-parity`. All exit 0.
- **Tests and typechecks:**
- `@objectstack/core` test: 62 files, 1803 tests passed. Typecheck exit
0, and its `check:test-typecheck` covers the new test.
- `@objectstack/driver-sql`: typecheck exit 0; the full suite as above.
- `@objectstack/service-analytics` full suite with
`OS_TEST_POSTGRES_URL` set: 151 files, 3468 tests passed. Typecheck exit
0; `tsc --listFiles` includes the new pin.
- `@objectstack/rest`: the two pin files, SQLite and PostgreSQL, 8/8.
Typecheck exit 0, with `check:test-typecheck` over `tsconfig.test.json`.
- **ESLint, a declared narrowing** (the repo-wide `pnpm lint` is CI's):
- Touched files: `eslint --no-inline-config --format json` over the 8
touched `.ts` files. From the JSON: 8 files, 0 errors, 0 warnings, 0
ignored.
- Population: `eslint.config.mjs`'s `files:
['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']` minus `NEVER_LINTED`, which
contains all 8.
- Invariance: the config enables no type-aware linting
(`eslint.config.mjs:326-328`, no `parserOptions.project`), and this diff
does not touch the config, so no untouched file's verdict can move.
## Acceptance notes
- **MySQL: NOT MEASURED** (no MySQL server here). The presenter is not
gated by dialect, so a `DECIMAL` string would be presented the same way.
Separately, a read-only inference, also unmeasured: `plugin.ts:500`
returns `engine.execute`'s result as the rows whenever it is an array,
and `knex.raw` on mysql2 resolves to a `[rows, fields]` pair, so on
MySQL the native path may read that pair as its rows.
- **`formatOutput`'s inline copy of the presenter** (`sql-driver.ts`,
the numeric pass at about `:20303`) is not folded onto `presentAsNumber`
here, per the ruling. It is the same rule written a second time, against
`presentReadValue`'s "never a re-derivation" docblock. For
`domain:engine`.
- **Prose that now points to the old home:**
`sql-driver-20335-aggregate-numeric-presentation.test.ts:31` and
`sql-driver-20387-aggregate-double-accumulation.test.ts:7` say
`AGGREGATE_ANSWER_KIND` lives in `sql-driver.ts`. These are driver-sql
test docblocks outside this card's two regions. For `domain:engine`, or
whoever next touches those files.
- **Expression measures** (`type: 'number' | 'string' | 'boolean'`) are
untouched: `fields[]` still declares `number` while PostgreSQL answers
`"250.000…"`, text or a JS boolean. #20943 (PR #20998) retires SQL
expressions in cube member `sql`.
- **`min` / `max` over a relationship-path column** (`account.revenue`)
stay as the client gave them. `declaredFieldType` looks a column up on
the base object only, the same "cannot answer, do not block" tier
`measureResultType` takes.
- **Not this card:** the SUM / AVG accumulation divergence on the native
statement (`0.1 + 0.2` answers `0.3` on PostgreSQL native, against
`0.30000000000000004` elsewhere). It was measured once through the
route, as the ruling asked, and is reported to the seat. No pin
enshrines either side of it.
- The census of `analytics.query(` call sites
(`envelope-caller-census.test.ts`) is unchanged: the new pins drive the
route or call the service under another name, so no ledger row or
declared input moved.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent a75311d commit d1633f3
9 files changed
Lines changed: 930 additions & 71 deletions
File tree
- .changeset
- packages
- core/src
- utils
- drivers/driver-sql/src
- rest/src
- services/service-analytics/src
- __tests__
- strategies
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
59 | 62 | | |
60 | 63 | | |
61 | 64 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
24 | 28 | | |
25 | 29 | | |
26 | 30 | | |
| |||
1524 | 1528 | | |
1525 | 1529 | | |
1526 | 1530 | | |
1527 | | - | |
1528 | | - | |
1529 | | - | |
1530 | | - | |
1531 | | - | |
1532 | | - | |
1533 | | - | |
1534 | | - | |
1535 | | - | |
1536 | | - | |
1537 | | - | |
1538 | | - | |
1539 | | - | |
1540 | | - | |
1541 | | - | |
1542 | | - | |
1543 | | - | |
1544 | | - | |
1545 | | - | |
1546 | | - | |
1547 | | - | |
1548 | | - | |
1549 | | - | |
1550 | | - | |
1551 | | - | |
1552 | | - | |
1553 | | - | |
1554 | | - | |
1555 | | - | |
1556 | | - | |
1557 | | - | |
1558 | | - | |
1559 | | - | |
1560 | | - | |
1561 | | - | |
1562 | | - | |
1563 | | - | |
1564 | | - | |
1565 | | - | |
1566 | | - | |
1567 | | - | |
1568 | | - | |
1569 | | - | |
1570 | | - | |
1571 | | - | |
1572 | | - | |
1573 | | - | |
1574 | | - | |
1575 | | - | |
1576 | | - | |
1577 | | - | |
1578 | | - | |
1579 | | - | |
1580 | | - | |
1581 | | - | |
1582 | | - | |
1583 | | - | |
| 1531 | + | |
| 1532 | + | |
| 1533 | + | |
| 1534 | + | |
| 1535 | + | |
1584 | 1536 | | |
1585 | 1537 | | |
1586 | 1538 | | |
| |||
15496 | 15448 | | |
15497 | 15449 | | |
15498 | 15450 | | |
15499 | | - | |
15500 | | - | |
15501 | | - | |
15502 | | - | |
15503 | | - | |
15504 | | - | |
15505 | | - | |
15506 | | - | |
15507 | | - | |
15508 | | - | |
| 15451 | + | |
| 15452 | + | |
| 15453 | + | |
| 15454 | + | |
| 15455 | + | |
15509 | 15456 | | |
15510 | 15457 | | |
15511 | 15458 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
242 | 242 | | |
243 | 243 | | |
244 | 244 | | |
245 | | - | |
246 | | - | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
247 | 248 | | |
248 | 249 | | |
249 | 250 | | |
250 | 251 | | |
251 | 252 | | |
252 | 253 | | |
253 | | - | |
254 | | - | |
| 254 | + | |
| 255 | + | |
255 | 256 | | |
256 | 257 | | |
257 | 258 | | |
| |||
0 commit comments