Skip to content

Commit d1633f3

Browse files
fix(core,driver-sql,service-analytics): the analytics native-SQL path answers measures declared number as numbers (#20889) (#21040)
Fixes #20889 Clause-②: yes (widening) ## What changed On PostgreSQL, the analytics native-SQL path answered every measure its response declares `number` as a string: `count: "2"`, `sum: "500.000000000000000000000000000000"`. SQLite answered numbers. `driver-sql`'s own `aggregate()` has presented these answers since #20335, but through a private table and presenter that `service-analytics` could not reach. The seat's route ruling (`5922780640`) moves that one rule into `@objectstack/core`, and both faces now call it. - **`@objectstack/core`** (new `src/utils/aggregate-answer.ts`, one export line in `index.ts` beside `compensatedSum`): - `AGGREGATE_ANSWER_KIND`, with its precision-policy docblock, moved from `sql-driver.ts`. The text is byte-identical apart from the `export` keyword; the proof is below. - `presentAsNumber(value)`, the body of `presentReadValue`'s `'number'` arm as a named function. A string `Number()` reads as a number becomes that number. Any other value is returned as given. - **`@objectstack/driver-sql`**, `sql-driver.ts`, the two ruled regions only: - the `AGGREGATE_ANSWER_KIND` definition becomes an import, and a pointer comment marks where it was; - `presentReadValue`'s `'number'` arm returns `presentAsNumber(value)`. The import is a separate statement beside the `AggregationFunction` import (inserted after base line 23; line 27 at HEAD). It does NOT join the `@objectstack/core` import block at base lines 94-100, which PR #20988 rewrites. That hunk starts at base line 91, 67 lines below the insertion. No behaviour change: the full driver-sql suite is green on SQLite and PostgreSQL (below). - **`@objectstack/service-analytics`**, `NativeSQLStrategy.execute` only. After the raw statement runs, each measure column is presented by the measure's DECLARED aggregate function (`lookupMember(cube, m, 'measure').type`), never by whether a value looks numeric: - `count`, `count_distinct`, `sum` and `avg` (the table's `'number'` kind) always take `presentAsNumber`; - `min` / `max` (the `'column'` kind) take it only when `declaredFieldType(object, column)` is in `NUMERIC_VALUE_TYPES`. That is the rule `driver-sql`'s `readPresentationKind` applies to a declared numeric column. A host that cannot answer, or a relationship-path column, leaves the value as given; - expression metric types (`number` / `string` / `boolean`) are left as they are (ruling Q3). One point covers both doors: the cube read (`POST /api/v1/analytics/query`) and the dataset door (`POST /api/v1/analytics/dataset/query`, through `DatasetExecutor`). - **`.changeset/20889-analytics-native-measure-number.md`**: `@objectstack/core` minor, `@objectstack/driver-sql` and `@objectstack/service-analytics` patch, `Clause-②: yes (widening)` for the two new root exports. ## Per-measure answers, before and after Fixture: `text` category; `rating` stars (an integer column); `number` amount and frac (the exact-decimal column); `currency` price. Group `a` holds 2 rows. - **Before, PostgreSQL:** phase 0's reading at base `7fa67dada3`, through `AnalyticsService.query` (what the route relays verbatim). The red pins at `719b98795e` read the same strings. - **After:** read at `e827c237d1` through the real `dispatcher-plugin` mount of `POST /api/v1/analytics/query`, by a scratch probe that was deleted after the run. | function | column (declared type) | SQLite before | SQLite after | PostgreSQL 16.13 before | PostgreSQL 16.13 after | |:--|:--|:--|:--|:--|:--| | `count` | `*` | `2` | `2` | `"2"` | `2` | | `count_distinct` | note (`text`) | `2` | `2` | `"2"` | `2` | | `sum` | stars (`rating`, integer) | `7` | `7` | `"7"` | `7` | | `avg` | stars (`rating`, integer) | `3.5` | `3.5` | `"3.5000000000000000"` | `3.5` | | `sum` | amount (`number`, decimal) | `500` | `500` | `"500.000000000000000000000000000000"` | `500` | | `avg` | amount (`number`, decimal) | `250` | `250` | `"250.000000000000000000000000000000"` | `250` | | `min` | amount (`number`) | `100` | `100` | `"100.000000000000000000000000000000"` | `100` | | `max` | price (`currency`) | `20.5` | `20.5` | `"20.500000000000000000000000000000"` | `20.5` | | `max` | note (`text`), the control | not read at base | `"y"` | not read at base | `"y"` (left as the client gave it) | | `sum` | amount over `9007199254740993` | `9007199254740992` | `9007199254740992` | `"9007199254740993.000000000000000000000000000000"` | `9007199254740992` | | dataset `row_count` (`count`) | `*` | `2` | `2` | `"2"` | `2` | | dataset `filtered_count`, a group the statement answered | `*` | `1` | `1` | `"1"` | `1` | | expression measure, type `number` | `SUM(amount) / 2` | `250` | `250` | `"250.000000000000000000000000000000"` | unchanged, left as is (ruling Q3) | `fields[]` declared `number` for every measure on both dialects, before and after. On the ObjectQL face of the same route, every answer was already a number on both dialects, and it is unchanged. ## Move proof `move-proof.py` (scratch) reads the block from BASE `2f2fa11d75`'s `sql-driver.ts` and from HEAD's `aggregate-answer.ts`: ```text table+docblock: base lines 57, core lines 57; differing lines 1: ["export const AGGREGATE_ANSWER_KIND: Readonly(...) = {"] (type arguments elided in this copy only) sha1 base e899be861c6e5e13cab24558923e41b0ca7b740c sha1 core (export keyword removed) e899be861c6e5e13cab24558923e41b0ca7b740c presenter: base arm body lines 8, core function body lines 8; identical after de-indent: True MOVE PROOF: PASS ``` The table and its docblock are byte-identical apart from `export`. The presenter's eight body lines, comment included, are identical once de-indented. In core, a new file header says why the rule lives there. The moved docblock still speaks of `SqlDriver.aggregate` and `formatOutput` in the driver's voice, and the header says so. ## Pins (red first), the ablation, and the local PostgreSQL run - **Red, at `719b98795e`** (pins only, no fix), on a private PostgreSQL 16.13 (`initdb`, 127.0.0.1, trust): - `@objectstack/core` `aggregate-answer.test.ts`: 1 file failed, `Cannot find module './aggregate-answer'`. - `service-analytics` `native-sql-measure-number-presentation.test.ts`: 5 failed, all PostgreSQL; 5 passed, all SQLite. First failure: `a row_count is a number, never "2": expected 'string' to be 'number'`. - `rest`, the new `analytics-dataset-measure-number-door.test.ts` and the tightened `analytics-dataset-json-dimension-door.test.ts`: 2 failed, both PostgreSQL; 6 passed. Failures: `a row_count is a number, never "2"` and `expected [ [ 'x', '2' ], [ 'y', '1' ] ] to deeply equal [ [ 'x', 2 ], [ 'y', 1 ] ]`. - **Green, at `e827c237d1`:** core 5/5; service-analytics 10/10 (SQLite and PostgreSQL); rest 8/8 (SQLite and PostgreSQL). - **Ablation** (the strategy stops calling the presenter). Predicted before running: exactly the PostgreSQL cells go red, service-analytics 5 and rest 2; every SQLite test and the core pin stay green. - The mutation went through `scripts/ablation-replace.mjs` (WRAP mode, with its restore trap) on the absolute path. The guard `if (numberMeasures.length > 0 && Array.isArray(rows))` gained `&& String(numberMeasures) === 'ABLATION-20889'`, so it never holds. Anchor count went 1 to 0, blob `3bce7bb073` to `8f36f62dfe`. - `service-analytics` was rebuilt, and `ablation-dist-preflight.mjs` found the marker in 2 built files. - Observed: service-analytics 5 failed (all PostgreSQL), 5 passed; rest 2 failed (both PostgreSQL), 6 passed; core 5/5. That is exactly as predicted. - Restore: the blob after restore equals HEAD (`3bce7bb073`), `git diff HEAD` is 0 bytes, and porcelain is 0 lines. Then a rebuild, `ablation-dist-preflight.mjs --absent` (marker absent from all 6 built files, tree clean), and a re-run: all green. - **`driver-sql`'s full suite at `e827c237d1`**, SQLite and live PostgreSQL, `TZ=America/New_York`, with the server at `Asia/Shanghai` as CI runs it: 210 files passed, 3 skipped; 4061 tests passed, 95 skipped. `sql-driver-20335-aggregate-numeric-presentation.test.ts` 15 tests (1 skipped, the MySQL cell), and `sql-driver-20387-aggregate-double-accumulation.test.ts` 11 (1 skipped). A first run on a UTC server failed only the four suites whose precondition asserts a non-UTC server. - **CI:** no CI job provisions `OS_TEST_POSTGRES_URL` for `service-analytics` or `rest`, so their PostgreSQL cells are named skips there (`skipped: set OS_TEST_POSTGRES_URL to run this cell`). The core pin (string to number) and every SQLite cell run in CI. The private cluster was stopped and its data directory removed after the runs. ## Gates All at HEAD `e827c237d1`, as ONE sequential script under the shared verify lock, each exit code captured before any pipe. - **Derived families:** `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` names 67. `--ran` reconciles: `67 derived famil(ies) accounted for — 65 run, 2 NOT-MEASURED`, 0 unrun. All 65 exit 0. - **NOT MEASURED: `check:dual-build-cjs-loads` and `check:type-check-debt`.** Reason: both exit 3, `PREREQUISITE NOT MET`. They read the built output of every package, and this run built only the dependency closure it tests. CI's `lint.yml` builds the full closure before them. - **The four roster families the derivation marks ⛔ (roster in a directory this diff touches):** `check-changeset-fixed.mjs`, `check:authz-resolver`, `check:error-code-casing`, `check:filter-alias-parity`. All exit 0. - **Tests and typechecks:** - `@objectstack/core` test: 62 files, 1803 tests passed. Typecheck exit 0, and its `check:test-typecheck` covers the new test. - `@objectstack/driver-sql`: typecheck exit 0; the full suite as above. - `@objectstack/service-analytics` full suite with `OS_TEST_POSTGRES_URL` set: 151 files, 3468 tests passed. Typecheck exit 0; `tsc --listFiles` includes the new pin. - `@objectstack/rest`: the two pin files, SQLite and PostgreSQL, 8/8. Typecheck exit 0, with `check:test-typecheck` over `tsconfig.test.json`. - **ESLint, a declared narrowing** (the repo-wide `pnpm lint` is CI's): - Touched files: `eslint --no-inline-config --format json` over the 8 touched `.ts` files. From the JSON: 8 files, 0 errors, 0 warnings, 0 ignored. - Population: `eslint.config.mjs`'s `files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']` minus `NEVER_LINTED`, which contains all 8. - Invariance: the config enables no type-aware linting (`eslint.config.mjs:326-328`, no `parserOptions.project`), and this diff does not touch the config, so no untouched file's verdict can move. ## Acceptance notes - **MySQL: NOT MEASURED** (no MySQL server here). The presenter is not gated by dialect, so a `DECIMAL` string would be presented the same way. Separately, a read-only inference, also unmeasured: `plugin.ts:500` returns `engine.execute`'s result as the rows whenever it is an array, and `knex.raw` on mysql2 resolves to a `[rows, fields]` pair, so on MySQL the native path may read that pair as its rows. - **`formatOutput`'s inline copy of the presenter** (`sql-driver.ts`, the numeric pass at about `:20303`) is not folded onto `presentAsNumber` here, per the ruling. It is the same rule written a second time, against `presentReadValue`'s "never a re-derivation" docblock. For `domain:engine`. - **Prose that now points to the old home:** `sql-driver-20335-aggregate-numeric-presentation.test.ts:31` and `sql-driver-20387-aggregate-double-accumulation.test.ts:7` say `AGGREGATE_ANSWER_KIND` lives in `sql-driver.ts`. These are driver-sql test docblocks outside this card's two regions. For `domain:engine`, or whoever next touches those files. - **Expression measures** (`type: 'number' | 'string' | 'boolean'`) are untouched: `fields[]` still declares `number` while PostgreSQL answers `"250.000…"`, text or a JS boolean. #20943 (PR #20998) retires SQL expressions in cube member `sql`. - **`min` / `max` over a relationship-path column** (`account.revenue`) stay as the client gave them. `declaredFieldType` looks a column up on the base object only, the same "cannot answer, do not block" tier `measureResultType` takes. - **Not this card:** the SUM / AVG accumulation divergence on the native statement (`0.1 + 0.2` answers `0.3` on PostgreSQL native, against `0.30000000000000004` elsewhere). It was measured once through the route, as the ruling asked, and is reported to the seat. No pin enshrines either side of it. - The census of `analytics.query(` call sites (`envelope-caller-census.test.ts`) is unchanged: the new pins drive the route or call the service under another name, so no ledger row or declared input moved. --- _Generated by [Claude Code](https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent a75311d commit d1633f3

9 files changed

Lines changed: 930 additions & 71 deletions

File tree

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
---
2+
'@objectstack/core': minor
3+
'@objectstack/driver-sql': patch
4+
'@objectstack/service-analytics': patch
5+
---
6+
7+
fix: the analytics native-SQL path answers a measure its response declares `number` as a number on every dialect, presented by the one rule `driver-sql`'s `aggregate()` applies, which `@objectstack/core` now exports as `AGGREGATE_ANSWER_KIND` and `presentAsNumber` (#20889)
8+
9+
Clause-②: yes (widening)
10+
11+
**New exports.** `@objectstack/core` exports two names, moved here unchanged
12+
from `@objectstack/driver-sql`, which now imports them instead of keeping them
13+
private:
14+
15+
- `AGGREGATE_ANSWER_KIND`: what each declared aggregate function answers.
16+
`count`, `count_distinct`, `sum` and `avg` answer `'number'`; `min` and `max`
17+
answer `'column'`, a value of the aggregated column.
18+
- `presentAsNumber(value)`: the `'number'` presentation. A string `Number()`
19+
reads as a number becomes that number. Any other value is returned as given:
20+
a number, `null`, a boolean, empty or blank text, or text that reads as NaN.
21+
22+
**What changed.** On PostgreSQL, `POST /api/v1/analytics/query` and
23+
`POST /api/v1/analytics/dataset/query` answered through `NativeSQLStrategy`
24+
returned count, count_distinct, sum, avg, and min / max over a numeric column
25+
as strings, such as `count: "2"` and
26+
`sum: "500.000000000000000000000000000000"`, while `fields[]` declared
27+
`number`. A dataset's `row_count` did the same, and a measure-scoped count
28+
mixed `"1"` with the number `0` in one column. SQLite answered numbers. The
29+
strategy now presents each measure column by its declared aggregate function,
30+
through the same table and presenter as `SqlDriver.aggregate()`. `min` / `max`
31+
are presented only when their column is declared numeric, so `max` over a text
32+
column, every dimension, and expression measures keep the value the database
33+
returned.
34+
35+
**Precision.** The answer is one JS number, the policy `driver-sql`'s
36+
`aggregate()` already applies. A total that needs more digits than a double
37+
holds, such as `9007199254740993`, answers the nearest double
38+
(`9007199254740992`), which is also what SQLite and the engine path answer.
39+
40+
**What did not move.** `@objectstack/driver-sql`'s behaviour is unchanged: its
41+
`aggregate()` reads the same table, and its read presenter calls the same
42+
function. The answers on SQLite are byte-identical. The arithmetic of the
43+
analytics native statement did not change either. On PostgreSQL its `sum` and
44+
`avg` still add exact decimals, so `0.1 + 0.2` answers `0.3` where the engine
45+
path answers `0.30000000000000004`.

‎packages/core/src/index.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,9 @@ export * from './utils/datetime.js';
5656
// runtime dependencies, and a copy per face is how one `sum` came to answer
5757
// two doubles.
5858
export * from './utils/compensated-sum.js';
59+
// [#20889] What an aggregate ANSWERS and its `'number'` presenter, moved from
60+
// `driver-sql` so the analytics native-SQL face presents with the same rule.
61+
export * from './utils/aggregate-answer.js';
5962

6063
// Export the shared batched-write helper (framework#2678)
6164
export * from './utils/bulk-write.js';
Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#20889] `AGGREGATE_ANSWER_KIND` and `presentAsNumber` — what an aggregate
5+
* ANSWERS, and the one `'number'` presenter every face that hands a SQL
6+
* client's aggregate row to a caller applies: `driver-sql`'s own `aggregate()`
7+
* (#20335) and `service-analytics`' native-SQL face (`NativeSQLStrategy`).
8+
*
9+
* The wire strings below are the values measured on live PostgreSQL 16.13
10+
* through the analytics native-SQL path (node-postgres parses `bigint` and
11+
* `numeric` to strings), and the large totals are the ones #20335 pinned at the
12+
* driver door. Each face pins the same presentation through its own door, in
13+
* its own package, on SQLite and PostgreSQL.
14+
*/
15+
16+
import { describe, it, expect } from 'vitest';
17+
import { AggregationFunction } from '@objectstack/spec/data';
18+
import { AGGREGATE_ANSWER_KIND, presentAsNumber } from './aggregate-answer';
19+
20+
describe('[#20889] AGGREGATE_ANSWER_KIND — what each declared aggregate function answers', () => {
21+
it('has exactly one row per declared aggregate function', () => {
22+
expect(Object.keys(AGGREGATE_ANSWER_KIND).sort()).toEqual([...AggregationFunction.options].sort());
23+
});
24+
25+
it('count, count_distinct, sum and avg answer a number; min and max answer a value of the column', () => {
26+
expect(AGGREGATE_ANSWER_KIND).toEqual({
27+
count: 'number',
28+
count_distinct: 'number',
29+
sum: 'number',
30+
avg: 'number',
31+
min: 'column',
32+
max: 'column',
33+
});
34+
});
35+
});
36+
37+
describe("[#20889] presentAsNumber — the 'number' presenter", () => {
38+
it('turns the numeric text a SQL client hands back into the number it spells', () => {
39+
// count / count_distinct (`bigint`), sum over an integer column (`bigint`).
40+
expect(presentAsNumber('2')).toBe(2);
41+
expect(presentAsNumber('11')).toBe(11);
42+
// avg over an integer column (`numeric`, 16 places).
43+
expect(presentAsNumber('3.5000000000000000')).toBe(3.5);
44+
// sum / avg / min / max over the exact-decimal column (`numeric(65,30)`).
45+
expect(presentAsNumber('500.000000000000000000000000000000')).toBe(500);
46+
expect(presentAsNumber('20.500000000000000000000000000000')).toBe(20.5);
47+
expect(presentAsNumber('0.125000000000000000000000000000')).toBe(0.125);
48+
expect(presentAsNumber('-7.250000000000000000000000000000')).toBe(-7.25);
49+
});
50+
51+
it('precision policy: a total a double cannot hold answers the nearest double, never a string', () => {
52+
const big = presentAsNumber('9007199254740993.000000000000000000000000000000');
53+
expect(typeof big).toBe('number');
54+
expect(big).toBe(9007199254740992);
55+
expect(big).toBe(Number('9007199254740993'));
56+
const decimal = presentAsNumber('12345678901234567.123456789000000000000000000000');
57+
expect(typeof decimal).toBe('number');
58+
expect(decimal).toBe(12345678901234568);
59+
expect(decimal).toBe(Number('12345678901234567.123456789'));
60+
});
61+
62+
it('passes every value that is not numeric text through as given', () => {
63+
expect(presentAsNumber(7)).toBe(7);
64+
expect(presentAsNumber(0.30000000000000004)).toBe(0.30000000000000004);
65+
expect(presentAsNumber(null)).toBeNull();
66+
expect(presentAsNumber(undefined)).toBeUndefined();
67+
expect(presentAsNumber(true)).toBe(true);
68+
expect(presentAsNumber('')).toBe('');
69+
expect(presentAsNumber(' ')).toBe(' ');
70+
// PostgreSQL's `numeric` 'NaN', and text `Number()` cannot read, stay as written.
71+
expect(presentAsNumber('NaN')).toBe('NaN');
72+
expect(presentAsNumber('abc')).toBe('abc');
73+
const date = new Date(0);
74+
expect(presentAsNumber(date)).toBe(date);
75+
});
76+
});
Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#20889] What an aggregate ANSWERS, and the `'number'` presenter that gives a
5+
* count or a total its type — defined once, for every face that hands a SQL
6+
* client's aggregate row to a caller.
7+
*
8+
* ## Why it lives here
9+
*
10+
* Two faces run an aggregate statement on a SQL client and must present its
11+
* answer the same way:
12+
*
13+
* - `@objectstack/driver-sql`'s native `aggregate()` (#20335), where this rule
14+
* was written: `aggregate()` reads {@link AGGREGATE_ANSWER_KIND} to choose
15+
* the result columns that take the `'number'` presentation, and
16+
* `presentReadValue`'s `'number'` arm is {@link presentAsNumber};
17+
* - `@objectstack/service-analytics`' native-SQL face
18+
* (`NativeSQLStrategy.execute`), which runs its own statement through the
19+
* host's raw-SQL bridge — `engine.execute`, or an embedder's own
20+
* `executeRawSql` — and so never passes through the driver's `aggregate()`.
21+
* On PostgreSQL it answered `count: "2"` under a `fields[]` that declared
22+
* `number`.
23+
*
24+
* `service-analytics` holds `driver-sql` as a development dependency only — it
25+
* is multi-driver, and its native path also runs over an embedder's raw SQL —
26+
* and this is the package both already stand on, the reason `compensatedSum`
27+
* lives here too. A second transcription is how a face comes to answer its own
28+
* type again.
29+
*
30+
* The table and its docblock below moved here from `driver-sql` unchanged, so
31+
* they speak of `SqlDriver.aggregate` and `formatOutput` in that driver's
32+
* voice.
33+
*/
34+
35+
import type { AggregationFunction } from '@objectstack/spec/data';
36+
37+
/**
38+
* [#20335] What each declared aggregate function ANSWERS — a derived `number`,
39+
* or a value OF the aggregated column — and therefore which read presentation
40+
* {@link SqlDriver.aggregate} gives its result column.
41+
*
42+
* - `'number'` — `count`, `count_distinct`, `sum`, `avg`. A count or a total is
43+
* a number whatever the column held, and it is presented as one (`'number'`,
44+
* the presenter `formatOutput` applies to a numeric field on a `find()` row).
45+
* - `'column'` — `min`, `max`. The answer is one of the column's own values, so
46+
* it takes that column's presentation ({@link SqlDriver.readPresentationKind}),
47+
* exactly as before this table existed.
48+
*
49+
* Why the `'number'` half needs presenting at all: the SQL client hands a
50+
* result back as the wire type of the SQL expression, not as the platform's
51+
* value type. Measured on live PostgreSQL 16.13 and MySQL 8.0.46 through this
52+
* driver's own connections: node-postgres parses `bigint` (OID 20 — `count`,
53+
* and `sum` over an integer column) and `numeric` (OID 1700 — `sum` / `avg`
54+
* over the exact-decimal numeric family, `avg` over an integer column) to
55+
* STRINGS (`"2"`, `"500.000000000000000000000000000000"`), and mysql2 does the
56+
* same for `DECIMAL` (`SUM` / `AVG`; its `COUNT` arrives as a number). The
57+
* engine's rows path (`objectql`'s `in-memory-aggregation.ts`) and SQLite answer
58+
* numbers for the same query, so `having { n: { $in: [2] } }` kept c1, c2 on
59+
* those and no group on PostgreSQL's native path.
60+
*
61+
* Keyed on the function the query ASKED for, never on whether a value looks
62+
* numeric, and deliberately not gated by dialect: the presenter only rewrites a
63+
* STRING, so a client that already answers a number (better-sqlite3, mysql2's
64+
* `COUNT`) passes through untouched — measured byte-identical on SQLite — and
65+
* no list of "string-answering dialects" exists to drift.
66+
*
67+
* ## The precision policy — one JS number, the loss declared
68+
*
69+
* The answer is `Number(text)`: an IEEE-754 double, on every dialect. A `sum` /
70+
* `avg` over the exact-decimal column (`numeric(65,30)` / `DECIMAL(65,30)`)
71+
* whose value needs more than a double's ~15-17 significant digits, or an
72+
* integer at or above 2^53, is ROUNDED to the nearest double — declared, not
73+
* silent: it is the same bound `formatOutput` already puts on a `find()` read of
74+
* that column (#16318, `valueSchemaFor`'s `z.number().finite()`, ADR-0104 D1),
75+
* and the bound the rows path has always had (`toNumber` sums JS doubles). A
76+
* value-dependent type — a number when it fits, a string when it does not — was
77+
* rejected: it would reopen this defect for exactly the large totals, where a
78+
* `having` `$in` or a chart silently stops matching. Only a string `Number()`
79+
* reads as NaN (PostgreSQL's `numeric` `'NaN'`) is left as written, the
80+
* presenter's existing rule.
81+
*
82+
* A `Record` over `AggregationFunction` on purpose: a function that joins the
83+
* declared vocabulary without an answer here fails `tsc` rather than reaching a
84+
* caller unpresented.
85+
*/
86+
export const AGGREGATE_ANSWER_KIND: Readonly<Record<AggregationFunction, 'number' | 'column'>> = {
87+
count: 'number',
88+
count_distinct: 'number',
89+
sum: 'number',
90+
avg: 'number',
91+
min: 'column',
92+
max: 'column',
93+
};
94+
95+
/**
96+
* [#20335, #20889] The `'number'` presentation: a STRING `Number()` reads as a
97+
* number becomes that number; every other value — a number, `null`, a boolean,
98+
* empty or blank text, text `Number()` reads as NaN (PostgreSQL's `numeric`
99+
* `'NaN'`) — is returned as given.
100+
*
101+
* It is the body of `driver-sql`'s `presentReadValue` `'number'` arm, moved
102+
* here unchanged: the presenter `formatOutput` applies to a declared numeric
103+
* column on a `find()` row (#16318), and the one {@link AGGREGATE_ANSWER_KIND}
104+
* names for a count or a total. The answer is one JS double — the precision
105+
* policy that table's docblock states.
106+
*
107+
* ⛔ Ask it by a column's DECLARED meaning — the aggregate function the query
108+
* asked for, or a column declared numeric — never because a value looks
109+
* numeric: a text column holding `'007'` is text.
110+
*/
111+
export function presentAsNumber(value: unknown): unknown {
112+
// Only strings are repaired, exactly as in `formatOutput`: a fresh
113+
// REAL/INTEGER column already yields a number, and genuinely
114+
// non-numeric legacy junk is left intact rather than turned into NaN.
115+
if (typeof value === 'string' && value.trim() !== '') {
116+
const n = Number(value);
117+
if (!Number.isNaN(n)) return n;
118+
}
119+
return value;
120+
}

‎packages/drivers/driver-sql/src/sql-driver.ts‎

Lines changed: 14 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,10 @@ import { parseAutonumberFormat, renderAutonumber, resolveAutonumberFormat, readA
2121
// "the protocol has no such function" refusal cannot drift from what
2222
// `AggregationNodeSchema.function` actually admits.
2323
import { AggregationFunction, emptyGroupValueFor } from '@objectstack/spec/data';
24+
// [#20335, #20889] What each aggregate function ANSWERS, and the `'number'`
25+
// presenter its counts and totals take — defined once in core, for this
26+
// driver's `aggregate()` and the analytics native-SQL face alike.
27+
import { AGGREGATE_ANSWER_KIND, presentAsNumber } from '@objectstack/core';
2428
import { STRUCTURED_JSON_TYPES, FILE_REFERENCE_TYPES, MULTI_OPTION_TYPES, NUMERIC_VALUE_TYPES, isMultiValueField } from '@objectstack/spec/data';
2529
// [#16318] The per-field-type physical representation of the NUMERIC family.
2630
// `os generate migration` reads the SAME table, in both of its formats — that
@@ -1524,63 +1528,11 @@ const SQL_AGGREGATE_FUNCTIONS: ReadonlyMap<string, SqlAggregateLowering> = new M
15241528
['count_distinct', { sql: 'count', distinct: true }],
15251529
]);
15261530

1527-
/**
1528-
* [#20335] What each declared aggregate function ANSWERS — a derived `number`,
1529-
* or a value OF the aggregated column — and therefore which read presentation
1530-
* {@link SqlDriver.aggregate} gives its result column.
1531-
*
1532-
* - `'number'` — `count`, `count_distinct`, `sum`, `avg`. A count or a total is
1533-
* a number whatever the column held, and it is presented as one (`'number'`,
1534-
* the presenter `formatOutput` applies to a numeric field on a `find()` row).
1535-
* - `'column'` — `min`, `max`. The answer is one of the column's own values, so
1536-
* it takes that column's presentation ({@link SqlDriver.readPresentationKind}),
1537-
* exactly as before this table existed.
1538-
*
1539-
* Why the `'number'` half needs presenting at all: the SQL client hands a
1540-
* result back as the wire type of the SQL expression, not as the platform's
1541-
* value type. Measured on live PostgreSQL 16.13 and MySQL 8.0.46 through this
1542-
* driver's own connections: node-postgres parses `bigint` (OID 20 — `count`,
1543-
* and `sum` over an integer column) and `numeric` (OID 1700 — `sum` / `avg`
1544-
* over the exact-decimal numeric family, `avg` over an integer column) to
1545-
* STRINGS (`"2"`, `"500.000000000000000000000000000000"`), and mysql2 does the
1546-
* same for `DECIMAL` (`SUM` / `AVG`; its `COUNT` arrives as a number). The
1547-
* engine's rows path (`objectql`'s `in-memory-aggregation.ts`) and SQLite answer
1548-
* numbers for the same query, so `having { n: { $in: [2] } }` kept c1, c2 on
1549-
* those and no group on PostgreSQL's native path.
1550-
*
1551-
* Keyed on the function the query ASKED for, never on whether a value looks
1552-
* numeric, and deliberately not gated by dialect: the presenter only rewrites a
1553-
* STRING, so a client that already answers a number (better-sqlite3, mysql2's
1554-
* `COUNT`) passes through untouched — measured byte-identical on SQLite — and
1555-
* no list of "string-answering dialects" exists to drift.
1556-
*
1557-
* ## The precision policy — one JS number, the loss declared
1558-
*
1559-
* The answer is `Number(text)`: an IEEE-754 double, on every dialect. A `sum` /
1560-
* `avg` over the exact-decimal column (`numeric(65,30)` / `DECIMAL(65,30)`)
1561-
* whose value needs more than a double's ~15-17 significant digits, or an
1562-
* integer at or above 2^53, is ROUNDED to the nearest double — declared, not
1563-
* silent: it is the same bound `formatOutput` already puts on a `find()` read of
1564-
* that column (#16318, `valueSchemaFor`'s `z.number().finite()`, ADR-0104 D1),
1565-
* and the bound the rows path has always had (`toNumber` sums JS doubles). A
1566-
* value-dependent type — a number when it fits, a string when it does not — was
1567-
* rejected: it would reopen this defect for exactly the large totals, where a
1568-
* `having` `$in` or a chart silently stops matching. Only a string `Number()`
1569-
* reads as NaN (PostgreSQL's `numeric` `'NaN'`) is left as written, the
1570-
* presenter's existing rule.
1571-
*
1572-
* A `Record` over `AggregationFunction` on purpose: a function that joins the
1573-
* declared vocabulary without an answer here fails `tsc` rather than reaching a
1574-
* caller unpresented.
1575-
*/
1576-
const AGGREGATE_ANSWER_KIND: Readonly<Record<AggregationFunction, 'number' | 'column'>> = {
1577-
count: 'number',
1578-
count_distinct: 'number',
1579-
sum: 'number',
1580-
avg: 'number',
1581-
min: 'column',
1582-
max: 'column',
1583-
};
1531+
// [#20335, #20889] `AGGREGATE_ANSWER_KIND` — what each declared aggregate
1532+
// function ANSWERS, and the one-double precision policy for that answer — lives
1533+
// in `@objectstack/core` (`utils/aggregate-answer.ts`), imported above with its
1534+
// `'number'` presenter, so the analytics native-SQL face presents a count or a
1535+
// total with this driver's own rule.
15841536

15851537
/**
15861538
* [#20387] What each declared aggregate function ACCUMULATES IN on PostgreSQL
@@ -15496,16 +15448,11 @@ export class SqlDriver implements IDataDriver {
1549615448
return presentAuditTimestampOutput(value);
1549715449
case 'boolean':
1549815450
return Boolean(value);
15499-
case 'number': {
15500-
// Only strings are repaired, exactly as in `formatOutput`: a fresh
15501-
// REAL/INTEGER column already yields a number, and genuinely
15502-
// non-numeric legacy junk is left intact rather than turned into NaN.
15503-
if (typeof value === 'string' && value.trim() !== '') {
15504-
const n = Number(value);
15505-
if (!Number.isNaN(n)) return n;
15506-
}
15507-
return value;
15508-
}
15451+
case 'number':
15452+
// [#20889] The presenter lives in `@objectstack/core`, beside
15453+
// `AGGREGATE_ANSWER_KIND`, so the analytics native-SQL face presents
15454+
// a count or a total with this same function.
15455+
return presentAsNumber(value);
1550915456
}
1551015457
}
1551115458

‎packages/rest/src/analytics-dataset-json-dimension-door.test.ts‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -242,16 +242,17 @@ for (const cell of CELLS) {
242242
const before = { ...reads };
243243
const res = await query({ measures: ['row_count'], dimensions: ['title_dim'] });
244244
expect(res.status, JSON.stringify(res.body)).toBe(200);
245-
const groups = (res.body.rows as Array<{ title_dim: string; row_count: number | string }>)
246-
.map((r) => [r.title_dim, Number(r.row_count)] as const)
245+
// [#20889] `row_count` is a JSON number on every dialect — never `"2"`.
246+
const groups = (res.body.rows as Array<{ title_dim: string; row_count: unknown }>)
247+
.map((r) => [r.title_dim, r.row_count] as const)
247248
.sort(([a], [b]) => a.localeCompare(b));
248249
expect(groups).toEqual([['x', 2], ['y', 1]]);
249250
expect(reads.rawSql - before.rawSql, 'the native strategy answered').toBeGreaterThanOrEqual(1);
250251

251252
const joined = await query({ measures: ['row_count'], dimensions: ['acct_name'] });
252253
expect(joined.status, JSON.stringify(joined.body)).toBe(200);
253-
const joinedGroups = (joined.body.rows as Array<{ acct_name: string; row_count: number | string }>)
254-
.map((r) => [r.acct_name, Number(r.row_count)] as const)
254+
const joinedGroups = (joined.body.rows as Array<{ acct_name: string; row_count: unknown }>)
255+
.map((r) => [r.acct_name, r.row_count] as const)
255256
.sort(([a], [b]) => a.localeCompare(b));
256257
expect(joinedGroups).toEqual([['A', 2], ['B', 1]]);
257258
});

0 commit comments

Comments
 (0)