Skip to content

Commit b1befe2

Browse files
committed
fix(service-analytics): refuse a grouped dimension on a structured-JSON field at the analytics door
A `dimensions` entry, or a bucketed `timeDimensions` entry, whose column is a declared structured-JSON field (the spec's STRUCTURED_JSON_TYPES, the class the engine's groupBy door reads) is refused INVALID_FIELD / 400 in `ensureCube`, naming the member the caller wrote, before either strategy builds anything. NativeSQLStrategy compiled GROUP BY by hand and never reached the engine's door: one group per serialized document on SQLite, 500 on PostgreSQL. The ObjectQL face reached the engine's door but was refused under `groupBy[0]`, a name the caller never wrote. Claude-Session: https://claude.ai/code/session_01XY5uCwTjZj7884yYtyur4H Co-authored-by: Claude <noreply@anthropic.com>
1 parent 46ee85e commit b1befe2

2 files changed

Lines changed: 180 additions & 0 deletions

File tree

‎packages/services/service-analytics/src/analytics-service.ts‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,9 @@ import { evaluateAnalyticsQueryOverRows } from './preview-evaluator.js';
8383
// member as the request spelled it (see `dataset-refusal.ts`'s header for why
8484
// that code and not `DATASET_INVALID`).
8585
import { invalidMemberError } from './dataset-refusal.js';
86+
// [#20807] A grouped dimension on a structured-JSON field is refused in
87+
// `ensureCube`, ahead of both strategies, naming the member the caller wrote.
88+
import { assertNoStructuredJsonDimension } from './structured-json-dimension-door.js';
8689
// [#16206] The `sqlDialect` hook's DECLARED accept set, and the predicate that
8790
// says whether a host answered outside it. Both live next to the membership set
8891
// the compilers read, so the contract has one definition and this file states
@@ -2373,6 +2376,12 @@ export class AnalyticsService implements IAnalyticsService {
23732376
* where), so a query that gets several wrong is answered about one at a time,
23742377
* naming a real mistake either way.
23752378
*
2379+
* [#20807] One more door runs between the dimension gate and the `where`
2380+
* gate, on the same paths: {@link assertDimensionsGroupScalarColumns}, which
2381+
* refuses a grouped dimension whose column EXISTS but is declared
2382+
* structured JSON — a question about the column's type, not its existence,
2383+
* and so asked after it.
2384+
*
23762385
* [#20356] "Registered" means registered in `scope`: the cube is read from it
23772386
* and what this method mints is recorded in it — the call's own request
23782387
* scope, on every door, so nothing minted here reaches the shared registry
@@ -2401,6 +2410,8 @@ export class AnalyticsService implements IAnalyticsService {
24012410
// spelling is in there — which is why the suggestion list is computed by
24022411
// subtraction inside the gate rather than echoed verbatim.
24032412
this.assertDimensionFields(query, cube, Object.keys(cube.dimensions));
2413+
// [#20807] …and a grouped dimension's column must not be structured JSON.
2414+
this.assertDimensionsGroupScalarColumns(query, cube);
24042415
// [#5669] …and the `where`'s, third and last of the three request keys that
24052416
// carry a field name. Its members are read from the filter TREE, not from
24062417
// `cube.dimensions` — which on this path was minted from this very query,
@@ -2472,6 +2483,7 @@ export class AnalyticsService implements IAnalyticsService {
24722483
// authored/compiled list IS the vocabulary a caller may name — and the one
24732484
// the rejection suggests.
24742485
this.assertDimensionFields(query, augmented, Object.keys(cube.dimensions));
2486+
this.assertDimensionsGroupScalarColumns(query, augmented);
24752487
// [#5669] The `where` gate resolves a filter member through dimensions AND
24762488
// measures (that is what the strategies do for a filter member), so it is
24772489
// handed the AUGMENTED cube — a caller filtering on a suffix-inferred
@@ -2486,10 +2498,44 @@ export class AnalyticsService implements IAnalyticsService {
24862498
// authored cube can declare a measure over a field the object dropped.
24872499
this.assertMeasureFields(query, cube, Object.keys(cube.measures));
24882500
this.assertDimensionFields(query, cube, Object.keys(cube.dimensions));
2501+
this.assertDimensionsGroupScalarColumns(query, cube);
24892502
this.assertWhereFields(query, cube, Object.keys(cube.dimensions));
24902503
}
24912504
}
24922505

2506+
/**
2507+
* [#20807] Refuse a GROUPED dimension whose column is a declared
2508+
* structured-JSON field — `INVALID_FIELD` / 400, naming the member the caller
2509+
* wrote — before either strategy builds anything. The rule, the
2510+
* measurements and the envelope are {@link assertNoStructuredJsonDimension}'s
2511+
* (`structured-json-dimension-door.ts`); this method supplies the two
2512+
* answers only the service has.
2513+
*
2514+
* - The column a member groups by is {@link resolveMemberSource}'s, the
2515+
* resolver the dimension source-field gate above reads, with the same
2516+
* `'dimension'` kind the strategies' `resolveDimensionSql` /
2517+
* `resolveFieldName(…, 'dimension')` resolve by.
2518+
* - Its declared type is {@link AnalyticsServiceConfig.sourceFieldMeta}'s.
2519+
*
2520+
* Runs after the dimension source-field gate on every `ensureCube` path, so a
2521+
* member naming a column the object does not have is answered as that first.
2522+
* Same stand-downs as that gate: no `sourceFieldMeta`, or a cube whose `sql`
2523+
* is not a bare object name.
2524+
*/
2525+
private assertDimensionsGroupScalarColumns(query: AnalyticsQuery, cube: Cube): void {
2526+
const fieldMeta = this.sourceFieldMeta;
2527+
if (!fieldMeta) return;
2528+
const object = typeof cube.sql === 'string' ? cube.sql.trim() : '';
2529+
if (!object || !BARE_IDENTIFIER.test(object)) return;
2530+
assertNoStructuredJsonDimension(
2531+
query,
2532+
cube,
2533+
object,
2534+
(member) => resolveMemberSource(cube, member, 'dimension').source,
2535+
(o, field) => fieldMeta(o, field)?.type,
2536+
);
2537+
}
2538+
24932539
/**
24942540
* [#4437] Reject a measure whose SOURCE FIELD the backing object does not
24952541
* have, BEFORE the strategy compiles it into SQL.
Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#20807] A GROUPED dimension on a STRUCTURED-JSON field — `json`,
5+
* `composite`, `repeater`, `record`, `location`, `address`, `vector` — is
6+
* refused `INVALID_FIELD` / 400 at the analytics door, naming the member the
7+
* caller wrote, before either strategy builds anything.
8+
*
9+
* ## What ran before this door, measured on `origin/main` `793fb839`
10+
*
11+
* `POST /api/v1/analytics/query` over the service `AnalyticsServicePlugin`
12+
* composes on a real engine, three rows with a different `meta` document each:
13+
*
14+
* | `dimensions` | SQLite | PostgreSQL 16 |
15+
* |:--|:--|:--|
16+
* | a `text` member (the control) | 200, one group per value | same |
17+
* | a cube or dataset member over a `json` field | 200, one group per serialized document | **500 `DATABASE_ERROR`** ("could not identify an equality operator for type json") |
18+
*
19+
* `NativeSQLStrategy` compiled `GROUP BY meta` itself and ran it once through
20+
* the raw-SQL bridge; `engine.aggregate` was asked 0 times, so the engine's own
21+
* `groupBy` door (#20783, `packages/objectql`'s
22+
* `group-by-structured-json-door.ts`) never saw the query. `ObjectQLStrategy`
23+
* (a driver without raw SQL, or any bucketed time dimension) did reach that
24+
* door, and was refused there under the engine's position (`groupBy[0]`), a
25+
* name the caller never wrote.
26+
*
27+
* ## Where it stands
28+
*
29+
* In `AnalyticsService.ensureCube`, right after the dimension source-field
30+
* gate, on every path out of it — so it runs ahead of strategy selection, for
31+
* `query()` (the `/analytics/query` door, and every query a dataset selection
32+
* runs through `DatasetExecutor`) and for `generateSql()` (the `/analytics/sql`
33+
* dry run) alike. This is the first compile step that knows both halves of the
34+
* verdict: the member the caller wrote (a `dimensions` entry, or a dataset
35+
* dimension's name, which is the cube key it compiled to) and the declared type
36+
* of the column it resolves to (`sourceFieldMeta`). One door ahead of both
37+
* strategies gives one answer on every driver.
38+
*
39+
* ## What it judges
40+
*
41+
* - The members that GROUP: every `dimensions` entry, and every
42+
* `timeDimensions` entry that carries a `granularity` (a bucket is a group
43+
* key; no granularity makes a JSON document a date). A `timeDimensions`
44+
* entry with no granularity only bounds a range and groups nothing.
45+
* - The class is `@objectstack/spec/data`'s {@link STRUCTURED_JSON_TYPES},
46+
* the predicate the engine's door reads — never a list minted here.
47+
*
48+
* **Not judged** (the same "cannot answer, do not block" tiering as every
49+
* sibling gate in `ensureCube`): a host that wires no `sourceFieldMeta`, a cube
50+
* whose `sql` is not a bare object name, a member the resolver cannot pin to a
51+
* bare base column (an expression `sql`, a dotted relation traversal: its
52+
* column lives on a joined object that `sourceFieldMeta` does not answer for),
53+
* and every other field type.
54+
*
55+
* ## The envelope
56+
*
57+
* `INVALID_FIELD` / 400 through {@link invalidMemberError} (ADR-0112): the
58+
* verdict is about ONE MEMBER the request named, the family the three
59+
* source-field gates and the engine's door already answer with
60+
* `INVALID_FIELD`. `member` is the entry as the request spelled it; `field` is
61+
* the column it groups by — it exists, and its declared type is the verdict —
62+
* and `object` is the object that declares it.
63+
*
64+
* @see https://github.com/objectstack-ai/objectstack/issues/20807
65+
*/
66+
67+
import type { Cube } from '@objectstack/spec/data';
68+
import { STRUCTURED_JSON_TYPES } from '@objectstack/spec/data';
69+
import type { AnalyticsQuery } from '@objectstack/spec/contracts';
70+
import { invalidMemberError } from './dataset-refusal.js';
71+
72+
/** One grouped member, tagged with the request key that carried it. */
73+
interface GroupedMember {
74+
readonly member: string;
75+
readonly param: 'dimensions' | 'timeDimensions';
76+
}
77+
78+
/**
79+
* The members of `query` that group the result, in request-key order:
80+
* `dimensions` first, then each bucketed `timeDimensions` entry.
81+
*/
82+
function groupedMembers(query: AnalyticsQuery): GroupedMember[] {
83+
return [
84+
...(query.dimensions ?? []).map((member) => ({ member, param: 'dimensions' as const })),
85+
...(query.timeDimensions ?? [])
86+
.filter((td) => !!td.granularity)
87+
.map((td) => ({ member: td.dimension, param: 'timeDimensions' as const })),
88+
];
89+
}
90+
91+
/**
92+
* Refuse the first grouped member of `query` whose column is a declared
93+
* structured-JSON field — `INVALID_FIELD` / 400. See the module header.
94+
*
95+
* @param object - The object `cube.sql` names (the caller has checked it is a
96+
* bare object name).
97+
* @param sourceOf - The bare base column a dimension member groups by, or
98+
* `null` when this door cannot pin one (the dimension gate's resolver).
99+
* @param declaredFieldType - The declared `FieldType` of a column on
100+
* `object`, or `undefined` when nothing authoritative answers.
101+
*
102+
* The words put the verdict first, then that the query did not run, then the
103+
* route, then the reason: a door that bounds a 4xx message keeps the front of
104+
* it.
105+
*/
106+
export function assertNoStructuredJsonDimension(
107+
query: AnalyticsQuery,
108+
cube: Cube,
109+
object: string,
110+
sourceOf: (member: string) => string | null,
111+
declaredFieldType: (object: string, field: string) => string | undefined,
112+
): void {
113+
for (const { member, param } of groupedMembers(query)) {
114+
const field = sourceOf(member);
115+
if (!field) continue;
116+
const type = declaredFieldType(object, field);
117+
if (typeof type !== 'string' || !STRUCTURED_JSON_TYPES.has(type)) continue;
118+
119+
const kind = param === 'timeDimensions' ? 'Time dimension' : 'Dimension';
120+
const verb = param === 'timeDimensions' ? 'buckets' : 'groups by';
121+
const err = invalidMemberError(
122+
`${kind} '${member}' on cube '${cube.name}' ${verb} field '${field}', which object '${object}' `
123+
+ `declares as ${type} — a structured-JSON value, which analytics does not group by. `
124+
+ 'The query was NOT run. Group by a field that stores one scalar value: store the part you '
125+
+ 'group on in a field of its own and group by that field. A JSON document is no group key '
126+
+ 'the SQL dialects share: one grouped each serialized document apart, another refused the '
127+
+ 'statement.',
128+
{ member, param, cube: cube.name },
129+
) as Error & { field?: string; object?: string };
130+
err.field = field;
131+
err.object = object;
132+
throw err;
133+
}
134+
}

0 commit comments

Comments
 (0)