fix(service-analytics)!: the dataset draft preview refuses a runtimeFilter on a field the object does not have, and a dimension over a JSON column, as the live strategies do

.changeset/28257-preview-filter-field-json-dimension.md

@@ -0,0 +1,20 @@
+---
+'@objectstack/service-analytics': patch
+---
+
+fix(service-analytics)!: the dataset draft preview refuses a runtimeFilter on a field the object does not have, and a dimension over a JSON column, as the live strategies do, instead of serving an empty or object-keyed answer
+
+Clause-②: no (narrowing) — a draft-preview dataset query (`queryDataset` with `previewDrafts: true` over a pending seed draft) now refuses a `runtimeFilter` member whose field the object does not have, and a grouped dimension over a `json` column, both of which the native and ObjectQL strategies refuse and the preview served before. Nothing is widened: no export, answer member, error code or status is added.
+Metadata-protocol: none
+
+<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped, and no export or type changes: the draft preview now answers an unknown filter field and a JSON dimension the way the published dataset already answers them. Both refusals are the live faces' existing ones, so there is nothing for a migration to rewrite and no tombstone. -->
+
+**BREAKING** accept-set narrowing on the draft preview of `queryDataset` (`@objectstack/service-analytics`).
+
+- **Before.** The preview evaluated a `runtimeFilter` over the seed rows without asking whether the object has each filtered field. A misspelled `{ bogus: 'x' }` matched no row and served `[]`; under an `$or`, `{ $or: [{ bogus: 1 }, { region: 'amer' }] }` served the other arm's rows. Both live strategies refuse it `INVALID_FIELD` / 400 with `param: 'where'` ("Filter member 'bogus' in 'where' on cube … constrains field 'bogus', which object … does not have …"). The preview also grouped by a dimension over a `json` column, one group per object value, where live refuses it `INVALID_FIELD` / 400 with `param: 'dimensions'` (#20807).
+- **Now.** The preview asks both live gates before it reads a seed row, so each mistake is refused with the same envelope and message on all three faces: the filter in every spelling (object, `$or` arm, array triple), the JSON dimension whether the dataset declares it or not.
+- **Gate order.** Both gates sit where `ensureCube` asks them: the structured-JSON door after the dimension-field gate and before the aggregate × field-type door, the `where` gate last among `ensureCube`'s gates (after the record-currency door) and before `query()`'s caller-measure column-reference gate. So a query with two mistakes is answered about the same one first on every face: a JSON dimension beside an unknown filter field answers the dimension; `amount + 1` beside an unknown filter field answers the filter (400), where the preview answered the column-reference gate (403).
+- **Unchanged.** A filter on a declared dimension, on an undeclared field the object has (`amount: { $gt: 40 }`) and an `$in` filter serve the same rows as before on every face. The live strategies are unchanged.
+- **Who sees it.** A Studio author whose draft widget filters on a field name the object does not have, or groups by a JSON field: the mistake now shows the refusal it meets once published, instead of an empty chart or one bar per object value.
+
+**The remedy.** Filter on one of the object's own fields, spelled as the object declares it; group by a scalar field, not a `json` one.

packages/services/service-analytics/src/__tests__/preview-filter-field-json-dimension-parity.test.ts

@@ -0,0 +1,182 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#28257] A `runtimeFilter` on a field the dataset object does not have, and
+ * a dimension over a structured-JSON column (#20807), through `queryDataset` on
+ * the native strategy, the ObjectQL strategy and the draft preview.
+ *
+ * ## What these pins hold
+ *
+ * - A filter member whose field the object does not have is refused
+ *   `INVALID_FIELD` / 400 with `param: 'where'` on every face, with the same
+ *   message, whether it is the whole filter, one arm of an `$or` or the array
+ *   spelling. Before, the preview matched no row and served `[]`, or under the
+ *   `$or` served the other arm's rows.
+ * - A dimension over a `json` column, declared or not, is refused
+ *   `INVALID_FIELD` / 400 with `param: 'dimensions'` on every face, with the
+ *   same message. Before, the preview grouped by each row's object value.
+ * - CONTROL: a filter on a declared dimension, on an undeclared real field and
+ *   an `$in` filter serve the same rows on every face.
+ * - The preview asks both gates in the live order: the JSON door after the
+ *   dimension-field gate and before the field-type door, the `where` gate last
+ *   among `ensureCube`'s and before the column-reference gate.
+ */
+
+import { describe, it, expect, beforeAll, afterAll } from 'vitest';
+import { ObjectQL } from '@objectstack/objectql';
+import { SqlDriver } from '@objectstack/driver-sql';
+import { DatasetSchema } from '@objectstack/spec/ui';
+import type { AnalyticsService } from '../analytics-service.js';
+import { AnalyticsServicePlugin } from '../plugin.js';
+import { installDefaultOrganization } from './default-organization.testkit.js';
+
+const OBJECT = 'os28257_deal';
+
+const ROWS = [
+  { id: 'd1', region: 'amer', amount: 100, payload: { k: 1 } },
+  { id: 'd2', region: 'amer', amount: 50, payload: { k: 2 } },
+  { id: 'd3', region: 'emea', amount: 30, payload: { k: 1 } },
+] as const;
+
+const DATASET = DatasetSchema.parse({
+  name: 'os28257_deals',
+  label: 'Deals',
+  object: OBJECT,
+  dimensions: [
+    { name: 'region', field: 'region', type: 'string' },
+    { name: 'payload_dim', field: 'payload', type: 'string' },
+  ],
+  measures: [
+    { name: 'total', aggregate: 'sum', field: 'amount' },
+    { name: 'deals', aggregate: 'count' },
+  ],
+});
+
+type Face = 'native' | 'objectql' | 'preview';
+const FACES: readonly Face[] = ['native', 'objectql', 'preview'];
+
+const quiet = { debug() {}, info() {}, warn() {}, error() {}, trace() {}, fatal() {}, child() { return quiet; } };
+
+type Refusal = Error & { code?: string; status?: number; param?: string; field?: string };
+
+const refusalOf = (p: Promise<unknown>): Promise<Refusal> =>
+  p.then<never, Refusal>(
+    () => {
+      throw new Error('expected a refusal, but the query resolved');
+    },
+    (e) => e as Refusal,
+  );
+
+/** A `'protocol'` service answering one pending seed draft, which routes `previewDrafts` onto the preview path. */
+const pendingSeedProtocol = {
+  getMetaItems: async () => [{ name: 'os28257_seed', object: OBJECT }],
+  getMetaItem: async () => ({ item: { records: ROWS } }),
+};
+
+describe('[#28257] an unknown filter field and a json dimension on the native, ObjectQL and draft-preview faces', () => {
+  let engine: ObjectQL;
+  const services: Partial<Record<'native' | 'objectql', AnalyticsService>> = {};
+
+  const ask = (face: Face, selection: Record<string, unknown>) =>
+    services[face === 'preview' ? 'objectql' : face]!.queryDataset(
+      DATASET,
+      selection as never,
+      undefined,
+      face === 'preview' ? { previewDrafts: true } : undefined,
+    );
+
+  beforeAll(async () => {
+    const driver = new SqlDriver({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true } as never);
+    engine = new ObjectQL({ logger: quiet } as never);
+    engine.registerDriver(driver as never, true);
+    await engine.init();
+    engine.registry.registerObject({
+      name: OBJECT,
+      label: 'Deal',
+      fields: {
+        region: { name: 'region', type: 'text' },
+        amount: { name: 'amount', type: 'number' },
+        payload: { name: 'payload', type: 'json' },
+      },
+    } as never);
+    await engine.syncSchemas();
+    await installDefaultOrganization(engine as never);
+    for (const row of ROWS) await engine.insert(OBJECT, { ...row } as never);
+
+    for (const [face, caps] of [
+      ['native', undefined],
+      ['objectql', () => ({ nativeSql: false, objectqlAggregate: true, inMemory: false })],
+    ] as const) {
+      const registered: Record<string, unknown> = {};
+      const provided: Record<string, unknown> = { data: engine, protocol: pendingSeedProtocol };
+      await new AnalyticsServicePlugin({ ...(caps ? { queryCapabilities: caps } : {}) } as never).init({
+        getService: (name: string) => provided[name] ?? registered[name],
+        registerService: (name: string, svc: unknown) => { registered[name] = svc; },
+        replaceService: (name: string, svc: unknown) => { registered[name] = svc; },
+        hook: () => {},
+        logger: quiet,
+      } as never);
+      services[face] = registered.analytics as AnalyticsService;
+    }
+  });
+
+  afterAll(async () => {
+    try { await engine?.destroy(); } catch { /* noop */ }
+  });
+
+  const refusedAlike = async (selection: Record<string, unknown>, param: string, field: string) => {
+    const refusals = await Promise.all(FACES.map((face) => refusalOf(ask(face, selection))));
+    for (const err of refusals) {
+      expect(err.code, err.message).toBe('INVALID_FIELD');
+      expect(err.status).toBe(400);
+      expect(err.param).toBe(param);
+      expect(err.field).toBe(field);
+    }
+    const [native, objectql, preview] = refusals;
+    expect(preview.message).toBe(native.message);
+    expect(objectql.message).toBe(native.message);
+    return native;
+  };
+
+  for (const [label, runtimeFilter] of [
+    ['the whole filter', { bogus: 'x' }],
+    ['one arm of an $or', { $or: [{ bogus: 1 }, { region: 'amer' }] }],
+    ['the array spelling', [['bogus', '=', 'x']]],
+  ] as const) {
+    it(`refuses a filter on 'bogus' as ${label}, INVALID_FIELD / 400 param where, with one message on every face`, async () => {
+      const err = await refusedAlike({ dimensions: ['region'], measures: ['total'], runtimeFilter }, 'where', 'bogus');
+      expect(err.message).toContain("Filter member 'bogus' in 'where' on cube 'os28257_deals' constrains field 'bogus'");
+    });
+  }
+
+  for (const [label, runtimeFilter, expected] of [
+    ['a declared dimension', { region: 'amer' }, [{ key: 'amer', total: 150 }]],
+    ['an undeclared real field', { amount: { $gt: 40 } }, [{ key: 'amer', total: 150 }]],
+    ['an $in on a declared dimension', { region: { $in: ['emea'] } }, [{ key: 'emea', total: 30 }]],
+  ] as const) {
+    it(`CONTROL: a filter on ${label} serves the same rows on every face`, async () => {
+      for (const face of FACES) {
+        const result = await ask(face, { dimensions: ['region'], measures: ['total'], runtimeFilter });
+        const rows = (result.rows as Array<Record<string, unknown>>).map((r) => ({ key: r.region, total: Number(r.total) }));
+        expect(rows, face).toEqual(expected);
+      }
+    });
+  }
+
+  for (const dimension of ['payload', 'payload_dim'] as const) {
+    it(`refuses the json dimension '${dimension}' INVALID_FIELD / 400 param dimensions, with one message on every face`, async () => {
+      const err = await refusedAlike({ dimensions: [dimension], measures: ['total'] }, 'dimensions', 'payload');
+      expect(err.message).toContain('declares as json');
+    });
+  }
+
+  for (const [label, selection, param, field] of [
+    ['a json dimension beside a bogus filter: the json door', { dimensions: ['payload'], measures: ['total'], runtimeFilter: { ghost: 1 } }, 'dimensions', 'payload'],
+    ["a json dimension beside 'region_sum': the json door", { dimensions: ['payload'], measures: ['region_sum'] }, 'dimensions', 'payload'],
+    ["'amount + 1' beside a bogus filter: the where gate", { dimensions: ['region'], measures: ['amount + 1'], runtimeFilter: { ghost: 1 } }, 'where', 'ghost'],
+  ] as const) {
+    it(`two mistakes, ${label} first, on every face`, async () => {
+      await refusedAlike(selection, param, field);
+    });
+  }
+});

packages/services/service-analytics/src/analytics-service.ts

@@ -2772,6 +2772,10 @@ export class AnalyticsService implements IAnalyticsService {
             // that names a real field (`amount`) groups by it, here as there.
             this.assertMeasureFields(q, cube, Object.keys(compiled.cube.measures));
             this.assertDimensionFields(q, cube, Object.keys(compiled.cube.dimensions));
+            // [#28257] …and the structured-JSON dimension door (#20807): a
+            // grouped dimension over a `json` column is refused there, where
+            // the evaluator grouped by each row's object value.
+            this.assertDimensionsGroupScalarColumns(q, cube);
             // [#21044] …and the aggregate × field-type table: a minted
             // `region_sum` over a text field is refused there, and here it
             // would sum no number at all and read `0`.
@@ -2780,6 +2784,12 @@ export class AnalyticsService implements IAnalyticsService {
             // must not serve the sum across currencies that the published
             // dataset refuses.
             this.assertRecordCurrencyGrouped(q, cube);
+            // [#28257] …and the `where` field gate (#5669), last in
+            // `ensureCube`: a filter member whose field the object does not
+            // have (a `runtimeFilter` typo) is refused `INVALID_FIELD` / 400
+            // with `param: 'where'`, where the evaluator matched no row and
+            // served `[]` — or, under an `$or`, served the other arm's rows.
+            this.assertWhereFields(q, cube, Object.keys(compiled.cube.dimensions));
             // [#21156] `query()` asks the caller-member gate after `ensureCube`.
             this.assertCallerMeasuresResolvable(q, compiled.cube, compiled.cube, compiled.durations, context);
             // [#21120] The body-column refusal rides this preview path too —