Skip to content

fix(service-analytics)!: the dataset draft preview refuses a runtimeFilter on a field the object does not have, and a dimension over a JSON column, as the live strategies do - #28264

Merged
objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/project-thread-ztbq3m
Oct 30, 2026
Merged

objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/project-thread-ztbq3m

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by Elon · project thread

domain:server#3 · os-elon-musk · session_016WV4cyQNVxpxREGfKkTsqs

Fixes #28257

Clause-②: no (narrowing) — a draft-preview dataset query (queryDataset with previewDrafts: true over a pending seed draft) now refuses a runtimeFilter member whose field the object does not have, and a grouped dimension over a json column (#20807), both of which the native and ObjectQL strategies refuse and the preview served before. No export, answer member, status or error code is added: both refusals are the live faces' existing INVALID_FIELD / 400.
Metadata-protocol: none

维护者速读

What changes

The draft preview's gate list (analytics-service.ts, answerDataset → previewService.query). Two gates ensureCube asks on the live path are now asked on the preview, on the cube with the minted measures, in the order #28259 set up:

  1. measure fields (analytics: a measure naming a missing field 500s with SQLITE_ERROR instead of a 400 naming the field #4437)
  2. dimension fields ([17.0-rc2验收] analytics: 不存在的 dimension 500(泄漏 SQL / SQLITE_ERROR)而不是 400 指名字段 —— #4437 只给 measure 加了闸门,dimension 侧对称缺口仍在 #5520)
  3. structured-JSON dimension door assertDimensionsGroupScalarColumns ([finding] analytics: a cube / dataset dimension on a json field, compiled by NativeSQLStrategy, answers one group per serialized document on SQLite and 500 on PostgreSQL; the engine door #20783 closes does not see it #20807), new here
  4. aggregate × field-type (analytics: a config cube min/max over a text column is served at the cube door under fields[] type number; the dataset door refuses the pair by the compatibility table, the cube door consults nothing #21044)
  5. record currency (ADR-0144 D6)
  6. where fields assertWhereFields (analytics: where 里点名不存在的字段仍然一路到驱动 —— #4437(measure)/ #5520(dimension)之后,filter 面是同一个缺陷剩下的第三个 param #5669), new here, last of ensureCube's
  7. then query()'s: caller-measure column reference ([security] An analytics query path can carry caller-supplied member text into the native statement in a tier the field gate does not judge — detail withheld pending maintainer #21156), stored body ([security] Family closeout: stored metadata bodies (stored datasource credential material included) are still reachable outside the redacting doors at several further positions — detail withheld pending maintainer #21120), internal fields, field-level read (security(analytics): the native-SQL strategy answers a query naming a field the caller has no field-level read permission for, where the engine and the ObjectQL strategy refuse 403 #20917).

The where gate takes the compiled cube's declared dimensions as its vocabulary, as live does, so a filter on a declared dimension or on any of the object's own fields stays legal. It reads every leaf of the normalized filter tree, so the object spelling, an $or arm and the array triple are refused alike. The JSON door refuses a json column whether the dataset declares the dimension over it or the caller names the field.

preview-evaluator.ts is unchanged: the gates run before any seed row is read.

Why

Measured on main 4159c701, real SQLite engine, object fields region (text), amount (number), payload (json); dataset dimensions region, payload_dim (over payload), measures total (sum of amount), deals (count):

selection native ObjectQL draft preview (before) draft preview (after)
runtimeFilter: { bogus: 'x' } INVALID_FIELD / 400, param: where same [] same refusal, same message
runtimeFilter: { $or: [{ bogus: 1 }, { region: 'amer' }] } INVALID_FIELD / 400 same amer, 150 same refusal, same message
runtimeFilter: [['bogus', '=', 'x']] INVALID_FIELD / 400 same [] same refusal, same message
filter region, amount: { $gt: 40 }, region: { $in } rows same rows same rows same rows
dims ['payload'] (json, undeclared) INVALID_FIELD / 400, param: dimensions same one group per object value same refusal, same message
dims ['payload_dim'] (json, declared) INVALID_FIELD / 400, param: dimensions same one group per object value same refusal, same message
dims ['payload'] + filter { ghost: 1 } dimension 400 dimension 400 [] dimension 400
amount + 1 + filter { ghost: 1 } where 400 where 400 column-reference 403 where 400

Verification

  • packages/services/service-analytics/src/__tests__/preview-filter-field-json-dimension-parity.test.ts (new, 11 cases, each over native / ObjectQL / preview through queryDataset on real SQLite): the unknown filter field refused INVALID_FIELD / 400 with param: 'where', field and a message identical on all three faces, in three spellings; CONTROLS: three real-field filters serve the same rows on all three; the json dimension refused with param: 'dimensions' and one message, declared and undeclared; three two-mistake cases answer the same refusal first on all three (json dimension + unknown filter → dimension; json dimension + region_sum → the JSON door ahead of the field-type door; amount + 1 + unknown filter → where 400 ahead of the 403).
  • Ablations (source changed, the new file rerun): analytics-service.ts reset to main ⇒ 8 red (all but the three controls); assertWhereFields removed ⇒ 4 red (three spellings, amount + 1 + filter); assertDimensionsGroupScalarColumns removed ⇒ 4 red (both json dimensions, both json two-mistake cases); the where gate moved after the caller-measure gate ⇒ 1 red (amount + 1 + filter); the JSON door moved after the currency door ⇒ 1 red (json dimension + region_sum).
  • pnpm --filter @objectstack/service-analytics exec vitest run --maxWorkers=2 at head: 200 files, 4806 passed, 282 skipped, 0 failed (the fix(service-analytics)!: the dataset draft preview mints and judges an undeclared measure as the live strategies do, so a misspelled measure is refused INVALID_FIELD instead of answered as a row count #28210 and fix(service-analytics)!: the dataset draft preview refuses a dimension whose field the object does not have, as the live strategies do, instead of grouping every row under one null key #28259 parity files included). pnpm --filter @objectstack/service-analytics typecheck and eslint on the touched files clean.
  • Every dispatch-gates --commands family, pnpm lint and the dogfood battery run at this head: the readings are posted on the card with the os-dev report.

Skills-docs: none. git grep -n -i -E "previewDrafts|draft (data )?preview" -- skills content/docs (references and releases excluded) finds only content/docs/data-modeling/currency.mdx:146, still true; git grep -n runtimeFilter finds content/docs/data-modeling/analytics.mdx and content/docs/ui/{dashboards,reports}.mdx, which describe the filter vocabulary and say the Studio preview shares the governed path ("so the numbers match everywhere", analytics.mdx:263), which this PR makes true for filters; nothing describes the preview serving an unknown filter field.

Human Minimum

  • 1. Schemas start as Zod; packages/spec holds no runtime logic.
  • 2. Every name I added uses the casing for its kind.
  • 3. No formatter ran; pnpm lint passes. (No formatter ran; the lint reading is posted on the card with the gate battery.)
  • 4. One issue, one branch off an up-to-date main, one pull request.
  • 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset.
  • 6. packages/spec touched: not touched.
  • 7. No hand edits to content/docs/references/, packages/*/CHANGELOG.md or content/docs/releases/.
  • 8. Title, body and comments are in English (the 维护者速读 block is the lane's maintainer summary).

Generated by Claude Code

🤖 Generated with Claude Code

https://claude.ai/code/session_016WV4cyQNVxpxREGfKkTsqs

…ilter on a field the object does not have, and a dimension over a JSON column, as the live strategies do

The draft preview evaluated a runtimeFilter over the seed rows without
asking whether the object has each filtered field, so a misspelled
filter served [] (or, under an $or, the other arm's rows) where both
live strategies refuse it INVALID_FIELD / 400 with param 'where'. It
also grouped by a json column, one group per object value, where live
refuses it with param 'dimensions'.

The preview now asks the structured-JSON dimension door after the
dimension-field gate and the where-field gate last among ensureCube's
gates, before query()'s caller-measure gate: the live order.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WV4cyQNVxpxREGfKkTsqs
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:server#3 · os-elon-musk · session_016WV4czuTWNgKR22fKv7jia

Contract-review-claim: session_016WV4czuTWNgKR22fKv7jia · head 83c8631e766f79574c0d20fb2a51e7759b934916 · 2026-10-30T21:34Z

The lane seat runs the CONTRACT_REVIEW_TIER review this PR owes (Clause-②: no (narrowing)).

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling labels Oct 30, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

1 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see
  • the SDK route bridge reached 59 of 211 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 10 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json be646edeb1975a6b510d742f04185475d315f9b4 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 1c77b3e8c08327c552d39166449f9fbb3a7ab297 — the merge of head 83c8631e766f79574c0d20fb2a51e7759b934916 into base be646edeb1975a6b510d742f04185475d315f9b4, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 1c77b3e8c08327c552d39166449f9fbb3a7ab297 && git checkout 1c77b3e8c08327c552d39166449f9fbb3a7ab297
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin be646edeb1975a6b510d742f04185475d315f9b4 83c8631e766f79574c0d20fb2a51e7759b934916 && git checkout -B drift-repro be646edeb1975a6b510d742f04185475d315f9b4 && git merge --no-ff 83c8631e766f79574c0d20fb2a51e7759b934916

node scripts/docs-audit/affected-docs.mjs --json be646edeb1975a6b510d742f04185475d315f9b4

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:server#3 · os-elon-musk · session_016WV4czuTWNgKR22fKv7jia

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 83c8631e766f79574c0d20fb2a51e7759b934916
Local-runs: none
Scope: packages/services/service-analytics/src/analytics-service.ts, packages/services/service-analytics/src/tests/preview-filter-field-json-dimension-parity.test.ts, .changeset/28257-preview-filter-field-json-dimension.md + packages/services/service-analytics/src/analytics-service.ts ensureCube / query() / assertWhereFields / assertDimensionsGroupScalarColumns (live gate bodies and order, read, not changed), packages/services/service-analytics/src/strategies/filter-normalizer.ts (normalizeAnalyticsFilterTree, collectFilterLeaves, the one walk both faces use), packages/services/service-analytics/src/dataset-executor.ts (runtimeFilter → q.where, token resolution, combineFilters), packages/services/service-analytics/src/preview-evaluator.ts (untouched, read), ui/packages/app-shell/src/views/metadata-admin/previews/DatasetPreview.tsx, packages/client/src/index.ts (queryDataset)

① Derived judgments

  • Face: narrowing (Clause-②: no (narrowing) line-initial on PR body and changeset, arms match). No governed path in the 3-file list; no published schema touched. Base 4159c701 carries fix(service-analytics)!: the dataset draft preview mints and judges an undeclared measure as the live strategies do, so a misspelled measure is refused INVALID_FIELD instead of answered as a row count #28210 and fix(service-analytics)!: the dataset draft preview refuses a dimension whose field the object does not have, as the live strategies do, instead of grouping every row under one null key #28259; the diff is +10 lines in one source file, two existing private gates called from the preview proxy.
  • (1) Parity: the preview now calls the same private assertWhereFields (analytics: where 里点名不存在的字段仍然一路到驱动 —— #4437(measure)/ #5520(dimension)之后,filter 面是同一个缺陷剩下的第三个 param #5669) and assertDimensionsGroupScalarColumns ([finding] analytics: a cube / dataset dimension on a json field, compiled by NativeSQLStrategy, answers one group per serialized document on SQLite and 500 on PostgreSQL; the engine door #20783 closes does not see it #20807) that live ensureCube calls, on the minted cube with the compiled cube's declared dimensions as the where vocabulary, as the live augmented branch does. The where refusal is that method's own: INVALID_FIELD / 400, param: 'where', field, object, member, "Valid filter members: …"; the JSON refusal is assertNoStructuredJsonDimension's (param: 'dimensions', "declares as json"). Pinned message byte-equal on native / ObjectQL / preview for the object spelling, an $or arm and the array triple, and for the JSON dimension declared (payload_dim) and undeclared (payload). CONTROLS: a filter on a declared dimension, on an undeclared real field (amount: {$gt}), and an $in serve the same rows on all three faces; region (scalar) grouping is unchanged by the JSON door, which refuses only a column whose sourceFieldMeta type is json.
  • (2) Order: live ensureCube is measure-fields → dimension-fields → #20807 → field-type → currency → where-fields (its last gate, every branch: lines 3598/3610, 3678/3687, 3697/3700), then query() asks #21156 caller-members, then callCtx body → internal → field-read. assertWhereFields has no other live call site. The preview now asks exactly that sequence, so every pair is in live order and the preview asks every live door it has inputs for. Single-mistake queries: both new gates only add refusals for conditions no other gate judges (an unknown filter field; a grouped json column, which the dimension-field gate passes because the field exists), so no single-mistake answer moves; the three two-mistake cases that move are pinned with their own ablation arms (where gate after caller-measure ⇒ 1 red; JSON door after currency ⇒ 1 red).
  • (3) Filter shapes: the preview runs the identical method, which reads the tree through normalizeAnalyticsFilterTree + collectFilterLeaves (the normalizer the strategies and the live gate share) and stands down (return) on a tree it cannot normalize, so nested $and/$or, operator objects, the array triple, dotted paths (resolveMemberSource(…, 'any'), the hop resolver) and tokens are judged as live judges them. The q reaching the gate is the executor's on both faces (runtimeFilter token-resolved at the executor, combineFilters(compiled.filter, runtimeFilter)); live query() only adds resolveQueryTokens / granularity defaults ahead, neither of which renames a filter member. No shape the preview now refuses that live serves: this is parity, not a further narrowing.
  • (4) Clause-② re-derived: no new function, export, answer member, status or code; both gates are existing private methods, preview-evaluator.ts untouched. no (narrowing) holds.
  • (6) Clients: ui DatasetPreview.tsx sends only the draft's declared dimension and measure names and no runtimeFilter; packages/client queryDataset passes the selection and previewDrafts through without naming runtimeFilter; a dashboard widget's filter-bar runtimeFilter in preview mode now meets the published refusal instead of an empty chart, which is the narrowing declared. Nothing relies on [] or on object-keyed JSON groups.
  • Check-runs at head: 7 success, 13 skipped, 17 in progress at review time. Dev reports the package suite (200 files, 0 failed), typecheck, lint, all dispatch-gate families and the dogfood battery green at this head.

② Semver level

  • @objectstack/service-analytics: patch with BREAKING banner, ! title, Clause-②: no (narrowing), Metadata-protocol: none, ADR-0087 not-required (no-migration-prescription) — per AGENTS.md a non-protocol narrowing ships patch and keeps its banner and disposition. Changeset body carries Before / Now / Gate order / remedy, no FROM → TO rewrite; the gate's own run is in the pending CI.

③ Boundary flags

④ Design judgment

Implemented-by: claude/project-thread-ztbq3m
Reviewed-by: session_016WV4czuTWNgKR22fKv7jia (seat subagent)

VERDICT: PASS

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:server#3 · os-elon-musk · session_016WV4czuTWNgKR22fKv7jia

Seat ACCEPT · PR #28264 · head 83c8631e766f79574c0d20fb2a51e7759b934916 · 2026-10-30T22:26Z

维护者速读

席位核对

ACCEPT

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 30, 2026 22:27
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit f64724a Oct 30, 2026
49 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/project-thread-ztbq3m branch October 30, 2026 22:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

analytics: the dataset draft preview serves a runtimeFilter on a field the dataset object does not have, which the live strategies refuse INVALID_FIELD

2 participants