Skip to content

Commit 85be049

Browse files
fix(lint): filter-token-unknown states the runtime's refusal, FILTER_TOKEN_UNKNOWN / 400 (#24312)
Fixes #24293 Clause-②: no ## What changes `filter-token-unknown`'s message said an unresolvable filter placeholder "is sent to the data engine as a literal string, matches no record, and the surface renders empty". The runtime does not do that: the data engine refuses a query that carries one, `FILTER_TOKEN_UNKNOWN` / 400. The message now says so: ```text Filter value "{current_user}" is not a resolvable placeholder. The data engine refuses a query that carries it (`FILTER_TOKEN_UNKNOWN`, 400), so the surface never shows the data the filter meant. ``` Message and comment text only. Unchanged: the rule id, severity (`error`), `path`, `hint`, and which values the rule reports. The message's first sentence is unchanged, so the negative pin at `validate-filter-tokens.test.ts` (the `{record_id}` refusal must not read as an unknown spelling) still guards the same text. ## Every site of the false sentence, on base `21070a2f0a` | site | done | |:--|:--| | `packages/lint/src/validate-filter-tokens.ts:236`–`:237`, the message | rewritten as above | | `validate-filter-tokens.ts:25`–`:26`, the module header ("passed to the data engine **verbatim**, matches no row") | rewritten: the engine refuses the query, naming `UnknownFilterTokenError` in `@objectstack/core` | | `validate-filter-tokens.ts:28`–`:41`, the header's "Why this is an error" section ("The runtime failure mode is silent", present tense) | put in the past tense: the silent zero is what the rule was written against, before the runtime resolved placeholders. Says why the rule stays an error now that the runtime refuses: the refusal arrives only when a surface is opened, after the build passed, which an AI author's author, validate, correct loop never sees | | `validate-filter-tokens.ts:106`, the `severity` field doc ("silently matches nothing") | rewritten: the engine refuses a query carrying an unresolvable placeholder | | `packages/lint/src/validate-dataset-references.ts:36`–`:39`, the docblock quoting the old message (triage `6248475229`) | corrected. The quote is gone, and the comparison it carried is inverted: the value `filter-token-unknown` reports is refused loudly, while a dangling field path at the same node is the quiet failure, per that file's own severity note | | `packages/lint/src/authoring-rules.ts:917`–`:920`, the comment above `validateFilterTokens`' registration ("reaches the data engine as a literal and matches nothing. The surface renders empty with no error"). The comment ships inside `@objectstack/lint`'s `dist/index.js` and `dist/runtime.js` | rewritten in `f3d1666d9`, comment text only. The claim's file surface was amended to add it (`6249110232`). The `#3574` provenance and the silent-zero history are kept, in the past tense, as in the module header. Before and after are below | The `authoring-rules.ts` comment before (`21070a2f0`, `:917`–`:920`): ```text // #3574 — a filter value like `{current_user}` resolves in no vocabulary, // reaches the data engine as a literal and matches nothing. The surface // renders empty with no error, and a silent zero is indistinguishable from a // genuine one at review time. ``` and after (`f3d1666d9`, `:917`–`:921`): ```text // #3574 — a filter value like `{current_user}` resolves in no vocabulary. The // data engine refuses a query carrying it (`FILTER_TOKEN_UNKNOWN`, 400). When // #3574 was filed it reached the engine as a literal and matched nothing: the // surface rendered empty with no error, a silent zero indistinguishable from a // genuine one at review time. ``` `packages/lint/src/rule-explanations.ts` (`os explain filter-token-unknown`, from #24292) already says "the data engine refuses the query (`FILTER_TOKEN_UNKNOWN`, 400) rather than compare the placeholder as a literal string, so the surface never shows the data the filter meant". The new message takes its words, so the two agree. That file is not touched. ## What the runtime does, read on base `21070a2f0a` - `packages/core/src/utils/filter-tokens.ts`: `resolveFilterTokens` throws `UnknownFilterTokenError` (`status = 400`, `code = 'FILTER_TOKEN_UNKNOWN'`) for a whole-string placeholder outside both vocabularies (`:547`), and for one the resolver drops (`:552`). - Callers: the ObjectQL engine on every read verb (`find`, `findOne`, `count`, `aggregate`, through `resolveWhereTokens`) and on `update` / `delete` (`withResolvedWhere`); the analytics service on `where`, `timeDimensions[].dateRange` and a dataset's own filter and measure filters (`analytics-service.ts:1852`–`:1883`, `dataset-executor.ts:128`). - Pins: `packages/objectql/src/engine-filter-token-today-spelling.test.ts:71`–`:74` (`FILTER_TOKEN_UNKNOWN` / 400) and `packages/rest/src/rest-aggregate-positions.test.ts:171`, which asserts `_status` 400 and `code` `FILTER_TOKEN_UNKNOWN` at `POST /api/v1/data/:object/query` for a `where` carrying `'{not_a_token}'`. - objectui's client resolver (`packages/core/src/utils/filter-tokens.ts` there, `resolveContextTokens`) passes an unknown token through untouched, so it reaches the server and is refused there. - The message claims the engine's door only, and its claim is conditional ("refuses a query that carries it"). A surface whose filter never becomes a server query is outside the claim. The one such door this read found is in Acceptance notes. ## Tests On head `f3d1666d9` (round 1's `6be194c50`, plus the `authoring-rules.ts` comment commit): - `pnpm --workspace-concurrency=2 --filter '@objectstack/lint...' build` (lint and its build closure): `VERDICT command-exit 0`. The old sentence's phrases ("reaches the data engine as a literal", "renders empty with no error", "sent to the data engine as a literal string", "passed to the data engine", "matches no record, and the surface renders empty", "unresolved placeholder silently matches nothing") have 0 hits in `packages/lint/src` and 0 in `packages/lint/dist`. The controls hit: the new comment's "data engine refuses a query carrying it" (src 1, dist 4 files) and the new message (dist 4 files). - `pnpm --filter @objectstack/lint exec vitest run --maxWorkers=2`: `Test Files 144 passed (144)`, `Tests 6858 passed (6858)`. The 5 cases that `6be194c50`'s run skipped (`6853 passed | 5 skipped`) are the `skipIf` cases in `lazy-deps.test.ts` and `runtime-lazy-deps.test.ts`. They need lint's own built `dist/`, which that run did not have. This time they ran, against the rebuilt `dist/`. - `pnpm --filter @objectstack/lint typecheck`: `VERDICT command-exit 0`. `check:test-typecheck: OK`. `tsc -p tsconfig.test.json --listFiles` lists `validate-filter-tokens.test.ts`, so the test layer compiles the new case. - New case in `validate-filter-tokens.test.ts`: "names the code the data engine refuses the query with, near miss or not". It holds both arms of the unknown-token finding (a near miss `{current_user}`, and `{whoever_is_watching}` with none) to the named code `FILTER_TOKEN_UNKNOWN`, not to the prose around it. - No test pinned the old prose. A repo-wide `git grep` for its phrases ("literal string, matches no", "surface renders empty", "sent to the data engine", "passed to the data engine", "is not a resolvable placeholder") (plus "reaches the data engine as a literal") outside the changeset, which quotes the old text as its FROM, finds only the kept first sentence and the negative pin above. The same grep over objectui at `32b492b67` finds nothing. `packages/cli/test/authoring-rule-command-parity.test.ts:107` names the rule id only. - Ablation, one-off, through `scripts/ablation-replace.mjs` (anchor hit x1 to x0, blob `47d8294046cf` to `a81f9a88dae0`): with `` (`FILTER_TOKEN_UNKNOWN`, 400) `` removed from the message, `Tests 1 failed | 33 passed (34)`, failing on exactly the new case: `expected 'Filter value "{current_user}" is not …' to contain` the code. Restored: blob equals HEAD (`47d8294046cf`), `git diff HEAD` empty, `git status` clean. The test imports the source by relative path, so no `dist/` sits between the mutation and the run. ## Gates (local subset; the whole farm is CI's) Derived on head `f3d1666d9` with `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack`, no paths given (change set: these 5 files vs merge base `21070a2f0`). That gives 64 commands, the same set as on `6be194c50`, and 11 more than the claim's path-only lead of 53. The extra 11 are the changeset-kind families. Each was run with its exit code captured before any pipe, and the record was reconciled with `--ran`: "64 derived famil(ies) accounted for — 62 run, 2 NOT-MEASURED (2 DERIVED from a recorded exit 3)", exit 0. - 62 exited 0, `pnpm check:docs-transcript-drift` among them, since lint's `dist/` was built this time. On `6be194c50`, 61 exited 0 on the first pass. These include `check:nul-bytes`, `check:issue-citations`, `check:type-check-coverage`, `check:type-check-debt`, `check:slot-lookup`, `check:changeset-gate-self-tests`, `check-changeset-no-major.mjs --base origin/main` and `check-empty-changeset.mjs --base origin/main`. - `pnpm check:docs-transcript-drift`: on `6be194c50` it exited 3 on the first pass (PREREQUISITE NOT MET: no `packages/lint/dist`), then 0 after a lint build. On `f3d1666d9` it exited 0: "4 declared transcript value(s) across 417 page(s) under content/docs/ equal what the registry derives today, and no undeclared block quotes one". The built `dist/index.js` carries the new message, and the old sentence is gone from it (0 hits). - NOT MEASURED, PREREQUISITE NOT MET (exit 3): `pnpm check:dual-build-cjs-loads` needs every package's `dist/` (a whole `pnpm build`), and `pnpm check:lean-entry-closure` needs `@objectstack/objectql`'s `dist/`. Both are left to CI. - `origin/main` has moved past the base (`2ee5861b0`, `0c3db0f6a`, `30da04f8e` and later). None of these commits touches `packages/lint`, its build closure, or `packages/core/src/utils/filter-tokens.ts`. The branch is not merged with them, as the patch round directs, and CI runs on GitHub's merge ref. The derivation on `f3d1666d9` flags 4 gate inputs that changed on `main` across that range: `scripts/check-single-authz-resolver.mjs`, `scripts/slot-lookup-baseline.json`, `scripts/check-route-envelope.mjs` and one `scripts/adr-anchors/` file. Their local readings are of the base's copies, and CI's readings on the merge ref are the ones that count. ## Acceptance notes - **objectui's in-memory `ValueDataSource`** (`provider: 'value'`, `packages/core/src/adapters/ValueDataSource.ts` at objectui `32b492b67`) evaluates a filter on the client and sends no query, so no engine refuses there. Its matcher has no placeholder handling, so a literal `{…}` compares as a string. This is read from code, NOT MEASURED. The new message does not claim that door, because its claim is conditional on a query reaching the engine. - The REST refusal is cited from the existing pin (`rest-aggregate-positions.test.ts:171`), not re-run here. CI's `Test Core` runs it. --- _Generated by [Claude Code](https://claude.ai/code/session_019oFY6Hi35J7atvgVFnmoaK)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 2ce2e36 commit 85be049

5 files changed

Lines changed: 57 additions & 22 deletions

File tree

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
---
2+
"@objectstack/lint": patch
3+
---
4+
5+
fix(lint): `filter-token-unknown` states what the runtime does with an unresolvable filter placeholder: the data engine refuses the query with `FILTER_TOKEN_UNKNOWN` / 400
6+
7+
Clause-②: no
8+
9+
The finding's message said the value "is sent to the data engine as a literal string, matches no record, and the surface renders empty". The runtime does not do that. The data engine refuses a query that carries an unresolvable placeholder, `FILTER_TOKEN_UNKNOWN` with HTTP 400, rather than compare it as a literal string. The message now reads:
10+
11+
```text
12+
Filter value "{current_user}" is not a resolvable placeholder. The data engine refuses a query that carries it (`FILTER_TOKEN_UNKNOWN`, 400), so the surface never shows the data the filter meant.
13+
```
14+
15+
Unchanged: the rule id, its severity (`error`), the finding's `path`, its `hint` (the `fix:` line), and which values the rule reports. The message's first sentence is unchanged as well. `os explain filter-token-unknown` already stated the refusal and is unchanged.

‎packages/lint/src/authoring-rules.ts‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -914,9 +914,10 @@ export const AUTHORING_RULES: readonly AuthoringRule[] = [
914914
'saves, a rollout decision over stored tenant rows the in-repo corpus does not measure.',
915915
run: (stack) => validateDashboardWidgetOptions(stack),
916916
},
917-
// #3574 — a filter value like `{current_user}` resolves in no vocabulary,
918-
// reaches the data engine as a literal and matches nothing. The surface
919-
// renders empty with no error, and a silent zero is indistinguishable from a
917+
// #3574 — a filter value like `{current_user}` resolves in no vocabulary. The
918+
// data engine refuses a query carrying it (`FILTER_TOKEN_UNKNOWN`, 400). When
919+
// #3574 was filed it reached the engine as a literal and matched nothing: the
920+
// surface rendered empty with no error, a silent zero indistinguishable from a
920921
// genuine one at review time.
921922
{
922923
name: 'validateFilterTokens',

‎packages/lint/src/validate-dataset-references.ts‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,10 +33,11 @@
3333
*
3434
* ## Why a dataset's dangling reference is worse than most
3535
*
36-
* `filter-token-unknown`'s own wording states the failure mode it exists to
37-
* prevent: the value is *"sent to the data engine as a literal string, matches
38-
* no record, and the surface renders empty."* A dangling field path produces
39-
* the same outcome from the same node.
36+
* `filter-token-unknown` judges the VALUE at that same node, and the value it
37+
* reports does not fail quietly: the data engine refuses the query
38+
* (`FILTER_TOKEN_UNKNOWN`, 400). A dangling field path there is the quiet
39+
* failure. By the severity note below, every outcome it produces reaches a
40+
* human as a chart that rendered successfully.
4041
*
4142
* A dataset is the semantic layer — dashboards and reports bind its dimensions
4243
* and measures BY NAME (ADR-0021) — and the CONSUMER end of that binding is

‎packages/lint/src/validate-filter-tokens.test.ts‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,18 @@ describe('validateFilterTokens', () => {
5656
expect(findings[0].hint).toContain('{current_user_id}');
5757
});
5858

59+
// [#24293] The message states the runtime's answer: the data engine refuses a
60+
// query carrying the value (`UnknownFilterTokenError` in `@objectstack/core`,
61+
// `FILTER_TOKEN_UNKNOWN` / 400). Held as that code, the named subject, not as
62+
// the prose around it.
63+
it('names the code the data engine refuses the query with, near miss or not', () => {
64+
const findings = validateFilterTokens({
65+
dashboards: [{ name: 'd', widgets: [{ id: 'w', filter: { a: '{current_user}', b: '{whoever_is_watching}' } }] }],
66+
});
67+
expect(findings.map((f) => f.path)).toEqual(['dashboards[0].widgets[0].filter.a', 'dashboards[0].widgets[0].filter.b']);
68+
for (const f of findings) expect(f.message, f.path).toContain('`FILTER_TOKEN_UNKNOWN`');
69+
});
70+
5971
it('accepts the correct spelling alongside date macros', () => {
6072
expect(
6173
validateFilterTokens({

‎packages/lint/src/validate-filter-tokens.ts‎

Lines changed: 21 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -22,22 +22,27 @@ import { recordsOf } from './object-graph.js';
2222
* Exactly two vocabularies resolve inside a filter value: context tokens
2323
* (`{current_user_id}`, `{current_org_id}` — see `context-tokens.zod.ts`) and
2424
* date macros (`{today}`, `{30_days_ago}` — see `date-macros.zod.ts`).
25-
* Anything else is passed to the data engine **verbatim**, matches no row, and
26-
* the surface renders an empty result.
25+
* Anything else does not resolve: the data engine refuses a query that carries
26+
* it (`FILTER_TOKEN_UNKNOWN`, 400 — `UnknownFilterTokenError` in
27+
* `@objectstack/core`'s `filter-tokens.ts`), so the surface never shows the
28+
* data the filter meant.
2729
*
2830
* ## Why this is an error, not a warning
2931
*
30-
* The runtime failure mode is silent and indistinguishable from success. A
31-
* metric widget filtered on an unresolved `{current_user}` renders `0`, which
32-
* looks exactly like a metric that is legitimately zero — no console error, no
33-
* server log, nothing for a human reviewer to notice. Issue #3574 found a
34-
* dashboard that had been broken this way since the day it was written.
32+
* The rule is older than that refusal. When issue #3574 was filed, the runtime
33+
* substituted no placeholder: the value reached the data engine verbatim and
34+
* matched no row. A metric widget filtered on an unresolved `{current_user}`
35+
* rendered `0`, which looks exactly like a metric that is legitimately zero —
36+
* no console error, no server log, nothing for a human reviewer to notice.
37+
* Issue #3574 found a dashboard that had been broken this way since the day it
38+
* was written. The runtime now refuses the query instead, so the failure is no
39+
* longer silent, but it still shows only when someone opens the surface.
3540
*
36-
* That failure mode is worse for AI authors than for humans. An AI reads a
37-
* successful query returning `0` as a correct answer and builds on it — it has
38-
* no instinct that the number looks wrong. Its correction loop is
39-
* "author → validate → fix", so a diagnostic only reaches it if the diagnostic
40-
* can fail the build. A runtime warning in a server log is invisible to it.
41+
* Neither form reaches an AI author. An AI read the old `0` as a correct answer
42+
* and built on it — it has no instinct that the number looks wrong. Its
43+
* correction loop is "author → validate → fix", so a diagnostic only reaches it
44+
* if the diagnostic can fail the build. A server log line, or a refusal that
45+
* arrives when a surface is opened after the build passed, is invisible to it.
4146
* Hence: authoring-time error.
4247
*
4348
* The near-miss spellings this catches are not hypothetical. Each is a correct
@@ -103,7 +108,7 @@ export const FILTER_TOKEN_UNKNOWN = 'filter-token-unknown';
103108
export type FilterTokenSeverity = 'error' | 'warning';
104109

105110
export interface FilterTokenFinding {
106-
/** Always `error` today — an unresolved placeholder silently matches nothing. */
111+
/** Always `error` today — the data engine refuses a query carrying an unresolvable placeholder. */
107112
severity: FilterTokenSeverity;
108113
/** Diagnostic rule id. */
109114
rule: string;
@@ -233,8 +238,9 @@ function walkFilterValues(
233238
where,
234239
path,
235240
message:
236-
`Filter value "${node}" is not a resolvable placeholder. It is sent to the ` +
237-
`data engine as a literal string, matches no record, and the surface renders empty.`,
241+
`Filter value "${node}" is not a resolvable placeholder. The data engine refuses a ` +
242+
`query that carries it (\`FILTER_TOKEN_UNKNOWN\`, 400), so the surface never shows ` +
243+
`the data the filter meant.`,
238244
hint: suggestion && isRecordContextToken(suggestion)
239245
? `Did you mean "{${suggestion}}"? It resolves only in a filter on a component of a ` +
240246
`\`type: 'record'\` page, to the id of the record in view.`

0 commit comments

Comments
 (0)