diff --git a/.changeset/25269-page-window-declared.md b/.changeset/25269-page-window-declared.md new file mode 100644 index 0000000000..c0a98304be --- /dev/null +++ b/.changeset/25269-page-window-declared.md @@ -0,0 +1,20 @@ +--- +'@objectstack/spec': patch +'@objectstack/metadata-protocol': patch +--- + +fix(spec)!: the published list-query schemas declare the page window the Data API enforces — a non-negative integer — instead of any number + +Clause-②: no (narrowing) +Metadata-protocol: none + + + +**BREAKING** accept-set narrowing on two published request schemas. It changes what a schema-validated Data API request may carry, not what a metadata author can write, so it ships as `patch` with this banner. + +- **Before.** `FindDataRequestSchema` (the `POST /api/v1/data/:object/query` body) declared `limit` / `offset` / `top` as any number and `$top` / `$skip` / `skip` as any number or string, and `HttpFindQueryParamsSchema` (the `GET /api/v1/data/:object` querystring) declared `top` / `skip` as `z.coerce.number()`. A client generated from them, or a request validated against them, was told `-1`, `1.5` or a blank `?top=` was valid, and the door then refused it with a `400`. +- **Now.** Each of those parameters declares the page window the door enforces: a non-negative integer, or on the string-carrying spellings the digits a querystring carries it as (`'50'`). `-1`, `1.5`, `NaN`, `Infinity`, a blank or whitespace-only string, and a string that is not a count fail the parse. The bound is exported once, as `QueryPageWindowSchema` / `QueryPageWindowParamSchema` (`@objectstack/spec/data`), and the list-query normalizer in `@objectstack/metadata-protocol` now judges every window by that same schema, so the two accept sets are one declaration. +- **One answer moves, and only its code.** On `POST /api/v1/data/:object/query` (and the runtime dispatcher's `POST` twin) a malformed window is now refused by the body schema, like every other malformed body: `400 VALIDATION_FAILED` with `fields[0].field` naming `query.limit` / `query.offset`, where it was `400 INVALID_REQUEST` from the normalizer. Still a `400`, still before any read. `GET /api/v1/data/:object` has no body schema and answers `400 INVALID_REQUEST` naming the parameter as before. +- **Unchanged.** A well-formed window parses and pages exactly as before, `0` included (`$top=0` asks for the total alone). Windows inside `expand` and the engine and driver contracts (`QuerySchema`, `EngineQueryOptionsSchema`) are not narrowed: no door refuses those. + +**The remedy.** Send the window as a whole count of zero or more, or leave the parameter out. diff --git a/content/docs/references/api/protocol.mdx b/content/docs/references/api/protocol.mdx index 5bef891fcc..7ba9c49b4d 100644 --- a/content/docs/references/api/protocol.mdx +++ b/content/docs/references/api/protocol.mdx @@ -904,9 +904,9 @@ Enable package response | **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional | Full-text search — the query text (canonical, ADR-0061 D1), or a structured FullTextSearch configuration | | **searchFields** | `string[]` | optional | Narrow the search to these fields (server-intersected with the allowed searchable set — can only narrow, never widen; ADR-0061 D1) | | **orderBy** | `{ field: string; order?: Enum<'asc' \| 'desc'> }[]` | optional | Sorting instructions (ORDER BY) | -| **limit** | `number` | optional | Max records to return (LIMIT) | -| **offset** | `number` | optional | Records to skip (OFFSET) | -| **top** | `number` | optional | Alias for limit (OData compatibility) | +| **limit** | `integer` | optional | Max records to return (LIMIT) | +| **offset** | `integer` | optional | Records to skip (OFFSET) | +| **top** | `integer` | optional | Alias for limit (OData compatibility) | | **cursor** | `never` | optional | [REMOVED] `query.cursor` was removed in @objectstack/spec 17 (ADR-0049) — no driver ever implemented keyset pagination, so the cursor was accepted and ignored and every page came back identical (a caller looping "until hasMore is false" never terminates). Delete the key; `QueryBuilder.cursor()` was removed with it. Express the keyset as an ordinary `where` predicate on your sort key — `where: { created_at: { $gt: last.created_at } }` with the matching `orderBy` — which every driver executes with canonicalised comparands. A first-class cursor, if ever built, will be a response-minted opaque token, not this caller-built record. | | **joins** | `never` | optional | [REMOVED] `query.joins` was removed in @objectstack/spec 17 (ADR-0049) — no engine or driver ever read it: a query carrying `joins` behaved exactly as if the key were absent, while its name squatted on the reserved REST parameter set. Delete the key. Related records are read through `expand` — `expand: { owner_id: { object: 'user', fields: ['name'] } }` — which the engine resolves via batch $in queries, and whose nested query selects the related record's own columns. Keep the foreign key in your own projection (`fields: ['title', 'owner_id']`): the relation is carried by that column, so projecting it away leaves expansion nothing to resolve. A dotted `fields` path is NOT a replacement — no driver ever resolved one and the ingress refuses it (`400 INVALID_FIELD`). | | **aggregations** | `{ function: Enum<'count' \| 'sum' \| 'avg' \| 'min' \| 'max' \| 'count_distinct'>; field?: string; alias: string; filter?: any }[]` | optional | Aggregation functions | @@ -917,8 +917,8 @@ Enable package response | **expand** | `Record` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. | | **$filter** | `[string, string, any] \| [string, string] \| [string, object, ...object[]] \| object[] \| Record \| any \| null` | optional | Transport spelling of `where` (OData `$filter`) | | **filters** | `[string, string, any] \| [string, string] \| [string, object, ...object[]] \| object[] \| Record \| any \| null` | optional | Transport spelling of `where` (plural of `filter`) | -| **$top** | `number \| string \| null` | optional | Transport spelling of `limit` (OData `$top`) — a number, or the digits a querystring carries it as | -| **$skip** | `number \| string \| null` | optional | Transport spelling of `offset` (OData `$skip`) — a number, or the digits a querystring carries it as | +| **$top** | `integer \| string \| null` | optional | Transport spelling of `limit` (OData `$top`) — a non-negative integer, or the digits a querystring carries it as | +| **$skip** | `integer \| string \| null` | optional | Transport spelling of `offset` (OData `$skip`) — a non-negative integer, or the digits a querystring carries it as | | **$orderby** | `Record> \| Record \| { field: string; order?: Enum<'asc' \| 'desc'> }[] \| null` | optional | Transport spelling of `orderBy` (OData `$orderby`) | | **$select** | `string \| string[] \| null` | optional | Transport spelling of `fields` (OData `$select`) | | **$expand** | `string \| string[] \| Record \| null` | optional | Transport spelling of `expand` (OData `$expand`) | @@ -929,7 +929,7 @@ Enable package response | **filter** | `[string, string, any] \| [string, string] \| [string, object, ...object[]] \| object[] \| Record \| any \| null` | optional | Transport spelling of `where` | | **select** | `string \| string[] \| null` | optional | Transport spelling of `fields` | | **sort** | `Record> \| Record \| { field: string; order?: Enum<'asc' \| 'desc'> }[] \| null` | optional | Transport spelling of `orderBy` | -| **skip** | `number \| string \| null` | optional | Transport spelling of `offset` | +| **skip** | `integer \| string \| null` | optional | Transport spelling of `offset` — a non-negative integer, or the digits a querystring carries it as | | **populate** | `string \| string[] \| null` | optional | Transport spelling of `expand` | @@ -2055,8 +2055,8 @@ The published metadata item body, opaque by ruling (1C). Shape is the item's own | **select** | `string` | optional | Comma-separated list of fields to retrieve. | | **sort** | `string` | optional | Sort expression (e.g. "name asc,created_at desc" or "-created_at"). | | **orderBy** | `string` | optional | Alias for sort (OData compatibility). | -| **top** | `number` | optional | Max records to return (limit). | -| **skip** | `number` | optional | Records to skip (offset). | +| **top** | `integer \| string` | optional | Max records to return (limit) — a non-negative integer. | +| **skip** | `integer \| string` | optional | Records to skip (offset) — a non-negative integer. | | **expand** | `string` | optional | Comma-separated list of lookup/master_detail field names to expand. Resolved to populate array and passed to the engine for batch $in expansion. | | **search** | `string` | optional | Full-text search query. | | **distinct** | `never` | optional | [REMOVED] `query.distinct` was removed in @objectstack/spec 17 (ADR-0049 / ADR-0078) — no driver ever rendered SELECT DISTINCT; the flag's only observable effect was MIS-WIRED: the REST list path treated a distinct query as not countable and silently degraded `total`/`hasMore` to a page-local estimate while still returning duplicate rows. Delete the key; `QueryBuilder.distinct()` was removed with it, and the count suppression is gone (`total` is truthful again). For unique values of one column use the SQL/memory drivers' `distinct(object, field)` door; for unique combinations, `groupBy`; for a deduplicated count, the `count_distinct` aggregation. | diff --git a/content/docs/references/data/data-engine.mdx b/content/docs/references/data/data-engine.mdx index 585f6eadba..a097f663f1 100644 --- a/content/docs/references/data/data-engine.mdx +++ b/content/docs/references/data/data-engine.mdx @@ -22,8 +22,8 @@ The Data Engine acts as the "Driver" layer in the Hexagonal Architecture. ## TypeScript Usage ```typescript -import { BaseEngineOptionsSchema, DataEngineAggregateOptionsSchema, DataEngineAggregateRequestSchema, DataEngineCountOptionsSchema, DataEngineCountRequestSchema, DataEngineDeleteOptionsSchema, DataEngineDeleteRequestSchema, DataEngineExecuteRequestSchema, DataEngineFilterSchema, DataEngineFindOneRequestSchema, DataEngineFindRequestSchema, DataEngineInsertOptionsSchema, DataEngineInsertRequestSchema, DataEngineQueryOptionsSchema, DataEngineRequestSchema, DataEngineSortSchema, DataEngineUpdateOptionsSchema, DataEngineUpdateRequestSchema, DataEngineVectorFindRequestSchema, DroppedFieldsEventSchema, EngineAggregateOptionsSchema, EngineCountOptionsSchema, EngineDeleteOptionsSchema, EngineQueryOptionsSchema, EngineUpdateOptionsSchema, QueryTransportParamsSchema, QueryWithTransportSchema, ValidationAdvisoryEventSchema } from '@objectstack/spec/data'; -import type { BaseEngineOptions, DataEngineAggregateOptions, DataEngineCountOptions, DataEngineDeleteOptions, DataEngineExecuteRequest, DataEngineFilter, DataEngineInsertOptions, DataEngineInsertRequest, DataEngineQueryOptions, DataEngineRequest, DataEngineSort, DataEngineUpdateOptions, DataEngineVectorFindRequest, DroppedFieldsEvent, EngineAggregateOptions, EngineCountOptions, EngineDeleteOptions, EngineQueryOptions, EngineUpdateOptions, QueryTransportParams, QueryWithTransport, ValidationAdvisoryEvent } from '@objectstack/spec/data'; +import { BaseEngineOptionsSchema, DataEngineAggregateOptionsSchema, DataEngineAggregateRequestSchema, DataEngineCountOptionsSchema, DataEngineCountRequestSchema, DataEngineDeleteOptionsSchema, DataEngineDeleteRequestSchema, DataEngineExecuteRequestSchema, DataEngineFilterSchema, DataEngineFindOneRequestSchema, DataEngineFindRequestSchema, DataEngineInsertOptionsSchema, DataEngineInsertRequestSchema, DataEngineQueryOptionsSchema, DataEngineRequestSchema, DataEngineSortSchema, DataEngineUpdateOptionsSchema, DataEngineUpdateRequestSchema, DataEngineVectorFindRequestSchema, DroppedFieldsEventSchema, EngineAggregateOptionsSchema, EngineCountOptionsSchema, EngineDeleteOptionsSchema, EngineQueryOptionsSchema, EngineUpdateOptionsSchema, QueryPageWindowSchema, QueryPageWindowParamSchema, QueryTransportParamsSchema, QueryWithTransportSchema, ValidationAdvisoryEventSchema } from '@objectstack/spec/data'; +import type { BaseEngineOptions, DataEngineAggregateOptions, DataEngineCountOptions, DataEngineDeleteOptions, DataEngineExecuteRequest, DataEngineFilter, DataEngineInsertOptions, DataEngineInsertRequest, DataEngineQueryOptions, DataEngineRequest, DataEngineSort, DataEngineUpdateOptions, DataEngineVectorFindRequest, DroppedFieldsEvent, EngineAggregateOptions, EngineCountOptions, EngineDeleteOptions, EngineQueryOptions, EngineUpdateOptions, QueryPageWindow, QueryPageWindowParam, QueryTransportParams, QueryWithTransport, ValidationAdvisoryEvent } from '@objectstack/spec/data'; // Validate data const result = BaseEngineOptionsSchema.parse(data); @@ -1258,6 +1258,40 @@ QueryAST-aligned options for DataEngine.update operations | **traceId** | `string` | optional | | +--- + +## QueryPageWindow + +Page window — a non-negative integer (0 asks for the total alone) + +**Type:** `integer` + + +--- + +## QueryPageWindowParam + +Page window — a non-negative integer, or the digits a querystring carries it as + +### Union Options + +This schema accepts one of the following structures: + +#### Option 1 + +Page window — a non-negative integer (0 asks for the total alone) + +Type: `integer` + +--- + +#### Option 2 + +Type: `string` + +--- + + --- ## QueryTransportParams @@ -1270,8 +1304,8 @@ Transport spellings of the QueryAST slots — folded to canonical keys at the bo | :--- | :--- | :--- | :--- | | **$filter** | `[string, string, any] \| [string, string] \| [string, object, ...object[]] \| object[] \| Record \| any \| null` | optional | Transport spelling of `where` (OData `$filter`) | | **filters** | `[string, string, any] \| [string, string] \| [string, object, ...object[]] \| object[] \| Record \| any \| null` | optional | Transport spelling of `where` (plural of `filter`) | -| **$top** | `number \| string \| null` | optional | Transport spelling of `limit` (OData `$top`) — a number, or the digits a querystring carries it as | -| **$skip** | `number \| string \| null` | optional | Transport spelling of `offset` (OData `$skip`) — a number, or the digits a querystring carries it as | +| **$top** | `integer \| string \| null` | optional | Transport spelling of `limit` (OData `$top`) — a non-negative integer, or the digits a querystring carries it as | +| **$skip** | `integer \| string \| null` | optional | Transport spelling of `offset` (OData `$skip`) — a non-negative integer, or the digits a querystring carries it as | | **$orderby** | `Record> \| Record \| { field: string; order?: Enum<'asc' \| 'desc'> }[] \| null` | optional | Transport spelling of `orderBy` (OData `$orderby`) | | **$select** | `string \| string[] \| null` | optional | Transport spelling of `fields` (OData `$select`) | | **$expand** | `string \| string[] \| Record \| null` | optional | Transport spelling of `expand` (OData `$expand`) | @@ -1282,7 +1316,7 @@ Transport spellings of the QueryAST slots — folded to canonical keys at the bo | **filter** | `[string, string, any] \| [string, string] \| [string, object, ...object[]] \| object[] \| Record \| any \| null` | optional | Transport spelling of `where` | | **select** | `string \| string[] \| null` | optional | Transport spelling of `fields` | | **sort** | `Record> \| Record \| { field: string; order?: Enum<'asc' \| 'desc'> }[] \| null` | optional | Transport spelling of `orderBy` | -| **skip** | `number \| string \| null` | optional | Transport spelling of `offset` | +| **skip** | `integer \| string \| null` | optional | Transport spelling of `offset` — a non-negative integer, or the digits a querystring carries it as | | **populate** | `string \| string[] \| null` | optional | Transport spelling of `expand` | ### Nested Shape: `QueryTransportParams.$expand[string]` @@ -1335,9 +1369,9 @@ Transport spellings of the QueryAST slots — folded to canonical keys at the bo | **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional | Full-text search — the query text (canonical, ADR-0061 D1), or a structured FullTextSearch configuration | | **searchFields** | `string[]` | optional | Narrow the search to these fields (server-intersected with the allowed searchable set — can only narrow, never widen; ADR-0061 D1) | | **orderBy** | `{ field: string; order?: Enum<'asc' \| 'desc'> }[]` | optional | Sorting instructions (ORDER BY) | -| **limit** | `number` | optional | Max records to return (LIMIT) | -| **offset** | `number` | optional | Records to skip (OFFSET) | -| **top** | `number` | optional | Alias for limit (OData compatibility) | +| **limit** | `integer` | optional | Max records to return (LIMIT) | +| **offset** | `integer` | optional | Records to skip (OFFSET) | +| **top** | `integer` | optional | Alias for limit (OData compatibility) | | **cursor** | `never` | optional | [REMOVED] `query.cursor` was removed in @objectstack/spec 17 (ADR-0049) — no driver ever implemented keyset pagination, so the cursor was accepted and ignored and every page came back identical (a caller looping "until hasMore is false" never terminates). Delete the key; `QueryBuilder.cursor()` was removed with it. Express the keyset as an ordinary `where` predicate on your sort key — `where: { created_at: { $gt: last.created_at } }` with the matching `orderBy` — which every driver executes with canonicalised comparands. A first-class cursor, if ever built, will be a response-minted opaque token, not this caller-built record. | | **joins** | `never` | optional | [REMOVED] `query.joins` was removed in @objectstack/spec 17 (ADR-0049) — no engine or driver ever read it: a query carrying `joins` behaved exactly as if the key were absent, while its name squatted on the reserved REST parameter set. Delete the key. Related records are read through `expand` — `expand: { owner_id: { object: 'user', fields: ['name'] } }` — which the engine resolves via batch $in queries, and whose nested query selects the related record's own columns. Keep the foreign key in your own projection (`fields: ['title', 'owner_id']`): the relation is carried by that column, so projecting it away leaves expansion nothing to resolve. A dotted `fields` path is NOT a replacement — no driver ever resolved one and the ingress refuses it (`400 INVALID_FIELD`). | | **aggregations** | `{ function: Enum<'count' \| 'sum' \| 'avg' \| 'min' \| 'max' \| 'count_distinct'>; field?: string; alias: string; filter?: any }[]` | optional | Aggregation functions | @@ -1348,8 +1382,8 @@ Transport spellings of the QueryAST slots — folded to canonical keys at the bo | **expand** | `Record` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. | | **$filter** | `[string, string, any] \| [string, string] \| [string, object, ...object[]] \| object[] \| Record \| any \| null` | optional | Transport spelling of `where` (OData `$filter`) | | **filters** | `[string, string, any] \| [string, string] \| [string, object, ...object[]] \| object[] \| Record \| any \| null` | optional | Transport spelling of `where` (plural of `filter`) | -| **$top** | `number \| string \| null` | optional | Transport spelling of `limit` (OData `$top`) — a number, or the digits a querystring carries it as | -| **$skip** | `number \| string \| null` | optional | Transport spelling of `offset` (OData `$skip`) — a number, or the digits a querystring carries it as | +| **$top** | `integer \| string \| null` | optional | Transport spelling of `limit` (OData `$top`) — a non-negative integer, or the digits a querystring carries it as | +| **$skip** | `integer \| string \| null` | optional | Transport spelling of `offset` (OData `$skip`) — a non-negative integer, or the digits a querystring carries it as | | **$orderby** | `Record> \| Record \| { field: string; order?: Enum<'asc' \| 'desc'> }[] \| null` | optional | Transport spelling of `orderBy` (OData `$orderby`) | | **$select** | `string \| string[] \| null` | optional | Transport spelling of `fields` (OData `$select`) | | **$expand** | `string \| string[] \| Record \| null` | optional | Transport spelling of `expand` (OData `$expand`) | @@ -1360,7 +1394,7 @@ Transport spellings of the QueryAST slots — folded to canonical keys at the bo | **filter** | `[string, string, any] \| [string, string] \| [string, object, ...object[]] \| object[] \| Record \| any \| null` | optional | Transport spelling of `where` | | **select** | `string \| string[] \| null` | optional | Transport spelling of `fields` | | **sort** | `Record> \| Record \| { field: string; order?: Enum<'asc' \| 'desc'> }[] \| null` | optional | Transport spelling of `orderBy` | -| **skip** | `number \| string \| null` | optional | Transport spelling of `offset` | +| **skip** | `integer \| string \| null` | optional | Transport spelling of `offset` — a non-negative integer, or the digits a querystring carries it as | | **populate** | `string \| string[] \| null` | optional | Transport spelling of `expand` | ### Nested Shape: `QueryWithTransport.search` diff --git a/content/docs/references/index.mdx b/content/docs/references/index.mdx index 0de63b3cc5..78fefed4c9 100644 --- a/content/docs/references/index.mdx +++ b/content/docs/references/index.mdx @@ -1,7 +1,7 @@ --- title: Protocol reference — every schema by module navTitle: Protocol Reference -description: Every schema published by @objectstack/spec — 1542 schemas across 14 protocol modules +description: Every schema published by @objectstack/spec — 1544 schemas across 14 protocol modules --- {/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} @@ -23,7 +23,7 @@ counts are sums of the rows they head. Regenerate with | [AI Protocol](/docs/references/ai) | 12 | 68 | Agents, tools, skills, RAG and knowledge sources, model registry, conversations. | | [API Protocol](/docs/references/api) | 32 | 438 | REST contracts, endpoints, routing, realtime, batch, discovery. | | [Automation Protocol](/docs/references/automation) | 13 | 71 | Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execution records. | -| [Data Protocol](/docs/references/data) | 29 | 176 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. | +| [Data Protocol](/docs/references/data) | 29 | 178 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. | | [Identity Protocol](/docs/references/identity) | 5 | 27 | Users and accounts, organizations, positions, SCIM provisioning. | | [Integration Protocol](/docs/references/integration) | 1 | 12 | The single connector protocol (ADR-0097) — catalog descriptors and provider-bound instances. | | [Kernel Protocol](/docs/references/kernel) | 30 | 161 | Plugin lifecycle and manifests, capabilities and security, metadata loading, service registry. | @@ -34,7 +34,7 @@ counts are sums of the rows they head. Regenerate with | [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. | | [System Protocol](/docs/references/system) | 36 | 285 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. | | [UI Protocol](/docs/references/ui) | 16 | 169 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. | -| **Total** | **197** | **1542** | 14 protocol modules | +| **Total** | **197** | **1544** | 14 protocol modules | --- @@ -130,7 +130,7 @@ Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execu ## Data Protocol -**Source:** `packages/spec/src/data/` · **Import:** `@objectstack/spec/data` · **29 pages, 176 schemas** +**Source:** `packages/spec/src/data/` · **Import:** `@objectstack/spec/data` · **29 pages, 178 schemas** Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. @@ -138,7 +138,7 @@ Objects, fields, queries, filters, datasources and drivers — the ObjectQL laye | :--- | :--- | | [`analytics.zod.ts`](/docs/references/data/analytics) | `AggregationMetricType`, `AnalyticsDateRange`, `AnalyticsDateRangePreset`, `AnalyticsQuery`, `Cube`, `CubeJoin`, `Dimension`, `DimensionType`, `Metric`, `TimeUpdateInterval` | | [`context-tokens.zod.ts`](/docs/references/data/context-tokens) | `ContextToken`, `ContextTokenPlaceholder` | -| [`data-engine.zod.ts`](/docs/references/data/data-engine) | `BaseEngineOptions`, `DataEngineAggregateOptions`, `DataEngineAggregateRequest`, `DataEngineCountOptions`, `DataEngineCountRequest`, `DataEngineDeleteOptions`, `DataEngineDeleteRequest`, `DataEngineExecuteRequest`, `DataEngineFilter`, `DataEngineFindOneRequest`, `DataEngineFindRequest`, `DataEngineInsertOptions`, `DataEngineInsertRequest`, `DataEngineQueryOptions`, `DataEngineRequest`, `DataEngineSort`, `DataEngineUpdateOptions`, `DataEngineUpdateRequest`, `DataEngineVectorFindRequest`, `DroppedFieldsEvent`, `EngineAggregateOptions`, `EngineCountOptions`, `EngineDeleteOptions`, `EngineQueryOptions`, `EngineUpdateOptions`, `QueryTransportParams`, `QueryWithTransport`, `ValidationAdvisoryEvent` | +| [`data-engine.zod.ts`](/docs/references/data/data-engine) | `BaseEngineOptions`, `DataEngineAggregateOptions`, `DataEngineAggregateRequest`, `DataEngineCountOptions`, `DataEngineCountRequest`, `DataEngineDeleteOptions`, `DataEngineDeleteRequest`, `DataEngineExecuteRequest`, `DataEngineFilter`, `DataEngineFindOneRequest`, `DataEngineFindRequest`, `DataEngineInsertOptions`, `DataEngineInsertRequest`, `DataEngineQueryOptions`, `DataEngineRequest`, `DataEngineSort`, `DataEngineUpdateOptions`, `DataEngineUpdateRequest`, `DataEngineVectorFindRequest`, `DroppedFieldsEvent`, `EngineAggregateOptions`, `EngineCountOptions`, `EngineDeleteOptions`, `EngineQueryOptions`, `EngineUpdateOptions`, `QueryPageWindow`, `QueryPageWindowParam`, `QueryTransportParams`, `QueryWithTransport`, `ValidationAdvisoryEvent` | | [`datasource.zod.ts`](/docs/references/data/datasource) | `Datasource`, `DriverDefinition`, `DriverType`, `ExternalDatasourceSettings`, `SchemaMode` | | [`date-macros.zod.ts`](/docs/references/data/date-macros) | `DateMacroPlaceholder`, `DateMacroToken` | | [`driver.zod.ts`](/docs/references/data/driver) | `DriverCapabilities`, `DriverConfig`, `DriverOptions`, `PoolConfig` | diff --git a/packages/metadata-protocol/src/protocol.page-window-declared.test.ts b/packages/metadata-protocol/src/protocol.page-window-declared.test.ts new file mode 100644 index 0000000000..29ee5a9d29 --- /dev/null +++ b/packages/metadata-protocol/src/protocol.page-window-declared.test.ts @@ -0,0 +1,163 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * #25269 — the published schemas that describe a list door's page window + * declare the accept set `findData` enforces (#25229), value for value. + * + * Before this, `FindDataRequestSchema` (the `POST /data/:object/query` body, + * through `QueryWithTransportSchema`) declared `limit` / `offset` / `top` as any + * number and `$top` / `$skip` / `skip` as any number or string, and + * `HttpFindQueryParamsSchema` (the `GET /data/:object` querystring) declared + * `top` / `skip` as `z.coerce.number()` — so `-1`, `1.5` and a blank `?top=` + * validated against the published schema and were then refused by the door. + * + * ## The oracle is the door, not the spec + * + * Each value is first handed to the real `findData` normalizer (a recording + * engine behind it): accepted iff the engine was reached, and the window it was + * handed is the expected parse output. Every schema row is then held to that + * verdict. The normalizer's own refusal table (`protocol.page-window.test.ts`) + * is the seed of the value list below, widened at the edges the bound has + * (`-0`, `Number.MAX_SAFE_INTEGER` and one past it, `Infinity`, the `Number` + * spellings `'1e2'` / `'0x10'` / `' 5 '`). + * + * A member that declares only the NUMBER arm (the canonical `limit` / `offset` + * / `top` of a JSON body) refuses every string — it did before this change + * too, so the string arm there is asserted refused, never widened. + */ + +import { describe, it, expect, vi } from 'vitest'; +import { FindDataRequestSchema, HttpFindQueryParamsSchema } from '@objectstack/spec/api'; +import { QueryPageWindowParamSchema, QueryPageWindowSchema, QueryWithTransportSchema } from '@objectstack/spec/data'; +import { ObjectStackProtocolImplementation } from './protocol.js'; + +const SCHEMA = { + name: 'invoice', + nameField: 'name', + fields: { name: { name: 'name', type: 'text' } }, +}; + +type Verdict = { accepted: true; window: number } | { accepted: false }; + +/** What `findData` does with `value` in the canonical `limit` slot. */ +async function doorVerdict(value: unknown): Promise { + const find = vi.fn(async () => [] as unknown[]); + const engine = { + registry: { getObject: (n: string) => (n === 'invoice' ? SCHEMA : undefined) }, + find, + aggregate: vi.fn(async () => [] as unknown[]), + count: vi.fn(async () => 0), + }; + const p = new ObjectStackProtocolImplementation(engine as any); + try { + await p.findData({ object: 'invoice', query: { limit: value } } as never); + } catch (e) { + expect((e as { code?: string }).code, `${String(value)}: refused with another code`).toBe('INVALID_REQUEST'); + expect(find).not.toHaveBeenCalled(); + return { accepted: false }; + } + expect(find).toHaveBeenCalledTimes(1); + return { accepted: true, window: ((find.mock.calls[0] as unknown[])[1] as { limit: number }).limit }; +} + +const NUMBERS: readonly number[] = [ + 0, -0, 1, 5, 50, 5000, Number.MAX_SAFE_INTEGER, + -1, -5, 0.5, 1.5, 100.5, -0.5, Number.MAX_SAFE_INTEGER + 1, + Number.NaN, Number.POSITIVE_INFINITY, Number.NEGATIVE_INFINITY, +]; + +const STRINGS: readonly string[] = [ + '0', '5', '50', ' 5 ', '1e2', '0x10', '9007199254740991', + '', ' ', 'abc', 'ten', '-1', '-5', '1.5', '100.5', 'Infinity', 'NaN', '9007199254740992', +]; + +const OTHERS: readonly unknown[] = [true, false, {}]; + +/** + * schema · spelling · the arms the member declares · parse `value` through it + * and return the window it produced (or `undefined` for a refusal). + */ +type Row = readonly [string, string, 'number' | 'both', (value: unknown) => number | undefined]; + +const viaQuery = (schema: typeof QueryWithTransportSchema, spelling: string) => (value: unknown) => { + const parsed = schema.safeParse({ object: 'invoice', [spelling]: value }); + return parsed.success ? (parsed.data as { limit?: number; offset?: number }).limit + ?? (parsed.data as { offset?: number }).offset : undefined; +}; + +const viaFindRequest = (spelling: string) => (value: unknown) => { + const parsed = FindDataRequestSchema.safeParse({ object: 'invoice', query: { object: 'invoice', [spelling]: value } }); + if (!parsed.success) return undefined; + const query = parsed.data.query as { limit?: number; offset?: number }; + return query.limit ?? query.offset; +}; + +const viaHttpParams = (spelling: 'top' | 'skip') => (value: unknown) => { + const parsed = HttpFindQueryParamsSchema.safeParse({ [spelling]: value }); + return parsed.success ? parsed.data[spelling] : undefined; +}; + +const ROWS: readonly Row[] = [ + ['FindDataRequest', 'limit', 'number', viaFindRequest('limit')], + ['FindDataRequest', 'offset', 'number', viaFindRequest('offset')], + ['FindDataRequest', 'top', 'number', viaFindRequest('top')], + ['FindDataRequest', '$top', 'both', viaFindRequest('$top')], + ['FindDataRequest', '$skip', 'both', viaFindRequest('$skip')], + ['FindDataRequest', 'skip', 'both', viaFindRequest('skip')], + ['QueryWithTransport', 'limit', 'number', viaQuery(QueryWithTransportSchema, 'limit')], + ['QueryWithTransport', '$top', 'both', viaQuery(QueryWithTransportSchema, '$top')], + ['QueryWithTransport', '$skip', 'both', viaQuery(QueryWithTransportSchema, '$skip')], + ['HttpFindQueryParams', 'top', 'both', viaHttpParams('top')], + ['HttpFindQueryParams', 'skip', 'both', viaHttpParams('skip')], + ['QueryPageWindowParam', '(value)', 'both', (v) => { + const parsed = QueryPageWindowParamSchema.safeParse(v); + return parsed.success ? parsed.data : undefined; + }], + ['QueryPageWindow', '(value)', 'number', (v) => { + const parsed = QueryPageWindowSchema.safeParse(v); + return parsed.success ? parsed.data : undefined; + }], +]; + +describe('#25269 — every published window schema accepts exactly what the door accepts', () => { + it('the value list covers both verdicts on both arms (no vacuous table)', async () => { + const verdicts = await Promise.all([...NUMBERS, ...STRINGS].map(doorVerdict)); + const numberVerdicts = verdicts.slice(0, NUMBERS.length); + const stringVerdicts = verdicts.slice(NUMBERS.length); + expect(numberVerdicts.some((v) => v.accepted)).toBe(true); + expect(numberVerdicts.some((v) => !v.accepted)).toBe(true); + expect(stringVerdicts.some((v) => v.accepted)).toBe(true); + expect(stringVerdicts.some((v) => !v.accepted)).toBe(true); + }); + + describe.each(ROWS)('%s · %s', (_schema, _spelling, arms, parse) => { + it.each(NUMBERS.map((v) => [String(v), v] as const))('number %s — the door\'s verdict and window', async (_label, value) => { + const verdict = await doorVerdict(value); + const window = parse(value); + if (verdict.accepted) { + expect(window, 'the door accepts it; the schema must too').toBe(verdict.window); + } else { + expect(window, 'the door refuses it; the schema must too').toBeUndefined(); + } + }); + + it.each(STRINGS.map((v) => [JSON.stringify(v), v] as const))('string %s — the door\'s verdict and window', async (_label, value) => { + const verdict = await doorVerdict(value); + const window = parse(value); + if (arms === 'number') { + // A JSON body's canonical member declares the number arm only, as + // it always has: no string is a window there. + expect(window).toBeUndefined(); + } else if (verdict.accepted) { + expect(window, 'the door accepts it; the schema must too').toBe(verdict.window); + } else { + expect(window, 'the door refuses it; the schema must too').toBeUndefined(); + } + }); + + it.each(OTHERS.map((v) => [JSON.stringify(v), v] as const))('%s is neither a count nor its digits — refused by both', async (_label, value) => { + expect((await doorVerdict(value)).accepted).toBe(false); + expect(parse(value)).toBeUndefined(); + }); + }); +}); diff --git a/packages/metadata-protocol/src/protocol.ts b/packages/metadata-protocol/src/protocol.ts index 70662a24bd..d065a00722 100644 --- a/packages/metadata-protocol/src/protocol.ts +++ b/packages/metadata-protocol/src/protocol.ts @@ -172,6 +172,7 @@ import { classifyDottedFilterHead, foldQueryAliasSlots, QUERY_TRANSPORT_ALIAS_SLOTS, QUERY_TRANSPORT_DOLLAR_ALIASES, QUERY_TRANSPORT_DOLLAR_PARAMS, + QueryPageWindowParamSchema, type QueryAliasConflict, type DroppedFieldsEvent, type QueryAST, type EngineQueryOptionsParsed, type ValidationAdvisoryEvent, @@ -4777,14 +4778,15 @@ function repeatedQueryParamError(param: string, count: number): Error { * Only the two arms a door can deliver are read: a number (a JSON body) and a * string (a querystring). Anything else — `true`, which `Number` would quietly * make `1` — is refused like any other value that is not a count. + * + * [#25269] The accept set is the spec's `QueryPageWindowParamSchema`, parsed + * here rather than restated: the same declaration is what `FindDataRequestSchema` + * and `HttpFindQueryParamsSchema` publish for these parameters, so a client + * generated from them is told exactly what this check lets through. */ function pageWindowValue(value: unknown, param: string): number { - const n = typeof value === 'number' - ? value - : typeof value === 'string' && value.trim() !== '' - ? Number(value) - : Number.NaN; - if (Number.isSafeInteger(n) && n >= 0) return n; + const window = QueryPageWindowParamSchema.safeParse(value); + if (window.success) return window.data; // `String` for a number: `JSON.stringify(Infinity)` is `null`. const shown = typeof value === 'number' ? String(value) : JSON.stringify(value) ?? String(value); const err: any = new Error( diff --git a/packages/rest/src/data-page-window-door.test.ts b/packages/rest/src/data-page-window-door.test.ts index 292e0a75b1..cb731c2337 100644 --- a/packages/rest/src/data-page-window-door.test.ts +++ b/packages/rest/src/data-page-window-door.test.ts @@ -29,6 +29,14 @@ * is the door-level half: the status and `code` each door answers, the * spelling it names, that the driver was never asked, and the controls paging * exactly as before. + * + * [#25269] The `POST` body's published schema (`FindDataRequestSchema`) now + * declares that same window, so the `POST` door refuses a malformed one where + * it refuses every other malformed body — at the schema, `400 + * VALIDATION_FAILED`, `fields[0]` naming `query.` — exactly as it already + * refused `{"limit":"ten"}`. Still a 400 before any read; the code is the + * schema door's rather than the normalizer's, because the schema now says what + * the normalizer enforces. */ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; @@ -152,13 +160,13 @@ describe('[#25229] the page window on both list doors — sqlite', () => { expect(reads.n - before, 'no read of the object — every refusal precedes the driver').toBe(0); }); - it('POST: 400 INVALID_REQUEST naming the body key — no read', async () => { + it('POST: 400 VALIDATION_FAILED naming the body key — the declared schema refuses it, no read', async () => { const before = reads.n; for (const [name, body, param] of REFUSED_POST) { const res = await query(body); expect(res.status, `${name}: ${JSON.stringify(res.body)}`).toBe(400); - expect(res.body.code, name).toBe('INVALID_REQUEST'); - expect(res.body.error, name).toContain(`Query parameter '${param}' must be a non-negative integer`); + expect(res.body.code, name).toBe('VALIDATION_FAILED'); + expect(res.body.fields?.[0]?.field, name).toBe(`query.${param}`); } expect(reads.n - before, 'no read of the object — every refusal precedes the driver').toBe(0); }); diff --git a/packages/spec/api-surface/data.json b/packages/spec/api-surface/data.json index 574527ace1..672059599d 100644 --- a/packages/spec/api-surface/data.json +++ b/packages/spec/api-surface/data.json @@ -598,6 +598,12 @@ "QueryFilter (type)", "QueryFilterSchema (const)", "QueryInput (type)", + "QueryPageWindow (type)", + "QueryPageWindowParam (type)", + "QueryPageWindowParamParsed (type)", + "QueryPageWindowParamSchema (const)", + "QueryPageWindowParsed (type)", + "QueryPageWindowSchema (const)", "QuerySchema (const)", "QueryTransportParams (type)", "QueryTransportParamsParsed (type)", diff --git a/packages/spec/declaration-map/data.json b/packages/spec/declaration-map/data.json index 6e0b150641..a1bd9cd274 100644 --- a/packages/spec/declaration-map/data.json +++ b/packages/spec/declaration-map/data.json @@ -251,6 +251,10 @@ "PostgresConfigSchema": "data/PostgresConfig", "QueryFilter": "data/QueryFilter", "QueryFilterSchema": "data/QueryFilter", + "QueryPageWindow": "data/QueryPageWindow", + "QueryPageWindowParam": "data/QueryPageWindowParam", + "QueryPageWindowParamSchema": "data/QueryPageWindowParam", + "QueryPageWindowSchema": "data/QueryPageWindow", "QuerySchema": "data/Query", "QueryTransportParams": "data/QueryTransportParams", "QueryTransportParamsSchema": "data/QueryTransportParams", diff --git a/packages/spec/dropped-refinements.baseline.json b/packages/spec/dropped-refinements.baseline.json index 8fbb73c721..35e94854a3 100644 --- a/packages/spec/dropped-refinements.baseline.json +++ b/packages/spec/dropped-refinements.baseline.json @@ -2,8 +2,8 @@ "description": "Shrink-only ledger of every PUBLISHED JSON Schema that is STILL WIDER than the Zod type it was generated from, because a rule written as `.refine()` reaches the runtime and not the file (#18670). `z.toJSONSchema()` has no arm for a `custom` check: a plain record, the same record with a `.refine()`, and the same record with an ABORTING `.refine()` all project byte-identically (measured on zod 4.4.3, the version packages/spec resolves). So a document one of these files ACCEPTS can still be refused at parse time, and an author -- or an AI -- validating against packages/spec/json-schema/** finds out a release later. Each `sites` path is a position under that schema at which a refinement is dropped; the same paths are written onto the artifact itself as `x-dropped-refinements`. Item 2 closed the first patterns: a refinement DECLARED through the closed list in src/shared/refinement-projection.ts is emitted into the published file, reads `projected` rather than `dropped`, and its row LEAVES this ledger in the same PR -- which is why the ledger shrinks and never grows on a repair. Every refinement outside that closed list stays here, and adding an arm to the list is a public-contract decision, not a refactor. Hand-edited on purpose and with no `gen:` script: a generator would let a new gap be admitted by running a command instead of by a decision, which is the silence this ledger exists to end. Adding, removing or moving a site fails packages/spec/scripts/build-schemas.ts until the line moves with it, and the failure prints the corrected entry in full. ⛔ Do not delete or weaken a refinement to shorten this file -- the runtime rule is correct; it is the projection that is silent, and the remedy is to teach the closed list a NAMED pattern, never to drop the rule.", "measured": { "zod": "4.4.3", - "publishedSchemasWithDroppedRefinements": 240, - "droppedRefinementSites": 811, + "publishedSchemasWithDroppedRefinements": 242, + "droppedRefinementSites": 813, "refinementSitesThatDidProject": 369, "refinementSitesWithNoJsonFormToCompare": 0 }, @@ -281,6 +281,11 @@ "list.timeline.groupByField" ] }, + "api/HttpFindQueryParams": { + "sites": [ + "top.options[1].in" + ] + }, "api/ImportRequest": { "sites": [ "" @@ -947,6 +952,11 @@ "where.lazy" ] }, + "data/QueryPageWindowParam": { + "sites": [ + "options[1].in" + ] + }, "data/QueryTransportParams": { "sites": [ "$filter.options[0].lazy.options[0].items[0]", diff --git a/packages/spec/export-origins/data.json b/packages/spec/export-origins/data.json index aa38eae691..aa3e5350c1 100644 --- a/packages/spec/export-origins/data.json +++ b/packages/spec/export-origins/data.json @@ -587,6 +587,12 @@ "QueryFilter": "src/data/filter.zod.ts#QueryFilter (type)", "QueryFilterSchema": "src/data/filter.zod.ts#QueryFilterSchema (const)", "QueryInput": "src/data/query.zod.ts#QueryInput (type)", + "QueryPageWindow": "src/data/data-engine.zod.ts#QueryPageWindow (type)", + "QueryPageWindowParam": "src/data/data-engine.zod.ts#QueryPageWindowParam (type)", + "QueryPageWindowParamParsed": "src/data/data-engine.zod.ts#QueryPageWindowParamParsed (type)", + "QueryPageWindowParamSchema": "src/data/data-engine.zod.ts#QueryPageWindowParamSchema (const)", + "QueryPageWindowParsed": "src/data/data-engine.zod.ts#QueryPageWindowParsed (type)", + "QueryPageWindowSchema": "src/data/data-engine.zod.ts#QueryPageWindowSchema (const)", "QuerySchema": "src/data/query.zod.ts#QuerySchema (const)", "QueryTransportParams": "src/data/data-engine.zod.ts#QueryTransportParams (type)", "QueryTransportParamsParsed": "src/data/data-engine.zod.ts#QueryTransportParamsParsed (type)", diff --git a/packages/spec/json-schema.manifest/data.json b/packages/spec/json-schema.manifest/data.json index 0e6dc30206..008b39787c 100644 --- a/packages/spec/json-schema.manifest/data.json +++ b/packages/spec/json-schema.manifest/data.json @@ -138,6 +138,8 @@ "data/PostgresConfig", "data/Query", "data/QueryFilter", + "data/QueryPageWindow", + "data/QueryPageWindowParam", "data/QueryTransportParams", "data/QueryWithTransport", "data/RangeOperator", diff --git a/packages/spec/src/api/protocol.test.ts b/packages/spec/src/api/protocol.test.ts index 296e226e2a..8bf5e97197 100644 --- a/packages/spec/src/api/protocol.test.ts +++ b/packages/spec/src/api/protocol.test.ts @@ -664,6 +664,24 @@ describe('HttpFindQueryParamsSchema', () => { } }); + // [#25269] The page window the list door enforces — not `z.coerce.number()`, + // which admitted `-1`, `1.5` and a blank `?top=` (as `0`). + it.each([['top', '-1'], ['top', '1.5'], ['top', ''], ['top', ' '], ['top', 'abc'], ['skip', '-5'], ['skip', 0.5], ['skip', -1]])( + 'refuses %s=%j — not a page window', + (key, value) => { + expect(HttpFindQueryParamsSchema.safeParse({ [key]: value }).success).toBe(false); + }, + ); + + it('keeps 0 and the digits a querystring carries as page windows', () => { + const result = HttpFindQueryParamsSchema.safeParse({ top: '0', skip: ' 5 ' }); + expect(result.success).toBe(true); + if (result.success) { + expect(result.data.top).toBe(0); + expect(result.data.skip).toBe(5); + } + }); + it('should accept sort, orderBy, expand, search, count', () => { const result = HttpFindQueryParamsSchema.safeParse({ sort: 'name asc,created_at desc', diff --git a/packages/spec/src/api/protocol.zod.ts b/packages/spec/src/api/protocol.zod.ts index de5dd95f32..a9e52b2a7b 100644 --- a/packages/spec/src/api/protocol.zod.ts +++ b/packages/spec/src/api/protocol.zod.ts @@ -18,7 +18,7 @@ import { retiredKey } from '../shared/retired-key'; // through it, so `api/UpdateAiConversationRequest.json` states it. import { requiredOneOf } from '../shared/refinement-projection'; import { MetadataItemNameSchema } from '../shared/identifiers.zod'; -import { DroppedFieldsEventSchema, QueryWithTransportSchema, ValidationAdvisoryEventSchema } from '../data/data-engine.zod'; +import { DroppedFieldsEventSchema, QueryPageWindowParamSchema, QueryWithTransportSchema, ValidationAdvisoryEventSchema } from '../data/data-engine.zod'; import { AnalyticsQueryRequestSchema, AnalyticsResultResponseSchema, @@ -2372,8 +2372,11 @@ export const HttpFindQueryParamsSchema = lazySchema(() => z.object({ select: z.string().optional().describe('Comma-separated list of fields to retrieve.'), sort: z.string().optional().describe('Sort expression (e.g. "name asc,created_at desc" or "-created_at").'), orderBy: z.string().optional().describe('Alias for sort (OData compatibility).'), - top: z.coerce.number().optional().describe('Max records to return (limit).'), - skip: z.coerce.number().optional().describe('Records to skip (offset).'), + // [#25269] The page window the list door enforces, not `z.coerce.number()`, + // which admitted `-1`, `1.5` and a blank `?top=` (as `0`) that the door + // refuses with a 400. + top: QueryPageWindowParamSchema.optional().describe('Max records to return (limit) — a non-negative integer.'), + skip: QueryPageWindowParamSchema.optional().describe('Records to skip (offset) — a non-negative integer.'), expand: z.string().optional().describe( 'Comma-separated list of lookup/master_detail field names to expand. ' + 'Resolved to populate array and passed to the engine for batch $in expansion.' diff --git a/packages/spec/src/data/data-engine.zod.ts b/packages/spec/src/data/data-engine.zod.ts index ece3d06ebf..544987d8f5 100644 --- a/packages/spec/src/data/data-engine.zod.ts +++ b/packages/spec/src/data/data-engine.zod.ts @@ -907,6 +907,55 @@ const TransportFilterValueSchema = lazySchema(() => z.union([FilterArraySchema, */ const TransportCountValueSchema = lazySchema(() => z.union([z.boolean(), z.enum(['true', 'false'])])); +/** + * [#25269] The page window — `limit` / `offset` and every spelling that folds + * onto them (`top`, `$top`, `skip`, `$skip`) — as the Data API list doors + * enforce it: a non-negative safe integer. `0` is a window (`$top=0` asks for + * the total alone); a fraction, a negative, `NaN` or `Infinity` is not. + * + * ONE declaration of the bound, read by both sides: the door schemas below + * declare it, and `@objectstack/metadata-protocol`'s `findData` normalizer — + * the one statement every list door reaches — judges each window by + * {@link QueryPageWindowParamSchema}, built on this. A second spelling of the + * bound in either place is how the schemas came to admit `limit: -1` and + * `offset: 1.5` that the door refuses. + * + * ⛔ Not declared on `QuerySchema` itself. That schema is also the driver + * contract and the recursive `expand` value, and neither is judged by the + * normalizer: narrowing it would refuse at a schema windows no server refuses. + */ +export const QueryPageWindowSchema = lazySchema(() => z.number().int().nonnegative() + .describe('Page window — a non-negative integer (0 asks for the total alone)')); + +/** + * A page window as a transport carries it: the count itself (a JSON body), or + * the text a querystring carries it as (`?$top=50` arrives as `'50'`), which + * parses to the count. A blank string is refused, not read as absent: an + * absent window means every row, so reading `?$top=` that way would widen the + * read the caller bounded. + * + * This is the whole accept set of the `findData` window check, not a copy of + * it: the normalizer parses every window through this schema. The string arm + * converts with `Number`, exactly as the door always has, so `' 5 '` and + * `'1e2'` are counts here because they are counts there. + */ +export const QueryPageWindowParamSchema = lazySchema(() => z.union([ + QueryPageWindowSchema, + z.string() + .refine( + (text) => text.trim() !== '' && QueryPageWindowSchema.safeParse(Number(text)).success, + 'Page window must be a non-negative integer, written as digits', + ) + .transform((text) => Number(text)), +]).describe('Page window — a non-negative integer, or the digits a querystring carries it as')); + +export type QueryPageWindow = z.input; +/** Post-parse shape of {@link QueryPageWindow} — the same count; named so a parse result has its ADR-0122 name. */ +export type QueryPageWindowParsed = z.infer; +export type QueryPageWindowParam = z.input; +/** Post-parse shape of {@link QueryPageWindowParam} — the count, the querystring digits converted (ADR-0122). */ +export type QueryPageWindowParamParsed = z.infer; + /** * The transport query parameters — every spelling the two tables above name * that is not itself a QueryAST key: the `$`-prefixed forms, the plural @@ -971,11 +1020,11 @@ export const QueryTransportParamsSchema = lazySchema(() => z.object({ filters: TransportFilterValueSchema.nullable().optional() .describe('Transport spelling of `where` (plural of `filter`)'), /** OData spelling of `limit` (two hops: `$top` to `top` to `limit`). */ - $top: z.union([z.number(), z.string()]).nullable().optional() - .describe('Transport spelling of `limit` (OData `$top`) — a number, or the digits a querystring carries it as'), + $top: z.union([QueryPageWindowSchema, z.string()]).nullable().optional() + .describe('Transport spelling of `limit` (OData `$top`) — a non-negative integer, or the digits a querystring carries it as'), /** OData spelling of `offset` (two hops: `$skip` to `skip` to `offset`). */ - $skip: z.union([z.number(), z.string()]).nullable().optional() - .describe('Transport spelling of `offset` (OData `$skip`) — a number, or the digits a querystring carries it as'), + $skip: z.union([QueryPageWindowSchema, z.string()]).nullable().optional() + .describe('Transport spelling of `offset` (OData `$skip`) — a non-negative integer, or the digits a querystring carries it as'), /** OData spelling of `orderBy`. */ $orderby: DataEngineSortSchema.nullable().optional() .describe('Transport spelling of `orderBy` (OData `$orderby`)'), @@ -1007,8 +1056,8 @@ export const QueryTransportParamsSchema = lazySchema(() => z.object({ sort: DataEngineSortSchema.nullable().optional() .describe('Transport spelling of `orderBy`'), /** Bare transport spelling of `offset`. */ - skip: z.union([z.number(), z.string()]).nullable().optional() - .describe('Transport spelling of `offset`'), + skip: z.union([QueryPageWindowSchema, z.string()]).nullable().optional() + .describe('Transport spelling of `offset` — a non-negative integer, or the digits a querystring carries it as'), /** Bare transport spelling of `expand`. */ populate: z.union([z.string(), z.array(z.string())]).nullable().optional() .describe('Transport spelling of `expand`'), @@ -1129,19 +1178,21 @@ function lowerFoldedTransportValues( }); }; // `?$top=50` / `?$skip=10` arrive as digits; canonical `limit` / `offset` - // declare numbers. A string that is not a number is refused rather than + // declare a page window. A string that is not one is refused rather than // passed on: `$top: 'abc'` used to reach the engine as `limit: null` — an - // UNBOUNDED read under a 200 — and `$top: ''` as `limit: 0`. + // UNBOUNDED read under a 200 — and `$top: ''` as `limit: 0`. [#25269] The + // judge is the door's own ({@link QueryPageWindowParamSchema}), so `'-1'` + // and `'1.5'` are refused here exactly as `findData` refuses them; the + // number arm was already judged on the member that carried it. for (const key of ['limit', 'offset'] as const) { const value = bag[key]; if (typeof value !== 'string') continue; - const trimmed = value.trim(); - const n = trimmed === '' ? Number.NaN : Number(trimmed); - if (!Number.isFinite(n)) { - refuse(key, `carries ${JSON.stringify(value)}, which is not a number.`); + const window = QueryPageWindowParamSchema.safeParse(value); + if (!window.success) { + refuse(key, `carries ${JSON.stringify(value)}, which is not a non-negative integer.`); continue; } - bag[key] = n; + bag[key] = window.data; } for (const key of ['fields', 'searchFields'] as const) { if (typeof bag[key] === 'string') { @@ -1285,6 +1336,17 @@ const describedCanonicalExpand = (): z.ZodType => { return member.describe(member.description ?? ''); }; +/** + * [#25269] A canonical window member (`limit` / `offset` / `top`) declared as + * the page window the door enforces ({@link QueryPageWindowSchema}), with the + * text `QuerySchema` gives it — READ from the member, as + * {@link describedCanonicalExpand} does. `QuerySchema`'s own member stays a + * bare number: it is also the recursive `expand` value, whose windows the + * door does not judge. + */ +const describedPageWindowMember = (key: 'limit' | 'offset' | 'top'): z.ZodType => + QueryPageWindowSchema.optional().describe(queryObjectFace().shape[key].description ?? ''); + /** * A query slot whose declared INPUT is the canonical AST or its transport * spelling, and whose OUTPUT is CONSTRUCTED: the folded bag is parsed by @@ -1311,12 +1373,16 @@ const describedCanonicalExpand = (): z.ZodType => { * so a transport bag parses to canonical keys. The canonical members keep * `QuerySchema`'s own declarations verbatim — except `where`, which accepts the * `FilterArray` sugar on every spelling of its slot because the door serves it - * on every spelling of its slot. + * on every spelling of its slot, and the page window (`limit` / `offset` / + * `top`), which declares the non-negative integer the door enforces. */ export const QueryWithTransportSchema = lazySchema( () => queryObjectFace() .extend({ ...(QueryTransportParamsSchema as unknown as z.ZodObject).shape, + limit: describedPageWindowMember('limit'), + offset: describedPageWindowMember('offset'), + top: describedPageWindowMember('top'), expand: describedCanonicalExpand(), // The lowering sink the sentence below names is the one maintainer // ruling C on #5158 fixed. The id lives in this comment and not in the diff --git a/packages/spec/src/data/query-transport.test.ts b/packages/spec/src/data/query-transport.test.ts index fdcdde5f10..6dd2682b1b 100644 --- a/packages/spec/src/data/query-transport.test.ts +++ b/packages/spec/src/data/query-transport.test.ts @@ -250,6 +250,27 @@ describe('§3 the fold is TOTAL — every admitted value shape lowers, or the pa }); it('an empty-string `$top` — the `limit: 0` case', () => refuses({ object: 'a', $top: '' })); it('a non-numeric `$skip`', () => refuses({ object: 'a', $skip: 'abc' })); + // [#25269] The page window the door enforces: a non-negative integer, on + // every spelling and both arms. The table over the door's own verdicts is + // `metadata-protocol`'s `protocol.page-window-declared.test.ts`. + it.each([ + ['limit', -1], ['limit', 1.5], ['offset', -5], ['offset', 0.5], ['top', -1], + ['$top', -1], ['$top', 1.5], ['$skip', -1], ['skip', 2.5], + ])('a `%s` of %s — a number that is not a page window', (key, value) => { + const issues = refuses({ object: 'a', [key]: value }); + expect(issues).toHaveLength(1); + }); + it.each([['$top', '-1'], ['$top', '1.5'], ['$skip', '-5'], ['skip', '0.5']])( + 'a `%s` of %j — digits that are not a page window, refused at the canonical path naming the spelling', + (key, value) => { + const issues = refuses({ object: 'a', [key]: value }); + expect(issues).toHaveLength(1); + expect(issues[0].path).toEqual([key === '$top' ? 'limit' : 'offset']); + expect(issues[0].message).toContain(`'${key}'`); + }, + ); + it('`0` stays a window on every spelling (`$top=0` asks for the total alone)', () => + parsesTo({ object: 'a', $top: '0', skip: 0 }, { object: 'a', limit: 0, offset: 0 })); it('a `$count` that is neither the boolean nor its two spellings', () => refuses({ object: 'a', $count: 'yes' })); it('a non-object query', () => expect(QueryWithTransportSchema.safeParse('nope').success).toBe(false));