Skip to content

Commit 5c9138b

Browse files
fix(objectql)!: an unprojected read serves the declared fields, never an orphaned column (#21612)
Fixes #21571 Clause-②: no (narrowing) An unprojected read now serves the object's declared fields plus the platform's system columns. A column no metadata declares, such as a field retired in an upgrade whose column additive sync leaves behind, no longer leaves the engine. The decision is made once, in the engine (`packages/objectql`), so every driver and every door gets the same answer. No driver source and no `metadata-protocol` source is edited. ## Measured first (before any fix) The probe was a temporary, env-gated recorder at the engine's read sites (commit 920cecf, reverted by a7500b3; the net diff carries none of it). In trim mode it deleted every undeclared key at `find`, `findOne`, the by-id update result and the update/delete prior reads, a superset of this fix. | suite (tree) | files | under trim | undeclared-key hits | | --- | --- | --- | --- | | objectql (a7ab047) | 366 | 8 failed, 7374 passed | 50 | | rest (a7ab047) | 258 | all pass | 0 | | plugin-auth (a7ab047) | 118 | all pass | 1 (a mock `sys_account` row) | | service-automation, plugin-sharing, plugin-audit (a7ab047) | all | all pass | 0 | | runtime (778bf5c, with #21545 merged) | 318 | all pass | 0 | The 8 objectql failures, by cause: - 2 in `engine.test.ts` (#3617): the `sys_migration` double declared only `id`. The production object declares `blocking`, `verified_at` and `last_run_at`, and `readMigrationFlagVerified` reads those. The fixture now declares what production declares. - 5 in `engine.test.ts` / `plugin.integration.test.ts`: red only at the prior-read probe sites (a fixture's `readonlyWhen` reads an undeclared `locked`). This fix does not touch prior reads. - 1 in `driver-fault-boundary-redaction.test.ts`: the mock driver hands back live references to its store, and an in-place delete corrupted it. So the fix never mutates a driver row: a row carrying an undeclared key is copied without it. In-process readers of undeclared columns, by call site: - `os migrate plan`'s `unmapped_column` detection introspects the table through the driver (`driver-sql/src/schema-drift.ts`). Unaffected. - `os migrate account-issuer` reads `sys_account.issuer` through the driver (`cli/src/commands/migrate/account-issuer.ts`). Unaffected. - The `os migrate apply` account preflight (`cli/src/commands/migrate/apply.ts`, then plugin-auth `account-identity-preflight.ts`) calls `engine.find` with an explicit projection naming the retired `issuer`. The engine's unknown-plain filter already drops that name today. Unchanged by this PR (see Acceptance notes). - The metadata column migrations (`packages/metadata/src/migrations/*`) use driver raw DDL. Unaffected. - `cloneData` copies every key of its `findOne` source into an insert. Before this fix, an orphaned column made the clone fail at the write door (`Unknown field 'mailing_street'`, `INVALID_FIELD`); it now succeeds. This reader was hurt by the orphaned column, it did not rely on it. - Export, search, the RPC dispatcher, `getData` and `findData` all read through `find`/`findOne`. They narrow with the door, which is intended. - hotcrm's `scripts/backfill-contact-mailing-address.ts` is external and reads through REST. The changeset names its interim route. Hypothesis verdicts: - **H1 (confirmed):** `find` reached the driver with `fields` undefined, and driver-sql answered with `select('*')`. - **H2 (confirmed, and it decided the shape):** on the composed REST harness, an explicit projection naming a declared field with no column fails, and driver-sql's ladder returns the whole row, retired column included (the reach pin's ladder case, red before the fix). So the engine shapes the rows the driver returns instead of pushing a projection down. That holds whichever rung answered, and needs no driver edit. - **H3:** one authority, the registry field map plus `PLATFORM_PROVISIONED_COLUMNS`. That list moved into `declared-read-columns.ts`. The explicit-projection filter (`find`/`findOne`), the new default projection and the write door's `undeclaredWriteFieldErrors` all read it. No second list. - **H4:** the shaping runs before formulas, `expand`, file references and the hooks. Internal omission, credential masking and the `__search` strip still run after the hooks. The conformance matrix pins formula, password mask, `internal`, and each system column. - **H6:** only `find` and `findOne` return row bodies. `count` returns a number. `aggregate` returns aggregates, and its `groupBy` names are gated at the door. `findStream` was retired in 17.0. `expand` re-enters `this.find`. ## Changes - `packages/objectql/src/declared-read-columns.ts` (new): `PLATFORM_PROVISIONED_COLUMNS`, `declaredColumnSet` (no opinion for an absent, array or empty field map, the rule the read and write doors already share), and the non-mutating `withDeclaredColumnsOnly` / `rowsWithDeclaredColumnsOnly`. - `packages/objectql/src/engine.ts`: `find` and `findOne` shape the driver's rows right after the driver call. The explicit filter and the write door read the shared list. - `packages/objectql/src/no-operator-object-door.ts`: a comment pointer to the list's new home. - Tests: `packages/rest/src/data-query-unprojected-declared-fields.test.ts` (the reach pin), `packages/objectql/src/unprojected-read-declared-fields-conformance.test.ts` (the conformance matrix), and the `sys_migration` fixture in `engine.test.ts`. - `.changeset/21571-unprojected-read-declared-fields.md`: `@objectstack/objectql` minor, BREAKING narrowing, ADR-0087 `not-required (no-migration-prescription)`, and the interim route. - `content/docs/data-modeling/queries.mdx`: one paragraph stating the default projection. ## Pins - **Reach pin.** The harness is the composed REST harness this package already uses: `RestServer`, then `ObjectStackProtocolImplementation`, then `ObjectQL`, then a real `SqlDriver` on better-sqlite3 with a file database. Boot one declares two mailing fields and writes values. Boot two retires them, the card's upgrade. Then `POST /api/v1/data/rq_contact/query` with no `fields`. - Before the fix: 4 failed, 2 passed. The retired columns were present on the query and on `GET /data/:object/:id`, every-key-declared was red, and the ladder rung was red. The system-columns pin and the explicit `INVALID_FIELD` / 400 pin were green. - After the fix: 6 passed. - **Conformance matrix.** `objectql`, three driver behaviours (`whole row`, `projection`, `ladder`) × doors (`find`, bare `find`, `findOne`, ladder projection, retired-only projection, `expand`, `findData`, `getData`, `cloneData`), plus declared-treatment and store-not-mutated pins. A future driver shape is one entry. - **Explicit projection** of a retired column still answers `INVALID_FIELD` / 400. The test asserts `code` and `status`. - **Pin sweep.** Repo-wide `git grep` over tests for orphaned, unmapped, retired or undeclared column readings, plus the trim-mode runs above: no test asserted that an unprojected read returns an undeclared column. The only fallout was the `sys_migration` fixture, which now declares its columns. No refusal assertion was changed. ## Reverse verification (fix committed first, at 9593fbd) Each shaping call was replaced in turn with `node scripts/ablation-replace.mjs` (anchor hit 1 to 0, blob changed). Then objectql was rebuilt (exit 0), and `ablation-dist-preflight --absent` confirmed the call is absent from all 14 built files. - `find` call removed: reach pin 3 failed, 3 passed (query, every-key-declared, ladder). Matrix 17 failed, 19 passed. - `findOne` call removed: reach pin 1 failed, 5 passed (GET by id). Matrix 9 failed, 27 passed (`findOne`, `getData`, `cloneData` × 3 shapes). The clone's failure reads `Unknown field 'mailing_street' on object 'rq_contact'`. - Restore: `git checkout HEAD -- packages/objectql/src/engine.ts`. Blob aadf401 equals the HEAD blob, `git diff HEAD` is empty and `git status --porcelain` is clean. After a rebuild, the preflight finds both calls present in dist. Reach pin 6/6 and matrix 36/36. ## Local verification, at 4bc22fe (the PR head, after merging main, which includes #21545) - `pnpm --filter @objectstack/objectql test`: 367 files, 7415 passed. `test:repo`: 1 file, 5 passed. `typecheck` (tsc plus test-typecheck): exit 0. - The reach pin: 6 passed. `pnpm --filter @objectstack/rest run typecheck`: exit 0. - The full rest suite ran at 99033a9: 259 files, 4889 passed, 326 skipped. The last merge (3 commits) touched neither `rest` nor `objectql`, so that run was not repeated. - Gates: `node scripts/pm/dispatch-gates.mjs --commands` (no paths) re-derived the same 96 commands at this head. All 96 were run and reconciled with `--ran` (96 run, 0 unrun). - 95 exit 0. - `node scripts/check-engine-split-ratio.mjs --days 90` answered exit 2: NOT MEASURED. The clone is shallow inside the 90-day window, and this is a report-only metric. - Lint, narrowed and proven: - Population, from eslint's own config: the 8 changed files went to `eslint --no-inline-config --format json`. eslint reports the `.md`/`.mdx` pair as "File ignored because no matching configuration was supplied". - Count, from the JSON: 8 files in the report, 6 TypeScript files linted, 0 errors on them. The 2 warnings are the two ignore notices. - Invariance: `eslint.config.mjs` enables no type-aware linting (no `parserOptions.project`, no typed rules), so this diff cannot move an untouched file's verdict. - Not run locally, declared to CI: the remaining downstream consumers of `@objectstack/objectql`. The trim-mode measurement above covered rest, runtime, plugin-auth, plugin-sharing, plugin-audit and service-automation under a superset of this change. ## Acceptance notes - **Write responses still carry orphaned columns. This is a finding for the seat, outside this card's read ruling.** - Measured after this fix on the same harness: `PATCH /api/v1/data/rq_contact/c1` with `{ name }` answers 200, and its `record` carries `mailing_street: "1 Retired Way"`. - Cause: the engine's by-id update returns driver-sql's readback (`select *`), and `updateData` strips only `internal` fields from it. - The create 201 and clone 201 bodies are built from driver-sql's `returning('*')` too. That is not measured here; on a new row the orphaned columns would read null. - The A-prime ruling keeps engine write results whole for privileged writers, so where to cut this is a decision. It is not taken here. - Prior reads (the update/delete `previous` rows handed to hooks) still carry orphaned columns. In process only, no public door measured. Carrier: none. - In process, `engine.aggregate` does not refuse an undeclared `groupBy` name. The data door does (`assertGroupByFieldsExist`). Carrier: none. - The `os migrate apply` account preflight's `issuers` label reads `(none)` on the engine path, because the engine already drops the retired `issuer` from an explicit projection. The collision verdict keys on provider and account ids and is unaffected. `os migrate account-issuer` reads through the driver and shows real issuers. Carrier: none. - #21573 (the operator-only `os migrate` read of unmapped columns) is not addressed here. It is the interim route the changeset names. --- _Generated by [Claude Code](https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 440cd32 commit 5c9138b

8 files changed

Lines changed: 717 additions & 20 deletions
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
'@objectstack/objectql': minor
3+
---
4+
5+
fix(objectql)!: a read with no projection serves the object's declared fields and the platform's system columns, never a column no metadata declares (#21571)
6+
7+
**BREAKING (narrowing)** — what a released read door serves shrinks. A column that
8+
no metadata declares, typically a field retired in an upgrade whose column additive
9+
schema sync leaves in the table until `os migrate apply --allow-destructive`, is no
10+
longer returned by any read through the engine.
11+
12+
| read | before | now |
13+
| --- | --- | --- |
14+
| `POST /api/v1/data/:object/query` or `GET /api/v1/data/:object` with no `fields` | every column of the table, retired ones and their values included | the declared fields, the registry's system columns, `id`, `created_at`, `updated_at` |
15+
| `GET /api/v1/data/:object/:id`, export, search hits, the RPC dispatcher, `expand`ed records | the same whole row | the same declared set |
16+
| `engine.find` / `engine.findOne` in process (hooks, flows, plugins), no `fields` | the whole row | the declared set |
17+
| an explicit `fields` naming a declared field whose column does not exist yet (driver-sql retries `select('*')`) | the whole row, retired columns included | the declared set |
18+
| `POST /api/v1/data/:object/:id/clone` of a record whose table carries a retired column | refused `INVALID_FIELD` (the copy carried the retired column into the insert) | cloned |
19+
20+
**Unchanged:** naming a retired column in `fields` still answers `400 INVALID_FIELD`
21+
on the data door. Declared fields keep their treatment: `internal: true` omission,
22+
credential masking, formula evaluation and the hidden `__search` strip run as
23+
before, and the registry-injected tenant, owner and audit columns are still served.
24+
No driver changed: the engine shapes the rows any driver returns, so the answer is
25+
the same on every driver and every door. Writes, and the rows a write returns, are
26+
not changed by this release.
27+
28+
**If you still read a retired column's values** (for example a one-time conversion
29+
that copies the old columns into their replacement field): run that conversion
30+
BEFORE upgrading to this release, while the old field is still declared, or, once
31+
it lands, read the unmapped columns through the operator-only `os migrate` read
32+
(objectstack#21573). There is no flag that re-opens undeclared columns on a runtime
33+
door. An in-process reader that needs a column must declare it as a field.
34+
35+
Clause-②: no (narrowing)
36+
37+
<!-- adr-0087: not-required (no-migration-prescription) No authorable key, spelling, export, config field or stored shape is removed or renamed, and no stored row is read or rewritten: the change narrows which physical columns the engine's read verbs return, to the field map the metadata already declares. A retired column's values stay in the table; what an app does with them is an operator action stated above (convert before upgrading, or the operator-only os migrate read), not a FROM to TO mapping that objectstack migrate meta could apply. The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id covers a read projection (not registered / already-registered); and the change is runtime behaviour, not a declaration (not runtime-interface-only / type-surface-only). -->

‎content/docs/data-modeling/queries.mdx‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -285,6 +285,13 @@ pattern the metadata-revision / flow-run / notification list endpoints already u
285285
}
286286
```
287287

288+
Without `fields`, a read returns the object's **declared** fields plus the platform's
289+
system columns (`id`, `created_at`, `updated_at`, and the tenant, owner and audit columns
290+
the registry adds). A column no metadata declares is never returned — for example one a
291+
retired field left in the table until `os migrate apply --allow-destructive` drops it — and
292+
naming it in `fields` on the data API is refused with `400 INVALID_FIELD`. To read such a column's values
293+
for a one-time conversion, run the conversion before the field is retired.
294+
288295
### Nested / Related Fields
289296

290297
{/* os:check */}
Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#21571] The columns a read may serve: the object's DECLARED fields plus the
5+
* platform-provisioned columns — one set, answered from the registry's field
6+
* map, and the same set an explicit projection is judged against.
7+
*
8+
* ## The defect this closes
9+
*
10+
* A read that names no `fields` reached the driver with no projection, and
11+
* every driver answers that with the whole row (`SELECT *` on SQL). A column
12+
* that no metadata declares — a field retired in an upgrade, whose column
13+
* additive schema sync leaves in the table until an operator runs a
14+
* destructive apply — therefore rode back in every record body, on every door
15+
* that reads through the engine (`POST /data/:object/query`, the list route,
16+
* `GET /data/:object/:id`, export, search, the RPC dispatcher, a hook's or a
17+
* flow's in-process read). Naming the same column in `fields` answered
18+
* `400 INVALID_FIELD`: the platform knew it was not a field and served it
19+
* anyway, outside any field-level rule, because there is no field to attach a
20+
* rule to.
21+
*
22+
* ## Where it is decided, and why there
23+
*
24+
* Triage's ruling on #21571: the default projection of an unprojected read is
25+
* decided ONCE, in the engine, so every driver and every door gets the same
26+
* answer — not per driver, not per door. The declared field set is that
27+
* projection; no allow-list of column names and no flag re-opens undeclared
28+
* columns on a runtime door.
29+
*
30+
* The engine SHAPES THE ROWS the driver hands back, rather than pushing a
31+
* projection down to the driver. Measured on driver-sql: its recovery ladder
32+
* retries `select('*')` whenever a projected statement fails on an
33+
* unresolvable column, so a pushed-down projection naming a declared field
34+
* whose column does not exist yet (not migrated, or the read races a schema
35+
* sync) would be answered by the whole row — orphaned columns included. An
36+
* EXPLICIT projection reaches that rung today too. Shaping the returned rows
37+
* holds whichever rung answered, and needs no driver edit.
38+
*
39+
* It runs on the rows as they arrive from the driver — before formula
40+
* evaluation, `expand`, file-reference resolution and the `afterFind` hooks —
41+
* so everything downstream of storage sees the declared record, the same scope
42+
* `materializeDeclaredFields` gives every CEL surface. Keys added AFTER that
43+
* (a formula's value, an expanded record, a hook's derived key) are the
44+
* engine's or the hook's, not storage's, and are untouched. Declared fields
45+
* keep their existing treatment: `internal: true` omission, credential
46+
* masking and the `__search` companion strip all still run after the hooks.
47+
*
48+
* ## Where it deliberately has no opinion
49+
*
50+
* The rule the read and write doors already share: a door that cannot see the
51+
* field map must not invent a verdict about it. A schema with no field map, an
52+
* ARRAY field map (not checkable — `Object.keys` yields indices), or an EMPTY
53+
* one (indistinguishable from an unpopulated map: a registered object always
54+
* carries at least the injected system columns) leaves the rows exactly as the
55+
* driver returned them.
56+
*
57+
* ## In-process readers of undeclared columns — measured before this landed
58+
*
59+
* The objectql, rest, runtime, plugin-auth, plugin-sharing, plugin-audit and
60+
* service-automation suites ran with every undeclared key removed at this
61+
* seam; no production reader needed one (the fallout was test fixtures). The
62+
* operator reads that legitimately need a retired column's values go through
63+
* the driver, never through this path: `os migrate plan`'s `unmapped_column`
64+
* detection introspects the table, and `os migrate account-issuer` reads
65+
* `sys_account` through the driver the engine routes it to.
66+
*/
67+
68+
/**
69+
* The columns the platform provisions on every physical table without an
70+
* author declaring them. `id` is the driver's primary key; the two audit
71+
* timestamps are engine/driver-stamped. The registry injects the rest of the
72+
* system columns (tenant, owner, the audit actors) INTO the field map, so they
73+
* need no entry here.
74+
*
75+
* ONE list: the read verbs' explicit-projection filter, this module's row
76+
* shaping and the write path's undeclared-key door all read it, so a key a read
77+
* accepts is never refused by a write and never trimmed from a row.
78+
*/
79+
export const PLATFORM_PROVISIONED_COLUMNS = ['id', 'created_at', 'updated_at'] as const;
80+
81+
/**
82+
* The declared column set of an object — its field-map keys plus
83+
* {@link PLATFORM_PROVISIONED_COLUMNS} — or `undefined` when there is no
84+
* checkable field map (absent, an array, or empty). `undefined` means "no
85+
* opinion", never "nothing is declared".
86+
*/
87+
export function declaredColumnSet(
88+
schema: { fields?: unknown } | null | undefined,
89+
): ReadonlySet<string> | undefined {
90+
const fields = schema?.fields;
91+
if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return undefined;
92+
const names = Object.keys(fields as Record<string, unknown>);
93+
if (names.length === 0) return undefined;
94+
const declared = new Set(names);
95+
for (const provisioned of PLATFORM_PROVISIONED_COLUMNS) declared.add(provisioned);
96+
return declared;
97+
}
98+
99+
/**
100+
* The row with every key outside `declared` removed — a NEW object when there
101+
* was anything to remove, the same reference otherwise. Never mutates the row
102+
* it is given: a driver that hands back a live reference into its own store
103+
* (a contract violation, but one a test double commits) must not lose data
104+
* because a read happened.
105+
*/
106+
export function withDeclaredColumnsOnly<T>(row: T, declared: ReadonlySet<string> | undefined): T {
107+
if (!declared || !row || typeof row !== 'object' || Array.isArray(row)) return row;
108+
const source = row as unknown as Record<string, unknown>;
109+
let undeclared = false;
110+
for (const key of Object.keys(source)) {
111+
if (!declared.has(key)) { undeclared = true; break; }
112+
}
113+
if (!undeclared) return row;
114+
const shaped: Record<string, unknown> = {};
115+
for (const key of Object.keys(source)) {
116+
if (declared.has(key)) shaped[key] = source[key];
117+
}
118+
return shaped as unknown as T;
119+
}
120+
121+
/** {@link withDeclaredColumnsOnly} over a driver's result page. */
122+
export function rowsWithDeclaredColumnsOnly<T>(rows: T[], declared: ReadonlySet<string> | undefined): T[] {
123+
if (!declared) return rows;
124+
let changed = false;
125+
const shaped = rows.map((row) => {
126+
const next = withDeclaredColumnsOnly(row, declared);
127+
if (next !== row) changed = true;
128+
return next;
129+
});
130+
return changed ? shaped : rows;
131+
}

‎packages/objectql/src/engine.test.ts‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2565,6 +2565,20 @@ describe('ObjectQL — file-as-reference migration flag (#3617)', () => {
25652565
let engine: ObjectQL;
25662566
let driver: IDataDriver;
25672567

2568+
// [#21571] The flag columns the real `sys_migration` declares
2569+
// (`platform-objects` sys-migration.object.ts). A read serves only declared
2570+
// columns, so a double declaring `id` alone would read every flag row as
2571+
// empty — the fixture declares what the production object declares.
2572+
const SYS_MIGRATION_DEF = {
2573+
name: 'sys_migration',
2574+
fields: {
2575+
id: { type: 'text' },
2576+
last_run_at: { type: 'datetime' },
2577+
verified_at: { type: 'datetime' },
2578+
blocking: { type: 'number' },
2579+
},
2580+
};
2581+
25682582
const verifiedRow = {
25692583
id: 'adr-0104-file-references',
25702584
last_run_at: '2026-07-27T00:00:00.000Z',
@@ -2594,7 +2608,7 @@ describe('ObjectQL — file-as-reference migration flag (#3617)', () => {
25942608
const withMediaObject = () => {
25952609
vi.mocked(SchemaRegistry.getObject).mockImplementation((name: string) => {
25962610
if (name === 'note') return { name: 'note', fields: { doc: { type: 'file' } } } as any;
2597-
if (name === 'sys_migration') return { name: 'sys_migration', fields: { id: { type: 'text' } } } as any;
2611+
if (name === 'sys_migration') return SYS_MIGRATION_DEF as any;
25982612
return undefined;
25992613
});
26002614
};
@@ -2636,7 +2650,7 @@ describe('ObjectQL — file-as-reference migration flag (#3617)', () => {
26362650
it('costs no query for an object that declares no media field', async () => {
26372651
vi.mocked(SchemaRegistry.getObject).mockImplementation((name: string) => {
26382652
if (name === 'invoice') return { name: 'invoice', fields: { amount: { type: 'number' } } } as any;
2639-
if (name === 'sys_migration') return { name: 'sys_migration', fields: { id: { type: 'text' } } } as any;
2653+
if (name === 'sys_migration') return SYS_MIGRATION_DEF as any;
26402654
return undefined;
26412655
});
26422656
vi.mocked(driver.findOne).mockResolvedValue(verifiedRow as any);
@@ -2736,7 +2750,7 @@ describe('ObjectQL — file-as-reference migration flag (#3617)', () => {
27362750
vi.mocked(SchemaRegistry.getObject).mockImplementation((name: string) => objects[name]);
27372751
vi.mocked((SchemaRegistry as any).getAllObjects).mockImplementation(() => Object.values(objects));
27382752
};
2739-
const SYS_MIGRATION = { name: 'sys_migration', fields: { id: { type: 'text' } } };
2753+
const SYS_MIGRATION = SYS_MIGRATION_DEF;
27402754
const lines = (info: any) => info.mock.calls.map((c: any[]) => String(c[0])).join('\n');
27412755

27422756
it('names the command that closes an open value-shape gate', async () => {

‎packages/objectql/src/engine.ts‎

Lines changed: 35 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -290,6 +290,14 @@ import { readonlyWhenFkJudgementReadsParent } from './validation/rule-validator.
290290
// total over the MASTER's declared fields before it leaves this engine — the
291291
// same helper every other server seam materialises with (#1871/#4649/#4953).
292292
import { materializeDeclaredFields } from './declared-fields.js';
293+
// [#21571] The declared column set — the read verbs' default projection, their
294+
// explicit-projection filter and the write path's undeclared-key door, one list.
295+
import {
296+
PLATFORM_PROVISIONED_COLUMNS,
297+
declaredColumnSet,
298+
rowsWithDeclaredColumnsOnly,
299+
withDeclaredColumnsOnly,
300+
} from './declared-read-columns.js';
293301
import { applyInMemoryAggregation } from './in-memory-aggregation.js';
294302
import {
295303
resolveEngineDeleteDispatch,
@@ -1912,19 +1920,17 @@ function assertProjectionHasNoDottedPaths(
19121920
* because the partial-success path (`insertMany`) reports per row and must
19131921
* cull the bad rows instead of failing the batch around them.
19141922
*/
1915-
const PLATFORM_PROVISIONED_COLUMNS = ['id', 'created_at', 'updated_at'] as const;
1916-
19171923
function undeclaredWriteFieldErrors(
19181924
object: string,
19191925
schema: { fields?: unknown } | undefined,
19201926
rows: readonly unknown[],
19211927
): Array<Error | undefined> {
19221928
const out: Array<Error | undefined> = new Array(rows.length);
1923-
const fields = schema?.fields;
1924-
if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return out;
1925-
const declared = new Set(Object.keys(fields as Record<string, unknown>));
1926-
if (declared.size === 0) return out;
1927-
for (const provisioned of PLATFORM_PROVISIONED_COLUMNS) declared.add(provisioned);
1929+
// [#21571] The declared set and its "no opinion" cases (no map, an array
1930+
// map, an empty map) are `declaredColumnSet`'s, shared with the read verbs'
1931+
// default projection — one answer to "is this a column of the object".
1932+
const declared = declaredColumnSet(schema);
1933+
if (!declared) return out;
19281934
for (let i = 0; i < rows.length; i++) {
19291935
const row = rows[i];
19301936
if (!row || typeof row !== 'object' || Array.isArray(row)) continue;
@@ -11894,13 +11900,12 @@ export class ObjectQL implements IObjectQLEngine {
1189411900
// projection is a different fact and no longer reaches this filter via
1189511901
// the engine — `assertProjectionHasNoDottedPaths` above refused it.
1189611902
if (_findSchema?.fields && Array.isArray(ast.fields) && ast.fields.length > 0) {
11897-
const known = new Set(Object.keys(_findSchema.fields));
1189811903
// Always allow the primary key + audit columns even if not present in
1189911904
// schema.fields. Without this, callers requesting `select=id,name`
1190011905
// silently get the `id` projected away, breaking record navigation.
11901-
known.add('id');
11902-
known.add('created_at');
11903-
known.add('updated_at');
11906+
// [#21571] The same three the default projection and the write door
11907+
// admit — `PLATFORM_PROVISIONED_COLUMNS`, one list.
11908+
const known = new Set<string>([...Object.keys(_findSchema.fields), ...PLATFORM_PROVISIONED_COLUMNS]);
1190411909
// Whole names, no head-splitting: only plain entries reach here (the
1190511910
// dotted refusal above fired on anything carrying a '.').
1190611911
const filtered = ast.fields.filter(f => known.has(f));
@@ -11939,6 +11944,19 @@ export class ObjectQL implements IObjectQLEngine {
1193911944
try {
1194011945
let result = await driver.find(object, hookContext.input.ast as QueryAST, hookContext.input.options as any);
1194111946

11947+
// [#21571] The read's default projection is the DECLARED field set:
11948+
// a column no metadata declares (a field retired in an upgrade,
11949+
// whose column additive sync leaves behind) never leaves the engine.
11950+
// Shaped here, on the rows as the driver returned them, so it holds
11951+
// whichever driver answered and whichever rung of a driver's
11952+
// recovery ladder answered (driver-sql retries `select('*')` when a
11953+
// projected statement names a missing column) — and before formulas,
11954+
// `expand`, file references and the hooks, so all of them see the
11955+
// declared record. See `declared-read-columns.ts`.
11956+
if (Array.isArray(result)) {
11957+
result = rowsWithDeclaredColumnsOnly(result, declaredColumnSet(_findSchema));
11958+
}
11959+
1194211960
// Post-process: evaluate formula virtual fields against the raw rows.
1194311961
// [#20082] With the caller's permission map when a formula calls
1194411962
// `can` — one resolution for the whole result set, never per row.
@@ -12174,12 +12192,9 @@ export class ObjectQL implements IObjectQLEngine {
1217412192
// the rationale, and for why this tolerance is plain-columns-only ([#7589]
1217512193
// refused any dotted entry above, so none reaches this filter).
1217612194
if (_findOneSchema?.fields && Array.isArray(ast.fields) && ast.fields.length > 0) {
12177-
const known = new Set(Object.keys(_findOneSchema.fields));
1217812195
// Always allow the primary key + audit columns even if not present
12179-
// in schema.fields (matches `find()` behavior).
12180-
known.add('id');
12181-
known.add('created_at');
12182-
known.add('updated_at');
12196+
// in schema.fields (matches `find()` behavior, one list).
12197+
const known = new Set<string>([...Object.keys(_findOneSchema.fields), ...PLATFORM_PROVISIONED_COLUMNS]);
1218312198
const filtered = ast.fields.filter(f => known.has(f));
1218412199
ast.fields = filtered.length > 0 ? filtered : undefined;
1218512200
}
@@ -12215,6 +12230,10 @@ export class ObjectQL implements IObjectQLEngine {
1221512230

1221612231
let result = await driver.findOne(objectName, hookContext.input.ast as QueryAST, hookContext.input.options as any);
1221712232

12233+
// [#21571] Same default projection as `find`, same position: the
12234+
// declared field set, applied to the row as the driver returned it.
12235+
result = withDeclaredColumnsOnly(result, declaredColumnSet(_findOneSchema));
12236+
1221812237
// Post-process: evaluate formula virtual fields against the raw row
1221912238
// ([#20082] with the caller's permission map when a formula calls `can`).
1222012239
if (result != null) {

‎packages/objectql/src/no-operator-object-door.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -206,7 +206,7 @@ export function declaredNoOperatorObjectColumn(def: unknown): NoOperatorObjectCo
206206
* [#20745] The columns every record carries whether or not the declared map
207207
* lists them, with the type each stores: the same three names `find` /
208208
* `findOne` add to their known set and the write gate admits unconditionally
209-
* (`PLATFORM_PROVISIONED_COLUMNS` in `engine.ts`), because the platform
209+
* (`PLATFORM_PROVISIONED_COLUMNS` in `declared-read-columns.ts`), because the platform
210210
* provisions them rather than the author declaring them.
211211
*/
212212
const PLATFORM_PROVISIONED_COLUMN_TYPES: ReadonlyMap<string, string> = new Map([

0 commit comments

Comments
 (0)