Skip to content

Commit d537a31

Browse files
feat(objectql,lint): a formula field reads one hop through a reference — record.book.title computes, a second hop is refused (#24274)
Part of #24197 — the code part of the ruling. The ruling's ADR-0079 note (`docs/adr/**`, Tier H) follows as its own PR, so this one does not close the card. Clause-②: yes (narrowing) (the lint and the engine narrow what they accept; `@objectstack/formula` also widens its published index by one optional parameter and one exported type, `TraversalSubject`. Per `clause2-line.mjs`, a diff that widens one surface and narrows another reads `yes (narrowing)`. No level moves.) ## What changes A formula field that reads through a reference field (`record.book.title`) now computes. Before, `planFormulaProjection` gave a formula only the row's own columns, so the reference held an id, the read faulted, and the field read `null` on every row. Meanwhile the lint admitted the expression. **Engine (`packages/objectql`).** - The formula read path preloads the related fields a formula names ONE hop through a lookup or master-detail field. This covers `find`, `findOne`, and the record a create or update hands back. The read is one batched `find` per reference field per page, never one per row, and it names only `id` and the related fields the formulas name. - There is one loader. The validation-rule path's `resolvePredicateRelated` body moved into `resolveOneHopRelated(schema, wanted, rows, context, authority)`, and both seams call it. Validation predicates keep `authority: 'predicate'`: a SYSTEM read behind the caller's read doors, unchanged. Formulas use `authority: 'caller'`: the hop is read under the reader's own context, through every layer `find` applies. The hop collector `addReferenceTraversals` and the conflict check `referenceTraversalConflicts` are extracted from `rule-validator.ts`, not re-implemented. - Verdicts are the validation path's own (`RelatedFieldBinding`). A related record that is not found, or that the reader may not read, a related field the related object does not declare, and a failed read each leave the formula **unevaluated**. It reads `null`, and the formula fault sink reports why, naming the related object and field. An **empty** reference is evaluated, so `record.reader.?name.orValue('—')` gives it a value. A plain read through an empty reference faults, and the reported reason names that optional read. - One hop is the depth. A second hop written out, and a reference read both through and as a plain value, are not evaluated. They read `null` with the lint's own sentence. A `find` that IS a formula's hop read (tracked by query-object identity in a `WeakSet`, never on a context) serves no hop of its own, so two objects whose titles read each other terminate. **Lint (`packages/lint`) and `@objectstack/formula`.** `judgeFieldFormula` passes `traversalHydration: true`. `validateExpression` now honours the flag in the `value` role, and `findTraversalConflicts` takes a `subject` (`'predicate'` default, or `'formula'`) for the wording. Results: - `record.book.title + ' · ' + record.reader.name` passes. - `record.book.shelf.code` is refused (`traversal-multi-hop`), naming the field and the one-hop alternative. - `record.reader == null ? … : record.reader.name` is refused (`traversal-bare-and-traversed`), with the optional read as the repair. - Callers that do not opt in see no change. **Spec (`packages/spec`).** The blueprint schema's `expression` and `nameField` describe strings, and their strict twins, gave `"record.order_no + ' · ' + record.customer"` and `"{order_no} · {customer}"`. `customer` is a lookup, so those compose the stored id into the title. They now teach `record.customer.name`, say that a second hop is refused, and say that a formula title cannot be sorted by or searched. `content/docs/references/ai/solution-blueprint.mdx` is regenerated (`check:generated --fix`). **Two edits outside the claim's file list, both forced by gates on the moved code:** - `content/docs/permissions/system-context.mdx` row 29b: the anchor moves from `engine.ts#resolvePredicateRelated` to `engine.ts#resolveOneHopRelated`, because the elevation read moved there (`check:system-context-census`). - `federated-injected-column-readers.test.ts`: its ledger rows are renamed to the moved function names (`resolveOneHopRelated`, `addReferenceTraversals`, `referenceTraversalConflicts`), with the same dispositions. ## Sort and search (the ruling's measured choice) A formula's value is stored nowhere. Drivers materialise no column for it, and the engine drops it from the driver projection. So a title formula over lookups is **unsortable and unsearchable**, and it is already loud wherever a sort or a search names it. Measured over HTTP on this branch, below: `?sort=title` answers `400 INVALID_SORT` and `?search=…&searchFields=title` answers `400 INVALID_FIELD`. The engine-level sort refusal is pinned in `engine-formula-one-hop.test.ts`. No engine change was needed. The describe strings now say so. ## Verification Head `e9c02e6813` (after merging `origin/main` at `53ef752436`). **Live backend over HTTP.** `examples/app-todo` was booted with `objectstack dev --fresh` on this branch. A junction object was saved through `PUT /api/v1/meta/object/lib_loan`, with lookups `book` and `reader` and `nameField: title`, where `title = record.book.title + ' · ' + record.reader.name`. | request | answer | |---|---| | save `lib_loan` with `title = record.book.shelf.name` (two hops) | `422 INVALID_METADATA`, `object 'lib_loan' · field 'title' expression`: "`record.book` is read through more than one relationship hop. A formula resolves ONE hop …" | | save `lib_loan` with the one-hop composed title | `200` | | `POST /api/v1/data/lib_loan` `{book, reader}` | `record.title` = `Dune · Bo` | | `GET /api/v1/data/lib_loan?fields=title,book,reader` | `Dune · Ann`, `Emma · Ann`, `Dune · Bo`, and `null` for the loan with an empty `reader`. Rows still carry the foreign-key ids. The empty-reader fault is logged once, with the optional-read repair. | | `GET …?sort=title` | `400 INVALID_SORT` | | `GET …?search=Dune&searchFields=title` | `400 INVALID_FIELD` | **Engine pins** (`packages/objectql/src/engine-formula-one-hop.test.ts`, 17 tests, real `ObjectQL` and a driver that pages): - A page of 4 loans over 3 books and 2 readers makes exactly 1 `lib_book` read and 1 `lib_reader` read. The book read is `where id $in [b1,b2,b3]` with fields `[id, title]`. The next page reads only its own ids. - A projection naming only `title` answers `{ title }` alone. `findOne` and the insert response are served. - Caller authority: a middleware scoping `lib_reader` rows for non-system callers leaves the formula `null`, with "not found by this read (it does not exist, or this reader may not read it)". The hidden name is absent from the response, and SYSTEM reads it. A middleware masking `name` keeps it out of the value. - A dangling reference reads `null` with a reason (no fallback evaluated). An empty reference with `.?name.orValue('—')` gives `Emma · —`. An undeclared related field is named on the related object. - A second hop is refused with no related read. The related object's own-column formulas compute on the hop read. Its own hop is not served (two hops via a formula). A ping/pong cycle terminates. - `orderBy: title` rejects with `{ code: 'INVALID_SORT', status: 400 }`. **Lint pins** (`packages/lint/src/validate-expressions.formula-one-hop.test.ts`, 10 tests): admit one hop and the optional read; refuse two hops and the bare-and-traversed read; the four blueprint describe strings carry the one-hop example; the example passes the field-formula pass with `customer` a lookup; the `nameField` strings state the sort and search answer. **Formula pins** (`relationship-traversal.test.ts`): the `value` role is judged when opted in, with formula wording and no rule repair, and is unchanged when not opted in. **Pin flipped (load-bearing).** `relationship-traversal.test.ts` "checks nothing in the `value` role" asserted that an opted-in `value` source earned no traversal refusal. It now asserts the refusal's `code`, `params` and formula wording. **Ablations** (`scripts/ablation-replace.mjs`; each mutation proven on disk, each restore proven `blob == HEAD`, `git diff HEAD` empty): | mechanism | mutation | result | |---|---|---| | hop preload | `resolveFormulaRelated` returns no lookup | 13 of 17 engine pins red | | caller authority | hop read as SYSTEM | exactly the 2 caller-permission pins red | | one-hop depth guard | `formulaHopReads.add` removed | second-hop-through-formula pin red; the full file no longer terminates (cycle recursion; killed at the timeout) | | engine second-hop refusal | `hopConflict` never set | the second-hop pin red | | lint verdict | `traversalHydration` dropped from `judgeFieldFormula` | the 2 lint refusal pins red | | formula `value` arm | arm back to `predicate` only | the 2 formula refusal pins red | **Suites and typechecks** (exit codes captured before any pipe): - `pnpm --filter @objectstack/formula test`: 47 files, 1330 tests passed. - `pnpm --filter @objectstack/lint test`: 144 files, 6835 tests passed. - `pnpm --filter @objectstack/objectql test` ran on the pre-merge head: 437 of 438 files passed. The red file was the federated reader ledger, which this PR then fixed. After the merge, the 5 formula, predicate and ledger files were re-run: 82 tests passed. - `typecheck` passed (exit 0) for `spec`, `formula`, `objectql` (its `check:test-typecheck` included) and `lint`. - `pnpm --filter @objectstack/spec check:generated`: all 14 artifacts up to date. **Gates.** `node scripts/pm/dispatch-gates.mjs --commands` (no paths) derived 117 commands on `e9c02e6813`. All 117 ran, and every one exited 0. The `--ran` reconciliation: "117 derived famil(ies) accounted for — 117 run, 0 NOT-MEASURED (a DERIVED zero …)". ## Acceptance notes - `evaluateFormulaField`, the synchronous hook helper, preloads no hop. A traversing formula yields `null` there, as a `can()` formula already does. Its docblock says so. - A formula reached through a hop does not read a hop of its own (the declared depth). The lint does not see this shape, because it would have to resolve the related object's formula. The engine reports it at run time with the second-hop sentence. - H3's "a hop through a non-reference" is not refused. `record.address.city` on an object-valued field traverses today, and the validation path leaves it alone the same way. - Measured on the live backend and outside this card's fence: on an object with no stored text field (the loan above, whose only text-ish field is the formula title), a bare `GET /api/v1/data/lib_loan?search=zzzz-nothing` answers `200` with all 5 rows. The search is dropped silently, because the auto-default search set is empty. This is reported to the seat as a finding, not fixed here. - Over HTTP, the caller-permission case was measured with the engine pins (middleware composition), not with a second seeded user on the live backend. --- _Generated by [Claude Code](https://claude.ai/code/session_01M6VD9XhattUEHKD99C7KUR)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 7a8733f commit d537a31

17 files changed

Lines changed: 1074 additions & 68 deletions
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
'@objectstack/formula': minor
3+
---
4+
5+
feat(formula)!: `validateExpression` judges relationship traversal in the `value` role when the caller opts in, with formula wording
6+
7+
Clause-②: yes (narrowing)
8+
9+
<!-- adr-0087: not-required (no-migration-prescription) No spec key, stored shape or authorable spelling is added, removed or renamed. `validateExpression('value', …)` with `traversalHydration: true` now reports the two traversal conflicts it already reported for `predicate`; a caller that does not pass the flag sees no change. Nothing stored changes, so there is nothing for `objectstack migrate meta` to rewrite. -->
10+
11+
- `validateExpression('value', source, { fieldTypes, traversalHydration: true, … })` now refuses a second hop through a reference (`traversal-multi-hop`) and a reference read both through and as a plain value (`traversal-bare-and-traversed`). The messages say the formula reads null, and the repair for an empty reference is the optional read `record.LOOKUP.?FIELD.orValue('…')`. Before, the `value` role ignored the flag.
12+
- `findTraversalConflicts` takes an optional fourth argument, `subject` (`'predicate'`, the default, or `'formula'`), which selects that wording. The `TraversalSubject` type is exported.
13+
14+
Without the flag, every role behaves as before.
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
'@objectstack/lint': minor
3+
---
4+
5+
feat(lint)!: a formula field's `expression` is judged on the one-hop line — one hop through a reference passes, a second hop is refused
6+
7+
Clause-②: no (narrowing)
8+
9+
<!-- adr-0087: not-required (no-migration-prescription) No spec key, stored shape or authorable spelling is added, removed or renamed. The field-formula pass now refuses two expression shapes it accepted before, and the runtime never computed a value for either over a set reference: a read two hops through references, and a reference read both through and as a plain value. Nothing stored changes, so there is nothing for `objectstack migrate meta` to rewrite. -->
10+
11+
The field-formula pass now passes `traversalHydration` for a formula `expression`, because the engine's formula read path loads one hop through a reference field.
12+
13+
- `record.book.title + ' · ' + record.reader.name` passes, as before. It now also computes at run time.
14+
- `record.book.shelf.code` (two hops) is refused. The refusal names the field and says a formula resolves one hop.
15+
- `record.reader == null ? '—' : record.reader.name` (the reference read both through and as a plain value) is refused. The refusal names `record.reader.?name.orValue('—')`, the read that gives an empty reference a value.
16+
17+
What to do. Read a field stored on the related object instead of a second hop, and replace a null test on a reference with the optional read.
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
'@objectstack/objectql': minor
3+
---
4+
5+
feat(objectql)!: a formula field reads one hop through a reference field — `record.book.title` on a loan now reads the book's title instead of `null`
6+
7+
Clause-②: no (narrowing)
8+
9+
<!-- adr-0087: not-required (no-migration-prescription) No spec key, stored shape or authorable spelling is added, removed or renamed. A formula field that reads `record.LOOKUP.FIELD` now computes; before, it read null on every row. The narrowing is in what a formula over a reference evaluates to in two shapes that never computed a value for a set reference (a second hop, and a reference read both through and as a plain value): both now read null on every row with a reported reason, so there is nothing stored for `objectstack migrate meta` to rewrite. -->
10+
11+
The read path (`find`, `findOne`, and the record a create or update hands back) now preloads the related fields a formula names ONE hop through a lookup or master-detail field, before it computes the formula. A junction object's composed title, `record.book.title + ' · ' + record.reader.name`, shows both names.
12+
13+
- **How it reads.** One batched read per reference field per page of rows, never one per row, naming only `id` and the related fields the formulas name. The loader is the one validation rules already use for `record.LOOKUP.FIELD`. The row a read returns still holds the foreign key; only the formula sees the related record.
14+
- **As whom.** The hop is read under the reader's own permissions (row-level and field-level security, the organization wall). A related record the reader may not see reads as not found, and a related field it may not see reads as empty, so a formula never shows the reader a value they could not read directly.
15+
- **When there is no related record.** A reference to a record that is not found, or that the reader may not read, leaves the formula null and reports why, naming the related object and field. An empty reference is evaluated: write `record.reader.?name.orValue('—')` to give the formula a value while the reference is empty.
16+
- **One hop only.** A second hop (`record.book.shelf.code`) and a reference read both through and as a plain value are not evaluated: the field reads null and the reported reason uses the same wording the lint uses. A formula on the related object that reads a hop of its own reads null on a hop read, so two objects whose titles read each other terminate.
17+
- **Sorting and search.** A formula value is stored nowhere, so a formula title still cannot be sorted by (`400 INVALID_SORT`) or searched, as before.
18+
19+
What to do. Nothing for a formula that reads its own fields. A formula that reads through a reference now shows a value. One that reads two hops should read a field stored on the related object instead.
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
'@objectstack/spec': patch
3+
---
4+
5+
docs(spec): the AI blueprint schema's composed-title example reads the lookup's name one hop through it
6+
7+
The `expression` and `nameField` descriptions on `BlueprintFieldSchema` / `BlueprintObjectSchema` and their strict twins gave `"record.order_no + ' · ' + record.customer"` and `"{order_no} · {customer}"` as a composed title. `customer` is a lookup, so those compose the stored id into the title. They now give `"record.order_no + ' · ' + record.customer.name"` and say that a formula reads a lookup's related record one hop through it, that a second hop is refused, and that a formula title is shown everywhere but cannot be sorted by or searched. No schema shape changes.

‎content/docs/permissions/system-context.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -137,7 +137,7 @@ that silently does not happen.
137137
| 27 | Search-companion column **kept** in a read's rows when it was explicitly requested | objectql | Get: the internal companion column is readable. Lose: nothing for app code — this is the engine reading its own index | `packages/objectql/src/engine.ts#stripSearchCompanionFromRead` |
138138
| 28 | Dependent-count disclosure on a blocked delete | objectql | Get: the count of blocking children. Nothing was elevated past the caller, so nothing is withheld | `packages/objectql/src/engine.ts#dependentCountIsDisclosable` |
139139
| 29 | Reference-cleanup log attributes the write to `'system'` | objectql | Get: an honest actor label instead of `anonymous` when the context carries neither `userId` nor `actor` | `packages/objectql/src/engine.ts#recordReferenceCheckElevation` |
140-
| 29b | A traversing validation rule's related read skips the **caller's bounds** | objectql | Get: no row bound from the caller's own read on a related object no organization wall scopes, no read door on a header the rule reads through a `master_detail` field, and a related read even with no organization under a walled posture (a `tenantId` on the context still scopes it). Every caller that is not system is held to all three: a header its read door does not return is unavailable to the rule exactly as a header that exists nowhere is, whatever it holds, and a header it can read is judged on its stored values. A plain `lookup` to an object an organization wall scopes asks no read door, system caller or not | `packages/objectql/src/engine.ts#resolvePredicateRelated` |
140+
| 29b | A traversing validation rule's related read skips the **caller's bounds** | objectql | Get: no row bound from the caller's own read on a related object no organization wall scopes, no read door on a header the rule reads through a `master_detail` field, and a related read even with no organization under a walled posture (a `tenantId` on the context still scopes it). Every caller that is not system is held to all three: a header its read door does not return is unavailable to the rule exactly as a header that exists nowhere is, whatever it holds, and a header it can read is judged on its stored values. A plain `lookup` to an object an organization wall scopes asks no read door, system caller or not | `packages/objectql/src/engine.ts#resolveOneHopRelated` |
141141
| 29c | A detail write's master-detail header read asks **no read door** | objectql | Get: the write binds the header a parent-scoped `requiredWhen` / `readonlyWhen` reads as `parent` from the header's stored row whoever can read it, as before. Every caller that is not system meets its own read door first: a header that read does not return answers as a header that exists nowhere — an unbound `parent`, and `reference_not_found` on the reference check (which a system caller's write skips altogether) | `packages/objectql/src/engine.ts#masterDetailHeadersCallerReads` |
142142
| 30 | **Bulk data event `organizationId` OMITTED** — the batch is published "not asserted" | plugin-security | Get: nothing — the `data.records.*` event still publishes. Lose: the per-organization attribution: this exit is taken before the security middleware composes any tenant wall, so it records no Layer 0 verdict on the operation (`OperationContext.tenantLayer0Verdict`, #15813), and the engine's bulk producer — which reads that recorded verdict and nothing else — omits the key rather than filling it from the caller's `tenantId`; a tenant-scoped consumer then does not deliver the event inside an organization wall (#15225) | `packages/plugins/plugin-security/src/security-plugin.ts#start` |
143143

0 commit comments

Comments
 (0)