Skip to content

Commit 87e902d

Browse files
feat(objectql,core): the engine composes the ReadableIds leaf at its executor boundary, fail closed, and refuses it by name where it cannot (ADR-0139 E2) (#23301)
Fixes #22861 Clause-②: no ADR-0139 E2 (part of the execution plan in #22788), built on ruling 6121741984 (letter A, both sub-questions) and the seat answers 6123453723. The engine composes every `ReadableIds` leaf once per parent object, at the executor boundary, on the final AST, immediately before each read verb's driver call. It fails closed, and it refuses the leaf by name wherever it cannot be composed. ## What changes **Composed** (`packages/objectql/src/engine.ts`, `composeReadableIdsAtBoundary`). On `find`, `findOne`, `count` and `aggregate`, the composition runs after the comparand doors, the middleware chain and the `beforeFind` hooks, immediately before the driver call. `aggregate` composes once, for both of its exclusive driver paths. Each `{ FIELD: { $in: ReadableIds.of(P) } }` in `where` becomes `ReadableIds.compose(leaf, scope)` or `$in: []`. For each parent object `P`, once per read, in this order: 1. D5's three classes are refused by name, before any security call: a delegated (on-behalf-of) caller (`context.onBehalfOf.userId`); a parent with an object-scoped read middleware (`hasObjectMiddleware(P)`); a parent bound to another datasource than the object read (`resolveDatasourceBinding`). 2. A parent the registry does not hold is answered `$in: []`, and nothing is asked. 3. The `security` service is absent or unusable (no source, a source that answers nothing or throws, no `getReadFilter`, neither `canReadObject` nor `explain`) ⇒ `$in: []`. A `warn` is logged once per engine for each of the two states. 4. `canReadObject(P, caller)` false or a throw ⇒ `$in: []`. A service that predates the optional method is asked `explain`, and only `allowed: true` admits. 5. `getReadFilter(P, caller)`: `undefined` ⇒ composed with no row restriction. `RLS_DENY_FILTER`, recognised by identity, or a throw ⇒ `$in: []`. Any other filter ⇒ composed with it as the scope, by reference and marked `'policy'`, otherwise passed exactly as `P`'s own read passes its injected scope (seat answer 6123453723, finding 2). The tenant wall is the driver's (D3). Nothing is memoised across reads. The copy-on-write rewrite leaves the hooks' `input.ast` untouched, carries each rewritten node's provenance mark, and returns the AST by reference when it holds no leaf, which is every read today. **Refused** with `INTERNAL_ERROR` / 500 (the ADR-0112 envelope), naming the position and the leaf, before any driver is asked (`packages/objectql/src/filter-comparand-shape.ts`, `collectComposableReadableIds`, one rule asked at two points): - `update` and `delete`. The caller's leaf is refused at the comparand door. A leaf the middleware chain writes is refused at the executor's entry, before the prior-row reads reach the driver (seat finding 3). - `having` and an aggregation's own `filter`: at the door, and again at the boundary for a leaf a middleware wrote. - Any position but a `$in` comparand. This includes a leaf that the array form of `where` lowers into another position (`lowerWhereFilterArray`'s array branch now asks the same rule). The array sugar's own `in` spelling stays `parseFilterAST`'s `INVALID_FILTER` / 400, because that lowering does not opt in. - A composed leaf the engine did not compose, written by a middleware or a hook. The caller's own composed leaf in a `$in` slot stays the comparand door's 400. The interim refusal `refuseUncomposedReadableIds` and its "does not compose that leaf yet" wording are gone. The door still admits the `$in` leaf on the read verbs (ruling A). **Wired** (`packages/objectql/src/plugin.ts`). `ObjectQLPlugin.init()` writes a source, `() => ctx.getService('security')`, into the engine's symbol-keyed slot `SECURITY_SERVICE_SOURCE_SLOT`. That is a `Symbol.for` key, not a member of `ObjectQL`, and neither package entry re-exports it, so the published surface does not grow (contract review 6150749585, ②; seat direction 6150767225, arm ii). The service is read when a leaf is composed, never at init (seat answer Q2). plugin-security registers the service in its own `start()`. **Branded by reference** (`packages/core/src/utils/filter-tokens.ts`, ruled sub-question 1). `resolveFilterTokens` returns a `ReadableIds` leaf by reference. Before this change, a leaf beside a resolved placeholder was rebuilt as an unbranded `{ object }`. The landing point is the one the ruling names. No producer was found elsewhere. ## The premise, re-measured on this head (fork clause not tripped) These are source readings on `693b450c90`. - `find` and `findOne`: the composition sits between `triggerHooks('beforeFind')` plus `buildDriverOptions` and `driver.find` / `driver.findOne`. Nothing between them rewrites the AST. - `count`: the composition sits immediately before `driver.count`. `count` has no before-hook, and its driver-without-`count` fallback calls `find`, which composes at its own boundary. - `aggregate`: the native `drv.aggregate(ast)` and the rows path's `driver.find({ ...ast })` are exclusive and consume the one composed AST. - The predicate writes: the update and delete hooks receive no AST, so the AST is final once the chain returns. The refusal sits at the executor's entry, ahead of `readPriorRows`, the delete pre-image read, `updateMany` and `deleteMany`. - D5's checks read the context, the registry and the middleware registrations, so a parent costs one `canReadObject` (or `explain`) and one `getReadFilter` per read. This is pinned by "each parent is asked once per read". ## The three notes from #23009 (6137591851): populations, each pinned fail-safe, no fast path added The pin evaluates each composed scope with `@objectstack/formula`'s `matchesFilterCondition` (the JS two-valued surface) over parent rows. Each scope matches no row. The control is that the non-deny layer alone matches. - **(a) A deny ANDed with another layer.** `getReadFilter` returns `andComposeLayers(andComposeLayers(rls, cbp), sharing)`. When the RLS layer is the deny and the controlled-by-parent or the sharing layer also constrains, the answer is `{ $and: [RLS_DENY_FILTER, …] }`. One case is an org-less caller on a tenant object whose sharing model is private. The population is per deployment: objects with a sharing predicate or controlled-by-parent, read by a caller whose RLS layer denies. The answer composes to a subquery whose scope keeps the sentinel arm. - **(b) Dual package.** `@objectstack/spec`, `@objectstack/objectql` and `@objectstack/plugin-security` are `type: commonjs` packages with an `import` entry to `.mjs`. `@objectstack/core`, `@objectstack/cli` and `@objectstack/runtime` are `type: module`. An ESM host loads all of them through `import`, and a CJS host through `require`, so identity holds within one format. Only a host that loads the engine and the security plugin through different conditions misses the identity check. Pinned: a copy of the sentinel composes to a scope that matches no row. - **(c) Sharing's `{ id: '__deny_all__' }`.** `buildReadFilter` answers it for a non-system context that carries a position or a permission set but no user id, on a non-public object with an owner field. plugin-security answers a context with no principal at all with the sentinel first. The scope is composed as an ordinary scope. Pinned: no row. No change was made in `plugin-security` or `plugin-sharing`. ## Compile surfaces (`.claude/skills/pm-dispatch/references/compile-surfaces.md`) | surface | conclusion | |:--|:--| | 1 `driver-sql` (and its wasm and local-Turso heirs) | Out of scope. It compiles the composed leaf in E3 (ADR-0139 D3). This PR hands it the composed leaf, which no producer writes today. | | 2 Turso `RemoteTransport` | Out of scope. It is E3's, as above. | | 3 analytics `read-scope-sql` | Already compliant. The leaf never reaches it: only the engine's door passes `admitReadableIds` (`git grep -n admitReadableIds -- 'packages/**/*.ts'` finds one non-test caller, `packages/objectql/src/filter-comparand-shape.ts`), so analytics filters refuse the leaf 400 at the spec door. | | 4 analytics `filter-normalizer` | Already compliant, for the same reason as 3. | | 5 `formula` `matchesFilterCondition` | Out of scope. It evaluates authored filters, where no author can spell the leaf (D1). It is used here only as the oracle of the fail-safe pin. | | half `objectql` `having-filter` | Changed. `having` refuses the leaf at the door and at the boundary (pinned, including a middleware-written `having`). | | unfrozen `driver-memory` / `driver-mongodb` | Out of scope. They are E4's (memory implements, mongodb refuses). | ## Pins and the ablation of each arm New: `engine-readable-ids-composition.test.ts` (23 cases) and `plugin-readable-ids-security-source.test.ts` (1). Rewritten for E2: `engine-readable-ids-door.test.ts`. In `filter-tokens.test.ts`: 1 case. Every refusal pin asserts `code` and `status`, plus the position or the class it names. Each ablation was run through `scripts/ablation-replace.mjs` (WRAP mode: the anchor must hit, and the restore is proven blob == HEAD with `git diff HEAD` empty). They ran on `3653f70150` and `87545e5a0e`, whose subject files are byte-identical to this head. Every ablation turned its target pin red: | arm | red | |:--|:--| | find / findOne / count / aggregate boundary call (4) | each read verb's composition pins (16 / 4 / 4 / 6 failed) | | deny-sentinel identity arm | the deny sentinel ⇒ `$in: []` | | `canReadObject` false arm | `canReadObject` false; the `explain` fallback | | absent or unusable service fails closed | no usable security service; read at use | | `'policy'` mark | each read verb (scope marked `'policy'`) | | update / delete executor-entry refusal (2) | the write-verb pin | | `having` at the boundary | the `having` and aggregation-filter pin | | D5 delegated / middleware-gated / federated (3) | its own D5 pin | | unregistered parent | the unregistered-parent pin | | array-branch rule | the array-branch pin | | once per parent | each parent asked once; the unregistered-parent pin | | forged composed leaf | the forged composed-leaf pin | | position arm | three position pins | | door's write-verb arm | the door pin (refused before the chain) | | plugin source | the plugin wiring pin | | `filter-tokens` by reference (core `src`) | core's by-reference pin | | `filter-tokens` by reference, through `@objectstack/core`'s `dist/` | the engine-level "beside a resolved placeholder" pin | The last row is a dist-resolved ablation. Core was rebuilt, and `ablation-dist-preflight.mjs` reported the marker absent from `packages/core/dist`; the pin failed (1 failed). Core was then restored, rebuilt and preflighted with the marker present in `dist/index.js`, and the pin passed. The first count-boundary ablation was a no-op that the tool refused (its replacement text already occurred 7 times, so the count could not rise). It was rerun with a unique replacement and went red. ## Tests (on `693b450c90` unless stated) - **Patch round on `586d7b57ff`** (seat direction 6150767225): - **The seam is off the published surface.** After the build, `registerSecurityServiceSource`, `SECURITY_SERVICE_SOURCE_SLOT` and `securityServiceSource` have 0 hits in `dist/index.d.ts`, `dist/core.d.ts`, their `.d.mts` twins and the shared declaration chunk. The controls `registerEffectiveObjectPermissionsResolver` (1) and `registerWriteGateProbe` (2) still hit in that chunk. - **The census row.** `Test Core (3/6)`'s red at `693b450c90` was `packages/core`'s `row-serving-door-exposure.pin.test.ts`, which classifies every non-test caller of `canReadObject`. `engine.ts` is now classified `not-a-door`: it asks a parent's admission only to compose the scope of a `$in` leaf narrowing a read whose own door already admitted it. No parent row or field is served, and a refused parent answers `$in: []`. Core `test:repo` went from 1 failed / 82 passed to 6 files and 83 tests passed. ⛔ No test was skipped or loosened. - **The slot-read ablation.** The slot read was replaced by a read of the old method's field, which nothing writes: 11 failed / 13 passed, including the plugin source pins. The restore is proven by blob. - **Re-runs:** both typechecks; the focused suites (6 files, 60 tests, plus core `filter-tokens` with 52); objectql `test:repo` (5); and the derived gates: 79 derived, 78 run at exit 0, 1 NOT-MEASURED (`check:dual-build-cjs-loads`, left to CI). - **The three engine test doubles** are back to byte-identical with `main`. - `@objectstack/objectql` `typecheck` (tsc plus the test layer: 40 files, 234 errors, 65 pinned signatures, unchanged): exit 0. `@objectstack/core` `typecheck`: exit 0. - `@objectstack/core` local suite: 95 files, 2,402 tests passed. - `@objectstack/objectql` local suite, run in four chunks to fit the foreground cap: 106 + 100 + 107 + 100 = 413 files, 8,060 tests passed; the `repo` project 1 file, 5 tests. Measured on `87545e5a0e`, whose `packages/objectql` is identical to this head. The later merge moved only core's security and tenancy modules and two spec data schemas. After it: the focused set (10 files, 211 tests) and both typechecks again on this head. - Derived gates (`dispatch-gates --commands --repo objectstack-ai/objectstack`, 79 commands, the same list on `87545e5a0e` and `693b450c90`): 78 run at exit 0 on `693b450c90`, the ratchet families included (`check:query-options-erasure`, `check:slot-lookup`, `check:type-check-coverage`, and `check:type-check-debt` with a 607 s re-measure). `check:dual-build-cjs-loads` exited 3 (PREREQUISITE NOT MET: it needs every package's `dist/`), so it is NOT MEASURED and left to CI. `dispatch-gates --ran` reads: 79 derived, 78 run, 1 NOT-MEASURED (derived from the recorded exit 3), 0 UNRUN. - `main` moved by three commits after `693b450c90` (metadata-protocol, rest, plugin-auth tests, the spec api protocol schema). None of them touches `packages/objectql`, `packages/core` or a path of this diff, and `git merge-tree` against it is clean. The branch is not re-merged, so the readings above stay on one head. CI runs on the merge ref. ## Acceptance notes (not filed) - **The unregistered-parent answer.** A parent the registry does not hold is answered `$in: []`, unasked. ADR-0139 D4 says rows naming an unregistered object are excluded, and the emitters name registered parents only. - **Over-inclusion under D5.2.** `hasObjectMiddleware` also counts plugin-security's own object-scoped doors (`sys_position`, `sys_permission_set`, `sys_capability` and the position-assignment object), so a leaf over those parents is refused under D5.2. The record accepts that over-inclusion. - **No live exposure.** No driver evaluates the composed leaf until E3/E4, and nothing in the platform writes a `ReadableIds` leaf today: `git grep "ReadableIds.of("` finds only doc text and error prose outside tests. - **An unverified inference.** `resolveFilterTokens` exempts only `Date` among class instances when it rebuilds a tree that holds a placeholder. This was read from the code, not reproduced. Carrier: none. --- _Generated by [Claude Code](https://claude.ai/code/session_01P88ehrZ6hM19hBzdpsQTTo)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 404fc47 commit 87e902d

11 files changed

Lines changed: 1121 additions & 115 deletions
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
---
2+
'@objectstack/objectql': minor
3+
'@objectstack/core': patch
4+
---
5+
6+
feat(objectql): the engine composes a `ReadableIds` leaf at its executor boundary, on the final query immediately before the driver call, and refuses it by name wherever it cannot (ADR-0139 D2, D5)
7+
8+
Clause-②: no
9+
10+
- **Composed.** On `find`, `findOne`, `count` and `aggregate`, after the middleware chain and the `beforeFind` hooks, every `{ FIELD: { $in: ReadableIds.of(P) } }` in `where` is replaced by `ReadableIds.compose(leaf, scope)`, where `scope` is the registered security service's `getReadFilter(P, caller)`, marked `'policy'` (`undefined` composes with no row restriction). The comparand becomes `$in: []`, which matches no row, when:
11+
- `canReadObject(P, caller)` is false or throws (a service without `canReadObject` is asked `explain`, and only `allowed: true` admits);
12+
- `getReadFilter` answers `RLS_DENY_FILTER` (by identity) or throws;
13+
- no security service is registered, or the registered one cannot answer both halves (the engine reads the kernel's `security` service through `ObjectQLPlugin` when a leaf is composed, so an engine used without that plugin has none);
14+
- the registry holds no object `P`.
15+
16+
Each parent object is asked once per read, and nothing is kept across reads. A leaf that a middleware or a `beforeFind` hook writes is composed the same way.
17+
- **Refused, `INTERNAL_ERROR` / 500, before any driver is asked:**
18+
- on `update` and `delete`: the caller's leaf at the comparand door, a middleware's at the executor's entry, before the prior-row reads;
19+
- in `having` and in an aggregation's own `filter`;
20+
- anywhere but a `$in` comparand, including a leaf the array form of `where` lowers into another position;
21+
- a composed leaf the engine did not compose, written by a middleware or a hook (the caller's own, as a `$in` comparand, stays the comparand door's `INVALID_FILTER` / 400);
22+
- ADR-0139 D5's three classes: a delegated (on-behalf-of) caller, a parent with an object-scoped read middleware, and a parent bound to another datasource than the object read.
23+
- **Before:** the comparand door refused every `ReadableIds` leaf `INTERNAL_ERROR` / 500 ("this engine does not compose that leaf yet"), and a leaf that a middleware or a hook wrote reached the driver unrefused.
24+
- **`@objectstack/core`:** `resolveFilterTokens` returns a `ReadableIds` leaf by reference. When the filter also held a placeholder, it used to rebuild the leaf as a plain `{ object }` and drop its brand.
25+
- **Not yet:** no driver evaluates the composed leaf (ADR-0139 E3 and E4), and no code in the platform writes a `ReadableIds` leaf today.

‎packages/core/src/security/row-serving-door-exposure.pin.test.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,13 @@ const CALLERS: Record<string, Classification> = {
9595
kind: 'not-a-door',
9696
reason: 'the provider: it implements the admission and binds it into the security service it registers; it serves no rows',
9797
},
98+
'packages/objectql/src/engine.ts': {
99+
kind: 'not-a-door',
100+
reason:
101+
'the ReadableIds composition (ADR-0139 D2): it asks the admission of a PARENT object to compose the scope of a `$in` leaf '
102+
+ 'that only narrows the read the caller addressed, whose own door already admitted it; no row and no field of the '
103+
+ 'parent is served, and a refused parent answers `$in: []`',
104+
},
98105
};
99106

100107
const SCAN_TIMEOUT_MS = 60_000;

‎packages/core/src/utils/filter-tokens.test.ts‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
22

33
import { describe, it, expect } from 'vitest';
4+
import { ReadableIds, isReadableIds } from '@objectstack/spec/data';
45
import {
56
resolveFilterToken,
67
resolveFilterTokens,
@@ -296,6 +297,21 @@ describe('resolveFilterTokens — tree walk', () => {
296297
it('accepts the `${token}` spelling', () => {
297298
expect(resolveFilterTokens({ d: '${30_days_ago}' }, ctx)).toEqual({ d: '2026-06-15' });
298299
});
300+
301+
// [ADR-0139 D1] The tree around a placeholder is rebuilt; a `ReadableIds`
302+
// leaf in it is not. Its brand is a non-enumerable symbol key, so a copy of
303+
// it is an unbranded `{ object }` that the engine's executor boundary cannot
304+
// compose.
305+
it('returns a ReadableIds leaf beside a resolved placeholder by reference, brand intact', () => {
306+
const leaf = ReadableIds.of('cpz_doc');
307+
const out = resolveFilterTokens(
308+
{ owner: '{current_user_id}', $or: [{ record_id: { $in: leaf } }, { note: 'x' }] },
309+
ctx,
310+
) as { owner: string; $or: Array<{ record_id?: { $in: unknown } }> };
311+
expect(out.owner).toBe('usr_1');
312+
expect(out.$or[0].record_id!.$in).toBe(leaf);
313+
expect(isReadableIds(out.$or[0].record_id!.$in)).toBe(true);
314+
});
299315
});
300316

301317
/**

‎packages/core/src/utils/filter-tokens.ts‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,7 @@
106106

107107
import {
108108
classifyFilterToken,
109+
isReadableIds,
109110
isRecordContextToken,
110111
parseDateMacroParam,
111112
type DateMacroUnit,
@@ -505,6 +506,9 @@ export function resolveFilterToken(
505506
function hasFilterToken(node: unknown): boolean {
506507
if (typeof node === 'string') return classifyFilterToken(node) !== null;
507508
if (Array.isArray(node)) return node.some(hasFilterToken);
509+
// A `ReadableIds` leaf is a comparand value: its `object` names a table and
510+
// is never a placeholder (see `resolveFilterTokens`).
511+
if (isReadableIds(node)) return false;
508512
if (node && typeof node === 'object' && !(node instanceof Date)) {
509513
return Object.values(node as Record<string, unknown>).some(hasFilterToken);
510514
}
@@ -518,6 +522,13 @@ function hasFilterToken(node: unknown): boolean {
518522
* bake one request's user id, and one day's dates, into every later render).
519523
*
520524
* Returns the input unchanged, by reference, when it holds no placeholders.
525+
*
526+
* [ADR-0139 D1] A branded `ReadableIds` leaf is returned BY REFERENCE, never
527+
* rebuilt. Its brand is a non-enumerable symbol key, so the plain-object copy
528+
* below would hand back an unbranded `{ object }`: a leaf beside a resolved
529+
* placeholder would then reach the engine's executor boundary as a plain
530+
* object in a `$in` comparand, which the engine cannot compose. The leaf holds
531+
* no placeholder and is frozen, so returning it unchanged is exact.
521532
*/
522533
export function resolveFilterTokens<T>(
523534
filter: T,
@@ -542,6 +553,8 @@ export function resolveFilterTokens<T>(
542553
return resolved;
543554
}
544555
if (Array.isArray(node)) return node.map(walk);
556+
// [ADR-0139 D1] The leaf by reference, brand intact (see above).
557+
if (isReadableIds(node)) return node;
545558
if (node && typeof node === 'object') {
546559
// Dates and other class instances are comparands, not filter structure.
547560
if (node instanceof Date) return node;

0 commit comments

Comments
 (0)