Repository navigation
Commit 87e902d
feat(objectql,core): the engine composes the ReadableIds leaf at its executor boundary, fail closed, and refuses it by name where it cannot (ADR-0139 E2) (#23301)
Fixes #22861
Clause-②: no
ADR-0139 E2 (part of the execution plan in #22788), built on ruling
6121741984 (letter A, both sub-questions) and the seat answers
6123453723. The engine composes every `ReadableIds` leaf once per parent
object, at the executor boundary, on the final AST, immediately before
each read verb's driver call. It fails closed, and it refuses the leaf
by name wherever it cannot be composed.
## What changes
**Composed** (`packages/objectql/src/engine.ts`,
`composeReadableIdsAtBoundary`). On `find`, `findOne`, `count` and
`aggregate`, the composition runs after the comparand doors, the
middleware chain and the `beforeFind` hooks, immediately before the
driver call. `aggregate` composes once, for both of its exclusive driver
paths. Each `{ FIELD: { $in: ReadableIds.of(P) } }` in `where` becomes
`ReadableIds.compose(leaf, scope)` or `$in: []`. For each parent object
`P`, once per read, in this order:
1. D5's three classes are refused by name, before any security call: a
delegated (on-behalf-of) caller (`context.onBehalfOf.userId`); a parent
with an object-scoped read middleware (`hasObjectMiddleware(P)`); a
parent bound to another datasource than the object read
(`resolveDatasourceBinding`).
2. A parent the registry does not hold is answered `$in: []`, and
nothing is asked.
3. The `security` service is absent or unusable (no source, a source
that answers nothing or throws, no `getReadFilter`, neither
`canReadObject` nor `explain`) ⇒ `$in: []`. A `warn` is logged once per
engine for each of the two states.
4. `canReadObject(P, caller)` false or a throw ⇒ `$in: []`. A service
that predates the optional method is asked `explain`, and only `allowed:
true` admits.
5. `getReadFilter(P, caller)`: `undefined` ⇒ composed with no row
restriction. `RLS_DENY_FILTER`, recognised by identity, or a throw ⇒
`$in: []`. Any other filter ⇒ composed with it as the scope, by
reference and marked `'policy'`, otherwise passed exactly as `P`'s own
read passes its injected scope (seat answer 6123453723, finding 2). The
tenant wall is the driver's (D3).
Nothing is memoised across reads. The copy-on-write rewrite leaves the
hooks' `input.ast` untouched, carries each rewritten node's provenance
mark, and returns the AST by reference when it holds no leaf, which is
every read today.
**Refused** with `INTERNAL_ERROR` / 500 (the ADR-0112 envelope), naming
the position and the leaf, before any driver is asked
(`packages/objectql/src/filter-comparand-shape.ts`,
`collectComposableReadableIds`, one rule asked at two points):
- `update` and `delete`. The caller's leaf is refused at the comparand
door. A leaf the middleware chain writes is refused at the executor's
entry, before the prior-row reads reach the driver (seat finding 3).
- `having` and an aggregation's own `filter`: at the door, and again at
the boundary for a leaf a middleware wrote.
- Any position but a `$in` comparand. This includes a leaf that the
array form of `where` lowers into another position
(`lowerWhereFilterArray`'s array branch now asks the same rule). The
array sugar's own `in` spelling stays `parseFilterAST`'s
`INVALID_FILTER` / 400, because that lowering does not opt in.
- A composed leaf the engine did not compose, written by a middleware or
a hook. The caller's own composed leaf in a `$in` slot stays the
comparand door's 400.
The interim refusal `refuseUncomposedReadableIds` and its "does not
compose that leaf yet" wording are gone. The door still admits the `$in`
leaf on the read verbs (ruling A).
**Wired** (`packages/objectql/src/plugin.ts`). `ObjectQLPlugin.init()`
writes a source, `() => ctx.getService('security')`, into the engine's
symbol-keyed slot `SECURITY_SERVICE_SOURCE_SLOT`. That is a `Symbol.for`
key, not a member of `ObjectQL`, and neither package entry re-exports
it, so the published surface does not grow (contract review 6150749585,
②; seat direction 6150767225, arm ii). The service is read when a leaf
is composed, never at init (seat answer Q2). plugin-security registers
the service in its own `start()`.
**Branded by reference** (`packages/core/src/utils/filter-tokens.ts`,
ruled sub-question 1). `resolveFilterTokens` returns a `ReadableIds`
leaf by reference. Before this change, a leaf beside a resolved
placeholder was rebuilt as an unbranded `{ object }`.
The landing point is the one the ruling names. No producer was found
elsewhere.
## The premise, re-measured on this head (fork clause not tripped)
These are source readings on `693b450c90`.
- `find` and `findOne`: the composition sits between
`triggerHooks('beforeFind')` plus `buildDriverOptions` and `driver.find`
/ `driver.findOne`. Nothing between them rewrites the AST.
- `count`: the composition sits immediately before `driver.count`.
`count` has no before-hook, and its driver-without-`count` fallback
calls `find`, which composes at its own boundary.
- `aggregate`: the native `drv.aggregate(ast)` and the rows path's
`driver.find({ ...ast })` are exclusive and consume the one composed
AST.
- The predicate writes: the update and delete hooks receive no AST, so
the AST is final once the chain returns. The refusal sits at the
executor's entry, ahead of `readPriorRows`, the delete pre-image read,
`updateMany` and `deleteMany`.
- D5's checks read the context, the registry and the middleware
registrations, so a parent costs one `canReadObject` (or `explain`) and
one `getReadFilter` per read. This is pinned by "each parent is asked
once per read".
## The three notes from #23009 (6137591851): populations, each pinned
fail-safe, no fast path added
The pin evaluates each composed scope with `@objectstack/formula`'s
`matchesFilterCondition` (the JS two-valued surface) over parent rows.
Each scope matches no row. The control is that the non-deny layer alone
matches.
- **(a) A deny ANDed with another layer.** `getReadFilter` returns
`andComposeLayers(andComposeLayers(rls, cbp), sharing)`. When the RLS
layer is the deny and the controlled-by-parent or the sharing layer also
constrains, the answer is `{ $and: [RLS_DENY_FILTER, …] }`. One case is
an org-less caller on a tenant object whose sharing model is private.
The population is per deployment: objects with a sharing predicate or
controlled-by-parent, read by a caller whose RLS layer denies. The
answer composes to a subquery whose scope keeps the sentinel arm.
- **(b) Dual package.** `@objectstack/spec`, `@objectstack/objectql` and
`@objectstack/plugin-security` are `type: commonjs` packages with an
`import` entry to `.mjs`. `@objectstack/core`, `@objectstack/cli` and
`@objectstack/runtime` are `type: module`. An ESM host loads all of them
through `import`, and a CJS host through `require`, so identity holds
within one format. Only a host that loads the engine and the security
plugin through different conditions misses the identity check. Pinned: a
copy of the sentinel composes to a scope that matches no row.
- **(c) Sharing's `{ id: '__deny_all__' }`.** `buildReadFilter` answers
it for a non-system context that carries a position or a permission set
but no user id, on a non-public object with an owner field.
plugin-security answers a context with no principal at all with the
sentinel first. The scope is composed as an ordinary scope. Pinned: no
row.
No change was made in `plugin-security` or `plugin-sharing`.
## Compile surfaces
(`.claude/skills/pm-dispatch/references/compile-surfaces.md`)
| surface | conclusion |
|:--|:--|
| 1 `driver-sql` (and its wasm and local-Turso heirs) | Out of scope. It
compiles the composed leaf in E3 (ADR-0139 D3). This PR hands it the
composed leaf, which no producer writes today. |
| 2 Turso `RemoteTransport` | Out of scope. It is E3's, as above. |
| 3 analytics `read-scope-sql` | Already compliant. The leaf never
reaches it: only the engine's door passes `admitReadableIds` (`git grep
-n admitReadableIds -- 'packages/**/*.ts'` finds one non-test caller,
`packages/objectql/src/filter-comparand-shape.ts`), so analytics filters
refuse the leaf 400 at the spec door. |
| 4 analytics `filter-normalizer` | Already compliant, for the same
reason as 3. |
| 5 `formula` `matchesFilterCondition` | Out of scope. It evaluates
authored filters, where no author can spell the leaf (D1). It is used
here only as the oracle of the fail-safe pin. |
| half `objectql` `having-filter` | Changed. `having` refuses the leaf
at the door and at the boundary (pinned, including a middleware-written
`having`). |
| unfrozen `driver-memory` / `driver-mongodb` | Out of scope. They are
E4's (memory implements, mongodb refuses). |
## Pins and the ablation of each arm
New: `engine-readable-ids-composition.test.ts` (23 cases) and
`plugin-readable-ids-security-source.test.ts` (1). Rewritten for E2:
`engine-readable-ids-door.test.ts`. In `filter-tokens.test.ts`: 1 case.
Every refusal pin asserts `code` and `status`, plus the position or the
class it names.
Each ablation was run through `scripts/ablation-replace.mjs` (WRAP mode:
the anchor must hit, and the restore is proven blob == HEAD with `git
diff HEAD` empty). They ran on `3653f70150` and `87545e5a0e`, whose
subject files are byte-identical to this head. Every ablation turned its
target pin red:
| arm | red |
|:--|:--|
| find / findOne / count / aggregate boundary call (4) | each read
verb's composition pins (16 / 4 / 4 / 6 failed) |
| deny-sentinel identity arm | the deny sentinel ⇒ `$in: []` |
| `canReadObject` false arm | `canReadObject` false; the `explain`
fallback |
| absent or unusable service fails closed | no usable security service;
read at use |
| `'policy'` mark | each read verb (scope marked `'policy'`) |
| update / delete executor-entry refusal (2) | the write-verb pin |
| `having` at the boundary | the `having` and aggregation-filter pin |
| D5 delegated / middleware-gated / federated (3) | its own D5 pin |
| unregistered parent | the unregistered-parent pin |
| array-branch rule | the array-branch pin |
| once per parent | each parent asked once; the unregistered-parent pin
|
| forged composed leaf | the forged composed-leaf pin |
| position arm | three position pins |
| door's write-verb arm | the door pin (refused before the chain) |
| plugin source | the plugin wiring pin |
| `filter-tokens` by reference (core `src`) | core's by-reference pin |
| `filter-tokens` by reference, through `@objectstack/core`'s `dist/` |
the engine-level "beside a resolved placeholder" pin |
The last row is a dist-resolved ablation. Core was rebuilt, and
`ablation-dist-preflight.mjs` reported the marker absent from
`packages/core/dist`; the pin failed (1 failed). Core was then restored,
rebuilt and preflighted with the marker present in `dist/index.js`, and
the pin passed. The first count-boundary ablation was a no-op that the
tool refused (its replacement text already occurred 7 times, so the
count could not rise). It was rerun with a unique replacement and went
red.
## Tests (on `693b450c90` unless stated)
- **Patch round on `586d7b57ff`** (seat direction 6150767225):
- **The seam is off the published surface.** After the build,
`registerSecurityServiceSource`, `SECURITY_SERVICE_SOURCE_SLOT` and
`securityServiceSource` have 0 hits in `dist/index.d.ts`,
`dist/core.d.ts`, their `.d.mts` twins and the shared declaration chunk.
The controls `registerEffectiveObjectPermissionsResolver` (1) and
`registerWriteGateProbe` (2) still hit in that chunk.
- **The census row.** `Test Core (3/6)`'s red at `693b450c90` was
`packages/core`'s `row-serving-door-exposure.pin.test.ts`, which
classifies every non-test caller of `canReadObject`. `engine.ts` is now
classified `not-a-door`: it asks a parent's admission only to compose
the scope of a `$in` leaf narrowing a read whose own door already
admitted it. No parent row or field is served, and a refused parent
answers `$in: []`. Core `test:repo` went from 1 failed / 82 passed to 6
files and 83 tests passed. ⛔ No test was skipped or loosened.
- **The slot-read ablation.** The slot read was replaced by a read of
the old method's field, which nothing writes: 11 failed / 13 passed,
including the plugin source pins. The restore is proven by blob.
- **Re-runs:** both typechecks; the focused suites (6 files, 60 tests,
plus core `filter-tokens` with 52); objectql `test:repo` (5); and the
derived gates: 79 derived, 78 run at exit 0, 1 NOT-MEASURED
(`check:dual-build-cjs-loads`, left to CI).
- **The three engine test doubles** are back to byte-identical with
`main`.
- `@objectstack/objectql` `typecheck` (tsc plus the test layer: 40
files, 234 errors, 65 pinned signatures, unchanged): exit 0.
`@objectstack/core` `typecheck`: exit 0.
- `@objectstack/core` local suite: 95 files, 2,402 tests passed.
- `@objectstack/objectql` local suite, run in four chunks to fit the
foreground cap: 106 + 100 + 107 + 100 = 413 files, 8,060 tests passed;
the `repo` project 1 file, 5 tests. Measured on `87545e5a0e`, whose
`packages/objectql` is identical to this head. The later merge moved
only core's security and tenancy modules and two spec data schemas.
After it: the focused set (10 files, 211 tests) and both typechecks
again on this head.
- Derived gates (`dispatch-gates --commands --repo
objectstack-ai/objectstack`, 79 commands, the same list on `87545e5a0e`
and `693b450c90`): 78 run at exit 0 on `693b450c90`, the ratchet
families included (`check:query-options-erasure`, `check:slot-lookup`,
`check:type-check-coverage`, and `check:type-check-debt` with a 607 s
re-measure). `check:dual-build-cjs-loads` exited 3 (PREREQUISITE NOT
MET: it needs every package's `dist/`), so it is NOT MEASURED and left
to CI. `dispatch-gates --ran` reads: 79 derived, 78 run, 1 NOT-MEASURED
(derived from the recorded exit 3), 0 UNRUN.
- `main` moved by three commits after `693b450c90` (metadata-protocol,
rest, plugin-auth tests, the spec api protocol schema). None of them
touches `packages/objectql`, `packages/core` or a path of this diff, and
`git merge-tree` against it is clean. The branch is not re-merged, so
the readings above stay on one head. CI runs on the merge ref.
## Acceptance notes (not filed)
- **The unregistered-parent answer.** A parent the registry does not
hold is answered `$in: []`, unasked. ADR-0139 D4 says rows naming an
unregistered object are excluded, and the emitters name registered
parents only.
- **Over-inclusion under D5.2.** `hasObjectMiddleware` also counts
plugin-security's own object-scoped doors (`sys_position`,
`sys_permission_set`, `sys_capability` and the position-assignment
object), so a leaf over those parents is refused under D5.2. The record
accepts that over-inclusion.
- **No live exposure.** No driver evaluates the composed leaf until
E3/E4, and nothing in the platform writes a `ReadableIds` leaf today:
`git grep "ReadableIds.of("` finds only doc text and error prose outside
tests.
- **An unverified inference.** `resolveFilterTokens` exempts only `Date`
among class instances when it rebuilds a tree that holds a placeholder.
This was read from the code, not reproduced. Carrier: none.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01P88ehrZ6hM19hBzdpsQTTo)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 404fc47 commit 87e902d
11 files changed
Lines changed: 1121 additions & 115 deletions
File tree
- .changeset
- packages
- core/src
- security
- utils
- objectql/src
- scripts/adr-anchors
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
95 | 95 | | |
96 | 96 | | |
97 | 97 | | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
98 | 105 | | |
99 | 106 | | |
100 | 107 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
296 | 297 | | |
297 | 298 | | |
298 | 299 | | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
299 | 315 | | |
300 | 316 | | |
301 | 317 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
| 109 | + | |
109 | 110 | | |
110 | 111 | | |
111 | 112 | | |
| |||
505 | 506 | | |
506 | 507 | | |
507 | 508 | | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
508 | 512 | | |
509 | 513 | | |
510 | 514 | | |
| |||
518 | 522 | | |
519 | 523 | | |
520 | 524 | | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
521 | 532 | | |
522 | 533 | | |
523 | 534 | | |
| |||
542 | 553 | | |
543 | 554 | | |
544 | 555 | | |
| 556 | + | |
| 557 | + | |
545 | 558 | | |
546 | 559 | | |
547 | 560 | | |
| |||
0 commit comments