refactor(objectql): move file-field hydration on read out of engine.ts into engine-file-reference-hydration.ts (#25522 cut 11) (#26351)

<!-- claude-projects-attribution: {"github_login":"os-project-manager"}
-->
_Requested by **Leehom** · [project
thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1XSqi3W2xJmAZg7troEZcuuT)_
Part of #25522
Clause-②: no
Metadata-protocol: none
Ruling-ref: 6348891160

Cut 11 of the `packages/objectql/src/engine.ts` split (#24586 ruling A,
recorded in `6270097840`). The claim is `6426204595`. The whole
"File-reference hydration" section moves unedited into a new module,
`packages/objectql/src/engine-file-reference-hydration.ts`:
`resolveFileReferences` (a read's file-field ids expanded to the
`sys_file` read form), `callerMayReadSysFile` (#24335, the admission
asked before the caller's own `sys_file` read),
`serveRefusedFileHydration` (the refused arm), `servableFileRows` (the
one servability rule both arms use) and `readParentDerivedFiles`
(#22637, the field-owned verdict). The three module helpers only they
use move with them: `isReadRefusal`, `toFileValue` and
`PARENT_DERIVED_FILE_COLUMNS`. No behaviour changes. #25522 stays open
for the later cuts.

Branch `claude/project-thread-umvn9e` (the card thread's harness-pinned
branch), cut from `origin/main` at `bdfad1fc7`, which is cut 10's merge
(#26308). Nothing was edited before that merge landed. `origin/main`
(`a8e7393dd`) was merged in once before this PR opened; none of the
commits it brought touch `packages/objectql/**`,
`packages/core/src/security/**`, `content/docs/permissions/**` or
`scripts/adr-anchors/**`.

## Why this area

Re-measured on `bdfad1fc7` with cut 10's scripts, unchanged (both
screens: erasure sites and module-helper closure). `engine.ts` is 22,942
lines, 7,374 of them from the last 30 days (`git blame -s
--since=2026-09-26T21:39Z`). `scripts/pm/git-history.mjs count`: 129
commits in 30 days, 35 in 7. `class ObjectQL` holds 318 members, 255
methods and accessors. The file at `bdfad1fc7` is byte-identical to cut
10's head, where the table was first pre-measured while #26308 was in
CI.

| range | section | first … last member | lines | 30-day lines | methods
| inferred public | U | private statics |
|---|---|---|---|---|---|---|---|---|
| 4755-4980 | Instance state and constructor | `txStore` … `constructor`
| 226 | 50 | 0 (0 public) | - | 0 | - |
| 4981-5003 | Status and registry accessor | `getStatus` … `registry` |
23 | 0 | 2 (2 public) | getStatus | 0 | - |
| 5004-5347 | Hook and function registry | `registerHook` …
`triggerHooks` | 344 | 6 | 16 (16 public) | registerHook, triggerHooks |
1 | - |
| 5348-5703 | Hook dispatch internals | `hasHooksFor` …
`assertBulkPerRowHookBudget` | 356 | 61 | 7 (0 public) | - | 0 | - |
| 5704-5864 | Action registry and activation | `registerAction` …
`setActionActive` | 161 | 19 | 10 (10 public) | - | 1 | - |
| 5865-6146 | Registered resolvers and write-gate probes |
`_heldFileResolver` … `registerInsertRecomputeListener` | 282 | 116 | 6
(6 public) | - | 0 | - |
| 6147-6900 | Readable-ids security and permission/formula resolution |
`readableIdsSecurityReported` … `resolveDefaultPermissions` | 754 | 546
| 15 (2 public) | - | 0 | - |
| 6901-7007 | Authz invalidation and middleware registration |
`writeEpoch` … `hasObjectMiddleware` | 107 | 18 | 4 (4 public) | - | 0 |
- |
| 7008-7185 | Execution plumbing (middleware, session, provenance,
users) | `executeWithMiddleware` … `buildUser` | 178 | 45 | 7 (0 public)
| - | 0 | - |
| 7186-7516 | Driver options and engine posture | `buildDriverOptions` …
`resolveEnginePosture` | 331 | 209 | 6 (0 public) | - | 0 | - |
| 7517-7726 | Organization probe and system-insert organization |
`probeInstallOrganizations` … `transactionCoversDriverFor` | 210 | 45 |
4 (0 public) | - | 0 | - |
| 7727-8092 | Field defaults | `resolveNowDefault` …
`defaultFieldEntries` | 366 | 4 | 6 (0 public) | - | 0 | - |
| 8093-8705 | Autonumber | `applyAutonumbers` … `seedAutonumber` | 613 |
80 | 6 (0 public) | - | 1 | - |
| 8706-9370 | Package, app, plugin and driver registration |
`registrationConversionWarned` … `registerDriver` | 665 | 170 | 6 (3
public) | registerApp, registerDriver | 0 | - |
| 9371-9579 | Datasource registry | `supplyFileColumnsMovedResolver` …
`listUnavailableDatasources` | 209 | 1 | 8 (7 public) | - | 3 | - |
| 9580-9731 | Write guard and realtime publish | `assertWriteAllowed` …
`publishBulkDataEvent` | 152 | 18 | 4 (1 public) | - | 0 | - |
| 9732-9830 | Validation-message i18n | `setI18nService` …
`objectDisplayLabel` | 99 | 14 | 4 (1 public) | - | 0 | - |
| 9831-10834 | Referential integrity and master-detail resolution |
`assertReferencesResolve` … `masterDeclaredFields` | 1004 | 415 | 12 (0
public) | - | 0 | - |
| 10835-10950 | Dangling-reference and cascade inspection |
`inspectDanglingReferences` … `inspectCascadeDelete` | 116 | 48 | 2 (2
public) | - | 0 | - |
| 10951-11045 | Crypto and tenancy providers | `setCryptoProvider` …
`setTenancyPostureProvider` | 95 | 20 | 4 (4 public) | - | 1 | - |
| 11046-11465 | Secret and internal-field handling |
`encryptSecretFields` … `resolveInternalField` | 420 | 61 | 9 (3 public)
| - | 0 | - |
| 11466-11534 | Schema lookup and filter judging | `getSchema` …
`judgeFilter` | 69 | 48 | 3 (2 public) | - | 0 | - |
| 11535-11800 | Datasource routing | `LIFECYCLE_DATASOURCE` …
`setDatasourceMapping` | 266 | 20 | 8 (3 public) | setDatasourceMapping
| 0 | - |
| 11801-11968 | Lifecycle init and health | `init` …
`resolvePrimaryDatasource` | 168 | 16 | 3 (3 public) | init | 0 | - |
| 11969-12042 | Media and value-shape presence | `objectHasMediaField` …
`valueShapeStrictFor` | 74 | 13 | 4 (0 public) | - | 0 |
mediaFieldPresence, coveredValueFieldPresence |
| 12043-12725 | Migration flags and attestations |
`isFileReferencesMigrationVerified` … `wasDatastoreCreatedFromEmpty` |
683 | 81 | 17 (8 public) | - | 0 | - |
| 12726-12748 | destroy | `destroy` … `destroy` | 23 | 0 | 1 (1 public)
| destroy | 0 | - |
| 12749-12767 | Expand/cascade limits (statics) | `MAX_EXPAND_DEPTH` …
`MAX_BULK_PER_ROW_HOOK_ROWS` | 19 | 0 | 0 (0 public) | - | 0 |
MAX_EXPAND_DEPTH, MAX_CASCADE_DEPTH |
| 12768-13218 | Roll-up summaries | `autonumberCounters` …
`recomputeSummaries` | 451 | 0 | 7 (1 public) | - | 0 | - |
| 13219-13472 | Related-record expansion | `expandRelatedRecords` …
`expandRelatedRecords` | 254 | 10 | 1 (0 public) | - | 0 | - |
| 13473-13916 | File-reference hydration | `resolveFileReferences` …
`readParentDerivedFiles` | 444 | 310 | 5 (0 public) | - | 0 | - |
| 13917-14395 | Query lowering | `resolveWhereTokens` …
`requireFindOnePredicate` | 479 | 272 | 9 (0 public) | - | 0 | - |
| 14396-14952 | find / findOne | `find` … `staticReadonlyCreateStrip` |
557 | 142 | 4 (2 public) | - | 0 | - |
| 14953-15610 | validate | `validate` … `validate` | 658 | 447 | 1 (1
public) | - | 0 | - |
| 15611-16957 | insert / insertMany | `readUpdatePriorRow` …
`insertMany` | 1347 | 439 | 3 (2 public) | - | 1 | - |
| 16958-18891 | update | `update` … `update` | 1934 | 275 | 1 (1 public)
| - | 0 | - |
| 18892-20432 | Delete cascade and reference walking |
`planCascadeAtomicity` … `organizationOwnedRowCounts` | 1541 | 733 | 22
(1 public) | - | 0 | storedReferenceIncludes,
remainderAfterMemberRemoval, referenceCheckContext,
outlivesOrganization, keepsOrganizationReference,
organizationCountContext |
| 20433-21002 | delete | `delete` … `delete` | 570 | 47 | 1 (1 public) |
- | 0 | - |
| 21003-21622 | count / aggregate | `count` … `aggregate` | 620 | 236 |
3 (2 public) | - | 0 | - |
| 21623-21931 | execute and transactions | `execute` …
`enforceTransactionOrigin` | 309 | 74 | 8 (2 public) | - | 0 | - |
| 21932-22224 | Compatibility API | `registerObject` … `create` | 293 |
52 | 16 (16 public) | - | 0 | - |

The full per-method screen is in the project file
`pm-orders/server/25522-cut11-seam-table.md`; the move script is
`pm-orders/server/evidence/25522/move-cut11.mjs`.

| area | paying bodies under both screens | 30-day lines in them | net |
verdict |
|---|---|---|---|---|
| **File-reference hydration** (this PR) | all 5 bodies (288 lines) and
their 3 helpers, every user of which moves too | 176 of 288 | priced
-243 by the rule; **-291 measured** (the helpers and three import
bindings go too) | pays most; nothing in the section stays behind |
| Driver options and engine posture | `buildDriverOptions`,
`driverReadScope`, `wallExemptedOperation`, `resolveEnginePosture`;
`engineLayer0Verdict` only together with `driverWriteTarget` (they share
`accessibleOrganizationsOf`) | about 84 of 146 | about -110 to -120 |
next; hot |
| Migration flags and attestations | 8 bodies | 50 of 154 | about -91 |
its host needs setters (a seat ruling first); not taken |
| Readable-ids, Package registration, count, Delete cascade and the rest
| - | - | -87 to 0 | hot, blocked by a shared helper, or pays little |

## The shape

- The class keeps all five methods with name, modifiers (`private
async`), signature, docblock and position. Each body is now one host
literal of exactly the `FileReferenceHydrationHost` members it reads,
plus a `.call`, marked `[#25522 cut 11]`, as cuts 8 and 10 did. Fields
go as getters over a `readX` closure (`_registry`, `logger`,
`_heldFileResolver`, `_fieldOwnedFileReadAuthorizer`); methods go as
late-bound closures (`find`, `callerMayReadSysFile`,
`serveRefusedFileHydration`, `servableFileRows`,
`readParentDerivedFiles`, `resolveReadableIdsSecurity`), so a spy or
double set on the engine is still the one a call reaches.
- **One named host.** `FileReferenceHydrationHost` types each member as
the class member (`ObjectQL['find']` and so on); each function's `this`
is the `Pick` of what it reads. The module imports the engine only as a
type.
- **The helpers.** `isReadRefusal`, `toFileValue` and
`PARENT_DERIVED_FILE_COLUMNS` move with their docblocks byte for byte
and are not exported; every user of each was one of the five bodies. One
copy, no re-derivation.
- **Imports.** The module imports `FILE_REFERENCE_TYPES`,
`isFileIdToken`, `RAW_FILE_VALUES_CONTEXT_KEY` and the type
`FileRefusedValue` (`@objectstack/spec/data`), `ExecutionContext`
(type), `withoutOperationPrivateKeys` (`@objectstack/core`, with its
three-line import comment, which names `readParentDerivedFiles`) and
`isMissingTableError` (`@objectstack/metadata/errors`): the bindings
`engine.ts` used. `engine.ts` drops `isFileIdToken`, `FileRefusedValue`
and `withoutOperationPrivateKeys` (their last readers moved) and gains
the module import with a two-line comment. `FILE_REFERENCE_TYPES`,
`RAW_FILE_VALUES_CONTEXT_KEY` and `isMissingTableError` have readers
left in `engine.ts` and stay imported there.
- The module header names what moved and why nothing in the section
stays. It cites no ADR decision letter.

## Net lines (the seat's rule)

`engine.ts` goes from 22,942 to 22,651 lines, **net -291** (44 added,
335 removed). The module is 451 lines. With the pin row below, the diff
against `origin/main` is 838 changed lines over 3 files (+503 / -335).

## The published `.d.ts` does not change (ruling `6348891160`)

Base: `@objectstack/objectql` built with `engine.ts` restored from
`bdfad1fc7` (blob `0603fb25` checked) and the new module removed;
restored with `git checkout HEAD --`, both blob hashes compared, `git
diff HEAD` empty. Head: built at `596517727`.
`resolveFileReferences.call(host` occurs 0 times in the base
`dist/index.mjs` and once in the head's, so each leg built the source it
names. All six declaration files are **byte-identical**, and equal to
the hashes cuts 7 to 10 recorded:

| file | sha256, base = head |
|---|---|
| `core.d.mts` |
`ef178bf0725eccc2f358f082ce13dc5122a6d5a53c19ef630017256f67de03b4` |
| `core.d.ts` |
`1d20303a455a909e4b55fd0802fb0b0bcb59ec51784daf27d5ef4c0659410953` |
| `index.d.mts` |
`ccf900240aa137abe72f82b8abfc872d42c5645467400134c580285f2a3b3090` |
| `index.d.ts` |
`0454d5e70b156d8effbf27b8915587c7cbee241dcd7d698013c0a4c369a02aa5` |
| `util-CVzOGxAn.d.mts` |
`85ef1cbec2b016d5f7afabaa21689c86c6dffc573076fdfdeb8ea0f1a1b1ad53` |
| `util-CVzOGxAn.d.ts` |
`85ef1cbec2b016d5f7afabaa21689c86c6dffc573076fdfdeb8ea0f1a1b1ad53` |

**Positive control:** dropping `private` from the `servableFileRows`
stub (anchor landed once; 0 private hits after, 1 public) and rebuilding
changed all six hashes and the chunk name (`util-CBKDhiuL.d.ts`
`f9beab24567a…`, `index.d.ts` `2a876e08b028…`), and the chunk now prints
`servableFileRows(objectName: string, rows: any[]): Promise of Map of
string to any`. Restored with `git checkout HEAD --` (blob hash
compared, `git diff HEAD` empty, the private spelling back to 1 hit),
rebuilt, and all six files compared equal to the head build again.

## Proof of move-only

A TypeScript-AST leaf-token compare
(`pm-orders/server/evidence/25522/astcmp.mjs`, unchanged from cuts 9 and
10) of base `engine.ts` (`bdfad1fc7`) against the head module:

- All five bodies are **SAME** in parameters, return type, type
parameters, body tokens and `async`, and every body line equals the base
line less two leading spaces, comments included: `resolveFileReferences`
584 tokens / 150 lines, `callerMayReadSysFile` 81 / 8,
`serveRefusedFileHydration` 239 / 30, `servableFileRows` 190 / 49,
`readParentDerivedFiles` 427 / 51.
- The three helpers: the base text of each from its docblock to its end
is a substring of the module, and `engine.ts` no longer declares them.
- Class: 318 members at both ends, in the same order. 313 are
byte-identical, and the 5 stubs keep docblock, modifiers and signature
byte-identical up to the body.
- Top level: 226 of 233 base statements are byte-identical in head. The
other seven: the class; the three helpers, moved; the
`@objectstack/spec/data` value import, less `isFileIdToken`; the
`FileRefusedValue` type import, dropped; the `@objectstack/core` import,
less `withoutOperationPrivateKeys` and its comment.
`readStoredAutonumberCounter`, which followed the helpers, keeps its own
text and its one leading blank line. Head adds the new module import
with its comment.

## Readers of the moved lines

- **The row-serving door pin (migrated in this PR).**
`packages/core/src/security/row-serving-door-exposure.pin.test.ts`
(core's `repo` vitest project, CI's `Test Core`) classifies every file
that calls `.canReadObject(`, and `callerMayReadSysFile` makes such a
call. It gets a `not-a-door` row for
`engine-file-reference-hydration.ts`, built from the method's own
docblock: the ask only narrows what the caller's own `sys_file` read
reads, and serves no row. The `engine.ts` row stays, since the
ReadableIds composition still calls the admission there (2 sites).
**Control:** without the row, the pin fails with exactly one
unclassified file, the new module; with it, 6 of 6 files and 83 of 83
tests pass.
- **Every `repo` project** (`core`, `objectql`, `rest`, `runtime`,
`spec`, `types`) was run after the build: all green (6, 1, 5, 3, 79 and
1 files). No other census keys a moved body:
`second-object-read-exposure.pin.test.ts` keys callers of
`referenceTargetOf`, which none of the five call;
`operation-private-keys.pin.test.ts` keys declarations of the helper,
not its callers.
- **The system-context census page.** `readParentDerivedFiles` writes
`isSystem: true` into its sub-read's context; that is not a read site,
so the page needs no row. `check-system-context-census` at base
`a8e7393dd` and at head: 153 read sites across 82 files, 145 of 145
symbols cited, OK at both ends.
- **The federated injected-column readers table** keys no moved body
(none calls `isFederatedObject`, `referenceTargetOf` or the tenant-field
readers).
- **ADR anchors.** No `scripts/adr-anchors/` entry describes a moved
body (the engine entry's one paragraph is the transaction seam), so no
entry follows. Two body comments cite a decision letter (`ADR-0104 D3`
in `readParentDerivedFiles`, `ADR-0110 D3` in `resolveFileReferences`'
`catch`) and leave the anchored `engine.ts` with their bodies, so the
decision-letter count drops by two. `check-adr-anchors` at base
`a8e7393dd` (clean detached worktree): 90 anchored files, **1,880**
decision-letter citations, 59,270 resolving citations; at head: 90,
**1,878**, 59,270. The method docblocks that cite ADR-0104 D3 stay on
the class.
- No `objectql/src/engine.ts#NAME` pointer under `packages/spec/**`,
`docs/adr/**`, `docs/qa/**` or `content/**` names any of the five or the
helpers. Prose that names `resolveFileReferences` (`driver-sql`'s
`sql-driver.ts`, the hydration tests, `service-storage`'s agreement
test, `check-durability-degradation-log-level.mjs` header comments,
CHANGELOGs) still resolves, because the method stays on the class. No
gate baseline keys a moved body
(`durability-read-invention.baseline.json` does not; the moved bodies
hold no `query-options/no-any-erasure` site, and
`query-options-erasure-baseline.json` keeps `engine.ts` at 8).

## Acceptance notes

- Serial check, re-read before `pr_create` (29 open PRs): one opened
after this cut started touches `engine.ts`. #26337 (draft, ADR-0142
phase 2, opened 21:46Z) edits it in nine hunks: an import near line 241,
three type and constant declarations, and the `insert` / `update` bodies
(from base line 15727 on). None is in the moved section (base lines
13473 to 13916) or on the import lines this cut edits, and `git
merge-tree --write-tree` of this head with its head `578e7a417` merges
cleanly (exit 0; its own generated `packages/spec` files are regenerated
by their merge driver, as on any merge). Under the card's rule 4
whichever lands second merges `main`. Otherwise #26138 edits
`validation/rule-validator.test.ts` only and #25640 the objectql
`CHANGELOG.md` and `package.json` only; no open PR touches the
row-serving pin, `system-context.mdx` or `scripts/adr-anchors/`.
- One receiver note, not a change in what runs: `servableFileRows` calls
`this._heldFileResolver(tombstoned)`. In the class the call's receiver
was the engine; in the module it is the host, whose getter returns the
same function. Both registrations (`service-storage`'s plugin and the
parent-derived test) pass arrow functions, and `HeldFileResolver`
declares no `this`, so nothing reads the receiver.
`readParentDerivedFiles` copies its authorizer into a local before
calling it, so that call has no receiver at either end.
- Next cut from this screen: Driver options and engine posture (about
-110 to -120; `engineLayer0Verdict` moves only with `driverWriteTarget`,
so `accessibleOrganizationsOf` stays module-local; 84 of 146 body lines
are recent). Migration flags still needs a setter ruling.

## Verification

At `596517727` (the cut and the pin row; the merge of `origin/main`
after it changes no file under `packages/objectql/**` or
`packages/core/**`):

- `pnpm --filter @objectstack/objectql typecheck` (`tsc --noEmit` +
`check:test-typecheck`): VERDICT command-exit 0. `pnpm exec eslint
--no-inline-config packages/objectql packages/core/src/security`: exit
0.
- `pnpm --filter @objectstack/objectql exec vitest run --project local
--maxWorkers=2`: 472 files, 8,789 tests passed (VERDICT command-exit 0).
The three hydration suites (`engine-file-hydrate-outage`,
`-parent-derived`, `-refused`) are among them, 53 tests.
- `vitest run --project repo` in every package that has one, after the
`^...` closure builds: `core` 6/6 files (83 tests), `objectql` 1/1,
`rest` 5/5, `runtime` 3/3, `spec` 79/79, `types` 1/1, all VERDICT
command-exit 0.
- Dependants: all five members keep their declarations and all six
published `.d.ts` files are byte-identical, so no dependant can see a
difference. CI's `TypeScript Type Check` covers them.

At `4acced1d8` (final head):

- Gates: `dispatch-gates --commands` derived 62 families on the real
diff. Every one was run, each exit code captured before any pipe, then
reconciled with `--ran`: 62 accounted for, 61 run green, 1 **NOT
MEASURED** (exit 3, PREREQUISITE NOT MET; CI runs it):
`check:dual-build-cjs-loads` (needs every package's `dist/`). Named
readings: `check-system-context-census` OK, `check:durability-log-level`
(82 read seams, none invents an answer), `check:query-options-erasure`
(none new), `check:engine-double-contract` OK,
`check:lean-entry-closure`, `check:type-check-debt` (0 raw tsc errors
above the ledger), `check:logger-receiver-detach`,
`check:org-identifier`, `check:nul-bytes`, `check-affected-docs` exit 0.
- Hand gates outside the derivation, all exit 0: `check-adr-anchors` at
base and head (above), `check-adr-symbol-anchors`,
`check-tenant-audit-census`, `check-system-context-census`,
`check-platform-checklist --changed-from origin/main` (13 anchors
judged, 0 lost) and `check-commit-card-trailers --range
origin/main..HEAD`.
- No ablation, because no tested behaviour changed. The whole `Test
Core` farm is CI's.

`skip-changeset`: move-only. The published `.d.ts` files are
byte-identical, `liveness/` and every other published path are
untouched, and the module's runtime is the same code under a different
file name. The pin edit is a test file.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_011J7U85Djs3JuvHhG5gdcM9

---
_Generated by [Claude
Code](https://claude.ai/code/session_011J7U85Djs3JuvHhG5gdcM9)_

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: objectstack-fleet[bot]

packages/core/src/security/row-serving-door-exposure.pin.test.ts

@@ -108,6 +108,14 @@ const CALLERS: Record<string, Classification> = {
       + 'that only narrows the read the caller addressed, whose own door already admitted it; no row and no field of the '
       + 'parent is served, and a refused parent answers `$in: []`',
   },
+  // [#25522 cut 11] File-field hydration's `sys_file` admission, moved out of engine.ts.
+  'packages/objectql/src/engine-file-reference-hydration.ts': {
+    kind: 'not-a-door',
+    reason:
+      "file-field hydration's ask before the caller's own `sys_file` sub-read (`callerMayReadSysFile`, #24335): it only "
+      + 'narrows what is read (`true` still reads as the caller, through the engine middleware; anything else serves the '
+      + "refused marker), and it serves no row of its own; the record read it hydrates was admitted by that read's door",
+  },
 };
 
 const SCAN_TIMEOUT_MS = 60_000;