fix(objectql): a refused sys_file read marks the file field refused instead of reading as no file (#22620)

Fixes #22593
Clause-②: yes

## What changes

A file field (`file` / `image` / `avatar` / `video` / `audio`) holds a
`sys_file` id. The engine expands that id by reading `sys_file` as the
caller (`ObjectQL.resolveFileReferences`,
`packages/objectql/src/engine.ts`). A reader who may read the record but
not `sys_file` gets that sub-read refused. The record still came back,
but with the bare id in the field. A bare id is also what an uncommitted
id or a storage outage leaves behind, so the console showed "no file"
and the server logged a `warn` telling the operator to check storage.

After this change, the security layer's refusal (`PERMISSION_DENIED`)
marks each held id as refused, in the value itself:

- **From:** `{ "attachment": "f_7cQx2Lm90a" }`, plus `warn: sys_file
lookup failed; file fields keep their raw ids and will render as "no
file" for this read`.
- **To:** `{ "attachment": { "id": "f_7cQx2Lm90a", "metadataRefused":
true } }`. Multi-value fields mark each id, and a legacy URL string in
the array passes through unchanged. The refusal is logged at `debug`
only, because the reader is told in the value (AGENTS "Degradation log
levels": a failure handed to the caller).
- **Unchanged:**
- A reader with `sys_file` read still gets the hydrated `{ id, name,
size, mimeType, url }`.
  - An empty field still reads empty.
  - A missing `sys_file` table still leaves the bare id, silently.
- Any other failure (connection, timeout, a database-level ACL fault
such as `42501`) still leaves the bare id with the existing `warn`.
  - The record read still succeeds. Fail-open is unchanged.

Nothing that was not read is served: the marker has no `name`, `size`,
`mimeType` or `url`. A consumer that needs a link builds the stable
`/api/v1/storage/files/:fileId` endpoint from the id, as it already does
for a bare id. The download door makes its own access decision (see "Not
in this PR").

`@objectstack/spec` declares the shape. `FileRefusedValueSchema` /
`FileRefusedValue` is a closed `{ id, metadataRefused: true }`, an
ADR-0122 isomorphic alias pinned in `type-alias-convention.pin.test.ts`
(773 to 774). `valueSchemaFor(def, 'expanded')` for the file types now
accepts it beside the bare id and `FileValueSchema`. It is a read form
only: the stored form stays the bare id, and a pin checks that
`valueSchemaFor(def)` (the stored form) rejects the marker object.
Changeset: `@objectstack/objectql` minor and `@objectstack/spec` minor,
with the FROM and TO above.

## Measured on main first (the symptom, before the change)

Both pin files were run against `main`'s `engine.ts` (`d748ae80af`, blob
`df2300664b`, byte-identical by `git hash-object`), with every other
file at this head (`faf689872c`). The file was then restored from `HEAD`
(blob `1d1b78f500`, and `git diff HEAD` is empty).

- Real security layer
(`plugin-security/src/file-field-hydration-refused.test.ts`, real
`ObjectQL` + `SecurityPlugin` + `SqlDriver`): the reader with record
read and no `sys_file` read got `attachment === 'f_7cQx2Lm90a'`, the
bare id. 1 failed and 3 passed. The 3 that passed are the two controls
and the direct-refusal probe, which shows `sys_file` refused with
`PERMISSION_DENIED` / 403.
- Seam pins (`objectql/src/engine-file-hydrate-refused.test.ts`): 4
failed and 4 passed. The `warn` captured on main is exactly the line
from the report: `sys_file lookup failed; file fields keep their raw ids
and will render as "no file" for this read`. The 4 controls passed on
main as well.

The symptom from the 17.7.0 report reproduces on current `main`.

## Pins (green at `faf689872c`)

- A reader without `sys_file` read sees `{ id, metadataRefused: true }`,
through both `find` and `findOne`, on the real security layer and on the
seam.
- Control: a record with no file still reads `null`.
- Control: a reader with `sys_file` read gets the hydrated file.
- Control: a missing `sys_file` table (`42P01`) still reads the bare id,
with no `warn`.
- Control: a database-level ACL fault (`42501`) is an outage, not a
refusal of this caller. It still reads the bare id and still logs the
seam's `warn`.
- The served value parses as `FileRefusedValueSchema` and as
`valueSchemaFor(def, 'expanded')`, and carries exactly the keys `id` and
`metadataRefused`.

## Ablation

`node scripts/ablation-replace.mjs` replaced the anchor `if
(isReadRefusal(error)) {` with `if (false && isReadRefusal(error)) {`,
which restores the fall-through.

- The anchor hit once (count 1 to 0), and the blob changed from
`1d1b78f500` to `1565ec17ca`.
- objectql pins: 4 failed and 4 passed. The first pin received the bare
id.
- plugin-security pins: 1 failed and 3 passed. The first pin received
the bare id.
- Restore: the blob equals `HEAD` (`1d1b78f500`), `git diff HEAD` is
empty and the porcelain status is clean.

Both legs read source, not `dist/`: the objectql test imports
`./engine`, and plugin-security's `vitest.config.ts` aliases
`@objectstack/objectql` to `../../objectql/src/index.ts`.

## Cross-lane paths (the seat declares these on the lane posts before
this PR leaves draft)

- `packages/spec/src/data/field-value.zod.ts`, its pins
(`field-value.test.ts`, `type-alias-convention.pin.test.ts`, and
`scripts/strictness-ledger.test.ts`, whose site count over
`field-value.zod.ts` moves from 3 to 4) and the regenerated spec
artifacts (`api-surface/`, `authorable-surface/`, `declaration-map/`,
`export-origins/`, `json-schema.manifest/` data shards) are
`domain:spec`.
-
`packages/plugins/plugin-security/src/file-field-hydration-refused.test.ts`
(test only) is `domain:services`.
- `content/docs/references/data/field-value.mdx` and
`content/docs/references/index.mdx` (generated by spec `build-docs`),
and the hand-written `content/docs/protocol/objectql/types.mdx` (the
`file` read forms) and `content/docs/permissions/attachments-access.mdx`
(one sentence), are `domain:devx`.
- `docs/audits/2026-07-unknown-key-strictness-ledger.counts/data.md`
(generated, `check:strictness-ledger`) gets +1 strict site in `data/`.
`docs/audits/2026-07-unknown-key-strictness-ledger.md` gets a one-row
update on the `field-value.zod.ts` row naming the third `strictObject`.

## Consumer relay (objectui, not edited here)

At objectui `origin/main` `12ff256`,
`packages/fields/src/widgets/file-value.ts` `readFileValue` takes the
marker through its object arm:

- The id is kept, and the URL is built from the id (`fileUrlFromId`).
- The name falls back to the generic "File", and there is no `mimeType`.
- `metadataRefused` is not carried onto `FileValueView`.

So today the console already shows a non-empty generic file chip, not
`EmptyValue`, but the refusal itself never reaches the UI. What the
console's file renderers (`FileField` read-only list, `FileCell`,
`ImageField`, `AvatarField`, the grid cell) should show for
`metadataRefused: true`:

- A distinct restricted state: a lock affordance and a translated label
(for example "File (details restricted)"), not the generic fallback
name.
- Never `EmptyValue`.
- No thumbnail attempt for image or avatar types, because no MIME type
is known.
- Keep the open/download affordance on the id-derived stable endpoint.
That door decides on its own and may answer `403 FILE_DOWNLOAD_DENIED`.
- The editable widget submits only the bare id (`fileIdOf` already reads
`id`), never the marker object.

Once objectui takes a spec that carries `FileRefusedValueSchema`, it can
tell the form apart by shape.

## Not in this PR

- **The access half.** This PR makes no change to `sys_file`'s read
rule. Whether file metadata should follow the read access of the parent
record goes to the maintainer on a separate decision card, which the
seat files. The inputs were measured read-only and are in the report on
the card.
- Row-level narrowing on `sys_file` (a reader with object read whose
sub-read comes back short) still reads as a bare id. The engine cannot
tell a hidden row from an uncommitted one without an existence probe,
and an existence probe is itself an access decision. It belongs to the
access half.

## Acceptance notes

- Export: `formatCsvCell` already serialises an object-valued cell as
JSON, so a refused reader's CSV cell now reads
`{"id":"…","metadataRefused":true}` instead of the bare id. This is
distinct from both empty and hydrated. XLSX and JSON export pass the
value through.
- `plugin-approvals` `normalizeActionAttachment` would drop the marker
(it maps any object to `{ id, name?, url? }`), but its only caller,
`listActions`, reads `sys_approval_action` with the system context, so
its hydration never sees a refusal.
- The header of
`packages/services/service-storage/src/attachment-access-hooks.ts` still
says the default member permission sets grant wildcard CRUD.
`member_default` has named no `'*'` since that wildcard was removed, so
the comment is stale. This is noted only, not filed (carrier: none).

## Verification (head `faf689872c`, which is `origin/main` `d748ae80af`
merged in)

- Build: `pnpm --filter '@objectstack/plugin-security...' build`, which
covers spec, objectql and plugin-security in one closure. Then `pnpm
--filter @objectstack/spec check:generated` reported all 15 artifacts up
to date.
- Full suites:
  - `@objectstack/objectql` `test`: 393 files and 7738 tests passed.
- `@objectstack/plugin-security` `test`: 194 files, with 4079 passed and
45 skipped.
  - `@objectstack/spec` `test`: 640 files, with 19103 passed and 1 todo.
- `typecheck` passed for spec, objectql and plugin-security.
- `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack
--commands` derived 119 commands, and all 119 exited 0. The `--ran`
reconciliation reads: 119 derived, 119 run, 0 NOT-MEASURED, all with
recorded exit codes.
- Narrowed lint over the 6 changed TypeScript files, with `eslint
--no-inline-config --format json`:
  - The `--format json` output reports 6 files, 0 errors and 0 warnings.
- The population comes from the `eslint.config.mjs` blocks for
`packages/**/*.{ts,tsx,mts,cts}`.
- The config never enables type-aware linting, so this diff cannot
change the verdict on any file it does not touch.
  - The repository-wide `pnpm lint` is left to CI.
- Not run locally, left to CI: the 6 path-scheduled CI jobs and the
workspace type-check lanes that the derivation lists as out of reach
locally.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Bw3y2DWhT9RPnrmDsNqEVG)_

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: objectstack-fleet[bot]

.changeset/22593-file-field-hydration-refused.md

@@ -0,0 +1,18 @@
+---
+'@objectstack/objectql': minor
+'@objectstack/spec': minor
+---
+
+A file field whose `sys_file` read was refused to the reader reads as refused, not as "no file"
+
+Clause-②: yes
+
+The engine expands a stored file id (`file` / `image` / `avatar` / `video` / `audio`) by reading `sys_file` as the caller. A reader who may read the record but not `sys_file` (the default for any member whose application grants no `sys_file` read) is refused that read, and the field used to come back as the bare id: the same value an id with no committed file row produces. A refused file and an absent one were one answer, and the server logged a `warn` per read telling the operator to check storage availability.
+
+- **From.** `{ "attachment": "f_7cQx2Lm90a" }` for a reader refused `sys_file`, plus `warn: sys_file lookup failed; file fields keep their raw ids and will render as "no file"`.
+- **To.** `{ "attachment": { "id": "f_7cQx2Lm90a", "metadataRefused": true } }`. Multi-value fields mark each id. Nothing else is served, because nothing else was read: no `name`, `size`, `mimeType` or `url`. A consumer that needs the download link derives the stable `/api/v1/storage/files/:fileId` endpoint from the id, as it does for a bare id; the download door judges access by the record that owns the file. The refusal is logged at `debug` only, because the reader is told in the value.
+- **Unchanged.** A reader allowed to read `sys_file` gets the hydrated `{ id, name, size, mimeType, url }`. An empty field reads empty. A missing `sys_file` table still leaves the bare id, silently. Any other read failure (connection, timeout, a database-level ACL fault such as `42501`) still leaves the bare id with the `warn`. Only the security layer's `PERMISSION_DENIED` refusal takes the new form.
+
+`@objectstack/spec` declares the shape: `FileRefusedValueSchema` / `FileRefusedValue` (`{ id, metadataRefused: true }`, closed), and `valueSchemaFor(def, 'expanded')` for the file types admits it beside the bare id and `FileValueSchema`. It is a read form only. The stored form stays the bare id.
+
+**For consumers.** Code that reads a file field's expanded value as `string | FileValue` should handle the third form: an object with `metadataRefused: true` and no `url` is a file this reader may not see the details of, not an empty field.

content/docs/permissions/attachments-access.mdx

@@ -165,7 +165,9 @@ can be shared across records). Reclamation is handled by the platform LifecycleS
   file `GET /storage/files/:id` serves is a file a record read expands into
   `{ id, name, size, mimeType, url }`. The row itself stays tombstoned until a
   sweep tidies it, and a file with no holder left still answers
-  `FILE_NOT_FOUND` (404) and keeps its bare id in a record payload.
+  `FILE_NOT_FOUND` (404) and keeps its bare id in a record payload — except to
+  a reader refused `sys_file` read, who gets `{ id, metadataRefused: true }` for
+  every file id, held or not ([`file` read forms](/docs/protocol/objectql/types#file)).
 - **`sys_upload_session`** — abandoned/terminal chunked-upload sessions are
   reaped, and a reap guard aborts the underlying backend multipart upload
   (S3 `AbortMultipartUpload` / local parts dir) first, so already-uploaded

content/docs/protocol/objectql/types.mdx

@@ -1156,21 +1156,44 @@ avatar:
   multiple: false  # set true to store an array of file references
 ```
 
-**Stored value:** an opaque `sys_file` id string. The expanded read form is the
-media metadata object, whose only required member is `url`:
+**Stored value:** an opaque `sys_file` id string. What a read returns leaves it
+unchanged: the stored form is the id alone, and the two object forms below are
+read forms only. A record read expands each stored id in place (each element,
+with `multiple: true`) into one of **three** forms, and none of them fails the
+record read that asked for it.
+
+**The file object** — the caller's `sys_file` read found the file (a committed
+row, or a tombstone something still holds). `url` is its only required member,
+derived from the id rather than stored:
 
 ```json
 {
-  "url": "https://cdn.example.com/files/abc123.jpg",
+  "id": "abc123",
   "name": "profile.jpg",
   "size": 1024000,
-  "mimeType": "image/jpeg"
+  "mimeType": "image/jpeg",
+  "url": "/api/v1/storage/files/abc123"
 }
 ```
 
 `alt` and `durationSeconds` are the other optional members. The keys are `name` and
 `mimeType` — not `filename` / `content_type`.
 
+**The bare id, as stored** — the engine had nothing to resolve it against: the
+read found no such file, `sys_file` is not registered or its table is not
+provisioned yet, or the read failed for any reason other than a refusal (an
+outage, which the engine logs as a warning).
+
+**`{ "id": "abc123", "metadataRefused": true }`** — the caller may read the record
+but is refused the `sys_file` read. The value is the id the record holds plus the
+marker, nothing else: no `url`, `name`, `size` or `mimeType`, because none was
+read, and it says nothing about whether the file exists. It is distinct from an
+empty field, which holds no file at all, and from the file object, which always
+carries `url`.
+[`FileRefusedValueSchema`](/docs/references/data/field-value#filerefusedvalue)
+declares it, closed to exactly those two keys. Download access is judged on its
+own, by the record that owns the file rather than by `sys_file` read.
+
 <Callout>
 Deployments predating the file-as-reference migration may still hold the inline
 metadata object (or a bare URL) as the *stored* value. The engine warns rather

content/docs/references/data/field-value.mdx

@@ -1,7 +1,7 @@
 ---
 title: Field Value schema — Data Protocol reference
 navTitle: Field Value
-description: "Field runtime VALUE-shape contract. Reference for Address, AddressValue, CalendarDateValue, ClockTimeValue and 6 more: every property with its type and default."
+description: "Field runtime VALUE-shape contract. Reference for Address, AddressValue, CalendarDateValue, ClockTimeValue and 7 more: every property with its type and default."
 ---
 
 {/* ⚠️  AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
@@ -41,8 +41,8 @@ this contract has (ADR-0104 performance budget).
 ## TypeScript Usage
 
 ```typescript
-import { AddressSchema, AddressValueSchema, CalendarDateValueSchema, ClockTimeValueSchema, FileLikeValueSchema, FileReferenceIdValueSchema, FileValueSchema, InstantValueSchema, LocationValueSchema, ReferenceIdValueSchema } from '@objectstack/spec/data';
-import type { Address, AddressValue, CalendarDateValue, ClockTimeValue, FileLikeValue, FileReferenceIdValue, FileValue, InstantValue, LocationValue, ReferenceIdValue } from '@objectstack/spec/data';
+import { AddressSchema, AddressValueSchema, CalendarDateValueSchema, ClockTimeValueSchema, FileLikeValueSchema, FileReferenceIdValueSchema, FileRefusedValueSchema, FileValueSchema, InstantValueSchema, LocationValueSchema, ReferenceIdValueSchema } from '@objectstack/spec/data';
+import type { Address, AddressValue, CalendarDateValue, ClockTimeValue, FileLikeValue, FileReferenceIdValue, FileRefusedValue, FileValue, InstantValue, LocationValue, ReferenceIdValue } from '@objectstack/spec/data';
 
 // Validate data
 const result = AddressSchema.parse(data);
@@ -134,6 +134,18 @@ Type: `string`
 **Type:** `string`
 
 
+---
+
+## FileRefusedValue
+
+### Properties
+
+| Property | Type | Required | Description |
+| :--- | :--- | :--- | :--- |
+| **id** | `string` | ✅ | The sys_file id the record holds |
+| **metadataRefused** | `true` | ✅ | The reader was refused the file's metadata (no read on sys_file): name, size and type are withheld. Distinct from an empty field, which holds no file at all. |
+
+
 ---
 
 ## FileValue

content/docs/references/index.mdx

@@ -1,7 +1,7 @@
 ---
 title: Protocol reference — every schema by module
 navTitle: Protocol Reference
-description: Every schema published by @objectstack/spec — 1514 schemas across 14 protocol modules
+description: Every schema published by @objectstack/spec — 1515 schemas across 14 protocol modules
 ---
 
 {/* ⚠️  AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
@@ -23,7 +23,7 @@ counts are sums of the rows they head. Regenerate with
 | [AI Protocol](/docs/references/ai) | 12 | 68 | Agents, tools, skills, RAG and knowledge sources, model registry, conversations. |
 | [API Protocol](/docs/references/api) | 32 | 431 | REST contracts, endpoints, routing, realtime, batch, discovery. |
 | [Automation Protocol](/docs/references/automation) | 13 | 70 | Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execution records. |
-| [Data Protocol](/docs/references/data) | 29 | 174 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. |
+| [Data Protocol](/docs/references/data) | 29 | 175 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. |
 | [Identity Protocol](/docs/references/identity) | 5 | 27 | Users and accounts, organizations, positions, SCIM provisioning. |
 | [Integration Protocol](/docs/references/integration) | 1 | 12 | The single connector protocol (ADR-0097) — catalog descriptors and provider-bound instances. |
 | [Kernel Protocol](/docs/references/kernel) | 30 | 157 | Plugin lifecycle and manifests, capabilities and security, metadata loading, service registry. |
@@ -34,7 +34,7 @@ counts are sums of the rows they head. Regenerate with
 | [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. |
 | [System Protocol](/docs/references/system) | 34 | 275 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. |
 | [UI Protocol](/docs/references/ui) | 16 | 166 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. |
-| **Total** | **195** | **1514** | 14 protocol modules |
+| **Total** | **195** | **1515** | 14 protocol modules |
 
 ---
 
@@ -130,7 +130,7 @@ Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execu
 
 ## Data Protocol
 
-**Source:** `packages/spec/src/data/` · **Import:** `@objectstack/spec/data` · **29 pages, 174 schemas**
+**Source:** `packages/spec/src/data/` · **Import:** `@objectstack/spec/data` · **29 pages, 175 schemas**
 
 Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer.
 
@@ -154,7 +154,7 @@ Objects, fields, queries, filters, datasources and drivers — the ObjectQL laye
 | [`external-catalog.zod.ts`](/docs/references/data/external-catalog) | `ExternalCatalog`, `ExternalColumn`, `ExternalTable` |
 | [`feed.zod.ts`](/docs/references/data/feed) | `FeedFilterMode`, `FeedItemType` |
 | [`field.zod.ts`](/docs/references/data/field) | `CurrencyConfig`, `CurrencyValue`, `Field`, `FieldMaskingKeep`, `FieldMaskingRule`, `FieldType`, `InlineGridColumn`, `LocationCoordinates`, `SelectOption`, `UniqueScope` |
-| [`field-value.zod.ts`](/docs/references/data/field-value) | `Address`, `AddressValue`, `CalendarDateValue`, `ClockTimeValue`, `FileLikeValue`, `FileReferenceIdValue`, `FileValue`, `InstantValue`, `LocationValue`, `ReferenceIdValue` |
+| [`field-value.zod.ts`](/docs/references/data/field-value) | `Address`, `AddressValue`, `CalendarDateValue`, `ClockTimeValue`, `FileLikeValue`, `FileReferenceIdValue`, `FileRefusedValue`, `FileValue`, `InstantValue`, `LocationValue`, `ReferenceIdValue` |
 | [`filter.zod.ts`](/docs/references/data/filter) | `ComparisonOperator`, `EqualityOperator`, `FieldOperators`, `FieldReference`, `FilterArray`, `FilterCondition`, `NormalizedFilter`, `QueryFilter`, `RangeOperator`, `SetOperator`, `SpecialOperator`, `StringOperator` |
 | [`hook.zod.ts`](/docs/references/data/hook) | `Hook`, `HookContext`, `HookEvent` |
 | [`hook-body.zod.ts`](/docs/references/data/hook-body) | `ExpressionBody`, `HookBody`, `HookBodyCapability`, `ScriptBody` |

docs/audits/2026-07-unknown-key-strictness-ledger.counts/data.md

@@ -21,7 +21,7 @@ The `strict` column is the one the campaign schedules against; it counts both th
 
 | Dir | Sites | strict | passthrough | catchall | strip |
 |---|---|---|---|---|---|
-| `data/` | 155 | 79 | 2 | 0 | 74 |
+| `data/` | 156 | 80 | 2 | 0 | 74 |
 
 ## `data/` — sites
 
@@ -44,7 +44,7 @@ classify and is not listed (it becomes reportable the day it grows its first sit
 | `driver/sqlite.zod.ts` | 2 |
 | `driver/turso.zod.ts` | 2 |
 | `external-catalog.zod.ts` | 4 |
-| `field-value.zod.ts` | 3 |
+| `field-value.zod.ts` | 4 |
 | `field.zod.ts` | 13 |
 | `filter.zod.ts` | 12 |
 | `hook-body.zod.ts` | 2 |
@@ -56,15 +56,15 @@ classify and is not listed (it becomes reportable the day it grows its first sit
 | `seed-loader.zod.ts` | 12 |
 | `seed.zod.ts` | 1 |
 | `validation.zod.ts` | 6 |
-| **total** | **155** |
+| **total** | **156** |
 
 ## `data/` — open
 
 Per file, how many of its sites still silently discard unknown keys. The `Class`
 column that decides the bucket split is hand-written in the ledger; the arithmetic
 over it is here.
 
-**74 strip of 155**, in 10 file(s).
+**74 strip of 156**, in 10 file(s).
 
 | File | Strip | Sites |
 |---|---|---|
@@ -78,7 +78,7 @@ over it is here.
 | `hook.zod.ts` | 5 | 7 |
 | `query.zod.ts` | 4 | 5 |
 | `seed-loader.zod.ts` | 12 | 12 |
-| **total** | **74** | **155** |
+| **total** | **74** | **156** |
 
 | Bucket | Sites |
 |---|---|