fix(storage)!: a record write claims or copies only a file its caller may read

.changeset/25692-file-reference-claim-read-verdict.md

@@ -0,0 +1,21 @@
+---
+'@objectstack/service-storage': patch
+---
+
+fix(storage)!: a record write claims or copies only a file its caller may read
+
+The file-reference claim path (ADR-0104 D3: a file field holds a `sys_file` id, at most one record's field owns a file, and writing an id another record owns copies the bytes into a fresh file the writing record owns) now asks, before it copies or claims a referenced file, whether the write's caller may read that file. The question is the download door's own verdict, asked under the write's caller context: the uploader, a reader of the one record whose field owns the file (its object's `fileAccessDelegate` where one is declared), a member of the organization for a file no record owns, a reader of a parent record for an attachments-scope file, anyone for a `public_read` file. The door and the claim path now call one judgement, so one file has one access rule.
+
+A file the caller may not read is answered as the organization-scoped lookup answers a file it cannot see: the value is kept as written, nothing is claimed or copied, and constraint verdicts no longer run ahead of the read check. A copy this write made is claimed by the same write without a second question. System writes and context-less programmatic calls have no caller to ask and reference files as before; so does a kernel with no `auth` service, where the download door binds no gate. An authorization-store outage fails the write with its declared envelope instead of reading as either answer.
+
+Clause-②: no (narrowing) — a record write reaches fewer files: a file its caller may not read is no longer copied or claimed for it. Nothing that was refused becomes accepted.
+
+Metadata-protocol: none (a write path's file bookkeeping; nothing a metadata author writes changes)
+
+<!-- adr-0087: not-required (no-migration-prescription) No metadata moves: no spec key, authorable spelling, object definition, stored row or stored shape is added, removed, renamed or re-shaped, and no exported declaration is removed or narrowed, so there is nothing for `objectstack migrate meta` to rewrite. What changes is which files the file-reference claim path copies or claims for a record write (those its caller may read). The remedy is a change of who writes the reference (a caller who may read the file), never a rewrite of a package source or a stored row. The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id is named or touched (not registered or already-registered); and nothing exported is removed or narrowed (not runtime-interface-only, type-surface-only or runtime-export-retired). -->
+
+**BREAKING** for a write that referenced a file its caller may not read, shipped as `patch` because the metadata protocol does not move.
+
+- **Before.** The claim path copied or claimed a referenced file without asking whether the write's caller may read it.
+- **Now.** Both happen only when the caller may read the file under the download door's rule. Otherwise the record keeps the id as written and owns nothing: the file's readers are unchanged, and the written record's readers gain no access to it through that record.
+- **Fix.** Reference a file as a caller who may read it (its uploader, or a reader of the record that owns it), or upload the file again. A server-side process that copies files between records on purpose runs as a system write.

content/docs/permissions/system-context.mdx

@@ -178,6 +178,7 @@ One of the largest consumers of the flag;
 | 47c | The inbound-mail IMAP sync: `syncMailbox` and `syncAll` admit the system context ALONE | service-mailbox | Get: the sync job reads every mailbox row (its organization, address and IMAP cursor), writes the cursor back, and hands each fetched message to `ingest` under the system context. Lose: nothing a user held — every other caller is refused (`PERMISSION_DENIED`), because a sync stores a mailbox's mail and moves its cursor | `packages/services/service-mailbox/src/sync/mailbox-sync-service.ts#syncMailbox`, `#syncAll` |
 | 47d | The print service REFUSES the system context: `printRecordToFile` admits a human user acting for themselves alone | service-print | Get: nothing — a system caller is refused (`PERMISSION_DENIED`) before any page is read, session minted or browser started. Lose: nothing a user held — the PDF is rendered as the requesting user and never as the system, so a record is archived only by someone who could read it | `packages/services/service-print/src/print-service.ts#refuseUnlessHumanCaller` |
 | 47e | The caller-scoped attach verbs REFUSE the system context: `attachFileAs` and `assertMayAttach` admit a signed-in user alone | service-storage | Get: nothing — a system caller is refused (`PERMISSION_DENIED`); the system attaches through `attachFile`, whose rows name no user. Lose: nothing a user held — the caller-scoped verbs insert the attachment under the caller's context so every insert gate judges that user and both rows name them | `packages/services/service-storage/src/file-attachment-service.ts#requireUserContext` |
+| 47f | The file-reference claim path asks no read verdict of a system write: a record write claims or copies a referenced file only as its caller may read it (the download door's verdict), and a system write carries no caller to ask | service-storage | Get: a system write claims or copies any file the organization-scoped lookup reaches, as before #25692; a deliberate server-side copy between records runs this way. Lose: nothing a user held — a user write is judged by the download door's verdict, and a context-less programmatic write is not asked either | `packages/services/service-storage/src/file-reference-lifecycle.ts#referencingCallerOf` |
 
 ### 5. Actions, metadata plane, provenance, the organization wall
 

packages/services/service-storage/src/attachment-access-hooks.ts

@@ -268,7 +268,10 @@ function asIdList(id: unknown): Array<string | number> | null {
  * context resolved for the object of the CURRENT operation, which here is
  * `sys_attachment`. Every gate in this module asks about the PARENT record's
  * object, never about `sys_attachment`, so carrying any of them across is one
- * object's widening applied to another object's question.
+ * object's widening applied to another object's question. [#25692] The
+ * file-reference lifecycle reads a record write's caller through the same
+ * {@link callerContext}, for the same reason: it asks about a FILE's owning
+ * record, never about the object being written.
  *
  * [#7145] The general rule — which keys those are, why they are dropped by
  * PREFIX rather than by a name list, and why the copy is load-bearing in both
@@ -318,7 +321,7 @@ function asIdList(id: unknown): Array<string | number> | null {
  * object's grant. Where that judgement is not served, record sharing is asked
  * with no depth, so its owner-match stays at its narrowest (`own`) — the safe
  * direction. */
-function callerContext(ctx: any): ExecutionContext {
+export function callerContext(ctx: any): ExecutionContext {
   const exec = ctx?.input?.options?.context;
   if (exec && typeof exec === 'object') {
     return withoutOperationPrivateKeys(exec as Record<string, unknown>);

packages/services/service-storage/src/file-reference-lifecycle.test.ts

@@ -1070,3 +1070,290 @@ describe('File Reference Ownership (ADR-0104 D3 wave 2)', () => {
     expect(engine.calls.filter((c) => c.op === 'update' && c.object === 'sys_file')).toHaveLength(0);
   });
 });
+
+/**
+ * [#25692] A write references only a file its caller may read.
+ *
+ * The hooks ask the read verdict the storage plugin binds (the download
+ * door's own judgement) of the WRITE's caller, before copy-on-claim and
+ * before the claim. Here the verdict is a stand-in that admits a fixed set of
+ * file ids; `file-reference-read-verdict.integration.test.ts` pins the real
+ * binding against the door. Each refusal pin below is ablated in the PR that
+ * introduced it.
+ */
+describe('a write references only a file its caller may read', () => {
+  const CALLER = { userId: 'u_writer', tenantId: 'org_1' };
+  const session = { userId: CALLER.userId, organizationId: CALLER.tenantId };
+
+  /** A verdict that admits only `readable`, recording every question asked. */
+  function verdict(readable: string[]) {
+    const asked: Array<{ fileId: unknown; context: any }> = [];
+    const authorizeRead = vi.fn(async (f: Record<string, unknown>, context: any) => {
+      asked.push({ fileId: f.id, context });
+      return readable.includes(String(f.id));
+    });
+    return { authorizeRead, asked };
+  }
+
+  function installWith(engine: Engine, authorizeRead: any, storage: any = fakeStorage()) {
+    installFileReferenceHooks(engine, () => storage, silentLogger(), authorizeRead);
+    return { storage };
+  }
+
+  /** {@link driveInsert}, as a signed-in caller's write carries it: a hook
+   * session plus the caller's context on the operation options. */
+  async function insertAs(
+    engine: Engine,
+    object: string,
+    data: Record<string, unknown>,
+    id: string,
+    caller: { session?: any; context?: any } = { session, context: CALLER },
+  ) {
+    const ctx: any = {
+      object,
+      event: 'beforeInsert',
+      input: { data, options: caller.context ? { context: caller.context } : {} },
+      session: caller.session,
+      dispatch: recordDispatch(),
+    };
+    await engine.trigger('beforeInsert', ctx);
+    const row: Record<string, unknown> & { id: string } = { ...(ctx.input.data as Record<string, unknown>), id };
+    (engine.tables[object] ??= []).push(row);
+    ctx.event = 'afterInsert';
+    ctx.result = row;
+    await engine.trigger('afterInsert', ctx);
+    return row;
+  }
+
+  async function updateAs(engine: Engine, object: string, id: string, data: Record<string, unknown>) {
+    const ctx: any = {
+      object,
+      event: 'beforeUpdate',
+      input: { id, data, options: { context: CALLER } },
+      session,
+      dispatch: recordDispatch(),
+    };
+    await engine.trigger('beforeUpdate', ctx);
+    const row = (engine.tables[object] ?? []).find((r) => String(r.id) === String(id));
+    if (row) Object.assign(row, ctx.input.data);
+    ctx.event = 'afterUpdate';
+    ctx.result = row;
+    await engine.trigger('afterUpdate', ctx);
+    return row;
+  }
+
+  const ownedByP1 = () => file({ ref_object: 'product', ref_id: 'p1', ref_field: 'image' });
+  const sysFileWrites = (engine: Engine) =>
+    engine.calls.filter((c) => (c.op === 'update' || c.op === 'insert') && c.object === 'sys_file');
+
+  describe('a file another record owns', () => {
+    it('is not copied for a caller the verdict refuses: the value is kept as written', async () => {
+      const engine = fakeEngine({ files: [ownedByP1()], records: { product: [{ id: 'p1', image: 'file_a' }] } });
+      const { authorizeRead } = verdict([]);
+      const { storage } = installWith(engine, authorizeRead);
+
+      const row = await insertAs(engine, 'product', { image: 'file_a' }, 'p2');
+
+      expect(row.image).toBe('file_a');
+      expect(storage.download).not.toHaveBeenCalled();
+      expect(storage.upload).not.toHaveBeenCalled();
+      expect(engine.tables.sys_file).toHaveLength(1);
+      expect(engine.tables.sys_file[0]).toMatchObject({ id: 'file_a', ref_object: 'product', ref_id: 'p1', ref_field: 'image' });
+      expect(sysFileWrites(engine)).toEqual([]);
+    });
+
+    it('is not copied by an update for a caller the verdict refuses', async () => {
+      const engine = fakeEngine({
+        files: [ownedByP1()],
+        records: { product: [{ id: 'p1', image: 'file_a' }, { id: 'p2', image: null }] },
+      });
+      const { authorizeRead } = verdict([]);
+      const { storage } = installWith(engine, authorizeRead);
+
+      const row = await updateAs(engine, 'product', 'p2', { image: 'file_a' });
+
+      expect(row?.image).toBe('file_a');
+      expect(storage.download).not.toHaveBeenCalled();
+      expect(engine.tables.sys_file).toHaveLength(1);
+      expect(engine.tables.sys_file[0]).toMatchObject({ ref_id: 'p1' });
+    });
+
+    it('CONTROL — is copied for a caller the verdict admits, exactly as before; the copy is claimed without a second question', async () => {
+      const engine = fakeEngine({ files: [ownedByP1()], records: { product: [{ id: 'p1', image: 'file_a' }] } });
+      const { authorizeRead, asked } = verdict(['file_a']);
+      const { storage } = installWith(engine, authorizeRead);
+
+      const row = await insertAs(engine, 'product', { image: 'file_a' }, 'p2');
+
+      expect(row.image).not.toBe('file_a');
+      expect(storage.download).toHaveBeenCalledWith('user/file_a.png');
+      const copy = engine.tables.sys_file.find((f) => f.id === row.image)!;
+      expect(copy).toMatchObject({ ref_object: 'product', ref_id: 'p2', ref_field: 'image' });
+      expect(engine.tables.sys_file[0]).toMatchObject({ id: 'file_a', ref_id: 'p1' });
+      // The verdict was asked about the SOURCE only — the stand-in admits
+      // nothing else, so a claim that re-asked about the copy would have
+      // left it unclaimed.
+      expect(asked.map((q) => q.fileId)).toEqual(['file_a']);
+    });
+  });
+
+  describe('a file no record owns', () => {
+    it('is not claimed for a caller the verdict refuses', async () => {
+      const engine = fakeEngine({ files: [file()] });
+      const { authorizeRead } = verdict([]);
+      installWith(engine, authorizeRead);
+
+      const row = await insertAs(engine, 'product', { image: 'file_a' }, 'p1');
+
+      expect(row.image).toBe('file_a');
+      expect(engine.tables.sys_file[0].ref_id ?? null).toBeNull();
+      expect(sysFileWrites(engine)).toEqual([]);
+    });
+
+    it('is not claimed by an update for a caller the verdict refuses', async () => {
+      const engine = fakeEngine({ files: [file()], records: { product: [{ id: 'p1', image: null }] } });
+      const { authorizeRead } = verdict([]);
+      installWith(engine, authorizeRead);
+
+      await updateAs(engine, 'product', 'p1', { image: 'file_a' });
+
+      expect(engine.tables.sys_file[0].ref_id ?? null).toBeNull();
+      expect(sysFileWrites(engine)).toEqual([]);
+    });
+
+    it('a tombstoned file is not brought back for a caller the verdict refuses', async () => {
+      const engine = fakeEngine({ files: [file({ status: 'deleted', deleted_at: '2026-01-01T00:00:00.000Z' })] });
+      const { authorizeRead } = verdict([]);
+      installWith(engine, authorizeRead);
+
+      await insertAs(engine, 'product', { image: 'file_a' }, 'p1');
+
+      expect(engine.tables.sys_file[0]).toMatchObject({ status: 'deleted', deleted_at: '2026-01-01T00:00:00.000Z' });
+      expect(engine.tables.sys_file[0].ref_id ?? null).toBeNull();
+    });
+
+    it('CONTROL — is claimed for a caller the verdict admits', async () => {
+      const engine = fakeEngine({ files: [file()] });
+      const { authorizeRead } = verdict(['file_a']);
+      installWith(engine, authorizeRead);
+
+      await insertAs(engine, 'product', { image: 'file_a' }, 'p1');
+
+      expect(engine.tables.sys_file[0]).toMatchObject({ ref_object: 'product', ref_id: 'p1', ref_field: 'image' });
+    });
+
+    it('each id of a multiple:true field is judged on its own', async () => {
+      const engine = fakeEngine({ files: [file(), file({ id: 'file_b', key: 'user/file_b.png' })] });
+      const { authorizeRead } = verdict(['file_b']);
+      installWith(engine, authorizeRead);
+
+      const row = await insertAs(engine, 'product', { gallery: ['file_a', 'file_b'] }, 'p1');
+
+      expect(row.gallery).toEqual(['file_a', 'file_b']);
+      expect(engine.tables.sys_file.map((f) => [f.id, f.ref_id ?? null])).toEqual([
+        ['file_a', null],
+        ['file_b', 'p1'],
+      ]);
+    });
+  });
+
+  it('constraint verdicts do not run ahead of the read check', async () => {
+    const engine = fakeEngine({
+      files: [file({ mime_type: 'application/pdf', size: 10_000_000 })],
+      registry: {
+        sys_file: REGISTRY.sys_file,
+        product: { name: 'product', fields: { id: { type: 'text' }, image: { type: 'image', accept: ['image/*'], maxSize: 1000 } } },
+      },
+    });
+    const { authorizeRead } = verdict([]);
+    installWith(engine, authorizeRead);
+
+    const row = await insertAs(engine, 'product', { image: 'file_a' }, 'p1');
+
+    expect(row.image).toBe('file_a');
+    expect(engine.tables.sys_file[0].ref_id ?? null).toBeNull();
+  });
+
+  it('CONTROL — a file the caller may read is still judged against its field\'s declaration', async () => {
+    const engine = fakeEngine({
+      files: [file({ mime_type: 'application/pdf', size: 10 })],
+      registry: {
+        sys_file: REGISTRY.sys_file,
+        product: { name: 'product', fields: { id: { type: 'text' }, image: { type: 'image', accept: ['image/*'] } } },
+      },
+    });
+    const { authorizeRead } = verdict(['file_a']);
+    installWith(engine, authorizeRead);
+
+    const err = await insertAs(engine, 'product', { image: 'file_a' }, 'p1').then(() => null, (e) => e);
+
+    expect(err).toBeInstanceOf(FileConstraintError);
+    expect({ code: err.code, status: err.status }).toEqual({ code: 'ERR_FILE_CONSTRAINT', status: 400 });
+  });
+
+  it('asks the verdict of the write\'s caller context, taken from the operation options', async () => {
+    const engine = fakeEngine({ files: [ownedByP1()], records: { product: [{ id: 'p1', image: 'file_a' }] } });
+    const { authorizeRead, asked } = verdict([]);
+    installWith(engine, authorizeRead);
+    const context = { ...CALLER, permissions: ['member_default'], __operationPrivate: 'dropped' };
+
+    await insertAs(engine, 'product', { image: 'file_a' }, 'p2', { session, context });
+
+    expect(asked.length).toBeGreaterThan(0);
+    for (const q of asked) {
+      expect(q.fileId).toBe('file_a');
+      expect(q.context).toMatchObject({ userId: 'u_writer', tenantId: 'org_1', permissions: ['member_default'] });
+      expect(q.context).not.toHaveProperty('__operationPrivate');
+    }
+  });
+
+  it('an authorization-store outage fails the write rather than reading as either answer', async () => {
+    const engine = fakeEngine({ files: [ownedByP1()], records: { product: [{ id: 'p1', image: 'file_a' }] } });
+    const outage = Object.assign(new Error('authorization store unavailable'), { code: 'SERVICE_UNAVAILABLE', status: 503 });
+    const authorizeRead = vi.fn(async () => { throw outage; });
+    const { storage } = installWith(engine, authorizeRead);
+
+    const err = await insertAs(engine, 'product', { image: 'file_a' }, 'p2').then(() => null, (e) => e);
+
+    expect(err).toBe(outage);
+    expect(storage.download).not.toHaveBeenCalled();
+    // The before hook threw, so the driver never wrote the record.
+    expect((engine.tables.product ?? []).map((r) => r.id)).toEqual(['p1']);
+  });
+
+  describe('writes with no caller to ask reference files as before', () => {
+    it('a system write is not asked, and copies as before', async () => {
+      const engine = fakeEngine({ files: [ownedByP1()], records: { product: [{ id: 'p1', image: 'file_a' }] } });
+      const { authorizeRead } = verdict([]);
+      installWith(engine, authorizeRead);
+
+      const row = await insertAs(engine, 'product', { image: 'file_a' }, 'p2', {
+        session: { ...session, isSystem: true },
+        context: { ...CALLER, isSystem: true },
+      });
+
+      expect(authorizeRead).not.toHaveBeenCalled();
+      expect(row.image).not.toBe('file_a');
+    });
+
+    it('a context-less programmatic write is not asked, and claims as before', async () => {
+      const engine = fakeEngine({ files: [file()] });
+      const { authorizeRead } = verdict([]);
+      installWith(engine, authorizeRead);
+
+      await insertAs(engine, 'product', { image: 'file_a' }, 'p1', {});
+
+      expect(authorizeRead).not.toHaveBeenCalled();
+      expect(engine.tables.sys_file[0]).toMatchObject({ ref_id: 'p1' });
+    });
+
+    it('no verdict bound (a bare kernel): nothing is asked, and files are claimed as before', async () => {
+      const engine = fakeEngine({ files: [file()] });
+      install(engine);
+
+      await insertAs(engine, 'product', { image: 'file_a' }, 'p1');
+
+      expect(engine.tables.sys_file[0]).toMatchObject({ ref_id: 'p1' });
+    });
+  });
+});