fix(metadata): the object read's sortability follows the caller's field permission for every caller class (#22250)

.changeset/22250-meta-fls-runtime-sortability.md

@@ -0,0 +1,16 @@
+---
+"@objectstack/metadata-core": minor
+"@objectstack/rest": minor
+"@objectstack/runtime": patch
+---
+
+fix(metadata): the object read's `sortability` follows the caller's field permission for every caller class
+
+Clause-②: yes (widening)
+
+- **What was wrong.** ADR-0106 D4 serves a caller holding `manage_metadata`, `studio.access` or `setup.access` (a platform admin, and an organization admin through `setup.access`) the object schema UNMASKED, because Studio and Setup authoring need the whole definition. The `sortability` projection served beside it on `GET /meta/object/:name` (and the runtime dispatcher's `/metadata/object/:name`) was derived from that unmasked document, so it marked a field `sortable: true` that the same caller's field permission marks unreadable, while the data route refuses a sort or filter on that field with `403 PERMISSION_DENIED`. A client building its sort picker from the projection offered a click the server refuses.
+- **What changes.** `sortability` is now derived from the caller's runtime view: the served document for every caller the mask projects (unchanged), and for a D4-exempt caller the fields their own field permission reads. The definition (`item.fields`) is unchanged for every class: whole for an exempt caller, masked for everyone else. The data route and `/auth/me/permissions` are unchanged.
+- **Caching.** On the cached `GET /meta/object/:name` branch an exempt caller's ETag folds a fingerprint of the fields withheld from their projection, and the conditional request is judged after it is folded. An exempt caller whose permission withholds nothing keeps the byte-identical ETag. An exempt caller and a projected caller denied the same field never share a validator.
+- **Failure posture.** The exempt caller's field permission is asked lazily, only by the item read that serves `sortability`, and never turns their definition read into a fault: if the security service throws or cannot answer, `sortability` is derived from the served document as before, with one `warn`.
+- **New in `@objectstack/metadata-core`.** `resolveObjectSchemaRuntimeView(document, posture)` returns `{ document, withheld, fingerprint }`, the view the runtime projections beside an object schema are derived from. The types `ObjectSchemaRuntimeView` and `ObjectSchemaRuntimeFieldPosture` are exported with it, and the `passthrough` member of `ObjectSchemaMaskPosture` gains an optional `runtime` (set on an `exempt` posture for a capability-exempt caller in a deployment with a `security` service).
+- **New in `@objectstack/rest`.** `translateMetaEnvelope` takes an optional fifth argument, `runtimeDocument` (default: the served document), and `MetaItemAnswerSources.translateEnvelope` an optional third. Nothing is removed, renamed or narrowed.

content/docs/references/api/protocol.mdx

@@ -1360,7 +1360,7 @@ Enable package response
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **fields** | `Record<string, { sortable: boolean; reason?: 'virtual-type'; caveat?: 'unprovisioned-anchor' }>` | ✅ | Verdict per sortable-addressable column, keyed by field name. The domain is the served field map plus the always-provisioned `id`; a name absent from this map (an unknown field, a dotted path, an unprovisioned audit column) has no platform sort behind it and must get no sort affordance. |
+| **fields** | `Record<string, { sortable: boolean; reason?: 'virtual-type'; caveat?: 'unprovisioned-anchor' }>` | ✅ | Verdict per sortable-addressable column, keyed by field name. The domain is the served field map plus the always-provisioned `id`, less any field the caller's own field permission withholds: a caller exempt from the object-schema mask (ADR-0106 D4) is served such fields in `item` and gets no entry for them; for every other caller the served map already excludes them. A name absent from this map (an unknown field, a dotted path, an unprovisioned audit column, a field the caller may not read) has no platform sort behind it and must get no sort affordance. |
 
 
 ---

content/docs/references/api/sortability.mdx

@@ -36,12 +36,15 @@ records as deliberately rejected).
 `assertSortFieldsExist` (`@objectstack/metadata-protocol`, #6994) and
 `assertOrderByIsMaterializable` (`@objectstack/objectql`, #7095) — actually
 do with an `orderBy` over the name. Three verdicts are REFUSALS, and one
-measured degradation is not refused; the projection covers all four:
+measured degradation is not refused; the projection covers all four, and
+one more absence that is a fact about the CALLER, not the column (5.):
 
 1. **Unknown name** — not a field of the object. Refused (`400
    INVALID_SORT`). Encoded as ABSENCE: the projection's domain is exactly
-   the served field map plus the always-provisioned `id`, so a name with no
-   entry has no platform sort behind it and gets no affordance.
+   the served field map plus the always-provisioned `id`, less any field
+   the caller's own field permission withholds (5. — which removes nothing
+   for every caller but a D4-exempt one), so a name with no entry has no
+   platform sort behind it and gets no affordance.
 2. **Dotted path** (`account.name`) — crosses into a related record no
    driver joins for. Refused. Encoded as absence too: entries are keyed by
    whole-column field names, and a dotted name can never appear as one.
@@ -62,6 +65,15 @@ measured degradation is not refused; the projection covers all four:
    so the entry stays `sortable: true` — the enforcement fact — and carries
    `caveat: 'unprovisioned-anchor'` so a consumer can choose a conservative
    affordance without re-deriving federation from the document.
+5. **Withheld by the caller's field permission** (ADR-0106 D1/D4, #22250) —
+   a field the caller's own field-level security marks unreadable. Refused
+   for that caller by the data route's field guard (`403`: filtering or
+   sorting on a hidden field would leak its values). Encoded as absence: no
+   entry, whatever the caller's class — including a caller exempt from the
+   object-schema mask (ADR-0106 D4: `manage_metadata`, `studio.access`,
+   `setup.access`), who IS served the field in `item.fields` because
+   authoring needs the whole definition. For every other caller the mask
+   has already removed the field from `item`, so this is case 1's absence.
 
 ## Considered and deliberately NOT members
 
@@ -132,7 +144,7 @@ const result = FieldSortabilitySchema.parse(data);
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **fields** | `Record<string, { sortable: boolean; reason?: 'virtual-type'; caveat?: 'unprovisioned-anchor' }>` | ✅ | Verdict per sortable-addressable column, keyed by field name. The domain is the served field map plus the always-provisioned `id`; a name absent from this map (an unknown field, a dotted path, an unprovisioned audit column) has no platform sort behind it and must get no sort affordance. |
+| **fields** | `Record<string, { sortable: boolean; reason?: 'virtual-type'; caveat?: 'unprovisioned-anchor' }>` | ✅ | Verdict per sortable-addressable column, keyed by field name. The domain is the served field map plus the always-provisioned `id`, less any field the caller's own field permission withholds: a caller exempt from the object-schema mask (ADR-0106 D4) is served such fields in `item` and gets no entry for them; for every other caller the served map already excludes them. A name absent from this map (an unknown field, a dotted path, an unprovisioned audit column, a field the caller may not read) has no platform sort behind it and must get no sort affordance. |
 
 ### Nested Shape: `ObjectSortability.fields[string]`
 

packages/metadata-core/src/object-schema-fls.test.ts

@@ -15,6 +15,7 @@ import {
     normalizeIfNoneMatch,
     objectFieldVisibilityFingerprint,
     resolveObjectSchemaMaskPosture,
+    resolveObjectSchemaRuntimeView,
     OBJECT_SCHEMA_MASK_DISABLE_ENV,
     OBJECT_SCHEMA_MASK_EXEMPT_CAPABILITIES,
     OBJECT_SCHEMA_MASK_UNDETERMINED_METRIC,
@@ -164,7 +165,10 @@ describe('[#7020] the D4 read exemption is derived from the write gate', () => {
             security: { getMetadataReadableFields: () => { asked++; throw new Error('must not be consulted'); } },
             enabled: true,
         });
-        expect(posture).toEqual({ kind: 'passthrough', reason: 'exempt' });
+        // [#22250] The posture carries the LAZY runtime question — resolving the
+        // posture still asks nothing, and the definition still passes through.
+        expect(posture).toMatchObject({ kind: 'passthrough', reason: 'exempt' });
+        expect(posture.kind === 'passthrough' && typeof posture.runtime).toBe('function');
         expect(asked).toBe(0);
     });
 
@@ -291,3 +295,136 @@ describe('[ADR-0106 D7] the metadata-plane query is preferred when the service o
         expect(posture).toMatchObject({ kind: 'project', readable: new Set(['id']) });
     });
 });
+
+/**
+ * [#22250] D4 exempts the DEFINITION (Studio/Setup authoring needs the whole
+ * schema), not the runtime projections served beside it: those follow the
+ * caller's own field permission, which the data route enforces for every
+ * class. Pinned per caller class, with the member as the control.
+ */
+describe('[#22250] the D4 exemption covers the definition, not the runtime view', () => {
+    const CLASSES = {
+        'platform admin': { userId: 'u_platform_admin', systemPermissions: ['manage_metadata', 'studio.access', 'setup.access'] },
+        'organization admin': { userId: 'u_org_admin', systemPermissions: ['manage_org_users', 'setup.access', 'setup.write'] },
+        member: { userId: 'u_member', systemPermissions: [] as string[] },
+    } as const;
+
+    const resolveFor = (context: unknown, readable: () => string[] | undefined, telemetry?: Parameters<typeof resolveObjectSchemaMaskPosture>[0]['telemetry']) => {
+        let asked = 0;
+        const security = { getMetadataReadableFields: () => { asked++; return readable(); } };
+        return {
+            posture: resolveObjectSchemaMaskPosture({ objectName: 'account', context, security, enabled: true, telemetry }),
+            asked: () => asked,
+        };
+    };
+
+    for (const [label, context] of Object.entries(CLASSES)) {
+        it(`${label}: the runtime view holds only the readable fields, whatever the definition serves`, async () => {
+            const { posture: pending } = resolveFor(context, () => ['id']);
+            const posture = await pending;
+            const served = applyObjectSchemaMask(OBJECT, posture).document;
+            const view = await resolveObjectSchemaRuntimeView(served, posture);
+
+            expect(Object.keys(view.document.fields)).toEqual(['id']);
+            if (label === 'member') {
+                // Control: the definition mask already projects a member, so the
+                // runtime view IS the served document — same reference, nothing
+                // withheld on top, no second fingerprint.
+                expect(posture.kind).toBe('project');
+                expect(Object.keys(served.fields)).toEqual(['id']);
+                expect(view.document).toBe(served);
+                expect(view.withheld).toEqual([]);
+                expect(view.fingerprint).toBe('');
+            } else {
+                // The exempt definition stays whole, for authoring...
+                expect(posture).toMatchObject({ kind: 'passthrough', reason: 'exempt' });
+                expect(served).toBe(OBJECT);
+                expect(Object.keys(served.fields)).toEqual(['id', 'salary_grade']);
+                // ...and only the runtime view drops what the caller cannot read.
+                expect(view.withheld).toEqual(['salary_grade']);
+                expect(view.fingerprint).not.toBe('');
+            }
+        });
+    }
+
+    it('one field permission, one runtime view, whatever the class', async () => {
+        const views = await Promise.all(Object.values(CLASSES).map(async (context) => {
+            const posture = await resolveFor(context, () => ['id']).posture;
+            return (await resolveObjectSchemaRuntimeView(applyObjectSchemaMask(OBJECT, posture).document, posture)).document;
+        }));
+        for (const view of views) expect(view.fields).toEqual(views[2].fields);
+    });
+
+    it('never shares a validator with the definition mask over the same denied set', async () => {
+        const admin = await resolveFor(CLASSES['platform admin'], () => ['id']).posture;
+        const member = await resolveFor(CLASSES.member, () => ['id']).posture;
+        const adminView = await resolveObjectSchemaRuntimeView(OBJECT, admin);
+        const memberMask = applyObjectSchemaMask(OBJECT, member);
+        // Same denied field, different served bodies (the admin's carries the
+        // field in `item`): the two fingerprints must differ or a 304 crosses over.
+        expect(memberMask.fingerprint).toBe(objectFieldVisibilityFingerprint(['salary_grade']));
+        expect(adminView.fingerprint).not.toBe(memberMask.fingerprint);
+    });
+
+    it('an unrestricted exempt caller: same reference, empty fingerprint (D3 byte-identity)', async () => {
+        const posture = await resolveFor(CLASSES['platform admin'], () => ['id', 'salary_grade']).posture;
+        const view = await resolveObjectSchemaRuntimeView(OBJECT, posture);
+        expect(view.document).toBe(OBJECT);
+        expect(view.fingerprint).toBe('');
+    });
+
+    it('asks the service lazily and at most once per posture, and never for a document without `fields`', async () => {
+        const { posture: pending, asked } = resolveFor(CLASSES['platform admin'], () => ['id']);
+        const posture = await pending;
+        expect(asked()).toBe(0);
+        await resolveObjectSchemaRuntimeView({ name: 'not_an_object_schema' }, posture);
+        expect(asked()).toBe(0);
+        await resolveObjectSchemaRuntimeView(OBJECT, posture);
+        await resolveObjectSchemaRuntimeView(OBJECT, posture);
+        expect(asked()).toBe(1);
+        // The input is never mutated.
+        expect(Object.keys(OBJECT.fields)).toEqual(['id', 'salary_grade']);
+    });
+
+    it('`undefined` settles undetermined: the view is the served document, warned and counted once', async () => {
+        const warns: unknown[] = [];
+        const counters: unknown[] = [];
+        const posture = await resolveFor(CLASSES['platform admin'], () => undefined, {
+            warn: (m, meta) => warns.push([m, meta]),
+            counter: (n, labels) => counters.push([n, labels]),
+        }).posture;
+        const view = await resolveObjectSchemaRuntimeView(OBJECT, posture);
+        expect(view.document).toBe(OBJECT);
+        expect(view.fingerprint).toBe('');
+        expect(warns).toHaveLength(1);
+        expect(counters).toEqual([[OBJECT_SCHEMA_MASK_UNDETERMINED_METRIC, { object: 'account' }]]);
+    });
+
+    it('a throwing service never faults the exempt caller (D4) — the view settles undetermined, warned', async () => {
+        const warns: unknown[] = [];
+        const posture = await resolveObjectSchemaMaskPosture({
+            objectName: 'account',
+            context: CLASSES['platform admin'],
+            security: { getMetadataReadableFields: () => { throw new Error('security down'); } },
+            enabled: true,
+            telemetry: { warn: (m, meta) => warns.push([m, meta]) },
+        });
+        const view = await resolveObjectSchemaRuntimeView(OBJECT, posture);
+        expect(view.document).toBe(OBJECT);
+        expect(view.fingerprint).toBe('');
+        expect(warns).toHaveLength(1);
+    });
+
+    it('`isSystem` and a deployment with no security service carry no runtime question', async () => {
+        const system = await resolveFor({ isSystem: true }, () => ['id']);
+        const systemPosture = await system.posture;
+        expect(systemPosture).toEqual({ kind: 'passthrough', reason: 'exempt' });
+        expect((await resolveObjectSchemaRuntimeView(OBJECT, systemPosture)).document).toBe(OBJECT);
+        expect(system.asked()).toBe(0);
+
+        const noService = await resolveObjectSchemaMaskPosture({
+            objectName: 'account', context: CLASSES['platform admin'], security: undefined, enabled: true,
+        });
+        expect(noService).toEqual({ kind: 'passthrough', reason: 'exempt' });
+    });
+});