fix(objectql): extend the credential-column guard from aggregates to filter and sort keys

.changeset/credential-column-filter-sort-guard.md

@@ -0,0 +1,29 @@
+---
+'@objectstack/objectql': patch
+---
+
+Clause-②: no (narrowing)
+Metadata-protocol: none
+
+<!-- adr-0087: not-required (no-migration-prescription) No metadata moves: no spec key, authorable spelling, export or stored shape is removed, renamed or re-shaped, so there is nothing for `objectstack migrate meta` to rewrite. What narrows is a runtime read door: the engine stops EVALUATING a `password` / `secret` column in a non-system caller's filter or sort position, a column every generic read already masks, and the remedy is a change to the caller's query, never to anyone's metadata. No exported declaration is removed or narrowed (not runtime-interface-only or type-surface-only); the package publishes (not unpublished); no ADR-0087 id is named or touched (not registered or already-registered). -->
+
+**BREAKING** (an accept-set narrowing), shipped as `patch`: nothing a metadata author writes changes (`Metadata-protocol: none`), so the level follows the kind, and the banner and the ADR-0087 disposition stay.
+
+Hardening (ADR-0100): the engine's credential-column guard now covers filter and sort keys, not only aggregates. A `password` or `secret` field was already refused as a group-by key or a min / max operand; the same refusal now covers every position that evaluates the stored value.
+
+**FROM.** For every caller except a system context, on `find`, `findOne`, `count`, `aggregate` and predicate `update` / `delete`, and so on every door that reaches them with a caller context (the REST data routes, the MCP bridges, flows, hooks' `ctx.api`, plugin routes):
+
+- A `where` / `filter` predicate on a `password` or `secret` field was evaluated, including a `{ "$field": … }` comparand, a per-aggregation `filter`, an `expand` entry's own `where` / `orderBy`, and a nested-relation condition on a related object's credential field.
+- An `orderBy` key on such a field was admitted.
+- A search scanned such a field when the object listed it in `searchableFields`.
+- An import in `upsert` or `update` mode whose `matchFields` names such a field looked rows up by that field.
+
+**TO.**
+
+- Each of those filter and sort positions is refused `400 INVALID_FIELD`, the same ADR-0112 envelope as the aggregate refusal (with `field`, `object` and `param` naming the position), before any driver is asked. The group-by and min / max refusals are unchanged.
+- A has-a-value check stays accepted: `{ "$null": … }`, `{ "$exists": … }`, `{ "$empty": … }`, or equality with `null`.
+- A search never scans a credential column, even one an object lists in `searchableFields`.
+- An import whose `matchFields` names such a field now meets the same refusal in its match lookup: each row that reaches the lookup is reported failed with that refusal and is not written.
+- A system context is not bound, so the server's own credential lookups are unchanged.
+
+**Fix.** A caller that sorted or filtered on a `password` / `secret` field now receives `400 INVALID_FIELD`: sort or filter by another field, or use a has-a-value check. An import that matched on such a field matches on another field.

content/docs/data-modeling/field-types.mdx

@@ -121,6 +121,15 @@ Reversible, encrypted-at-rest value (DB password, API key, token). Unlike `passw
 { name: 'api_key', label: 'API Key', type: 'secret' }
 ```
 
+**Querying a `password` or `secret` field.** Neither type is a filter, sort or aggregation key.
+Only a has-a-value check is accepted on one: `{ "$null": true | false }`, `{ "$exists": … }`,
+`{ "$empty": … }`, or equality with `null`. Any other operator in `where` or `filter` (equality
+with a value, a range, `$startsWith` / `$contains`, `$in`, a `{ "$field": … }` comparand), an
+`orderBy` entry, a group-by or a min / max over one is refused with `400 INVALID_FIELD` before
+the query runs. That holds for every caller, administrators included. A search also never scans
+such a column, even one listed in `searchableFields`. Only server-side code running as the
+system context may match on the stored value (ADR-0100).
+
 ---
 
 ## Rich Content Types

content/docs/permissions/system-context.mdx

@@ -121,6 +121,7 @@ bugs — it is the protection or the side effect that silently does not happen.
 | 17 | Approval snapshot payload redaction skipped, and so is the snapshot query guard and the reader-locale label overlay | plugin-approvals | Get: the whole snapshot on `find` / `findOne` — the audit/replay channel — and a filter, sort or grouping by it on any read; and a request's `process_label` / `current_step_label` as stored, never in a locale the context carries, so nothing the platform reads back and writes again holds a reader's language. Lose: field-visibility redaction over approval payloads, the refusal of a query over the snapshot for a reader withheld a field of the objects it can reach, and the flow and step labels in the reader's language | `bindSnapshotRedactionMiddleware`, `bindSnapshotPredicateGuard`, `bindReaderLocaleLabelsMiddleware` |
 | 18 | REST anonymous-deny seam satisfied | rest | Get: `enforceAuth` passes with no `userId`. Not reachable from the wire — `isSystem` is never set on an inbound request | `enforceAuth` |
 | 18b | Settings namespace read scope not applied | service-settings | Get: a read of `sys_setting`, `sys_setting_audit` or `sys_platform_setting` across every namespace, whatever each namespace's `readPermission` names — the settings service's own reads of its stores take this path. Lose: the per-namespace read gate the generic data API applies to every other caller, the same rule the settings routes enforce | `settingsReadDoorMiddleware` |
+| 18c | **Credential-column filter / sort refusal not applied** — a `where`, `orderBy`, per-aggregation `filter` or `expand` entry naming a `password` / `secret` column is evaluated | objectql | Get: the server's own match on a stored credential value, a lookup that never returns it. Every other caller, administrators included, is refused `400 INVALID_FIELD` for anything but a has-a-value check (ADR-0100). Lose: nothing a wire caller reaches — `isSystem` is never set on an inbound request | `rejectCredentialEvaluation` |
 
 ### 2. Write pipeline and data integrity
 
@@ -523,6 +524,7 @@ and CI refuses one that differs from what the tree produces.
 | `packages/metadata-core/src/meta-write-capability.ts#metaWriteCapabilityVerdict` | 1 | 51 |
 | `packages/metadata-core/src/meta-write-capability.ts#walledCatalogWriteVerdict` | 1 | 55b |
 | `packages/metadata-core/src/object-schema-fls.ts#isObjectSchemaMaskExempt` | 1 | 8 |
+| `packages/objectql/src/credential-evaluation-guard.ts#rejectCredentialEvaluation` | 1 | 18c |
 | `packages/objectql/src/engine-delete-cascade.ts#recordReferenceCheckElevation` | 1 | 29 |
 | `packages/objectql/src/engine-driver-options.ts#buildDriverOptions` | 1 | 24 |
 | `packages/objectql/src/engine-execution-plumbing.ts#buildSession` | 1 | 63 |

packages/core/src/security/second-object-read-exposure.pin.test.ts

@@ -209,6 +209,15 @@ const CALLERS: Record<string, Classification> = {
     kind: 'not-a-served-read',
     reason: "resolves a seed's references at install, under the system identity; serves nothing to a caller",
   },
+  // ADR-0100: the credential-column guard's filter / sort half.
+  'packages/objectql/src/credential-evaluation-guard.ts': {
+    kind: 'not-a-served-read',
+    reason:
+      "names the object an `expand` entry the caller wrote targets, only to read that object's DECLARED credential " +
+      'fields from the registry and refuse (`INVALID_FIELD` / 400) a filter or sort key on one; it reads no record and ' +
+      "serves nothing of the target — the expanded read itself is the engine's `expandRelatedRecords`. Pinned by " +
+      '`packages/objectql/src/credential-evaluation-guard.test.ts` (the expand refusals assert the driver was never asked)',
+  },
   // [#25522 cut 10] The write-path reference check, moved out of engine.ts.
   'packages/objectql/src/engine-referential-integrity.ts': {
     kind: 'not-a-served-read',

packages/objectql/src/credential-evaluation-guard.test.ts

@@ -0,0 +1,255 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * The credential-column guard's filter / sort half (ADR-0100), against a REAL
+ * {@link ObjectQL} engine over a minimal recording driver: every refusal is
+ * the ADR-0112 `INVALID_FIELD` / 400 envelope and leaves the driver unasked;
+ * a has-a-value check, a system context and every non-credential field are
+ * served as before.
+ */
+
+import { describe, it, expect, beforeEach } from 'vitest';
+import { ObjectQL } from './engine.js';
+import { isCredentialPresenceCondition } from './credential-evaluation-guard.js';
+import type { EngineAggregateOptions, EngineDeleteOptions, EngineQueryOptions, EngineUpdateOptions } from '@objectstack/spec/data';
+
+function makeRecordingDriver() {
+  const rows = new Map<string, Record<string, unknown>[]>();
+  const calls: string[] = [];
+  const matches = (row: Record<string, unknown>, where: any): boolean => {
+    if (!where || typeof where !== 'object') return true;
+    for (const [k, v] of Object.entries(where)) {
+      if (k.startsWith('$')) continue;
+      if (v !== null && typeof v === 'object') continue;
+      if ((row[k] ?? null) !== (v ?? null)) return false;
+    }
+    return true;
+  };
+  const driver: any = {
+    name: 'memory', version: '0.0.0', supports: {},
+    async connect() {}, async disconnect() {}, async checkHealth() { return true; },
+    async execute() { return null; },
+    async find(object: string, ast: any) {
+      calls.push(`find:${object}`);
+      const found = (rows.get(object) ?? []).filter((r) => matches(r, ast?.where));
+      const page = typeof ast?.limit === 'number' ? found.slice(0, ast.limit) : found;
+      return page.map((r) => ({ ...r }));
+    },
+    async findOne(object: string, ast: any) {
+      calls.push(`findOne:${object}`);
+      const r = (rows.get(object) ?? []).find((x) => matches(x, ast?.where));
+      return r ? { ...r } : null;
+    },
+    async count(object: string, ast: any) {
+      calls.push(`count:${object}`);
+      return (rows.get(object) ?? []).filter((r) => matches(r, ast?.where)).length;
+    },
+    async aggregate(object: string) {
+      calls.push(`aggregate:${object}`);
+      return [{ n: (rows.get(object) ?? []).length }];
+    },
+    async create(object: string, data: Record<string, unknown>) {
+      const list = rows.get(object) ?? [];
+      const row = { id: `r${list.length + 1}`, ...data };
+      list.push(row);
+      rows.set(object, list);
+      return { ...row };
+    },
+    async update(object: string, id: string, data: Record<string, unknown>) {
+      calls.push(`update:${object}`);
+      return { id, ...data };
+    },
+    async updateMany(object: string) { calls.push(`updateMany:${object}`); return 0; },
+    async delete(object: string) { calls.push(`delete:${object}`); return true; },
+    async deleteMany(object: string) { calls.push(`deleteMany:${object}`); return 0; },
+    async upsert(_o: string, data: Record<string, unknown>) { return data; },
+    async bulkCreate() { return []; }, async bulkUpdate() { return []; }, async bulkDelete() {},
+    async beginTransaction() { return { commit: async () => {}, rollback: async () => {} }; },
+    async commit() {}, async rollback() {},
+  };
+  return { driver, rows, calls };
+}
+
+const person = {
+  name: 'cg_person', label: 'Person',
+  fields: {
+    id: { name: 'id', label: 'ID', type: 'text' as const },
+    name: { name: 'name', label: 'Name', type: 'text' as const },
+    pin: { name: 'pin', label: 'PIN', type: 'password' as const },
+  },
+};
+
+const vault = {
+  name: 'cg_vault', label: 'Vault',
+  searchableFields: ['name', 'pin'],
+  fields: {
+    id: { name: 'id', label: 'ID', type: 'text' as const },
+    name: { name: 'name', label: 'Name', type: 'text' as const },
+    pin: { name: 'pin', label: 'PIN', type: 'password' as const },
+    token: { name: 'token', label: 'Token', type: 'secret' as const },
+    holder: { name: 'holder', label: 'Holder', type: 'lookup' as const, reference: 'cg_person' },
+  },
+};
+
+const USER: EngineQueryOptions['context'] = { userId: 'u1' };
+const SYSTEM: EngineQueryOptions['context'] = { isSystem: true };
+
+async function buildEngine() {
+  const engine = new ObjectQL();
+  const recording = makeRecordingDriver();
+  engine.registerDriver(recording.driver, true);
+  await engine.init();
+  engine.registry.registerObject(person as any);
+  engine.registry.registerObject(vault as any);
+  recording.rows.set('cg_person', [{ id: 'p1', name: 'Pat', pin: '1234' }]);
+  recording.rows.set('cg_vault', [{ id: 'v1', name: 'alpha', pin: 'p@ssw0rd!', token: 'secret:h1', holder: 'p1' }]);
+  return { engine, ...recording };
+}
+
+/** The ADR-0112 envelope the refusal carries, asserted on the thrown error. */
+async function expectRefused(run: () => Promise<unknown>, field: string, param: string) {
+  let thrown: any;
+  try { await run(); } catch (e) { thrown = e; }
+  expect(thrown, 'the call was refused').toBeDefined();
+  expect(thrown.code).toBe('INVALID_FIELD');
+  expect(thrown.status).toBe(400);
+  expect(thrown.httpStatus).toBe(400);
+  expect(thrown.field).toBe(field);
+  expect(thrown.param).toBe(param);
+}
+
+describe('ADR-0100 credential-column guard — filter and sort keys', () => {
+  let ctx: Awaited<ReturnType<typeof buildEngine>>;
+  beforeEach(async () => { ctx = await buildEngine(); });
+
+  const refusedWheres: Array<[string, EngineQueryOptions['where'], string]> = [
+    ['equality', { pin: 'p@ssw0rd!' }, 'pin'],
+    ['$eq with a value', { pin: { $eq: 'p@ssw0rd!' } }, 'pin'],
+    ['$ne with a value', { pin: { $ne: 'x' } }, 'pin'],
+    ['range', { pin: { $gt: 'o' } }, 'pin'],
+    ['$startsWith', { pin: { $startsWith: 'p@' } }, 'pin'],
+    ['$contains', { pin: { $contains: 'ssw' } }, 'pin'],
+    ['$in', { pin: { $in: ['a', 'b'] } }, 'pin'],
+    ['a secret column', { token: { $startsWith: 'secret:' } }, 'token'],
+    ['inside $or', { $or: [{ name: 'alpha' }, { pin: 'x' }] }, 'pin'],
+    ['inside $not', { $not: { pin: 'x' } }, 'pin'],
+    ['$not over a value operator', { pin: { $not: { $eq: 'x' } } }, 'pin'],
+    ['a cross-field comparand', { name: { $eq: { $field: 'pin' } } }, 'pin'],
+    ['the array form', [['pin', '=', 'p@ssw0rd!']], 'pin'],
+  ];
+
+  for (const [label, where, field] of refusedWheres) {
+    it(`find refuses ${label} on a credential column, before the driver is asked`, async () => {
+      await expectRefused(() => ctx.engine.find('cg_vault', { where, context: USER }), field, 'where');
+      expect(ctx.calls).toEqual([]);
+    });
+  }
+
+  it('a context-less engine call is judged like any non-system caller', async () => {
+    await expectRefused(() => ctx.engine.find('cg_vault', { where: { pin: 'x' } }), 'pin', 'where');
+  });
+
+  it('refuses orderBy on a password and on a secret column', async () => {
+    await expectRefused(
+      () => ctx.engine.find('cg_vault', { orderBy: [{ field: 'pin', order: 'asc' }], context: USER }), 'pin', 'orderBy',
+    );
+    await expectRefused(
+      () => ctx.engine.find('cg_vault', { orderBy: [{ field: 'token', order: 'desc' }], context: USER }), 'token', 'orderBy',
+    );
+    expect(ctx.calls).toEqual([]);
+  });
+
+  it('refuses on findOne, count, aggregate (where and per-aggregation filter), update and delete', async () => {
+    await expectRefused(() => ctx.engine.findOne('cg_vault', { where: { pin: 'x' }, context: USER }), 'pin', 'where');
+    await expectRefused(() => ctx.engine.count('cg_vault', { where: { pin: 'x' }, context: USER }), 'pin', 'where');
+    await expectRefused(
+      () => ctx.engine.aggregate('cg_vault', { where: { pin: 'x' }, aggregations: [{ function: 'count', alias: 'n' }] }, { context: USER }),
+      'pin', 'where',
+    );
+    await expectRefused(
+      () => ctx.engine.aggregate('cg_vault', {
+        aggregations: [{ function: 'count', alias: 'n', filter: { pin: { $gt: 'o' } } }],
+      } satisfies EngineAggregateOptions, { context: USER }),
+      'pin', 'aggregations[0].filter',
+    );
+    await expectRefused(
+      () => ctx.engine.update('cg_vault', { name: 'z' }, { where: { pin: 'x' }, multi: true, context: USER } satisfies EngineUpdateOptions), 'pin', 'where',
+    );
+    await expectRefused(
+      () => ctx.engine.delete('cg_vault', { where: { pin: 'x' }, multi: true, context: USER } satisfies EngineDeleteOptions), 'pin', 'where',
+    );
+    expect(ctx.calls).toEqual([]);
+  });
+
+  it('refuses a credential column in an expand entry, judged against the target object', async () => {
+    await expectRefused(
+      () => ctx.engine.find('cg_vault', { expand: { holder: { object: 'cg_person', where: { pin: '1234' } } }, context: USER } satisfies EngineQueryOptions),
+      'pin', 'expand.holder.where',
+    );
+    await expectRefused(
+      () => ctx.engine.find('cg_vault', { expand: { holder: { object: 'cg_person', orderBy: [{ field: 'pin' }] } }, context: USER } satisfies EngineQueryOptions),
+      'pin', 'expand.holder.orderBy',
+    );
+    expect(ctx.calls).toEqual([]);
+  });
+
+  it('refuses a nested-relation condition on the related object’s credential column', async () => {
+    await expectRefused(
+      () => ctx.engine.find('cg_vault', { where: { holder: { pin: '1234' } }, context: USER }), 'pin', 'where',
+    );
+  });
+
+  const presence: Array<[string, EngineQueryOptions['where']]> = [
+    ['bare null', { pin: null }],
+    ['$null: true', { pin: { $null: true } }],
+    ['$null: false', { pin: { $null: false } }],
+    ['$exists', { pin: { $exists: true } }],
+    ['$empty', { pin: { $empty: false } }],
+    ['$eq: null', { pin: { $eq: null } }],
+    ['$ne: null', { token: { $ne: null } }],
+    ['$not over a presence check', { pin: { $not: { $null: true } } }],
+  ];
+  for (const [label, where] of presence) {
+    it(`serves a has-a-value check (${label})`, async () => {
+      await expect(ctx.engine.find('cg_vault', { where, context: USER })).resolves.toBeInstanceOf(Array);
+      expect(ctx.calls).toContain('find:cg_vault');
+    });
+  }
+
+  it('serves a system context unchanged — the server’s own match on a credential column', async () => {
+    const rows = await ctx.engine.find('cg_vault', { where: { pin: 'p@ssw0rd!' }, context: SYSTEM });
+    expect(rows.map((r: any) => r.id)).toEqual(['v1']);
+    await expect(ctx.engine.find('cg_vault', { orderBy: [{ field: 'pin', order: 'asc' }], context: SYSTEM })).resolves.toHaveLength(1);
+    await expect(ctx.engine.count('cg_vault', { where: { pin: 'p@ssw0rd!' }, context: SYSTEM })).resolves.toBe(1);
+  });
+
+  it('leaves every non-credential field unaffected', async () => {
+    const rows = await ctx.engine.find('cg_vault', {
+      where: { name: 'alpha', holder: 'p1' }, orderBy: [{ field: 'name', order: 'asc' }], context: USER,
+    });
+    expect(rows.map((r: any) => r.id)).toEqual(['v1']);
+    await expect(ctx.engine.count('cg_vault', { where: { name: 'alpha' }, context: USER })).resolves.toBe(1);
+  });
+
+  it('a search never scans a credential column, even one the author listed as searchable', async () => {
+    let seenWhere: unknown;
+    const original = ctx.driver.find;
+    ctx.driver.find = async (object: string, ast: any) => { seenWhere = ast?.where; return original.call(ctx.driver, object, ast); };
+    await ctx.engine.find('cg_vault', { search: 'p@ss', context: USER });
+    expect(JSON.stringify(seenWhere)).toContain('"name"');
+    expect(JSON.stringify(seenWhere)).not.toContain('"pin"');
+  });
+});
+
+describe('isCredentialPresenceCondition', () => {
+  it('accepts only presence shapes', () => {
+    expect(isCredentialPresenceCondition(null)).toBe(true);
+    expect(isCredentialPresenceCondition({ $null: true })).toBe(true);
+    expect(isCredentialPresenceCondition({ $ne: null })).toBe(true);
+    expect(isCredentialPresenceCondition({})).toBe(false);
+    expect(isCredentialPresenceCondition('x')).toBe(false);
+    expect(isCredentialPresenceCondition({ $null: 'yes' })).toBe(false);
+    expect(isCredentialPresenceCondition({ $null: true, $eq: 'x' })).toBe(false);
+    expect(isCredentialPresenceCondition({ $exists: true, $empty: false })).toBe(true);
+  });
+});