diff --git a/.changeset/24677-reference-into-stored-metadata-body-refused.md b/.changeset/24677-reference-into-stored-metadata-body-refused.md new file mode 100644 index 00000000000..e21818c1c72 --- /dev/null +++ b/.changeset/24677-reference-into-stored-metadata-body-refused.md @@ -0,0 +1,23 @@ +--- +'@objectstack/lint': minor +'@objectstack/spec': patch +--- + +fix(lint)!: a `lookup` or `master_detail` field whose `reference` is a stored-metadata-body table is refused where it is declared + +Clause-②: no (narrowing) + + + +**BREAKING** (an authoring narrowing), shipped as `minor` under the launch-window convention for breaking changes. + +A `lookup` or `master_detail` whose `reference` names a table of the stored-metadata-body family (`STORED_METADATA_BODY_OBJECTS`: `sys_metadata`, `sys_metadata_history`) is an error. The engine never hydrates such a field: `$expand` keeps the stored id and never nests the row, a formula or a validation rule never reads through it, and a master-detail child's `parent` root stays unbound, so every write a `parent` rule judges is refused. `os validate`, `os build` and the object save door (`PUT /api/v1/meta/object/:name`, and a draft's promotion) run the same pass, under the existing `EXPRESSION_INVALID` rule, located at the field's `reference`. + +- FROM: `meta: { type: 'lookup', reference: 'sys_metadata' }` (or a `master_detail`, or `sys_metadata_history`) saved with a 200 and built with exit 0, and then only ever showed an id. TO: refused, with `422 INVALID_METADATA` at the save door and an error at `os validate` / `os build`. The message names the field, its type and the target, and ends on the family's one prescription. +- A reference to any other table, platform tables such as `sys_user` included, is unchanged. +- A `user` field whose authored `reference` names a family table is refused the same way: the parse accepts an authored `reference` on a `user` field, and it is the target every reader resolves. +- The refusal of a formula or a validation rule that reads through such a field is unchanged, and is still reported at that expression. +- A stored object row that already declares such a field keeps loading: no authoring rule runs when a stored row is rehydrated, and the engine guards keep refusing every relationship read into the family. Its next active save, or a draft's promotion, through the metadata API is refused until the field is removed or retargeted. The engine guards stay, for every path that runs no authoring rule. +- `@objectstack/spec` (`patch`): the `reference` describe states the convention, so the generated field reference says it too. + +What to do. Remove the field, or retarget it. To point a record at a metadata item, store the item's type and name in `text` fields and read the item through the metadata API (`GET /api/v1/meta/:type/:name`), where its body is projected. diff --git a/content/docs/data-modeling/field-types.mdx b/content/docs/data-modeling/field-types.mdx index 58cf6664b25..321d5fc400d 100644 --- a/content/docs/data-modeling/field-types.mdx +++ b/content/docs/data-modeling/field-types.mdx @@ -307,7 +307,7 @@ Reference to a record in another object (foreign key). | Property | Type | Default | Description | |:---|:---|:---|:---| -| `reference` | `string` | **required** | Target object name (snake_case) | +| `reference` | `string` | **required** | Target object name (snake_case). A stored-metadata table (`sys_metadata`, `sys_metadata_history`) is refused — see [below](#a-relationship-never-targets-a-stored-metadata-table) | | `referenceFilters` | `string[]` | — | **Removed** (#2377, ADR-0049) — no longer a recognized field property. `FieldSchema` is a strict object, so an unknown key is **rejected with guidance**, not silently stripped (ADR-0078): the error echoes the offending key and prescribes the replacement. Use structured `lookupFilters` + `dependsOn` instead; see [Relationships](/docs/data-modeling/relationships) | | `deleteBehavior` | `'restrict' \| 'cascade' \| 'set_null'` | `'set_null'` | Behavior when referenced record is deleted. On a *required* lookup `set_null` is escalated to `restrict`, since a NOT NULL foreign key cannot be cleared — **whether the `set_null` was defaulted or written out explicitly**. On a `multiple: true` required lookup the escalation is judged per referencing row: only a row the member removal would leave EMPTY is refused. `cascade` and `restrict` are the values honored as written. Where `set_null` does run, a `multiple: true` lookup loses only the deleted **member** — the other members are kept, and a set emptied that way is stored as `[]`, never `null` | @@ -329,7 +329,7 @@ Parent-child relationship (cascading delete by default). | Property | Type | Default | Description | |:---|:---|:---|:---| -| `reference` | `string` | **required** | Target (master) object name | +| `reference` | `string` | **required** | Target (master) object name. A stored-metadata table is refused — see [below](#a-relationship-never-targets-a-stored-metadata-table) | | `referenceFilters` | `string[]` | — | **Removed** (#2377, ADR-0049) — no longer a recognized field property. `FieldSchema` is a strict object, so an unknown key is **rejected with guidance**, not silently stripped (ADR-0078): the error echoes the offending key and prescribes the replacement. Use structured `lookupFilters` + `dependsOn` instead; see [Relationships](/docs/data-modeling/relationships) | | `deleteBehavior` | `'restrict' \| 'cascade'` | `'cascade'` | Behavior when parent is deleted. `restrict` refuses the parent delete while children exist; `cascade` (and omitting the key) deletes the children with the parent. An explicit `set_null` is a **parse-time rejection** on this type (#9689) — a detail row cannot outlive its master; use a `lookup` if children must survive the parent | | `inlineEdit` | `boolean \| 'grid' \| 'form'` | — | Edit child records inline on the parent create/edit form (`true` = auto-pick, `'grid'`, or `'form'`) | @@ -347,6 +347,10 @@ Parent-child relationship (cascading delete by default). } ``` +#### A relationship never targets a stored-metadata table + +A `lookup` or `master_detail` whose `reference` is `sys_metadata` or `sys_metadata_history` is refused by `os validate`, `os build` and the object save door alike. Those rows carry serialized metadata bodies, which are served only through the metadata API (`GET /api/v1/meta/:type/:name`), where the body is projected; the engine never hydrates a reference into them, so `$expand` would keep the stored id and never nest the row, and a master-detail child's `parent` root would stay unbound, refusing every write a `parent` rule judges. To point a record at a metadata item, store its type and name in `text` fields and read the item through the metadata API. + ### `tree` Self-referential hierarchy (e.g., categories, org chart). diff --git a/content/docs/references/data/field.mdx b/content/docs/references/data/field.mdx index 2ccdfe9827e..0597c0bee21 100644 --- a/content/docs/references/data/field.mdx +++ b/content/docs/references/data/field.mdx @@ -76,7 +76,7 @@ const result = CurrencyConfigSchema.parse(data); | **maxSize** | `integer` | optional | Maximum permitted file size in BYTES for media fields. Enforced on write against the stored file size, not just checked in the browser. | | **options** | `{ label: string; value: string; description?: string; color?: string; … }[]` | optional | Static options for the option types. A `select` / `radio` field needs a non-empty list here or a `picklist` — with neither (or `options: []`) it is refused at parse. | | **picklist** | `string` | optional | Name of a shared `picklist` whose options this field offers — instead of `options`, never with it. Option types only (select, radio, multiselect, checkboxes, tags). A select / radio declares this or a non-empty `options`; with neither it is refused at parse. | -| **reference** | `string` | optional | Target object name (snake_case) for lookup/master_detail fields. Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. On a `tree` field it is optional and, if given, must be the declaring object's own name — the object schema refuses any other target, and writes the own name when it is left out. | +| **reference** | `string` | optional | Target object name (snake_case) for lookup/master_detail fields. Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. Never a stored-metadata table (`sys_metadata`, `sys_metadata_history`): their rows are served only through the metadata API, so the engine never hydrates a reference into one, and `os validate`, `os build` and the object save door refuse it. On a `tree` field it is optional and, if given, must be the declaring object's own name — the object schema refuses any other target, and writes the own name when it is left out. | | **referenceVia** | `string` | optional | Declares this text field as the id half of a polymorphic pointer pair (ADR-0052 §5 ActivityPointer): the value is a record id of the object named by the SIBLING FIELD this key names — e.g. `record_id` with `referenceVia: 'object_name'`. The sibling must be a declared field on the same object holding an object machine name. Text fields only; mutually exclusive with `reference` (a static and a per-record target contradict). Enforced today at seed load: the value resolves as a natural key against the object the sibling column names, and an unresolvable pointer is refused loudly instead of stored verbatim. Adds no referential integrity or $expand behavior. | | **referenceDeletedWhen** | `{ field: string; value: string }` | optional | Marks the record a `referenceVia` pointer names as PLATFORM-DELETED on rows where a sibling `select` field holds one declared option — e.g. `record_id` with `{ field: 'cancel_reason', value: 'record_deleted' }`. A renderer then draws a tombstone from that option's own label, reads nothing and links nothing. Valid only beside `referenceVia`; `field` must be a single-value `select` on the same object declaring inline `options`, and `value` one of its option values — any other shape is refused at parse. | | **mailAssociation** | `boolean` | optional | Inbound mail association (`email` fields only): TRUE marks this field as identifying a correspondent. When a mailbox service ingests a message, it associates the message with every record of this object whose value in this field equals one of the message's participant addresses (From, Sender, Reply-To, To and Cc, plus those of a forwarded message it carries; ASCII case-insensitive; the receiving mailbox's own addresses excluded; only within the mailbox's organization). A message no record matches waits in the mailbox's unassociated pool. Refused on any other type. Takes effect only where a mailbox service is composed. | diff --git a/content/docs/references/data/object.mdx b/content/docs/references/data/object.mdx index 82c3872a432..61bf66bb979 100644 --- a/content/docs/references/data/object.mdx +++ b/content/docs/references/data/object.mdx @@ -242,7 +242,7 @@ const result = ApiMethod.parse(data); | **maxSize** | `integer` | optional | Maximum permitted file size in BYTES for media fields. Enforced on write against the stored file size, not just checked in the browser. | | **options** | `{ label: string; value: string; description?: string; color?: string; … }[]` | optional | Static options for the option types. A `select` / `radio` field needs a non-empty list here or a `picklist` — with neither (or `options: []`) it is refused at parse. | | **picklist** | `string` | optional | Name of a shared `picklist` whose options this field offers — instead of `options`, never with it. Option types only (select, radio, multiselect, checkboxes, tags). A select / radio declares this or a non-empty `options`; with neither it is refused at parse. | -| **reference** | `string` | optional | Target object name (snake_case) for lookup/master_detail fields. Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. On a `tree` field it is optional and, if given, must be the declaring object's own name — the object schema refuses any other target, and writes the own name when it is left out. | +| **reference** | `string` | optional | Target object name (snake_case) for lookup/master_detail fields. Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. Never a stored-metadata table (`sys_metadata`, `sys_metadata_history`): their rows are served only through the metadata API, so the engine never hydrates a reference into one, and `os validate`, `os build` and the object save door refuse it. On a `tree` field it is optional and, if given, must be the declaring object's own name — the object schema refuses any other target, and writes the own name when it is left out. | | **referenceVia** | `string` | optional | Declares this text field as the id half of a polymorphic pointer pair (ADR-0052 §5 ActivityPointer): the value is a record id of the object named by the SIBLING FIELD this key names — e.g. `record_id` with `referenceVia: 'object_name'`. The sibling must be a declared field on the same object holding an object machine name. Text fields only; mutually exclusive with `reference` (a static and a per-record target contradict). Enforced today at seed load: the value resolves as a natural key against the object the sibling column names, and an unresolvable pointer is refused loudly instead of stored verbatim. Adds no referential integrity or $expand behavior. | | **referenceDeletedWhen** | `{ field: string; value: string }` | optional | Marks the record a `referenceVia` pointer names as PLATFORM-DELETED on rows where a sibling `select` field holds one declared option — e.g. `record_id` with `{ field: 'cancel_reason', value: 'record_deleted' }`. A renderer then draws a tombstone from that option's own label, reads nothing and links nothing. Valid only beside `referenceVia`; `field` must be a single-value `select` on the same object declaring inline `options`, and `value` one of its option values — any other shape is refused at parse. | | **mailAssociation** | `boolean` | optional | Inbound mail association (`email` fields only): TRUE marks this field as identifying a correspondent. When a mailbox service ingests a message, it associates the message with every record of this object whose value in this field equals one of the message's participant addresses (From, Sender, Reply-To, To and Cc, plus those of a forwarded message it carries; ASCII case-insensitive; the receiving mailbox's own addresses excluded; only within the mailbox's organization). A message no record matches waits in the mailbox's unassociated pool. Refused on any other type. Takes effect only where a mailbox service is composed. | @@ -598,7 +598,7 @@ const result = ApiMethod.parse(data); | **maxSize** | `integer` | optional | Maximum permitted file size in BYTES for media fields. Enforced on write against the stored file size, not just checked in the browser. | | **options** | `{ label: string; value: string; description?: string; color?: string; … }[]` | optional | Static options for the option types. A `select` / `radio` field needs a non-empty list here or a `picklist` — with neither (or `options: []`) it is refused at parse. | | **picklist** | `string` | optional | Name of a shared `picklist` whose options this field offers — instead of `options`, never with it. Option types only (select, radio, multiselect, checkboxes, tags). A select / radio declares this or a non-empty `options`; with neither it is refused at parse. | -| **reference** | `string` | optional | Target object name (snake_case) for lookup/master_detail fields. Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. On a `tree` field it is optional and, if given, must be the declaring object's own name — the object schema refuses any other target, and writes the own name when it is left out. | +| **reference** | `string` | optional | Target object name (snake_case) for lookup/master_detail fields. Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. Never a stored-metadata table (`sys_metadata`, `sys_metadata_history`): their rows are served only through the metadata API, so the engine never hydrates a reference into one, and `os validate`, `os build` and the object save door refuse it. On a `tree` field it is optional and, if given, must be the declaring object's own name — the object schema refuses any other target, and writes the own name when it is left out. | | **referenceVia** | `string` | optional | Declares this text field as the id half of a polymorphic pointer pair (ADR-0052 §5 ActivityPointer): the value is a record id of the object named by the SIBLING FIELD this key names — e.g. `record_id` with `referenceVia: 'object_name'`. The sibling must be a declared field on the same object holding an object machine name. Text fields only; mutually exclusive with `reference` (a static and a per-record target contradict). Enforced today at seed load: the value resolves as a natural key against the object the sibling column names, and an unresolvable pointer is refused loudly instead of stored verbatim. Adds no referential integrity or $expand behavior. | | **referenceDeletedWhen** | `{ field: string; value: string }` | optional | Marks the record a `referenceVia` pointer names as PLATFORM-DELETED on rows where a sibling `select` field holds one declared option — e.g. `record_id` with `{ field: 'cancel_reason', value: 'record_deleted' }`. A renderer then draws a tombstone from that option's own label, reads nothing and links nothing. Valid only beside `referenceVia`; `field` must be a single-value `select` on the same object declaring inline `options`, and `value` one of its option values — any other shape is refused at parse. | | **mailAssociation** | `boolean` | optional | Inbound mail association (`email` fields only): TRUE marks this field as identifying a correspondent. When a mailbox service ingests a message, it associates the message with every record of this object whose value in this field equals one of the message's participant addresses (From, Sender, Reply-To, To and Cc, plus those of a forwarded message it carries; ASCII case-insensitive; the receiving mailbox's own addresses excluded; only within the mailbox's organization). A message no record matches waits in the mailbox's unassociated pool. Refused on any other type. Takes effect only where a mailbox service is composed. | diff --git a/content/docs/references/system/migration.mdx b/content/docs/references/system/migration.mdx index 63ac5b26c39..1efca06db3a 100644 --- a/content/docs/references/system/migration.mdx +++ b/content/docs/references/system/migration.mdx @@ -77,7 +77,7 @@ Add a new field to an existing object | **maxSize** | `integer` | optional | Maximum permitted file size in BYTES for media fields. Enforced on write against the stored file size, not just checked in the browser. | | **options** | `{ label: string; value: string; description?: string; color?: string; … }[]` | optional | Static options for the option types. A `select` / `radio` field needs a non-empty list here or a `picklist` — with neither (or `options: []`) it is refused at parse. | | **picklist** | `string` | optional | Name of a shared `picklist` whose options this field offers — instead of `options`, never with it. Option types only (select, radio, multiselect, checkboxes, tags). A select / radio declares this or a non-empty `options`; with neither it is refused at parse. | -| **reference** | `string` | optional | Target object name (snake_case) for lookup/master_detail fields. Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. On a `tree` field it is optional and, if given, must be the declaring object's own name — the object schema refuses any other target, and writes the own name when it is left out. | +| **reference** | `string` | optional | Target object name (snake_case) for lookup/master_detail fields. Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. Never a stored-metadata table (`sys_metadata`, `sys_metadata_history`): their rows are served only through the metadata API, so the engine never hydrates a reference into one, and `os validate`, `os build` and the object save door refuse it. On a `tree` field it is optional and, if given, must be the declaring object's own name — the object schema refuses any other target, and writes the own name when it is left out. | | **referenceVia** | `string` | optional | Declares this text field as the id half of a polymorphic pointer pair (ADR-0052 §5 ActivityPointer): the value is a record id of the object named by the SIBLING FIELD this key names — e.g. `record_id` with `referenceVia: 'object_name'`. The sibling must be a declared field on the same object holding an object machine name. Text fields only; mutually exclusive with `reference` (a static and a per-record target contradict). Enforced today at seed load: the value resolves as a natural key against the object the sibling column names, and an unresolvable pointer is refused loudly instead of stored verbatim. Adds no referential integrity or $expand behavior. | | **referenceDeletedWhen** | `{ field: string; value: string }` | optional | Marks the record a `referenceVia` pointer names as PLATFORM-DELETED on rows where a sibling `select` field holds one declared option — e.g. `record_id` with `{ field: 'cancel_reason', value: 'record_deleted' }`. A renderer then draws a tombstone from that option's own label, reads nothing and links nothing. Valid only beside `referenceVia`; `field` must be a single-value `select` on the same object declaring inline `options`, and `value` one of its option values — any other shape is refused at parse. | | **mailAssociation** | `boolean` | optional | Inbound mail association (`email` fields only): TRUE marks this field as identifying a correspondent. When a mailbox service ingests a message, it associates the message with every record of this object whose value in this field equals one of the message's participant addresses (From, Sender, Reply-To, To and Cc, plus those of a forwarded message it carries; ASCII case-insensitive; the receiving mailbox's own addresses excluded; only within the mailbox's organization). A message no record matches waits in the mailbox's unassociated pool. Refused on any other type. Takes effect only where a mailbox service is composed. | @@ -506,7 +506,7 @@ Add a new field to an existing object | **maxSize** | `integer` | optional | Maximum permitted file size in BYTES for media fields. Enforced on write against the stored file size, not just checked in the browser. | | **options** | `{ label: string; value: string; description?: string; color?: string; … }[]` | optional | Static options for the option types. A `select` / `radio` field needs a non-empty list here or a `picklist` — with neither (or `options: []`) it is refused at parse. | | **picklist** | `string` | optional | Name of a shared `picklist` whose options this field offers — instead of `options`, never with it. Option types only (select, radio, multiselect, checkboxes, tags). A select / radio declares this or a non-empty `options`; with neither it is refused at parse. | -| **reference** | `string` | optional | Target object name (snake_case) for lookup/master_detail fields. Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. On a `tree` field it is optional and, if given, must be the declaring object's own name — the object schema refuses any other target, and writes the own name when it is left out. | +| **reference** | `string` | optional | Target object name (snake_case) for lookup/master_detail fields. Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. Never a stored-metadata table (`sys_metadata`, `sys_metadata_history`): their rows are served only through the metadata API, so the engine never hydrates a reference into one, and `os validate`, `os build` and the object save door refuse it. On a `tree` field it is optional and, if given, must be the declaring object's own name — the object schema refuses any other target, and writes the own name when it is left out. | | **referenceVia** | `string` | optional | Declares this text field as the id half of a polymorphic pointer pair (ADR-0052 §5 ActivityPointer): the value is a record id of the object named by the SIBLING FIELD this key names — e.g. `record_id` with `referenceVia: 'object_name'`. The sibling must be a declared field on the same object holding an object machine name. Text fields only; mutually exclusive with `reference` (a static and a per-record target contradict). Enforced today at seed load: the value resolves as a natural key against the object the sibling column names, and an unresolvable pointer is refused loudly instead of stored verbatim. Adds no referential integrity or $expand behavior. | | **referenceDeletedWhen** | `{ field: string; value: string }` | optional | Marks the record a `referenceVia` pointer names as PLATFORM-DELETED on rows where a sibling `select` field holds one declared option — e.g. `record_id` with `{ field: 'cancel_reason', value: 'record_deleted' }`. A renderer then draws a tombstone from that option's own label, reads nothing and links nothing. Valid only beside `referenceVia`; `field` must be a single-value `select` on the same object declaring inline `options`, and `value` one of its option values — any other shape is refused at parse. | | **mailAssociation** | `boolean` | optional | Inbound mail association (`email` fields only): TRUE marks this field as identifying a correspondent. When a mailbox service ingests a message, it associates the message with every record of this object whose value in this field equals one of the message's participant addresses (From, Sender, Reply-To, To and Cc, plus those of a forwarded message it carries; ASCII case-insensitive; the receiving mailbox's own addresses excluded; only within the mailbox's organization). A message no record matches waits in the mailbox's unassociated pool. Refused on any other type. Takes effect only where a mailbox service is composed. | diff --git a/packages/lint/src/authoring-rules.ts b/packages/lint/src/authoring-rules.ts index e20b3e3df1a..87718b3b9be 100644 --- a/packages/lint/src/authoring-rules.ts +++ b/packages/lint/src/authoring-rules.ts @@ -699,6 +699,13 @@ export const AUTHORING_RULES: readonly AuthoringRule[] = [ // `validate-expressions.ts`), where a declared block resolves and its // leaves are judged: the build and the door accept all 8, and refuse a // misspelt leaf. + // + // [#24677] The object pass also refuses a `lookup` / `master_detail` + // whose `reference` is a stored-metadata-body table — the declaration the + // #24316 hop refusal presupposes, which the engine never hydrates. No + // entry-level change: it rides the object crossing above. MEASURED first: + // 0 in-repo producers reference either family table, against 45 files + // that reference an ordinary platform table. surfaces: CLI_AND_RUNTIME, runtimeTypes: ['flow', 'action', 'hook', 'object'], run: (stack, ctx) => diff --git a/packages/lint/src/validate-expressions.stored-metadata-body-reference.test.ts b/packages/lint/src/validate-expressions.stored-metadata-body-reference.test.ts new file mode 100644 index 00000000000..a149c786eec --- /dev/null +++ b/packages/lint/src/validate-expressions.stored-metadata-body-reference.test.ts @@ -0,0 +1,161 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#24677] The authoring doors refuse a `lookup` or `master_detail` field whose + * `reference` is a stored-metadata-body table. + * + * The engine never hydrates such a field: `expand` keeps its stored id and + * never nests the row, and a master-detail `parent` root over a family header + * stays unbound (`@objectstack/objectql`'s + * `engine-hydration-stored-metadata-body.test.ts`). So as declared it saved, + * built and validated, and then never worked. These pins hold that the author + * learns it where the field is declared, at every authoring door alike: + * + * - `os validate` and `os build` (`runAuthoringRules`, the CLI's own entry); + * - the object save door (`runRuntimeAuthoringRules` on an `object` write), + * whose finding is the build's finding; the protocol-level half, through + * the real `saveMetaItem`, is `@objectstack/metadata-protocol`'s + * `protocol.runtime-authoring-gate.test.ts`; + * + * and that a reference to an ordinary platform table still saves (the + * control), and that the #24316 hop refusal is unchanged beside it. + */ +import { describe, expect, it } from 'vitest'; +import { STORED_METADATA_BODY_OBJECTS, STORED_METADATA_BODY_PRESCRIPTION } from '@objectstack/spec/kernel'; +import { EXPRESSION_INVALID, runAuthoringRules } from './authoring-rules.js'; +import { runRuntimeAuthoringRules } from './runtime-gate.js'; +import { validateStackExpressions } from './validate-expressions.js'; + +const RELATIONSHIP_TYPES = ['lookup', 'master_detail'] as const; + +/** `sharingModel` is authored so `security-owd-unset` stays quiet and the finding is this rule's. */ +const carrier = (type: string, target: string, fields: Record = {}) => ({ + name: 'probe_carrier', + label: 'Probe Carrier', + sharingModel: 'private', + fields: { + name: { type: 'text', label: 'Name' }, + meta: { type, label: 'Meta', reference: target }, + ...fields, + }, +}); + +/** Where the refusal is located: the KEY the author edits. */ +const WHERE = "object 'probe_carrier' · field 'meta' reference"; + +const atReference = (found: readonly T[]): T[] => + found.filter((f) => f.where === WHERE); + +const gateObject = (item: unknown, objects: unknown[] = []) => + runRuntimeAuthoringRules({ type: 'object', item, context: { objects } }); + +const dump = (r: unknown) => JSON.stringify(r, null, 2); + +describe('[#24677] a lookup or master_detail into a stored-metadata-body table is refused at author time', () => { + for (const target of STORED_METADATA_BODY_OBJECTS) { + for (const type of RELATIONSHIP_TYPES) { + it(`a \`${type}\` to '${target}' is an error naming the field, its type and the target, ending on the prescription`, () => { + const found = atReference(validateStackExpressions({ objects: [carrier(type, target)] })); + + expect(found, dump(found)).toHaveLength(1); + const [issue] = found; + expect(issue.severity).toBe('error'); + expect(issue.message).toContain(`\`meta\` is a \`${type}\``); + expect(issue.message).toContain(`names '${target}'`); + expect(issue.message.endsWith(STORED_METADATA_BODY_PRESCRIPTION)).toBe(true); + }); + + it(`\`os validate\` and \`os build\` refuse a \`${type}\` to '${target}' under the expression rule`, () => { + const stack = { objects: [carrier(type, target)] }; + for (const command of ['validate', 'build'] as const) { + const found = atReference(runAuthoringRules(command, { normalized: stack, parsed: stack })); + expect(found, `${command}: ${dump(found)}`).toHaveLength(1); + expect(found[0]).toMatchObject({ rule: EXPRESSION_INVALID, severity: 'error', path: WHERE }); + } + }); + + it(`the object save door refuses a \`${type}\` to '${target}' with the build's own finding`, () => { + const result = gateObject(carrier(type, target)); + + expect(result.rulesRun).toContain('validateStackExpressions'); + const atDoor = atReference(result.errors); + expect(atDoor, dump(result)).toHaveLength(1); + const stack = { objects: [carrier(type, target)] }; + const atBuild = atReference(runAuthoringRules('build', { normalized: stack, parsed: stack })); + for (const key of ['rule', 'where', 'path', 'message', 'hint'] as const) { + expect(atDoor[0][key], `door and build disagree on '${key}'`).toBe(atBuild[0][key]); + } + }); + } + } + + it('a `user` field whose authored `reference` names the family is refused too — the collector\'s type set, not a second one', () => { + // `user`'s target is implied (`sys_user`), but an authored `reference` on it + // is not refused at parse and is the target every reader resolves, so the + // hop refusal's collector admits it, and this refusal judges the same set. + const [issue] = atReference(validateStackExpressions({ objects: [carrier('user', 'sys_metadata')] })); + expect(issue?.message).toContain('`meta` is a `user`'); + expect(atReference(validateStackExpressions({ objects: [carrier('user', 'sys_user')] }))).toEqual([]); + }); + + it('a master_detail refusal says its `parent` root stays unbound; a lookup refusal does not', () => { + const [md] = atReference(validateStackExpressions({ objects: [carrier('master_detail', 'sys_metadata')] })); + const [lk] = atReference(validateStackExpressions({ objects: [carrier('lookup', 'sys_metadata')] })); + expect(md.message).toContain('`parent` root stays unbound'); + expect(lk.message).not.toContain('`parent` root'); + }); + + it('a stored object that already declares one is still refused on its next save', () => { + // The written item's stored self carries the same field: a finding located + // on the written item is judged against the baseline without it, so a + // stored row is not grandfathered past the door it next crosses. + const stored = carrier('lookup', 'sys_metadata'); + const result = gateObject({ ...stored, label: 'Probe Carrier (renamed)' }, [stored]); + + expect(atReference(result.errors), dump(result)).toHaveLength(1); + }); + + it('a stored sibling that declares one is not this write\'s to answer for (the differential)', () => { + const sibling = carrier('lookup', 'sys_metadata'); + const written = { name: 'probe_other', label: 'Other', sharingModel: 'private', fields: { name: { type: 'text', label: 'Name' } } }; + + const result = gateObject(written, [sibling]); + + expect(result.errors.filter((f) => f.where.includes(' reference')), dump(result)).toEqual([]); + }); +}); + +describe('[#24677] the control: a reference to an ordinary table still saves', () => { + for (const target of ['sys_user', 'probe_account']) { + for (const type of RELATIONSHIP_TYPES) { + it(`a \`${type}\` to '${target}' draws no finding at the build or at the object save door`, () => { + const objects = [ + carrier(type, target), + { name: 'probe_account', label: 'Account', sharingModel: 'private', fields: { name: { type: 'text', label: 'Name' } } }, + ]; + expect(atReference(validateStackExpressions({ objects }))).toEqual([]); + const stack = { objects }; + expect(atReference(runAuthoringRules('build', { normalized: stack, parsed: stack }))).toEqual([]); + expect(atReference(gateObject(objects[0], [objects[1]]).errors)).toEqual([]); + }); + } + } +}); + +describe('[#24677] the #24316 hop refusal is unchanged beside it', () => { + it('a formula reading through the family reference draws the hop refusal at its expression, and the reference refusal at the field', () => { + const stack = { + objects: [carrier('lookup', 'sys_metadata', { body_out: { type: 'formula', label: 'Out', expression: 'record.meta.metadata' } })], + }; + const found = validateStackExpressions(stack); + + const hop = found.filter((i) => i.message.includes('a stored-metadata table')); + expect(hop.map((i) => i.where), dump(found)).toEqual(["object 'probe_carrier' · field 'body_out' expression"]); + expect(hop[0].message).toContain('this formula reads null on every row'); + expect(atReference(found), dump(found)).toHaveLength(1); + // Each refusal is selected by its own location alone: the reference + // refusal never reads as a hop, and the hop never as the reference. + expect(atReference(hop)).toEqual([]); + expect(atReference(found)[0].message).not.toContain('a stored-metadata table'); + }); +}); diff --git a/packages/lint/src/validate-expressions.ts b/packages/lint/src/validate-expressions.ts index a8c1b422784..2f0c605228c 100644 --- a/packages/lint/src/validate-expressions.ts +++ b/packages/lint/src/validate-expressions.ts @@ -22,6 +22,12 @@ * (which reads as a guard and is not one) never reaches a production write. * See `validate-null-guards.ts` for the decision procedure and its scope. * + * Since #24677 it also refuses the one declaration the #24316 hop refusal + * presupposes: a `lookup` / `master_detail` field whose `reference` is a + * stored-metadata-body table, located at that field's `reference`. It is not a + * predicate, but it is judged here so that `os validate`, `os build` and the + * object save door, which all run this rule's object passes, refuse it alike. + * * ## Scope — the keys this rule reads, and the ones it deliberately does not * * This rule is registered `input: 'parsed'` (`authoring-rules.ts`), so what it @@ -116,7 +122,7 @@ import type { FlowNodeParsed, FlowEdgeParsed } from '@objectstack/spec/automatio // shared refusal to prevent. import { EvaluatedExpressionInputSchema, EVALUATED_EXPRESSION_SOURCE_REQUIRED } from '@objectstack/spec/shared'; import { referenceCarrierOf, referenceTargetOf, REFERENCE_VALUE_TYPES } from '@objectstack/spec/data'; -import { isStoredMetadataBodyObject } from '@objectstack/spec/kernel'; +import { isStoredMetadataBodyObject, STORED_METADATA_BODY_PRESCRIPTION } from '@objectstack/spec/kernel'; // [#22394] The acting user's declared members — the contract half of the option // verdict's member allowlist for `current_user` and its ADR-0068 aliases. Read // off the schema at first use, never restated here. @@ -1729,9 +1735,12 @@ function singleMasterOf(obj: AnyRec): string | undefined { * serve a hop, so the lint refuses it where it is written. * * The literal `.reference` read stays in the argument for the #5017 receiver - * scan, exactly as in {@link masterDetailCount}. A type whose target is implied - * (`user`, `tree`) never names the family, so the authored carrier is the only - * one read. + * scan, exactly as in {@link masterDetailCount}. Only the authored carrier is + * read: a type whose target is implied never implies the family. A `tree` + * field's authored `reference` cannot name it (the object schema refuses any + * target but the declaring object); a `user` field's authored `reference` is + * not refused at parse, so a `user` naming the family is collected too + * (#24677 measured it). */ function storedMetadataBodyReferencesOf(obj: AnyRec): Map { const out = new Map(); @@ -1766,6 +1775,35 @@ function storedMetadataBodyHopMessage( + 'stored id, if the id is what you need.'; } +/** + * [#24677] The refusal of the declaration every hop above presupposes: a + * `lookup` or `master_detail` field (or any other type the collector admits) + * whose `reference` is a stored-metadata-body table + * ({@link storedMetadataBodyReferencesOf}). The engine never hydrates + * such a field — `expand` keeps its stored id, and a master-detail `parent` + * root over a family header stays unbound — so the field saves and never + * works. It names the field, its type, the target and the rule, and ends on + * the family's one prescription, as the hook-body and flow-write refusals do. + * + * Worded apart from {@link storedMetadataBodyHopMessage} on purpose ("a table + * of stored metadata", never that message's "a stored-metadata table"), so + * the hop refusal's pins, which select its findings by that phrase, read the + * hop refusal alone. + */ +function storedMetadataBodyReferenceMessage(field: string, type: string, target: string): string { + const consequence = type === 'master_detail' + ? 'a formula or a validation rule never reads through the field, and this object\'s `parent` root stays ' + + 'unbound, so every write a `parent` rule judges is refused' + : 'and a formula or a validation rule never reads through the field'; + return `\`${field}\` is a \`${type}\` whose \`reference\` names '${target}', a table of stored metadata. ` + + 'A relationship field may not target a stored-metadata-body table: the engine never ' + + `hydrates one, so \`$expand\` keeps the stored id and never nests the row, ${consequence}. ` + + 'Those rows carry stored metadata bodies, served only ' + + 'through the metadata API (`GET /api/v1/meta/:type/:name`), where the body is projected; to point a ' + + 'record at a metadata item, store its type and name in `text` fields instead. ' + + STORED_METADATA_BODY_PRESCRIPTION; +} + /** * [#24410] The refusal of a formula's one-hop read of a field the reference's * target does not declare. It opens in the shared validator's `unknown field` @@ -1928,9 +1966,10 @@ export function runStackExpressionPasses(stack: AnyRec, options: StackExpression const fieldTypeIndex = buildFieldTypeIndex(objects); const nullableIndex = buildNullableFieldIndex(objects); // [#24316] object name → its references into a stored-metadata-body table, - // read the first time a site of that object serves a hop — never before the - // walk reaches the object, so a carrier the spec cannot read is refused by the - // pass that owned that verdict before this one existed. + // read the first time a site of that object serves a hop, or (#24677) when + // the walk reaches the object's fields, whichever comes first — never before + // the walk reaches the object, so a carrier the spec cannot read is refused + // by the pass that owned that verdict before this one existed. const storedMetadataBodyReferenceIndex = new Map>(); const storedMetadataBodyReferencesFor = (objectName: string): Map => { let familyRefs = storedMetadataBodyReferenceIndex.get(objectName); @@ -2844,6 +2883,34 @@ export function runStackExpressionPasses(stack: AnyRec, options: StackExpression // below. Computed once per object, not per field. const masters = masterDetailCount(obj); + // [#24677] A `lookup` / `master_detail` whose `reference` is a + // stored-metadata-body table is refused where it is declared: the engine + // never hydrates it, so it saves and never works. Read through the hop + // refusal's own collector and index, so the two verdicts judge one set, + // and AFTER `masterDetailCount`, so an unreadable `reference` carrier is + // still refused by the pass that owned that verdict first. NOT fenced on + // an object write: the field is the object's own body, so the object save + // door gives the build's verdict, at the build's position in this walk. + // The type is read from the field-type index, never off the field again: + // the #5017 receiver meta-test pins the keys this walk reads off `f`. The + // collector admits only a field whose `type` is a string, and the index + // records every such field, so each collected field has its type here. + if (objectName) { + const familyRefs = storedMetadataBodyReferencesFor(objectName); + const ownFieldTypes = fieldTypeIndex.get(objectName); + for (const [fname] of fieldList) { + const target = familyRefs.get(fname); + const type = ownFieldTypes?.[fname]; + if (!target || !type) continue; + issues.push({ + where: `object '${objectName}' · field '${fname}' reference`, + message: storedMetadataBodyReferenceMessage(fname, type, target), + source: '', + severity: 'error', + }); + } + } + // [#20078] The roots a field-level predicate binds, each with the record // shape it binds: `record` / `previous` are this object's, `parent` is its // single master's — judged only when that master is in the stack, since diff --git a/packages/metadata-protocol/src/protocol.runtime-authoring-gate.test.ts b/packages/metadata-protocol/src/protocol.runtime-authoring-gate.test.ts index 5b62ceefecd..5f23cfabaa5 100644 --- a/packages/metadata-protocol/src/protocol.runtime-authoring-gate.test.ts +++ b/packages/metadata-protocol/src/protocol.runtime-authoring-gate.test.ts @@ -3069,6 +3069,123 @@ describe('runtime authoring gate on OBJECT writes — the object action predicat }); }); +/** + * [#24677] The object save door refuses a `lookup` or `master_detail` whose + * `reference` is a stored-metadata-body table. + * + * The engine never hydrates such a field (`expand` keeps the stored id; a + * master-detail `parent` root over a family header stays unbound), and this + * door saved it with a 200, as `os validate` and `os build` built it. The + * refusal is in `@objectstack/lint`'s expression rule, beside the refusal of a + * formula reading through such a field; no code here moves. Pinned through + * the REAL `saveMetaItem` / `publishMetaItem`: + * + * (a) each family table, as a `lookup` and as a `master_detail`, is refused + * with the 422 `INVALID_METADATA` envelope carrying the build's located + * finding, on an active save AND on a draft's promotion, and nothing + * lands; + * (b) the control: a reference to an ordinary platform table still saves; + * (d) the door's issue and the build's finding are the same finding. + * + * The rule's own pins are `@objectstack/lint`'s + * `validate-expressions.stored-metadata-body-reference.test.ts`. + * + * ⚠️ As in the #22019 block above: this package reaches `@objectstack/lint` + * through its built `dist/`, so an edit to the rule is invisible here until + * `pnpm --filter @objectstack/lint build` has run. + */ +describe('runtime authoring gate on OBJECT writes — a relationship into a stored-metadata-body table (#24677)', () => { + /** `sharingModel` is authored so `security-owd-unset` stays quiet and the refusal is the reference's. */ + const fxRef = (type: string, target: string) => ({ + name: 'fx_ref', + label: 'Reference Probe', + sharingModel: 'private', + fields: { + name: { type: 'text', label: 'Name' }, + meta: { type, label: 'Meta', reference: target }, + }, + }); + const FAMILY = ['sys_metadata', 'sys_metadata_history'] as const; + const TYPES = ['lookup', 'master_detail'] as const; + /** Where the build locates the refusal — the KEY the author edits. */ + const WHERE = "object 'fx_ref' · field 'meta' reference"; + + const refRows = (rows: Map) => + Array.from(rows.values()).filter((r) => r.type === 'object' && r.name === 'fx_ref'); + const buildFindings = (obj: unknown) => { + const stack = { objects: [obj] }; + return runAuthoringRules('build', { normalized: stack, parsed: stack }) + .filter((f) => f.rule === EXPRESSION_INVALID && f.where === WHERE); + }; + + for (const target of FAMILY) { + for (const type of TYPES) { + it(`(a) REFUSES an active save of a \`${type}\` to '${target}' with a 422 carrying the build's located finding`, async () => { + const { protocol, rows } = makeProtocol(); + + const err = await protocol + .saveMetaItem({ type: 'object', name: 'fx_ref', item: fxRef(type, target) }) + .catch((e: any) => e); + + expect(err, 'the save resolved — the door still accepts the reference').toBeInstanceOf(Error); + expect({ code: err.code, status: err.status }).toEqual({ code: 'INVALID_METADATA', status: 422 }); + expect(err.rulesRun).toContain('validateStackExpressions'); + const issue = err.issues.find((i: any) => i.rule === EXPRESSION_INVALID && i.where === WHERE); + expect(issue, `issues: ${JSON.stringify(err.issues)}`).toBeDefined(); + expect(issue.path).toBe(WHERE); + expect(issue.severity).toBe('error'); + // The named subjects: the field, its type and the target. + expect(issue.message).toContain(`\`meta\` is a \`${type}\` whose \`reference\` names '${target}'`); + // And nothing landed. + expect(refRows(rows)).toEqual([]); + }); + } + } + + it("(a) REFUSES the same body on a draft's PROMOTION — the draft door is not a bypass", async () => { + const { protocol, rows } = makeProtocol(); + await expect( + protocol.saveMetaItem({ type: 'object', name: 'fx_ref', item: fxRef('master_detail', 'sys_metadata'), mode: 'draft' }), + ).resolves.toMatchObject({ success: true }); + + const err = await protocol.publishMetaItem({ type: 'object', name: 'fx_ref' }).catch((e: any) => e); + + expect({ code: err?.code, status: err?.status }).toEqual({ code: 'INVALID_METADATA', status: 422 }); + const issue = err.issues.find((i: any) => i.rule === EXPRESSION_INVALID && i.where === WHERE); + expect(issue, `issues: ${JSON.stringify(err.issues)}`).toBeDefined(); + expect(refRows(rows).map((r) => r.state)).not.toContain('active'); + }); + + for (const type of TYPES) { + it(`(b) the control: a \`${type}\` to an ordinary platform table (\`sys_user\`) still saves, and the row lands`, async () => { + const { protocol, rows } = makeProtocol(); + + const result = await protocol.saveMetaItem({ type: 'object', name: 'fx_ref', item: fxRef(type, 'sys_user') }); + + expect(result.success).toBe(true); + expect(refRows(rows).map((r) => r.state)).toEqual(['active']); + }); + } + + it('(d) the door and `os build` give the SAME finding for the same field', async () => { + const { protocol } = makeProtocol(); + const err = await protocol + .saveMetaItem({ type: 'object', name: 'fx_ref', item: fxRef('lookup', 'sys_metadata') }) + .catch((e: any) => e); + const atDoor = (err.issues ?? []).filter((i: any) => i.rule === EXPRESSION_INVALID && i.where === WHERE); + + const atBuild = buildFindings(fxRef('lookup', 'sys_metadata')); + + expect(atBuild).toHaveLength(1); + expect(atBuild[0]!.severity).toBe('error'); + expect(atDoor).toHaveLength(1); + for (const k of ['rule', 'where', 'path', 'message', 'hint'] as const) { + expect(atDoor[0][k], `door and build disagree on '${k}'`).toBe(atBuild[0]![k]); + } + expect(buildFindings(fxRef('lookup', 'sys_user'))).toEqual([]); + }); +}); + // #23388 — the two page component rules at the runtime save door, ruling of // record 6161173492 (letter B): `validateComponentProps` and // `validateComponentTypes` run on a `page` write (`runtimeTypes: ['page']`), diff --git a/packages/metadata-protocol/src/stored-metadata-body-family.pin.test.ts b/packages/metadata-protocol/src/stored-metadata-body-family.pin.test.ts index 24147e28467..3a7954f8d29 100644 --- a/packages/metadata-protocol/src/stored-metadata-body-family.pin.test.ts +++ b/packages/metadata-protocol/src/stored-metadata-body-family.pin.test.ts @@ -43,7 +43,9 @@ * - [#24358] the engine's two relationship hydrations outside that loader, * `expand` and the master-detail `parent` root → `@objectstack/objectql` * (`engine-hydration-stored-metadata-body.test.ts`: the engine never - * hydrates a reference or a header into the family); + * hydrates a reference or a header into the family), and [#24677] the + * author-time refusal of the reference field itself → `@objectstack/lint` + * (`validate-expressions.stored-metadata-body-reference.test.ts`); * - the generic data door reads / groupBy / filter+sort → this package * (`protocol.data-door-stored-metadata-redaction.test.ts`, and the behaviour * asserted below). @@ -166,6 +168,14 @@ const FAMILY_SURFACES = [ disposition: 'refuses', pin: '@objectstack/objectql (engine-hydration-stored-metadata-body.test.ts)', }, + // [#24677] The declaration both hydrations above presuppose, refused where it + // is written: the engine guards stay for installed artifacts, which run no + // authoring rule. + { + surface: 'author time: a lookup or master_detail field whose reference is in the family (os validate, os build, the object save door)', + disposition: 'refuses', + pin: '@objectstack/lint (validate-expressions.stored-metadata-body-reference.test.ts) + this package (protocol.runtime-authoring-gate.test.ts)', + }, ] as const; /** A stored datasource body as it sits in the `metadata` column: serialized JSON with credential material. */ @@ -238,8 +248,9 @@ describe('[#21120] stored-metadata-body family — the data door exposes only co // + [#21207] the thirteen surfaces of the stored content hash, and the two // of the history change note that can quote one, + [#24316] the three of // the one-hop read through a reference, + [#24358] the two relationship - // hydrations outside it. - expect(FAMILY_SURFACES).toHaveLength(28); + // hydrations outside it, + [#24677] the author-time refusal of the + // reference field they presuppose. + expect(FAMILY_SURFACES).toHaveLength(29); }); }); diff --git a/packages/spec/src/data/field.zod.ts b/packages/spec/src/data/field.zod.ts index b5017655656..c63f57be633 100644 --- a/packages/spec/src/data/field.zod.ts +++ b/packages/spec/src/data/field.zod.ts @@ -1472,10 +1472,23 @@ export const FieldSchema = lazySchema(() => { * engine's object registry for a definition that reaches it unparsed, so every * reader of the target — the delete walk that honours `deleteBehavior`, * `$expand`, the write-time reference check — sees an ordinary self-reference. + * + * [#24677] A `lookup` / `master_detail` never targets a stored-metadata-body + * table (`STORED_METADATA_BODY_OBJECTS`: `sys_metadata`, + * `sys_metadata_history`). Its rows are served only through the metadata + * protocol's own door, so the engine never hydrates a reference into one + * (`$expand` keeps the stored id; a master-detail `parent` root stays + * unbound). The refusal is the authoring rules', beside the refusal of a + * formula or validation rule reading through such a field + * (`@objectstack/lint`'s expression rule), so `os validate`, `os build` and + * the object save door refuse it alike. */ reference: z.string().optional().describe( 'Target object name (snake_case) for lookup/master_detail fields. ' + 'Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. ' + + 'Never a stored-metadata table (`sys_metadata`, `sys_metadata_history`): their rows are served only ' + + 'through the metadata API, so the engine never hydrates a reference into one, and `os validate`, ' + + '`os build` and the object save door refuse it. ' + 'On a `tree` field it is optional and, if given, must be the declaring object\'s own name — ' + 'the object schema refuses any other target, and writes the own name when it is left out.' ),