feat(spec,objectql): report a caller value a stored expression replaced, as DroppedFieldsEvent reason stored_expression (#24778)

Closes #24690
Clause-②: yes (widening)

## What

PR #24673 (`c0803b04d2`) made the engine compute a stored
(non-`formula`) field's declared `expression` on every write, so a value
the caller sends for such a field never stands. Before this PR nothing
said so. `onFieldsDropped` stayed silent, so the PATCH and `/batch`
doors answered with no `droppedFields`, and a flow's `update_record`
reported success over a row that holds the computed value.

Scope follows the maintainer's ruling 「甲」 (#6015 comment 6292005854,
applied on decision card #24854): in 18.x only the loud half ships. That
half is the new `reason` value, its emit, and its report. The
`strictReadonlyWrites` refusal is runtime behaviour that no conversion
can carry, so it waits for 19.0.

- **spec** (`packages/spec/src/data/data-engine.zod.ts`):
`DroppedFieldsEventSchema.reason` gains `stored_expression`, with its
describe. The field IS written, with the computed value in place of the
caller's, so `computed` (a `formula` with no column, whose key is
dropped) would be untrue of it. The name follows the existing arms,
which name the field's role (`primary_key`, `computed`). The `fields`
describe and the schema describe now also cover a field written with the
computed value. In `contracts/data-engine.ts`, the
`strictReadonlyWrites` TSDoc names `stored_expression` as the one arm
outside its coverage in 18.x.
- **objectql** (`engine.ts`): `replacedStoredExpressionFields` reports a
field when three things hold: the caller sent it, no other strip took
it, and the value stored differs from the caller's. It runs on insert
(single, batch, and `insertMany`'s per-row `droppedFields`), on update
by id and on predicate update, in every context, `isSystem` included. A
client that sends the computed value (the console grid's consistent
save) or no value is told nothing. A value a static `readonly` strip
already took is reported once, as `readonly`.
- **strictReadonlyWrites is unchanged.** The update path sends the
report through a listener-only channel (`notifyDrop`). On insert, the
strict check judges only the strips' own drops. Both strict checks sit
exactly where they sit on `main`, before the computation. So a strict
write whose only drop is a `stored_expression` replacement completes
with the computed value and is reported to `onFieldsDropped`. A strict
write that also carries a refusable drop is refused for that drop alone,
writes nothing, and fires no listener. `readonly-strict-errors.ts` gains
only the `REASON_PHRASE` row its exhaustive key type requires. No
refusal carries that row in 18.x.
- **service-automation** (`crud-nodes.ts`): `DROPPED_REASON_LABEL` is an
exhaustive `Record` over the reason union, so it needs the new row. For
this reason the step warning says the requested value(s) were NOT
stored, and that the write succeeded with the computed value(s). The
other reasons keep their exact text.
- **Generated**: `content/docs/references/**` regenerated with
`check:generated --fix`.
- **Changeset**: `Clause-②: yes (widening)`. `@objectstack/spec` and
`@objectstack/objectql` are `minor`, `@objectstack/service-automation`
is `patch`. Not breaking. It records that the strict refusal is deferred
to 19.0 per 「甲」.

## Landing order: the objectui pin must move first

objectui's `STRIPPED_LINE`
(`packages/app-shell/src/providers/writeWarningToast.ts`) is a `Record`
keyed by `DroppedFieldsEvent['reason']`. `app-shell` builds with `tsc`,
and `scripts/build-console.sh` installs this tree's `@objectstack/*`
into the pinned objectui. The pin on `main` today is `616df6598b`. It
does not contain objectui PR 13233 (objectstack-ai/objectui#13178),
which adds the `stored_expression` row ahead of the spec pin: merge
commit `8803ca54ac269643101f98dd0cc24ab25feb55a0`, which is 36 commits
ahead of `616df6598b` and is on objectui `main`. Landing this PR at
today's pin would turn the full-battery `Console Pin Gate` red.

The pin is not bumped here. `docs/releases-maintenance.md` says the pin
moves by hand, in its own PR, with `scripts/bump-objectui.sh`, to a
revision chosen in an objectstack issue. **This PR must not land before
that pin PR does.** The pin PR must:

1. Run `scripts/bump-objectui.sh
8803ca54ac269643101f98dd0cc24ab25feb55a0` (or a later objectui `main`
commit). It writes `.objectui-sha` and the `@objectstack/console`
changeset for `616df6598b..NEW`.
2. Regenerate the committed manifest in the same PR: `pnpm
objectui:build`, then `node scripts/gen-sdui-manifest-node.mjs`. That
rewrites `sdui.manifest.json` and `scripts/sdui-manifest.record.json`,
and `scripts/check-sdui-manifest.mjs` must be green.
3. Land before this PR. The objectui row is keyed by the spec union plus
`stored_expression`, so the pinned objectui compiles against `main`'s
spec both before and after this PR.

## Tests (on the head named in the latest report)

-
`packages/objectql/src/engine-stored-expression-dropped-fields.test.ts`
covers by-id update (the PATCH door and each by-id `/batch` op), insert
single, batch and `insertMany`, and predicate update. It has controls
for an equal value and an unsent value, and checks the single report
when the `readonly` strip took the value. Under `strictReadonlyWrites`
it checks three things. By id, on insert and on a predicate update, the
write completes and is reported. A write that also has a `readonly_when`
drop is refused for that drop alone, writes nothing and fires no
listener. An insert with a refusable drop is refused before the
computation, carrying no `stored_expression` drop.
- `packages/rest/src/rest-stored-expression-dropped-fields.test.ts` runs
through the composed stack (`RestServer`, then the protocol, then
`ObjectQL`, then a real `SqlDriver` on better-sqlite3). PATCH answers
`droppedFields` and the `X-ObjectStack-Dropped-Fields` header. `/batch`
tags the op index. Each has a consistent-value control.
- `crud-dropped-fields.test.ts` (service-automation) pins the new
warning. `data-engine.test.ts` (spec) pins the five-value enum.
- Two ablations, one-time and with no test file left behind, via
`scripts/ablation-replace.mjs`. The tool restored and verified the file
each time (blob equal to HEAD, `git diff HEAD` empty).
- The reporter's guard was forced to `return []`, and 12 of 30 objectql
tests failed (first round).
- The by-id `notifyDrop` was routed back through the strict-or-listener
`emitDrop`, and the strict by-id report test failed (1 of 17).
- Reverse type check (first round): with the `stored_expression` row
commented out of `DROPPED_REASON_LABEL`, service-automation `tsc` failed
with TS2741.

## Acceptance notes

- `Object.is` is the comparison, as in the compute passes, over caller
values normalized before the snapshot. Take a stored expression whose
computed value has a different JS type from an equal value the caller
sends, such as a date expression computing a `Date` against an ISO
string: it would be reported. This was not measured.
- The `ImportRowResult.droppedFields` (`api/export`), `/batch` and
protocol `droppedFields` describes still say "caller-supplied fields the
engine LEGALLY strips". They carry the fifth reason through the shared
enum, and their prose was left alone because it is outside this card's
file surface.
- The 19.0 half, refusing a `stored_expression` replacement under
`strictReadonlyWrites`, needs its own card, filed by the seat.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01D6F9axfoPzR7t8G1J7Ytzz)_

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: objectstack-fleet[bot]

.changeset/24690-stored-expression-dropped-fields.md

@@ -0,0 +1,18 @@
+---
+'@objectstack/spec': minor
+'@objectstack/objectql': minor
+'@objectstack/service-automation': patch
+---
+
+feat(objectql): a caller value that a stored field's `expression` replaced is reported through `onFieldsDropped` / `droppedFields` under the new `DroppedFieldsEvent.reason` value `stored_expression`
+
+Clause-②: yes (widening)
+
+The engine computes a stored (non-`formula`) field's declared `expression` on every write, so a value the caller sends for such a field never stands. Until now nothing said so. `onFieldsDropped` stayed silent, so the PATCH and `/batch` doors answered with no `droppedFields`, and a flow's `update_record` reported success over a row that holds the computed value instead of the one sent.
+
+- **New `reason` value.** `DroppedFieldsEventSchema.reason` (`@objectstack/spec/data`) gains `stored_expression`. It means the field is a stored column the engine computes from its `expression`, and the caller's value was replaced by the computed one. The field IS written, so `computed` (a `formula` with no column, whose key is dropped) would be wrong for it. The four existing values are unchanged.
+- **When it is reported.** On insert (single, batch, and `insertMany`'s per-row `droppedFields`), on update by id, and on a predicate update, in every context, `isSystem` included. It is reported only when the caller sent the field and the value stored differs from the caller's value. A client that sends the computed value (the console's line-item grid) or no value is told nothing. A value another strip already took, such as a stored expression field that is also `readonly`, is reported once, under that strip's reason.
+- **`strictReadonlyWrites` is unchanged.** The maintainer's ruling 「甲」 says only the loud half ships in 18.x. A strict write whose value for such a field differs from the computed one is NOT refused: it completes with the computed value, and the replacement is reported through `onFieldsDropped` under strict as well. A refusal is runtime behaviour that no conversion can carry, so the strict refusal of a `stored_expression` replacement is deferred to 19.0. A strict write that also carries a refusable drop (`readonly`, `readonly_when`, `primary_key`, `computed`) is refused as before, for those drops only, and fires no listener.
+- **Flow step warning.** `@objectstack/service-automation` words the new reason. Its warning says the requested value(s) were not stored, rather than that the field was not written, and its closing sentence says the write succeeded with the computed value(s).
+
+**For consumers that branch on `reason`.** A `Record<DroppedFieldsEvent['reason'], …>` table fails its typecheck until it has a `stored_expression` row. That is the intended signal: add the row. A client running behind the server should treat a reason it does not recognise as "not applied", never as one of the known reasons.

content/docs/references/api/batch.mdx

@@ -68,7 +68,7 @@ const result = BatchConfigSchema.parse(data);
 | **errors** | `{ code: Enum<'VALIDATION_ERROR' \| 'INVALID_FIELD' \| 'MISSING_REQUIRED_FIELD' \| …>; declaredCode?: string; message: string; userMessage?: string; … }[]` | optional | Array of errors if operation failed. Branch on `errors[0].code` — an atomic batch that rolled back marks rows that were written then undone with code ROLLED_BACK and rows never reached with NOT_ATTEMPTED, while the causal row keeps its own error. A NON-atomic batch that stopped (the `continueOnError: false` default) marks its un-attempted tail with the same NOT_ATTEMPTED code — rows before the failure stay written and keep reporting success, since nothing was rolled back. |
 | **data** | `Record<string, any>` | optional | Full record data (if returnRecords=true) |
 | **index** | `number` | optional | Index of the record in the request array |
-| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed'> }[]` | optional | Write-observability: caller-supplied fields LEGALLY stripped from THIS row before it was written — static `readonly` / TRUE `readonlyWhen` on update, or the in-engine static `readonly` strip on create. Per-row because a batch can drop different fields on different rows (`readonlyWhen` is record-state-dependent). Present ONLY when ≥1 field was dropped for this row; the row still succeeded (success unchanged). A single response header cannot express per-row drops, so this body field is the canonical bulk channel — REST does not emit `X-ObjectStack-Dropped-Fields` for batches. Optional — omit-when-empty keeps the shape backward-compatible. |
+| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed' \| 'stored_expression'> }[]` | optional | Write-observability: caller-supplied fields LEGALLY stripped from THIS row before it was written — static `readonly` / TRUE `readonlyWhen` on update, or the in-engine static `readonly` strip on create. Per-row because a batch can drop different fields on different rows (`readonlyWhen` is record-state-dependent). Present ONLY when ≥1 field was dropped for this row; the row still succeeded (success unchanged). A single response header cannot express per-row drops, so this body field is the canonical bulk channel — REST does not emit `X-ObjectStack-Dropped-Fields` for batches. Optional — omit-when-empty keeps the shape backward-compatible. |
 
 ### Nested Shape: `BatchOperationResult.errors[number]`
 
@@ -86,13 +86,13 @@ const result = BatchConfigSchema.parse(data);
 
 ### Nested Shape: `BatchOperationResult.droppedFields[number]`
 
-A write-path strip event: caller-supplied fields legally dropped from the payload
+A write-path strip event: caller-supplied fields legally dropped from the payload, or written with the engine's computed value in place of the caller's
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
 | **object** | `string` | ✅ | Object the write targeted (resolved object name) |
-| **fields** | `string[]` | ✅ | Caller-supplied field names the engine removed from the write payload |
-| **reason** | `Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed'>` | ✅ | Why the fields were dropped: static readonly, a TRUE readonlyWhen predicate, the primary-key strip of a payload id the engine ruled is not an identifier, or a computed (formula) field no driver has a column for |
+| **fields** | `string[]` | ✅ | Caller-supplied field names whose value the engine did not store: removed from the write payload, or (`stored_expression`) written with the computed value in place of the caller's |
+| **reason** | `Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed' \| 'stored_expression'>` | ✅ | Why the fields were dropped: static readonly, a TRUE readonlyWhen predicate, the primary-key strip of a payload id the engine ruled is not an identifier, a computed (formula) field no driver has a column for, or a stored field whose declared expression the engine computes on every write, so the caller's different value was replaced by the computed one |
 
 
 ---
@@ -213,7 +213,7 @@ A write-path strip event: caller-supplied fields legally dropped from the payloa
 | **errors** | `{ code: Enum<'VALIDATION_ERROR' \| 'INVALID_FIELD' \| 'MISSING_REQUIRED_FIELD' \| …>; declaredCode?: string; message: string; userMessage?: string; … }[]` | optional | Array of errors if operation failed. Branch on `errors[0].code` — an atomic batch that rolled back marks rows that were written then undone with code ROLLED_BACK and rows never reached with NOT_ATTEMPTED, while the causal row keeps its own error. A NON-atomic batch that stopped (the `continueOnError: false` default) marks its un-attempted tail with the same NOT_ATTEMPTED code — rows before the failure stay written and keep reporting success, since nothing was rolled back. |
 | **data** | `Record<string, any>` | optional | Full record data (if returnRecords=true) |
 | **index** | `number` | optional | Index of the record in the request array |
-| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed'> }[]` | optional | Write-observability: caller-supplied fields LEGALLY stripped from THIS row before it was written — static `readonly` / TRUE `readonlyWhen` on update, or the in-engine static `readonly` strip on create. Per-row because a batch can drop different fields on different rows (`readonlyWhen` is record-state-dependent). Present ONLY when ≥1 field was dropped for this row; the row still succeeded (success unchanged). A single response header cannot express per-row drops, so this body field is the canonical bulk channel — REST does not emit `X-ObjectStack-Dropped-Fields` for batches. Optional — omit-when-empty keeps the shape backward-compatible. |
+| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed' \| 'stored_expression'> }[]` | optional | Write-observability: caller-supplied fields LEGALLY stripped from THIS row before it was written — static `readonly` / TRUE `readonlyWhen` on update, or the in-engine static `readonly` strip on create. Per-row because a batch can drop different fields on different rows (`readonlyWhen` is record-state-dependent). Present ONLY when ≥1 field was dropped for this row; the row still succeeded (success unchanged). A single response header cannot express per-row drops, so this body field is the canonical bulk channel — REST does not emit `X-ObjectStack-Dropped-Fields` for batches. Optional — omit-when-empty keeps the shape backward-compatible. |
 
 
 ---
@@ -227,8 +227,8 @@ A cross-object batch strip event: dropped fields plus the operation index
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
 | **object** | `string` | ✅ | Object the write targeted (resolved object name) |
-| **fields** | `string[]` | ✅ | Caller-supplied field names the engine removed from the write payload |
-| **reason** | `Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed'>` | ✅ | Why the fields were dropped: static readonly, a TRUE readonlyWhen predicate, the primary-key strip of a payload id the engine ruled is not an identifier, or a computed (formula) field no driver has a column for |
+| **fields** | `string[]` | ✅ | Caller-supplied field names whose value the engine did not store: removed from the write payload, or (`stored_expression`) written with the computed value in place of the caller's |
+| **reason** | `Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed' \| 'stored_expression'>` | ✅ | Why the fields were dropped: static readonly, a TRUE readonlyWhen predicate, the primary-key strip of a payload id the engine ruled is not an identifier, a computed (formula) field no driver has a column for, or a stored field whose declared expression the engine computes on every write, so the caller's different value was replaced by the computed one |
 | **index** | `integer` | ✅ | Index of the operation in the request `operations` array |
 
 
@@ -276,7 +276,7 @@ A cross-object batch strip event: dropped fields plus the operation index
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
 | **results** | `any[]` | ✅ | Per-operation result, index-aligned with the request operations |
-| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed'>; index: integer }[]` | optional | Write-observability: caller-supplied fields the engine LEGALLY stripped from an operation before it was written — static `readonly` or a TRUE `readonlyWhen` predicate. This endpoint is the console record form's save path (master-detail writes parent + children in one transaction), so without it the ONE surface where a user edits a `readonlyWhen` field reported plain success while the value never landed. Each event carries the `index` of its operation. Present ONLY when ≥1 field was dropped; the batch still committed without them (results/success semantics unchanged). Optional — omit-when-empty keeps the shape backward-compatible. |
+| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed' \| 'stored_expression'>; index: integer }[]` | optional | Write-observability: caller-supplied fields the engine LEGALLY stripped from an operation before it was written — static `readonly` or a TRUE `readonlyWhen` predicate. This endpoint is the console record form's save path (master-detail writes parent + children in one transaction), so without it the ONE surface where a user edits a `readonlyWhen` field reported plain success while the value never landed. Each event carries the `index` of its operation. Present ONLY when ≥1 field was dropped; the batch still committed without them (results/success semantics unchanged). Optional — omit-when-empty keeps the shape backward-compatible. |
 
 ### Nested Shape: `CrossObjectBatchResponse.droppedFields[number]`
 
@@ -285,8 +285,8 @@ A cross-object batch strip event: dropped fields plus the operation index
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
 | **object** | `string` | ✅ | Object the write targeted (resolved object name) |
-| **fields** | `string[]` | ✅ | Caller-supplied field names the engine removed from the write payload |
-| **reason** | `Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed'>` | ✅ | Why the fields were dropped: static readonly, a TRUE readonlyWhen predicate, the primary-key strip of a payload id the engine ruled is not an identifier, or a computed (formula) field no driver has a column for |
+| **fields** | `string[]` | ✅ | Caller-supplied field names whose value the engine did not store: removed from the write payload, or (`stored_expression`) written with the computed value in place of the caller's |
+| **reason** | `Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed' \| 'stored_expression'>` | ✅ | Why the fields were dropped: static readonly, a TRUE readonlyWhen predicate, the primary-key strip of a payload id the engine ruled is not an identifier, a computed (formula) field no driver has a column for, or a stored field whose declared expression the engine computes on every write, so the caller's different value was replaced by the computed one |
 | **index** | `integer` | ✅ | Index of the operation in the request `operations` array |