Merge remote-tracking branch 'origin/main' into claude/issue-20116-stage-2-type-and-widget

Author: claude

.changeset/19886-stored-list-ordering-refused.md

@@ -0,0 +1,40 @@
+---
+"@objectstack/formula": minor
+"@objectstack/plugin-security": minor
+"@objectstack/spec": patch
+---
+
+A row-level write check that orders a field against a bound (`>`, `>=`, `<`, `<=`) is refused with `INVALID_FILTER` / 400 when that field holds a list or an object on the record being written, instead of comparing the list's string form and admitting the write (#19886).
+
+**BREAKING** — an accept-set narrowing, shipped by `@objectstack/formula` and `@objectstack/plugin-security` as `minor` under the repo's launch-window convention for accept-set narrowings. The hand-migration prescription is registered under protocol major 18 as `rls-predicate-stored-list-ordering-refused`.
+
+Clause-②: no (narrowing)
+
+**Security fix for RLS write checks.** Measured through the real plugin-security on driver-sql and driver-memory: `record.tags > 'a'`, with `tags` a `json` field holding `['m']`, compared `'m' > 'a'` and admitted and stored the write. `record.meta < 'a'` with `meta` holding `{ a: 1 }` compared `'[object Object]' < 'a'` and did the same, and so did a `multiple` lookup. `using` stands in as the check for a policy that declares no `check`, so the same predicate in `using` was enforced the same way on writes. No shipped row-level or sharing-rule predicate orders a field at all.
+
+What changes:
+
+- `@objectstack/formula`: `matchesFilterCondition` refuses `$gt` / `$gte` / `$lt` / `$lte` and `$between` on a field whose value on the record is a list or a plain object, whatever the comparand, with the same `INVALID_FILTER` / 400 and the same message as the stage 2d refusals. The refusal is per record: a record whose `json` field holds one scalar is compared as before. `null` and `Date` values are unchanged, and so is every equality against a stored list (`$eq`, `$ne`, implicit equality, `$in`, `$nin`). `$between` is not produced by the CEL lowering, so it reaches this only through a filter passed to `matchesFilterCondition` directly.
+- `@objectstack/plugin-security`: a check insert or by-id update whose post-image holds a list or an object in an ordered field is refused 400 and stores nothing. That includes a by-id update that edits another field of a row whose stored `json` column holds a list, because the post-image merges the stored row.
+- `@objectstack/spec`: the migration registry carries the entry. The stage 2a entry `rls-predicate-array-comparand-refused` now ends "Scalar != and ==, null, Date comparands, and { $field } references between single-valued columns evaluate exactly as before", which is true since stage 2d.
+
+Three moves, named:
+
+1. **The write check now matches driver-sql's read.** driver-sql refuses every ordering comparison, and `$between`, on a column it stores as JSON text, by declared type (400, #7398). The in-process check now refuses the same predicate on the same row (400).
+2. **driver-memory's read parts from the write check.** driver-memory, a test driver, compares a stored list element by element on a read and keeps returning those rows (`record.tags > 'a'` reads a row holding `['m']`), while the check now refuses writing it. This is declared on #15104, as for stage 2d's `{ $field }` half.
+3. **A list written into a scalar field under an ordering check now answers 400.** `status: ['m']` into a `text` field under `record.status > 'a'`, or `amount: [500]` into a `number` field under `record.amount > 10`, was admitted, and driver-sql stored it as the text `'["m"]'` / `'[500]'`. It is now refused before anything is stored.
+
+**The explain answer.** `security/explain` evaluates the business RLS predicate in-process on the fetched record, so it now answers `INVALID_FILTER` / 400 where the record holds a list or an object under an ordering predicate (this stage). It already answered 400 for a `{ $field }` comparison against a list-holding column (stage 2d). For both, per operation:
+
+| explain operation | driver | the enforced operation answers | same as explain's 400? |
+|---|---|---|---|
+| `read` | driver-sql | 400 `INVALID_FILTER` (the driver's refusal) | yes |
+| `update` | driver-memory | 400 `INVALID_FILTER` (the post-image check) | yes |
+| `update` | driver-sql | 403 `PERMISSION_DENIED`: the pre-image gate fails closed on the driver's 400 | no — both deny |
+| `read` | driver-memory | the rows its element-wise read admits | no — the test driver's read |
+
+Explain itself is unchanged.
+
+**What to change.** Order a single-valued column (`record.priority > 2`), or test membership in the list with `in` (`record.status in ['open', 'pending']`). A `json` or `multiple` field has no ordering.
+
+<!-- adr-0087: registered rls-predicate-stored-list-ordering-refused -->

.changeset/19961-decision-branch-expression-absent-refused.md

@@ -0,0 +1,61 @@
+---
+'@objectstack/spec': minor
+'@objectstack/lint': minor
+---
+
+fix(spec)!: a `decision` branch with no `expression` — the key absent, or `null` — is refused at authoring (#19961)
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: registered flow-decision-branch-expression-absent-refused -->
+
+**BREAKING** — an accept-set narrowing on one authored flow-node slot, shipped as
+`minor` under the launch-window convention (`check-changeset-no-major` refuses
+`major` until GA; breaking-ness is carried by this banner and the ADR-0087
+disposition above, not by the level).
+
+**What changed.** `DecisionConditionSchema` declares a branch `{ label, expression }`
+with `expression` a required `z.string()`. Nothing enforced that: a decision node's
+`config` is an open record no schema is parsed against, and the expression ledger's
+resolver skipped an absent value as "not authored". So `conditions: [{ label: 'y' }]`
+passed `FlowSchema.parse`, `AutomationEngine.registerFlow` and `objectstack validate`,
+and the run then failed at that branch — the executor evaluates every branch it
+reaches, and a branch with no `expression` is a condition with no `source`, which
+`evaluateCondition` refuses. The ledger now marks the slot `required` (reconciled
+against the schema's own `required` list), and the branch is refused at all three
+doors through the walk and the function that already refuse a blank one — by
+`FlowSchema.parse` with a `custom` issue anchored at the slot (for example
+`nodes.1.config.conditions.0.expression`), by `registerFlow` and `objectstack validate`
+through that same parse, and by `validateStackExpressions` for a stack handed to it
+directly — with one message, led by the published `PREDICATE_SLOT_STRING_REFUSAL`
+sentence. `expression: null` is refused the same way, and so is a branch that wrote
+its predicate under `condition` (the edge's spelling), which has no `expression`
+either. The Studio flow designer writes the refused shape when a branch row's
+expression cell is left empty. Where such a branch already sits, the whole flow is
+refused: registered from the metadata
+registry or `sys_metadata` at boot, it is skipped with a `failed to register flow`
+warn naming it while the flows beside it register; a `defineStack({ flows })` source
+throws `StackSchemaInvalidError` for the whole stack; an artifact file is refused
+whole at load.
+
+## FROM → TO
+
+| you wrote | write instead |
+|:--|:--|
+| `conditions: [{ label: 'high' }]` on a `decision` node | the predicate you meant — `{ label: 'high', expression: 'record.amount > 10000' }` |
+| `conditions: [{ label: 'high', condition: 'record.amount > 10000' }]` | the same predicate under `expression` |
+| `conditions: [{ label: 'high', expression: null }]` | the predicate you meant, or `expression: 'false'` to keep the branch and never take it |
+
+**One-line fix:** write the predicate under `expression`. `expression: 'false'` keeps
+the branch and its label and never takes it — a change of behaviour, not a preserved
+one: a run that reached the branch used to FAIL there, and now routes on to the next
+branch or the declared fallback. ⚠️ Do not drop a decision's only branch: the node
+then routes by its out-edges alone, and the out-edge that branch labelled is no
+longer held back.
+
+**Unchanged.** A branch carrying a non-blank predicate parses, registers and
+validates as before; a blank one keeps its refusal and its own prescription
+(`flow-predicate-slot-blank-string-refused`); a `decision` with no `conditions`, or
+an empty list, still routes by its out-edges; an absent screen field `visibleWhen`
+is still legal (that slot is not required); and `PREDICATE_SLOT_STRING_REFUSAL`
+keeps its name and its text.

.changeset/20233-ui-plugin-migration-guidance-tracker-free.md

@@ -0,0 +1,21 @@
+---
+'@objectstack/spec': patch
+---
+
+fix(spec): `os migrate meta` guidance for the `ui-*` and `plugin-*` migration entries states each lesson in words instead of citing tracker numbers
+
+Clause-②: no
+
+The ADR-0087 semantic entries of the `ui-*` family (component props rows, form-field
+and list-view refusals, the react-tier `ListView` aliases, and the retired
+interaction, notification, embed, widget and i18n vocabularies) and of the `plugin-*`
+family (the plugin manifest, runtime, health-monitor and security-scanner retirements)
+are printed by `os migrate meta` as the header, `why:` and `verify:` lines of a manual
+change. Their text sent the reader to issue-tracker numbers — some of which no longer
+resolve — for what a ruling, measurement or fix had decided; it now says what was
+decided, in the sentence being read. The same holds for the two `surface` headers that
+carried a number. ADR ids are kept.
+
+Text only: no entry id, `from` / `to`, conversion or matching logic changes, and the
+chain rewrites exactly what it rewrote before. The generated migration registry,
+`spec-changes.json` and the protocol upgrade guide carry the same text.

.changeset/20237-dispatcher-meta-list-gate.md

@@ -0,0 +1,67 @@
+---
+'@objectstack/runtime': patch
+'@objectstack/rest': minor
+---
+
+fix(runtime): the dispatcher's `/meta/:type` list prunes what `RestServer`'s list prunes, through one list gate that `@objectstack/rest` publishes (#20237)
+
+Clause-②: yes
+
+`GET /meta/:type` has two implementations: `RestServer`, and the runtime
+dispatcher's `/meta` domain. On a host that mounts only the `${prefix}/*`
+catch-all (`@objectstack/hono`'s `createHonoApp`, the documented embed shape,
+and any adapter built on the public `HttpDispatcher` API), the dispatcher is the
+only one that answers. Its list branch applied **no** per-caller filter. An
+authenticated member who does not hold `crm_admin` got these answers through
+`dispatch()`:
+
+```
+request                         before                                     after (= RestServer)
+GET /meta/doc?include=content   the set-gated doc listed WITH its body     the doc left out
+GET /meta/book                  the set-gated book listed                  the book left out
+GET /meta/app                   an app whose requiredPermissions the       that app left out; the other app
+                                member lacks, and an ungated app with      pruned of its gated entry
+                                its requiredPermissions-gated entry
+GET /meta/dashboard (anyone)    every widget                               minus the widget whose service is off
+```
+
+The plural spellings (`/meta/docs`, `/meta/books`, `/meta/apps`) answer the
+same. Holders are still listed everything in full. Object lists are masked as
+before. An anonymous caller still gets `401 UNAUTHENTICATED`.
+
+**One gate, not two.** `RestServer`'s list filters moved unchanged into
+`createMetaListReadGate`, beside the item gate in
+`packages/rest/src/meta-item-read-gate.ts`. It covers the ADR-0046 §6.7 doc
+and book audience prunes, the app nav filter (`requiredPermissions`, the
+ADR-0045 §3 publish gate and the docs-audience entry arm) and the ADR-0057
+D10 dashboard widget gate. `RestServer`'s list route and the dispatcher's list
+branch both call it, over the same ports the item gate takes, and each rewraps
+the pruned items in its own list envelope. There is no second audience
+resolver in `packages/runtime`. `RestServer`'s list answers are unchanged.
+
+Every exit of the dispatcher's list branch now runs the gate and the ADR-0106
+object mask: the protocol list and the two fallbacks, the runtime metadata
+service's list and the ObjectQL registry. The fallbacks used to serve
+unmasked object schemas as well as ungated docs, books and apps. A gate input
+that cannot be read (a doc list's books read throws) is answered as that fault,
+never as the unfiltered list.
+
+**`@objectstack/rest`'s published export surface widens**, and that is why this
+changeset declares `Clause-②: yes`. Its only export subpath (`.`) gains one
+value:
+
+- `createMetaListReadGate(sources, metaType)`: the list gate. It takes the
+  same `MetaItemReadGateSources` the item gate takes, and it answers a judge
+  from a list's items to the items this caller may be served.
+
+It is public because the runtime dispatcher's `/meta` domain in
+`@objectstack/runtime` consumes this one gate. `@objectstack/rest` cannot import
+the runtime, so the shared decision has to live here and travel as an export,
+the way `createMetaItemReadGate` does. A caller outside the platform does not
+need it.
+
+Nothing is removed or renamed from the package's exports, and no authorable key
+moves. The dispatcher's prunes are not the widening: they pull a second
+transport back to the rules the contract already declares (ADR-0046 §6.7,
+ADR-0045 §3, `apps.mdx`'s `requiredPermissions` row), which is why
+`@objectstack/runtime` stays a `patch`.

.changeset/view-overlay-owner-hidden-retired.md

@@ -0,0 +1,101 @@
+---
+'@objectstack/spec': minor
+---
+
+feat(spec)!: retire the flattened view overlay's `owner` and `hidden` keys — accepted at the save door, stored, and read by nothing (#20230)
+
+**BREAKING** — `owner` and `hidden` are removed from the flattened view overlay:
+the lean `view` body with no `config` that `PUT /api/v1/meta/view/:name` (the
+Studio and MCP save) accepts, members 3 and 4 of the `view` metadata door, and
+the same members in the assembled-manifest `viewItems:` channel. ADR-0049
+enforce-or-remove; triage direction, verbatim: 「follow #20085's disposition for
+the same key pair」. This completes the family: the view item record's `owner` /
+`hidden` are retired in this same release by its own entry, with the same texts.
+
+⚠️ **This supersedes one sentence of the view item retirement's note in this same
+release.** That note says the flattened overlay's own `owner` / `hidden` are
+untouched and that a `{ object, viewKind, hidden: true }` overlay still parses.
+True of that change alone; after this one, such an overlay is refused too. Read the
+two notes together: after this release, neither door accepts either key.
+
+Clause-②: no (narrowing)
+
+The overlay door declared both keys separately from the view item's pair. A bound
+overlay such as `{ object, viewKind, hidden: true }` saved clean and one row was
+stored with the key, and nothing ever read it. Both view-switcher read paths
+(`GET /meta/view?object=` and `getViewsByObject`) filter on `viewKind` + `object`
+and sort on `order`, so `hidden: true` hid nothing, and a view with `owner` set
+was listed for every user who can read the object.
+
+Writer census, taken before removal: no writer of either overlay key in this
+framework or its examples, in objectui at its pinned commit and at `main` (the
+toolbar writes only `rowHeight`, `sort`, `hiddenFields`, `columnState` and
+`inlineEdit`; the switcher only `label`, `isPinned`, `isDefault` and `sortOrder`),
+or in the HotCRM app. The cloud repository was not reachable from the census.
+
+### FROM → TO
+
+| removed | what to write instead |
+| --- | --- |
+| flattened overlay `owner` | delete the key. Nothing restricts a view to one user today; a view is visible to everyone who can read its object. |
+| flattened overlay `hidden` | delete the key. To take a view out of the switcher, delete the view item (or stop shipping it from source). |
+
+**The one-line fix: delete `owner:` and `hidden:` from every view body you save.**
+`os migrate meta --from 17` lists the mechanical edits for existing sources.
+
+⚠️ Runtime behaviour is deliberately **unchanged**. Neither key ever changed what
+a view showed or to whom. What changes is the answer an author gets: a save that
+carries either key is refused `422 INVALID_METADATA`, with the prescription
+located at the key, instead of being stored with no effect. The prescriptions are
+the view item's own texts, so the family answers with one voice on both doors.
+
+### Stored rows
+
+Every read of a stored `view` row replays the conversion chain before the row is
+served or badged, and the D2 conversion strips both keys there. What that leaves
+depends on what else the row holds:
+
+- **A row with any other view key** (a column state, a sort, a default flag, an
+  order): served and badged valid without the keys. A GET then a PUT of the whole
+  row saves (if it was otherwise valid), so the console's next read-merge-write of
+  it saves, and `os migrate meta --stored --apply` rewrites it.
+- **A hide-only row**, holding nothing but its identity (`name`, `object`,
+  `viewKind`, `label`) and `owner` / `hidden`, such as
+  `{ object, viewKind, hidden: true }`: the strip leaves identity only, which the
+  `view` door refuses ("only identity fields"). The row is served badged invalid
+  (it was badged valid before this release). A whole-row re-save, or one that adds
+  only identity (a rename sets `label`), answers `422 INVALID_METADATA`.
+  `--apply` reports it `failed` and leaves it as stored; every read strips it
+  again. A write that adds a real view key, such as a toolbar toggle, saves.
+  **Fix: delete the row** (it never changed what anyone saw), or add the
+  personalization setting its author meant and save that.
+
+### The retirement kit
+
+- **Tombstones on both overlay members.** `retiredKey()` in
+  `flattenedViewOverlayFields()`, with the view item's prescription texts. Both
+  members `.strip()`, so a bare deletion would have dropped the key in silence
+  (ADR-0104).
+- **D2 conversion `view-overlay-owner-hidden-removed`** (step 18, retired from the
+  load path). A lossless delete from the flattened spelling (no `config`, no
+  container slot) in `views` (stack sources and stored rows) and `viewItems`
+  (assembled artifacts). It is disjoint from `view-item-owner-hidden-removed` by
+  `config`, so no row is judged by both.
+- **D3 semantic entry `view-overlay-owner-hidden-retired`**: the family's one D3
+  record, naming its D2 conversion. The view item record's pair is a separate
+  family with its own conversion and its own D3 entry; the two share the
+  prescription texts.
+- **`RETIRED_KEYS_BY_MAJOR[18]`**: `ui/ViewMetadata:owner`, `ui/ViewMetadata:hidden`.
+  `ui/ViewMetadata` is unemitted (its `z.undefined()` guards have no JSON Schema
+  form), so no build gate judges these rows and the four surface ratchets are
+  byte-identical on this retirement. The rows are pinned by the retirement test.
+- **No liveness row**: the `view` ledger walks the container keys only.
+- **No deprecation window**, per the project's startup-stage posture.
+
+⚠️ **The out-of-repo population is NOT MEASURED.** `@objectstack/spec` is published,
+and production `sys_metadata` rows are not reachable from the repository. Stored
+rows are stripped on read by the conversion above, and a hide-only row among them
+needs the fix above. A client that still sends either key is refused at its next
+save.
+
+<!-- adr-0087: registered view-overlay-owner-hidden-removed, view-overlay-owner-hidden-retired -->