Merge origin/main into claude/issue-21163-autonumber-like-escape

Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp
Co-authored-by: Claude <noreply@anthropic.com>

Author: claude

.changeset/20752-rest-strings-state-the-decision.md

@@ -0,0 +1,16 @@
+---
+'@objectstack/rest': patch
+---
+
+REST refusals, warnings and the served OpenAPI text no longer cite tracker numbers; each one states the decision behind it in words
+
+Clause-②: no
+
+A few strings `@objectstack/rest` sends to callers and operators pointed at an issue-tracker number for the reason behind them. The number goes; where the sentence did not already say what was decided, it now does.
+
+- `POST /data/:object/import` with a named mapping that declares a `javascript` transform: the `UNSUPPORTED_TRANSFORM` message now says the import path does not execute it (there is no server-side sandbox), so the import is refused rather than run with that transform skipped.
+- `GET /openapi.json`: the built-in section's response description says the section is built from the routes this server actually mounts and that payload schemas are deliberately not invented; the request-body description says the document leaves the route-specific shape undescribed rather than invent one.
+- The boot warning for a config that still sets the retired `api.requireAuth` drops its citation; it already says the key was removed and anonymous access to object data is always denied.
+- `/discovery`'s `capabilities.transactionalBatch.description` cites ADR-0034 alone.
+
+Text only: no status, error code, field, route or control flow moves. A client that matches the old message text (for example the tracker-number suffix the `UNSUPPORTED_TRANSFORM` message used to end with) needs the new spelling.

.changeset/20822-having-contains-membership.md

@@ -0,0 +1,18 @@
+---
+'@objectstack/objectql': patch
+'@objectstack/driver-mongodb': patch
+'@objectstack/formula': patch
+'@objectstack/spec': patch
+---
+
+fix(objectql,driver-mongodb,formula): the `having` and per-aggregation evaluator compiles the whole-day comparison it is handed, and `$contains` asks membership on a JSON-stored field in `MongoDBDriver` and in `matchesFilterCondition` (ADR-0053 D-D1 items 5 and 9; the `FILTER_OPERATORS` `$contains` contract, #20822)
+
+Clause-②: no
+
+- **`@objectstack/objectql`: the aggregate evaluator's own whole-day copy is deleted.** The walker behind `having` and `aggregations[i].filter` no longer widens a bare `YYYY-MM-DD` `$lte`, or a `$between` maximum, on a `datetime` column to the whole day, and no longer drops the bound on `9999-12-31`. Through `engine.aggregate` nothing changes: the engine's seam lowers both positions with the shared `lowerFilterCondition` (`@objectstack/spec/data`) before the walker runs, by the object's declared `datetime` fields for the per-aggregation `filter` and by the aggregated column's type for `having`. A caller that passes no seam gets the comparison it wrote: `applyInMemoryAggregation(rows, ast, tz, fields)` called directly now counts `{ at: { $lte: '2026-02-01' } }` against that day's midnight. To keep the seam's reading on a direct call, lower each `filter` first with `lowerFilterCondition(filter, { isDatetimeColumn })`.
+- **`@objectstack/driver-mongodb`: `$contains` / `$notContains` ask membership on a declared JSON-stored field.** On a field `syncSchema` recorded as `multiple: true`, a multi-option type (`tags`, `multiselect`, `checkboxes`) or a JSON type, `translateFilter` (every verb, and the aggregation `$match`) now emits an array-only `$elemMatch` over the members the comparand names, with the candidate rule the SQL dialects bind (`jsonMembershipCandidates`, `@objectstack/core`): `'1'` names the string `'1'` or the number `1`, `'true'` the string or `true`. It used to emit a `$regex`, which MongoDB applies to each element, so `{ owners: { $contains: 'u1' } }` matched a stored `['u10']` and `{ tags: { $contains: 'red' } }` a stored `['redwood']`. `$notContains` is the exact complement, and still admits a row with no value. A scalar column, and a field whose declaration the driver does not hold (an object never synced, a standalone `translateFilter` call), keep the substring `$regex`.
+- **`@objectstack/formula`: `matchesFilterCondition` asks membership of a JSON-stored column.** When the caller supplies `options.fields` and it names the column, the declaration decides: membership on a JSON-stored column, substring on any other. Otherwise the stored value decides: an array asks membership, anything else substring. A stored array used to fail `$contains` and pass `$notContains` whatever it held.
+  - **The RLS write check, which evaluates a policy with this function, moves with it.** Under a `check` such as `record.tags.contains('x')` on a multi-valued field, a write whose post-image holds `['x']` (a row the same policy's read shows) is now admitted; it was refused `PERMISSION_DENIED` / 403. `['xy']` stays refused, and the read hides it.
+  - A scalar written to a declared multi-valued field is judged as written, before the write door wraps it in a list. So `tags: 'xy'`, which the check used to admit while the read hides the stored `['xy']`, is now refused 403. And `tags: 'x'` is now refused 403 too, although the read shows the stored `['x']`. Send the list, `tags: ['x']`.
+- **`@objectstack/spec`: docblock only, in the shipped `src/data/filter.zod.ts`.** The three pointers to the deleted `SqlDriver.calendarDayUpperBoundRewrite` / `calendarDayBetweenRewrite` now name the shared `lowerFilterCondition` at the seams, and the `FILTER_OPERATORS` `$contains` implementation-status list gains `driver-mongodb` and `formula`. No schema, type or export changes.
+- No exported name changes.

.changeset/21042-analytics-native-aggregate-policies.md

@@ -0,0 +1,27 @@
+---
+'@objectstack/core': minor
+'@objectstack/driver-sql': patch
+'@objectstack/service-analytics': patch
+---
+
+fix: the analytics native-SQL path aggregates with the engine's own aggregate policies, so one query answers one number whichever strategy serves it: `sum` / `avg` accumulate in double, a PostgreSQL boolean aggregand is cast, and an all-NULL `sum` answers `0`. The operand policies move from `@objectstack/driver-sql` to `@objectstack/core` (#21042)
+
+Clause-②: yes (widening)
+
+**New exports.** `@objectstack/core` exports the aggregate operand policies, moved here from `@objectstack/driver-sql`, where they were module-private. The driver now imports them and emits byte-identical statements.
+
+- `AGGREGATE_ACCUMULATION`: what each declared aggregate function accumulates in on PostgreSQL and MySQL. `avg` accumulates in double; `sum` accumulates in double over a fractional column; the counts, `min` and `max` take the column as stored.
+- `aggregandColumnClass(shape)`: the one column-class predicate those policies read, over a column's declared `{ type, multiple }`. It answers `'fractional'`, `'integral'`, `'boolean'`, or `undefined` for every other column, a multi-valued one included. The type `AggregandColumnClass` names the three classes.
+- `POSTGRES_BOOLEAN_AGGREGAND_CAST`: the functions whose boolean aggregand is cast to `int` on PostgreSQL. These are `sum`, `avg`, `min` and `max`; the two counts are never cast.
+- `doubleAccumulationOperand(operand, dialect)`: the column's text, parsed as a double, spelled for `'postgres'` or `'mysql'`.
+- `aggregandOperandSql(func, columnClass, dialect, operand)`: the operand an aggregate wraps, with the cast inside the double operand. The type `AggregandSqlDialect` names its dialects (`'sqlite'`, `'postgres'`, `'mysql'`, `'unknown'`).
+
+**What changed.** `POST /api/v1/analytics/query` and `POST /api/v1/analytics/dataset/query` served by `NativeSQLStrategy` (the default on a SQL driver) skipped three policies `SqlDriver.aggregate()` applies. So the ObjectQL strategy and `engine.aggregate` answered differently for the same query. Measured on SQLite and PostgreSQL 16.13:
+
+- On PostgreSQL, `sum` / `avg` over an exact-decimal column, and `avg` over an integer one, added exact decimals. For example, `0.1 + 0.2` answered `0.3` and `11 / 9` answered `1.222222222222222`, where the engine answers `0.30000000000000004` and `1.2222222222222223`. The native statement now accumulates in double, as the driver does.
+- On PostgreSQL, `sum` / `avg` / `min` / `max` over a boolean field answered `500` (`function sum(boolean) does not exist`). The native statement now casts the boolean aggregand to `int`, as the driver does, and answers the numbers the engine answers.
+- On every dialect, a group whose aggregand is NULL in every row, and a measure-scoped `sum` that admits no row, answered `sum` `null` at the cube door. The strategy now folds a `null` answer to `emptyGroupValueFor` (`@objectstack/spec`) for every measure, so that `sum` answers `0`. `avg`, `min` and `max` over nothing stay `null`. The dataset door already answered `0`.
+
+This is no narrowing: each answer moves to the value the platform already declared for the same query.
+
+**What did not move.** `@objectstack/driver-sql`'s statements and answers are unchanged: a move-proof test compiles each aggregate function over each column class on SQLite, PostgreSQL and MySQL, and the statements equal the ones captured before the move. SQLite's native statement is unchanged, because neither operand policy applies there. A host that relays no field declarations to the analytics service, or names no SQL dialect, gets today's native arithmetic.

.changeset/21131-dashboard-widget-options-door.md

@@ -0,0 +1,15 @@
+---
+'@objectstack/lint': minor
+---
+
+`os validate`, `os build` and `os lint` now warn on a dashboard widget `options` key that no renderer reads, by name, as `unconsumed-widget-option` — the check the SDUI page save gate already ran on a `dashboard` node, now run over dashboard metadata (`*.dashboard.ts`, `defineStack({ dashboards })`).
+
+Clause-②: yes
+
+**What is flagged.** Every key in a dataset-bound widget's `options` outside the read set `CONSUMED_WIDGET_OPTION_KEYS` from `@objectstack/sdui-parser`: `dateGranularity`, `description`, `limit`, `sortBy`, `sortOrder`, `stageOrder`. `DashboardWidgetOptionsSchema` stays open (`.passthrough()`), so such a key still parses; it just stops being silent. Typical cases are `icon`, `columns`, `format`, `currency`, `color`, `suffix`, `showLegend` and `horizontal`, none of which styles anything on a widget bound to a dataset, and a misspelled declared key such as `sortDirection` or `granularity`. The finding is reported at `dashboards[N].widgets[M].options`, and its message names the key.
+
+**Where presentation goes instead.** A number's format and currency are the dataset measure's `format` and `currency`; a tile's accent is the widget's `colorVariant`; a chart's look is the widget's `chartConfig`.
+
+**Same check, same level, same exemptions.** The rule is `validateDashboardWidgetOptions`, exported from `@objectstack/lint` and registered for all three commands. It calls `checkDashboardWidgetOptions` from `@objectstack/sdui-parser` and keeps its `warning` level and `code`. Widgets with no `dataset`, legacy `component` widgets and widgets carrying `suppressWarnings: ['unconsumed-widget-option']` are not flagged. It does not run on the Studio/REST/MCP publish path.
+
+**What changes for a project.** Nothing is refused, and nothing changes without `--strict`. `os validate --strict` and `os lint --strict` now exit 1 instead of 0 on a stack that was otherwise warning-clean and writes such a key. Across this repository's example apps (`app-crm`, `app-todo`, `app-showcase`, `app-multi-package`), no dashboard writes one: zero findings, and no example's exit code changes. To clear the warning, delete the key, or move the intent to the home named above.

.changeset/21174-admin-audit-metadata.md

@@ -0,0 +1,13 @@
+---
+'@objectstack/plugin-auth': patch
+---
+
+fix(plugin-auth): the compliance-ledger rows the admin identity endpoints write record the admin's decisions, never a value of a field of the user (#21174)
+
+Clause-②: no
+
+The admin create-user and set-user-password endpoints each write their own `sys_audit_log` row beside the rows plugin-audit's CRUD mirror writes for the same call. That row's free `metadata` copied values the call had just written into fields of the user. The ledger's read side narrows the mirror's before/after snapshots to what each reader is served, but it cannot narrow free metadata without deriving masking a second time, so a ledger reader the data plane withholds one of those fields from was served its value through the explicit row.
+
+The explicit row now carries only the admin's decisions — which operation ran, whether the password was generated, whether the account's address is a generated placeholder, whether the membership was bound and to which organization — plus its reference to the user (`object_name` and `record_id`). The values the call writes into the user's fields are recorded where they already were: on the mirror's `create` and `update` rows for those same writes, in the snapshot columns the read side narrows per reader. The decision set is a closed type, so a field value no longer compiles into the row.
+
+Migration: a reader that took a user field's value from the explicit row's metadata reads it from the mirror's row for the same write instead (its after-snapshot), served according to the reader's field access. Rows written before this release are stored data and are not rewritten.

.changeset/21178-remote-json-column-gate.md

@@ -0,0 +1,25 @@
+---
+"@objectstack/driver-turso": minor
+---
+
+fix(driver-turso)!: in remote mode, a filter on a declared JSON-stored field is refused with `INVALID_FILTER` / 400 for every operator the local face refuses there, and `$contains` / `$notContains` answer membership instead of a substring of the stored text (#21178)
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: not-required (no-migration-prescription) a refusal of a QUERY shape at the remote transport's filter compiler: the operators refused on a JSON-stored column are exactly the ones driver-sql's where refuses there (and so this driver's local transport), read from the one JSON_COLUMN_INCOMPATIBLE_OPERATORS set @objectstack/core holds, and $contains / $notContains move from a substring test to the membership test the local transport already answers, through the same jsonMembershipPredicate. No authorable key, spelling or stored metadata shape moves: FilterConditionSchema and every object, view and dataset definition parse and save as before, and nothing reads or rewrites a stored row. There is nothing for objectstack migrate meta to rewrite, since what changes is which query this transport answers, not what any metadata says; the refusal itself names the spelling to use. The other categories are closed on facts: the bumped package publishes (not unpublished); no ADR-0087 id covers a filter operator on a JSON-stored column and this diff adds none (not registered / already-registered); and the change is runtime behaviour, with no published export or type narrowed or removed — the one interface change is an ADDED optional method, a widening (not runtime-interface-only / type-surface-only). -->
+
+**BREAKING** (`@objectstack/driver-turso`, remote mode): this narrows what `TursoDriver` answers when its `url` is a remote libSQL endpoint (such as `libsql://` or `https://`), the transport every hosted tenant database runs on, for every door that compiles a `where`: `find`, `findOne`, `count`, `updateMany`, `deleteMany`, `aggregate` and distinct values. It ships as `minor` under the launch-window convention for accept-set narrowings. Local and embedded-replica mode inherit `driver-sql`'s compiler and already answered this way; nothing moves there.
+
+**What is refused.** On a field the object declares JSON-stored (a structured-JSON type such as `json` or `address`, an inherently multi-value option type such as `tags`, `multiselect` or `checkboxes`, or a `select`, `radio`, `lookup`, `user`, `file` or `image` field declared `multiple: true`), a `where` that aims any operator in `@objectstack/core`'s `JSON_COLUMN_INCOMPATIBLE_OPERATORS` at the field is refused with `INVALID_FILTER` / 400, at any depth under `$and` / `$or` / `$not`, before any statement runs: `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$between`, `$in`, `$nin`, `$startsWith`, `$endsWith`, `$icontains`, `$like`, `$ilike`, and implicit equality (`{ "owners": "u1" }`), whatever the comparand, `null` included.
+
+**What `$contains` / `$notContains` answer now.** Membership: `{ "owners": { "$contains": "u1" } }` matches the rows whose stored list holds `u1` as an element, so it no longer matches a row holding only `u10`; `$notContains` is its exact complement, a row with no value included; and a structured-JSON object answers no member at all, instead of matching text inside its serialization. On a scalar text field both remain the substring test they were.
+
+**What an author sees now.** The body the local transport answers for the same filter, byte for byte: the filter WAS NOT APPLIED, the comparison can never equal one member of a stored list, and the spelling to use, `{ "FIELD": { "$contains": "a" } }` for membership, or an `$or` of `$contains` for any-of. The field and the operator are withheld from the message, and the full diagnostic, naming both, is written to the driver's logger at `warn`.
+
+**Why.** The remote transport compiles its own SQL and read neither the shared refused set nor the membership construct, so over a `multiple: true` lookup holding `["u1","u2"]`, `["u2"]`, `["u3","u1"]` and `["u10"]` it answered: `$nin: ["u1"]` and `$ne: "u1"` every row, the rows holding `u1` included; `$eq`, `$in` and implicit equality no row; `$lt` / `$lte` a lexicographic verdict over the serialization; `$startsWith: "["` and `$endsWith: "]"` every row with a value; `$contains: "u1"` the row holding only `u10` too. One driver gave two answers to one filter depending only on the connection string, and the exclusion operators failed open.
+
+**Who is affected.** A caller, saved filter, list view, report or read scope that reaches a remote-mode `TursoDriver` with one of those operators on a JSON-stored field and read the rows it got as the answer. Write `$contains` for "holds this member", an `$or` of `$contains` for "holds any of these", `$not` around either for the exclusion, and `$null` / `$exists` / `$empty` for presence.
+
+**New optional API.** `RemoteTransport.setJsonColumnResolver(resolver)` in `@objectstack/driver-turso`, which `TursoDriver` wires to its own `isJsonColumn`, beside `setDeclaredValueShapeResolver`. A `RemoteTransport` driven standalone without it treats no column as JSON-stored and compiles as before.
+
+**Unchanged.** `$contains` and `$notContains` on a scalar field, `$exists`, `$null` and `$empty`; every operator on a field that is not declared JSON-stored; and a table this driver holds no declaration for, where nothing is judged.