Skip to content

Commit 7a6e91f

Browse files
committed
Merge remote-tracking branch 'origin/main' into claude/issue-19727-ui-predicate-fault-submit-refuses
2 parents 326ad25 + a0920b4 commit 7a6e91f

28 files changed

Lines changed: 2502 additions & 132 deletions
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
---
2+
"@objectstack/service-analytics": minor
3+
---
4+
5+
fix(service-analytics)!: the analytics `where` door refuses a list in the equality slot instead of reading it as `IN` (#19888)
6+
7+
Clause-②: no (narrowing)
8+
9+
<!-- adr-0087: not-required (already-registered filter-equality-array-comparand-refused) the transition from a list in the equality slot to the operator it stood in for was registered by #19757 for the FilterCondition equality slot at the runtime filter doors; this change adds no new transition, it brings the analytics object-form `where` and the draft-data preview under the one already on the ledger, and the entry's own prescription and acceptance criteria apply verbatim -->
10+
11+
**BREAKING**: this narrows what the analytics faces of `@objectstack/service-analytics` accept. A caller `where`, a dataset `filter` or a measure `filter` that carries a list in the equality slot, `{ field: [...] }` (the empty list included) or `{ field: { $eq: [...] } }`, at any depth under `$and` / `$or` / `$not` or inside a nested relation, compiled before this change. It is now refused with `INVALID_FILTER` / 400. It ships as `minor` under the launch-window convention for accept-set narrowings. The remedy is `$in`:
12+
13+
| you wrote | write instead |
14+
|:--|:--|
15+
| `{ stage: ['won', 'lost'] }` or `{ stage: { $eq: ['won', 'lost'] } }`, meaning "one of these values" | `{ stage: { $in: ['won', 'lost'] } }` |
16+
| `{ stage: ['won'] }`, meaning one value | `{ stage: 'won' }` |
17+
| `{ stage: [] }` | `{ stage: { $in: [] } }`, which matches no row, as the bare list did |
18+
19+
`$in` charts the rows the implicit list charted before. The refusal also names `{ "$contains": "…" }` for "the stored list holds a value" on a multi-value field.
20+
21+
Ruling 乙 of #19757 refuses a list in the equality slot at the shared comparand-shape face in `@objectstack/spec`, for every driver at once. The analytics `where` door met that face only for the `FilterArray` spelling (`['stage', '=', ['won', 'lost']]`), which was already refused. The object spelling was compiled by the analytics filter normalizer, which read one condition four ways:
22+
23+
- `{ stage: ['won', 'lost'] }` compiled to `stage IN (...)`. On the ObjectQL path the engine received `{ stage: { $in: [...] } }`, so the engine's own shared-face check never saw the list.
24+
- `{ stage: { $eq: ['won', 'lost'] } }` compiled to `stage = 'won'`, and `'lost'` was dropped without a word.
25+
- `{ stage: { $eq: [] } }` compiled to no predicate at all, so the chart was drawn over every row.
26+
- `{ stage: [] }` compiled to the FALSE constant.
27+
28+
Each list is now handed to the shared face's equality arm before any node is built. Both spellings of one condition therefore get the same refusal, with the same wording, path and `$in` prescription. The draft-data preview runs the same gate, so a drafted chart refuses what the published chart refuses. Before, it compared each row against the list's string form.
29+
30+
Who is affected: nothing in this repository's examples, seeds or docs authors the shape (measured over `examples/**`, `packages/**` and the fenced code in the docs). Stored datasets, dashboard widget filters, report runtime filters and measure filters in a deployment were NOT measured. The authoring schema still admits the shape, so such a document still publishes, and it is refused when it is charted. The refusal names the field and the path. `$ne` with a list is not part of the ruling and is not judged here. The list operators (`$in`, `$nin`, `$between`) keep their lists, and every scalar, `null` included, compiles as before.
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
'@objectstack/driver-turso': minor
3+
'@objectstack/cli': minor
4+
---
5+
6+
fix(driver-turso): a REMOTE `TursoDriver` refuses to plan the ADR-0104 media column move instead of answering that there is nothing to move, and `os migrate files-to-references` reports that refusal as a column step it could not judge (#19894)
7+
8+
Clause-②: yes (narrowing)
9+
10+
**BREAKING for callers that plan the media column move on a remote Turso datasource** — `TursoDriver.planMediaColumnMove()` in `remote` transport mode (for example a `libsql://` URL) now throws a `NOT_IMPLEMENTED` / `501` error, where it used to answer `{ plans: [], refusals: [] }`. The `local` (`:memory:` and `file:`) and `replica` modes plan exactly as before, and the local modes plan exactly what `SqlDriver` plans for the same declaration.
11+
12+
What the refusal replaces, measured on the transport's SQLite-backed test double: a table with a `file` and an `image` field, synced through each of the three remote schema doors (`syncSchemasBatch`, `syncSchema`, `initObjects`), held its two TEXT media columns on the remote database, and the remote face answered an empty scan on every door. The inherited planner walks the objects `SqlDriver`'s own schema sync registers, which no remote schema door reaches, and probes each table through the placeholder in-memory Knex connection a remote driver is built with. The local and embedded-replica faces planned two `unquote` moves for the same declaration. `os migrate files-to-references` printed that empty scan as "Column step: nothing to move — this datastore declares no single-value media column".
13+
14+
- **The command reports the refusal instead of failing on it.** `os migrate files-to-references` calls the planner only after the backfill and its self-check have passed, and an `--apply` run has recorded the deployment flag by then. Measured on the command's own test doubles before this change, a planner that throws ended the run with the error alone (`--json`: `{"error": …, "code": "NOT_IMPLEMENTED"}`) and exit 1, with no backfill report, no verify report and no word about the flag it had recorded. The column step now catches a `NOT_IMPLEMENTED` refusal by its code and reports it as a skip: the text face prints `Column step: NOT JUDGED` with the driver's message, and `--json` gains `columnMoveRefused` — `{ error, code }` when the driver refused, `null` otherwise — beside `columnMove: null` and `columnsMovedAt: null`. The backfill, verify and flag reports are emitted as on any other run, nothing is stamped, and the exit code is the self-check's, as it already was for the command's other column-step skips.
15+
- **Any other throw from the planner still fails the command**, through the same error report and exit 1 as before.
16+
- **A genuinely empty scan still reads "nothing to move".**
17+
- **No new error code.** `NOT_IMPLEMENTED` / `501` is a standard code, the envelope this transport already uses for its remote transaction, auto-number, deferred-DDL and drift-detection refusals.
18+
19+
**If you are refused:** the backfill, its self-check and, on `--apply`, the deployment flag are unaffected. A remote Turso datasource keeps its single-value media columns on the JSON encoding: measured on the same double, the remote face writes a file id as a JSON string and reads it back as the id, and it does not read the record of a completed column move.
20+
21+
<!-- adr-0087: not-required (no-migration-prescription) Nothing authorable is removed, renamed or reshaped: no spec key, no export, no stored row and no config key — `planMediaColumnMove` keeps its name and its signature, and the command's `--json` document only gains a key. There is no old spelling that maps to a new one: the refused call asked the remote transport for a capability it never delivered, and the refusal itself says what the datasource keeps. -->

‎.changeset/19912-json-backfill-depth-limit.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,6 @@ fix(driver-sql): the local SQLite `Field.json` storage backfill no longer turns
77

88
The backfill that converges legacy json cells on their JSON-encoded form (it came with the change that made the SQLite write path JSON-encode every json value; the issue number that change cites no longer resolves, and its live record is the `SqlDriver.backfillCanonicalJsonEncoding` doc block and `sql-driver-12380-json-roundtrip.test.ts`) ran one `UPDATE … set col = json_quote(col)` over every TEXT cell SQLite's `json_valid()` rejects. `json_valid()` answers 0 for JSON nested more than 1000 levels deep (SQLite's JSON depth limit in every build this repository bundles), while the driver reads such a cell with `JSON.parse` without trouble. So a deep array written correctly through the driver was quoted into a JSON string by the next `syncSchema` / `initObjects`, and read back as a string from then on — silently, with no error.
99

10-
SQL now only pre-selects the candidate cells, a page at a time. The driver's own codec decides each one: a cell `JSON.parse` reads is left exactly as stored; a cell it cannot read is a legacy plain string and is rewritten to `JSON.stringify` of that string, byte-for-byte what the old statement wrote for it wherever the engine reads the stored bytes back verbatim. A cell the engine does not read back verbatim is left as stored and keeps reading as it did, where the old statement rewrote it: on `SqliteWasmDriver`, a legacy text with a leading U+FEFF or an embedded NUL (sql.js drops both when it reads the text); on any engine, text holding invalid UTF-8. Each rewrite is a compare-and-set on the text it was decided from, so a value written concurrently is never overwritten, and a re-run over a converged table still writes nothing. This covers every local SQLite face that inherits the backfill: `SqlDriver` on every client it treats as SQLite (`better-sqlite3`, `sqlite3` and its alias `sqlite`), `SqliteWasmDriver`, and `TursoDriver` in local mode.
10+
SQL now only pre-selects the candidate cells, a page at a time. The driver's own codec decides each one: a cell `JSON.parse` reads is left exactly as stored; a cell it cannot read is a legacy plain string and is rewritten to `JSON.stringify` of that string, byte-for-byte what the old statement wrote for it wherever the engine reads the stored bytes back verbatim. A cell the engine does not read back verbatim is left as stored and keeps reading as it did, where the old statement rewrote it: text holding invalid UTF-8, measured on better-sqlite3 and sql.js. `SqlDriver` on better-sqlite3, `TursoDriver` in local mode (which runs on better-sqlite3 too) and `SqliteWasmDriver` (sql.js) were each measured to read such a cell back with U+FFFD in place of the invalid bytes, so the text the rewrite would be decided from is not the stored text, and the cell is left alone. A legacy text with a leading U+FEFF or an embedded NUL is read back verbatim on all three faces, and is rewritten like any other plain string. Each rewrite is a compare-and-set on the text it was decided from, so a value written concurrently is never overwritten, and a re-run over a converged table still writes nothing. This covers every local SQLite face that inherits the backfill: `SqlDriver` on every client it treats as SQLite (`better-sqlite3`, `sqlite3` and its alias `sqlite`), `SqliteWasmDriver`, and `TursoDriver` in local mode.
1111

1212
The decision rule is exported from `@objectstack/driver-sql` as `recoverUnencodedJsonText(stored)`, and `@objectstack/driver-turso`'s remote codec-residue backfill now imports it instead of carrying its own copy, so the local and remote backfills apply one rule. That is a new public export on `@objectstack/driver-sql`'s root entry, and the reason this package takes `minor`: the function returns `null` for text `JSON.parse` accepts and `JSON.stringify(stored)` for text it rejects, and it is exported so that `SqlDriver.backfillCanonicalJsonEncoding` and the remote backfill's `recoverResidueCell` decide each cell by one shared rule rather than by two copies that could drift apart. The remote backfill's behaviour is unchanged.
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
"@objectstack/driver-sqlite-wasm": patch
3+
---
4+
5+
fix(driver-sqlite-wasm): a text value now round-trips byte-for-byte, as it does through `SqlDriver` on better-sqlite3 — an embedded U+0000 no longer cuts the stored value short, and a leading U+FEFF is no longer dropped when the value is read (#19978)
6+
7+
Clause-②: no
8+
9+
`SqliteWasmDriver` changed a text value without raising, at two points in sql.js (measured on sql.js 1.14.1, the version the lockfile installs, against better-sqlite3, which round-trips every value below):
10+
11+
- **Write.** sql.js binds a string with a length of `-1`, so SQLite stores it only up to the first U+0000: `'a'` + U+0000 + `'b'` was stored as the single byte `61`, and `'ab'` + U+0000 as `6162`.
12+
- **Read.** sql.js decodes a text cell up to the first NUL byte, through a decoder that drops a leading byte-order mark: a stored `610062` read back as `'a'`, and a stored text beginning with U+FEFF read back without it. A U+FEFF was always stored, because the write keeps it.
13+
14+
What changes:
15+
16+
- Every text cell the driver reads is decoded from its stored bytes, so a U+0000 anywhere in it and a U+FEFF at its start come back as stored. This includes values already on disk: a stored text that begins with U+FEFF now reads back with it.
17+
- A string value holding U+0000 is bound as its UTF-8 bytes, and the parameter that receives it becomes `+CAST(<parameter> AS TEXT)`, so SQLite stores the same TEXT value better-sqlite3 stores, and compares it the same way. Positional `?`, numbered `?NNN` and named parameters are all numbered as SQLite numbers them. A statement that binds no such string runs exactly as before.
18+
- An equality filter on such a value now compares the whole value. Before, the comparand was cut at the same U+0000 as the stored value, so `{ v: 'a' }` also matched a row written as `'a'` + U+0000 + `'b'`. It no longer does.
19+
- The local `Field.json` storage backfill (`SqlDriver.backfillCanonicalJsonEncoding`) now converges a legacy json text cell holding a leading U+FEFF or an embedded U+0000 on this driver too: measured on the next `initObjects`, a stored `EFBBBF78` becomes `22EFBBBF7822` and a stored `610062` becomes `22615C75303030306222`, the same bytes better-sqlite3 writes, where before this change both were left as stored because sql.js did not read them back verbatim.
20+
- If the loaded sql.js has no `Statement.getBlob` (the one sql.js read that carries a byte length), reading a text cell throws instead of decoding through the lossy path. sql.js 1.14.1 has it in its Node, browser and debug builds.
21+
22+
What does not change: a value already stored cut short stays cut short. The bytes after the U+0000 were never written, so nothing can restore them.
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
---
2+
'@objectstack/runtime': patch
3+
---
4+
5+
fix(runtime): `POST /automation/:name/runs/:runId/resume` checks WHO is resuming — the run's own starter, or the `sys_automation_run` read grant (#19987)
6+
7+
Clause-②: no
8+
9+
**The defect.** The resume route checked the body and then resumed. It never asked who the caller was. An authenticated user holding another user's run id could continue that user's paused run, and the run then went on under the identity STORED on the run: its data nodes ran as the user who started it, with the values the other user submitted. The screen read on the same pause, `GET /automation/:name/runs/:runId/screen`, already refused that caller.
10+
11+
**The fix.** The route now asks the screen read's own question, through the same function: the caller must be the user who started the run (`getRun(runId).trigger.userId`), OR hold read access to `sys_automation_run` (the operator override), OR be a system context. Anyone else gets `403 PERMISSION_DENIED`, the screen read's code and status, and the run is not touched: nothing reaches the engine, so the pause stays parked for the person it is waiting on. The message states the screen read's requirement word for word under this route's own verb: "Resuming a paused run requires being the identity that triggered the run, or read access to 'sys_automation_run'."
12+
13+
**What stays the same.**
14+
15+
- The user who started a run resumes it exactly as before, with no grant needed, and still does while the permission subsystem is down.
16+
- The suspended node's own gate (`resumeAuthority`) is unchanged and still answers behind this one. An `approval` pause is still refused on this route for every caller; approvers decide through the approvals API, which never uses this route.
17+
- Body checks come first, as before, so every `400` for a malformed body is unchanged for every caller.
18+
- For the starter, a `sys_automation_run` reader and a system context, every engine answer is unchanged, including the `404` for an unknown or finished run.
19+
20+
**Who needs to act.** A caller that resumes runs it did NOT start (for example an integration that feeds a `wait` node's signal, or a support tool) now needs read access to `sys_automation_run`. Without it, it gets the `403` above. Such a caller also gets that `403`, not the old `404`, for a run id that does not resolve. The check fails closed there because a run that is still paused can read as "not found" while the run store is degraded.

‎content/docs/automation/flows.mdx‎

Lines changed: 32 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -764,7 +764,7 @@ POST /api/v1/automation/{flow}/runs/{runId}/resume
764764
| `approval` | a human decision | the approvals service (`POST /api/v1/approvals/requests/:id/approve\|reject`) — resumes down the matching `approve` / `reject` edge. **Decide through the approvals API**; the resume route above **refuses** an approval pause outright (see below). |
765765
| `screen` | a user submits the form | the UI runner posting the collected `inputs` — **validated server-side against the screen's declared `fields`** (see below); a `paused` response carrying the next `screen` chains multi-step wizards under one stable `runId` |
766766
| `wait` (timer) | an ISO-8601 duration elapses | **automatically** — a one-shot job resumes the run; after a cold boot the engine re-arms pending timers from the durable store (overdue timers resume immediately) |
767-
| `wait` (signal) | a named external event | any caller invoking `resume(runId)` |
767+
| `wait` (signal) | a named external event | the external producer calling the resume route — as the user who started the run, or with read access to `sys_automation_run` (see below); in process, `resume(runId)` on the automation service |
768768

769769
<Callout type="warn">
770770
**A pause ends the flow's retry-governed segment.** `errorHandling: { strategy:
@@ -807,11 +807,37 @@ change (there is no `FLOW_STRANDED`), and the trigger route's `400` carries
807807
referent there. The shape is `ResumeFailureDetailsSchema` in
808808
`@objectstack/spec/api`.
809809

810-
### Who may resume — the gate is the suspended node
811-
812-
The resume route is generic, so **the node the run is parked on** decides
813-
whether a raw resume is a legitimate continuation. Every action descriptor
814-
carries a `resumeAuthority`:
810+
### Who may resume — the caller, then the suspended node
811+
812+
The route asks two questions, in this order. Both must pass.
813+
814+
**1. Who is resuming.** A resumed run continues under the identity stored on
815+
the run, so its data nodes run as the user who **started** it, whoever sent the
816+
values. The route therefore admits only:
817+
818+
- the user who started the run (the run's `trigger.userId`) — no extra
819+
permission needed;
820+
- a caller with **read access to `sys_automation_run`** — the operator
821+
override, for support tooling or an integration that feeds a signal `wait` on
822+
runs it did not start;
823+
- a system context.
824+
825+
Anyone else gets **403** `PERMISSION_DENIED` — "Resuming a paused run requires
826+
being the identity that triggered the run, or read access to
827+
'sys_automation_run'." — and nothing reaches the engine, so the run stays
828+
parked for the person it is waiting on. This is the same rule, code and status
829+
as reading the run's screen (`GET /api/v1/automation/{flow}/runs/{runId}/screen`),
830+
checked by the same function. The check runs after the body checks, so a
831+
malformed body is still a `400` for every caller. If the run id does not
832+
resolve, only an admitted caller (a system context or a `sys_automation_run`
833+
reader) gets the engine's `404`. Everyone else gets the `403`, because a run
834+
that is still paused can read as "not found" while the run store is degraded.
835+
The MCP `resume_run` tool is stricter: only the user who started the run, with
836+
no operator override (see [actions as tools](/docs/ai/actions-as-tools)).
837+
838+
**2. What the run is parked on.** The resume route is generic, so **the node
839+
the run is parked on** decides whether a raw resume is a legitimate
840+
continuation. Every action descriptor carries a `resumeAuthority`:
815841

816842
- **`'any'`** (`screen`, `wait`): the caller supplies the continuation and the
817843
route is the intended door.

0 commit comments

Comments
 (0)