Skip to content

Commit b0705c1

Browse files
claudeclaude
committed
Merge origin/main into claude/issue-16094-liveness-dead-warns
2 parents 1d23970 + 9c8b65a commit b0705c1

110 files changed

Lines changed: 6867 additions & 2612 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.changeset/20821-plan-deferred-ddl-reads.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
'@objectstack/driver-sql': patch
3+
---
4+
5+
fix(driver-sql): `os migrate plan` on a database that does not exist yet no longer prints `DATABASE_ERROR` for the tables whose DDL it deferred (#20821)
6+
7+
`os migrate plan` (and the boot of `os migrate apply`) runs with the SQL driver's DDL deferred: the driver records every table as pending `create_table` and creates none of them. The same boot then reads `sys_metadata`, `sys_metadata_activation` and `sys_migration`. On a new database those tables do not exist yet, so each read was refused, and each refusal printed a line like this on the driver's warn channel (stderr by default):
8+
9+
```text
10+
[sql-driver] DATABASE_ERROR — the backend refused a read on 'sys_metadata' (SQLITE_ERROR) ... no such table: sys_metadata
11+
```
12+
13+
Nothing was wrong: every reader already answers from the refusal, and the plan lists the same tables as pending creates. A dry run on a new database printed six of these lines.
14+
15+
The driver now sends such a refusal to the logger's `debug` channel instead of `warn`, when all three hold:
16+
17+
- this driver has DDL deferred;
18+
- the refused statement targets a table whose DDL this driver deferred;
19+
- the shared `isMissingTableError` predicate from `@objectstack/types` recognises the refusal as that table being missing.
20+
21+
The default logger has no `debug`, so the line is not printed. The refusal is still thrown to the caller with the same envelope (`DATABASE_ERROR`, status 500), and the plan's output is unchanged.
22+
23+
What still warns:
24+
25+
- every other refusal on a deferred driver, such as a malformed statement on a table that exists;
26+
- a missing table that the driver did not defer, such as a table nothing in the boot declares;
27+
- every refusal once the deferred DDL has been applied, or on a driver that never deferred any.
28+
29+
There is nothing to migrate.

‎.changeset/20890-dataset-dimension-json-stored-refused.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
"@objectstack/lint": minor
3+
---
4+
5+
fix(lint)!: `os validate`, `os build` and `os lint` refuse a dataset dimension over a JSON-stored field, the group key the analytics door already refuses at query time
6+
7+
Clause-②: yes (narrowing)
8+
9+
<!-- adr-0087: not-required (no-migration-prescription) a refusal of a grouping TARGET at authoring time: a dataset dimensions entry whose field resolves to a declared structured-JSON field (json, composite, repeater, record, location, address, vector) or multi-value field (multiselect, checkboxes, tags, or a select, radio, lookup, user, file or image declared multiple: true). It is the authoring leg of the analytics door that already refuses every query grouping by such a column with 400 INVALID_FIELD, and that door's own changeset declared this category for this surface. No authorable key, spelling, export or stored shape moves: DatasetSchema keeps parsing every dimension, no stored row is read or rewritten, and which scalar part of a document, or which member of a list, an author meant to group on is not something a ledger entry can rewrite. The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id covers a grouping target, and dataset-measure-aggregate-field-type-refused says in its own reason that a field used as a DIMENSION is untouched (not already-registered); and the change is a rule verdict, not a declaration (not runtime-interface-only or type-surface-only). -->
10+
11+
**BREAKING**: metadata that passed `os validate`, `os build` and `os lint` can now fail, and so can a runtime dataset save (Studio, REST `/meta`, MCP), which runs the same rule. A dataset dimension is a group key, and the analytics door refuses a query that groups by a JSON-stored column with `400 INVALID_FIELD` before any SQL is built, so such a dimension could be declared but never served. The new rule `dimension-json-stored-field-refused` (gating, `error`) refuses it where the author writes it. It ships as `minor` under the launch-window convention for accept-set narrowings. No export is removed. The package entry exports the new id as `DIMENSION_JSON_STORED_FIELD_REFUSED`, beside `MEASURE_AGGREGATE_FIELD_TYPE_REFUSED`, and the `rule` member of `DatasetMeasureAggregateFinding` gains it.
12+
13+
**What is refused.** A dimension whose `field` resolves, on the dataset's object or across its join chain, to a field declared with a structured-JSON type (`json`, `composite`, `repeater`, `record`, `location`, `address`, `vector`) or a multi-value declaration (`multiselect`, `checkboxes`, `tags`, or a `select`, `radio`, `lookup`, `user`, `file` or `image` declared `multiple: true`). The two classes are `@objectstack/spec/data`'s `STRUCTURED_JSON_TYPES` and `isMultiValueField`, the predicates the analytics door reads.
14+
15+
**What an author sees now.** The finding names the dataset, the dimension, the field, the object that declares it and its declaration, and says the analytics door refuses every query that groups by it. It names the route: group by a field that stores one scalar value, storing the part of the document you group on in a field of its own; for a multi-value field, filter by one member with `$contains` in a record query, one query per member. It is located at `datasets[N].dimensions[M].field`, name-keyed on the runtime wire.
16+
17+
**Unchanged.** A dimension over any other field, a single-value `select` or `lookup` included; a dimension whose field does not resolve (`dataset-field-unknown` reports that) or declares no type; a dataset over an object this stack does not define; measures, filters and every other position. A cube dimension (`analyticsCubes`) is not judged: no authoring rule reads cubes.

‎.changeset/20890-dataset-distinct-multiple-refused.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
"@objectstack/lint": minor
3+
---
4+
5+
fix(lint)!: a dataset `count_distinct` measure over a field declared `multiple: true` is refused by `measure-aggregate-field-type-refused`, as the compile leg and the engine already refuse it
6+
7+
Clause-②: yes (narrowing)
8+
9+
<!-- adr-0087: not-required (already-registered dataset-measure-aggregate-field-type-refused) the registered entry carries this family's hand-migration, an aggregate the field accepts with count as the one that stays for a JSON-stored field, and the count_distinct narrowing over the JSON-stored types already rides it. A select, radio, lookup, user, file or image field declared multiple: true is the one JSON-stored shape the per-type table cannot see; the dataset compile leg and the engine's count_distinct door refuse it beside the table's row, and this is the authoring leg of that same pair, so it adds no surface of its own. -->
10+
11+
**BREAKING**: metadata that passed `os validate`, `os build` and `os lint` can now fail, and so can a runtime dataset save, which runs the same rule. `measure-aggregate-field-type-refused` reads the field's declaration, not its type alone: `count_distinct` over a `select`, `radio`, `lookup`, `user`, `file` or `image` field declared `multiple: true` is refused, because that field is a list stored as JSON and no two backends compare such values for equality alike. The dataset compile leg already answers the pair `400 DATASET_INVALID`, and the engine's `count_distinct` door answers it `400 INVALID_FIELD`. It ships as `minor` under the launch-window convention for accept-set narrowings.
12+
13+
**What an author sees now.** The finding names the measure, the field, the object and the declaration with its flag (`select` with `multiple: true`), and says the aggregate accepts its row's types, none of them with `multiple: true`. The hint names the aggregates the declaration does accept, read from the same predicate: `count`.
14+
15+
**What to write instead.** `count` over the field, or `count_distinct` over a field that stores one scalar value. To count the records holding one member, filter by it with `$contains` in a record query.
16+
17+
**Unchanged.** `count_distinct` over the same types without the flag; `count` over any field; every other aggregate, whose rows accept no multi-capable type and whose verdicts therefore do not move; and the skips the rule already had.

‎.changeset/20901-step18-rationale-form-view-subform-columns.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
---
2+
'@objectstack/spec': patch
3+
---
4+
5+
fix(spec): protocol 18's migration rationale now covers the form view's inline grid columns and the identity-only currency `scale` refusal (#20901)
6+
7+
**`@objectstack/spec`**
8+
9+
- **`MIGRATIONS_BY_MAJOR[18].rationale` gains one fragment, `form-view-subform-columns-closed`.** It is the paragraph `os migrate meta --step` shows for the protocol 17 → 18 hop. The new sentences say that a form view's `subforms[].columns` now takes the strict `InlineGridColumnSchema` a relationship field's `inlineColumns` takes, that the conversion `form-view-subform-columns-canonicalized` respells a `{ field }` column as `{ name }` in stored rows and assembled artifacts while an author writing `field` is refused, and that `defineStack` refuses `scale` on a column that declares no `type` when its `name` is a `currency` field of a child object declared in the same stack (`inline-grid-column-identity-only-currency-scale-refused`).
10+
- Text only: no schema, conversion or migration entry changes, and `conversionIds` and `semantic` for step 18 are unchanged.

‎.changeset/20919-core-import-runner.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
'@objectstack/core': minor
3+
---
4+
5+
feat(core): the bulk-import runner, its row coercion, the mapping apply and the field-meta map now live in `@objectstack/core`, beside `bulkWrite` (#20919)
6+
7+
`runImport` (with `sanitizeRowError` and its option/result types), the cell
8+
coercion (`coerceRow`, `coerceFieldValue`, `parseDateCell`, `parseNumberCell`,
9+
`parseBooleanCell`, `matchOption`, `splitMulti`, `isBlank`), the `mapping`
10+
artifact pipeline (`applyMappingToRows`, `refuseUnknownMappingTargets`,
11+
`MappingArtifactLike`, `MappingFailure`, `ApplyMappingOptions`) and the field
12+
metadata map (`buildFieldMetaMap`, `ExportFieldMeta`) are exported from
13+
`@objectstack/core`. They moved here unchanged from `@objectstack/rest` so the
14+
connector sync executor in `@objectstack/service-automation` writes through the
15+
same runner as the HTTP import door without depending on the HTTP layer. Nothing
16+
to change for consumers: `@objectstack/rest` re-exports every name it exported
17+
before.

‎.changeset/20919-rest-re-exports.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
'@objectstack/rest': patch
3+
---
4+
5+
refactor(rest): the import runner, coercion, mapping apply, field-meta map and error classification moved to `@objectstack/core` / `@objectstack/types`; `rest` re-exports them (#20919)
6+
7+
`runImport`, `coerceRow`, `buildFieldMetaMap` and their types (from the package
8+
index), and `mapDataError` with its sibling classification exports, are now
9+
re-exported from their new homes — byte-identical code, the same names, the same
10+
behaviour at both import routes and at `plugin-auth`'s identity import. Nothing to
11+
change for consumers.

‎.changeset/20919-service-automation-connector-pull.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
'@objectstack/service-automation': minor
3+
---
4+
5+
feat(service-automation): the connector sync executor pulls a `mapping`'s `connectorSource` and writes it through the import runner (#20919)
6+
7+
`AutomationServicePlugin.pullConnectorSource({ mapping, context })` (and the
8+
exported `pullConnectorSource(deps, opts)`) reads the mapping through the
9+
protocol's `getMetaItem`, resolves `connectorSource.connector` to a declared
10+
(`connectors[]`) `rest` or `openapi` instance, makes ONE call to its read action,
11+
takes the array at `recordsPath` (default `body`), projects it through the
12+
mapping's `fieldMapping` and writes it with `@objectstack/core`'s `runImport` —
13+
the import door's coercion, `mode` / `upsertKey` matching and per-row verdicts,
14+
with the door's defaults for every knob `connectorSource` does not declare.
15+
16+
- **Watermark, read from the target.** For `connectorSource.watermark`, the
17+
starting point sent as `query[watermark.param]` is the highest value already
18+
stored in the target field a `fieldMapping` entry copies `watermark.field` onto
19+
(transform `none`). Nothing else stores it.
20+
- **One response per pull.** The connector's paging is not followed.
21+
- **Loud refusals,** each a `ConnectorPullError` (`code`, `status`, `reason`)
22+
raised before anything is written: a plugin-registered or unregistered
23+
connector, a degraded instance, a provider other than `rest` / `openapi`, an
24+
undeclared action, `update` / `upsert` with an empty `upsertKey`, a
25+
`javascript` transform, an unmapped `watermark.field`, an `ok: false` answer,
26+
a non-array at `recordsPath` and a non-object record.
27+
- **Nothing schedules a pull** — a `job` will drive it; the caller supplies the
28+
execution context.
29+
30+
The plugin now records the provider of each declared connector instance it
31+
materializes, which the executor reads.

‎.changeset/20919-spec-connector-source-live.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---
2+
'@objectstack/spec': patch
3+
---
4+
5+
docs(spec): `mapping.connectorSource` is pulled when a job drives it — the describes say where the watermark is read from and that a pull reads one response (#20919)
6+
7+
The `connectorSource` describe no longer says the pull is not executed: the
8+
connector sync executor in `@objectstack/service-automation` reads the binding,
9+
and nothing schedules a pull until the `job` stage lands. `watermark.field` now
10+
states that the next pull's starting point is read from the TARGET field a
11+
`fieldMapping` entry copies it onto (an unmapped one is refused at pull time), and
12+
`watermark` states the one-response limit: the connector's paging is not followed,
13+
so a paged endpoint yields its first page only. The liveness ledger's
14+
`connectorSource` rows are `live` and keep `authorWarn` (nothing schedules a pull
15+
yet). The retired `connector.syncConfig` prescription and the
16+
`connector-sync-keys-retired` upgrade entry say the same, and the entry's
17+
acceptance criterion no longer claims the connector is validated at authoring.
18+
No key, value or default changed.

‎.changeset/20919-types-data-error-classification.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
'@objectstack/types': minor
3+
---
4+
5+
feat(types): the data-error classification table (`mapDataError` and its judgements) is exported from `@objectstack/types` (#20919)
6+
7+
The classification half of `@objectstack/rest`'s error boundary — `mapDataError`
8+
(a thrown error → the `{ status, body }` ADR-0112 answer, without emitting it),
9+
`declaredHttpStatus`, `declaredServerFaultAnswer`, `sandboxBusinessMessage`,
10+
`boundedDeclaredUserMessage`, `boundedDeclaredRefusalMessage` and
11+
`isEngineDuplicateRecordEnvelope` — moved here unchanged, beside the primitives it
12+
composes, so the bulk-import runner in `@objectstack/core` judges a failed row with
13+
the same table the REST door answers with. `@objectstack/rest` keeps the emitters
14+
and re-exports every name it exported before. The module also exports the eight
15+
helpers the REST emitters compose (`truncateClientMessage`, `thrownCodeFields`,
16+
`withoutDeclaredCodePrefix`, `withDeclaredUserMessage`, `isSandboxOrigin`,
17+
`isSandboxCrash`, `fiveXxArmDisplacesDeclared4xx`, `structuredCodeAnswer`).

‎.changeset/20951-inline-grid-derived-columns.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
'@objectstack/spec': minor
3+
'@objectstack/lint': patch
4+
---
5+
6+
`deriveInlineGridColumns` (`@objectstack/spec/data`) derives the default columns of an inline master-detail grid, and `field-no-consumers` stops calling two kinds of in-use child field "inert" (#20951).
7+
8+
Clause-②: yes (widening)
9+
10+
- **`@objectstack/spec`.** New exports from `@objectstack/spec/data`: `deriveInlineGridColumns(def, { relationshipField?, exclude?, maxColumns? })`, its element type `DerivedInlineGridColumn`, and `DEFAULT_MAX_INLINE_GRID_COLUMNS` (`6`). The function answers which child fields an inline grid draws when its author listed no columns: a relationship field with `inlineEdit` and no `inlineColumns`, or a `subforms` entry with no `columns`. It returns identity-only entries (`{ name }`, plus `defaultHidden: true` on columns past the visible budget, which collapse into the column chooser and are never dropped), in the child's field order, skipping system, audit, tenancy, ownership and sort-position names, the relationship field, `system` / `readonly` / `hidden` fields and the types a grid cell cannot edit. It is the renderer's current rule, reproduced exactly; the renderer hydrates each column from the child field. No schema accepts anything new or refuses anything new.
11+
- **`@objectstack/lint`.** `field-no-consumers` now reads a `subforms` entry's `amountField` and `relationshipField` against the entry's `childObject`, and keeps `totalField` on the parent. It also credits the columns of a derived inline grid through `deriveInlineGridColumns`. Before, `os validate` warned that the child's summed amount column, the subform's relationship field and every derived grid column were inert, and credited a same-named parent field in the amount column's place. A field the derivation leaves out (for example a `hidden` one) is still reported.

0 commit comments

Comments
 (0)