Repository navigation
Commit d7c0241
Part of #17321
Clause-②: no
A stored record-form `filter` at the converged rule-array doors is now
converted to the `ViewFilterRule` array wherever that mapping is
lossless — a new ADR-0087 D2 conversion,
`page-component-filter-record-to-rule-array` (toMajor 18,
`retiredFromLoadPath: true`). A filter carrying `$and` / `$or` / `$not`
is left byte-identical and is never flattened. `Part of`, not a closing
keyword: ruling item 2's printed TODO list is **not** delivered here. It
needs changes outside `packages/spec`, and this dispatch told me to stop
at that line and report it (see *Stopped half* below).
## The ruling, item by item
Ruling record 5644018752 (director seat, decision batch #121 item 4,
maintainer 「同意」):
| # | Ruled | Here |
| --- | --- | --- |
| 1 | 「A D2 conversion in the
`element-data-source-and-object-block-filter-rule-array` family
covering: flat record `{k: v}` → `[{field, operator: 'equals', value}]`;
operator object `{k: {$op: v}}` → the mapped operator; several keys →
several AND rules; single-level AST tuples where the seat's mapping is
total.」 | **Delivered.** `packages/spec/src/conversions/registry.ts`,
and replayed by id from `step18.conversionIds`. |
| 2 | 「A record carrying `$and` / `$or` / `$not` is passed through
unchanged and reported as a structured TODO naming the page/block and
the combinator — `os migrate meta --stored` prints the list; …」 |
**Pass-through: delivered and pinned. TODO list: stopped** (outside
`packages/spec`, see below); plus every filter of an inline-row
component (contract review 5853094156). |
| 3 | 「The refusal message at the `filter` door gains the "how to fix"
sentence (the separate message-gap card is folded in if it is still
open; otherwise cross-referenced).」 | The message-gap card #17320 has
state `completed`, so it is cross-referenced here. I measured all 12
doors, and each one already gives the "how to fix" sentence. **One gap
was found and fixed.** For a record carrying a combinator, the sentence
rendered the combinator as a field (`{ field: '$or', operator: …, value:
… }`), which does not fix anything. That case is exactly what this
conversion leaves at rest. |
| 4 | 「`Clause-②: no` (no accept-set change; a conversion is added to
the ledger).」 | Held **because** the conversion is
`retiredFromLoadPath`. `normalizeStackInput` (`defineStack` / `os
validate`) never replays it, so an author who writes the record form is
still refused at the door. Pinned in §7 of the new test. |
C (flattening) is excluded, as ruled: 「C converts combinators by
flattening and silently changes what some pages select — excluded on the
fault axis alone.」
## Premises, re-measured on this tree (card measured on `501959b72`)
- **The doors still refuse the record form**: yes. All 12 doors listed
below answer `invalid_type` at the key, with the prescription.
- **The stored read path still replays conversions without validating**:
yes. `conversions/stored.ts` says 「Never throws, never validates」.
`ObjectStackProtocolImplementation.convertStoredItem` is the item read
path: `getMetaItem` for both draft and active rows calls it, and it
replays the full chain, retired entries included.
- **No D2 conversion existed for this shape**: yes. At base `49144fcc`,
`conversions/registry.ts` held 103 `toMajor:` lines and not one mention
of `dataSource`, `defaultFilters` or a rule-array conversion. The family
id appeared only as a semantic entry. At head there are 104.
- **The 「one table」**: the doors fold `operator` through
`normalizeFilterOperator` (`VIEW_FILTER_OPERATOR_ALIASES`). The
conversion introduces no new operator table. A record `$op` must be a
declared `FILTER_OPERATORS` member, and its name without the `$` is
folded through `normalizeFilterOperator` into `VIEW_FILTER_OPERATORS`.
An AST infix spelling with no rule word (`=`, `!=`, `>=` …) is lowered
through `parseFilterAST`, which is the AST's own single sink, and read
back from the `$` operator it produces.
- **Seam (objectui)**: read at the `.objectui-sha` pin `f8a9d0fb0596`.
`toFilterNode` / `mergeFilterNodes`
(`packages/core/src/utils/filter-converter.ts`) already lower all three
shapes, including a spec `ViewFilterRule[]`, so a stored row served as
the rule array renders — except where a block matches the filter against
its OWN inline rows. `object-map`, `object-tree`, `object-calendar` and
`object-gantt` hand `schema.filter` unlowered to an in-memory
`ValueDataSource` (`ObjectMap.tsx:831-833`, `ObjectTree.tsx:835-837`,
`ObjectCalendar.tsx:645-647`, `ObjectGantt.tsx:661,865` via
`resolveDataSource.ts:70`), whose `find` reads an object `$filter` in
the record dialect but an array one as an AST that refuses a rule object
and excludes every row (`ValueDataSource.ts:1100-1101`, `:564`,
`:600-603`, `:70-73`). So every filter of a component whose rows are
inline (data provider `value`, a data array, `staticData`), the binding
included, is left as stored (contract review `5853094156`, rework round
1). The same read **changed the mapping in one place**.
`convertFiltersToAST` **skips a record key whose value is `null`**, so
today that key constrains nothing, whereas an `equals null` rule would
test IS NULL. So a `null` value is declined rather than converted.
Combinators are lowered at that pin (`$and` / `$or`) or refused
(`$not`), so leaving them as stored keeps what the page does today.
## Which doors 「the family」 covers — measured door by door
Each door below refuses `{ status: 'active' }` with `invalid_type` at
the key and the rule-array prescription, including the 「Write one rule
per record key」 sentence. The conversion reaches exactly these doors. §6
of the test derives this set from `ComponentPropsMap` and
`ElementDataSourceSchema` and holds the conversion's reach equal to it.
| Door | Migration entry named by the refusal |
| --- | --- |
| `dataSource.filter` (any page component, `ElementDataSourceSchema`) |
`element-data-source-and-object-block-filter-rule-array` |
| `object-grid` / `object-metric` / `object-kanban` / `object-calendar`
/ `object-map` / `object-gantt` / `object-tree` / `object-timeline`
`.properties.filter` |
`element-data-source-and-object-block-filter-rule-array` |
| `element:number.properties.filter` |
`element-number-filter-rule-array` |
| `element:record_picker.properties.filter` |
`element-record-picker-filter-rule-array` |
| `object-grid.properties.defaultFilters` |
`object-grid-default-filters-rule-array` |
The `filter` of any other component type is not touched (pinned).
## What converts, and what is left exactly as stored
Converts: a flat record, where each scalar value becomes an `equals`
rule; an operator object, where each `$op` becomes the mapped operator
and several operators on one field become several rules; several keys,
which AND; a single-level AST, meaning one comparison or a flat list of
comparisons; and `{}`, which becomes `[]`. Values are carried verbatim.
Every produced rule must parse against `ViewFilterRuleSchema`, which is
the door's own check, so a converted row re-saves cleanly.
Left byte-identical (identity by reference, no notice):
- `$and` / `$or` / `$not`, or any other top-level `$` key.
- An AST `and` / `or` group, or a flat list that nests one.
- `$null` and `$exists`, whose meaning lives in the value and so is not
in the one table.
- `like` / `ilike`.
- A `null` value.
- An array or object comparand in equality position, a field reference,
an empty operator object.
- Any rule the door would refuse, such as `in` with a scalar or an empty
`icontains`.
- Every filter, the binding's `dataSource.filter` included, of a
component whose rows are inline (`properties.data` with provider
`value`, a bare `properties.data` array, or a truthy
`properties.staticData`). The same filter on an object-bound block
converts.
Conversion is all-or-nothing per filter, because converting the mappable
keys and dropping the rest would widen an AND list.
**Lossless is measured, not asserted** (§4). For every
`FILTER_OPERATORS` member, the mapped rule lowered through
`parseFilterAST` equals the source FilterCondition. Only `$null` and
`$exists` are declined. For every `VALID_AST_OPERATORS` spelling, the
mapped rule lowers exactly as the source comparison did. Only `like` and
`ilike` are declined.
## Changed published text
**Refusal at every rule-array `filter` door, for a record carrying a
combinator.** For every other value the text is byte-identical to base.
I measured this by loading base `49144fcc`'s `filter-rule-array.ts` next
to head's and comparing the full issue text (`code|path|message`) for 13
inputs: identical for 11 inputs (flat, `{}`, null value, operator
object, more than 3 keys, array value, `$text`, string, number, AST
array, `Date`), changed for the 2 combinator inputs. New text for `{
$or: [{ a: 1 }] }` at the binding, read from the built CJS `dist/ui`:
```text
`filter` on this element data source takes the ViewFilterRule ARRAY form `[{ field, operator, value }, ...]`, and this value is the MongoDB-style record form this door took before the one-filter-orthography convergence. `$or` is a combinator, not a field, and the rule array has no spelling for it: its rules only AND, so `$or` and `$not` cannot be written as rules at all, and `$and` only as the separate rules it joins. Nothing rewrites a combinator for you — flattening one changes which rows the filter selects — so decide which rows this filter should select, and write the rules that select exactly those. Legacy operator shorthands (`eq`, `gt`, `notIn`, …) are accepted and normalized on parse. Full conversion table: migration `element-data-source-and-object-block-filter-rule-array`.
```
For a record that has field keys beside the combinator, the rewrite
sentence becomes 「Write one rule per field key — they AND — so its field
keys become `[…]`.」 and the combinators are named in the plural: 「`$and`
and `$not` are combinators, not fields, …」. The code (`invalid_type`)
and path (`filter`) do not change. Both cases are pinned in
`filter-rule-array-guidance.test.ts` §1 (code + path + text).
**ADR-0087 semantic entries whose prose this PR made false** (published
in the upgrade guide):
- `element-data-source-and-object-block-filter-rule-array`, `reason`.
「⚠️ Metadata AT REST is deliberately NOT rewritten, and this disposition
adds no D2 conversion … so it has nothing to rewrite for this shape …」
is replaced with a paragraph that names the conversion and what it
converts. It also names what it leaves as stored and why. It says in so
many words that 「`os migrate meta --stored` does not list these rows
yet: a row the conversion leaves as stored reports there as already on
protocol.」
- `object-grid-default-filters-rule-array`, `reason`. 「Metadata AT REST
is deliberately NOT rewritten and this entry adds no D2 conversion …」 is
replaced to match. The two sibling entries (`element-number-…`,
`element-record-picker-…`) made no at-rest claim and are unchanged.
- The hand-written `step18.rationale` gains one closing sentence naming
the conversion. The generated regions of `migrations/registry.ts` were
regenerated with `gen:migration-registry`, not hand-edited.
## Stopped half — ruling item 2's structured TODO list (exact sites)
Measured: the existing D2 machinery does **not** report pass-through
rows. Ruled item 2's list needs all four of the following, and three of
them are outside `packages/spec`:
1. `packages/spec/src/conversions/types.ts` + `apply.ts` + `index.ts`.
There is no TODO channel. `ConversionContext` offers only
`reportConflict`, and `ApplyConversionsOptions` only `onNotice` and
`onConflict`. A TODO sink would be a new notice with a new stable code,
next to `OS_METADATA_CONVERTED` / `OS_METADATA_CONVERSION_CONFLICT`.
2. `packages/runtime/src/dispatcher-error-vocabulary.ts`. `pnpm
check:dispatcher-error-vocabulary` reds a new `code:` stamp in
`apply.ts` until a `foreign-vocabulary` row is added. Its two twins are
the `OS_METADATA_CONVERTED` and `OS_METADATA_CONVERSION_CONFLICT` rows.
**This is why the spec half was not landed alone**: it would turn a gate
red outside this lane.
3. `packages/metadata-protocol/src/protocol.ts`.
`convertStoredItemDetailed` forwards only `onNotice` to
`applyConversionsToStoredItem`. `migrateStoredMetadata` decides with
`changed = … notices.length > 0`, so a row that emitted no notice is
recorded `canonical` and is never itemised.
4. `packages/metadata-protocol/src/stored-migration.ts`.
`StoredMigrationRow` has no TODO field, and
`formatStoredMigrationReport` has no TODO section.
`packages/cli/src/commands/migrate/meta.ts` `runStored` prints
`formatStoredMigrationReport` and emits the whole report as JSON, so it
needs no change once the report carries the list.
Until that lands, a combinator row keeps loading unchanged, is counted
canonical by `os migrate meta --stored`, and is refused at its door on
its next save, now with the corrected prescription. The entry prose and
the changeset say exactly this.
## Verification — all at head `a0360aad` (rework round 1; round 0 was
`b1c9143f`)
- `pnpm --filter @objectstack/spec build` passed (dts 36/36).
- `check:generated`: all 15 generated artifacts up to date.
`spec-changes.json` and `docs/protocol-upgrade-guide.md` do not move,
because protocol 18 is the major in preparation (`PROTOCOL_VERSION`
17.0.0).
- `pnpm --filter @objectstack/spec typecheck` passed, including
`check:test-typecheck` with 142 pinned signatures and no new debt.
- Full `@objectstack/spec` suite (`vitest run --project local`): **541
files, 15871 passed, 2 todo**.
- New `conversions/page-component-filter-record-to-rule-array.test.ts`:
57 tests, §0–§7 (round 1 adds 6 inline-row cases, the inline-grid
`defaultFilters` case and the object-bound control). The guidance test
gains 2 combinator pins.
- **Ablation** (`scripts/ablation-replace.mjs`, wrap mode, committed
head): I deleted the door-acceptance gate (`return rules.every((rule) =>
ViewFilterRuleSchema.safeParse(rule).success) ? rules : undefined;`
became `return rules;`), with anchor 1→0 and blob `17b8a6c1` →
`188a52b8`. Expected direction: red. Observed: 3 red, namely a
field-reference comparand, `in` with a scalar, and an empty `icontains`.
The restore left blob == HEAD `17b8a6c1` and `git diff HEAD` empty. The
fourth door-refused row, an AST scalar comparison with no value, stayed
green, because `parseFilterAST` already declines it one step earlier.
- **Second ablation (round 1)**: the inline-rows decline removed (`if
(rendersInlineRows(component.properties)) return component;` → a
comment), anchor 1→0, blob `7de8807d` → `9d28b85b`. Expected direction:
red. Observed: 9 red (the 6 inline rows, the inline-grid
`defaultFilters` row, the replay-no-op pin and the registry fixture
pair); the object-bound control stayed green. The restore left blob ==
HEAD `7de8807d` and `git diff HEAD` empty.
- Consumer suites: `@objectstack/metadata-core`
`artifact-forward-conversion.test.ts` 25/25. `@objectstack/lint`
`validate-component-props` + `validate-preset-comparands` +
`authoring-rule-input-tier` 107/107. No test outside `packages/spec`
seeds a page row with a record-form filter at these doors (swept every
consumer test of `migrateStoredMetadata` /
`applyConversionsToStoredItem`), and none pins the conversion list.
- CJS and ESM smoke of the built `dist`:
`applyConversionsToStoredItem('page', …)` converts in both.
- Gates: `dispatch-gates.mjs --commands` re-derived at this head found
87 families. **84 exit 0; 3 exit 3 (prerequisite not met, NOT
MEASURED)**: `check:dual-build-cjs-loads` and `check:type-check-debt`
need a full `pnpm build`, and `check:lean-entry-closure` needs the
`objectql` dist. `--ran` reconciliation: 87 accounted for, 84 run, 3 NOT
MEASURED, 0 unrun. The roster gates whose roster sits under my paths
were also run, all exit 0: `check:filter-alias-parity`,
`check:error-code-casing`, `check:authz-resolver`, spec
`check:meta-url-spelling`, spec `check:error-code-provenance`, and
`check:nul-bytes`.
- Lint, narrowed and shown to exclude nothing (the repo-wide `pnpm lint`
is CI's): `eslint --no-inline-config --format json` over the 7 changed
TS files reported 7 files, 0 errors, 0 warnings. `eslint.config.mjs`
matches them under `packages/**/*.{ts,…}`. It has no
`parserOptions.project` and no typed rules, so this diff cannot move a
verdict in any untouched file.
- `main` was merged in before these runs (it had moved
`packages/spec/src/migrations/registry.ts` and `ui/report.zod.ts`). The
merge was clean, and `check:migration-registry` passed after it. Main
has moved again since (`e2c4e125`); the newer commits touch none of
these 8 files.
## Acceptance notes
- **Settled choices, each on the four axes.**
- *Several operators on one field* (`{ $gte: 1, $lte: 9 }`) convert to
one rule per operator. It is the same AND as several keys, so it is
lossless. The alternative, leaving such a filter silent and unconverted,
is worse on the fault axis.
- *`null` values are declined.* Converting them would silently change
what the page selects, per the pin read above.
- *All-or-nothing per filter.* A partial conversion widens the filter.
- *The family is every door the prescription helper serves.* The helper
recognises the same record form, the four entries carry the same table,
and the doors that joined later (`object-map` / `-gantt` / `-tree` /
`-timeline`) name the same migration.
- `.changeset/19514-…` (pending, another card's) says for
`object-grid.defaultFilters` 「Stored metadata … keeps loading and keeps
rendering exactly as it does today; what changes is that RE-SAVING it is
refused」. After this PR that sentence is still true for what the
conversion leaves as stored, but not for the converted forms. This PR's
changeset states the new behaviour and ships in the same release. Not
edited here (carrier: none).
- `$null` / `$exists` operator objects stay unconverted, because the
one-table rule excludes value-dependent operators. They are listed by
nothing until the stopped half lands.
- **objectui carrier**: `ValueDataSource` refuses a canonical
`ViewFilterRule` array, so a spec-conformant filter on an inline-row
map, tree, calendar or gantt already renders nothing today, independent
of this PR (measured by probe at the pin, round 1 report on #17321). The
seat files it in objectui; the inline-rows decline here can be dropped
once it lands.
- The `os migrate meta --from 17` sentence needs no `--to 18`: the CLI's
default terminus is the highest major with a step
(`packages/cli/src/commands/migrate/meta.ts:88`, `:393`; pinned by
`packages/cli/test/migrate-meta-default-range.test.ts`).
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent 8ecbe0f commit d7c0241
8 files changed
Lines changed: 1209 additions & 38 deletions
File tree
- .changeset
- packages/spec/src
- conversions
- migrations
- entries/semantic
- ui
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
Large diffs are not rendered by default.
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
69 | 83 | | |
70 | 84 | | |
71 | 85 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
58 | 62 | | |
59 | 63 | | |
60 | 64 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5279 | 5279 | | |
5280 | 5280 | | |
5281 | 5281 | | |
5282 | | - | |
| 5282 | + | |
| 5283 | + | |
| 5284 | + | |
| 5285 | + | |
| 5286 | + | |
| 5287 | + | |
| 5288 | + | |
| 5289 | + | |
| 5290 | + | |
| 5291 | + | |
| 5292 | + | |
| 5293 | + | |
5283 | 5294 | | |
5284 | 5295 | | |
5285 | 5296 | | |
| |||
5315 | 5326 | | |
5316 | 5327 | | |
5317 | 5328 | | |
| 5329 | + | |
5318 | 5330 | | |
5319 | 5331 | | |
5320 | 5332 | | |
| |||
7863 | 7875 | | |
7864 | 7876 | | |
7865 | 7877 | | |
7866 | | - | |
7867 | | - | |
7868 | | - | |
7869 | | - | |
7870 | | - | |
7871 | | - | |
7872 | | - | |
7873 | | - | |
7874 | | - | |
| 7878 | + | |
| 7879 | + | |
| 7880 | + | |
| 7881 | + | |
| 7882 | + | |
| 7883 | + | |
| 7884 | + | |
| 7885 | + | |
| 7886 | + | |
| 7887 | + | |
| 7888 | + | |
| 7889 | + | |
| 7890 | + | |
| 7891 | + | |
| 7892 | + | |
| 7893 | + | |
| 7894 | + | |
| 7895 | + | |
| 7896 | + | |
| 7897 | + | |
| 7898 | + | |
| 7899 | + | |
| 7900 | + | |
7875 | 7901 | | |
7876 | 7902 | | |
7877 | 7903 | | |
| |||
10900 | 10926 | | |
10901 | 10927 | | |
10902 | 10928 | | |
10903 | | - | |
10904 | | - | |
10905 | | - | |
10906 | | - | |
10907 | | - | |
10908 | | - | |
10909 | | - | |
| 10929 | + | |
| 10930 | + | |
| 10931 | + | |
| 10932 | + | |
| 10933 | + | |
| 10934 | + | |
| 10935 | + | |
| 10936 | + | |
| 10937 | + | |
| 10938 | + | |
| 10939 | + | |
10910 | 10940 | | |
10911 | 10941 | | |
10912 | 10942 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
160 | 160 | | |
161 | 161 | | |
162 | 162 | | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
163 | 203 | | |
164 | 204 | | |
165 | 205 | | |
| |||
0 commit comments