Skip to content

Commit 170fd83

Browse files
os-billclaude
andauthored
feat(spec)!: publish the $-prefix key ban the normalized filter enforces, and make the ratchet able to see it (#19335)
⛔ **PARKED — 本 head 落不了地,且挡住它的不是本 PR。** 卡 #18670 已转 `pm:blocked`,门禁卡是 **#19240**(认领读者 `claimRetractions` 只认**同一 login** 的 `Release:`,`SKILL.md` :496 的死认领回收写不进它)。本 PR 的落地前置 ① 与 ③ 成立(达档 `## Contract review` 记录 `5749728565` 在 head `1dfe2f40bc` 上;checks 全绿);② 不成立:`check-clause2-carriers.mjs --pair 19335` = **exit 4**,唯一 ✗ 行是 C9(本卡线程上两条他席认领仍 LIVE)。完整读数、对照与本席自纠见 #18670 评论 `5752999363`。⛔ 保持 draft,⛔ 不挂 auto-merge。 Part of #18670 — item 2, the **fifth** arm the batch #193 ruling added to the closed projection list, plus that ruling's **second acceptance item**. This body carries no closing keyword for that number on purpose: 566 dropped refinement sites remain across 205 published schemas, and whether the card closes is the seat's call rather than this PR's. Clause-②: yes **Carrier:** the published artefact `packages/spec/json-schema/data/NormalizedFilter.json`. The published JSON Schema **narrows** toward what the runtime already refuses, and no document the runtime accepts becomes refused. Director ruling `5749025303`, batch #193 item 3, letter **A**, maintainer 「其他同意」 2026-09-20T09:44Z: 「A **fifth arm** joins the closed projection list: `propertyNames: { not: { pattern } }`, scoped to that one site and to the `^\$` ban, under the same one-ledger-row-at-a-time discipline as the four landed arms; the published keyword and the enforced predicate are built from a **single source** so they cannot name different things; an ablation proves the pin (the emitter removed ⇒ the rows return).」 Base `f93beea0a6`; head after merging `origin/main` (`e3b3cdd2df`) through `scripts/pm/os-regen-merge.sh`: **`1dfe2f40bc`**. --- ## 1. The measurement that decided step 1 — and it came out YES The ruling put one measurement **before** the arm: can those three `NormalizedFilter.json` nodes hold a ledger row at all? They read `undecidable`, and the thread's worry was that closing the rule would buy a narrower file with **no testable row** — the opposite trade from every arm landed so far. ⛔ It is not a grep question, and the card's own instruction says so: `packages/spec/json-schema/**` is **0 tracked files** on `origin/main` (lit control, same instrument: `packages/spec/src/data/` reads **167 tracked**), because `.gitignore:63` ignores it. Every reading below is against a tree **generated by the repo's own tooling** — `pnpm --filter @objectstack/spec build`, whose first step is `gen:schema` (`OS_EAGER_SCHEMAS=1 tsx scripts/build-schemas.ts`). **The answer: a row CAN be held, and the reason it was not is a defect in the detector.** The generator publishes `NormalizedFilter` through its **THIRD** projection attempt — `projectByPruningUnionBranches`, which drops the `z.date()` union branches and publishes the rest. The detector's `projectOrNull` stopped at the two strict rungs. So it was asking what a projection **nobody publishes** says, and answering `undecidable`: | node | plain output rung | plain input rung | branch-pruning rung | differential under it | |:---|:---|:---|:---|:---| | `lazy.$and.element.options[0]` | throws | throws | ok, 16772 bytes | **identical ⇒ `dropped`** | | `lazy.$or.element.options[0]` | throws | throws | ok, 16772 bytes | **identical ⇒ `dropped`** | | `lazy.$not.options[0]` | throws | throws | ok, 16772 bytes | **identical ⇒ `dropped`** | ⇒ the ruling's **first** branch applies: the detector **judges** those three nodes. The `undecidable` row shape was its fallback 「if a row cannot be held」, and that antecedent is false, so ⛔ no unread ledger field was added for an empty population. What the hole got instead is §2. ## 2. Second acceptance item — the blind spot, measured to zero and then pinned there `projectOrNull` now carries the generator's third rung and reports **which rung answered**, so a differential can never compare a pruned projection with an unpruned one (nothing observed reaches that guard; it is written down so the day it stops holding reads `undecidable` and is counted, rather than reading `projected` and vanishing). Repo-wide effect, from the generator's own census line: | | published schemas | dropped sites | projected | **undecidable** | |:---|---:|---:|---:|---:| | base `f93beea0a6` | 204 | 560 | 357 | **9** | | + the ladder rung | 205 | 569 | 357 | **0** | | + the arm (this PR) | 205 | 566 | 360 | **0** | ⚠️ **The ledger GREW before it shrank, and the growth is the whole point of the item.** Seven sites became countable that no ratchet could see — `data/FieldOperators` and `data/NormalizedFilter` each gained their `$between` pair, and `data/RangeOperator` entered the ledger at all, a **published** schema that had been holding **zero** entries. Then the arm deleted three. Net: 204 entries / 560 sites → **205 / 566**. And a published site that still cannot be adjudicated now **fails the build by name**, printing the paths and the two legitimate remedies (teach the ladder a rung the generator has; or take the decision to give the ledger an `undecidable` row shape). ⛔ The hole cannot reopen in silence. ## 3. The arm, and the single source `banned-key-pattern` — 「no document may carry a key matching this pattern」 — emitted as `propertyNames` with a `not` over a `pattern`. A `$`-prefix ban is an **open** key set, so the existing `banned-keys` arm cannot express it: a finite list that merely sampled the set would be wider than the rule, which the closed list forbids by construction. **Single source, asserted rather than argued.** `bannedKeyPattern` compiles its regular expression **from** the declared pattern string, so the keyword the file publishes and the rule the runtime enforces are one string read twice. A test reads the emitted `pattern` off the published artefact and the declaration off the predicate and compares them — an emitter that re-spelled the rule, or a declaration edited without its predicate, fails there rather than drifting. **Exact, not approximate.** A JSON object's properties are exactly its own enumerable string-keyed ones, and `propertyNames` judges exactly those names. JSON Schema specifies `pattern` as an ECMA-262 regular expression evaluated as a SEARCH — unanchored, "does a match occur anywhere" — which is `RegExp.prototype.test` and nothing else. So `^\$` and the hand-written `key.startsWith('$')` it replaces name one set, pinned over a key corpus. It is presence and never value: a matching key present with a `null` value is present to both. **Scoped mechanically, which is how the ③ objection is answered.** The standing objection to a regex-shaped arm is that its over-reach cannot be read off the declaration the way a key list's can. The bound is a **second closed list**: `BannedKeyPattern` is a union of the pattern strings this package publishes, exactly one today, so a call site cannot invent a regex — there is no plain string type to pass, and widening it is the same reviewed decision that adding an arm is. The compiler refuses the second pattern; it does not arrive by a call site's choice. ⛔ No flags on the regular expression, and that is part of the equality rather than a style choice: a JSON Schema `pattern` has none to carry, and the global flag would make `test` stateful through `lastIndex`, so a key's verdict would depend on which keys were judged before it. Pinned both ways. ⛔ The predicate reads OWN enumerable keys and never the `in` operator — pinned with a name planted on the prototype, where the two readings actually come apart. ## 4. The card's own class, before and after — measured with a real validator ajv 8 (draft 2020-12) compiled against the **generated** `data/NormalizedFilter.json` on each side: | document | ajv BEFORE | ajv AFTER | |:---|:---|:---| | `{}` | true | true | | `{"$and":[{"amount":{"$eq":1}}]}` | true | true | | `{"$and":[]}` | true | true | | `{"$and":[{"$and":[]}]}` | true | true | | `{"$or":[{}]}` | true | true | | `{"$not":{}}` | true | true | | `{"$not":{"amount":{"$eq":1}}}` | true | true | | `{"$and":[{"$bogus":{"$eq":1}}]}` | **true** | **false** | | `{"$or":[{"$bogus":{"$eq":1}}]}` | **true** | **false** | | `{"$not":{"$bogus":{"$eq":1}}}` | **true** | **false** | The three that move are refused by the runtime, which names the rule: 「a field condition's keys are field names, never `$`-prefixed operators」. ⇒ the validator stops answering PASS on metadata the platform refuses, and **nothing the runtime accepts became refused** — the empty combinators and the nested group members are the direction that would have broken had the ban landed on the union instead of on the field-condition branch, and they are pinned. All three published nodes now carry the rule, conjoined and never substituted (a record states `propertyNames: { type: 'string' }` of its own, and replacing it would trade a key-TYPE rule for a key-NAME rule — a narrowing bought with a widening): ```json { "type": "object", "propertyNames": { "type": "string" }, "additionalProperties": { "...": "the operator map" }, "allOf": [ { "propertyNames": { "not": { "pattern": "^\\$" } } } ] } ``` ## 5. Blast radius — the whole published tree The six source files were reverted to the base, the generator re-run, and the two trees compared byte for byte. **Revert leg proven on disk:** each path's blob hash equalled its base blob before anything ran. **Restore leg proven by bytes:** `git diff HEAD` printed **0 bytes**, `git status --porcelain` printed nothing, and each path's blob hash equalled its HEAD blob. | reading | value | |:---|:---| | files common to both trees | 1535 | | **byte-identical** | **1530** | | moved | **5** | The five, by name: `data/NormalizedFilter.json` (gains the ban at three nodes; gains the two `$between` annotation rows the ladder made visible), `data/FieldOperators.json` and `data/RangeOperator.json` (**annotation only** — they gain `x-dropped-refinements` rows, and `x-` keywords are ignored by every validator, so the set of documents they accept is unchanged), `objectstack.json` (the bundle; its 29 differing leaf paths sit under exactly those three definitions and nowhere else), and `.build-input-hash-schema`. ⭐ **`openapi.json` measured separately and with the right instrument.** `gen:schema` never writes it, so comparing it inside the sweep above would have read two copies of the same stale file and reported a false identical. `gen:openapi` was run on both trees: sha256 `34b1dc9c2cf103144fc0a174d4bc901836fd1f89d1d1a71c0aa36e2bfbeeebaa` on **both** sides — this arm reaches no schema that surface publishes. ## 6. Ablation — the pin can fail, and the rows do return `scripts/ablation-replace.mjs` replaced the one line dispatching the arm, with the mutation verified against the disk: anchor **1 → 0**, marker **0 → 1**, blob `4c5881bf5d1f` → `92da85bc6406`. ⭐ Resolution stated, because a false green here points the wrong way: every consumer reaches this module by a **relative** specifier, which resolves to source and never through the package `exports` to `dist`. There is no built artefact between the mutation and the verdict, so no dist preflight applies. | leg | result | |:---|:---| | `refinement-projection.test.ts` | **exit 1** — 12 failed / 66 passed, the single-source pin and the live seam among them | | `gen:schema` | **exit 1** — naming all three rows returning by name: `lazy.$and.element.options[0]`, `lazy.$not.options[0]`, `lazy.$or.element.options[0]` | | **restore** | blob back to `4c5881bf5d1f` **==** HEAD, `git diff HEAD` **0 bytes**, anchor back to 1 and marker back to 0 | The second leg is the ruling's own requirement: 「the emitter removed ⇒ the rows return」. They do — and they exist to return **only because** §2 made those nodes countable first. Regenerated afterwards, `data/NormalizedFilter.json` came back to sha256 `80041a0b…`, byte-identical to the pre-ablation artefact. ## 7. Verification — real exit codes, each captured before any pipe | check | exit | |:---|:---| | `pnpm --filter @objectstack/spec build` | **0** | | `pnpm --filter @objectstack/spec typecheck` | **0** | | `pnpm --filter @objectstack/spec test` | **0** — 500 test files / 14663 tests, all passed, dist built | | `pnpm --filter @objectstack/spec gen:schema` | **0** — ledger balanced | | `pnpm --filter @objectstack/spec gen:openapi` | **0** — byte-identical to base | | `pnpm --filter @objectstack/spec check:generated` | **0** — 16 of 16 generated artefacts up to date | | `pnpm lint` | **0** — the whole repository, `eslint . --no-inline-config`, not a narrowed subset | | derived gate families, reconciled by `scripts/pm/dispatch-gates.mjs --ran` | **86 derived / 82 exit 0 / 4 NOT MEASURED / 0 UNRUN** | The four NOT MEASURED each exit **3** — `PREREQUISITE NOT MET`, a code that is explicitly neither pass nor failure — because each needs a whole-repo build closure that CI produces: `check:doc-formula-expressions`, `check:dual-build-cjs-loads`, `check:lean-entry-closure`, `check:type-check-debt`. ⛔ Declared, not skipped. ⭐ **`api-surface-declarations/` moved, and the movement is order-only — but it IS mine.** `check:api-surface` (the name-level gate) stays green with no diff at all. The declaration-text artefact did move, and rather than assume, it was tested: with this branch's six source files reverted to the base and the package rebuilt, `check:api-surface-declarations` exits **0** — so the movement belongs here. Characterised by bytes: 10 changed lines, 9 of them a whole-line multiset identity (two enum members swapping places), and the tenth a union whose quoted tokens are the same set, the same count, and whose text is identical once the tokens are masked. ⇒ **no declaration added, removed, or changed in meaning.** Regenerated and committed as its own commit. ## 8. Merge hygiene `origin/main` was merged in through `scripts/pm/os-regen-merge.sh` — ⛔ never rebased, ⛔ never force-pushed. That path was taken because `git check-attr merge` reads **`os-regen`** on `packages/spec/api-surface-declarations/api.txt` and `system.txt`, per file rather than by counting `.gitattributes` rows. After the merge the implementation body was re-asserted by name (`bannedKeyPattern`, `OPERATOR_PREFIX_KEY_PATTERN`, `BannedKeyPattern`, `emitBannedKeyPattern`, `conjoinPropertyNames`, `undecidableEntries`), the whole chain was regenerated, and `check:generated` reported 16 of 16 current with **no** regeneration diff. ## Acceptance notes - ⚠️ **A dispatch instruction that the repository contradicts, named rather than quietly resolved.** The dispatch said to regenerate `packages/spec/dropped-refinements.baseline.json` 「with the repo's tooling; never hand-edit it」. There is no such tooling: the ledger has no `gen:` script by design, `build-schemas.ts` calls it 「a committed, hand-edited ledger」 in its own refusal text, and the module docblock argues the point at length — a generator would let a new gap be admitted by running a command instead of by a decision. The operative half of the ruling — 「⛔ do not serialise on it」 — was followed: this PR did not wait on #19147. Every ledger edit here is the **corrected entry the gate itself printed**, pasted verbatim, which is the closest thing to tooling the artefact has. - **Noted, not filed — the sibling changeset in this same release now contradicts the tree.** `.changeset/18670-project-banned-keys.md` records that the `$`-prefix sites 「stay unprojected … carry NO annotation and hold NO ledger row: published yet unratcheted」. True of its own tree, false of this one. ⛔ Not rewritten — a landed record of what that PR shipped — so this PR's changeset states the supersession instead, and the two read coherently as one CHANGELOG. Carrier: none needed; both entries publish together. - **Noted, not filed — and this PR IS the carrier the previous one named.** #19137 named 「the next PR that edits `packages/spec/scripts/build-schemas.ts`」 as carrier for a stale mention of the retired `api-surface-signatures.json`. This PR does edit that file, so it inherits the hand-off, and it is being declined deliberately: the line is a documentation nit in a comment, not one of the three filing classes, and it is not this ruling's defect class. It survives at `packages/spec/scripts/build-schemas.ts:874`. Carrier: the next PR that edits that file for a reason of its own. - **`dropped-refinements.baseline.json` is a shared hot file** held by #19147. Not serialised on, per the ruling; collisions resolve by regenerating through `scripts/pm/os-regen-merge.sh`, ⛔ never by hand-editing conflict markers. - The arm list's own roster pin and the new pattern-set pin are both asserted as exact equalities, so a sixth arm — or a second pattern — updates a reviewed line in a diff rather than widening the narrowing quietly. --- _Generated by [Claude Code](https://claude.ai/code/session_01JbZnqu8bt6YqfJsr9vaFb3)_ --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent e37ea4d commit 170fd83

8 files changed

Lines changed: 622 additions & 18 deletions
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
"@objectstack/spec": minor
3+
---
4+
5+
**BREAKING (published artifact narrows)** — `packages/spec/json-schema/**` now states the `$`-prefix key ban a normalized field condition enforces, so a validator reading the published files stops answering PASS on `{"$and":[{"$bogus":{"$eq":1}}]}` at `data/NormalizedFilter` — a document the runtime refuses by name (#18670 item 2, the fifth arm).
6+
7+
Clause-②: yes (narrowing)
8+
9+
One named pattern joins the closed list, and only one:
10+
11+
- **`banned-key-pattern` — "no document may carry a key matching this pattern"**, emitted as `propertyNames` with a `not` over a `pattern`. `NormalizedFilter`'s `$and` / `$or` members and its `$not` operand each admit a field condition whose keys are field names (`amount`, `account.name`) and never `$`-prefixed operators. The published file now says so at all three nodes.
12+
13+
**Scoped, and the scope is mechanical.** The ban is over an OPEN set of names, which is why the existing `banned-keys` arm cannot express it — a finite list that merely sampled the set would be wider than the rule. The pattern arm that can express it is bounded by a second closed list: `BannedKeyPattern` is a union of the pattern strings this package publishes, exactly one today (`^\$`), so a call site cannot invent a regex because there is no `string` to pass, and widening it is the same reviewed decision that adding an arm is. That is what answers the standing objection to a regex-shaped declaration — its over-reach cannot be read off the declaration the way a key list's can, so the bound is on how few declarations exist rather than on trusting the next caller.
14+
15+
**The rows retired, by name.** `packages/spec/dropped-refinements.baseline.json`, entry `data/NormalizedFilter`:
16+
17+
| row | before | after |
18+
|:---|:---|:---|
19+
| `lazy.$and.element.options[0]` | dropped | **deleted** — reads `projected`, arm `banned-key-pattern` |
20+
| `lazy.$or.element.options[0]` | dropped | **deleted** — reads `projected`, arm `banned-key-pattern` |
21+
| `lazy.$not.options[0]` | dropped | **deleted** — reads `projected`, arm `banned-key-pattern` |
22+
23+
**⛔ Not a behaviour change, and no document the runtime accepts becomes refused.** The arm is EXACT rather than approximate. A JSON object's properties are exactly its own enumerable string-keyed ones and `propertyNames` judges exactly those names; JSON Schema specifies `pattern` as an ECMA-262 regular expression evaluated as a SEARCH, which is `RegExp.prototype.test` and nothing else — so the same source text decides the same set of names on both sides. It is presence and never value: a matching key present with a `null` value is present to both. Measured with ajv 8 (draft 2020-12) on the generated file, the verdict vector moves in one direction only: the three `$`-prefixed specimens go `true` → `false`, and every document the runtime accepts — `{}`, the empty combinators `{"$and":[]}` / `{"$or":[{}]}` / `{"$not":{}}`, a nested group, an ordinary field condition — is accepted before and after. Across the published tree, **1530 of 1535 files are byte-identical**; one file changes what it accepts, two change annotation only, and the remaining two are the bundle and the build-input hash.
24+
25+
**The predicate and the keyword are ONE string.** `bannedKeyPattern` compiles its `RegExp` from the declared pattern, so the keyword the file publishes and the rule the runtime enforces cannot come to mean different things — the construction `requiredOneOf`, `dependentRequired` and `bannedKeys` already use, and the reason this arm needs no drift pin either. The `RegExp` carries no flags, which is part of the equality rather than a style choice: a JSON Schema `pattern` has none to carry, and `g` would make `test` stateful through `lastIndex` so a key's verdict would depend on which keys were judged before it.
26+
27+
**A ratchet repair ships with it, and it is what made the rows exist to delete.** The detector decided `dropped` vs `projected` on a two-rung projection ladder while the generator publishes on a three-rung one — a node whose every io direction refuses over an unrepresentable member still reaches its file when that member sits in a union position, because the emit loop drops the branch and publishes the rest. Nine PUBLISHED sites therefore read `undecidable`, the one verdict the ledger does not count: they held no row, carried no `x-dropped-refinements`, and no repair of them could ever have deleted a row. The three nodes this arm closes were three of the nine. The detector now carries the generator's third rung and reports which rung answered, so a differential can never compare a pruned projection with an unpruned one; and a published site that still cannot be adjudicated fails the build by name, so the blind spot cannot reopen in silence.
28+
29+
⚠️ **The ledger therefore GREW before it shrank, and the growth is the point.** Six sites became visible that were previously uncounted — `data/FieldOperators` and `data/RangeOperator` gained `$between.items[0]` / `[1]`, `data/NormalizedFilter` gained the same pair under `$not`, and `data/RangeOperator` entered the ledger as a published schema that had been holding no entry at all — then this arm deleted three. Net across the change: **204 entries / 560 sites → 205 / 566**, with the census at **566 dropped / 205 published schemas / 360 projected** (224 `non-blank-string`, 129 `required-one-of`, 3 `banned-key-pattern`, 2 `dependent-required`, 2 `banned-keys`) and **0 undecidable**, down from 9. Those two files gain annotation only: `x-` keywords are ignored by every validator, so the set of documents they accept is unchanged.
30+
31+
⭐ **Superseding a sibling entry in this same release.** `18670-project-banned-keys.md` records that the `$`-prefix sites 「stay unprojected … they carry NO annotation and hold NO ledger row: published yet unratcheted」. That was a correct reading of its own tree and is no longer true of this one: the sites are projected, the blind spot is closed, and the population it described is empty. The earlier entry is left as the record of what it landed.
32+
33+
<!-- adr-0087: not-required (no-migration-prescription) Nothing an author can write is removed, renamed or re-spelled: no spec key, no export and no config field changes, and the accepted set of metadata documents is byte-for-byte what it was. What changed is a machine-readable DECLARATION catching up with the runtime it always described, so there is nothing for `objectstack migrate meta` to rewrite and no stored representation to convert. -->

‎packages/spec/dropped-refinements.baseline.json‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,10 @@
22
"description": "Shrink-only ledger of every PUBLISHED JSON Schema that is STILL WIDER than the Zod type it was generated from, because a rule written as `.refine()` reaches the runtime and not the file (#18670). `z.toJSONSchema()` has no arm for a `custom` check: a plain record, the same record with a `.refine()`, and the same record with an ABORTING `.refine()` all project byte-identically (measured on zod 4.4.3, the version packages/spec resolves). So a document one of these files ACCEPTS can still be refused at parse time, and an author -- or an AI -- validating against packages/spec/json-schema/** finds out a release later. Each `sites` path is a position under that schema at which a refinement is dropped; the same paths are written onto the artifact itself as `x-dropped-refinements`. Item 2 closed the first patterns: a refinement DECLARED through the closed list in src/shared/refinement-projection.ts is emitted into the published file, reads `projected` rather than `dropped`, and its row LEAVES this ledger in the same PR -- which is why the ledger shrinks and never grows on a repair. Every refinement outside that closed list stays here, and adding an arm to the list is a public-contract decision, not a refactor. Hand-edited on purpose and with no `gen:` script: a generator would let a new gap be admitted by running a command instead of by a decision, which is the silence this ledger exists to end. Adding, removing or moving a site fails packages/spec/scripts/build-schemas.ts until the line moves with it, and the failure prints the corrected entry in full. ⛔ Do not delete or weaken a refinement to shorten this file -- the runtime rule is correct; it is the projection that is silent, and the remedy is to teach the closed list a NAMED pattern, never to drop the rule.",
33
"measured": {
44
"zod": "4.4.3",
5-
"publishedSchemasWithDroppedRefinements": 205,
6-
"droppedRefinementSites": 565,
7-
"refinementSitesThatDidProject": 366,
8-
"refinementSitesWithNoJsonFormToCompare": 9
5+
"publishedSchemasWithDroppedRefinements": 206,
6+
"droppedRefinementSites": 571,
7+
"refinementSitesThatDidProject": 369,
8+
"refinementSitesWithNoJsonFormToCompare": 0
99
},
1010
"entries": {
1111
"ai/BlueprintField": {
@@ -614,6 +614,8 @@
614614
},
615615
"data/FieldOperators": {
616616
"sites": [
617+
"$between.items[0]",
618+
"$between.items[1]",
617619
"$in",
618620
"$nin"
619621
]
@@ -677,6 +679,8 @@
677679
},
678680
"data/NormalizedFilter": {
679681
"sites": [
682+
"lazy.$not.options[0].valueType.$between.items[0]",
683+
"lazy.$not.options[0].valueType.$between.items[1]",
680684
"lazy.$not.options[0].valueType.$in",
681685
"lazy.$not.options[0].valueType.$nin"
682686
]
@@ -745,6 +749,12 @@
745749
"in.where.options[0].lazy.options[2].items[0]"
746750
]
747751
},
752+
"data/RangeOperator": {
753+
"sites": [
754+
"$between.items[0]",
755+
"$between.items[1]"
756+
]
757+
},
748758
"data/ReferenceIdValue": {
749759
"sites": [
750760
""

‎packages/spec/scripts/build-schemas.ts‎

Lines changed: 55 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3582,6 +3582,58 @@ if (hasDroppedRefinementProblems(droppedRefinementProblems)) {
35823582
process.exit(1);
35833583
}
35843584

3585+
// ⭐ #18670 item 2 — the ratchet's OWN blind spot, held at zero.
3586+
//
3587+
// A site reaches this census only from the successful-emit path above, so every
3588+
// entry here describes a PUBLISHED file. An `undecidable` site is therefore a
3589+
// node that is published and whose rule the comparison could not read: it holds
3590+
// no ledger row, it is named in no `x-dropped-refinements`, and no repair of it
3591+
// could ever delete a row. That is strictly worse than a declared gap — the gap
3592+
// is real and the instrument built to count it cannot see it — so it is refused
3593+
// rather than reported, on the same principle as every ratchet above: growth is
3594+
// a reviewed line in a diff, never a number that moved.
3595+
//
3596+
// This population was NINE sites when the item was ruled, all of them nodes the
3597+
// published file reaches through `projectByPruningUnionBranches` while the
3598+
// detector's ladder stopped at the two strict rungs. Giving the detector the
3599+
// generator's own third rung took it to ZERO and turned those nine into
3600+
// ordinary `dropped` rows, which is the reading this check now pins.
3601+
const undecidableEntries = refinementCensus.filter((entry) => entry.undecidable.length > 0);
3602+
if (undecidableEntries.length > 0) {
3603+
const undecidableTotal = undecidableEntries.reduce((sum, entry) => sum + entry.undecidable.length, 0);
3604+
console.error(
3605+
`\n❌ ${undecidableTotal} PUBLISHED refinement site(s) across ${undecidableEntries.length} schema(s) ` +
3606+
`carry a rule this build could not adjudicate (#18670 item 2):`,
3607+
);
3608+
for (const entry of undecidableEntries) {
3609+
console.error(` + ${entry.defKey} (${entry.undecidable.length} site(s))`);
3610+
for (const site of entry.undecidable) {
3611+
console.error(` ${site.path === '' ? '<the export itself>' : site.path} (${site.nodeType})`);
3612+
}
3613+
}
3614+
console.error(
3615+
`\n These files ARE published, so the rule is enforced by the runtime and the file says\n` +
3616+
` nothing about it — the same gap ${DROPPED_REFINEMENTS_BASELINE_FILE} exists to hold. What is\n` +
3617+
` different, and worse, is that an undecidable site holds NO ledger row: it is invisible to\n` +
3618+
` the ratchet, absent from \`x-dropped-refinements\`, and no repair of it can delete a row.\n` +
3619+
` ⛔ Leaving it uncounted is the one outcome this check exists to refuse.\n\n` +
3620+
` Two remedies, in order of preference:\n\n` +
3621+
` 1. TEACH THE LADDER. \`projectOrNull\` in scripts/lib/dropped-refinements.ts must project\n` +
3622+
` the way THIS generator does, or it answers a question about a file nobody publishes.\n` +
3623+
` It carries three rungs — output, input, and the branch-pruning pass — because those\n` +
3624+
` are the three attempts the emit loop above makes. If your export reaches its file by\n` +
3625+
` a fourth route, the ladder owes that route too, and the site becomes an ordinary\n` +
3626+
` \`dropped\` row you declare in the ledger.\n` +
3627+
` 2. GIVE THE LEDGER AN \`undecidable\` ROW SHAPE. If a site genuinely cannot be judged,\n` +
3628+
` the population stops being empty and needs recording rather than refusing — an entry\n` +
3629+
` shape of its own, ratcheted in both directions like \`sites\`. That is a decision about\n` +
3630+
` what the ledger counts, ⛔ not a refactor: take it, don't assume it.\n\n` +
3631+
` ⛔ Do not delete or weaken the refinement to clear this line, and ⛔ do not stop\n` +
3632+
` publishing the export — the rule is correct and the file is wanted.`,
3633+
);
3634+
process.exit(1);
3635+
}
3636+
35853637
// The accepted population, reported in full on every run — the same discipline
35863638
// as the never-published ledger above, and for the same reason: a population
35873639
// that passes in silence is the silence this ratchet was built to end.
@@ -3602,8 +3654,9 @@ if (droppedSiteTotal > 0) {
36023654
` the refinement, and ⛔ never an open-ended translator over the whole population.`,
36033655
);
36043656
console.log(
3605-
` Also measured this run: ${projectedSiteTotal} refinement site(s) DID reach the file, ` +
3606-
`${undecidableSiteTotal} had no JSON form on either side to compare.`,
3657+
` Also measured this run: ${projectedSiteTotal} refinement site(s) DID reach the file, and ` +
3658+
`${undecidableSiteTotal} published site(s) could not be adjudicated — a population the check\n` +
3659+
` above holds at zero, so this figure is a reading of it and not a tolerance.`,
36073660
);
36083661
}
36093662

‎packages/spec/scripts/lib/dropped-refinements.ts‎

Lines changed: 47 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,9 @@ import {
9494
projectableRefinementsOf,
9595
zodDefOf,
9696
} from './refinement-projection';
97+
// The generator's THIRD projection attempt. The detector owes the same ladder
98+
// for the same reason it owes the same `override` — see `projectOrNull`.
99+
import { projectByPruningUnionBranches } from './union-branch-projection';
97100

98101
/** File name of the committed ledger, resolved against the package root. */
99102
export const DROPPED_REFINEMENTS_BASELINE_FILE = 'dropped-refinements.baseline.json';
@@ -134,8 +137,17 @@ export interface RefinementSite {
134137
/**
135138
* `dropped` — removing the refinements leaves the projection byte-identical.
136139
* `projected` — the projection changed, so the rule DID reach a reader.
137-
* `undecidable` — the node has no JSON form in either io direction, so the
138-
* comparison has no two sides. Reported, never counted as a gap.
140+
* `undecidable` — no rung of the generator's own projection ladder gives this
141+
* node a JSON form, so the comparison has no two sides.
142+
*
143+
* ⛔ `undecidable` is NOT a third way to be fine. A node can be undecidable
144+
* here and PUBLISHED all the same — its export reaches the file through a
145+
* route this comparison could not reproduce — and such a node states nothing
146+
* about its rule while holding zero ledger rows, which is the ratchet's own
147+
* blind spot rather than an absence of gap. Since #18670's second acceptance
148+
* item the ladder carries the generator's third rung so far fewer nodes land
149+
* here, and the ones that still do are counted: see
150+
* {@link DroppedRefinementsEntry.undecidableSites}.
139151
*/
140152
readonly verdict: 'dropped' | 'projected' | 'undecidable';
141153
/**
@@ -229,8 +241,15 @@ function withoutCustomChecks(schema: z.ZodType): z.ZodType | null {
229241
return stripped;
230242
}
231243

244+
/** One side of the differential: which ladder rung answered, and what it said. */
245+
interface LadderReading {
246+
/** `output` / `input` — a strict pass; `pruned` — the branch-dropping pass. */
247+
readonly rung: 'output' | 'input' | 'pruned';
248+
readonly text: string;
249+
}
250+
232251
/**
233-
* `toJSONSchema` in the generator's own io ladder, or `null` when neither side
252+
* `toJSONSchema` in the generator's own io ladder, or `null` when no rung of it
234253
* has a JSON form.
235254
*
236255
* ⭐ It projects through `projectPublishedJsonSchema` — the SAME call the
@@ -248,15 +267,27 @@ function withoutCustomChecks(schema: z.ZodType): z.ZodType | null {
248267
* `projected` behind a green ledger while the published file went wide in
249268
* silence. The helper's own docblock carries that measurement.
250269
*/
251-
function projectOrNull(schema: z.ZodType): string | null {
270+
function projectOrNull(schema: z.ZodType): LadderReading | null {
252271
for (const io of ['output', 'input'] as const) {
253272
try {
254-
return JSON.stringify(projectPublishedJsonSchema(schema, { io }));
273+
return { rung: io, text: JSON.stringify(projectPublishedJsonSchema(schema, { io })) };
255274
} catch {
256275
// Try the other direction — the generator does the same, for the same reason.
257276
}
258277
}
259-
return null;
278+
// THIRD rung — the generator's own third attempt (#16431 (a)), owed here for
279+
// the same reason the `override` is. A node whose every io direction refuses
280+
// over one unrepresentable member still REACHES the published file when that
281+
// member sits in a union position: `build-schemas.ts` drops the branch and
282+
// publishes the rest. Stopping the ladder at two rungs therefore asked a
283+
// different question than "what does the published file say about this rule"
284+
// — it asked what a projection nobody publishes says — and answered
285+
// `undecidable`, which is the one verdict the ledger does not count. Measured
286+
// on `data/NormalizedFilter`: the three record nodes carrying the `$`-prefix
287+
// ban published as bare objects, held ZERO ledger rows, and no repair of them
288+
// could ever have deleted a row.
289+
const pruned = projectByPruningUnionBranches(schema);
290+
return pruned ? { rung: 'pruned', text: JSON.stringify(pruned.schema) } : null;
260291
}
261292

262293
/** One node's raw differential and the verdict adjudicated from it. */
@@ -306,7 +337,16 @@ function readProjection(schema: z.ZodType): NodeProjectionReading {
306337
const before = projectOrNull(schema);
307338
const after = projectOrNull(stripped);
308339
if (before === null || after === null) return { verdict: 'undecidable', projectionMoved: false };
309-
if (before === after) return { verdict: 'dropped', projectionMoved: false };
340+
// ⛔ Two rungs are not two readings of one question. A differential whose
341+
// sides were answered by different passes compares a pruned projection with
342+
// an unpruned one, and the bytes then differ over the branches one side
343+
// dropped — a `projected` verdict earned by the ladder rather than by the
344+
// rule. Nothing observed reaches here (a `custom` check cannot change which
345+
// types a subtree carries, so both sides refuse and survive alike), which is
346+
// exactly why it is written down: the day it stops holding, this reads
347+
// `undecidable` and is counted, instead of reading `projected` and vanishing.
348+
if (before.rung !== after.rung) return { verdict: 'undecidable', projectionMoved: false };
349+
if (before.text === after.text) return { verdict: 'dropped', projectionMoved: false };
310350
const stated = projectableRefinementsOf(schema).length;
311351
const total = customChecksOf(schema).length;
312352
return { verdict: total === stated ? 'projected' : 'dropped', projectionMoved: true };

0 commit comments

Comments
 (0)