Skip to content

Export numeric bounds as JSON numbers and label every looser divergence - #60

Merged
VSN2015 merged 5 commits into
masterfrom
fix/schema-bounds-and-divergences
Sep 25, 2026
Merged

VSN2015 merged 5 commits into
masterfrom
fix/schema-bounds-and-divergences

Conversation

@VSN2015

@VSN2015 VSN2015 commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

A bug fix in the JSON Schema exporter. One bug made the exported document invalid. The others were places where the docs accept what the server rejects, without saying so. No runtime behaviour changes. Updated after review: see the last two sections.

The bugs

1. A decimal range bound came out as strings, which makes the schema invalid. Range bounds went through json_value, the re-encoding used for an authored default:/example:. That method renders a BigDecimal as its precision-safe string. The natural way to bound a price therefore published:

optional :price, :decimal, in: BigDecimal("0.01")..BigDecimal("999.99")
{ "type": ["string", "number"], "format": "decimal", "minimum": "0.01", "maximum": "999.99" }

The metaschema requires minimum/maximum to be numbers, so the whole document failed validation.

2. A bounded :decimal sent as a string skipped its bounds, and this was not labelled. :decimal is typed ["string", "number"], but minimum/maximum only apply to numbers. So "5000" passed the docs, and the server answered inclusion.

3. normalize: was not in the export. With required :name, :string, length: 3..10, normalize: :squish:

  payload             docs (minLength 3)   server
  "   "               accept               missing   (squishes to "", which counts as absent)
  " a  "              accept               length    (squishes to "a")
  "  abcdefghij  "    reject (14 > 10)     accept    (squishes to 10 chars)

The fix

  • Bounds are JSON numbers. A whole-number bound is emitted as an Integer, anything else as a Float. Exclusive and endless ranges work as before.
  • An infinite bound is omitted. Float::INFINITY and BigDecimal("Infinity") mean "no bound", so no key is emitted. The same goes for NaN, which compares to nothing. Neither is a JSON number.
  • normalize: is exported as x-permittable-normalize. The value is the preset's name ("squish", "email", …), which a client can apply itself, or true for a custom proc. The contract stores only the resolved lambda, so the exporter gets the name back with NORMALIZERS.key(callable), a lookup by identity. Contract data is unchanged.
  • Every looser divergence is now documented: in the README's "What the schema deliberately does not say", in the JsonSchema header comment, and in spec/schema_conformance_spec.rb. The README now lists three safe cases and six looser ones.

The decision I'd most like reviewed: rounding a bound that a double cannot hold

to_f rounds to the nearest double, which can land on the wrong side of the bound. For example, BigDecimal("0.1000000000000000001").to_f is 0.1. A client that sends 0.1 passes minimum: 0.1 and the server then refuses it.

How far is "wrong" depends on the field's type:

  • a :decimal field reads the number back as BigDecimal("0.1") and compares exactly;
  • a :float field compares the Float through BigDecimal#<=>, which reads the Float at limited precision. So a :float field needs a few doubles more.

So the exporter doesn't model either comparison. It asks the server. It takes the most extreme value the published keyword admits:

  • the bound itself for minimum/maximum;
  • the value just below the bound for exclusiveMaximum;
  • on an :integer field, the ceiling or floor.

While that value would fail the field's own cast and comparison, the bound moves one double inward (a minimum up, a maximum down). The published range is then only ever narrower than the enforced one, on every numeric type, and the loop stops at the first double the server accepts. (Bounds are exact as published; a client that reads the document back with ordinary double-precision floats, rather than the digits as sent, can still round a value across a boundary — that's inherent to parsing any JSON number as a double, not something the exporter controls.)

Details:

  • Only the relevant side of the range is asked. A range narrower than the gap between two doubles admits no double at all, so checking both ends would never settle.
  • The bound never moves outward. Where a lossy comparison would also accept a few doubles beyond the nearest one, those stay unpublished (stricter, the safe way).
  • A decimal of up to 15 significant digits round-trips through a double, so every price is emitted as written: 999.99, not 999.9899999999999.
  • An INTEGRAL bound is published exactly at any magnitude (10**400 included — a JSON integer has no size limit), so it never goes through this nudge at all. A genuinely FRACTIONAL bound beyond Float::MAX has no such escape and is omitted, same as an infinite bound — see the "Round 2" section below for why this needed a separate fix.

The conformance spec

The looser labels live in a LOOSER table. Each label names the keyword the diverging field's own schema must carry. The spec checks both the direction (server rejects, docs accept) and that the keyword is on that field, not anywhere in the document:

Label Server rejects, docs accept Field must carry
:extension_only an over-nested :json value x-permittable-max-depth
:range_extension "zebra" for in: "a".."m" x-permittable-range
:custom_validation a value a validate: proc refuses x-permittable-custom-validation
:normalized_first " ", " a " under squish x-permittable-normalize
:format_annotation "abc" / "NaN" (:decimal), "2026-02-30" (:date), "not a time" (:datetime) format
:string_decimal "5000" against ..999.99 maximum (as a number)

The padded case (" abcdefghij ") goes the safe way and has its own label, :normalized_encoding.

CASES and LOOSER are now constants of a SchemaConformance module, instead of top-level constants defined inside RSpec.describe.

I mutation-checked the per-field lookup: pointing :custom_validation at x-permittable-range fails the spec. The old anywhere-in-the-document check would have passed, because a sibling field in the same case carries that extension.

TinyJsonSchema needed no change, and the "exporter emits no keyword the validator silently ignores" example still passes.

Also

  • The golden fixture is unchanged.
  • I did not touch the pattern code (UNTRANSLATABLE, apply_pattern!, ecma_pattern). In scalar_schema only the apply_in! call changed, because it now passes the field type.
  • An untranslatable format: regexp (x-permittable-pattern) is not one of the six LOOSER cases the spec asserts — #66, reworking Ruby → ECMA-262 pattern translation, owns adding that case. The README and CHANGELOG now say so plainly instead of listing it alongside the six.

Review follow-up

  1. Infinite bounds. BigDecimal("Infinity") crashed the export (to_i raised FloatDomainError), and Float::INFINITY was emitted as Infinity. Both are now omitted, and so is NaN. Specs: treats an infinite bound as no bound at all, omits a NaN bound.
  2. The nudge on :float fields. The nudge is now checked against the field's own cast and comparison, as described above. Specs cover both :decimal and :float: the published minimum and maximum are accepted, and the minimum is no further inward than it needs to be.
  3. Missing looser cases. Strings whose validity is only a format (:decimal, :date, :datetime), validate: procs, and non-numeric ranges are now labelled and direction-checked.
  4. Release notes. The Unreleased "Added" bullet now points to the full looser list under Fixed. The README's safe list counts three cases, including :normalized_encoding.
  5. The keyword check now looks at the diverging field's own schema.
  6. The spec tables moved into the SchemaConformance module.

master was merged in first, with master's CHANGELOG entries kept ahead of this PR's.

Round 2: final regression-pass fixes

A last regression pass compared every export against master's and found one real regression plus three documentation-accuracy issues. All fixed:

  1. A bound beyond Float's range was dropped entirely, where master published it validly. optional :x, :float, in: (10**400..) used to export no minimum at all — the inward-nudge loop asks the field's own cast whether the bound is "honoured" by routing it through to_f, which overflows a value this large to Infinity, walks the bound to Infinity, and drops it. Master published minimum: 10**400 outright. Fixed: json_bound now returns an integral bound exactly, before the nudge loop runs, whenever to_f would overflow it — a JSON integer has no size limit, so this needs no Float conversion and none of the machinery that exists only to protect a fractional bound from double-rounding. A genuinely fractional bound past Float::MAX (e.g. BigDecimal("1e400") + BigDecimal("0.5")) has no arbitrary-precision JSON fallback and stays omitted — now a documented, deliberate choice rather than a silent one. New specs: publishes an integral bound beyond Float::MAX exactly, matching master and omits a FRACTIONAL bound beyond Float::MAX, unlike an integral one.
  2. The exclusiveMaximum/JSON-rounding concern isn't an exporter bug. A published bound rounding across its boundary when a client parses it with an ordinary double is inherent to IEEE754 parsing, not something the exporter can fix from the document side. The CHANGELOG and README now say so explicitly next to the "narrower than enforced" claim, rather than leaving it as an unqualified guarantee.
  3. The CHANGELOG/README looser-divergence counts didn't match what the spec asserts. LOOSER has six keys; the README's "Six cases go the other way" header was accurate, but its bullet list bundled :custom_validation, :range_extension and the untranslatable format: regexp into one bullet — five bullets naming seven rules, one of which (the regexp case) has no conformance case at all. Split into six one-to-one bullets, and the regexp case is now called out separately as not asserted by this spec, owned by Translate format: regexps into patterns valid under Ajv's u flag #66. Same fix applied to the parallel CHANGELOG bullet.
  4. Removed a stale CHANGELOG line. "No behaviour changes: this release adds tests and documentation only" sat mid-Unreleased even though Unreleased contains substantial behaviour changes (this PR's own export changes among them). Deleted.

Verification

  • 2 new examples this round in json_schema_spec.rb (publishes an integral bound beyond Float::MAX exactly, matching master, omits a FRACTIONAL bound beyond Float::MAX, unlike an integral one), on top of the 30 from the original round (5 in json_schema_spec.rb, 25 conformance payloads). 673 examples in total, 0 failures.
  • RuboCop clean.

🤖 Generated with Claude Code

Sang and others added 3 commits September 24, 2026 19:17
A :decimal bounded by BigDecimals published "minimum": "0.01" as a string,
because range bounds went through the authored-value re-encoding; the
metaschema requires numbers, so the document was invalid. Bounds are now
Integers when exact, otherwise Floats rounded inward when a double cannot
round-trip the bound, so the published range is never wider than the
enforced one.

normalize: is now exported as x-permittable-normalize, and it and the
string encoding of a bounded :decimal join max_depth: as documented,
direction-asserted divergences in the README and the conformance spec.
No runtime behaviour changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An infinite or NaN range endpoint is now omitted instead of crashing the
export (BigDecimal("Infinity").to_i raised) or emitting Infinity. The
inward nudge for an unrepresentable bound now asks the field's own cast
and comparison, so a :float bound is safe too; it used to be one double
short there.

The documented looser divergences now also cover strings whose validity
is only a format annotation (:decimal, :date, :datetime), validate:
procs, and non-numeric ranges. The conformance spec checks each marker
on the diverging field's own schema, and its tables live in a module.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@VSN2015

VSN2015 commented Sep 25, 2026

Copy link
Copy Markdown
Owner Author

Addressed all six review points in dd5be8e (after merging master): infinite/NaN bounds omitted instead of crashing; the inward nudge now checks the field's own cast and comparison, so :float is safe too; format-only strings, validate: and non-numeric ranges added to the looser list and labelled; release notes and README counts reconciled; LOOSER markers checked on the diverging field's own schema; the tables moved into a SchemaConformance module. 671 examples, 0 failures; RuboCop clean. Details are in the updated description.

@VSN2015 VSN2015 changed the title Export decimal bounds as numbers and label two looser divergences Export numeric bounds as JSON numbers and label every looser divergence Sep 25, 2026
optional :x, :float, in: (10**400..) dropped `minimum` entirely: the
inward-nudge loop asks the field's own cast whether the published bound
is honoured, and that cast runs the bound through `to_f`, which overflows
a value this large to Infinity and walked the bound to Infinity too,
where it was omitted like a genuinely infinite one. Master published it
outright as `minimum: 10**400`, so this was a regression, not a labelled
divergence: a JSON integer has no size limit, so an integral bound needs
neither Float conversion nor the nudge machinery, which exists only to
protect a FRACTIONAL bound from double-rounding. `json_bound` now returns
an integral bound before the loop runs whenever `to_f` would overflow it.
A genuinely fractional bound past Float::MAX still has no arbitrary-
precision JSON representation to fall back on and stays omitted, now
documented as deliberate rather than silent.

Also, from the last regression pass:
- Qualify the "published range can only be narrower than the enforced
  one" claim in the CHANGELOG/README: bounds are exact as published, but
  a client parsing with ordinary double-precision floats can still round
  a value across the boundary. That is inherent to double parsing, not
  an exporter bug, so no code change.
- Make the CHANGELOG/README looser-divergence counts match what
  spec/schema_conformance_spec.rb actually asserts (LOOSER has six keys,
  not seven): split the "rules only an extension can carry" bullet so
  each of the six maps to one bullet, and say plainly that an
  untranslatable format: regexp has no conformance case yet, since PR
  #66 owns the regexp-translation rework it would depend on.
- Delete the stale "No behaviour changes: this release adds tests and
  documentation only" line — Unreleased now contains substantial
  behaviour changes, this bound fix included.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@VSN2015

VSN2015 commented Sep 25, 2026

Copy link
Copy Markdown
Owner Author

Final regression pass, addressed in fc88567:

  1. Regression fixed: optional :x, :float, in: (10**400..) was dropping minimum entirely (the inward-nudge loop overflowed the bound to Infinity via to_f and dropped it), where master published minimum: 10**400 outright. json_bound now returns an integral bound exactly, before the nudge loop, whenever it's beyond Float::MAX — a JSON integer needs no size limit and no Float round-trip. Failing spec added first (publishes an integral bound beyond Float::MAX exactly, matching master), confirmed red, then fixed. A companion spec documents the deliberate omission of a genuinely fractional bound past Float::MAX (no arbitrary-precision JSON fallback exists for that case).
  2. Qualified the "narrower than the enforced range" claim in CHANGELOG/README: exact as published, but a client parsing with ordinary doubles can still round across a boundary — inherent to double parsing, not an exporter bug, no code change.
  3. Fixed the CHANGELOG/README looser-divergence counts to match spec/schema_conformance_spec.rb's LOOSER table (six keys): split the bundled bullet so each of the six gets its own line, and called out the untranslatable format: regexp case (x-permittable-pattern) as not asserted here — owned by Translate format: regexps into patterns valid under Ajv's u flag #66.
  4. Deleted the stale "No behaviour changes: this release adds tests and documentation only" CHANGELOG line.

ASDF_RUBY_VERSION=3.2.2 bundle exec rspec: 673 examples, 0 failures (671 → 673: two new examples this round). ASDF_RUBY_VERSION=3.2.2 bundle exec rubocop: clean.

CHANGELOG only: keep master's entries ahead of this PR's.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@VSN2015
VSN2015 merged commit 0f279df into master Sep 25, 2026
17 checks passed
@VSN2015
VSN2015 deleted the fix/schema-bounds-and-divergences branch September 25, 2026 08:24
VSN2015 pushed a commit that referenced this pull request Sep 25, 2026
Resolves CHANGELOG/README conflicts (kept both sides' entries; master's
CHANGELOG entries first) and merges apply_in!'s numeric-bound-safe
rework (#60) with this branch's opaque/in_published in: handling.

Also reconciles an interaction the merge surfaced: assert_comparable_range!
(added on this branch, to catch a wrong-TYPED Range like a String range
on an :integer) treated a NaN endpoint the same way, since NaN's <=>
always returns nil — which would have raised at class load for
`in: Float::NAN..`, a Range #60's own new export logic explicitly
allows to load (and simply omits from the schema, since it's never
`finite?`). A NaN endpoint is not evidence of a wrong-typed bound, so
it is left alone here exactly as an infinite endpoint already is.

# Conflicts:
#	CHANGELOG.md
#	README.md
#	lib/permittable/json_schema.rb
#	spec/json_schema_spec.rb

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
VSN2015 pushed a commit that referenced this pull request Sep 25, 2026
Master had moved to include #59-#61, #64 and #66 since this branch's last
merge, plus #60 and #65 which both touch json_schema.rb's :decimal/:datetime
export and lib/permittable/rspec.rb's `in:`/`default:` matcher chains —
real conflicts, not just additive.

lib/permittable.rb: kept this branch's permittable_transform seam
(AuthoredValues needs it to walk a default without running transform: on
it) — same guard (`violations.length == before`) as master's inline form.

lib/permittable/json_schema.rb (4 hunks): combined rather than picked a
side.
- apply_in!'s enum export now uses BOTH #65's `field[:in_published] ||
  allowed` (keeps a :date/:datetime member's authored String form) AND
  this branch's `decimal: :number` (keeps a :decimal member numeric and
  consistent with its own default/example export).
- kept #60's `apply_normalize!` method (unrelated to this branch) ahead of
  json_value, whose signature this branch changes to `decimal: :string`.
- json_value's Hash/BigDecimal/Time cases: kept this branch's `decimal:`
  threading for Hash/BigDecimal, and #65's `exact_iso8601` for Time (fixes
  sub-second Time :in members exporting as whole seconds — this branch's
  plain `.utc.iso8601` would have regressed that).
- kept both decimal_json (this branch) and exact_iso8601 (#65) methods;
  each is used from the merged json_value body above.
Verified with a probe: a :decimal in: list now exports numerically AND
agrees with its own numeric default, and a :datetime in: list with
sub-second members still exports the authored string via in_published —
both at once, which neither branch alone tested.

lib/permittable/rspec.rb (2 hunks) and spec/matchers_spec.rb: purely
additive — #65's `cast_in`/`same_in?`/`within` alongside this branch's
`default_mismatch`/`cast_default`. Combined by keeping both.

README.md: combined the `default:` row (this branch, the transform:
exception) with the `validate:` row (master, the array-skip-on-failed-
validate note) — same table, different rows each PR had touched.

Fixed along the way (found while merging, not part of either PR):
- spec/permittable_spec.rb: a #61 perf spec asserted `valid_encoding?` is
  called on the caller's OWN string object, but this branch's
  permittable_own copies a request's String before Coercion.cast ever sees
  it (to avoid aliasing params) — so the original object is never touched,
  though the copy is (correctly) scanned exactly once. Rewrote the spec
  to count via a shared counter that survives `dup`, so it asserts the
  real invariant (one scan overall) rather than one specific object's
  identity.
- spec/matchers_spec.rb: my own conflict resolution (both PRs inserted a
  new `it` block at the same point in the file) left one block's closing
  `end` missing — a syntax error that silently dropped all 42 examples in
  this file from the suite total without failing the run. Fixed; the
  suite total is now 819 (was silently 777).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant