From 395f8c902fea570f7b575743d3d8bc3d01dd1862 Mon Sep 17 00:00:00 2001 From: Sang Date: Thu, 24 Sep 2026 19:17:45 +0700 Subject: [PATCH 1/3] Export decimal bounds as numbers and label two looser divergences A :decimal bounded by BigDecimals published "minimum": "0.01" as a string, because range bounds went through the authored-value re-encoding; the metaschema requires numbers, so the document was invalid. Bounds are now Integers when exact, otherwise Floats rounded inward when a double cannot round-trip the bound, so the published range is never wider than the enforced one. normalize: is now exported as x-permittable-normalize, and it and the string encoding of a bounded :decimal join max_depth: as documented, direction-asserted divergences in the README and the conformance spec. No runtime behaviour changes. Co-Authored-By: Claude Opus 5.5 (1M context) --- CHANGELOG.md | 4 ++ README.md | 4 +- lib/permittable/json_schema.rb | 60 +++++++++++++++++++++++++- spec/json_schema_spec.rb | 47 +++++++++++++++++++++ spec/schema_conformance_spec.rb | 74 ++++++++++++++++++++++++++------- 5 files changed, 170 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0166e30..f2521cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,10 @@ Contracts that don't opt in are byte-for-byte unaffected: the default format is ### Changed - **The compatibility range is now tested rather than asserted, and narrowed to what passes.** CI ran one combination — the newest of everything — while the gemspec advertised `activesupport >= 5.0, < 9`. Testing the range surfaced two real problems. On **activesupport 5.0 and 5.1 a contract cannot be declared at all**: the registry is a `class_attribute ... default: []`, and `default:` arrived in Rails 5.2, so `permit_params` died on `NoMethodError: undefined method '+' for nil`. And on **activesupport ≤ 7.0.8.4, `require "permittable"` itself raised** `NameError: uninitialized constant ActiveSupport::LoggerThreadSafeLevel::Logger`, because concurrent-ruby 1.3.5 stopped requiring `logger` for them. The floor is now **`>= 6.1`** — the oldest line the full suite is run against — and the load failure is fixed with one stdlib `require "logger"` ahead of `require "active_support"`, so the gem loads whatever the host's own boot order. +### Fixed +- **A `:decimal` bounded by BigDecimals exported an invalid schema.** `in: BigDecimal("0.01")..BigDecimal("999.99")` — the natural way to bound a price — published `"minimum": "0.01", "maximum": "999.99"`, because range bounds went through the same re-encoding as an authored `default:`, which renders a BigDecimal as its precision-safe string. The metaschema requires `minimum`/`maximum` to be numbers, so the whole document failed validation. Bounds are now JSON numbers: an Integer when the bound is one, otherwise a Float. Where a double cannot hold the bound closely enough to read back as itself (possible only past 15 significant digits — never a price), it is moved one double **inward**, a minimum up and a maximum down, so the published range can only be narrower than the enforced one. `default:`/`example:` keep their string encoding, which `:decimal`'s type permits. +- **Two places the docs are looser than the server were unlabelled.** A `normalize:` step runs before the checks, so with `length: 3..10, normalize: :squish` the string `" "` passed the docs and was `missing` on the server, and `" a "` passed them and failed `length`. It is now exported as `x-permittable-normalize` — the preset's name, or `true` for a custom proc. And a bounded `:decimal` is documented as string-or-number while `minimum`/`maximum` only constrain numbers, so `"5000"` passed the docs for a bound the server enforces. Both are now in the README's list of divergences next to `max_depth:`, and `spec/schema_conformance_spec.rb` labels each and asserts its direction, and that the rule is still visible in the schema, so neither can pass as agreement. No runtime behaviour changes. + ## 0.8.0 (2026-09-19) diff --git a/README.md b/README.md index 749bd30..a413edd 100644 --- a/README.md +++ b/README.md @@ -920,9 +920,11 @@ Where they legitimately differ, the spec names the reason and asserts the **dire - **Non-canonical encodings.** Coercion accepts `"30"` for an `:integer` and `1` for a `:string`, because form and query payloads are all strings. The schema documents the canonical JSON encoding only. - **`null` as absence.** The runtime reads `{"age": null}` as `{}` ([absence](#absence-defaults-and-partial-updates)); JSON Schema cannot express that, so `type: integer` rejects a null the server would accept and ignore. A [`nullable:`](#explicit-nulls-nullable) field is not this case — there the null is a value, the exported `type` widens to say so, and the two agree. -One case goes the other way, and is worth knowing before you hand the document to a client: +Three cases go the other way, and are worth knowing before you hand the document to a client: - **Bounds JSON Schema has no keyword for.** A `:json` field's `max_depth:` is enforced by the server but cannot be written as a JSON Schema keyword, so the published document is **looser** there and an over-nested payload still earns a 422. The bound is not dropped — it is exported as `x-permittable-max-depth` — so a generator or linter that wants it can read it. +- **`normalize:` runs before the checks.** The server validates the *normalized* string, and JSON Schema has no keyword for "transform, then check". With `required :name, :string, length: 3..10, normalize: :squish`, `" "` passes the docs' `minLength: 3` and then squishes to `""` — absent, so `missing` — and `" a "` passes them and squishes to `"a"`, which is too short. (It also goes the safe way: `" abcdefghij "` is too long for the docs and fine once squished.) The step is exported as `x-permittable-normalize` — the preset's name (`"squish"`, `"email"`, …), which a client can apply before validating, or `true` for a custom proc. +- **A bounded `:decimal` sent as a string.** A `:decimal` is documented as `["string", "number"]`, because the string is its precision-safe encoding, but `minimum`/`maximum` constrain only numbers — so `"5000"` passes the docs for `in: BigDecimal("0.01")..BigDecimal("999.99")` and the server answers `inclusion`. The bound is still published, as a JSON number, for a client that parses the string first. Everything else the exporter cannot translate stays visible as an `x-permittable-*` extension rather than being guessed at. diff --git a/lib/permittable/json_schema.rb b/lib/permittable/json_schema.rb index e5ad9fc..2c3dcf6 100644 --- a/lib/permittable/json_schema.rb +++ b/lib/permittable/json_schema.rb @@ -23,6 +23,14 @@ module Permittable # never surprised by a 422. spec/schema_conformance_spec.rb holds that line, # asserting the direction of every divergence it permits. # + # Three rules have no JSON Schema keyword at all, and there the schema is + # LOOSER: a `:json` field's `max_depth:`, a `normalize:` step the server + # runs before it checks, and the string encoding of a bounded `:decimal` + # (minimum/maximum only constrain numbers). Each stays visible in the + # document — `x-permittable-max-depth`, `x-permittable-normalize`, the + # numeric bound itself — and the conformance spec labels each one rather + # than letting it pass as agreement. + # # Emission is deterministic (fixed key insertion order, declaration-order # properties) so generated documents are committable and diff-stable. module JsonSchema @@ -171,8 +179,38 @@ def apply_in!(schema, allowed) schema["x-permittable-range"] = allowed.inspect return end - schema["minimum"] = json_value(allowed.begin) if allowed.begin - schema[allowed.exclude_end? ? "exclusiveMaximum" : "maximum"] = json_value(allowed.end) if allowed.end + schema["minimum"] = json_bound(allowed.begin, round: :up) if allowed.begin + schema[allowed.exclude_end? ? "exclusiveMaximum" : "maximum"] = json_bound(allowed.end, round: :down) if allowed.end + end + + # minimum/maximum must be JSON numbers — the metaschema says so — so a + # bound is NOT an authored value for json_value, which renders a + # BigDecimal as its precision-safe string and made `in: + # BigDecimal("0.01")..BigDecimal("999.99")` publish an invalid document. + # Integer and Float pass through; any other Numeric (BigDecimal, + # Rational) becomes an Integer when it is one, else a Float. + # + # A Float cannot hold every decimal. to_f rounds to the NEAREST double, + # which can land on the wrong side of the bound: 0.1000000000000000001 + # becomes 0.1, and a client sending 0.1 passes `minimum: 0.1` while the + # server — which reads that number back as BigDecimal("0.1") and compares + # exactly — refuses it. So when the double's own shortest spelling (what + # Float#to_s gives, and what coercion parses) falls outside the bound, + # the bound moves one double INWARD: a minimum up, a maximum down. The + # published range can then only be narrower than the enforced one, the + # safe direction. A bound a double holds exactly enough to round-trip — + # any decimal of up to 15 significant digits, so every price — is emitted + # as written. + def json_bound(value, round:) + return value if value.is_a?(Integer) || value.is_a?(Float) + return value.to_i if value == value.to_i + + float = value.to_f + spelled = BigDecimal(float.to_s) + return float.next_float if round == :up && spelled < value + return float.prev_float if round == :down && spelled > value + + float end def apply_string_bounds!(schema, field) @@ -242,9 +280,27 @@ def annotate(schema, field) end schema["x-permittable-custom-validation"] = true if field[:validate] schema["x-permittable-transformed"] = true if field[:transform] + apply_normalize!(schema, field) schema end + # The server checks the NORMALIZED string, so minLength/maxLength/pattern + # describe a value the client never sends: under `normalize: :squish`, + # " " satisfies a minLength of 3 and then squishes to "" (absent), and + # " a " satisfies it and squishes to "a". JSON Schema has no keyword for + # "transform, then check", so the step is flagged rather than dropped — + # by the preset's name, which a client can apply itself, or `true` for a + # host proc, which is as opaque as `transform:`. The preset is recovered + # by identity from the resolved callable, because the contract stores the + # lambda it runs rather than the name it was declared by. + def apply_normalize!(schema, field) + normalizer = field[:normalize] + return unless normalizer + + preset = NORMALIZERS.key(normalizer) + schema["x-permittable-normalize"] = preset ? preset.to_s : true + end + def json_value(value) case value when Array then value.map { |v| json_value(v) } diff --git a/spec/json_schema_spec.rb b/spec/json_schema_spec.rb index 6fcafee..decad62 100644 --- a/spec/json_schema_spec.rb +++ b/spec/json_schema_spec.rb @@ -69,6 +69,37 @@ def property(name, **opts, &contract) expect(property("pct") { optional :pct, :integer, in: 0...100 }).to include("minimum" => 0, "exclusiveMaximum" => 100) end + it "emits BigDecimal bounds as JSON numbers, which is all minimum/maximum may be" do + # The natural way to bound a price. Routed through the authored-value + # re-encoding, the bounds came out as the strings "0.01" / "999.99" — + # which the metaschema forbids, so the whole document was invalid. + prop = property("price") { optional :price, :decimal, in: BigDecimal("0.01")..BigDecimal("999.99") } + expect(prop).to include("minimum" => 0.01, "maximum" => 999.99) + expect(prop.values_at("minimum", "maximum")).to all(be_a(Float)) + + exact = property("qty") { optional :qty, :decimal, in: BigDecimal("1")...BigDecimal("100") } + expect(exact).to include("minimum" => 1, "exclusiveMaximum" => 100) + expect(exact.values_at("minimum", "exclusiveMaximum")).to all(be_a(Integer)) + + endless = property("tip") { optional :tip, :decimal, in: BigDecimal("0.5").. } + expect(endless).to include("minimum" => 0.5) + expect(endless.keys.grep(/maximum/i)).to be_empty + end + + it "rounds a bound a double cannot hold INWARD, so the docs never admit what the server refuses" do + # 0.1000000000000000001 has no double; to_f rounds it to 0.1, and a + # client sending 0.1 would pass a published `minimum: 0.1` and then be + # refused by the server's exact BigDecimal comparison. The minimum is + # nudged up and the maximum down to the neighbouring double instead. + bound = BigDecimal("0.1000000000000000001") + low = property("x") { optional :x, :decimal, in: bound.. }["minimum"] + high = property("x") { optional :x, :decimal, in: ..bound }["maximum"] + expect(BigDecimal(low.to_s)).to be >= bound + expect(low).to eq(0.1.next_float) + expect(high).to eq(0.1) + expect(BigDecimal(high.to_s)).to be <= bound + end + it "carries a non-numeric Range as an extension instead of guessing" do prop = property("code") { optional :code, :string, in: "a".."m" } expect(prop["x-permittable-range"]).to eq('"a".."m"') @@ -167,6 +198,22 @@ def property(name, **opts, &contract) expect(props["tags"]["x-permittable-transformed"]).to be(true) end + it "exports normalize: as x-permittable-normalize — the preset's name, or true for a custom proc" do + # The server checks the NORMALIZED value, so minLength/maxLength/pattern + # describe a string the client never sends. The step is not a keyword + # JSON Schema has; it is flagged so a client can apply it first. + props = schema_for do + required :name, :string, length: 3..10, normalize: :squish + optional :email, :string, normalize: "email" + optional :code, :string, normalize: ->(v) { v.delete("-") } + optional :plain, :string + end["properties"] + expect(props["name"]).to include("minLength" => 3, "maxLength" => 10, "x-permittable-normalize" => "squish") + expect(props["email"]["x-permittable-normalize"]).to eq("email") + expect(props["code"]["x-permittable-normalize"]).to be(true) + expect(props["plain"]).not_to have_key("x-permittable-normalize") + end + it "marks a child that inherited sensitive: from its container writeOnly too" do props = schema_for do optional :payment, sensitive: true do diff --git a/spec/schema_conformance_spec.rb b/spec/schema_conformance_spec.rb index e56bc03..d0ef4c0 100644 --- a/spec/schema_conformance_spec.rb +++ b/spec/schema_conformance_spec.rb @@ -22,13 +22,34 @@ # that: `type: integer` reads null as a present value of the wrong type. # (A `nullable:` field is not this case: there the null IS a value, the # exporter widens `type` to say so, and the two agree.) - # :extension_only — the one divergence that runs the OTHER way, so it is - # labelled separately rather than waved through: the contract enforces a - # bound JSON Schema has no keyword for (`max_depth:`), so the schema is - # LOOSER than the server and a client following it can still be - # surprised by a 422. The exporter does not drop the bound — it emits it - # as `x-permittable-max-depth` — so this asserts both the direction and - # that the extension is present to be read. + # :normalized_encoding — the runtime runs `normalize:` BEFORE it checks, + # so a padded value whose normalized form fits (" abcdefghij " under + # squish and maxLength 10) is accepted though its raw form is too long. + # + # The rest run the OTHER way — the schema is LOOSER than the server, and a + # client following it can still be surprised by a 422 — so each is labelled + # separately rather than waved through, and LOOSER names what the exported + # schema must still carry for a tool that wants to close the gap: + # + # :extension_only — the contract enforces a bound JSON Schema has no + # keyword for (`max_depth:`). The exporter does not drop the bound — it + # emits it as `x-permittable-max-depth`. + # :normalized_first — the same `normalize:` step, the unsafe way round: + # " " passes a minLength of 3 and then squishes to "" (absent), " a " + # passes it and squishes to "a" (too short). JSON Schema has no + # keyword for "transform, then check", so the step is exported as + # `x-permittable-normalize`. + # :string_decimal — a `:decimal` is documented as string OR number, + # because the string is its precision-safe encoding, but + # minimum/maximum only ever constrain numbers: "5000" sails past a + # `maximum` of 999.99 that the server enforces on the parsed value. The + # bound is still published, as a number, for a client that parses first. + LOOSER = { + extension_only: "x-permittable-max-depth", + normalized_first: "x-permittable-normalize", + string_decimal: "maximum" + }.freeze + CASES = { "scalars and bounds" => { contract: proc { @@ -132,6 +153,31 @@ [{ "metadata" => { "a" => { "b" => { "c" => 1 } } } }, :extension_only] ] }, + "a normalized string" => { + contract: proc { required :name, :string, length: 3..10, normalize: :squish }, + payloads: [ + [{ "name" => "Jo Jo" }, :agree], + [{ "name" => "ab" }, :agree], + [{ "name" => "waaaaytoolong" }, :agree], + [{ "name" => " abcdefghij " }, :normalized_encoding], + [{ "name" => " " }, :normalized_first], + [{ "name" => " a " }, :normalized_first] + ] + }, + "a decimal bounded by BigDecimals" => { + # The natural way to bound a price — and the bounds must come out as + # JSON numbers, or the validator has nothing to compare against. + contract: proc { optional :price, :decimal, in: BigDecimal("0.01")..BigDecimal("999.99") }, + payloads: [ + [{ "price" => 5 }, :agree], + [{ "price" => 0.01 }, :agree], + [{ "price" => 999.99 }, :agree], + [{ "price" => 0.001 }, :agree], + [{ "price" => 1000 }, :agree], + [{ "price" => "5.00" }, :agree], + [{ "price" => "5000" }, :string_decimal] + ] + }, "arrays" => { contract: proc { array :tags, of: :string, length: 1..3 }, payloads: [ @@ -219,14 +265,14 @@ def runtime_verdict(klass, payload) expect(documented).to eq(runtime), "contract said #{runtime}, its own schema said #{documented} " \ "(#{TinyJsonSchema.errors(schema, payload).inspect}); schema: #{schema.inspect}" - elsif expectation == :extension_only + elsif LOOSER.key?(expectation) # The unsafe direction, allowed only where JSON Schema has no - # keyword at all — and only with the bound still visible as an - # extension, so a tool that wants it can find it. + # keyword that says it — and only with the rule still visible in + # the schema, so a tool that wants it can find it. expect([runtime, documented]).to eq(%i[reject accept]), - "extension_only is for a bound the schema cannot carry; " \ + "#{expectation} is for a rule the schema cannot carry; " \ "got runtime=#{runtime}, documented=#{documented}" - expect(extensions_in(schema)).to include("x-permittable-max-depth") + expect(keywords_in(schema)).to include(LOOSER.fetch(expectation)) else # Only ever in the safe direction: a client following the docs is # conservative, never surprised by a 422. @@ -257,10 +303,6 @@ def runtime_verdict(klass, payload) "the exporter emits #{unchecked.inspect}, which TinyJsonSchema does not check" end - def extensions_in(schema) - keywords_in(schema).grep(/\Ax-permittable-/) - end - def keywords_in(node) return [] unless node.is_a?(Hash) From dd5be8edfb2a7c4a809e5a855a5d48d383c5680c Mon Sep 17 00:00:00 2001 From: Sang Date: Fri, 25 Sep 2026 14:03:59 +0700 Subject: [PATCH 2/3] Verify bounds against the field's cast and complete the looser list An infinite or NaN range endpoint is now omitted instead of crashing the export (BigDecimal("Infinity").to_i raised) or emitting Infinity. The inward nudge for an unrepresentable bound now asks the field's own cast and comparison, so a :float bound is safe too; it used to be one double short there. The documented looser divergences now also cover strings whose validity is only a format annotation (:decimal, :date, :datetime), validate: procs, and non-numeric ranges. The conformance spec checks each marker on the diverging field's own schema, and its tables live in a module. Co-Authored-By: Claude Opus 5.5 (1M context) --- CHANGELOG.md | 12 +++- README.md | 9 ++- lib/permittable/json_schema.rb | 114 +++++++++++++++++++++++--------- spec/json_schema_spec.rb | 60 ++++++++++++++--- spec/schema_conformance_spec.rb | 97 +++++++++++++++++++++------ 5 files changed, 227 insertions(+), 65 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 606d6d4..de5ec0b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ - **A CI matrix over the whole supported range.** The suite now runs against activesupport 6.1, 7.0, 7.1, 7.2, 8.0 and 8.1 across the supported Rubies, via one pinned gemfile per line in [`gemfiles/`](gemfiles/README.md) (no new dev dependency — plain `BUNDLE_GEMFILE`). The matrix is an explicit include list rather than a cross product, so it doubles as the answer to "which combinations are actually supported?". Variant lockfiles are deliberately not committed: each run resolves the newest patch of its line, so a regression in a supported version fails CI instead of being frozen out by a stale lock. - **A `runtime-deps` CI job that proves activesupport is the only runtime dependency.** The spec suite can't: it bundles actionpack and activerecord to exercise the integration and schema-drift paths. This job installs the *built gem* with nothing but its declared dependencies, asserts actionpack/activerecord/rails are genuinely absent, and then exercises every controller-free surface — standalone contracts (casting, defaults, `finalize`, 400/422 semantics, nested plain hashes), the concern on a plain params duck, `unknown: :error` with no logger to warn through, `sensitive:` registration with no Railtie, monitor mode, instrumentation, JSON Schema, OpenAPI assembly, and the generator. Every `respond_to?`/`defined?` guard in the gem is a promise; a missing one now fails CI. - **`spec/schema_conformance_spec.rb` — the "docs cannot drift" claim is now tested rather than argued.** Every other spec checks one side or the other; this one checks that the two **agree**, walking canonical JSON payloads through both a contract and its own exported schema and comparing the verdicts. It covers scalars and bounds, enums, exclusive and endless ranges, exact lengths, formats, arrays of scalars and of hashes, nested hashes under `unknown: :error`, rooted contracts, `nullable:` fields, and opaque `:json` fields with bounds. `spec/support/tiny_json_schema.rb` is a deliberately small validator covering exactly the keywords the exporter emits and nothing else — a measuring instrument, not a dependency — and one example asserts the exporter emits no keyword the validator silently ignores, so the two cannot fall out of step. The spec was mutation-tested: dropping `maxItems`, `additionalProperties: false`, the required-string `minLength: 1`, or the `root:` required wrapper each makes it fail. -- **Every place the schema and the runtime differ is now labelled, and its direction asserted.** Two go the safe way — the server accepts what the docs reject, so a client following the docs is merely conservative: non-canonical encodings (`"30"` for an `:integer`, `1` for a `:string`, because form and query payloads are all strings), and an explicit `null` read as absence on a field that is not `nullable:`. One goes the other way and is now stated plainly instead of being left to be discovered: a `:json` field's `max_depth:` is a bound JSON Schema has no keyword for, so the published document is **looser** than the server there and an over-nested payload still earns a 422. The bound is exported as `x-permittable-max-depth` rather than dropped, and the spec asserts that extension is present. A new divergence in either direction fails the suite rather than shipping quietly. +- **Every place the schema and the runtime differ is now labelled, and its direction asserted.** Two go the safe way — the server accepts what the docs reject, so a client following the docs is merely conservative: non-canonical encodings (`"30"` for an `:integer`, `1` for a `:string`, because form and query payloads are all strings), and an explicit `null` read as absence on a field that is not `nullable:`. The ones that go the other way are now stated plainly instead of being left to be discovered, starting with a `:json` field's `max_depth:`: a bound JSON Schema has no keyword for, so the published document is **looser** than the server there and an over-nested payload still earns a 422. The bound is exported as `x-permittable-max-depth` rather than dropped, and the spec asserts that extension is present. (The rest of the looser list — and a third safe case — are under **Fixed**.) A new divergence in either direction fails the suite rather than shipping quietly. No behaviour changes: this release adds tests and documentation only. - **`permittable:generate` now reads Rails 8 `params.expect` calls, not just `params.permit`.** The generator is the adoption on-ramp, and it was blind to the syntax Rails 8 apps actually use — a modern controller has no permit calls to scan, so the draft fell back to columns alone and lost everything the app already knew about its own params (the `root:`, the permitted key list, which keys aren't columns). `params.expect(user: [:name, tag_names: [], address: [:city], line_items: [[:sku]]])` is now scanned into the same `Scan`, merged with any permit calls in the same controller. - **Arrays of hashes are drafted without a TODO when the source says so.** `expect` distinguishes what `permit` cannot: `key: [:a]` is a nested hash, `key: [[:a]]` is an array of hashes. A scanned `[[...]]` therefore drafts as `array :key do ... end` with no "this may be an array of hashes" TODO, and `Scan` carries the new `nested_arrays` member alongside `nested`. @@ -61,8 +61,14 @@ Contracts that don't opt in are byte-for-byte unaffected: the default format is - **Enum columns drafted as `:integer`**, rejecting the `"shipped"` every form sends. A Rails `enum` now drafts as `:string, in: Model.statuses.keys` — the model's own accessor rather than today's keys inlined, so adding a value cannot leave the contract behind — and a database default is shown as its enum key (`"pending"`, not `0`). Rails also assigns an integer-backed enum its stored integer (`status: 1` from a JSON client), which a `:string` field turns into a rejected `"1"`; the line carries a TODO saying how to admit it rather than guessing that clients send it. An enum whose name is not a method identifier (`first-status`) is reached as `Model.defined_enums["first-status"]`, so the draft still loads. - **The STI inheritance column and `lock_version` were drafted as client-writable fields.** Mass-assigning `type` changes which class the record loads as, which is a privilege-escalation shape, not a field. Drafted from columns alone, both are now omitted from the fields and named in a TODO explaining why — including where to declare `lock_version` if the app's forms round-trip it for stale-update detection, worded for the rules actually drafted. When the controller's own permit call lists one, it stays a field with a TODO instead: omitting a `lock_version` the app sends would switch stale-update detection off the day the draft is enforced. Each counts only when the model uses it — a `type` column with `self.inheritance_column = nil`, or `lock_version` with `self.lock_optimistically = false`, is an ordinary column and drafts as one. A model left with no field to draft (only `type` and `lock_version`, or nothing else with a contract type) drafts nothing, as a model with no columns does, rather than a rule that raises `a contract must declare at least one field` when pasted. - **Column names that are not symbol literals produced a draft that did not parse.** `first-name`, `2fa_enabled` and `Email Address` were emitted as `:first-name` and friends; names are now emitted with `Symbol#inspect` (`:"first-name"`), so the draft is valid Ruby whatever the schema. -- **A `:decimal` bounded by BigDecimals exported an invalid schema.** `in: BigDecimal("0.01")..BigDecimal("999.99")` — the natural way to bound a price — published `"minimum": "0.01", "maximum": "999.99"`, because range bounds went through the same re-encoding as an authored `default:`, which renders a BigDecimal as its precision-safe string. The metaschema requires `minimum`/`maximum` to be numbers, so the whole document failed validation. Bounds are now JSON numbers: an Integer when the bound is one, otherwise a Float. Where a double cannot hold the bound closely enough to read back as itself (possible only past 15 significant digits — never a price), it is moved one double **inward**, a minimum up and a maximum down, so the published range can only be narrower than the enforced one. `default:`/`example:` keep their string encoding, which `:decimal`'s type permits. -- **Two places the docs are looser than the server were unlabelled.** A `normalize:` step runs before the checks, so with `length: 3..10, normalize: :squish` the string `" "` passed the docs and was `missing` on the server, and `" a "` passed them and failed `length`. It is now exported as `x-permittable-normalize` — the preset's name, or `true` for a custom proc. And a bounded `:decimal` is documented as string-or-number while `minimum`/`maximum` only constrain numbers, so `"5000"` passed the docs for a bound the server enforces. Both are now in the README's list of divergences next to `max_depth:`, and `spec/schema_conformance_spec.rb` labels each and asserts its direction, and that the rule is still visible in the schema, so neither can pass as agreement. No runtime behaviour changes. +- **A `:decimal` bounded by BigDecimals exported an invalid schema.** `in: BigDecimal("0.01")..BigDecimal("999.99")` — the natural way to bound a price — published `"minimum": "0.01", "maximum": "999.99"`, because range bounds went through the same re-encoding as an authored `default:`, which renders a BigDecimal as its precision-safe string. The metaschema requires `minimum`/`maximum` to be numbers, so the whole document failed validation. Bounds are now JSON numbers: an Integer when the bound is one, otherwise a Float. Where the nearest double would let through a value the server refuses (possible only past 15 significant digits — never a price), the bound moves **inward**, a minimum up and a maximum down, one double at a time until the field's own cast and comparison accept the most extreme value the docs admit. That check matters because the right distance depends on the type: a `:decimal` compares exactly, while a `:float` compares through BigDecimal's lossy reading of a Float and needs a few doubles more. So the published range can only be narrower than the enforced one, on either type. An infinite bound (`Float::INFINITY`, `BigDecimal("Infinity")`) means no bound and is omitted, as is a NaN one; neither is a JSON number. `default:`/`example:` keep their string encoding, which `:decimal`'s type permits. +- **Most of the places the docs are looser than the server were unlabelled.** Only `max_depth:` was listed. Now the README, the exporter's own comment and `spec/schema_conformance_spec.rb` also name these — the spec labels each, asserts its direction, and checks that the rule is still visible on the diverging field's own schema, so none can pass as agreement: + - **`normalize:` runs before the checks.** With `length: 3..10, normalize: :squish`, `" "` passed the docs and was `missing` on the server, and `" a "` passed them and failed `length`. The step was not exported at all; it is now `x-permittable-normalize` — the preset's name, or `true` for a custom proc. (It also runs the safe way — `" abcdefghij "` is too long for the docs and fine once squished — which is listed with the safe cases.) + - **A bounded `:decimal` sent as a string** skips `minimum`/`maximum`, which constrain only numbers, so `"5000"` passed the docs for a bound the server enforces. + - **Strings whose validity is a `format`.** `"abc"` or `"NaN"` for a `:decimal`, and `"2026-02-30"` for a `:date` or `:datetime`, pass the docs — `format` is an annotation in draft 2020-12 — and fail the cast. + - **Rules only an extension can carry:** a `validate:` proc (`x-permittable-custom-validation`), a Range of non-numbers (`x-permittable-range`), and an untranslatable `format:` regexp (`x-permittable-pattern`). + + No runtime behaviour changes. ## 0.8.0 (2026-09-19) diff --git a/README.md b/README.md index f0ce9d8..0854147 100644 --- a/README.md +++ b/README.md @@ -981,16 +981,19 @@ Output is deterministic (fixed key order, declaration-order properties), so the `spec/schema_conformance_spec.rb` holds the "cannot drift" claim to account: it walks canonical JSON payloads through both the contract and its own exported schema and asserts the verdicts agree. -Where they legitimately differ, the spec names the reason and asserts the **direction**, so a new divergence fails the suite instead of shipping quietly. Two cases go the safe way — the **server accepts what its docs reject**, leaving a client that follows the docs merely conservative: +Where they legitimately differ, the spec names the reason and asserts the **direction**, so a new divergence fails the suite instead of shipping quietly. Three cases go the safe way — the **server accepts what its docs reject**, leaving a client that follows the docs merely conservative: - **Non-canonical encodings.** Coercion accepts `"30"` for an `:integer` and `1` for a `:string`, because form and query payloads are all strings. The schema documents the canonical JSON encoding only. - **`null` as absence.** The runtime reads `{"age": null}` as `{}` ([absence](#absence-defaults-and-partial-updates)); JSON Schema cannot express that, so `type: integer` rejects a null the server would accept and ignore. A [`nullable:`](#explicit-nulls-nullable) field is not this case — there the null is a value, the exported `type` widens to say so, and the two agree. +- **Padding that normalizes away.** `normalize:` runs before the checks, so under `normalize: :squish` and `length: 3..10` the server accepts `" abcdefghij "` — ten characters once squished — while the docs reject its fourteen. -Three cases go the other way, and are worth knowing before you hand the document to a client: +Six cases go the other way, and are worth knowing before you hand the document to a client. Each rule stays visible on its own field, and the spec asserts that as well as the direction: - **Bounds JSON Schema has no keyword for.** A `:json` field's `max_depth:` is enforced by the server but cannot be written as a JSON Schema keyword, so the published document is **looser** there and an over-nested payload still earns a 422. The bound is not dropped — it is exported as `x-permittable-max-depth` — so a generator or linter that wants it can read it. -- **`normalize:` runs before the checks.** The server validates the *normalized* string, and JSON Schema has no keyword for "transform, then check". With `required :name, :string, length: 3..10, normalize: :squish`, `" "` passes the docs' `minLength: 3` and then squishes to `""` — absent, so `missing` — and `" a "` passes them and squishes to `"a"`, which is too short. (It also goes the safe way: `" abcdefghij "` is too long for the docs and fine once squished.) The step is exported as `x-permittable-normalize` — the preset's name (`"squish"`, `"email"`, …), which a client can apply before validating, or `true` for a custom proc. +- **`normalize:` runs before the checks.** The server validates the *normalized* string, and JSON Schema has no keyword for "transform, then check". With `required :name, :string, length: 3..10, normalize: :squish`, `" "` passes the docs' `minLength: 3` and then squishes to `""` — absent, so `missing` — and `" a "` passes them and squishes to `"a"`, which is too short. The step is exported as `x-permittable-normalize` — the preset's name (`"squish"`, `"email"`, …), which a client can apply before validating, or `true` for a custom proc. - **A bounded `:decimal` sent as a string.** A `:decimal` is documented as `["string", "number"]`, because the string is its precision-safe encoding, but `minimum`/`maximum` constrain only numbers — so `"5000"` passes the docs for `in: BigDecimal("0.01")..BigDecimal("999.99")` and the server answers `inclusion`. The bound is still published, as a JSON number, for a client that parses the string first. +- **Rules only an extension can carry.** A `validate:` proc is app code, so the schema can only flag it (`x-permittable-custom-validation`); a Range of non-numbers such as `in: "a".."m"` has no keyword and rides along as `x-permittable-range`; and a `format:` regexp that does not translate to ECMA-262 is published as `x-permittable-pattern` rather than as a `pattern` that would enforce something else. A value any of these refuses passes the docs. +- **Strings whose validity is a `format`.** `:decimal`, `:date` and `:datetime` are sent as strings, and what makes such a string valid is its `format` (`"decimal"`, `"date"`, `"date-time"`) — which draft 2020-12 treats as an annotation unless a validator opts into asserting it. So `"abc"` or `"NaN"` for a `:decimal` and `"2026-02-30"` for a `:date` pass most validators and fail the server's cast with `invalid_type`. Everything else the exporter cannot translate stays visible as an `x-permittable-*` extension rather than being guessed at. diff --git a/lib/permittable/json_schema.rb b/lib/permittable/json_schema.rb index 2c3dcf6..aa57036 100644 --- a/lib/permittable/json_schema.rb +++ b/lib/permittable/json_schema.rb @@ -23,13 +23,24 @@ module Permittable # never surprised by a 422. spec/schema_conformance_spec.rb holds that line, # asserting the direction of every divergence it permits. # - # Three rules have no JSON Schema keyword at all, and there the schema is - # LOOSER: a `:json` field's `max_depth:`, a `normalize:` step the server - # runs before it checks, and the string encoding of a bounded `:decimal` - # (minimum/maximum only constrain numbers). Each stays visible in the - # document — `x-permittable-max-depth`, `x-permittable-normalize`, the - # numeric bound itself — and the conformance spec labels each one rather - # than letting it pass as agreement. + # Where a rule has no JSON Schema keyword, the schema is LOOSER instead, + # and a client following it can still earn a 422. Each such rule stays + # visible on its own field, and the conformance spec labels each one + # rather than letting it pass as agreement: + # * a `:json` field's `max_depth:` — `x-permittable-max-depth`; + # * a Range of non-numbers (`in: "a".."m"`) — `x-permittable-range`; + # * a `validate:` proc — `x-permittable-custom-validation`; + # * a `normalize:` step, which the server runs BEFORE it checks, so + # " " passes a minLength of 3 and is then absent — + # `x-permittable-normalize`; + # * the string encoding of :decimal, :date and :datetime, whose validity + # rests on `format`, an annotation in draft 2020-12 — so "abc", "NaN" + # and "2026-02-30" pass the schema and fail the cast; + # * the string encoding of a BOUNDED :decimal, which minimum/maximum + # (number-only keywords) never see — the numeric bound is published + # for a client that parses first. + # (A `normalize:` step also runs the safe way: " abcdefghij " is too + # long for a maxLength of 10 and fine once squished.) # # Emission is deterministic (fixed key insertion order, declaration-order # properties) so generated documents are committable and diff-stable. @@ -125,7 +136,7 @@ def nullify!(schema, field) def scalar_schema(field) schema = SCALAR_SCHEMAS.fetch(field[:type]).dup apply_format_name!(schema, field) - apply_in!(schema, field[:in]) + apply_in!(schema, field[:in], type: field[:type]) apply_string_bounds!(schema, field) # A preset's pattern is authored by this gem rather than by the app, so # it needs no heuristic — see apply_pattern!. @@ -165,7 +176,7 @@ def array_schema(field, unknown:) schema end - def apply_in!(schema, allowed) + def apply_in!(schema, allowed, type:) return unless allowed unless allowed.is_a?(Range) @@ -179,10 +190,17 @@ def apply_in!(schema, allowed) schema["x-permittable-range"] = allowed.inspect return end - schema["minimum"] = json_bound(allowed.begin, round: :up) if allowed.begin - schema[allowed.exclude_end? ? "exclusiveMaximum" : "maximum"] = json_bound(allowed.end, round: :down) if allowed.end + min = json_bound(allowed, "minimum", type) if allowed.begin + schema["minimum"] = min if min + keyword = allowed.exclude_end? ? "exclusiveMaximum" : "maximum" + max = json_bound(allowed, keyword, type) if allowed.end + schema[keyword] = max if max end + # The types whose cast turns a JSON number into the value `in:` compares, + # so a published bound can be checked against the server's own verdict. + NUMERIC_TYPES = %i[integer float decimal].freeze + # minimum/maximum must be JSON numbers — the metaschema says so — so a # bound is NOT an authored value for json_value, which renders a # BigDecimal as its precision-safe string and made `in: @@ -190,27 +208,63 @@ def apply_in!(schema, allowed) # Integer and Float pass through; any other Numeric (BigDecimal, # Rational) becomes an Integer when it is one, else a Float. # + # An infinite endpoint (Float::INFINITY, BigDecimal("Infinity")) means + # "no bound", and neither it nor NaN — which compares to nothing — is a + # JSON number, so both are omitted: nil. (to_i on either raises, which + # used to take the whole export down with it.) + # # A Float cannot hold every decimal. to_f rounds to the NEAREST double, # which can land on the wrong side of the bound: 0.1000000000000000001 - # becomes 0.1, and a client sending 0.1 passes `minimum: 0.1` while the - # server — which reads that number back as BigDecimal("0.1") and compares - # exactly — refuses it. So when the double's own shortest spelling (what - # Float#to_s gives, and what coercion parses) falls outside the bound, - # the bound moves one double INWARD: a minimum up, a maximum down. The - # published range can then only be narrower than the enforced one, the - # safe direction. A bound a double holds exactly enough to round-trip — - # any decimal of up to 15 significant digits, so every price — is emitted - # as written. - def json_bound(value, round:) - return value if value.is_a?(Integer) || value.is_a?(Float) - return value.to_i if value == value.to_i - - float = value.to_f - spelled = BigDecimal(float.to_s) - return float.next_float if round == :up && spelled < value - return float.prev_float if round == :down && spelled > value - - float + # becomes 0.1, and a client sending 0.1 passes `minimum: 0.1` and is then + # refused. How far is "wrong" depends on the field's TYPE, not on the + # bound: a :decimal reads the number back as BigDecimal("0.1") and + # compares exactly, while a :float compares the Float through + # BigDecimal#<=>, which reads it at limited precision and so needs a few + # doubles more. Rather than model either, the bound asks the server: + # while the most extreme value the published keyword admits would be + # refused by the field's own cast and comparison, the bound moves one + # double INWARD. The published range can then only be narrower than the + # enforced one — the safe direction — and stops at the first double the + # server accepts. (It never moves outward: where a lossy comparison + # would also accept a few doubles beyond the nearest one, those stay + # unpublished.) A decimal of up to 15 significant digits — every price — + # round-trips through a double, so it is emitted as written. + def json_bound(range, keyword, type) + value = keyword == "minimum" ? range.begin : range.end + return nil unless value.finite? + + bound = value.is_a?(Float) || value != value.to_i ? value.to_f : value.to_i + step = keyword == "minimum" ? :next_float : :prev_float + # A fractional bound past Float::MAX converts to Infinity, which no + # step moves; it is then as unrepresentable as an infinite one. + bound = bound.to_f.public_send(step) until !bound.finite? || honoured?(range, keyword, type, bound) + bound if bound.finite? + end + + # Would the server accept the most extreme value `keyword: bound` + # admits? Only the one side is asked — a range narrower than a double + # can span admits no double at all, and checking both ends would never + # settle. A non-numeric field type has no cast to ask, so its bound is + # published as converted. + def honoured?(range, keyword, type, bound) + return true unless NUMERIC_TYPES.include?(type) + + side = keyword == "minimum" ? (range.begin..) : Range.new(nil, range.end, range.exclude_end?) + status, value = Coercion.cast(type, admitted_extreme(keyword, type, bound)) + status == :ok && side.cover?(value) + end + + # The value nearest the bound that the published keyword still lets + # through: the bound itself for minimum/maximum, the double (or, on an + # :integer field, the integer) just below it for exclusiveMaximum. + def admitted_extreme(keyword, type, bound) + if type == :integer + return bound.ceil if keyword == "minimum" + return bound.floor if keyword == "maximum" + + return bound.ceil - 1 + end + keyword == "exclusiveMaximum" ? bound.to_f.prev_float : bound end def apply_string_bounds!(schema, field) diff --git a/spec/json_schema_spec.rb b/spec/json_schema_spec.rb index decad62..6e8eb7c 100644 --- a/spec/json_schema_spec.rb +++ b/spec/json_schema_spec.rb @@ -86,18 +86,58 @@ def property(name, **opts, &contract) expect(endless.keys.grep(/maximum/i)).to be_empty end - it "rounds a bound a double cannot hold INWARD, so the docs never admit what the server refuses" do - # 0.1000000000000000001 has no double; to_f rounds it to 0.1, and a + # The server's own verdict on a value a client sends as the JSON number + # `value`: the field's cast, then its rules — exactly what a request runs. + def server_accepts?(field_rule, value) + Permittable::Coercion.check_scalar(field_rule[:fields].first, value).first == :ok + end + + it "rounds a bound a double cannot hold INWARD, verified against the field's own comparison" do + # 0.1000000000000000001 has no double. to_f rounds it to 0.1, and a # client sending 0.1 would pass a published `minimum: 0.1` and then be - # refused by the server's exact BigDecimal comparison. The minimum is - # nudged up and the maximum down to the neighbouring double instead. + # refused. How far inward is right depends on the TYPE: a :decimal reads + # the number back as BigDecimal("0.1…") and compares exactly, while a + # :float compares the Float itself, through BigDecimal#<=>'s own + # limited-precision reading of it — so one double inward is enough for + # the first and not for the second. bound = BigDecimal("0.1000000000000000001") - low = property("x") { optional :x, :decimal, in: bound.. }["minimum"] - high = property("x") { optional :x, :decimal, in: ..bound }["maximum"] - expect(BigDecimal(low.to_s)).to be >= bound - expect(low).to eq(0.1.next_float) - expect(high).to eq(0.1) - expect(BigDecimal(high.to_s)).to be <= bound + %i[decimal float].each do |type| + low_rule = rule_for { optional :x, type, in: bound.. } + high_rule = rule_for { optional :x, type, in: ..bound } + low = described_class.rule(low_rule)["properties"]["x"]["minimum"] + high = described_class.rule(high_rule)["properties"]["x"]["maximum"] + + expect(server_accepts?(low_rule, low)).to be(true), "#{type}: the server refuses the published minimum #{low}" + expect(server_accepts?(low_rule, low.prev_float)).to be(false), "#{type}: #{low} is further inward than needed" + expect(server_accepts?(high_rule, high)).to be(true), "#{type}: the server refuses the published maximum #{high}" + end + # The nearest double below the bound needs no nudge on either type. (On + # a :float the server's lossy comparison would accept a few doubles + # more; the bound only ever moves inward, so those stay unpublished — + # stricter, the safe way.) + expect(property("x") { optional :x, :decimal, in: ..bound }["maximum"]).to eq(0.1) + expect(property("x") { optional :x, :float, in: ..bound }["maximum"]).to eq(0.1) + expect(property("x") { optional :x, :decimal, in: bound.. }["minimum"]).to eq(0.1.next_float) + expect(property("x") { optional :x, :float, in: bound.. }["minimum"]).to be > 0.1.next_float + end + + it "treats an infinite bound as no bound at all" do + # BigDecimal("Infinity") used to crash the whole export (to_i raises + # FloatDomainError), and Float::INFINITY is not a JSON number. + decimal = property("x") { optional :x, :decimal, in: BigDecimal("0")..BigDecimal("Infinity") } + expect(decimal).to include("minimum" => 0) + expect(decimal.keys.grep(/maximum/i)).to be_empty + + float = property("x") { optional :x, :float, in: -Float::INFINITY...Float::INFINITY } + expect(float.keys.grep(/imum/i)).to be_empty + expect { JSON.generate(float) }.not_to raise_error + end + + it "omits a NaN bound, which compares to nothing" do + # Ruby refuses a two-sided Range with a NaN end, but an endless one + # builds — and NaN is no JSON number either. + prop = property("x") { optional :x, :float, in: Float::NAN.. } + expect(prop.keys.grep(/imum/i)).to be_empty end it "carries a non-numeric Range as an extension instead of guessing" do diff --git a/spec/schema_conformance_spec.rb b/spec/schema_conformance_spec.rb index d0ef4c0..50a8cd5 100644 --- a/spec/schema_conformance_spec.rb +++ b/spec/schema_conformance_spec.rb @@ -7,12 +7,18 @@ # Where they legitimately differ, the payload says so and why. Those exemptions # are deliberately narrow, and each asserts its DIRECTION: the server may # accept what its docs reject (a client following the docs is merely -# conservative), never the reverse (a client following the docs gets a -# surprise 422). A new divergence therefore fails this spec rather than -# shipping quietly. -RSpec.describe "the exported schema against what the contract enforces" do +# conservative). The reverse (a client following the docs gets a surprise +# 422) is allowed only where JSON Schema has no keyword for the rule at all, +# under its own label, and only with the rule still visible on the field. A +# new divergence therefore fails this spec rather than shipping quietly. + +# The payload table, kept out of the example group so its constants do not +# leak onto Object from inside an RSpec block. +module SchemaConformance # A payload's expectation: :agree, or the reason the two may differ. # + # Three run the SAFE way — the server accepts what its docs reject: + # # :coerced_encoding — the runtime accepts a non-canonical encoding of the # declared type ("30" for an integer, 1 for a string) because form and # query payloads are all strings. JsonSchema documents the canonical JSON @@ -28,25 +34,37 @@ # # The rest run the OTHER way — the schema is LOOSER than the server, and a # client following it can still be surprised by a 422 — so each is labelled - # separately rather than waved through, and LOOSER names what the exported - # schema must still carry for a tool that wants to close the gap: + # separately rather than waved through, and LOOSER names the keyword the + # DIVERGING FIELD'S OWN schema must still carry, so a tool that wants to + # close the gap can find the rule: # # :extension_only — the contract enforces a bound JSON Schema has no # keyword for (`max_depth:`). The exporter does not drop the bound — it # emits it as `x-permittable-max-depth`. - # :normalized_first — the same `normalize:` step, the unsafe way round: - # " " passes a minLength of 3 and then squishes to "" (absent), " a " - # passes it and squishes to "a" (too short). JSON Schema has no - # keyword for "transform, then check", so the step is exported as + # :range_extension — a Range of non-numbers (`in: "a".."m"`), which + # minimum/maximum cannot express, carried as `x-permittable-range`. + # :custom_validation — a `validate:` proc is opaque app code, so the + # schema can only flag it: `x-permittable-custom-validation`. + # :normalized_first — the `normalize:` step, the unsafe way round: " " + # passes a minLength of 3 and then squishes to "" (absent), " a " + # passes it and squishes to "a" (too short). JSON Schema has no keyword + # for "transform, then check", so the step is exported as # `x-permittable-normalize`. - # :string_decimal — a `:decimal` is documented as string OR number, - # because the string is its precision-safe encoding, but - # minimum/maximum only ever constrain numbers: "5000" sails past a + # :format_annotation — :decimal, :date and :datetime accept a STRING, and + # what makes that string valid is its `format` ("decimal", "date", + # "date-time"), which draft 2020-12 treats as an annotation unless a + # validator opts in. So "abc", "NaN" and "2026-02-30" pass the schema + # and fail the cast. + # :string_decimal — minimum/maximum only ever constrain numbers, so a + # bounded :decimal's string encoding skips them: "5000" sails past a # `maximum` of 999.99 that the server enforces on the parsed value. The # bound is still published, as a number, for a client that parses first. LOOSER = { extension_only: "x-permittable-max-depth", + range_extension: "x-permittable-range", + custom_validation: "x-permittable-custom-validation", normalized_first: "x-permittable-normalize", + format_annotation: "format", string_decimal: "maximum" }.freeze @@ -178,6 +196,29 @@ [{ "price" => "5000" }, :string_decimal] ] }, + "a string range, a validate: proc, and formats that only annotate" => { + contract: proc { + optional :code, :string, in: "a".."m" + optional :slug, :string, validate: ->(v) { v.match?(/\A[a-z-]+\z/) } + optional :price, :decimal + optional :day, :date + optional :at, :datetime + }, + payloads: [ + [{ "code" => "b" }, :agree], + [{ "code" => "zebra" }, :range_extension], + [{ "slug" => "a-slug" }, :agree], + [{ "slug" => "Not A Slug" }, :custom_validation], + [{ "price" => "12.50" }, :agree], + [{ "price" => 12.5 }, :agree], + [{ "price" => "abc" }, :format_annotation], + [{ "price" => "NaN" }, :format_annotation], + [{ "day" => "2026-02-28" }, :agree], + [{ "day" => "2026-02-30" }, :format_annotation], + [{ "at" => "2026-02-28T10:00:00Z" }, :agree], + [{ "at" => "not a time" }, :format_annotation] + ] + }, "arrays" => { contract: proc { array :tags, of: :string, length: 1..3 }, payloads: [ @@ -227,7 +268,9 @@ ] } }.freeze +end +RSpec.describe "the exported schema against what the contract enforces" do def host_for(root:, unknown:, &contract) klass = Class.new do include Permittable @@ -251,7 +294,7 @@ def runtime_verdict(klass, payload) :reject end - CASES.each do |label, spec| + SchemaConformance::CASES.each do |label, spec| context "with #{label}" do let(:klass) { host_for(root: spec[:root] || false, unknown: spec[:unknown] || :ignore, &spec[:contract]) } let(:schema) { Permittable::JsonSchema.rule(klass.permit_rule_for("call")) } @@ -265,14 +308,14 @@ def runtime_verdict(klass, payload) expect(documented).to eq(runtime), "contract said #{runtime}, its own schema said #{documented} " \ "(#{TinyJsonSchema.errors(schema, payload).inspect}); schema: #{schema.inspect}" - elsif LOOSER.key?(expectation) + elsif SchemaConformance::LOOSER.key?(expectation) # The unsafe direction, allowed only where JSON Schema has no - # keyword that says it — and only with the rule still visible in - # the schema, so a tool that wants it can find it. + # keyword that says it — and only with the rule still visible on + # the field that diverges, so a tool that wants it can find it. expect([runtime, documented]).to eq(%i[reject accept]), "#{expectation} is for a rule the schema cannot carry; " \ "got runtime=#{runtime}, documented=#{documented}" - expect(keywords_in(schema)).to include(LOOSER.fetch(expectation)) + expect(diverging_field_schema(schema, payload)).to include(SchemaConformance::LOOSER.fetch(expectation)) else # Only ever in the safe direction: a client following the docs is # conservative, never surprised by a 422. @@ -286,7 +329,7 @@ def runtime_verdict(klass, payload) end it "covers every keyword the exporter can emit for the declarations under test" do - emitted = CASES.each_value.flat_map do |spec| + emitted = SchemaConformance::CASES.each_value.flat_map do |spec| schema = Permittable::JsonSchema.rule( host_for(root: spec[:root] || false, unknown: spec[:unknown] || :ignore, &spec[:contract]).permit_rule_for("call") ) @@ -303,6 +346,22 @@ def runtime_verdict(klass, payload) "the exporter emits #{unchecked.inspect}, which TinyJsonSchema does not check" end + # The schema of the one field a looser payload is about: descend through + # `properties` along the payload's keys while the payload names exactly one + # of them, stopping at the field whose value is not itself described + # property by property (a scalar, or an opaque `:json` object). + def diverging_field_schema(schema, payload) + loop do + raise ArgumentError, "a looser payload must name one field: #{payload.inspect}" unless payload.is_a?(Hash) && payload.size == 1 + + key, value = payload.first + schema = schema.fetch("properties").fetch(key) + return schema unless value.is_a?(Hash) && schema["properties"] + + payload = value + end + end + def keywords_in(node) return [] unless node.is_a?(Hash) From fc88567caba30b38bebe2f80b44ac06a262fafd9 Mon Sep 17 00:00:00 2001 From: Sang Date: Fri, 25 Sep 2026 15:12:03 +0700 Subject: [PATCH 3/3] Publish an integral bound exactly, at any magnitude past Float::MAX MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit optional :x, :float, in: (10**400..) dropped `minimum` entirely: the inward-nudge loop asks the field's own cast whether the published bound is honoured, and that cast runs the bound through `to_f`, which overflows a value this large to Infinity and walked the bound to Infinity too, where it was omitted like a genuinely infinite one. Master published it outright as `minimum: 10**400`, so this was a regression, not a labelled divergence: a JSON integer has no size limit, so an integral bound needs neither Float conversion nor the nudge machinery, which exists only to protect a FRACTIONAL bound from double-rounding. `json_bound` now returns an integral bound before the loop runs whenever `to_f` would overflow it. A genuinely fractional bound past Float::MAX still has no arbitrary- precision JSON representation to fall back on and stays omitted, now documented as deliberate rather than silent. Also, from the last regression pass: - Qualify the "published range can only be narrower than the enforced one" claim in the CHANGELOG/README: bounds are exact as published, but a client parsing with ordinary double-precision floats can still round a value across the boundary. That is inherent to double parsing, not an exporter bug, so no code change. - Make the CHANGELOG/README looser-divergence counts match what spec/schema_conformance_spec.rb actually asserts (LOOSER has six keys, not seven): split the "rules only an extension can carry" bullet so each of the six maps to one bullet, and say plainly that an untranslatable format: regexp has no conformance case yet, since PR #66 owns the regexp-translation rework it would depend on. - Delete the stale "No behaviour changes: this release adds tests and documentation only" line — Unreleased now contains substantial behaviour changes, this bound fix included. Co-Authored-By: Claude Sonnet 5 --- CHANGELOG.md | 14 +++++++++----- README.md | 7 ++++--- lib/permittable/json_schema.rb | 14 ++++++++++++++ spec/json_schema_spec.rb | 27 +++++++++++++++++++++++++++ 4 files changed, 54 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index de5ec0b..aef610a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,6 @@ - **A `runtime-deps` CI job that proves activesupport is the only runtime dependency.** The spec suite can't: it bundles actionpack and activerecord to exercise the integration and schema-drift paths. This job installs the *built gem* with nothing but its declared dependencies, asserts actionpack/activerecord/rails are genuinely absent, and then exercises every controller-free surface — standalone contracts (casting, defaults, `finalize`, 400/422 semantics, nested plain hashes), the concern on a plain params duck, `unknown: :error` with no logger to warn through, `sensitive:` registration with no Railtie, monitor mode, instrumentation, JSON Schema, OpenAPI assembly, and the generator. Every `respond_to?`/`defined?` guard in the gem is a promise; a missing one now fails CI. - **`spec/schema_conformance_spec.rb` — the "docs cannot drift" claim is now tested rather than argued.** Every other spec checks one side or the other; this one checks that the two **agree**, walking canonical JSON payloads through both a contract and its own exported schema and comparing the verdicts. It covers scalars and bounds, enums, exclusive and endless ranges, exact lengths, formats, arrays of scalars and of hashes, nested hashes under `unknown: :error`, rooted contracts, `nullable:` fields, and opaque `:json` fields with bounds. `spec/support/tiny_json_schema.rb` is a deliberately small validator covering exactly the keywords the exporter emits and nothing else — a measuring instrument, not a dependency — and one example asserts the exporter emits no keyword the validator silently ignores, so the two cannot fall out of step. The spec was mutation-tested: dropping `maxItems`, `additionalProperties: false`, the required-string `minLength: 1`, or the `root:` required wrapper each makes it fail. - **Every place the schema and the runtime differ is now labelled, and its direction asserted.** Two go the safe way — the server accepts what the docs reject, so a client following the docs is merely conservative: non-canonical encodings (`"30"` for an `:integer`, `1` for a `:string`, because form and query payloads are all strings), and an explicit `null` read as absence on a field that is not `nullable:`. The ones that go the other way are now stated plainly instead of being left to be discovered, starting with a `:json` field's `max_depth:`: a bound JSON Schema has no keyword for, so the published document is **looser** than the server there and an over-nested payload still earns a 422. The bound is exported as `x-permittable-max-depth` rather than dropped, and the spec asserts that extension is present. (The rest of the looser list — and a third safe case — are under **Fixed**.) A new divergence in either direction fails the suite rather than shipping quietly. -No behaviour changes: this release adds tests and documentation only. - **`permittable:generate` now reads Rails 8 `params.expect` calls, not just `params.permit`.** The generator is the adoption on-ramp, and it was blind to the syntax Rails 8 apps actually use — a modern controller has no permit calls to scan, so the draft fell back to columns alone and lost everything the app already knew about its own params (the `root:`, the permitted key list, which keys aren't columns). `params.expect(user: [:name, tag_names: [], address: [:city], line_items: [[:sku]]])` is now scanned into the same `Scan`, merged with any permit calls in the same controller. - **Arrays of hashes are drafted without a TODO when the source says so.** `expect` distinguishes what `permit` cannot: `key: [:a]` is a nested hash, `key: [[:a]]` is an array of hashes. A scanned `[[...]]` therefore drafts as `array :key do ... end` with no "this may be an array of hashes" TODO, and `Scan` carries the new `nested_arrays` member alongside `nested`. - **Route params are not mistaken for fields.** In `params.expect(:id, user: [:name])` the `:id` is a routing key, not body input, so it stays visible in a TODO rather than being drafted as a contract field — as does a second envelope (`params.expect(user: [...], address: [...])`), which belongs under a different `root:` than one rooted contract can express. A rootless `params.expect(:q, :page)` still drafts as scalars, since there is no envelope to be a sibling of. @@ -61,14 +60,19 @@ Contracts that don't opt in are byte-for-byte unaffected: the default format is - **Enum columns drafted as `:integer`**, rejecting the `"shipped"` every form sends. A Rails `enum` now drafts as `:string, in: Model.statuses.keys` — the model's own accessor rather than today's keys inlined, so adding a value cannot leave the contract behind — and a database default is shown as its enum key (`"pending"`, not `0`). Rails also assigns an integer-backed enum its stored integer (`status: 1` from a JSON client), which a `:string` field turns into a rejected `"1"`; the line carries a TODO saying how to admit it rather than guessing that clients send it. An enum whose name is not a method identifier (`first-status`) is reached as `Model.defined_enums["first-status"]`, so the draft still loads. - **The STI inheritance column and `lock_version` were drafted as client-writable fields.** Mass-assigning `type` changes which class the record loads as, which is a privilege-escalation shape, not a field. Drafted from columns alone, both are now omitted from the fields and named in a TODO explaining why — including where to declare `lock_version` if the app's forms round-trip it for stale-update detection, worded for the rules actually drafted. When the controller's own permit call lists one, it stays a field with a TODO instead: omitting a `lock_version` the app sends would switch stale-update detection off the day the draft is enforced. Each counts only when the model uses it — a `type` column with `self.inheritance_column = nil`, or `lock_version` with `self.lock_optimistically = false`, is an ordinary column and drafts as one. A model left with no field to draft (only `type` and `lock_version`, or nothing else with a contract type) drafts nothing, as a model with no columns does, rather than a rule that raises `a contract must declare at least one field` when pasted. - **Column names that are not symbol literals produced a draft that did not parse.** `first-name`, `2fa_enabled` and `Email Address` were emitted as `:first-name` and friends; names are now emitted with `Symbol#inspect` (`:"first-name"`), so the draft is valid Ruby whatever the schema. -- **A `:decimal` bounded by BigDecimals exported an invalid schema.** `in: BigDecimal("0.01")..BigDecimal("999.99")` — the natural way to bound a price — published `"minimum": "0.01", "maximum": "999.99"`, because range bounds went through the same re-encoding as an authored `default:`, which renders a BigDecimal as its precision-safe string. The metaschema requires `minimum`/`maximum` to be numbers, so the whole document failed validation. Bounds are now JSON numbers: an Integer when the bound is one, otherwise a Float. Where the nearest double would let through a value the server refuses (possible only past 15 significant digits — never a price), the bound moves **inward**, a minimum up and a maximum down, one double at a time until the field's own cast and comparison accept the most extreme value the docs admit. That check matters because the right distance depends on the type: a `:decimal` compares exactly, while a `:float` compares through BigDecimal's lossy reading of a Float and needs a few doubles more. So the published range can only be narrower than the enforced one, on either type. An infinite bound (`Float::INFINITY`, `BigDecimal("Infinity")`) means no bound and is omitted, as is a NaN one; neither is a JSON number. `default:`/`example:` keep their string encoding, which `:decimal`'s type permits. -- **Most of the places the docs are looser than the server were unlabelled.** Only `max_depth:` was listed. Now the README, the exporter's own comment and `spec/schema_conformance_spec.rb` also name these — the spec labels each, asserts its direction, and checks that the rule is still visible on the diverging field's own schema, so none can pass as agreement: +- **A `:decimal` bounded by BigDecimals exported an invalid schema.** `in: BigDecimal("0.01")..BigDecimal("999.99")` — the natural way to bound a price — published `"minimum": "0.01", "maximum": "999.99"`, because range bounds went through the same re-encoding as an authored `default:`, which renders a BigDecimal as its precision-safe string. The metaschema requires `minimum`/`maximum` to be numbers, so the whole document failed validation. Bounds are now JSON numbers: an Integer when the bound is one, otherwise a Float. Where the nearest double would let through a value the server refuses (possible only past 15 significant digits — never a price), the bound moves **inward**, a minimum up and a maximum down, one double at a time until the field's own cast and comparison accept the most extreme value the docs admit. That check matters because the right distance depends on the type: a `:decimal` compares exactly, while a `:float` compares through BigDecimal's lossy reading of a Float and needs a few doubles more. So the published range is always exact as written and can only be narrower than the enforced one, on either type — a caveat worth stating plainly: a client that parses the published JSON with ordinary double-precision floats, rather than reading the digits as sent, can still round a value across the boundary. That is inherent to how a double reads any JSON number and is not something this exporter can fix from the document side. + **An INTEGRAL bound is exact at any magnitude**, `10**400` included: a JSON number literal has no size limit, so `optional :x, :float, in: (10**400..)` now publishes `minimum: 10**400` outright, matching master, instead of vanishing. (It briefly did vanish in an earlier draft of this fix: the inward-nudge above routes a candidate bound through `to_f` to ask the field's own cast whether it is honoured, which itself overflows a bound this large to `Infinity`, walking the bound to `Infinity` and dropping it — looser than master, not a labelled divergence.) A genuinely **fractional** bound past `Float::MAX` (`BigDecimal("1e400") + BigDecimal("0.5")`) has no such escape — there is no arbitrary-precision JSON number this exporter emits without going through `Float` — and stays omitted, same as an infinite bound. + An infinite bound (`Float::INFINITY`, `BigDecimal("Infinity")`) means no bound and is omitted, as is a NaN one; neither is a JSON number. `default:`/`example:` keep their string encoding, which `:decimal`'s type permits. +- **Most of the places the docs are looser than the server were unlabelled.** Only `max_depth:` was listed. Now the README, the exporter's own comment and `spec/schema_conformance_spec.rb` also name the other five — `LOOSER` in the spec, six keys in all — the spec labels each, asserts its direction, and checks that the rule is still visible on the diverging field's own schema, so none can pass as agreement: - **`normalize:` runs before the checks.** With `length: 3..10, normalize: :squish`, `" "` passed the docs and was `missing` on the server, and `" a "` passed them and failed `length`. The step was not exported at all; it is now `x-permittable-normalize` — the preset's name, or `true` for a custom proc. (It also runs the safe way — `" abcdefghij "` is too long for the docs and fine once squished — which is listed with the safe cases.) - **A bounded `:decimal` sent as a string** skips `minimum`/`maximum`, which constrain only numbers, so `"5000"` passed the docs for a bound the server enforces. - **Strings whose validity is a `format`.** `"abc"` or `"NaN"` for a `:decimal`, and `"2026-02-30"` for a `:date` or `:datetime`, pass the docs — `format` is an annotation in draft 2020-12 — and fail the cast. - - **Rules only an extension can carry:** a `validate:` proc (`x-permittable-custom-validation`), a Range of non-numbers (`x-permittable-range`), and an untranslatable `format:` regexp (`x-permittable-pattern`). + - **A `validate:` proc** is opaque app code, so the schema can only flag it — `x-permittable-custom-validation` — never enforce what it checks. + - **A Range of non-numbers**, such as `in: "a".."m"`, has no `minimum`/`maximum` equivalent and rides along as `x-permittable-range` instead. - No runtime behaviour changes. + An untranslatable `format:` regexp (`x-permittable-pattern`) is a seventh case in the same spirit — a value it would refuse still passes the docs — but it is **not** one of the spec's six: [#66](https://github.com/VSN2015/permittable/pull/66), reworking the Ruby → ECMA-262 `pattern` translation itself, owns adding its conformance case, so this PR only lists it as a known gap rather than asserting a direction for code it does not touch. + + No runtime behaviour changes to the request/response path — this is what the schema *says*, not what the server enforces. ## 0.8.0 (2026-09-19) diff --git a/README.md b/README.md index 0854147..5a75e08 100644 --- a/README.md +++ b/README.md @@ -991,11 +991,12 @@ Six cases go the other way, and are worth knowing before you hand the document t - **Bounds JSON Schema has no keyword for.** A `:json` field's `max_depth:` is enforced by the server but cannot be written as a JSON Schema keyword, so the published document is **looser** there and an over-nested payload still earns a 422. The bound is not dropped — it is exported as `x-permittable-max-depth` — so a generator or linter that wants it can read it. - **`normalize:` runs before the checks.** The server validates the *normalized* string, and JSON Schema has no keyword for "transform, then check". With `required :name, :string, length: 3..10, normalize: :squish`, `" "` passes the docs' `minLength: 3` and then squishes to `""` — absent, so `missing` — and `" a "` passes them and squishes to `"a"`, which is too short. The step is exported as `x-permittable-normalize` — the preset's name (`"squish"`, `"email"`, …), which a client can apply before validating, or `true` for a custom proc. -- **A bounded `:decimal` sent as a string.** A `:decimal` is documented as `["string", "number"]`, because the string is its precision-safe encoding, but `minimum`/`maximum` constrain only numbers — so `"5000"` passes the docs for `in: BigDecimal("0.01")..BigDecimal("999.99")` and the server answers `inclusion`. The bound is still published, as a JSON number, for a client that parses the string first. -- **Rules only an extension can carry.** A `validate:` proc is app code, so the schema can only flag it (`x-permittable-custom-validation`); a Range of non-numbers such as `in: "a".."m"` has no keyword and rides along as `x-permittable-range`; and a `format:` regexp that does not translate to ECMA-262 is published as `x-permittable-pattern` rather than as a `pattern` that would enforce something else. A value any of these refuses passes the docs. +- **A bounded `:decimal` sent as a string.** A `:decimal` is documented as `["string", "number"]`, because the string is its precision-safe encoding, but `minimum`/`maximum` constrain only numbers — so `"5000"` passes the docs for `in: BigDecimal("0.01")..BigDecimal("999.99")` and the server answers `inclusion`. The bound is still published, as a JSON number, for a client that parses the string first. (Numbers are published exactly as written, at any magnitude — `10**400` included, since a JSON integer has no size limit — but a client that reads the document back with ordinary double-precision floats, rather than the digits as sent, can still round a value across a boundary. That is inherent to parsing any JSON number as a double, not something this exporter controls.) +- **A `validate:` proc.** It is opaque app code, so the schema can only flag it — `x-permittable-custom-validation` — never enforce what it checks. A value the proc refuses still passes the docs. +- **A Range of non-numbers.** `in: "a".."m"` has no `minimum`/`maximum` equivalent (those constrain numbers only) and rides along as `x-permittable-range` instead. A value outside it still passes the docs. - **Strings whose validity is a `format`.** `:decimal`, `:date` and `:datetime` are sent as strings, and what makes such a string valid is its `format` (`"decimal"`, `"date"`, `"date-time"`) — which draft 2020-12 treats as an annotation unless a validator opts into asserting it. So `"abc"` or `"NaN"` for a `:decimal` and `"2026-02-30"` for a `:date` pass most validators and fail the server's cast with `invalid_type`. -Everything else the exporter cannot translate stays visible as an `x-permittable-*` extension rather than being guessed at. +A `format:` regexp that does not translate to ECMA-262 is looser in the same way — it publishes as `x-permittable-pattern` rather than a `pattern` that would enforce something else, so a value it refuses still passes the docs — but it is not one of the six above: `spec/schema_conformance_spec.rb` does not yet assert a case for it, since the Ruby → ECMA-262 translation it would depend on is being reworked separately. Everything else the exporter cannot translate stays visible as an `x-permittable-*` extension rather than being guessed at.
diff --git a/lib/permittable/json_schema.rb b/lib/permittable/json_schema.rb index aa57036..0be04cc 100644 --- a/lib/permittable/json_schema.rb +++ b/lib/permittable/json_schema.rb @@ -229,11 +229,25 @@ def apply_in!(schema, allowed, type:) # would also accept a few doubles beyond the nearest one, those stay # unpublished.) A decimal of up to 15 significant digits — every price — # round-trips through a double, so it is emitted as written. + # + # An INTEGRAL bound past Float::MAX (`10**400`) needs none of this: a + # JSON number literal has no size limit, so it is exact as published, + # with no Float rounding to guard against in the first place. Asking the + # server would instead break it — the field's own cast runs the bound + # through `to_f`, which overflows a value this large to Infinity — so + # the loop below walked the bound to Infinity and dropped it, though + # master published it as `minimum: 10**400` outright. It is returned + # here before the loop runs. A FRACTIONAL bound past Float::MAX has no + # such escape (there is no arbitrary-precision JSON number this exporter + # emits without going through Float) and stays omitted, same as a + # genuinely infinite bound — see CHANGELOG. def json_bound(range, keyword, type) value = keyword == "minimum" ? range.begin : range.end return nil unless value.finite? bound = value.is_a?(Float) || value != value.to_i ? value.to_f : value.to_i + return bound if bound.is_a?(Integer) && !bound.to_f.finite? + step = keyword == "minimum" ? :next_float : :prev_float # A fractional bound past Float::MAX converts to Infinity, which no # step moves; it is then as unrepresentable as an infinite one. diff --git a/spec/json_schema_spec.rb b/spec/json_schema_spec.rb index 6e8eb7c..094a7ff 100644 --- a/spec/json_schema_spec.rb +++ b/spec/json_schema_spec.rb @@ -133,6 +133,33 @@ def server_accepts?(field_rule, value) expect { JSON.generate(float) }.not_to raise_error end + it "publishes an integral bound beyond Float::MAX exactly, matching master" do + # 10**400 has no double — `to_f` overflows it to Infinity — but a JSON + # integer literal has no size limit, and master published it outright: + # `minimum: 10**400`. The inward-nudge machinery routes a candidate + # bound through `to_f` to ask the field's own cast whether it is + # "honoured", which itself overflows to Infinity for a bound this + # large and used to walk the bound to Infinity and drop it — looser + # than master, not merely a labelled divergence. + huge = 10**400 + expect(property("x") { optional :x, :float, in: huge.. }).to include("minimum" => huge) + expect(property("x") { optional :x, :float, in: ..(-huge) }).to include("maximum" => -huge) + expect(property("x") { optional :x, :decimal, in: BigDecimal("1e400").. }).to include("minimum" => huge) + end + + it "omits a FRACTIONAL bound beyond Float::MAX, unlike an integral one" do + # BigDecimal("1e400") + 0.5 has no double either, but unlike an + # integral bound it has no arbitrary-precision JSON representation + # this exporter emits without going through Float — publishing it + # exactly would mean a raw decimal number literal rather than a Ruby + # Integer/Float, which this exporter does not produce. It is omitted, + # same as a genuinely infinite bound, and deliberately so (see + # CHANGELOG) rather than silently. + fractional = BigDecimal("1e400") + BigDecimal("0.5") + prop = property("x") { optional :x, :decimal, in: fractional.. } + expect(prop.keys.grep(/imum/i)).to be_empty + end + it "omits a NaN bound, which compares to nothing" do # Ruby refuses a two-sided Range with a NaN end, but an endless one # builds — and NaN is no JSON number either.