Skip to content

Cast in: members with the field's type, and refuse a String in:, at class load - #65

Merged
VSN2015 merged 7 commits into
masterfrom
fix/in-checked-at-class-load
Sep 25, 2026
Merged

VSN2015 merged 7 commits into
masterfrom
fix/in-checked-at-class-load

Conversation

@VSN2015

@VSN2015 VSN2015 commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

A bug fix. Two in: defects, both reproduced, fixed together because they live in the same class-load check.

The bugs

1. in: members were compared as authored, against a cast value. The runtime casts the request value first, then asks in: whether it includes that value. So with in: %i[draft published] on a :string field, "draft" was compared with :draft, and every request got inclusion. Meanwhile the exported JSON Schema, which stringifies Symbols, published "enum": ["draft", "published"]: the docs listed the exact values the server refused. in: %w[1 2 3] on an :integer rejected every value in the same way. assert_satisfiable! didn't catch either one.

2. in: only had to answer include?. A String answers it with a substring test, so in: "free pro" accepted "e", "fr" and "ee p" as plans.

The fix

optional :status, :string,  in: %i[draft published]   # now accepts "draft" / "published"
optional :n,      :integer, in: %w[1 2 3]              # now accepts "2" and 2; enum is [1, 2, 3]
optional :plan,   :string,  in: "free pro"             # now fails at class load
  in: "free pro"               Permittable: :in for field :plan must be a Range, a list of values (an Array, Set, or a Hash read as its keys), or an object answering include? (got "free pro") — String#include? would accept any substring; list the values instead, e.g. in: %w[free pro]
  in: %w[1 two]  (:integer)    Permittable: :in for field :n contains "two", which is not a valid :integer (invalid_type)
  in: [nil, "pro"]             ... contains nil, which is not a valid :string (invalid_type) — an absent value never reaches in:; declare nullable: true to accept an explicit null
  in: "1".."5"   (:integer)    Permittable: :in for field :n is a Range of String ("1".."5"), which a :integer value cannot be compared with — no value could satisfy it; write the bounds as :integer values
  • in: is a Range, a list, or any other object that answers include?. Only a String, and anything that is neither a Range nor answers include?, is refused.
  • Coercion.in_list decides what counts as a list: only a plain Array, Set, Hash or Enumerator — never a subclass overriding include?. case allowed; when Hash ... dispatches with ===, which for a Class is is_a?, so a subclass first matched its ancestor's branch; a Hash/Array/Set subclass overriding include? for its own matching logic (a case-insensitive allowlist, a fuzzy Set, a registry) had that override silently discarded, read for its raw keys/elements instead — inverting which values it actually accepted, with no error at class load. Only an unoverridden include? now counts as a list. ActiveSupport::HashWithIndifferentAccess is the one exception kept anyway: its own override just canonicalises the argument before the same key lookup, and it's what Post.statuses actually returns. A Hash's keys and a Set are stored as a frozen Set, so membership stays O(1). An Enumerator (Lazy included, which doesn't override include?) is forced to an Array once at class load.
  • Each list member goes through Coercion.cast with the field's own type, the same cast a request value gets. The differences:
    • a Symbol is read as its String, since no request carries one;
    • a Time/DateTime on a :date field is read as its UTC date only when it's exactly midnight UTC, the one instant ActiveSupport ever found equal to a date. Any other instant raises;
    • normalize: isn't applied;
    • a nil member is dropped on a nullable: field.
  • The cast members are stored frozen, so request-time matching doesn't change at all.
  • Any other object answering include?, Enumerable or not, is stored exactly as given. That covers an app's case-insensitive allowlist or a DB-backed registry: it is never enumerated at class load and never replaced by an exact-match copy. It isn't cast, and it's exported as x-permittable-custom-validation rather than as an enum it can't list.
  • A list is a snapshot taken at class load. A later PLANS << "gold" isn't seen, where it used to be; an app that needs a live list passes its own include? object. The CHANGELOG lists this as a change.
  • Coercion.in_list and Coercion.cast_in_members are the only functions behind the builder and the matcher's within chain, so the two can't drift apart.

The decision I'd most like reviewed: Ranges are checked, not cast

Casting list members is straightforward. Casting Range endpoints isn't, because it changes what the bound means:

  • 0..Float::INFINITY on a :float and 1.5..3 on an :integer are real bounds whose endpoints no cast accepts. Casting them would turn working contracts into boot failures.
  • On a :decimal, 0..100 would get BigDecimal endpoints, and json_value exports those as the string "0.0", where minimum needs a number.

So a Range is kept exactly as written. What makes a Range reject every request is an endpoint that can't be compared with the cast value, and that's what gets refused. A probe value of the field's type is compared the way cover? itself compares (begin <=> value, then value <=> end). Whatever the host's own <=> allows is therefore allowed too: ActiveSupport lets Date compare with Time, so a Date range bounding a :datetime still loads, and a spec pins that down.

Exported members are ones the server accepts

A :date/:datetime member written as a String is exported as written, and a Time/DateTime/TimeWithZone member keeps as many fractional-second digits as it has (up to nine; this is in json_value, so an exported :datetime default:/example: gets it too). Re-encoding the cast Time printed whole seconds, so "2026-09-05T10:00:00.25Z" was published as "…10:00:00Z", a value the server refuses. The String went through the same cast a request does, so the server accepts it by construction. A spec sends every exported member back through the contract, and a conformance case covers the fractional-seconds instant. The authored forms live in field[:in_published], which is set only where they differ from the cast members. Matching still uses the cast values.

Agreement with #57's enum rule

check_column_types reads the stored in:. It therefore sees in: Post.statuses as the enum's names, and in: %i[pending shipped] as Strings, and accepts both. A spec pins this in the enum context.

Also

  • Arrays: in: isn't in ARRAY_OPTS, so arrays reject it today. This PR keeps that and does not add element-level in:.
  • Lists keep their container: a Set stays a Set (for an author who picked it for O(1) include?), and duplicates created by the cast (["1", 1, "01"] on :integer) are removed, because JSON Schema enum items should be unique.
  • The RSpec matcher: within(...) casts its argument with the same function, so within(%i[draft published]) can repeat the declaration as written, and within([1, 2, 3]) names what the runtime actually holds. The comparison uses the cast values, but a failure message shows the argument as the spec wrote it.
  • One exported-docs change for lists that already worked: the enum now uses the field's own encoding. in: [1.5] on :decimal exports "1.5", the same precision-safe string its default: already exports. in: [1, 2] on :float exports 1.0, 2.0.

This round: a Hash/Array/Set subclass's own include? was silently discarded

A regression from the round before this one. Coercion.in_list dispatched with case allowed; when Hash ..., which matches with === — is_a? for a Class — so a Hash/Array/Set subclass overriding include? (a case-insensitive allowlist, a fuzzy Set, a registry with its own matching logic entirely) matched its ancestor's branch anyway, and was read for its raw keys/elements instead of using its override:

class Registry < Hash
  def include?(v) = v.to_s.start_with?("custom-")
end
in: Registry[foo: 1]   # was silently read as Set["foo"] — the override never ran

Fixed by checking the object's own include?, not just its ancestry: Set/Array need instance_of?, and Hash needs instance_of?(Hash) || instance_of?(HashWithIndifferentAccess) (the one subclass override that's semantically the same lookup). Enumerator needs its include?'s method owner to still be Enumerable, since Enumerator::Lazy — which must keep working — never overrides it, but a hand-rolled Enumerator subclass that does now stays opaque too. New specs cover a Hash, an Array and a Set subclass each overriding include?, and confirm the override runs and nothing is cast.

Merging in master's #60 (numeric JSON-number bounds) surfaced one more interaction worth naming: this branch's own assert_comparable_range! (added two rounds ago, to catch a wrong-typed Range like a String range on an :integer) treated a NaN endpoint the same way <=> always returns nil for anything against NaN. That would have raised at class load for in: Float::NAN.., a Range #60's own new export logic explicitly allows to load and simply omits from the schema (never finite?). A NaN endpoint isn't evidence of a wrong-typed bound, so it's now left alone exactly as an infinite endpoint already was — a spec pins in: Float::NAN.. (:float) and in: ..BigDecimal("NaN") (:decimal) loading without error.

Compatibility

The cast only loosens what a request is judged against. No request that was accepted before is refused now. The declarations below used to boot and now fail at class load, because something in each could never match. A list's other, valid members did match before (in: [1, 2, "three"] was partly working), so only the Range case rejected everything:

  • a String in:, which matched substrings
  • a list member the field's type can't cast, including nil on a field that isn't nullable: and a non-midnight-UTC Time on a :date field
  • in: [nil] on a nullable: field, which is empty once the nil is dropped
  • a Range whose endpoints the field's values can't be compared with

Verification

🤖 Generated with Claude Code

Sang and others added 3 commits September 24, 2026 19:27
… class load

`in: %i[draft published]` on a :string field compared the cast String
against Symbols and rejected every request, while the exported enum
advertised "draft"/"published"; `in: %w[1 2 3]` on :integer did the same.
List members are now cast by the field's own type at class load and
stored cast, so the runtime, the JSON Schema enum and the RSpec matcher's
`within` read one list. A member no cast accepts fails at boot.

`in:` only had to answer include?, which let a String through as a
substring test (`in: "free pro"` accepted "e"). It must now be a Range or
a non-Hash Enumerable. A Range stays as written, but is refused when its
endpoints cannot be compared with the field's values.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Review follow-ups on the class-load in: check:
- A Hash in: (the Rails enum idiom, in: Post.statuses) is read as its
  keys, cast like any list, so check_column_types' enum rule sees names.
- An object that only answers include? is kept as given: not cast, and
  exported as x-permittable-custom-validation instead of an enum.
- Lists, Enumerator::Lazy included, are forced to an Array at class load,
  so casting never runs per request.
- A Time/DateTime member of a :date field is read as its date.
- A nil member is dropped on a nullable field.
- A String-authored :date/:datetime member is exported as written, so the
  enum never lists a value the server refuses.
- The matcher shares the contract's list predicate (Coercion.in_list).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@VSN2015

VSN2015 commented Sep 25, 2026

Copy link
Copy Markdown
Owner Author

Addressed the review in 00fffce, after merging master (which includes #57):

  1. A Hash in: is read as its keys and cast like a list, so in: Post.statuses works and passes Accept a Rails enum declared by its names in the column-type guard #57's enum rule.
  2. An object that only answers include? is kept as given. It isn't cast, and it's exported as x-permittable-custom-validation. Only a String is refused.
  3. A String-authored :date/:datetime member is exported as written, and a spec sends every exported member back through the server.
  4. Lists, including Enumerator::Lazy, are forced to an Array at class load.
  5. A Time or DateTime member of a :date field is converted with to_date.
  6. A nil member is dropped on a nullable: field. Elsewhere it still raises.
  7. The matcher shares Coercion.in_list and Coercion.cast_in_members with the contract.

The CHANGELOG's list of newly failing declarations is now exact. in: 5 failed on master already, and in: [nil], nullable: true is newly empty, so it's added to the list. 666 examples, 0 failures. RuboCop is clean.

🤖 Generated with Claude Code

@VSN2015 VSN2015 changed the title Cast in: members with the field's type, and refuse a non-list in:, at class load Cast in: members with the field's type, and refuse a String in:, at class load Sep 25, 2026
Regression-pass follow-ups on the class-load in: check:
- Only Array, Set, Hash (keys) and Enumerator are lists to cast. Any
  other object answering include?, Enumerable or not, is used as given,
  so an app's case-insensitive allowlist or DB-backed registry keeps its
  own include? and is never enumerated at class load.
- A Hash's keys (and a Set) are stored as a frozen Set: O(1) membership.
  The matcher compares lists as sets accordingly.
- A Time/DateTime member of a :date field is read as its UTC date only
  at exactly midnight UTC, the one instant that ever equalled a date;
  any other fails at class load.
- An exported Time keeps its fractional-second digits (up to nine), so
  every exported :datetime member is one the server accepts.
- CHANGELOG: the class-load snapshot of an Array is listed as a change,
  and the newly-failing declarations are described precisely.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@VSN2015

VSN2015 commented Sep 25, 2026

Copy link
Copy Markdown
Owner Author

Regression pass addressed in cc1ee81:

  1. Only Array, Set, Hash (keys) and Enumerator count as lists to cast. Any other object answering include?, Enumerable or not, is used as given, never enumerated, and exported as x-permittable-custom-validation.
  2. The Array snapshot at class load is kept, and the CHANGELOG now lists it as a change, with the include? object as the way to get a live list.
  3. The CHANGELOG now says newly failing declarations fail because a member could never match, and that their valid members did work before.
  4. Exported Time/DateTime/TimeWithZone values keep their fractional digits (iso8601(n), n ≤ 9). A spec checks that the server accepts each exported member.
  5. A Time/DateTime member of a :date field becomes its UTC date only at exactly midnight UTC, matching master's instant comparison. Any other raises at class load.
  6. A Hash's keys, and a Set, are stored as a frozen Set. The matcher compares lists as sets.

669 examples, 0 failures. RuboCop is clean. Pushed as a fast-forward.

🤖 Generated with Claude Code

Sang and others added 2 commits September 25, 2026 15:30
case/when dispatch on the collection's own class used === (is_a? for a
Class), so a subclass overriding include? for its own matching logic
(a case-insensitive allowlist, a fuzzy Set, a registry) matched its
ancestor's branch and had its override silently discarded, read for
its raw keys/elements instead — inverting which values it accepts,
with no error at class load.

Only a plain Array, Set, Hash, or Enumerator (an include? that is not
overridden) is now read as a list. ActiveSupport::HashWithIndifferentAccess
is kept as one anyway, since its own override just canonicalises the
argument before the same key lookup and it is what a Rails enum's own
reader (Post.statuses) actually returns.

Also tightens the README/CHANGELOG wording that implied cover? duck-
typing on a non-Range in:, which resolve_in! never did.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves CHANGELOG/README conflicts (kept both sides' entries; master's
CHANGELOG entries first) and merges apply_in!'s numeric-bound-safe
rework (#60) with this branch's opaque/in_published in: handling.

Also reconciles an interaction the merge surfaced: assert_comparable_range!
(added on this branch, to catch a wrong-TYPED Range like a String range
on an :integer) treated a NaN endpoint the same way, since NaN's <=>
always returns nil — which would have raised at class load for
`in: Float::NAN..`, a Range #60's own new export logic explicitly
allows to load (and simply omits from the schema, since it's never
`finite?`). A NaN endpoint is not evidence of a wrong-typed bound, so
it is left alone here exactly as an infinite endpoint already is.

# Conflicts:
#	CHANGELOG.md
#	README.md
#	lib/permittable/json_schema.rb
#	spec/json_schema_spec.rb

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@VSN2015

VSN2015 commented Sep 25, 2026

Copy link
Copy Markdown
Owner Author

Final regression addressed in 77890ec:

A Hash/Array/Set subclass overriding include? (a case-insensitive allowlist, a fuzzy Set, a registry) was silently swept into the cast-and-freeze list path, since case allowed; when Hash ... dispatches with === (is_a? for a Class). Its override was discarded and its raw contents used instead, inverting which values it accepted. Fixed by checking the object's own include?, not just its ancestry:

  • Set/Array need instance_of? exactly.
  • Hash needs instance_of?(Hash) || instance_of?(HashWithIndifferentAccess) — the one subclass whose override is the same key lookup, and what Post.statuses actually returns.
  • Enumerator needs its include?'s method owner to still be Enumerable, so Lazy (which never overrides it) still works, but a hand-rolled Enumerator subclass that does now stays opaque too.

New specs: a Hash, an Array and a Set subclass each overriding include?, confirming the override runs and nothing is cast — in the contract, the export, and the matcher.

Merging master's #60 also surfaced one interaction worth a small extra fix: this branch's assert_comparable_range! treated a NaN Range endpoint as an incomparable type (since <=> with NaN is always nil), which would have raised at class load for in: Float::NAN.. — a Range #60 explicitly allows to load and simply omits from the schema. NaN is now left alone there, like an infinite endpoint already was, with a spec pinning it.

775 examples, 0 failures, on current master (#57/#59/#60/#61/#64/#66 all included). RuboCop clean. Fast-forward push.

Also fixed the flagged doc-precision issue: README/CHANGELOG said in: fails when it "answers neither cover? nor include?," which overstated what resolve_in! actually checks (only Range gets special-cased; nothing else is duck-typed on cover?). Reworded to "neither a Range nor answers include?".

🤖 Generated with Claude Code

CHANGELOG only: keep master's entries ahead of this PR's.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@VSN2015
VSN2015 merged commit 4da71e8 into master Sep 25, 2026
16 checks passed
@VSN2015
VSN2015 deleted the fix/in-checked-at-class-load branch September 25, 2026 08:42
VSN2015 pushed a commit that referenced this pull request Sep 25, 2026
Master had moved to include #59-#61, #64 and #66 since this branch's last
merge, plus #60 and #65 which both touch json_schema.rb's :decimal/:datetime
export and lib/permittable/rspec.rb's `in:`/`default:` matcher chains —
real conflicts, not just additive.

lib/permittable.rb: kept this branch's permittable_transform seam
(AuthoredValues needs it to walk a default without running transform: on
it) — same guard (`violations.length == before`) as master's inline form.

lib/permittable/json_schema.rb (4 hunks): combined rather than picked a
side.
- apply_in!'s enum export now uses BOTH #65's `field[:in_published] ||
  allowed` (keeps a :date/:datetime member's authored String form) AND
  this branch's `decimal: :number` (keeps a :decimal member numeric and
  consistent with its own default/example export).
- kept #60's `apply_normalize!` method (unrelated to this branch) ahead of
  json_value, whose signature this branch changes to `decimal: :string`.
- json_value's Hash/BigDecimal/Time cases: kept this branch's `decimal:`
  threading for Hash/BigDecimal, and #65's `exact_iso8601` for Time (fixes
  sub-second Time :in members exporting as whole seconds — this branch's
  plain `.utc.iso8601` would have regressed that).
- kept both decimal_json (this branch) and exact_iso8601 (#65) methods;
  each is used from the merged json_value body above.
Verified with a probe: a :decimal in: list now exports numerically AND
agrees with its own numeric default, and a :datetime in: list with
sub-second members still exports the authored string via in_published —
both at once, which neither branch alone tested.

lib/permittable/rspec.rb (2 hunks) and spec/matchers_spec.rb: purely
additive — #65's `cast_in`/`same_in?`/`within` alongside this branch's
`default_mismatch`/`cast_default`. Combined by keeping both.

README.md: combined the `default:` row (this branch, the transform:
exception) with the `validate:` row (master, the array-skip-on-failed-
validate note) — same table, different rows each PR had touched.

Fixed along the way (found while merging, not part of either PR):
- spec/permittable_spec.rb: a #61 perf spec asserted `valid_encoding?` is
  called on the caller's OWN string object, but this branch's
  permittable_own copies a request's String before Coercion.cast ever sees
  it (to avoid aliasing params) — so the original object is never touched,
  though the copy is (correctly) scanned exactly once. Rewrote the spec
  to count via a shared counter that survives `dup`, so it asserts the
  real invariant (one scan overall) rather than one specific object's
  identity.
- spec/matchers_spec.rb: my own conflict resolution (both PRs inserted a
  new `it` block at the same point in the file) left one block's closing
  `end` missing — a syntax error that silently dropped all 42 examples in
  this file from the suite total without failing the run. Fixed; the
  suite total is now 819 (was silently 777).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant