Skip to content

LIVR 2.0: pass the full LIVR test suite, add aliased rules - #11

Open
koorchik wants to merge 7 commits into
erlangbureau:masterfrom
koorchik:livr-2.0-conformance
Open

koorchik wants to merge 7 commits into
erlangbureau:masterfrom
koorchik:livr-2.0-conformance

Conversation

@koorchik

Copy link
Copy Markdown

Hi @erlangbureau!

Thank you for liver! It's one of the two Erlang implementations listed on livr-spec.org, and its livr_spec rule set together with the JSON Schema and OpenAPI support makes it a really useful library.

The LIVR test suite was extended recently, and liver 1.1.0 fails 9 of its 70 cases (see #10): equal_to_field, email, or, and the six aliased-rule cases, because liver had no way to register aliased rules. This pull request fixes them, so liver passes the full LIVR 2.0 test suite (70/70). It adds a small API for aliased rules, fixes a return => proplist bug found on the way, and adds a rebar3 job to CI.

About this pull request: it was prepared with the help of an LLM (Claude Code) and then reviewed and tested: the LIVR conformance harness, the repository's tests on OTP 25 and 29 in Docker (with both make and rebar3), dialyzer, and a speed comparison with master (schemas without aliases run as fast as before). Still, please look through it before merging. You know the code and its users best, and I'm happy to change anything, rename the new API, split the PR, or drop parts you don't want. The commits are grouped by topic, so it may be easiest to review commit by commit.

What would help: if it looks good to you, merge it and publish a new release to hex (1.2.0, as the aliases API is new). The LIVR conformance run installs liver from hex, so once the release is out I'll update the LIVR harness to pass aliases through the new aliased_rules option, and #10 will close by itself after the next run.

What's changed

LIVR test suite

  • tests/cases/livr re-imported with scripts/livr_import.erl from koorchik/LIVR@4384f09e (LIVR 2.0). Only generated files change; running the importer on that commit reproduces them byte for byte.
  • The importer also imports the aliases_positive / aliases_negative suites (aliases go to opts => #{aliased_rules => [...]}), and livr_rules_SUITE runs them. livr_spec_check compares these suites with upstream too.

livr_spec fixes for LIVR 2.0

  • equal_to_field compares the two values as strings, so <<"10">> equals 10.
  • email: the pattern had "\s" in an Erlang string, which is a plain space, so tabs and newlines (also a trailing newline) passed. It now rejects whitespace outside quotes and @ inside a quoted local part, like the LIVR reference pattern.
  • 'or' also runs on a missing field. {'or' => [[required, ...], [required, ...]]} returned ok for a missing field and now returns REQUIRED. A branch without required that passes still leaves the field out.

Aliased rules (new)

  • liver:add_aliased_rule/1 registers a LIVR aliased rule as #{name, rules, error}. Keys can be atoms or binaries, so an alias decoded from JSON works as is. The aliased_rules option of validate/3 takes a list of the same terms for a single call, and these win over registered ones.
  • The alias rules use the rule set of the call (livr_spec or erlang_standard), and their output becomes the field value. If error is set, it replaces their errors. Aliases work at the top level, inside nested rules and inside other aliases, and they run on a missing field. Documented in the README and doc/livr_rules.md.
  • Registered aliases are kept in a persistent_term (register them at start-up). The aliases of a call are resolved once, when its options are normalized. Without any aliases, the options and the rule path stay as before, so schemas that don't use aliases run at the same speed.

Other fix

  • return => proplist combined with nested_list, list_of, 'or' or a list as the root schema returned format_error, or crashed with a case_clause when the list was the root schema. validate_term/3 now reads the result with liver_maps:get/2.

CI

  • New rebar3 job: rebar3 ct --dir tests on OTP 25 with rebar3 3.24.0 (the last rebar3 release for OTP 25) and on OTP 29 with rebar3 3.28.0.
  • actions/checkout v4 → v7 in the coverage job (v4 runs on the deprecated Node 20).

Behaviour changes

  • livr_spec equal_to_field: <<"10">> and 10 now count as equal. Floats are formatted the Erlang way, so 10.0 still does not equal <<"10">>.
  • livr_spec email: addresses with tabs, newlines, a backslash in the dotted local part, or @ inside quotes are now rejected.
  • 'or' on a missing field now runs its branches, so it can return REQUIRED or apply a default from a branch. A default placed before 'or' is now validated by the 'or' branches. With erlang_standard, which has no 'or' rule, a schema that uses 'or' now returns {unimplemented_rule, 'or'} for a missing field too, as it already did for a present one. A custom rule registered under the name 'or' now also runs on missing fields.
  • Aliases share the rule namespace: an alias named like a built-in rule (for example email) replaces that rule for every call in the VM, as add_rule/2 does.

How it was tested

  • Docker erlang:25 and erlang:29: make tests passes: aliased_rules 10/10, erlang_standard 9/9, json_schema 11/11, liver_internals 8/8, livr_rules 140/140, openapi_schema 13/13.

  • rebar3 ct --dir tests: 191 tests pass on erlang:25 with rebar3 3.24.0 and on erlang:29 with rebar3 3.28.0.

  • make dialyze on erlang:29 passes.

  • actionlint (rhysd/actionlint:latest): no findings.

  • livr_import:run/3 on koorchik/LIVR@4384f09e (erlang:27): its output is identical to tests/cases/livr, both at the sync commit (64 cases) and at the aliases-import commit (70 cases).

  • LIVR conformance harness (all 70 cases of the LIVR 2.0 suite, aliases passed through the aliased_rules option): 70/70.

  • Speed (erlang:27, ms per 100k liver:validate/3 calls, best of 15 runs, median of 6 interleaved runs):

    Schema origin/master before "Add LIVR aliased rules" this branch
    livr_spec, 5 fields, no aliases 496 521 522
    erlang_standard, 3 fields, no aliases 74 76 74
    livr_spec, same schema, one field uses an alias from the aliased_rules option – – 594
    livr_spec, aliased_rules option given but no field uses it – – 537

    The aliases commit costs nothing measurable without aliases. The ~5% on the livr_spec schema comes from the stricter email pattern in the LIVR 2.0 fixes; the schema has an email field.

  • Each commit was built and tested on its own. At the first two commits (the suite sync and the return => proplist fix), 6 livr_rules cases fail (equal_to_field, email, or, each in the maps and proplists forms). The livr_spec fix commit makes them pass.

Notes for the maintainer

  • Release: the CHANGELOG Unreleased section lists the changes. The new API (add_aliased_rule/1, aliased_rules) is a minor-version feature (1.2.0). Nothing breaks for erlang_standard users.
  • API shape: add_aliased_rule/1 takes the LIVR alias term (like registerAliasedRule in the JS and Perl validators), not positional arguments like add_rule/2. That way aliases stored as JSON can be loaded as they are. Registered aliases live in a persistent_term, not in the application env like add_rule/2 rules, because every validate/3 call reads them.
  • Not changed (already like this before): on a missing field only required-like rules run, so in [{default, X}, integer] the default value is not checked by integer (in JS it is). Compared with the JS email regex, \s here matches ASCII whitespace only.
  • The rebar3 CI job is optional extra coverage. Drop it if one build system in CI is enough. The coverage job uploads to Coveralls with COVERALLS_REPO_TOKEN, which is not available to pull requests from forks. That was already the case before this PR.

Thanks again for your work on liver!

Re-imported with scripts/livr_import.erl (make livr-spec-import) from
koorchik/LIVR@4384f09e; only the generated files under tests/cases/livr
change. New cases: equal_to_field on a numeric string vs a number,
emails with a newline, and 'or' with required in every branch. They
fail until the livr_spec fixes that follow.
validate_term/3 matched the result of validate_map/3 as a map, but with
return => proplist it is a proplist. So nested_list, list_of, 'or' and a
list as the root schema returned format_error (the case_clause was
caught by the rule runner) or crashed when called from validate/3.
Read the field with liver_maps:get/2, which handles both forms.
- equal_to_field: LIVR 2.0 compares the two values as strings, so
  <<"10">> is equal to 10 (positive/17-equal_to_field).
- email: the pattern had "\s" in an Erlang string, which is a plain
  space, so tabs and newlines passed, and $ matched before a trailing
  newline. Use the regex class \s, dollar_endonly, and the quoted local
  part of the LIVR reference pattern (no @ or newline inside the
  quotes); the dotted local part now excludes a backslash as well, as
  in the reference pattern (negative/16-email).
- 'or': a missing field only ran required, not_empty_list and default,
  so {'or': [[required, ...], [required, ...]]} passed without a value
  (negative/29-or). 'or' now runs on a missing field too, and
  validate_term/3 validates each branch as an absent field: only its
  required-like rules run, and a branch that passes leaves the field
  out.
liver:add_aliased_rule/1 registers a rule made of other rules, given
as a map with name, rules and error (atom or binary keys, so an alias
decoded from LIVR JSON works as is). The aliased_rules option of
validate/3 takes the same aliases for one call and wins over registered
ones.

The alias rules validate the value like a field of their own, with the
rule set of the call (livr_spec or erlang_standard), and their output
becomes the field value. If the alias has an error, it replaces their
errors. Aliases work at the top level, inside nested rules (nested_object,
list_of, 'or', nested_map, ...) and other aliases, and run on a missing
field, so an alias with required reports a missing value.

Registered aliases are kept in a persistent_term. The aliases of a call
are resolved once, when its options are normalized, so rules cost one
map match more only when there are aliases; without any, the options
and the rule path stay as before.
livr_import also imports aliases_positive and aliases_negative; the
aliases of a case go to opts => #{aliased_rules => [...]}, so
livr_rules_SUITE runs them with the existing case runner.
livr_spec_check compares those suites with upstream too. The files
under tests/cases/livr are the importer's output for koorchik/LIVR@4384f09e.
- The erlang.mk build (ci.erlang.mk) covers OTP 25 and later, but
  nothing builds liver with rebar3, the way most users depend on it
  (rebar.config.script pulls jsx below OTP 27). Run rebar3 ct on OTP 25
  with rebar3 3.24.0, the last release for OTP 25, and on OTP 29 with
  rebar3 3.28.0.
- actions/checkout v4 runs on Node 20, which GitHub Actions has
  deprecated; v7 runs on Node 24.
LIVR patterns follow JavaScript, where $ matches only at the end of the
value. PCRE also matches it before a trailing newline, so
{like, <<"^[0-9]{5}$">>} accepted <<"12345\n">>. The like rule now
compiles patterns with dollar_endonly.
@koorchik

Copy link
Copy Markdown
Author

I've added one more commit to this PR: 3a8c5f0 like: $ matches only at the end of the value.

LIVR patterns follow JavaScript, where $ matches only at the end of the value. PCRE also matches $ before a trailing newline, so {like, <<"^[0-9]{5}$">>} accepted <<"12345\n">> (the JS reference rejects it). The like rule now compiles patterns with dollar_endonly, a one-option change. The new like_end_of_value test in livr_rules_SUITE fails without it, and there is a line in the CHANGELOG.

Tested in Docker: rebar3 ct --dir tests on OTP 25, 27 and 29 (192 tests pass), and the LIVR 2.0 test suite through the livr-spec.org conformance runner (70/70). As with the rest of the PR, this was prepared with the help of an LLM and tested as described, but please review it anyway. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant