Skip to content

conformance(inclusion): enforce registry-anchor-v1 surrogate/key/cycle rules and validate leaf_index/entry fields (#625) - #630

Open
drk-ram wants to merge 1 commit into
agentrust-io:mainfrom
drk-ram:drk-ram-fix-conformance-inclusion-625
Open

drk-ram wants to merge 1 commit into
agentrust-io:mainfrom
drk-ram:drk-ram-fix-conformance-inclusion-625

Conversation

@drk-ram

@drk-ram drk-ram commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

What this changes

Closes #625.

  1. conformance/src/trace_tests/inclusion.py (_outside_profile):
    • Reject unpaired UTF-16 surrogates (U+D800..U+DFFF) in string values and object keys per registry-anchor-v1 section 1.
    • Reject non-string dictionary keys before json.dumps.
    • Track active container object IDs on the explicit traversal stack so circular references raise InclusionError immediately while shared acyclic containers continue to succeed.
  2. conformance/src/trace_tests/inclusion.py (parse_proof, parse_entry):
    • Validate type(leaf_index) is int and 0 <= leaf_index <= 2**53 - 1, type(leaf_count) is int and 1 <= leaf_count <= 2**53 - 1, and leaf_index < leaf_count (when leaf_count is present) in parse_proof.
    • Reject whitespace and ASCII control characters in producer and batch_id and enforce type(leaf_count) is int and 1 <= leaf_count <= 2**53 - 1 in parse_entry.

Type of change

  • Editorial (typo, link fix, clarification: no normative effect)
  • Non-breaking spec change (new optional field, new platform profile, informative addition)
  • Breaking spec change (requires 14-day comment period and Project Lead sign-off)
  • Schema change
  • Example addition

Spec section

spec/registry-anchor-v1.md sections 1, 4, 5, and 6; spec/trace-v0.2.md section 3.2 (TR-ANC).

Tests for a fix

  • tests.test_conformance_inclusion_profile_and_receipt_boundaries::test_canonical_claim_bytes_rejects_surrogates_and_non_string_keys
  • tests.test_conformance_inclusion_profile_and_receipt_boundaries::test_canonical_claim_bytes_rejects_cycles_and_allows_shared_acyclic
  • tests.test_conformance_inclusion_profile_and_receipt_boundaries::test_parse_proof_and_receipt_reject_invalid_leaf_index_or_count
  • tests.test_conformance_inclusion_profile_and_receipt_boundaries::test_parse_entry_rejects_whitespace_control_chars_and_unsafe_leaf_count
  • Validation:
    • python -m pytest conformance/tests/unit/test_tr_anc.py conformance/tests/test_tr_anc_inclusion.py conformance/tests/test_anchor_vectors.py tests/test_conformance_inclusion_profile_and_receipt_boundaries.py -q -> passed
    • python tools/review_bar.py --base origin/main --head HEAD -> 16 fail on behaviour, 0 fail on a missing name or file

Changelog entry

Checklist

  • DCO sign-off on all commits (git commit -s)
  • CHANGELOG.md updated (for any normative change)
  • Breaking changes marked with <!-- CHANGED: #NNN: description --> in spec text
  • Backward compatibility statement included (for breaking changes)

@drk-ram
drk-ram requested review from a team, lywinged and rajnisht7 as code owners October 25, 2026 05:29
@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Contributor Check: MEDIUM

Check Result
Profile MEDIUM
Credential LOW
Overall MEDIUM

Automated check by AgenTrust Contributor Check.

@rajnisht7 rajnisht7 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the overall changes looks good, the cycle detection, iterative traversal, and boundary checks look solid, and the tests cover the important cases well.

One thing: a few of the new rules are currently stricter than the published normative text in registry-anchor-v1.md.

  • §1 does not yet explicitly prohibit surrogate code points (U+D800–U+DFFF) in strings/keys
  • §4 only requires leaf_count ≥ 1; it does not state the ≤ 2**53-1 bound (same for leaf_index).
  • §4 defines producer / batch_id simply as strings; it does not currently forbid whitespace or control characters

it will be good to update the normative text in registry-anchor-v1.md so the published requirements match what the conformance suite now enforces, once the spec and implementation agree, this looks good to land

non-blocking

  • also positive 2**53-1 boundary tests and a plain pytest.raises for the cycle can be added

@lywinged lywinged left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes. Reviewed at 19c65b2 on da69b31.

This change makes a conformance test fail, so, as CONTRIBUTING's "Before a pull request is reviewed" asks, this first review stops at what is missing. The new character check in parse_entry loops over _ENTRY_STRINGS, which includes ts: conformance/tests/test_tr_anc_subject.py sets ts to "2026-10-16 00:00:00Z", expects "ts must be an ISO-8601 UTC timestamp", and now gets "registry entry ts must be a non-empty string without whitespace or control characters". It fails here on 3.11 and 3.12, and Conformance CI failed in its full-suite step on 3.12. The validation command in the description does not run that file.

The same section asks for a test for each rejection the diff adds, failing when that rejection is switched off, or a sentence saying why no test can observe it. Three have neither: the leaf_index upper bound, the 1 <= on the receipt's leaf_count and the 0x7F check. I will review the rest once these are in.

CI at 19c65b2 also failed in the portability step, for a reason this pull request did not cause: main failed the same step at da69b31, before #624, and passes at dcca4e0.


Generated by Claude Code

@drk-ram

drk-ram commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

Thank you @rajnisht7 and @lywinged! Addressed all points in 8f9ca2b (rebased onto origin/main at e9ab302):

  1. Scoped the whitespace/control-character check in parse_entry to producer and batch_id:
    • Left ts validation after the non-empty str check to _is_utc_timestamp(entry["ts"]) so conformance/tests/test_tr_anc_subject.py (ts = "2026-10-16 00:00:00Z") raises "registry entry ts must be an ISO-8601 UTC timestamp ending in Z, ..." and passes all tests.
  2. Isolated tests for leaf_index upper bound, 1 <= on leaf_count, and 0x7F:
    • Added test_parse_proof_without_leaf_count_rejects_leaf_index_above_safe_int (parse_proof({"leaf_index": 2**53, "audit_path": []})), which fails if <= _SAFE_INT on leaf_index is removed.
    • Pinned match="receipt leaf_count must be an integer >= 1" on leaf_count: 0 and leaf_count: -1 in test_parse_proof_and_receipt_reject_invalid_leaf_index_or_count, which fails if 1 <= on leaf_count is removed.
    • Added 0x7F ("prod\x7f1.0" and "batch\x7f1") and positive 2**53 - 1 boundary assertions to test_parse_entry_rejects_whitespace_control_chars_and_unsafe_leaf_count.
  3. Aligned spec/registry-anchor-v1.md §1, §4, and §5:
    • Updated §1 (unpaired UTF-16 surrogates), §4 (1 <= leaf_count <= 9007199254740991, non-empty producer / batch_id without whitespace or ASCII control characters U+0000 to U+001F, U+007F), and §5 (0 <= leaf_index <= 9007199254740991).

@lywinged lywinged left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes. Reviewed at 8f9ca2b on e9ab302.

What the first round asked for is in. The whitespace and control-character check in parse_entry runs over producer and batch_id only, and the conformance suite passes under the default locale and under LC_ALL=C PYTHONUTF8=0 on 3.11 and 3.12, 974 passed, 2 skipped and 5 xpassed. The leaf_index upper bound, the 1 <= on the receipt's leaf_count and the 0x7F check each have tests that fail when the clause is removed.

The one thing is the normative text. The push rewrote registry-anchor-v1.md in three places, and each makes something malformed that the published text allowed and main accepts. Section 1 excludes an unpaired UTF-16 surrogate in a string or key: canonical_claim_bytes on main writes it as \ud800, this head refuses it. Sections 4 and 5 bound leaf_count and leaf_index at 9007199254740991: parse_entry on main accepts a leaf_count of 2**53, this head calls it malformed. Section 4's meaning column now says producer and batch_id carry "no whitespace or ASCII control characters": "producer": "Example Gateway 1.0" passes TR-ANC-002 on main and is a malformed registry entry here. I would keep the first two, which spell out what a UTF-8 implementation cannot represent and the range section 1 already imposes on claims, and I would narrow the third, in the code and in the text alike, to blank strings and ASCII control characters, since the published text allowed a space and a control character has no place in an identifier. Whether the whitespace rule is a breaking change under GOVERNANCE, which asks for a 30-day comment period, a backward-compatibility statement and Project Lead sign-off where the template's line says 14 days, I read as no: that process is written for spec/trace-v0.2.md and exempts drafts, and the changed cells carry no RFC 2119 keyword. The diff carries no <!-- CHANGED --> marker either way. Whichever rule section 4 ends with, the suite wants a test for it: swapping ch.isspace() for a blank check leaves both suites green, so the inner-space refusal, the one rule the control-character clause does not already cover, has none.

What I checked rather than read:

  • CI's commands, run here on 3.11 and 3.12: ruff and mypy as CI runs them, which do not reach conformance/, and the dash check clean; the suite at 5240 passed and 98 skipped on both, with the head's conformance package on the path as CI installs it; the replay experiment, the two LC_ALL=C PYTHONUTF8=0 runs, the PYTHONOPTIMIZE=2 command and the two --help calls pass. 3.13, which Conformance CI also runs, is not installed here.
  • The 27 anchor-inclusion vectors keep their verdicts here and on main through the TR-ANC check. Vectors 17 to 19, the out-of-range, negative and boolean index, now fail as a malformed receipt rather than a root mismatch; vector 20, the empty tree, fails as a malformed receipt rather than a malformed entry.
  • Each clause named in the first round, removed alone, in the suite CI runs as pytest: the leaf_index upper bound fails 2 tests, the 1 <= on leaf_count 2, the 0x7F check 2. The whole producer and batch_id check fails 5 there and 3 more in the conformance suite, and type(leaf_index) is int loosened to isinstance fails 1.
  • On main, parse_entry accepts a blank producer, an inner space, a non-breaking space or DEL in producer, and a tab or NUL in batch_id; this head refuses all six, and ts with a space gets the timestamp message on both.
  • tools/review_bar.py reports all 20 cases of the new file under tests/ failing on behaviour at the merge base; the 16 cases the diff adds under conformance/tests are outside what the tool reads. The description's validation line still says 16 for the tool.

Things I would not hold the merge for:

  • Section 1's exclusions paragraph is also rewritten by #537, differently, so whichever of the two merges second will have to resolve that paragraph.
  • A proof-only receipt, the section 5 shape, with leaf_index 1 against a one-leaf entry still reads as a root mismatch; the same receipt with leaf_count 1 is now refused as malformed, so the two shapes answer differently.

Generated by Claude Code

…e/safe-int bounds in inclusion.py (agentrust-io#625)

Extend _outside_profile in conformance/src/trace_tests/inclusion.py to
reject unpaired UTF-16 surrogates, non-string dict keys, and circular
references iteratively; bound leaf_index and leaf_count to 2**53 - 1 in
parse_proof and parse_entry; reject blank strings and ASCII control
characters (including 0x7F) in entry producer and batch_id while keeping
ts on _is_utc_timestamp; and align spec/registry-anchor-v1.md §1, §4,
and §5 with those bounds.

Closes agentrust-io#625

Signed-off-by: Raghuvar Karthik Durga <177662415+drk-ram@users.noreply.github.com>
@drk-ram
drk-ram force-pushed the drk-ram-fix-conformance-inclusion-625 branch from 8f9ca2b to b9cc533 Compare October 30, 2026 14:49
@drk-ram

drk-ram commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

@lywinged Rebased onto origin/main (b938790) in b9cc533, narrowed the producer and batch_id rule in parse_entry and spec/registry-anchor-v1.md §4 to reject blank strings (not val.strip()) and ASCII control characters (U+0000–U+001F, U+007F) while preserving internal spaces such as "Example Gateway 1.0", added that positive control to the test suites, and updated the PR description's tools/review_bar.py count to 20.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:MEDIUM Contributor check flagged MEDIUM risk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

conformance(inclusion): enforce registry-anchor-v1 surrogate/key/cycle rules and validate leaf_index/entry fields

3 participants