Skip to content

fix(trace_verify,tools): enforce registry-anchor-v1 surrogate/number rules, reject private JWKs, and harden MMR/JSON parsing - #147

Open
drk-ram wants to merge 1 commit into
agentrust-io:mainfrom
drk-ram:drk-ram-fix-issue-146-anchor-jwk-mmr-json
Open

drk-ram wants to merge 1 commit into
agentrust-io:mainfrom
drk-ram:drk-ram-fix-issue-146-anchor-jwk-mmr-json

Conversation

@drk-ram

@drk-ram drk-ram commented Oct 23, 2026 •

Copy link
Copy Markdown

What and why

Closes #146.

Fixes four verification and registry-anchor-v1 parity bugs across src/trace_verify/ and tools/:

  1. Enforce registry-anchor-v1 section 1 profile rules in canonical_claim_bytes and anchor_profile_violation:
    • Check for unpaired UTF-16 surrogates (U+D800..U+DFFF) in string values and object keys, detect circular references (active: set[int]), and walk (list, tuple) in anchor_profile_violation.
    • Call anchor_profile_violation inside canonical_claim_bytes across src/trace_verify/_verify.py, tools/verify_inclusion.py, tools/anchor.py, and tools/batch_anchor.py, raising ValueError on floats, integers outside [-9007199254740991, 9007199254740991], unpaired surrogates, circular references, and RecursionError / TypeError, while preserving the explicit as-transmitted contract (raw_bytes returned verbatim without re-serialization).
  2. Reject private/symmetric JWK parameters and non-canonical base64url x:
    • Reject RFC 7517 / RFC 7518 / RFC 8037 private or symmetric JWK members (d, p, q, dp, dq, qi, oth, k) and non-canonical 32-byte base64url x encodings (including non-zero trailing padding bits) in src/trace_verify/_signature.py (validate_public_key_jwk / verify_claim_signature) and tools/validate_producers.py.
  3. Require exact int in MMR proof deserialization, builders, and verifiers:
    • Enforce type(v) is int (rejecting bool, IntEnum, str, and float) in InclusionProof.from_dict, ConsistencyProof.from_dict, inclusion_proof, consistency_proof, verify_inclusion, and verify_consistency in src/trace_verify/_mmr.py.
  4. Reject duplicate JSON member names across standalone, validation, and NDJSON tools:
    • Use loads_unique (object_pairs_hook) in tools/verify_inclusion.py, tools/verify_checkpoint_chain.py, tools/validate_registry.py, tools/validate_producers.py, tools/anchor.py (_registry_is_chained), tools/batch_anchor.py (is_already_anchored, load_published_entries), and src/trace_verify/_signature.py.

Validation

  • python -m unittest discover -s tests -q — passed (416 tests)
  • python -m pytest tests/test_mmr_core.py tests/test_cli_duplicate_members.py tests/test_trace_verify_pkg.py tests/test_signature.py -q — passed (95 passed, 74 subtests passed)
  • python -m ruff check src/ tools/ tests/ — passed (All checks passed!)
  • python tools/validate_registry.py — passed (OK: 3 day file(s) valid)
  • python tools/validate_producers.py — passed (OK: 3 producer key file(s) valid)

Registry impact

  • This PR does not modify any existing line under registry/

Existing entries are append-only. If you are changing one, say why here: mirrors
and auditors may already have observed the current commit hashes, and a rewrite
diverges them.

Checks

  • pytest passes
  • Registry and producer files still validate (tools/validate_registry.py,
    tools/validate_producers.py)
  • If the anchor format or verification semantics changed, docs/anchor-format.md
    is updated in the same PR, and existing inclusion proofs still verify
  • Commits are signed off (DCO)

@drk-ram
drk-ram requested review from a team, lywinged and rajnisht7 as code owners October 23, 2026 01:47

@dinakarjs dinakarjs left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head f1ea2ff (full commit attached to this review). Checked strict MMR integer/proof boundaries, the public-JWK/base64url restrictions, duplicate-member parsing in the CLI/tool paths, and sorted-key anchor profile validation without replacing the separate signing canonicalizer. Local full unittest suite: 415 tests passed; Ruff passed. Hosted CI and CodeQL succeed. ClusterFuzzLite remains action_required and is not represented as executed evidence. No blocking finding.

@rajnisht7 rajnisht7 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the fixes looks good, sorted-key profile checks, private JWK + canonical x, MMR exact-int, and loads_unique on the four named tools all check out. Scripts and new tests pass on this head.

One thing needs to be addressed:

  • Profile walk now runs before json.dumps, but there’s no cycle detection. A self-referential dict hangs instead of failing closed. kindly detect cycles by identity, return a clear error, allow shared-but-acyclic objects, and add tests for cyclic dict/list in both _verify.py and tools/verify_inclusion.py.

Note: as-transmitted still returning raw_bytes without a profile check is consistent with the docs and existing design (exact bytes, no re-serialization). Tool intake already rejects out-of-profile claims, and we already have coverage that as-transmitted preserves exact bytes including whitespace/key order.

Follow-up (non-blocking):

  • Surrogate wording can stay aligned with “unpaired” (behavior is fine in Python).
  • is_already_anchored / load_published_entries / _registry_is_chained still use plain json.loads on published NDJSON; loads_unique there would match the hardening but is outside the #146 scope.

@drk-ram

drk-ram commented Oct 23, 2026

Copy link
Copy Markdown
Author

Thanks @rajnisht7! Addressed in the latest commit: anchor_profile_violation in both src/trace_verify/_verify.py and tools/verify_inclusion.py now tracks the active ancestor object IDs on the explicit traversal stack (active: set[int] with (value, where, exiting) frames) so cyclic dicts/lists fail cleanly with a "circular reference at ..." violation while shared-but-acyclic dicts/lists continue to succeed, covered by unit tests in tests/test_trace_verify_pkg.py.

@rajnisht7

Copy link
Copy Markdown
Member

Cycle detection looks correct, active-set + exiting frames in both copies, shared-but-acyclic objects still allowed, clear error. Tests exercise both implementations.

needs follow up (non blocking):

  • as-transmitted path still returns raw_bytes without the profile walk (consistent with the existing design, but the contract should be made explicit in comments/tests).
  • is_already_anchored / load_published_entries / _registry_is_chained still use plain json.loads on published NDJSON; switching them to loads_unique would make the trust boundary consistent.
  • Add an explicit regression test that invokes tools/validate_producers.py for duplicate members.
  • inclusion_proof() should enforce type(size) is int at the top (IntEnum currently passes).

…rules, reject private JWKs, and harden MMR/JSON parsing (agentrust-io#146)

- Enforce registry-anchor-v1 section 1 profile rules (no floats, no integers outside -(2^53-1)..2^53-1, no unpaired UTF-16 surrogates U+D800..U+DFFF in object keys or strings, and cycle detection) in anchor_profile_violation and canonical_claim_bytes across src/trace_verify/_verify.py, tools/verify_inclusion.py, tools/anchor.py, and tools/batch_anchor.py, while documenting the as-transmitted exact-raw-bytes contract.
- Reject RFC 7517 / RFC 7518 / RFC 8037 private or symmetric JWK parameters (d, p, q, dp, dq, qi, oth, k) and non-canonical base64url x encodings in src/trace_verify/_signature.py and tools/validate_producers.py.
- Require exact int (type(v) is int, rejecting bool and IntEnum) in src/trace_verify/_mmr.py proof deserialization, builders (including inclusion_proof size), and verifiers.
- Reject duplicate JSON object member names via loads_unique in tools/verify_inclusion.py, tools/verify_checkpoint_chain.py, tools/validate_registry.py, tools/validate_producers.py, tools/anchor.py (_registry_is_chained), and tools/batch_anchor.py (is_already_anchored, load_published_entries).

Closes agentrust-io#146

Signed-off-by: Raghuvar Karthik Durga <177662415+drk-ram@users.noreply.github.com>
@drk-ram
drk-ram force-pushed the drk-ram-fix-issue-146-anchor-jwk-mmr-json branch from 7bade9b to fbb2c1e Compare October 29, 2026 16:24
@drk-ram

drk-ram commented Oct 29, 2026

Copy link
Copy Markdown
Author

Thanks @rajnisht7! Addressed all four follow-up items in fbb2c1e: (1) enforced type(size) is int at the top of inclusion_proof() in src/trace_verify/_mmr.py (with IntEnum regression coverage in tests/test_mmr_core.py); (2) switched is_already_anchored, load_published_entries, and _registry_is_chained in tools/batch_anchor.py and tools/anchor.py to use loads_unique; (3) made the as-transmitted exact-raw_bytes contract explicit in comments across src/trace_verify/_verify.py, tools/verify_inclusion.py, tools/anchor.py, and tools/batch_anchor.py and in test_as_transmitted_returns_raw_bytes_without_reserialization (tests/test_trace_verify_pkg.py); and (4) added an explicit regression test in tests/test_cli_duplicate_members.py invoking tools/validate_producers.py on a duplicate-member producer key file alongside the NDJSON helpers. Full unittest discovery (416 tests) and Ruff pass.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(trace_verify,tools): enforce registry-anchor-v1 surrogate/number rules, reject private JWKs, and harden MMR/JSON parsing

3 participants