Skip to content

test: add a portable offline verifier corpus - #167

Merged
rajnisht7 merged 2 commits into
agentrust-io:mainfrom
dinakarjs:test/portable-verifier-corpus-20261025
Nov 1, 2026
Merged

rajnisht7 merged 2 commits into
agentrust-io:mainfrom
dinakarjs:test/portable-verifier-corpus-20261025

Conversation

@dinakarjs

@dinakarjs dinakarjs commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Provide a bounded portable offline corpus with 28 accepted/refused cases across five operations: batch inclusion, producer signatures, MMR inclusion, MMR consistency and pinned-key checkpoint signatures.

The runner now rejects duplicate JSON members using the verifier's existing loads_unique parser. IDs are opaque. Ten new cases cover every position in two- and three-leaf trees plus tampered siblings, with a bounded independent SHA-256 reference in the tests. Exact case count and all five paired operation domains are required.

Validation on Linux/Python 3.12: fresh hash-locked CI/runtime/witness dependencies and no-deps editable install; full suite 416 tests passes, including 10 corpus harness tests. All 28 vectors pass through the real adapter subprocess. Restoring the old runner makes both duplicate-member regressions and all five missing-domain regressions fail. Ruff and whitespace pass.

Expected outcomes are omitted from adapter requests, not hidden from trusted local executables that can read the corpus. The original fixtures use synthetic keys/reference generators; the new multi-leaf reference is bounded, not a complete independent oracle. This is not full TRACE conformance, external MMR compatibility, certificate/hardware evidence, a new normative format or a new conformance level. No registry writes or service changes.

Prepared with OpenAI Codex. Signed-off-by: Srinivasa Dinakar Jainti dinakarjs@gmail.com.
Independent current-head review and hosted CI remain required; prior-head CI is historical.

Current head: 8605514. Published tree 9f0c840ae2dbe2dad850f5b48cd0f99f0562541f matches the locally validated source tree.

Hosted validation update: all returned current-head PR workflows completed successfully. Relevant runs: workflow 41166483376, workflow 41166483252. This workflow query is not a complete branch-protection inventory. Independent re-review has been requested in the PR conversation and remains pending.

Signed-off-by: Srinivasa Dinakar Jainti <dinakarjs@gmail.com>

@rajnisht7 rajnisht7 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the overall changes for corpus looks good, solid first slice overall.
a few points needs to be addressed:

  • Runner accepts duplicate JSON keys-json.loads last-wins, so an adapter can emit
    {"id":"bogus","id":"<correct>","accepted":false,"accepted":true}
    and still pass. The rest of the repo already rejects this with loads_unique; the runner should do this too, plus a couple of tests for it.
  • Case IDs leak the expected verdict- Names like *-valid / *-wrong-* / *-tamper let a trivial ID-lookup adapter pass all 18 cases. Opaque IDs (case-001 …) would make the “expectations withheld” claim actually hold.
  • All batch_inclusion cases are single-leaf- every proof has leaf_count=1 and (almost always) empty audit_path. A verifier that only handles the 1-leaf special case still passes the whole batch suite. At least one 2-leaf and one 3-leaf (left + right positions) plus a tampered-sibling negative would exercise the real inclusion algorithm.
  • Op-set assertion is too weak-the current “every present op has both True/False” check still passes if an entire domain (e.g. all checkpoint cases) is deleted. Explicitly assert the five expected operations (and ideally len(cases) == 18).

non-blocking: an adapter can just open conformance/verifier-v1.json and return the stored expected (no isolation). The “trusted local executable” note is already there, but the “expectations withheld” wording overstates what the protocol actually hides.

Prepared with OpenAI Codex.

Signed-off-by: Srinivasa Dinakar Jainti <dinakarjs@gmail.com>

Copy link
Copy Markdown
Contributor Author

@rajnisht7 please re-review 8605514. Addressed duplicate JSON replies with loads_unique and real subprocess regressions; opaque IDs; two-/three-leaf proofs at every position with tampered siblings; exact five-domain/28-case inventory. Expectations are omitted from requests, not isolated from trusted executables. Full suite: 416 tests; Ruff passes. Old runner fails both duplicate-key and all five missing-domain regressions. Independent bounded SHA-256 calculations cover the ten new vectors. Prepared with OpenAI Codex.

Copy link
Copy Markdown
Contributor Author

@rajnisht7 following up for your re-review and disposition of the corpus findings at 8605514. The published correction rejects duplicate JSON members, uses opaque IDs, adds two-/three-leaf proofs at every position with tampered-sibling negatives, and enforces the exact five-domain/28-case inventory. The trusted-local-executable limitation is explicit.

I have just rechecked this head: CI and CodeQL both succeeded. The description records 416 passing tests, real adapter subprocess checks, and behavioral regressions that fail with the old runner restored. Please confirm whether your requested changes are satisfied or identify remaining corrections. This is a conversation re-review request; prior local validation is cited, not a fresh suite run. Prepared with OpenAI Codex.

@rajnisht7
rajnisht7 merged commit bcef967 into agentrust-io:main Nov 1, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants