Repository navigation
conformance(loader,report): reject non-standard JSON constants in loads_strict and validate record_digest/build inputs (#648) - #653
Conversation
|
🟡 Contributor Check: MEDIUM
Automated check by AgenTrust Contributor Check. |
ce4cb3e to
43fb214
Compare
…ds_strict and validate record_digest/build inputs (agentrust-io#648) - Reject non-standard JSON constants (NaN, Infinity, -Infinity) via parse_constant and non-bytes inputs in trace_tests.loader.loads_strict with LoadError. - Reject non-dict record inputs in trace_tests.report.record_digest and trace_tests.report.build, and validate results_by_level level keys and module dicts in build. Closes agentrust-io#648 Signed-off-by: Raghuvar Karthik Durga <177662415+drk-ram@users.noreply.github.com>
43fb214 to
cd9bf9e
Compare
|
Hello @drk-ram, thank you for the continued interest in contributing to agentrust-io, before we go any further, we need to sort out what's already open. Quick recap of where things stand across agentrust-io: 120 PRs opened in the last 17 days, averaging over 7 a day. 69 are currently open, 46 merged, 5 closed without merging, and almost 98 PRs landed in earlier 7 days. You also mentioned in your maintainer interest issue #640 that you review all code diffs and test output yourself before submitting, with AI agents working under your supervision. Given what we're actually seeing in the open PRs, that review step doesn't seem to be happening consistently. Some examples:
If each of these was reviewed before opening, these issues would have been caught before submission, not after. At this volume and with these gaps, it's hard for us to review each, and also whether it's you behind each PR or something more automated opening them faster than they can reasonably be checked. So kindly avoid opening new PRs until the current backlog is cleared,kindly go back through your own open PRs and actually review them yourself first, fix the ones with pending changes requested, resolve the conflicts, fix the failing tests and CI, check if there are still bugs or issues which still exists in the diff and fix them and get them into merge ready state And separately, it would help if you could just clarify directly: are you personally testing, and reviewing each of these before they go up, or is there is an automated process submitting PRs on this account with less oversight than described? We'd rather have a straight answer than keep guessing from the pattern. |
|
Thank you @rajnisht7 for the direct feedback — point taken, and I am pausing all new PR submissions across To answer your question directly: I work on Windows 11 and had been batching my local fixes and submitting them together using a PowerShell I have gone back through every open PR across all 8 repositories to resolve all merge conflicts, fix the PR descriptions, and address every pending review comment:
I will hold off on opening any new PRs until the current queue is worked through. |
What this changes
Closes #648.
Hardens
conformance/src/trace_tests/loader.pyandconformance/src/trace_tests/report.py:NaN,Infinity,-Infinity) viaparse_constant=_refuse_constantsand non-bytesinputs (not isinstance(raw, bytes)) inloads_strictwithLoadError, preservingloads_strictandparse_record's documentedLoadError-only boundary while keepinginvalid_canonical_non_finite_float.json(1e400) reachable forTR-SIG.dictrecordinputs inrecord_digestandbuild, and validatesresults_by_level(requiring exactintkeys inLEVEL_NAMESanddictvalues) inbuild.Validation commands and results:
PYTHONPATH=src;conformance/src python -m pytest tests/test_conformance_report_parity.py -q-> passed (26 passed)PYTHONPATH=src;conformance/src python -m pytest conformance/tests/test_report.py conformance/tests/test_canonicalization_refusals.py conformance/tests/unit/test_loader.py -q-> passed (51 passed)python -m ruff check tests/test_conformance_report_parity.py-> passed (All checks passed!)python tools/check_dashes.py-> passed (no dashes outside examples/, spec/trace-v0.1.md)PYTHONPATH=src python tools/review_bar.py --base origin/main --head HEAD-> passed (14 changed test cases fail on behaviour at origin/main)Type of change
Spec section
conformance/src/trace_tests/loader.pyandconformance/src/trace_tests/report.py(conformance test suite loader and report builder; no normative change tospec/trace-v0.2.md)Tests for a fix
parse_constant=_refuse_constantsandisinstance(raw, bytes)inloads_strict:tests/test_conformance_report_parity.py::test_loads_strict_and_parse_record_reject_non_standard_constants_and_non_bytesisinstance(record, dict)inrecord_digestandbuild:tests/test_conformance_report_parity.py::test_record_digest_and_report_build_reject_non_dict_recordresults_by_levellevel and dict validation inbuild:tests/test_conformance_report_parity.py::test_report_build_rejects_invalid_results_by_levelChangelog entry
trace_tests.loader.loads_strictnow rejects non-standard JSON constants (NaN,Infinity,-Infinity) and non-bytesinputs withLoadError, andtrace_tests.report.record_digestandbuildreject non-dictrecords and invalidresults_by_levelmappings withValueError.Checklist
git commit -s)<!-- CHANGED: #NNN: description -->in spec text