Skip to content

[Wave 4] Low/info — Module sweep: Verification toolbelt: verify-audit, verify-annex-iv, verify-gguf, verify-integrity (part 2/2) #716

Open

Description

@cemililik

Wave 4 · 2 finding(s) in 2 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

  • CR-A09-053 (code, Low) — Annex IV and pipeline-manifest verifiers accept duplicate JSON keys, and manifest_hash does not change. verify-annex-iv (single and --pipeline, Tier 3 corroborated) exits 0 while a first-wins reader…
    severity Low · reachability default · intent intended-but-harmful · confidence likely · independently confirmed · reproduced end to end
    Failure scenario. After a pipeline whose stage_b actually stopped, an output-dir writer without the audit secret inserts "status": "completed", before the authentic "status": "failed" and "final_status": "completed", before the authentic value. The same works for an Annex IV intended_purpose. CI runs forgelm verify-annex-iv --pipeline and gets exit 0, hash verified and audit-corroborated, because ForgeLM sees the authentic values. A first-wins dashboard, a human reviewer, or any process that recorded t…
    Where. forgelm/verify/_annex_iv.py:112, forgelm/verify/_annex_iv.py:162-178, forgelm/verify/_io_safety.py:79, forgelm/verify/_pipeline_evidence.py:372
    Recommended fix. Give _io_safety one strict loader: an object_pairs_hook that raises on a repeated name. Use it in verify_annex_iv_artifact (which this same change can move onto _read_capped_json) and in both pipeline reads. Treat a repeat as an integrity violation (exit 6): a ForgeLM writer cannot produce one from str-keyed dicts. Also reject mixed int/str keys in build_annex_iv_artifact, the one input from whic…

  • CR-A09-054 (code, Low) — Mutation-adequacy gaps: ten non-equivalent mutants of _pipeline_evidence.py and the census replay survive the three test files
    severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
    Failure scenario. A refactor that swaps any/all, removes the islink guard, or changes the -1 default keeps CI green while deleting a middle stage row from a pipeline manifest, or following a symlinked legacy sibling out of the pipeline directory, becomes undetected.
    Where. forgelm/verify/_pipeline_evidence.py:271, forgelm/verify/_pipeline_evidence.py:360, forgelm/compliance.py:3290, forgelm/compliance.py:3357
    Recommended fix. Add tests: delete first and middle stage rows of a 3-stage run (assert contradicted); legacy manifest (forgelm_version 0.9.0) with a symlinked annex_iv_metadata.json sibling; all-blank annex_iv identity block routes to UNVERIFIED; _manifest_stage_positions with duplicate names; unpositioned-name restart retires only that name; reordered and duplicate-name manifests against corroboration.

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: verifyCode area: verifybugSomething isn't workingsecuritySecurity-relevantseverity: lowMinor defect, inaccuracy or rough edgesource: review-2026-10Found by the October 2026 code and documentation reviewwave: 4Roadmap wave 4: low / info / watch, when touching the module

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions