[Wave 4] Low/info — Module sweep: Verification toolbelt: verify-audit, verify-annex-iv, verify-gguf, verify-integrity (part 1/2) #715

Wave 4 · 20 finding(s) in 20 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

• CR-A09-031 (code, Low) — Documented non-zero exit branches of the verify-* CLIs (including verify-integrity's JSONDecodeError/IsADirectoryError handlers and text output) have no test
  severity Low · reachability default · intent intended-and-acceptable · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. A refactor of the exception ladder changes a public exit code with a green suite.
  Where. forgelm/cli/subcommands/_verify_integrity.py:94-99, forgelm/cli/subcommands/_verify_integrity.py:113-118, forgelm/cli/subcommands/_verify_integrity.py:144-154 (confirmed), tests/test_eu_ai_act.py:446-461
  Recommended fix. Add text-mode success and failure tests asserting printed lines and exit codes (0, 6, 1), a malformed-JSON manifest test and a directory-named-manifest test.

• CR-A09-032 (code, Low) — verify-gguf returns on a magic mismatch before the sidecar step, so a magic-byte tamper with a sidecar present is reported as exit 1 with sidecar_present=false
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A CI consumer reading checks.sidecar_present concludes that no sidecar was shipped, although one was, and treats a magic-byte tamper as a wrong-path typo.
  Where. forgelm/verify/_gguf.py:75-89 (confirmed), forgelm/verify/_gguf.py:254-255 (confirmed), forgelm/verify/_gguf.py:82-89, forgelm/verify/_model_integrity.py:176-187 (confirmed)
  Recommended fix. Set sidecar_present from os.path.isfile before returning. Optionally compare the sidecar on a magic mismatch and route a well-formed mismatching digest to exit 6.

• CR-A09-033 (code, Low) — Missing-evidence routing assumes the best-effort compliance export succeeded and reports a logged export failure as tampering
  severity Low · reachability default · intent intended-but-harmful · confidence likely · independently confirmed · not reproduced (code trace)
  Failure scenario. A stage's compliance export fails because the disk is full, and training completes. A later verification reports the stage evidence as deleted (exit 6).
  Where. forgelm/verify/_pipeline_evidence.py:298-321, forgelm/trainer.py:2148-2155, forgelm/verify/_pipeline_evidence.py:296-321, forgelm/cli/_pipeline.py:1642-1665
  Recommended fix. Record the export outcome on the stage state, for example an evidence_written flag or the pointer only on success, and route an evidence pointer that was never written as UNVERIFIED.

• CR-A09-034 (code, Low) — _audit_log_reader._parse_nonempty_line catches only JSONDecodeError, so lenient mode crashes and strict mode raises a bare ValueError/RecursionError
  severity Low · reachability default · intent contract-violation · confidence confirmed · single reviewer · reproduced end to end
  Failure scenario. One corrupted or crafted line in the output dir's audit log makes approvals enumeration and approve/reject crash rather than skip the line or report it.
  Where. forgelm/cli/subcommands/_audit_log_reader.py:93-102 (confirmed), forgelm/cli/subcommands/_audit_log_reader.py:123-128 (confirmed)
  Recommended fix. Catch (ValueError, RecursionError) in _parse_nonempty_line. JSONDecodeError is a ValueError subclass, so catching ValueError covers both. Raise AuditLogParseError in strict mode and skip in lenient mode.

• CR-A09-035 (code, Low) — The S1 split left orphaned comment fragments in _pipeline_evidence.py, _io_safety.py and _audit_log.py
  severity Low · reachability unreachable · intent intended-and-acceptable · confidence likely · independently confirmed · not reproduced (code trace)
  Failure scenario. Readers lose the rationale for _EmptyFileError and the RecursionError handlers; the docstring points maintainers at the wrong module.
  Where. forgelm/verify/_pipeline_evidence.py:200, forgelm/verify/_io_safety.py:29-31, forgelm/verify/_pipeline_evidence.py:47-52, forgelm/verify/_audit_log.py:11-15
  Recommended fix. Re-join the fragments next to the code they describe (the _EmptyFileError class and the RecursionError handlers), and update the module docstring.

• CR-A09-037 (code, Low) — verify-gguf: the real gguf-reader failure path and the CLI success path are unprotected; a mutant that makes any parser exception count as parsed survives
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A change that makes the reader failure fail-open would pass CI; the stubbed parse hides it.
  Where. forgelm/verify/_gguf.py:203-210, tests/test_verification_toolbelt.py:322, tests/test_verification_toolbelt.py:2124, tests/test_library_api.py:508-557
  Recommended fix. Add a GGUFWriter-built fixture, truncate and corrupt it, and assert exit 6 end to end (and exit 1 when a matching sidecar exists). Add a test of the no-sidecar valid result wording.

• CR-A09-039 (code, Low) — Docs misdescribe the stable verify result classes (called dataclasses / reason typed Optional[str]; overstated checks)
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Failure scenario. A consumer testing result.reason is None or relying on a tokenizer check gets a silently different answer. A reader assumes an Annex IV bundle with a model card is verified when only nine JSON keys are.
  Where. docs/reference/library_api_reference.md:93-98 (confirmed), docs/reference/library_api_reference-tr.md:93-98, forgelm/verify/_annex_iv.py:47-78, forgelm/verify/_annex_iv.py:122-185
  Recommended fix. Correct the types to str, rewrite the two descriptions to the real checks, and list the real checks keys.

• CR-A09-040 (code, Low) — Raises contract for verify_annex_iv_artifact / verify_gguf / verify_integrity is undocumented, and the code adds exceptions beyond what its own docstrings promise
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. A library caller catching OSError and checking valid lets a malformed artefact escape as a raw traceback. In a pipeline, that is an uncaught exception rather than a failed verification.
  Where. docs/reference/library_api_reference.md:93-98, docs/reference/library_api_reference.md:262-267, forgelm/verify/_annex_iv.py:101-112, forgelm/verify/_gguf.py:72-73
  Recommended fix. Add a verifier row to the error table listing OSError, ValueError (JSON/UTF-8) and RecursionError. Alternatively convert them into valid=False results, as verify_audit_log already does.

• CR-A09-041 (code, Low) — Mid-stream OSError while reading the audit log escapes approve, reject and approvals --show as a raw traceback (exit 1, no JSON envelope); the docs promise exit 2
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced with stubs
  Failure scenario. A CI step runs forgelm approvals --show r1 --output-format json on a flapping NFS mount. It gets a Python traceback, exit 1 and an empty stdout instead of the documented exit 2 plus envelope, and retry logic keyed on exit 2 does not fire.
  Where. forgelm/cli/subcommands/_audit_log_reader.py:166-167, forgelm/cli/subcommands/_approve.py:418-443, forgelm/cli/subcommands/_approve.py:475-500, forgelm/cli/subcommands/_approvals.py:514
  Recommended fix. Wrap the strict reads in approve/reject and both reads in --show in except OSError -> _output_error_and_exit(..., EXIT_TRAINING_ERROR), mirroring _run_approvals_list_pending.

• CR-A09-042 (code, Low) — verify-gguf docs omit or mis-type checks.metadata_error, which the code always emits (null when parsed or skipped)
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Failure scenario. A consumer that rejects unknown keys fails on real output.
  Where. docs/usermanuals/en/deployment/verify-gguf.md:58-66 (confirmed), docs/usermanuals/en/deployment/verify-gguf.md:80-88 (confirmed), forgelm/verify/_gguf.py:75-81, forgelm/verify/_gguf.py:95
  Recommended fix. Add metadata_error to the three examples in EN and TR.

• CR-A09-043 (code, Low) — manifest_hash_expected is echoed verbatim from the artefact, so it can be an integer or 'sha256:'-prefixed
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Failure scenario. A CI consumer runs jq -e '.manifest_hash_expected | test("^[0-9a-f]{64}$")' or a typed schema on the envelope and fails or crashes on a crafted artefact.
  Where. forgelm/verify/_annex_iv.py:155-165, forgelm/verify/_annex_iv.py:72-79
  Recommended fix. Coerce a non-string expected hash to '' (or its str form) and flag it as a mismatch, or document that the expected field mirrors the artefact verbatim.

• CR-A09-044 (code, Low) — CLAUDE.md exit-code split overgeneralises: 'never compared' exits 0, not 1, in three verifiers
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end · already tracked: KNOWN-OPEN C31-EXIT-CODE-PROSE — docs/roadmap/phase-16-trust-surface-hardening.md:783-785 (S15 open)
  Failure scenario. An operator follows CLAUDE.md:117, alarms on 6 and treats 0 as verified. An Annex IV artefact edited after generation with its metadata block removed exits 0. An audit log verified without FORGELM_AUDIT_SECRET exits 0 after a chain-only check.
  Where. CLAUDE.md:117, forgelm/cli/subcommands/_verify_audit.py:231-236, forgelm/verify/_annex_iv.py:182-185, forgelm/verify/_gguf.py:175-181
  Recommended fix. Reword CLAUDE.md:117 (and AGENTS.md) to say exit 0 can mean 'checked what was available'. Point CI authors at --require-hmac and at the JSON hash_state / sidecar_present fields.

• CR-A09-045 (code, Low) — An I/O error inside the gguf metadata reader is reported as an integrity failure (exit 6), not an I/O error (exit 2)
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced with stubs
  Failure scenario. A flaky network filesystem returns EIO while the reader memory-maps the file. CI pages the artefact owner for tampering when the fault was a disk or transport error.
  Where. forgelm/verify/_gguf.py:205-210, forgelm/verify/_gguf.py:222-269, docs/usermanuals/en/deployment/verify-gguf.md:150, forgelm/verify/_gguf.py:72-73
  Recommended fix. Let OSError propagate out of the metadata parse, catching only parse-type exceptions.

• CR-A09-046 (code, Low) — The error-handling standard says every JSON-mode error emits a stdout envelope, with no exception for argparse usage errors (L-A05-12, narrowed)
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Failure scenario. A contributor follows error-handling.md:264 and writes a CI consumer or test that runs json.loads(stdout) on every non-zero exit. A usage error then gives empty stdout and a JSONDecodeError instead of a handled failure.
  Where. docs/standards/error-handling.md:264, docs/usermanuals/en/reference/json-output.md:18 (the line where the exception is documented), docs/usermanuals/tr/reference/json-output.md:18, forgelm/cli/subcommands/_verify_audit.py:21-24
  Recommended fix. Add the json-output.md:18 exception to error-handling.md:264: the envelope guarantee starts after argparse succeeds, and usage errors exit 2 with an empty stdout. Do this or change the parser so it emits an envelope, whichever way M5r1-A05-001 is resolved.

• CR-A09-047 (code, Low) — Peak RSS amplification of a single 8 MiB evidence or manifest file (about 70x to 170x)
  severity Low · reachability default · intent intended-but-harmful · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. Memory-constrained CI runner (1 to 2 GB) running the verifier on a crafted 8 MiB list-of-integers evidence file reaches about 1.4 GB RSS and may be OOM-killed while other jobs share the host.
  Where. forgelm/verify/_io_safety.py:64-79, forgelm/compliance.py:1945, forgelm/verify/_pipeline_evidence.py:45, forgelm/compliance.py:2085
  Recommended fix. Either lower STAGE_EVIDENCE_MAX_BYTES (single-digit kB in practice, per the module's own comment; for example 1 MiB gives about 8x less peak memory and per-stage time), or add an element-count or depth-aware limit via object_pairs_hook counting. Also hashing the already-parsed artefact without the dumps/loads round-trip removes two of the three extra copies (about 1.3 to 2x less peak memory).

• CR-A09-048 (code, Low) — Audit-log reader: 'latest event wins' is not pinned (first-match mutant survives, including through approve)
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN TRUST-07 — docs/roadmap/phase-16-trust-surface-hardening.md:576-578 (S9 open; unit id inferred)
  Failure scenario. A regression to first-match makes approve and reject act on a stale required event for a re-staged run_id: a wrong staging_path, or 'Staging directory not found'. Nothing fails. For the decision event it also changes which prior decision the refusal message names.
  Where. forgelm/cli/subcommands/_audit_log_reader.py:222-228, forgelm/cli/subcommands/_approve.py:185-201, forgelm/cli/subcommands/_approve.py:538-548, forgelm/cli/subcommands/_approve.py:206-225
  Recommended fix. Add a reader-level test with two matching events asserting the later one is returned. Add an approve-level test with a re-staged run_id (two required events with different staging_path).

• CR-A09-049 (code, Low) — Audit-log reader: missing-file and directory handling plus the OSError-on-open path are untested
  severity Low · reachability default · intent intended-and-acceptable · confidence likely · single reviewer · reproduced end to end
  Failure scenario. The guard is dropped or reordered and every normal first-run lookup logs an ERROR traceback. Or the logger.exception is dropped and an unreadable log silently looks like 'no events', and no test notices.
  Where. forgelm/cli/subcommands/_audit_log_reader.py:158-164, forgelm/cli/subcommands/_audit_log_reader.py:246-248
  Recommended fix. Add caplog assertions: no records for a missing or directory path; one ERROR record when open raises OSError (monkeypatch builtins.open).

• CR-A09-050 (code, Low) — is_audit_log_readable has no direct test; only the os.access-mocked path is covered
  severity Low · reachability default · intent intended-and-acceptable · confidence likely · single reviewer · reproduced end to end
  Failure scenario. The gate is weakened for directory paths, or approve/reject lose the gate call, with no test failing.
  Where. forgelm/cli/subcommands/_audit_log_reader.py:246-248, forgelm/cli/subcommands/_approve.py:380-396
  Recommended fix. Add direct unit tests for the helper (missing path, directory, readable file, real chmod 000 skipped as root) and approve/reject unreadable-log tests.

• CR-A09-051 (code, Low) — Blank and whitespace-line skipping in the reader is not pinned (line 181 never executed)
  severity Low · reachability default · intent intended-and-acceptable · confidence likely · single reviewer · reproduced end to end
  Failure scenario. A change that makes blank lines fatal in strict mode would make approve and reject refuse on a log with a stray trailing newline (CRLF or editor-inserted), and nothing in the suite would fail.
  Where. forgelm/cli/subcommands/_audit_log_reader.py:179-181
  Recommended fix. Add a strict-mode test over a log containing blank, whitespace-only and CRLF lines, asserting the line numbers of the yielded events.

• CR-A09-052 (code, Low) — (c) No test covers any of the five crash inputs; the existing version-gate and status tests stop short of them
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end
  Failure scenario. Each crash above passes the full suite on this snapshot. A fix could regress unnoticed.
  Where. tests/test_verification_toolbelt.py:3470, tests/test_verification_toolbelt.py:3339, forgelm/verify/_pipeline_evidence.py:287-295
  Recommended fix. When M-X-027 is fixed, add tests for each crafted case: one at the library level (a violation with the right prefix) and one through the CLI (exit code, plus an envelope on stdout).

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}