[Wave 3] Medium — Module sweep: Verification toolbelt: verify-audit, verify-annex-iv, verify-gguf, verify-integrity #573

Wave 3 · 15 finding(s) in 14 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

• CR-A09-010 (code, Medium) — verify-gguf reports a correct uppercase-hex SHA-256 sidecar as tampering (exit 6)
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A Windows-built release ships an uppercase sidecar; the pre-serve gate pages the security owner for every artefact.
  Where. forgelm/verify/_gguf.py:29 (confirmed), forgelm/verify/_gguf.py:144-150, forgelm/verify/_gguf.py:144 (confirmed), forgelm/verify/_gguf.py:124-150
  Recommended fix. Compare expected.lower() (or hmac.compare_digest on lowercased values) and add the test.

• CR-A09-013 (code, Medium) — Pipeline per-stage evidence is not bound to its stage or run: reused, re-pointed or absolute-path artefacts count toward evidence_verified
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. Stage B's evidence is deleted or replaced by another run's file, and B still counts as verified with exit 0 and every recommended counter green.
  Where. forgelm/verify/_pipeline_evidence.py:500-511 (confirmed), forgelm/verify/_pipeline_evidence.py:242-243, forgelm/compliance.py:1818, forgelm/verify/_pipeline_evidence.py:233-249 (confirmed)
  Recommended fix. Reject duplicate evidence pointers across stages. Embed pipeline_run_id and the stage name (or trainer_type) in the per-stage artefact and cross-check them. Document that evidence_verified attests self-consistency only.

• CR-A09-014 (code, Medium) — verify-annex-iv (single), verify-gguf and verify-integrity open input with plain open()/json.load: a FIFO hangs forever and reads are uncapped
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN VERIFY-UNCAPPED-READ — docs/roadmap/risks-and-decisions.md:263
  Failure scenario. A model directory shipped to an integrator contains model_integrity.json as a FIFO or as a symlink to /dev/zero. The CI verify step hangs until the job times out, or is OOM-killed, and never produces a verdict.
  Where. forgelm/verify/_annex_iv.py:106-112 (confirmed), forgelm/verify/_gguf.py:82-83 (confirmed), forgelm/verify/_model_integrity.py:82-86 (confirmed), forgelm/compliance.py:2469-2470 (behind the isfile guard at 2457: FIFO not reachable, size uncapped only)
  Recommended fix. Switch the two uncapped reads to _io_safety._read_capped_json. Add an fstat S_ISREG check on the opened descriptor, using O_NONBLOCK or a stat-before-open check for FIFOs, in all three verifiers and in the genesis-manifest read. Correct the 'RAM bounded' docstring or stream the chain walk.

• CR-A09-015 (code, Medium) — approve/reject make gate decisions from unauthenticated audit-log lines (no chain/HMAC check), and the approval record pins no digest of the staged artefact
  severity Medium · reachability documented-opt-in · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A co-tenant with write access to the output directory appends a forged human_approval.rejected line. The approver's forgelm approve refuses with 'already has a terminal decision', and approvals shows a decision nobody made.
  Where. forgelm/cli/subcommands/_audit_log_reader.py:93-111, forgelm/cli/subcommands/_audit_log_reader.py:158-192, forgelm/cli/subcommands/_audit_log_reader.py:195-227 (confirmed), forgelm/cli/subcommands/_approve.py:185-224 (confirmed)
  Recommended fix. When a secret is available, verify the HMAC (and the chain) of the events a decision relies on, or run verify_audit_log first. Catch ValueError and RecursionError in _parse_nonempty_line: in lenient mode skip and count, in strict mode raise AuditLogParseError. Cap line length.

• CR-A09-017 (code, Medium) — Annex IV 'placeholder' detection only catches empty values; the writer's own stub sentences pass as populated §2(g)/§9 content
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator with no risk_assessment block submits an Annex IV bundle that the verifier certified as §1-9 complete, although §9 carries only the generator's own 'not configured' stub.
  Where. forgelm/verify/_annex_iv.py:80-92, forgelm/compliance.py:1843-1846,1857,1861-1865, forgelm/cli/_parser.py:829-836, docs/usermanuals/en/compliance/annex-iv.md:71
  Recommended fix. Have the writer leave unconfigured sections empty, or mark them with a sentinel key the verifier treats as unpopulated. Alternatively, reword the help text to drop the claim about detecting placeholders.

• CR-A09-018 (code, Medium) — No doc says the audit log is not bound to a run; RTP R-04 rates the write-access adversary LOW residual and the incident SOP tells responders to walk the chain to rule out splicing
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace)
  Failure scenario. A regulated operator wires verify-audit --require-hmac exit 0 and verify-annex-iv --pipeline exit 0 into a release gate as proof that this run's Article 12 record is authentic, as R-04 and verify_audit.md suggest. A substituted or spliced log (findings 1-3) passes the gate, and the SOP's chain walk confirms 'no splice'.
  Where. docs/reference/verify_audit.md:35, docs/reference/verify_annex_iv_subcommand.md:73, docs/reference/verify_annex_iv_subcommand.md:49, docs/reference/verify_annex_iv_subcommand.md:141-153
  Recommended fix. Add a 'What a pass does not prove' section to verify_audit.md and the Annex IV pipeline section, and mirror it in the TR pages: no binding to the dir, run or artefacts, and forks validate. Document Tier 3 and audit_corroboration with their run_id selection. Re-rate R-04's residual and fix the SOP step.

• CR-A09-021 (code, Medium) — Uppercase-hex GGUF sidecar and magic-tamper-with-sidecar are unpinned; the predicate test pins the wrong behaviour (G-A09-02, G-A09-03)
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Where. tests/test_verification_toolbelt.py:345, tests/test_verification_toolbelt.py:1888, tests/test_verification_toolbelt.py:2092, forgelm/verify/_gguf.py:29

• CR-A09-022 (code, Medium) — RecursionError, unreadable directories and FIFO/device paths are unpinned for the single-path verifiers (G-A09-06, G-07, G-09)
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN VERIFY-UNCAPPED-READ — docs/roadmap/risks-and-decisions.md:263
  Where. tests/test_verification_toolbelt.py:3778, tests/test_verification_toolbelt.py:2655-2711, tests/test_verification_toolbelt.py:1393, forgelm/verify/_annex_iv.py:106-112

• CR-A09-024 (code, Medium) — No export-to-verify-gguf round-trip test, and the verify tests hand-write the sidecar that export never produces (the gap that hides M-X-028)
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced with stubs · already tracked: KNOWN-OPEN TRUST-04 — docs/roadmap/phase-16-trust-surface-hardening.md:440-441 (S5 open)
  Failure scenario. The producer and consumer drift apart, as they have: verify-gguf passes tampered exports with exit 0, and every test is green.
  Where. tests/test_export.py:155-180, tests/test_export.py:238-271, tests/test_verification_toolbelt.py:2007-2020, tests/test_verification_toolbelt.py:2096-2135
  Recommended fix. Add a round-trip test: export_model with a stub converter, then assert that '.sha256' exists and verify_gguf is valid; mutate a byte, then assert integrity failure (exit 6 via the CLI dispatcher). Fix M-X-028 first.

• CR-A09-026 (code, Medium) — verify-integrity and verify-gguf text renderers print untrusted strings raw, so control characters and terminal escapes reach the terminal or CI log
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator verifies a downloaded/shared model directory; the supplier adds a tampered or extra file whose name carries CSI sequences. verify-integrity exits 6, but the operator's terminal (or an interactive CI log viewer that honours cursor movement) shows 'OK ... present and unchanged'. A human gate that reads the screen, not the exit code, approves the artefact.
  Where. forgelm/cli/subcommands/_verify_integrity.py:144-152, forgelm/verify/_model_integrity.py:142-146,166-195,199-206, forgelm/cli/subcommands/_verify_gguf.py:119-123, forgelm/verify/_gguf.py:120-126,210
  Recommended fix. Add one shared output sanitiser (replace non-printable chars, including C0, DEL, C1 and bidi controls, with a repr-style escape) and apply it to every attacker-derived string interpolated into print() or a logger message; at minimum use {rel!r} or ascii(rel) in these three lines. Add a regression test that crafts a file name with ESC and asserts no raw 0x1b in stdout.

• CR-A09-027 (code, Medium) — No test pins any of these verify-integrity cases, and there is no train-then-verify round trip
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A regression or a still-present bug in symlink, ordering or normalisation handling ships with a green suite (as now).
  Where. tests/test_verification_toolbelt.py:1101-1419, tests/test_eu_ai_act.py:379-413, tests/test_oom_recovery.py:76-77, forgelm/trainer.py:1504-1505 (order that causes the unpinned false 6)
  Recommended fix. Add parametrised verify_integrity tests for each case above (skip on platforms without symlink), plus one test that runs the trainer's _finalize_artifacts on a tmp directory with stubbed config and metrics and asserts valid=True.

• CR-A09-028 (code, Medium) — verify-annex-iv --pipeline: unbounded resource amplification, one 8 MiB evidence file re-parsed and re-hashed once per stage row (about 0.3 to 0.8 s per stage, hours for CI)
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A CI job runs verify-annex-iv --pipeline on a run directory taken from an untrusted producer (an artefact from a fork PR, a vendor, or a downloaded bundle). The producer ships a 60 KB to 8 MiB manifest with 10^4 to 10^5 'completed' rows, all pointing at one 8 MiB valid evidence file. The verifier spends about 0.3 to 0.8 s of CPU per row (about 52 min at 10k rows, multiple hours at 50k) with 0.65 to 1.4 GB RSS, so the gate job runs into its timeout or ties up the runner; the compliance verdict i…
  Where. forgelm/verify/_pipeline_evidence.py:451 (per-stage loop), forgelm/verify/_pipeline_evidence.py:296-298 (was cited as a mechanism site, but this range is the _missing_evidence_outcome docstring and not the repeated-work path; the repeated work…, forgelm/verify/_pipeline_evidence.py:45 (PIPELINE_MANIFEST_MAX_BYTES bounds bytes, not stage count), forgelm/verify/_pipeline_evidence.py:45
  Recommended fix. Fix 1 (memoise, gain about 780x on the measured attack, 311 s to 0.40 s at N=1000; for 80k rows of small evidence 9.1 s to 5.8 s): in verify_pipeline_stage_evidence keep a per-call dict keyed on (os.path.realpath(path), st_dev, st_ino, st_size, st_mtime_ns) mapping to the verdict of the parse, validate and hash step (outcome plus message with the label substituted); the per-stage path resolution…

• CR-A09-029 (code, Medium) — No doc tells responders how a crash-torn audit log is reported or how to tell it from tampering; the reference and the SOP classify it as a Critical security incident
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · not reproduced (code trace)
  Failure scenario. A responder gets exit 6 'line 412 is not valid JSON: Unterminated string' after an OOM-killed run. Following the SOP, they open a Critical security incident, lock the directory and notify the DPO. They have no documented way to establish that it was a crash.
  Where. docs/reference/verify_audit.md:35, docs/reference/verify_audit-tr.md:35, docs/qms/sop_incident_response.md:77-105, docs/qms/sop_incident_response.md:107-160
  Recommended fix. Add a 'Crash-interrupted append' subsection to verify_audit.md (EN and TR) and a triage step to SOP 4.1. If the failing line is the last line, the file does not end in '\n', and 'head -n ' verifies, treat it as an interrupted write: keep a forensic copy, truncate to the last newline, and re-verify. Better still, have the verifier emit a distinct verdict (see the routing finding) and document…

• CR-A09-036 (code, Low) — verify-* failure and error output is inconsistent across verifiers (stdout vs stderr, missing 'error' key, no log before non-zero exit)
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · not reproduced (code trace)
  Failure scenario. A JSON consumer that keys on 'error' for failures misses pipeline-mode I/O errors. A text-mode consumer that separates stdout data from stderr diagnostics sees pipeline I/O errors on stdout.
  Where. forgelm/cli/subcommands/_verify_annex_iv.py:160-190 (confirmed), forgelm/cli/subcommands/_verify_annex_iv.py:313-355, forgelm/cli/subcommands/_verify_integrity.py:143-157 (confirmed), forgelm/cli/subcommands/_verify_gguf.py:118-129 (confirmed)
  Recommended fix. Add 'error' to the pipeline-mode error envelopes and route text-mode errors through logger.error. Decide whether a FAIL verdict counts as an error that must be logged under STD-086, and document the decision.

• ↳ same root cause: DOC-D1B07-012 (docs, Medium) — docs/standards/error-handling.md: wrong detail — Shipped error envelope {"success": false, "error": ...} is 'canonical, used by every CLI subcommand'
  severity Medium
  Where. docs/standards/error-handling.md:264-273 · type: wrong detail
  The document says. Shipped error envelope {"success": false, "error": ...} is 'canonical, used by every CLI subcommand'
  What the code does. Several verify-* failure envelopes carry no error key. Measured: verify-gguf on a bad-magic file emits {success, valid, reason, checks, path}, and verify-audit on a tampered log emits {success, valid, entries_count, hmac_verified, errors:[...]}. Other subcommands (verify-integrity, verify-annex-iv on a missing file, approvals, audit, a config error) do emit error.
  Code references. forgelm/cli/subcommands/_verify_gguf.py, forgelm/cli/subcommands/_verify_audit.py, forgelm/cli/_config_load.py:42-43
  Proposed correction. Fix the code (add an error string to the verify-gguf / verify-audit failure envelopes), or document the exception: 'verify-audit reports errors (list) and verify-gguf reports reason; neither currently sets error.'

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}