Single-artefact verify-annex-iv exits 0 (valid=true, plain 'OK') when metadata.manifest_hash is removed, null or empty (CR-A09-009) #364

Wave 1 · theme D1-11: Other high-severity packages · highest severity High · 3 finding(s) · effort small

Summary

A CI gate keyed on exit 0 or valid accepts an edited artefact whose hash was simply deleted.

Primary finding

CR-A09-009 — Single-artefact verify-annex-iv exits 0 (valid=true, plain 'OK') when metadata.manifest_hash is removed, null or empty

severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN TRUST-05 — docs/roadmap/phase-16-trust-surface-hardening.md:440-442 (S5 open)

What happens. verify_annex_iv_payload / verify_annex_iv_artifact (forgelm/verify/_annex_iv.py:149-185) runs tamper detection only when metadata.manifest_hash is present and truthy. Otherwise it returns valid=True with 'tampering detection skipped', and the CLI (_verify_annex_iv.py:309-312) prints a plain OK and exits 0. The pipeline per-stage path routes the same artefact to UNVERIFIED (exit 1, _pipeline_evidence.py:413-417), and the chain manifest's absent hash exits 0 as 'OK (UNVERIFIED)'. So exit 0 does not imply that a comparison ran.

Where.

• forgelm/verify/_annex_iv.py:184
• forgelm/compliance.py:1878
• forgelm/verify/_annex_iv.py:158-185
• forgelm/verify/_annex_iv.py:180
• forgelm/cli/subcommands/_verify_annex_iv.py:296-306 (_print_artefact_result; the OK branch is at 303-306)
• forgelm/verify/_pipeline_evidence.py:413-417,505-507
• docs/reference/verify_annex_iv_subcommand.md:32,41-44,82-85
• forgelm/verify/_annex_iv.py:159-185
• forgelm/cli/subcommands/_verify_annex_iv.py:228-236 (disputed)
• forgelm/verify/_gguf.py:194-195

Evidence.

• X3/X4/X7/X8 (intended_purpose edited, hash stripped, null or empty) all exit 0 in text and json with valid=true.
• verify/_annex_iv.py:149-172 (expected falsy -> valid=True); s4_verify.py A2/A3/A4 valid=True; CLI: fpy -m forgelm verify-annex-iv ax_strip.json -> exit 0.
• scripts/bind.py: artefact edited with the hash kept -> exit 6. Edited with metadata deleted -> exit 0, 'OK: ... All Annex IV §1-9 fields populated; no manifest_hash present so tampering detection skipped'. The same hash-deleted file used as dpo_stage evidence -> --pipeline exit 1 'is complete but carries no manifest_hash — tampering could not be checked'.

Recommended fix. Mirror pipeline mode: add hash_state to the single-artefact envelope, print 'OK (UNVERIFIED)', or reject a missing hash on artefacts whose forgelm_version is at or above the version that started stamping.

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

DOC-BHELP-001 — forgelm: misleading — 'recomputes the manifest hash to detect post-generation tampering. Exits 0 on valid; 1 ...; 6 when every field is populated but the manifest hash no longer matches'.

severity High · already tracked: KNOWN-OPEN TRUST-05 — docs/roadmap/phase-16-trust-surface-hardening.md:438-441

Where. forgelm/cli/_parser.py:829-836 · type: misleading

The document says. 'recomputes the manifest hash to detect post-generation tampering. Exits 0 on valid; 1 ...; 6 when every field is populated but the manifest hash no longer matches'.

What the code does. If metadata.manifest_hash is absent the verifier returns valid=True and exit 0 ('tampering detection skipped'). The stamp is an unkeyed SHA-256 from a public function. Verified with a crafted artefact: good.json rc 0, edited-with-hash tampered.json rc 6, edited-and-stripped (metadata removed) rc 0. The help never says a hash-less artefact passes.

Code references. forgelm/verify/_annex_iv.py:159-186, forgelm/cli/subcommands/_verify_annex_iv.py:212-233

User impact. A verify-* command that is meant to be the exit-6 tamper signal exits 0 for a tampered file once the hash block is removed.

Proposed correction. Fail (exit 1 or 6) when a hash-less artefact is given, or add an explicit --allow-unhashed flag. Reword the help to say the check detects accidental edits, not an adversary who can edit the file.

DOC-D3B04A-005 — docs/usermanuals/en/reference/cli.md: misleading — Single-artefact verify-annex-iv 'validates the nine Annex IV §1-9 field categories and recomputes the manifest hash. Exit 0 when valid … 6 whe…

severity High · already tracked: KNOWN-OPEN TRUST-05 docs/roadmap/phase-16-trust-surface-hardening.md:413-419,440-441

Where. docs/usermanuals/en/reference/cli.md:225 (Turkish mirror: docs/usermanuals/tr/reference/cli.md:225) · type: misleading

The document says. Single-artefact verify-annex-iv 'validates the nine Annex IV §1-9 field categories and recomputes the manifest hash. Exit 0 when valid … 6 when every field is populated but the manifest hash no longer matches'.

What the code does. When the artefact carries no metadata.manifest_hash, the verifier returns valid=True with reason 'no manifest_hash present so tampering detection skipped' and the CLI exits 0. Reproduced: a hand-written artefact without a metadata block gave success:true, EXIT=0. ForgeLM's exporter always stamps the hash, so a missing hash means the file was edited. Pipeline mode treats the same condition as UNVERIFIED (exit 1). The hash is also unkeyed SHA-256.

Code references. forgelm/verify/_annex_iv.py:157-185, forgelm/compliance.py:1878, forgelm/verify/_pipeline_evidence.py:413-416, forgelm/cli/subcommands/_verify_annex_iv.py:348-355

User impact. An attacker who edits fields and deletes metadata gets exit 0 from the CI integrity gate. Readers assume tamper detection always runs.

Proposed correction. Append: "If the artefact carries no metadata.manifest_hash, the hash step is skipped and the command still exits 0 (reason: 'tampering detection skipped'); ForgeLM-exported artefacts always carry the hash, so treat a missing hash as suspect. The hash is an unkeyed SHA-256: it detects accidental or naive edits, not an adversary who can rewrite the file and recompute it."

Turkish mirror. the Turkish page has no such statement

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}