[Wave 4] Low/info — Module sweep: Compliance core: Annex IV, audit log, integrity manifest, provenance (part 2/5) #657

Wave 4 · 20 finding(s) in 20 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

• CR-A07-089 (code, Low) — iso_soc2_deployer_guide (EN+TR):188 says the no-load export 'always' produces hf_revision_source: unverified; the real --compliance-export gives local_path or unresolved in most configurations
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. An ISO/SOC2 auditor works from an air-gapped deployment, or a deployment that trains on local JSONL, and is handed a bundle regenerated with --compliance-export. Following iso_soc2_deployer_guide.md:188, the auditor expects unverified and finds unresolved or local_path, and flags a discrepancy that is in fact correct behaviour.
  Where. docs/guides/iso_soc2_deployer_guide.md:188, docs/guides/iso_soc2_deployer_guide-tr.md:188, docs/reference/compliance_summary.md:120, docs/reference/compliance_summary-tr.md:120
  Recommended fix. Say that a no-load export yields unverified only for an online, reachable Hub dataset id. It yields local_path for local corpora and unresolved (with a reason) when offline or when the lookup fails. Apply the same condition to compliance_summary.md:120 in EN and TR.

• CR-A07-090 (code, Low) — Deployer docs (guide and user manual) omit the 'local_path' hf_revision_source grade
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An auditor reviews a run that trained on local JSONL. The guide's table has no row for the grade the manifest shows, and the preceding sentence says pinning happens only under loaded. The auditor then either rejects valid content-hash evidence or treats an unknown grade as an anomaly.
  Where. docs/guides/iso_soc2_deployer_guide.md:176-189, docs/guides/iso_soc2_deployer_guide-tr.md:176-189, forgelm/compliance.py:1101-1113, docs/reference/compliance_summary.md:121
  Recommended fix. Add a local_path row in EN and TR: file pinned by sha256; directory identifies the path only and has no content hash. Reword :176-177 so that loaded is described as the only Hub-side evidence grade.

• CR-A07-091 (code, Low) — Four already-reported defects are pinned by no test: stripped-_hmac forgery, reserved-key overwrite, long-lived-logger re-root, non-finite metric hash (G-A07-01/04/05/08)
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A future fix for any of these four reported defects ships with no regression test, and a later refactor reintroduces it with the suite staying green.
  Where. forgelm/compliance.py:1897-1945 (compute_annex_iv_manifest_hash), forgelm/compliance.py:2407-2416 (skip-silently branch of _verify_hmac_for_entry, function 2396-2445), forgelm/compliance.py:543-544 (genesis guard only when _prev_hash == 'genesis'), forgelm/compliance.py:529-531
  Recommended fix. Add one regression test per scenario with the corrected expectation: a round trip with non-finite metrics, stripped-_hmac rejection when a secret is supplied, reserved keys not overwritten, and re-root refused after mid-process deletion.

• CR-A07-092 (code, Low) — fp4 labelling, evidence-bundle self-inclusion/symlink following and verify-integrity symlink handling have no test (G-A07-06/07/09)
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Failure scenario. The underlying defects (fp4 mislabelled as NF4, bundle self-inclusion, symlink content leaking into the bundle, false 'added' for an out-of-tree symlink) are all unpinned, so fixes and regressions go equally unseen.
  Where. forgelm/compliance.py:1523, forgelm/compliance.py:2342, forgelm/model_card.py:329, forgelm/compliance.py:2315-2322
  Recommended fix. Add a parametrised label test over nf4/fp4, a bundle test that places bundle_path inside and a symlink inside the output dir, and a verify_integrity test over a directory with an out-of-tree symlink (expectation depends on the fix chosen for the reported skipped_symlinks defect).

• CR-A07-093 (code, Low) — Catalog rows for compliance.* and human_approval.required contradict the code: no model card or audit zip in the bundle, SIGKILL cannot emit an event, section_missing also fires for a present-but-unr…
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. An auditor reads dataset_count=2 as 'two datasets used', or expects a model card and audit zip attested by compliance.artifacts_exported. They may also read section_missing as 'audit never run' when the audit file exists but is corrupt, or search the config for requires_human_approval and find nothing.
  Where. docs/reference/audit_event_catalog.md:51, docs/reference/audit_event_catalog.md:67, docs/reference/audit_event_catalog.md:68, docs/reference/audit_event_catalog.md:70
  Recommended fix. Fix the catalog wording in EN and TR. Rename dataset_count to split_count, or emit the real number of datasets (primary + extra_datasets). Add a reason (absent/unreadable) to governance_section_missing.

• CR-A07-094 (code, Low) — A crash-torn audit-log tail (SIGKILL/OOM/ENOSPC during log_event) is reported exactly like tampering: exit 6 / 'contradicted', with no tail diagnosis and no documented recovery
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A training container is OOM-killed, or the disk fills, during an audit append. The next CI verify-audit or verify-annex-iv --pipeline exits 6 'not valid JSON' and pages security for a non-event. The run directory can never be appended to again, and no document tells the operator this is a crash signature or how to recover.
  Where. forgelm/compliance.py:548-590, forgelm/compliance.py:279-293 (_read_chain_head trailing-newline guard raising ValueError), forgelm/compliance.py:2680-2683, forgelm/compliance.py:2543-2551 (_verify_chain_walk JSONDecodeError -> valid=False)
  Recommended fix. Classify 'only the final line is unparseable and the file does not end with a newline' as a distinct token (e.g. AUDIT_FAILURE_TORN_TAIL). Report entries_count of complete records, and route it consistently with the hard-kill rationale (1/UNVERIFIED), or keep 6 but with a reason that names the crash signature. Document the case and a recovery procedure in verify_audit.md and audit-log.md (EN+TR).

• CR-A07-095 (code, Low) — verify-integrity compares digests case-sensitively and never validates them: uppercase-hex, missing or null sha256 on an untouched dir gives exit 6
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · single reviewer · reproduced end to end
  Failure scenario. A Windows release step rebuilds or post-processes the manifest with Get-FileHash, which writes uppercase digests. Every subsequent verify-integrity run reports all weights as tampered.
  Where. forgelm/verify/_model_integrity.py:191-193, forgelm/verify/_model_integrity.py:161-175
  Recommended fix. Apply the same fix as for M-A09-002 across all verify-* digest comparisons.

• CR-A07-096 (code, Low) — verify-integrity's added-file walk compares exact relpath strings with no normalisation or allow-list, giving false 'added' (exit 6) on untouched dirs
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A model with a non-ASCII file name is sealed on Linux and verified on macOS after a copy that normalised names (or the reverse). verify-integrity exits 6 while reporting the same file as both verified and added.
  Where. forgelm/verify/_model_integrity.py:142-146, forgelm/verify/_model_integrity.py:176-195, forgelm/verify/_model_integrity.py:199-206
  Recommended fix. Normalise and deduplicate recorded entries once, before both the hashing loop and the walk comparison.

• CR-A07-097 (code, Low) — Hidden OS files, case-insensitive FS and duplicate stage evidence pointers are unpinned (G-A09-08, G-A09-10)
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Where. tests/test_verification_toolbelt.py:3182, forgelm/verify/_model_integrity.py:199-206 (the walk) and forgelm/compliance.py:787-805 (the writer records .DS_Store), forgelm/verify/_pipeline_evidence.py:242-243, forgelm/verify/_pipeline_evidence.py:500-511 (the _verify_stage_evidence call; no shared-pointer check)

• CR-A07-098 (code, Low) — audit_event_catalog.md payload drift: data.erasure_failed carries undocumented match_count / files_modified and an unlisted trigger; human_approval.required names a non-existent config key
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. A compliance parser built strictly against the catalog either rejects data.erasure_failed lines that carry match_count/files_modified or drops them, losing the partial-deletion list. An operator searching the config reference for requires_human_approval finds nothing.
  Where. docs/reference/audit_event_catalog.md:79, docs/reference/audit_event_catalog.md:51, docs/reference/audit_event_catalog-tr.md:51, forgelm/cli/subcommands/_purge.py:568
  Recommended fix. Extend row 79 to say '+ optional match_count (MultiMatchRefused), files_modified (run-mode partial deletion)' and add the path-traversal refusal trigger. Fix row 51 (EN+TR) to evaluation.require_human_approval: true.

• CR-A07-099 (code, Low) — verify-integrity hashes earlier entries before refusing a later escaping manifest entry, contrary to 'returns before any artifact is hashed'
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A third-party model directory ships a model_integrity.json that lists 100 GB of shards followed by '../x'. verify-integrity streams every shard through SHA-256 and then reports 'I refused to hash' with exit 1, after the doc promised the refusal happens before any hashing.
  Where. forgelm/verify/_model_integrity.py:153-195, docs/reference/verify_integrity_subcommand.md:32, docs/reference/verify_integrity_subcommand.md:36, docs/usermanuals/en/compliance/verify-integrity.md:98
  Recommended fix. Validate all entries (dict, string 'file', containment) in a first pass before any hash_file call, or reword the docs.

• CR-A07-100 (code, Low) — verify-integrity crashes with a raw ValueError traceback on a manifest entry containing a NUL character
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A crafted model_integrity.json with "file": "a\u0000b" makes forgelm verify-integrity --output-format json emit a Python traceback instead of the documented {success:false,...} envelope. CI parsers that expect JSON break.
  Where. forgelm/verify/_model_integrity.py:176-177, forgelm/cli/subcommands/_verify_integrity.py:85-129
  Recommended fix. Reject entries whose 'file' contains '\x00' with the same input-error VerifyIntegrityResult, or catch ValueError around realpath.

• CR-A07-101 (code, Low) — training_manifest.yaml does not carry config_hash despite ISO A.8.9 claim
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. An auditor follows the A.8.9 evidence row to bind training_manifest.yaml to the config that ran and finds no hash. The binding exists only in compliance_report.json.
  Where. forgelm/compliance.py:2177-2194, forgelm/compliance.py:2192, forgelm/cli/_pipeline.py:1634-1635, docs/reference/iso27001_control_mapping.md:110
  Recommended fix. Add manifest['config_hash'] to yaml_manifest in export_compliance_artifacts, or reword A.8.9 (EN+TR) to name compliance_report.json.

• CR-A07-102 (code, Low) — config_hash does not change when only secret values change (hf_token / api_key are redacted before hashing)
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. An auditor relies on the QMS statement to detect a credential swap mid-run by comparing config_hash values. A swap of the HF token or teacher API key leaves config_hash unchanged.
  Where. forgelm/compliance.py:1331-1362, forgelm/config.py:1453, forgelm/config.py:1518, forgelm/config.py:1654
  Recommended fix. Correct encryption_at_rest.md (EN+TR) to say secrets are redacted before hashing, so a secret-only change is NOT detectable via config_hash. Alternatively, hash a keyed digest of the secret values.

• CR-A07-103 (code, Low) — export_evidence_bundle ships the per-output-dir identifier salt alongside the audit log
  severity Low · reachability documented-opt-in · intent intended-but-harmful · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. An operator bundles ./outputs for an external auditor. The auditor, or anyone the zip leaks to, takes the salt from the bundle and brute-forces data.access_request_query.query_hash and data.erasure_requested.target_id against an email wordlist, recovering which data subjects filed Art. 15/17 requests.
  Where. forgelm/compliance.py:2299-2331, forgelm/cli/subcommands/_purge.py:76-81, docs/reference/architecture.md:147, forgelm/cli/subcommands/_purge.py:153-229
  Recommended fix. Exclude .forgelm_audit_salt (and other secret-bearing dotfiles) from export_evidence_bundle, and test that exclusion. Document that the salt must never leave the output dir.

• CR-A07-104 (code, Low) — Audit-log user-manual example shows a prev_hash/timestamp format the writer never emits
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. A SIEM ingest rule written from the manual's example expects 'Z' and a 'sha256:' prefix and rejects or mis-correlates every real entry.
  Where. docs/usermanuals/en/compliance/audit-log.md:15-18, forgelm/compliance.py:554-561, forgelm/compliance.py:606, docs/usermanuals/en/compliance/audit-log.md:16-18
  Recommended fix. Update the EN/TR example to the real shape (an isoformat timestamp with +00:00 and microseconds, and a bare-hex prev_hash).

• CR-A07-105 (code, Low) — library_api.md claims an unset FORGELM_OPERATOR aborts the run; the code silently falls back to @
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. An Airflow DAG forgets to set FORGELM_OPERATOR. Following the guide, the author expects a hard failure, but events are attributed to the worker's OS user@host and the run succeeds.
  Where. docs/guides/library_api.md:237, docs/guides/library_api.md:75, forgelm/compliance.py:192-229, docs/guides/library_api-tr.md:238
  Recommended fix. Reword library_api.md:237 and its TR mirror: 'If FORGELM_OPERATOR is unset, the operator falls back to <getpass.getuser()>@; only when no username resolves and FORGELM_ALLOW_ANONYMOUS_OPERATOR is not 1 does the run abort.'

• CR-A07-106 (code, Low) — Enterprise guide misplaces audit_log.jsonl and overstates the automatically generated evidence bundle
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Failure scenario. u233 single-stage run: out/compliance contains 4 files; audit_log.jsonl is at out/audit_log.jsonl.
  Where. docs/guides/enterprise_deployment.md:201-226, forgelm/compliance.py:175
  Recommended fix. Correct the tree to show audit_log.jsonl and audit_log.jsonl.manifest.json at the output_dir root, and mark which files need a compliance: or risk_assessment: block.

• CR-A07-107 (code, Low) — An 'unattested' corroboration from an unreadable audit log exits 2, while annex-iv.md and json-output.md say unattested exits 1
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end · already tracked: KNOWN-OPEN C31-EXIT-CODE-PROSE — docs/roadmap/phase-16-trust-surface-hardening.md:782-785 (S15 open)
  Failure scenario. A CI rule written from the doc ('unattested → 1') mishandles an audit_log_unreadable outcome.
  Where. forgelm/compliance.py:3229-3237, docs/usermanuals/en/compliance/annex-iv.md:174, forgelm/compliance.py:3235-3241, docs/usermanuals/en/reference/json-output.md:767
  Recommended fix. Document that unattested/audit_log_unreadable exits 2.

• CR-A07-108 (code, Low) — verify_integrity_subcommand.md exit-code table omits the empty-artifacts and other malformed-manifest exit-1 cases
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end · already tracked: KNOWN-OPEN C31-EXIT-CODE-PROSE — docs/roadmap/phase-16-trust-surface-hardening.md:782-785 (S15 open)
  Failure scenario. An integrator writes CI branching from the reference page alone and does not treat exit 1 from an empty manifest as 'nothing was verified'.
  Where. docs/reference/verify_integrity_subcommand.md:32, docs/reference/verify_integrity_subcommand.md:42, forgelm/verify/_model_integrity.py:92-138
  Recommended fix. Add the four missing exit-1 cases to the EN and TR reference tables so they match json-output.md:859.

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}