[Wave 4] Low/info — Module sweep: Compliance core: Annex IV, audit log, integrity manifest, provenance (part 1/5) #656

Wave 4 · 20 finding(s) in 20 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

• CR-A07-067 (code, Low) — The write-time re-root/truncation guard is skipped by a long-lived AuditLogger (cached _prev_hash) when the log is deleted or emptied mid-run
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. During a multi-hour training run, someone with write access to output_dir deletes audit_log.jsonl. The trainer keeps appending a new chain from 'genesis' with no error, although the guard is documented to refuse exactly this re-root at write time.
  Where. forgelm/compliance.py:552-553, forgelm/compliance.py:419 (confirmed), forgelm/compliance.py:313-320, forgelm/compliance.py:454-455
  Recommended fix. Run the manifest/re-root check under the lock whenever the under-lock head read yields 'genesis' (not when the cached value does), and define 'empty' as 'no decoded entries' rather than size 0. Optionally log an ERROR when a non-genesis logger finds the log re-rooted.

• CR-A07-068 (code, Low) — An empty FORGELM_AUDIT_SECRET at verify time is used as an HMAC key, giving a false 'HMAC mismatch' (exit 6), while the writer treats empty as 'no secret'
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A CI secret fails to template and is exported as an empty string. verify-audit pages the security on-call with exit 6 for an intact log.
  Where. forgelm/compliance.py:238-239, forgelm/compliance.py:2429, forgelm/compliance.py:2418, forgelm/cli/subcommands/_verify_audit.py:234
  Recommended fix. Normalise '' to None in the CLI (os.getenv(var) or None) and in _verify_audit_log_classified, matching the writer and the --require-hmac precheck.

• CR-A07-069 (code, Low) — QMS/ISO documents cite stale compliance.py line numbers for AuditLogger and the HMAC key derivation
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace)
  Failure scenario. An auditor following the citation lands on a comment rather than the derivation.
  Where. docs/qms/access_control.md:82 (confirmed), docs/qms/risk_treatment_plan.md:81 (confirmed), docs/qms/statement_of_applicability.md:132 (confirmed), docs/design/iso27001_soc2_alignment.md:255
  Recommended fix. Cite the symbol (AuditLogger.__init__) without a line number, or update to :258.

• CR-A07-070 (code, Low) — Art. 13 deployer instructions: hardcoded NVIDIA hardware requirements, non-float metrics silently dropped, staging path recorded as model location
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end
  Failure scenario. A deployer of a CPU-sized model, or of a 70B model, receives hardware guidance unrelated to the system and an incomplete metric table.
  Where. forgelm/compliance.py:1697, forgelm/compliance.py:1736-1738, forgelm/compliance.py:1747-1751, forgelm/trainer.py:1843
  Recommended fix. Derive hardware notes from resource_usage and quantization, or label them as generic. Render all numeric (non-bool) metrics. Regenerate or patch the document on approve.

• CR-A07-071 (code, Low) — Annex IV writer fills §2(g)/§9 with placeholders that defeat the verifier's completeness rule; the §2(g) placeholder cites a nonexistent --resource-tracking flag
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · not reproduced (code trace)
  Failure scenario. A high-risk run without a risk_assessment block passes verify-annex-iv with an 'all §1-9 populated' message.
  Where. forgelm/compliance.py:1843-1847, forgelm/compliance.py:1861-1864, forgelm/verify/_annex_iv.py:82-92
  Recommended fix. Use empty or explicitly typed 'not_provided' markers that the verifier recognises as missing (or reports as a warning class). Remove the nonexistent flag reference.

• CR-A07-072 (code, Low) — compliance.export_pipeline_manifest is dead in production, yet its docstring says the orchestrator calls it; the shipped writer (_pipeline._atomic_write_json) differs in fsync and default= handling
  severity Low · reachability unreachable · intent intended-but-harmful · confidence likely · independently confirmed · not reproduced (code trace)
  Failure scenario. A reviewer trusts the fsync and atomicity description of the pipeline manifest based on export_pipeline_manifest, while production uses a non-fsynced writer.
  Where. forgelm/compliance.py:3056-3059 (docstring: 'the orchestrator calls this on every stage transition'), forgelm/cli/_pipeline.py:255-260 (no-fsync is documented on _atomic_write_json), forgelm/cli/_pipeline.py:1373-1374 (_save_state docstring: 'Manifest export is delegated to forgelm.compliance'), forgelm/compliance.py:3053
  Recommended fix. Route the orchestrator through one writer (or delete the unused one), and make its docstring describe what ships.

• CR-A07-073 (code, Low) — verify_integrity re-hashes artefacts serially while generate_model_integrity uses a 4-thread pool (optional speedup)
  severity Low · reachability default · intent intended-and-acceptable · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. A 70B model with 30 or more safetensors shards is checked in a CI gate with verify-integrity. The check is bound by one thread of SHA-256 and one stream of I/O. With four threads, expected wall time on fast storage drops to roughly 25 to 40 percent.
  Where. forgelm/verify/_model_integrity.py:153-195, forgelm/compliance.py:807-811 (confirmed), forgelm/verify/_model_integrity.py:139-171, forgelm/compliance.py:743-757
  Recommended fix. Hash recorded entries with a ThreadPoolExecutor(max_workers=4) after the path-containment checks, and keep the result lists sorted. Expected gain is up to about 3 to 4x wall time on multi-shard models; it is not worth doing for single-file models.

• CR-A07-074 (code, Low) — Art. 10 governance metadata (data.governance) and local-dataset SHA-256 provenance are asserted by no test
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Where. forgelm/compliance.py:654-665, forgelm/compliance.py:726-728, forgelm/compliance.py:841-843 (fingerprint sha256), forgelm/compliance.py:750-757 (hash_file size_bytes)
  Recommended fix. Add a generate_data_governance_report test with config.data.governance populated asserting every key/value, and assert the local fingerprint's sha256 equals hashlib.sha256 of the file contents, plus size_bytes and a recorded modified timestamp.

• CR-A07-075 (code, Low) — Manifest content that backs Annex IV claims is unasserted: safety/judge/benchmark results, extra_datasets provenance, annex monitoring and risk_management sections, trust_remote_code and quantization…
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Where. forgelm/compliance.py:1523-1524, forgelm/compliance.py:1574-1579, forgelm/compliance.py:1620-1625, forgelm/compliance.py:1857-1863
  Recommended fix. Add manifest/annex tests with safety_result, judge_result, benchmark_result, extra_datasets, monitoring and risk_assessment set, asserting the values appear in compliance_report.json and in annex_iv_metadata.json sections 6 and 9; assert trust_remote_code=True and 4-bit lineage are reflected.

• CR-A07-076 (code, Low) — Dataset-fingerprint provenance tests are not hermetic: they fail when HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE / HF_DATASETS_OFFLINE is exported
  severity Low · reachability default · intent contract-violation · confidence confirmed · single reviewer · reproduced end to end
  Where. tests/test_compliance.py:1022-1063, tests/test_revision_pin_chains.py, tests/test_eu_ai_act.py:675-689, tests/conftest.py
  Recommended fix. Add an autouse fixture that deletes HF_HUB_OFFLINE, TRANSFORMERS_OFFLINE and HF_DATASETS_OFFLINE (monkeypatch.delenv) for these classes, or for the whole suite, so tests state their own offline mode.

• CR-A07-077 (code, Low) — Genesis-manifest hash and run_id checks in the audit verifier are each unpinned by tests (mutants survive) because the only test changes both at once
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A canonicalisation change makes old artefacts fail verification (false tamper alarms); no test notices.
  Where. forgelm/compliance.py:1897-1946 (confirmed), forgelm/compliance.py:2513-2519, tests/test_compliance.py:1968-2001, forgelm/compliance.py:2406-2413
  Recommended fix. Add a golden vector (artefact JSON plus expected digest) and a same-run_id re-root test.

• CR-A07-078 (code, Low) — audit_event_catalog.md row for cache.populate_tasks_failed omits the payload shape the code emits when no task is staged
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · not reproduced (code trace)
  Failure scenario. An auditor's parser written against the catalog expects error_class on every cache.populate_tasks_failed event and breaks, or misreports, on the all-uncached variant.
  Where. docs/reference/audit_event_catalog.md:99, forgelm/cli/subcommands/_cache.py:511-518, docs/reference/cache_subcommands.md:80
  Recommended fix. Copy the cache_subcommands.md:80 payload wording into the catalog row (and its TR mirror).

• CR-A07-079 (code, Low) — Audit-log docs give the wrong location, wrong run_id format and wrong log level
  severity Low · reachability default · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace)
  Where. docs/standards/logging-observability.md:189, docs/reference/verify_audit.md:48, docs/reference/verify_audit-tr.md:48, docs/reference/audit_event_catalog.md:15
  Recommended fix. Fix the paths (EN+TR), describe run_id per writer, and either raise the missing-manifest log to WARNING or correct the manual.

• CR-A07-081 (code, Low) — Pipeline chain binds stages by path only: no digest of a stage's input/output model; resume hash covers raw YAML bytes only
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · not reproduced (code trace)
  Failure scenario. After stage 1, an output-dir writer swaps final_model. Stage 2 resumes and trains on it. The chain manifest still verifies, because only paths are compared.
  Where. forgelm/compliance.py:3010-3040, forgelm/cli/_pipeline.py:178-188, forgelm/cli/_pipeline.py:1295-1340
  Recommended fix. Record the stage output's model_integrity digest as stage N+1 input digest and re-verify before loading; include dataset fingerprints in the resume guard.

• CR-A07-082 (code, Low) — FORGELM_AUDIT_SECRET with non-UTF-8 bytes crashes AuditLogger with a raw UnicodeEncodeError; non-ASCII secrets depend on the locale
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. CI exports a binary-derived secret; every training run dies at start with a Python traceback instead of a config error.
  Where. forgelm/compliance.py:258, forgelm/compliance.py:2429, forgelm/cli/subcommands/_verify_audit.py:231, forgelm/cli/_training.py:173
  Recommended fix. Encode with os.fsencode / surrogateescape consistently or reject non-UTF-8 secrets with a ConfigError that names the variable.

• CR-A07-083 (code, Low) — Key-handling hygiene: ad-hoc KDF without separation, audit secret reused as purge salt, and envelope keys that callers can override
  severity Low · reachability default · intent intended-but-harmful · confidence likely · independently confirmed · reproduced end to end
  Where. forgelm/compliance.py:258, forgelm/compliance.py:2429, forgelm/cli/subcommands/_purge.py:216-222, forgelm/compliance.py:555-562
  Recommended fix. Derive keys with HMAC(secret, 'forgelm-audit-v1' || run_id) and a separate label for the salt. Reject reserved envelope keys in **details. Treat an empty secret env var at verify time as unset. Fix the design doc and the stale RTP line reference.

• CR-A07-084 (code, Low) — TestBlackHoledHubDoesNotHang never reaches its black-hole server in a normal run, so the 'end-to-end timeout proof' is vacuous and order-dependent
  severity Low · reachability unreachable · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end
  Where. tests/test_revision_pin_chains.py:649, tests/test_revision_pin_chains.py:741, forgelm/compliance.py:1220
  Recommended fix. Pass the endpoint explicitly (HfApi(endpoint=...) via a patched factory or patch huggingface_hub.constants.ENDPOINT) and assert the server accepted a connection.

• CR-A07-085 (code, Low) — No-network guard is a connect() tripwire that production best-effort handlers swallow; seven tests make live Hub DNS/connect attempts on every run
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · reproduced with stubs
  Where. tests/conftest.py:73, forgelm/compliance.py:1220, tests/test_compliance.py:940, tests/test_grpo_reward.py:62
  Recommended fix. Record guard violations in the fixture and fail in teardown even when swallowed; stub prepare_revision_pin in the seven tests; also guard socket.getaddrinfo for non-loopback names.

• CR-A07-087 (code, Low) — data.governance metadata reaches only data_governance_report.json, not the Art. 11 manifest or Annex IV
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end
  Failure scenario. An auditor reading Annex IV 2(d) finds no declared collection method, biases or DPIA flag.
  Where. forgelm/compliance.py:707, forgelm/compliance.py:1848, forgelm/trainer.py:2085
  Recommended fix. Include the governance block in the manifest and Annex IV data_governance, and in the standalone export.

• CR-A07-088 (code, Low) — generate_model_integrity hangs forever on a FIFO (or other non-regular file) inside the model directory
  severity Low · reachability default · intent intended-but-harmful · confidence likely · independently confirmed · reproduced end to end
  Where. forgelm/compliance.py:743-757, forgelm/compliance.py:787-809, forgelm/trainer.py:2157-2175
  Recommended fix. Skip non-regular files (stat.S_ISREG after lstat) and record them under a skipped list, the same way skipped_symlinks is recorded.

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}