--compliance-export produces no evidence-bundle ZIP; export_evidence_bundle is unreachable from the CLI (CR-A04-006) #375

Wave 1 · theme D1-2: Audit chain, integrity manifest and verifier robustness · highest severity High · 2 finding(s) · effort small

Summary

The document states: Evidence Bundle Export: forgelm --config job.yaml --compliance-export ./audit_bundle/ packages all compliance artifacts into a single auditor-ready archive that includes annex_iv_metadata.json, compliance_report.json, data_provenance.json, data_governance_report.json, risk_assessment.json, model_integrity.json, deployer_instructions.md, audit_log.jsonl, model_card.md, safety_results.json and benchmark_results.json. Every artifact must be exportable via --compliance-export without GPU.. Observed in the code: --compliance-export writes up to five loose files: compliance_report.json, training_manifest.yaml, data_provenance.json, risk_assessment.json (when configured) and annex_iv_metadata.json. A run confirmed this. The ZIP helper export_evidence_bundle has no caller in forgelm/; only tests call it. Six of the eleven listed files are never produced by this command.

Primary finding

CR-A04-006 — --compliance-export produces no evidence-bundle ZIP; export_evidence_bundle is unreachable from the CLI

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end

What happens. _run_compliance_export (cli/_no_train_modes.py:248-261) only calls export_compliance_artifacts, which writes three to five loose files. export_evidence_bundle (compliance.py:2299-2330), the only ZIP writer, has no caller in forgelm/. Documents promising an auditor ZIP with audit log, model card and other artefacts (completed-phases, SoA, encryption_at_rest) are therefore unmet.

Where.

• forgelm/cli/_no_train_modes.py:248-259
• forgelm/compliance.py:2137-2291
• forgelm/compliance.py:2299-2328
• forgelm/cli/_no_train_modes.py:248-261 (confirmed)
• forgelm/cli/_parser.py:1445-1459
• docs/qms/statement_of_applicability.md:121
• docs/qms/encryption_at_rest.md:180-183
• forgelm/cli/_no_train_modes.py:281-283
• docs/qms/encryption_at_rest.md:30
• docs/reference/compliance_summary.md:27

Evidence.

• forgelm/cli/_no_train_modes.py:248-282; forgelm/compliance.py:2137-2290; forgelm/compliance.py:2299-2330
• The run produced comp/{compliance_report.json, training_manifest.yaml, data_provenance.json}; no .zip anywhere; --output bundle_dir was not created.

Recommended fix. Fix the code: wire export_evidence_bundle (for example a flag on --compliance-export) or remove the promise. In the meantime, rewrite the task text to: "--compliance-export DIR writes compliance_report.json, training_manifest.yaml, data_provenance.json, risk_assessment.json (if configured) and annex_iv_metadata.json; data_governance_report.json, model_integrity.json, deployer_instructions.md, audit_log.jsonl and the model card are produced only by a training run. No ZIP archive is produced from the CLI."

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

DOC-D2B07-005 — docs/roadmap/completed-phases.md: false claim — Evidence Bundle Export: forgelm --config job.yaml --compliance-export ./audit_bundle/ packages all compliance artifacts into a single auditor-ready a…

severity High

Where. docs/roadmap/completed-phases.md:343-348, 375 · type: false claim

The document says. Evidence Bundle Export: forgelm --config job.yaml --compliance-export ./audit_bundle/ packages all compliance artifacts into a single auditor-ready archive that includes annex_iv_metadata.json, compliance_report.json, data_provenance.json, data_governance_report.json, risk_assessment.json, model_integrity.json, deployer_instructions.md, audit_log.jsonl, model_card.md, safety_results.json and benchmark_results.json. Every artifact must be exportable via --compliance-export without GPU.

What the code does. --compliance-export writes up to five loose files: compliance_report.json, training_manifest.yaml, data_provenance.json, risk_assessment.json (when configured) and annex_iv_metadata.json. A run confirmed this. The ZIP helper export_evidence_bundle has no caller in forgelm/; only tests call it. Six of the eleven listed files are never produced by this command.

Code references. forgelm/cli/_no_train_modes.py:248-282, forgelm/compliance.py:2137-2290, forgelm/compliance.py:2299-2330

User impact. Auditors handed the export directory believe they hold the complete Art. 11/17 bundle. Governance, integrity, deployer instructions, audit log and evaluation results are missing, and no archive is produced.

Proposed correction. Fix the code: wire export_evidence_bundle (for example a flag on --compliance-export) or remove the promise. In the meantime, rewrite the task text to: "--compliance-export DIR writes compliance_report.json, training_manifest.yaml, data_provenance.json, risk_assessment.json (if configured) and annex_iv_metadata.json; data_governance_report.json, model_integrity.json, deployer_instructions.md, audit_log.jsonl and the model card are produced only by a training run. No ZIP archive is produced from the CLI."

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}