Skip to content

Documented run layout places the evidence bundle under artifacts/; code writes compliance/ and keeps audit_log.jsonl at the output root (CR-A02-027) #432

Open

Description

@cemililik

Wave 1 · theme D1-8: Docs describing features that do not exist (doc side of code findings) · highest severity High · 1 finding(s) · effort small

Summary

A CI step that archives checkpoints//artifacts/ as the regulator deliverable, per project-layout.md:126, archives nothing.

Primary finding

CR-A02-027 — Documented run layout places the evidence bundle under artifacts/; code writes compliance/ and keeps audit_log.jsonl at the output root

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end

What happens. first-run.md:160-176 and project-layout.md:28,53,126-131 describe checkpoints//artifacts/ holding annex_iv_metadata.json, audit_log.jsonl, data_audit_report.json, safety_report.json, benchmark_results.json and manifest.json. The trainer writes <output_dir>/compliance/ (trainer.py:2087) and audit_log.jsonl at the output root. data_audit_report.json is never produced by training, and safety_report.json is not a ForgeLM filename (the safety module writes safety_results.json). training/sft.md:136 correctly says compliance/, so the manual contradicts itself.

Where.

Evidence.

  • run1 file listing: out/compliance/*, out/audit_log.jsonl, no artifacts/.

Recommended fix. Align the getting-started pages (EN and TR) with the real layout.

Acceptance criteria

  • A regression test reproduces the failure scenario and asserts the corrected behaviour.
  • Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
  • Every finding in this issue is resolved or explicitly closed with a reason.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: trainerCode area: trainerbugSomething isn't workingseverity: highWrong behaviour or a false claim users rely onsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 1Roadmap wave 1: high severity, small effort

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions