Purge --run-id --kind artefacts can never match the compliance bundle ForgeLM writes; generic run-id tokens select unrelated files (CR-A10-002) #477

Wave 2 · theme D2-5: Data audit, masking and ingestion: medium effort · highest severity High · 5 finding(s) · effort medium

Summary

After a 1-step SFT run (run fg-ad448c0755ea), forgelm purge --run-id fg-ad448c0755ea --kind artefacts --output-dir runA/out exits 1 with 'No 'artefacts' artefacts found' and logs erasure_failed NoMatchingArtefacts.

Primary finding

CR-A10-002 — Purge --run-id --kind artefacts can never match the compliance bundle ForgeLM writes; generic run-id tokens select unrelated files

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN TRUST-02 — docs/roadmap/phase-16-trust-surface-hardening.md:553-571 (S9 open)

What happens. _artefact_targets_for_run (_purge.py:998-1014, with _filename_contains_run_id at :1041-1064) selects only compliance/ files whose name contains the run_id as a _-. delimited token. export_compliance_artifacts (compliance.py:2173-2236) and the trainer (trainer.py:2087-2096) write fixed names with no run_id, so a real run id always ends in NoMatchingArtefacts (exit 1), while word-like ids ('report', 'json') select shared bundle files. Tests pass only because they fabricate run-id-named files.

Where.

• forgelm/cli/subcommands/_purge.py:998-1014 (_artefact_targets_for_run, docstring claim at :999-1004)
• forgelm/compliance.py:2173-2236 (fixed output names; data_governance_report.json is written by forgelm/trainer.py:2087-2096)
• forgelm/compliance.py:2228-2236
• forgelm/trainer.py:2087-2096
• forgelm/cli/subcommands/_purge.py:1041-1064 (_filename_contains_run_id)
• forgelm/compliance.py:2173-2236 (confirmed)
• tests/test_gdpr_erasure.py:231-249 (test-gap: fabricated run-id-named fixtures)
• docs/guides/gdpr_erasure.md:56-60 and docs/reference/purge_subcommand.md:6
• forgelm/compliance.py:2173-2205
• tests/test_gdpr_erasure.py:231-254

Evidence.

• logs/A1_artefacts_realrun.out; ls runA/out/compliance shows no run-id names. A3 deleted notes_.txt too.
• Repro r5 uses the real export_compliance_artifacts. fpy -m forgelm purge --run-id fg-0123456789ab --kind artefacts --output-dir r5/out gives {success:false, 'No artefacts artefacts found'}, exit 1, and the audit log gains data.erasure_failed NoMatchingArtefacts. With --dry-run, --run-id report would delete [compliance_report.json, data_governance_report.json] and --run-id json would delete four JSON files.
• Repro: generate_training_manifest + export_compliance_artifacts into out/compliance with run_id fg-abcdef123456 yields ['compliance_report.json','data_provenance.json','training_manifest.yaml']; purge --run-id fg-abcdef123456 --kind artefacts -> exit 1, "No 'artefacts' artefacts found", nothing deleted.

Recommended fix. Bind the bundle to the run through content rather than file names: for example, delete compliance/ when compliance_report.json or the manifest records this run_id, or have the exporter write run-scoped names. Restrict deletion to the documented file kinds and update the docs.

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

• DOC-D2B02-005 (docs, High) — docs/design/gdpr_erasure.md: false claim — kind=artefacts deletes a fixed allow-list (compliance/, data_audit_report.json, data_governance_report.json), never a glob, kept in forgelm.compliance. Also…
  severity High · already tracked: KNOWN-OPEN TRUST-02 (inferred) — docs/roadmap/phase-16-trust-surface-hardening.md:564-566
  Where. docs/design/gdpr_erasure.md:233 · type: false claim
  The document says. kind=artefacts deletes a fixed allow-list (compliance/, data_audit_report.json, data_governance_report.json), never a glob, kept in forgelm.compliance. Also L193 (adds 'derived JSONL snapshots'), L169 ('removes compliance/*.json for the named run') and L62 ('purge --kind artefacts rewrites them').
  What the code does. There is no allow-list in forgelm.compliance. The resolver token-matches the run_id inside filenames under compliance/ only. It never touches data_audit_report.json or JSONL snapshots, never rewrites anything, and matches none of the exporter's fixed filenames (reproduced: exit 1 NoMatchingArtefacts).
  Code references. forgelm/cli/subcommands/_purge.py:998-1038, forgelm/compliance.py:2173-2236
  Proposed correction. Fix the code to delete the run's real artefact set via an allow-list, or document the actual behaviour: 'files under compliance/ whose name contains the run id as a token; none of ForgeLM's own artefacts currently do'.

• DOC-D2B02-007 (docs, High) — docs/guides/gdpr_erasure.md: misleading — forgelm purge --run-id <id> --kind artefacts deletes a run's compliance bundle. L60 says it removes <output_dir>/compliance/*.json files whose name embed…
  severity High · already tracked: KNOWN-OPEN TRUST-02 (inferred) — docs/roadmap/phase-16-trust-surface-hardening.md:554-566
  Where. docs/guides/gdpr_erasure.md:15 (Turkish mirror: docs/guides/gdpr_erasure-tr.md:60) · type: misleading
  The document says. forgelm purge --run-id <id> --kind artefacts deletes a run's compliance bundle. L60 says it removes <output_dir>/compliance/*.json files whose name embeds the run_id.
  What the code does. The resolver deletes only files in compliance/ whose filename contains the run_id as a delimited token. ForgeLM's exporter writes fixed names with no run_id: compliance_report.json, training_manifest.yaml, data_provenance.json, risk_assessment.json, annex_iv_metadata.json, data_governance_report.json. Against a real bundle the command finds nothing, emits data.erasure_failed (NoMatchingArtefacts)…
  Code references. forgelm/cli/subcommands/_purge.py:998-1014, forgelm/cli/subcommands/_purge.py:1044-1064, forgelm/compliance.py:2173-2236, forgelm/trainer.py:2087-2096
  Proposed correction. Fix the code: resolve the run's bundle by its real artefact names, scoped to the run (for example via run_id inside compliance_report.json), or delete an explicit allow-list. Until then, add: "Known limitation: ForgeLM's compliance artefacts do not embed the run id in their filenames, so --kind artefacts currently matches nothing for bundles pro…
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D3B02-004 (docs, High) — docs/usermanuals/en/compliance/gdpr-erasure.md: false claim — forgelm purge --run-id <id> --kind artefacts "atomically removes ... a compliance bundle".
  severity High · already tracked: KNOWN-OPEN TRUST-02 docs/roadmap/phase-16-trust-surface-hardening.md:553-562
  Where. docs/usermanuals/en/compliance/gdpr-erasure.md:17,32 (Turkish mirror: docs/usermanuals/tr/compliance/gdpr-erasure.md:17,32) · type: false claim
  The document says. forgelm purge --run-id <id> --kind artefacts "atomically removes ... a compliance bundle".
  What the code does. _artefact_targets_for_run only selects compliance/ files whose name contains the run_id as a token (e.g. compliance_<run_id>.json, annex_iv_<run_id>.json). The trainer writes fixed names: compliance_report.json, training_manifest.yaml, data_provenance.json, risk_assessment.json, annex_iv_metadata.json and data_governance_report.json. So the mode never finds a real bundle. Reproduced: with…
  Code references. forgelm/cli/subcommands/_purge.py:998-1016, forgelm/compliance.py:2173-2236, forgelm/trainer.py:2087-2096
  Proposed correction. Code fix (see rationale). Until then add a warning: "--kind artefacts currently matches only files whose names embed the run id; the standard compliance bundle (compliance/compliance_report.json etc.) is not matched and must be removed manually."
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D2B02-023 (docs, Medium) — docs/guides/gdpr_erasure.md: misleading — --kind artefacts deletes a run's compliance bundle (compliance/.json embedding run_id)
  severity Medium · already tracked: KNOWN-OPEN TRUST-02 (inferred) — docs/roadmap/phase-16-trust-surface-hardening.md:554-566
  Where. docs/guides/gdpr_erasure.md:15,60; docs/reference/purge_subcommand.md:51 · type: misleading
  The document says. --kind artefacts deletes a run's compliance bundle (compliance/.json embedding run_id)
  What the code does. No ForgeLM writer embeds the run_id in compliance file names, so the mode never matches a real run (A1). On matches it deletes any file type (A3). See F3.
  Code references. forgelm/cli/subcommands/_purge.py:998-1014, forgelm/compliance.py:2173,2228
  Proposed correction. See F3.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}