Repository navigation
purge memorisation check cannot see exported GGUF files: forgelm export records only a basename and no audit event #1071
Description
Summary
The memorisation warning of forgelm purge --row-id lists the model, staging and checkpoint-* directories under --output-dir and under the training layouts a loaded --config names, plus merge.output_dir. A model exported with forgelm export MODEL_PATH --output PATH lives wherever --output pointed, and ForgeLM keeps no usable record of where, so an exported GGUF is invisible to the check. CR-A10-042 (#102) named "exported models"; #1068 documents the limit instead of closing it.
Evidence
forgelm export(forgelm/cli/subcommands/_export.py,forgelm/export.py::export_model) writes no audit event. The destination reaches the operator only through the command's own output (output_pathin the JSON envelope).export.py::_update_integrity_manifestappends{"file": <basename>, "format", "quant", "sha256", "size_bytes"}toexported_artifactsin<model_dir>/model_integrity.json. It keeps the file name and drops the directory, so the manifest cannot be used to find the file. It does nothing when the manifest is absent, and the update is best effort.- Nothing else reads
exported_artifacts; the field is described only indocs/usermanuals/{en,tr}/deployment/gguf-export.md. - After purge: find the weights a run really leaves behind (fixes #102) #1068 the docs say that models exported or copied elsewhere are invisible to the check (
docs/guides/gdpr_erasure.md, the GDPR manual page, the QMS risk plan).
Why it matters
A GGUF quantised from weights trained on an erased row is the artefact most likely to leave the operator's machine, and the operator relies on data.erasure_warning_memorisation to be told which copies to chase. Today the one copy the tool created for distribution is the one it cannot name.
Acceptance criteria
- Decide where the destination is recorded and write the decision in this issue: an additive absolute
pathinexported_artifacts, a new audit event for a completed export (added todocs/reference/audit_event_catalog.mdand its TR mirror, whichtools/check_audit_event_catalog.pyenforces), or both. Say what happens when the model directory has nomodel_integrity.json. -
forgelm exportrecords the absolute destination of the file it wrote and, for a K-quant request, also the requested--output. For a K-quant,export_modelwrites<output>.f16.ggufas an intermediate (_resolve_kquant_path) andExportResult.output_pathis that file; the file the operator ships is produced later by the manualllama-quantizefollow-up at--output, which ForgeLM never sees. Manifests written before the change keep working unchanged (verify-integrityandverify-ggufincluded), and the new key is additive and ignored by their readers. -
forgelm purge --row-idadds the recorded export destinations that still exist tomemorisation_artefact_paths, reading them from the model directories it already finds. A missing file is skipped silently; an unreadable manifest is logged at WARNING and does not abort the erasure, which has already removed the row by then. - Tests cover: an export recorded then found by purge, a destination that was deleted since, a pre-change manifest, and an unreadable manifest.
- Docs in EN and TR: the audit catalog row, the purge reference, the GDPR guide and manual page (replace "models you export are invisible" with what is now covered), and
gguf-export.mdfor the recorded field. CHANGELOG entry. - State in the docs what stays invisible (a model copied by hand, or exported with
--no-integrity-updateif that path is kept), so the limit is explicit.
Context
Follow-up of #102 (CR-A10-042) and #1068. merge.output_dir is already covered by #1068 through the loaded config. Related: #1069 (retention horizon for checkpoint-*).
Sequencing: Phase 16 S5 (#891, CLI-03) changes how verify-integrity treats exported_artifacts, and S6 (#892) reworks export_model's temp and atomic-write handling. Settle the manifest schema with those before implementing. A new audit event would also serve CR-A09-019 (#652), which wants verify-gguf's sidecar anchored to a keyed export record, so decide the two together; but it needs its own answer to which audit_log.jsonl an export appends to, because forgelm export receives only a model path, not the training output directory.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity
cemililik commentedon Oct 23, 2026
Triaged on
developmente6bf28d, after #1068 merged during this triage. Each finding was first checked on 83e62f6 and on #1068's head (ab5ce5f), whose code is identical to the merge.Accepted: enhancement, severity medium, wave 3, under the W3 module-sweep epic #870.
What I checked
forgelm/export.pynorforgelm/cli/subcommands/_export.pytouches the audit log; the destination reaches the operator only through theoutput_pathof the JSON envelope._update_integrity_manifestappends{file: <basename>, format, quant, sha256, size_bytes}toexported_artifacts, only whenmodel_integrity.jsonalready exists, warns and carries on if it fails, and is skipped by--no-integrity-update.forgelm/readsexported_artifacts; it appears only in tests andgguf-export.md. Phase 16 S5 ([Phase 16] S5 — Verification truth: three states, not a success bit #891,CLI-03) is about its missing reader.Two findings that changed the acceptance criteria (body updated)
export_modelwrites<output>.f16.ggufas an intermediate andExportResult.output_pathis that file. The file the operator ships is produced later by the manualllama-quantizefollow-up at--output, which ForgeLM never sees. Record the written file and the requested--output; "destinations that still exist" then handles the not-yet-quantised case.verify-integrityreadsexported_artifacts, and S6 ([Phase 16] S6 — Invocation-owned resources: temp state, atomic writes, worker lifetime #892) reworksexport_model's temp and atomic-write handling. The new key must be additive and ignored by that reader; settle the schema with them before implementing.Maintainer decision still open (criterion 1). My recommendation is an additive absolute
pathinexported_artifacts: purge already finds the model directories and can read their manifests, and no new audit-log question arises. A new audit event is a larger step, becauseforgelm exportreceives only a model path, so whichaudit_log.jsonlit appends to needs its own design. It would also serve CR-A09-019 (#652: anchorverify-gguf's sidecar to a keyed export record), so it is worth deciding together, but it is not a prerequisite. Without amodel_integrity.jsonthe export stays invisible, and the docs should say so (last criterion).Size budget: as in #1069,
_purge.pyhas 0 LOC of headroom (decision C-18: record abudget_historyentry, or use a new module).Nothing blocks the purge side: it extends the
memorisation_artefact_pathspayload that #1068 added. The export side waits on the maintainer decision above.Not open for contribution yet: this needs a maintainer decision first. If you would like to work on it, please comment with your proposed approach before opening a pull request.