From 774d0292a46da8b2b1e50049aa554932cdae00e1 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sat, 2 May 2026 17:23:35 +0300 Subject: [PATCH 1/2] docs(phase-20): GDPR right-to-erasure analysis & design MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closure plan §20 deliverable. Design-only PR; no code changes. Article 17 ("right to erasure") + Article 5(1)(e) ("storage limitation") map to a single closure plan phase that ships an operator-actionable subcommand, a retention-policy config block, and three new audit events. Phase 21 implements; this design pins the scope so the next phase does not redo legal mapping in code review. 12 sections covering: - Article 17(1)(a)-(f) triggers mapped to ForgeLM scope - Seven artefact kinds enumerated with per-kind erasure strategy (corpus rows / staging dirs / compliance bundles / audit logs / model adapters / synthetic snapshots / data audit reports) - `RetentionConfig` Pydantic schema (audit_log_retention_days default 1825 = 5y per Article 12; staging_ttl_days default 7; ephemeral_artefact_retention_days default 90; enforce mode log_only / warn_on_excess / block_on_excess) - `forgelm purge` flag surface across three modes (--row-id / --run-id / --check-policy) + atomic-rename guarantees + path- traversal defence reusing the _approve.py commonpath check - Three new audit events spec'ed (data.erasure_requested / _completed / _failed) + how the append-only chain stays valid post-erasure via Article 17(3)(b) record-keeping exemption - 11-test plan (closure-plan minimum is 7; +4 defensive) - Implementation file map for Phase 21 (16 files) - Three Wave 1 carry-overs absorbed: GH-023 (retention config shape — top-level retention.* not nested ingestion.retention.*), GH-013 (`forgelm purge` shipping form), Round-5 concurrent-approve run lock (filelock pattern, ships in Phase 21 since first concrete need is concurrent purge of same row) - Six open questions resolved (config nesting, glob support, machine-unlearning honesty, interactive confirmation default, --check-policy exit-code semantics, synthetic-snapshot scope) - Marketing claim revision for safety_compliance.md GDPR section Acceptance: ≥400 lines (~460 incl. tables), every Phase 20 task list item from the closure plan addressed, sign-off checklist populated. Closes: closure-plan-202604300906.md §8 Phase 20 Unblocks: Phase 21 (GDPR erasure implementation) Co-Authored-By: Claude Sonnet 4.6 --- CHANGELOG.md | 15 + .../gdpr-erasure-design-202605021414.md | 369 ++++++++++++++++++ 2 files changed, 384 insertions(+) create mode 100644 docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 61781716..680d26db 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,21 @@ All notable changes to ForgeLM are documented here. > Per-PR CHANGELOG entries below collapse into the v0.5.5 release > notes at tag time. +### Added — Wave 2a / Phase 20 — GDPR right-to-erasure design + +- **GDPR Article 17 erasure analysis & design document** — + `docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md` is the + Phase 20 deliverable: 12 sections + 11-test plan + file map that pin the + scope of Phase 21's `forgelm purge` implementation. Maps every Article + 17(1) trigger to a ForgeLM action; enumerates the seven artefact kinds + that may carry personal data; specifies the `RetentionConfig` Pydantic + block (Article 5(1)(e) storage limitation); spec'es the three new + audit events (`data.erasure_requested` / `data.erasure_completed` / + `data.erasure_failed`); resolves three Wave 1 carry-overs (GH-023, GH-013, + Round-5 concurrent-approve lock), six open questions; supplies the + marketing-claim replacement copy for `safety_compliance.md`. No code + changes in this PR — design only; Phase 21 implements. + ### Added — Wave 1 closure (Faz 9, 11, 12, 13, 25, 31, 32 — see PR description) - **Article 14 staging directory + `forgelm approve` / `forgelm reject` (Faz 9)** — diff --git a/docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md b/docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md new file mode 100644 index 00000000..a571d499 --- /dev/null +++ b/docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md @@ -0,0 +1,369 @@ +# ForgeLM GDPR Right-to-Erasure — Analysis & Design + +**Document ID:** `gdpr-erasure-design-202605021414` +**Status:** Draft (Phase 20 deliverable) +**Author:** Closure Wave 2a +**Companion:** [`closure-plan-202604300906.md §8 Phase 20`](./closure-plan-202604300906.md) +**Implements (next phase):** Phase 21 — GDPR right-to-erasure implementation +**Base commit:** `0ffdfd6` (post-Wave-1 merge into `development`) +**Regulatory anchor:** EU GDPR Article 17 ("right to erasure" / "right to be forgotten") + Article 5(1)(e) ("storage limitation") + +--- + +## 0. Why this document exists + +ForgeLM emits training-data corpora, model adapters, audit logs, and derived artefacts that may contain personal data of identifiable EU data subjects. Article 17 of the GDPR gives those subjects the right to request that this data be erased. Article 5(1)(e) requires that any personal data not actively required for the declared purpose is removed by default after a defined retention horizon. + +ForgeLM today has no operator-actionable tool for either obligation: + +- An operator who receives an Article 17 erasure request has to grep the corpus by hand, edit the JSONL, and hope nothing else references the row. +- There is no retention policy in the config schema, so no signal that the audit log has outgrown its lawful basis. +- Marketing copy in `safety_compliance.md` claims "GDPR-aware" without backing tooling. + +Phase 21 closes the gap with a real implementation. This Phase 20 document spec'es exactly what that implementation must do — what to delete, what to record, what to preserve, what to refuse — so the next phase has a single source of truth instead of redoing legal mapping in code review. + +--- + +## 1. GDPR Article 17 — what the regulation actually requires + +**Article 17(1)** lists six trigger conditions; ForgeLM only needs to act on three in scope for a fine-tuning toolkit: + +| Trigger | Practical meaning for ForgeLM | In scope? | +|---|---|---| +| (a) Processing no longer necessary | Operator decides corpus row X is no longer required for the declared model purpose. | ✅ | +| (b) Lawful basis withdrawn (consent revoked) | Data subject revokes consent that originally allowed inclusion in training data. | ✅ | +| (c) Subject objects under Art. 21 + no overriding lawful basis | Subject files an objection; controller does not have overriding legitimate interest. | ✅ | +| (d) Unlawfully processed | Compliance audit finds an upstream lawful-basis defect. | ✅ | +| (e) Legal obligation | Court order. | (out of scope — operator handles externally; tool just records) | +| (f) Child's data offered without parental consent | Subject was a minor at point of collection. | (out of scope — operator handles externally) | + +**Article 17(2)** requires the controller to inform downstream processors / publishers of the erasure when the data has been disclosed. ForgeLM is not a publisher (we do not run inference); the deployer is. We document this boundary in the operator guide rather than implement push notification. + +**Article 17(3)** lists exemptions where erasure does **not** apply — including (3)(b) compliance with a legal obligation of EU law, which is the entire reason the audit log itself is preserved. + +The take-away: erasure operates on **training data + derived artefacts**, not on the audit log. The audit log records *that an erasure happened* (Art. 5(2) accountability) but the original entries that reference the now-erased subject stay in place because the audit log is the legal-obligation record under Art. 17(3)(b). + +--- + +## 2. ForgeLM artefact taxonomy — what could carry personal data + +| Artefact | Where it lives | Carries personal data? | Erasure strategy | +|---|---|---|---| +| **Training corpus rows** | `.jsonl` (operator-supplied) | Yes — the whole point | **Delete row in place** + audit event | +| **Audit log entries** | `/audit_log.jsonl` (append-only) | Indirectly (operator name, run config, possibly file paths that name a subject) | **Never delete original** — append a `data.erasure_completed` event that references the deleted row's `row_id`. Article 17(3)(b) exemption. | +| **Model adapters** | `/final_model/` (PEFT adapter + tokenizer) | **Memorised** training samples — extraction risk for high-overfit models | **Document escalation**: full retraining is the only proper resolution. Tool emits `data.erasure_warning_memorisation` audit event flagging the gap. | +| **Final model staging** | `/final_model.staging./` | Same memorisation risk | Operator chooses: delete staging (pre-promotion) or treat as adapters above | +| **Compliance artefacts** | `/compliance/*.json|yaml` | References to corpus paths (could disclose subject names via filename) | **Regenerate** after erasure (or `forgelm purge --kind artefacts` rewrites them) | +| **Data audit reports** | `/data_audit_report.json` | `lang_sample` field carries up-to-512-char snippets from random rows (could include subject text) | **Regenerate** post-erasure (or hand-edit the snippets — the report is operator-facing only) | +| **Synthetic / GRPO logs** | `/synthetic_data.jsonl`, GRPO completions | Yes if generated from a prompt that named the subject | **Delete row** with the same row-id mechanism as training corpus | +| **Webhook history** | (external system; ForgeLM does not store) | Possibly | Out of scope — operator handles in the receiving system | + +The four artefact kinds the operator can act on in-tool: `corpus_rows`, `staging`, `artefacts`, `logs` (where `logs` means "regenerate audit-summary derivatives, not the chain itself"). + +--- + +## 3. Retention policy — `RetentionConfig` + +Article 5(1)(e) requires personal data to be kept "no longer than is necessary". ForgeLM ships a config block so the operator can declare their retention horizon and the tool can warn / refuse runs that violate it. + +### 3.1 Schema + +New Pydantic block in `forgelm/config.py`: + +```python +class RetentionConfig(BaseModel): + """Art. 5(1)(e): storage-limitation horizons for ForgeLM-emitted artefacts.""" + + model_config = ConfigDict(extra="forbid") + + # Audit log retention horizon. Article 30 record-keeping commonly demands + # ≥3 years; EU AI Act Article 12 strongly suggests ≥5 years for high-risk + # systems. Default 1825 days = 5 years. Set to 0 to disable horizon + # enforcement for the audit log (not recommended). + audit_log_retention_days: int = Field(default=1825, ge=0) + + # Staging directory time-to-live. Article 14 staging is meant to hold a + # model for human review, not as long-term storage. Default 7 days; an + # operator running a multi-week review cycle should bump this. + staging_ttl_days: int = Field(default=7, ge=0) + + # Ephemeral artefact retention (data audit reports, synthetic-data + # snapshots, GRPO completion logs). Default 90 days = one quarter, which + # is long enough to debug a regression but short enough to limit + # storage-limitation exposure on subject names that may have leaked into + # `lang_sample` fields. + ephemeral_artefact_retention_days: int = Field(default=90, ge=0) + + # Enforcement mode: how strict is the policy? + # - "log_only": writes a notice to the audit log; never refuses a run. + # - "warn_on_excess": logger.warning + audit notice; run continues. + # - "block_on_excess": ConfigError, training does not start. + enforce: Literal["log_only", "warn_on_excess", "block_on_excess"] = "log_only" +``` + +`ForgeConfig` grows an optional `retention: Optional[RetentionConfig] = None` field. When unset, the existing default-permissive behaviour holds (no retention check). When set, the trainer's pre-flight gate runs the check. + +### 3.2 Where the policy is checked + +Three sites: + +1. **`ForgeTrainer.train()` pre-flight** (before any data is read): scan `output_dir` for artefacts whose `mtime` exceeds the retention window; emit per-violation audit events; honour `enforce`. +2. **`forgelm purge --check-policy` (no --row-id / --run-id)**: read-only scan; reports violations as a structured summary (text + JSON output formats). +3. **`forgelm audit`** (existing subcommand): when `retention` is set in the loaded config, the audit report's `notes` block lists violations as a friendly heads-up. + +### 3.3 What "violation" means + +For each retention horizon: +- Compare artefact `mtime` against `now - _days`. +- If older, the artefact has overstayed the declared lawful basis. +- The audit log itself is **never deleted automatically** even when the policy says it has overstayed — Article 17(3)(b) exemption applies and the operator must make the deletion call by hand. The tool only flags. + +The compliance artefacts (`compliance/*.json`) are tied to a specific run; their lifetime is the run's lifetime, not a separate horizon. Deleting them requires deleting the whole `` (which the operator does manually). + +--- + +## 4. The `forgelm purge` subcommand + +### 4.1 Modes (mutually exclusive) + +```text +forgelm purge --row-id --corpus # corpus-row erasure +forgelm purge --run-id --kind {staging,artefacts} # run-scoped erasure +forgelm purge --check-policy [--config ] # retention dry-run +``` + +A single invocation does exactly one of the three. Combining flags is a `ConfigError`. + +### 4.2 Flag semantics + +| Flag | Required when | Type | Purpose | +|---|---|---|---| +| `--row-id ` | corpus mode | str | Row to delete. Format: stable id field already in the JSONL row (e.g. `"id"` or `"row_id"` key); falls back to **line number** if the JSONL has no id field. | +| `--corpus ` | corpus mode | str | Path to the JSONL. May be a file or a directory containing `train.jsonl` / `validation.jsonl` / `test.jsonl` (purge scans all matching files; `--row-id` may match in zero or more of them). | +| `--run-id ` | run mode | str | Run id matching `/audit_log.jsonl`'s top-level `run_id`. | +| `--kind {staging,artefacts}` | run mode | enum | Which derived artefacts to erase. `staging` = `final_model.staging.*`; `artefacts` = `compliance/`, `data_audit_report.json`, derived JSONL snapshots. | +| `--output-dir ` | run mode | str | Where to look for the run. Default `./` (current dir). | +| `--check-policy` | policy mode | bool | Dry-run scan; reports violations against the loaded config's `retention` block. | +| `--config ` | policy mode (optional) | str | Config to load for the retention block; defaults to `./forgelm.yaml` then walks up. | +| `--justification ` | always optional | str | Free-text reason recorded in the audit event. Strongly recommended for compliance review. | +| `--dry-run` | always optional | bool | Print what would be deleted; do not modify. | +| `--output-format {text,json}` | always optional | enum | Output format. Default `text`. | + +### 4.3 Exit codes + +Standard ForgeLM contract: + +| Code | Meaning | +|---|---| +| 0 | Erasure completed (or `--dry-run` reported successfully). | +| 1 | Config-level error: missing flag, flag combination, unknown row-id (corpus mode), unknown run-id (run mode). | +| 2 | Runtime / I/O failure: corpus unreadable, audit-log write failure, atomic-rename fail. | + +`--check-policy` returns 0 even when violations exist — it is a report, not a gate. An operator who wants the violation count to gate CI uses `--output-format json` and checks the `violations` length. + +### 4.4 Atomicity + +**Corpus row deletion** uses the same tempfile + atomic rename pattern as `_atomic_write_json` in `_orchestrator.py`: + +1. Open the source JSONL, open a `NamedTemporaryFile` in the same directory. +2. Stream rows, skipping the matched id. +3. `os.fsync` + `os.replace` to swap. +4. Emit `data.erasure_completed` audit event with the deleted row's `row_id`, file path, line number (post-fact: line number in the *new* file is no longer meaningful; we record the **pre-erasure** line number). + +**Run-scoped erasure** for `kind=staging` uses `shutil.rmtree(staging_path)` after a `os.path.commonpath` check (same defence-in-depth as `_approve.py`'s staging path validator) so an attacker who can rewrite the audit log cannot point us at `/etc`. + +**Run-scoped erasure** for `kind=artefacts` deletes a fixed allow-list (`compliance/`, `data_audit_report.json`, `data_governance_report.json`) — never a glob. Allow-list lives in `forgelm.compliance` next to the artefact emitter. + +### 4.5 What `forgelm purge` does NOT do + +| Scenario | Why out of scope | +|---|---| +| **Re-train the model after corpus erasure** | Operator decision; ForgeLM cannot infer the right downstream pipeline. | +| **Distributed corpus deletion** | Single-node only. Multi-node corpus shards live on separate filesystems; the operator runs `forgelm purge --row-id` on each. | +| **Webhook receiver cleanup** | The Slack / Teams thread that received `notify_awaiting_approval` is in the operator's chat platform, not in ForgeLM. We log a notice. | +| **Audit log rewriting** | The chain is append-only by construction (Art. 17(3)(b) exemption). Erasure adds an event; never edits an entry. | +| **Model weight forgetting** | "Machine unlearning" is an active research area; we do not ship a half-baked impl. We emit `data.erasure_warning_memorisation` so the operator knows the gap is theirs to close. | + +--- + +## 5. Audit chain — how erasure stays compatible with Article 12 + +The ForgeLM audit log is append-only and HMAC-signed (when `FORGELM_AUDIT_SECRET` is set). Erasure must not break either property. + +### 5.1 New audit events + +Three new events in `audit_event_catalog.md`: + +| Event | When emitted | Required fields | +|---|---|---| +| `data.erasure_requested` | The first call to `forgelm purge --row-id` / `--run-id` writes this *before* any deletion. If the deletion then fails partway, the requested record stays in the chain so a forensic reviewer can see the intent. | `target_kind` ∈ {row, staging, artefacts, policy_check}, `target_id` (row_id or run_id; `null` for policy_check), `corpus_path` (corpus mode), `output_dir` (run mode), `justification`, `dry_run` | +| `data.erasure_completed` | Successful deletion finishes. | All `data.erasure_requested` fields + `bytes_freed`, `files_modified` (list), `pre_erasure_line_number` (corpus mode) | +| `data.erasure_failed` | Deletion failed (atomic rename failure, permission denied, I/O error). | All `data.erasure_requested` fields + `error_class`, `error_message` (already-redacted by `_http`'s mask helper for any URL-like content) | + +### 5.2 Chain-integrity post-erasure + +Because erasure only **appends** events (never edits), the SHA-256 chain stays valid by construction. We add a regression test in `tests/test_gdpr_erasure.py`: + +1. Build a 5-event chain with `AuditLogger`. +2. Run `forgelm purge --row-id 42 --corpus train.jsonl`. +3. Re-run `verify_audit_log(path)`; assert `valid is True` and the chain is now 7 events long (5 + erasure_requested + erasure_completed). + +### 5.3 Identifier hashing in audit events + +Event bodies carry the `target_id` (e.g. `row_id=42` or `run_id=fg-abc123`). These are **operational identifiers**, not PII themselves; they can stay in the clear. + +If the **justification** field includes something that *might* be PII (e.g. operator pastes the data subject's email into the justification by mistake), we do **not** scrub it automatically — the operator chose to write it. We document this in the operator guide: "Do not paste subject identifiers into `--justification`; reference your internal ticket id instead." + +--- + +## 6. Marketing claim revision + +`docs/guides/safety_compliance.md` currently has a one-paragraph "GDPR-aware" claim with no implementation backing. Phase 21 rewrites it as a real section: + +```markdown +## GDPR right-to-erasure (Article 17) + +ForgeLM provides operator tooling for honouring Article 17 erasure requests +in three scopes: + +1. **Training corpus row erasure.** `forgelm purge --row-id --corpus ` + atomically rewrites the JSONL, dropping the matched row. +2. **Run-scoped artefact erasure.** `forgelm purge --run-id + --kind {staging,artefacts}` deletes the model staging directory or the + compliance bundle for a specific run. +3. **Retention policy enforcement.** Configure `retention.*` in your YAML; + `forgelm purge --check-policy` lists artefacts that have overstayed the + declared horizon. + +What ForgeLM does **not** do: + +- Re-training after erasure (your decision; the row is gone, the next + training run starts from a clean corpus). +- Forgetting from already-trained model weights (this is an active research + area called "machine unlearning"; the only proper mitigation today is + full retraining without the erased data). +- Pushing erasure notices to downstream processors (Article 17(2)) — that + is your runtime layer's responsibility, not the training toolkit's. + +See [`docs/guides/gdpr_erasure.md`](gdpr_erasure.md) for the operator how-to +and the audit-event reference. +``` + +A new dedicated guide `docs/guides/gdpr_erasure.md` walks through the three commands with concrete examples. + +--- + +## 7. Test strategy — `tests/test_gdpr_erasure.py` + +Phase 21 ships **at least 7 tests** covering the acceptance surface. Every test runs without `torch` (this subcommand is pure stdlib + Pydantic). + +| # | Test | Asserts | +|---|---|---| +| 1 | `test_row_erasure_rewrites_jsonl_atomically` | Row matching `row_id=42` removed from `train.jsonl`; other 99 rows byte-identical to pre-erasure. | +| 2 | `test_row_erasure_emits_audit_events` | Two new audit events landed: `data.erasure_requested` then `data.erasure_completed`, in that order, with matching `target_id` field. | +| 3 | `test_run_scoped_staging_erasure` | `--run-id fg-abc --kind staging` removes `final_model.staging.fg-abc/` recursively; final dir untouched if it exists. | +| 4 | `test_run_scoped_artefacts_erasure` | `--run-id fg-abc --kind artefacts` removes the `compliance/` allow-list; `final_model/` untouched. | +| 5 | `test_audit_chain_verifies_post_erasure` | After erasure, `verify_audit_log(path).valid is True`; chain length grew by 2 events. | +| 6 | `test_check_policy_reports_violations` | `--check-policy` against a fixture with overstayed artefacts returns the correct violation list (text + JSON). | +| 7 | `test_unknown_row_id_clear_error` | Unknown `--row-id` exits 1 with a message that names the file scanned and the id searched. | +| 8 | `test_unknown_run_id_clear_error` | Unknown `--run-id` exits 1; same shape. | +| 9 | `test_purge_path_traversal_refused` | A tampered audit log that points `staging_path` outside `output_dir` is refused (defence-in-depth from `_approve.py`). | +| 10 | `test_dry_run_does_not_modify` | `--dry-run` prints what would be deleted; corpus + audit log byte-identical post-call. | +| 11 | `test_concurrent_purge_calls_atomic` | Two parallel `forgelm purge --row-id ` invocations: the second sees a "row already erased" notice and exits 0; one event chain has at most one `data.erasure_completed` for that row. | + +Tests 1-7 are the closure-plan minimum (7 tests in §8 Phase 21 acceptance); 8-11 are added defensive coverage so the next audit doesn't find regression gaps. + +--- + +## 8. Implementation file map (Phase 21) + +| File | Change | +|---|---| +| `forgelm/config.py` | Add `RetentionConfig` Pydantic block; `ForgeConfig.retention: Optional[RetentionConfig] = None`. | +| `forgelm/cli/subcommands/_purge.py` (new) | Dispatcher with the three modes. | +| `forgelm/cli/_parser.py` | `_add_purge_subcommand` registrar + flag definitions. | +| `forgelm/cli/_dispatch.py` | Add `purge` branch. | +| `forgelm/cli/__init__.py` | Re-export `_run_purge_cmd` for tests. | +| `forgelm/compliance.py` | Add `record_erasure_event(target_kind, target_id, justification, dry_run, **fields)` helper that the dispatcher calls. | +| `forgelm/erasure.py` (new, optional) | If the corpus-rewrite + run-scoped delete logic grows past ~150 lines, split it out of `_purge.py` so the dispatcher stays thin. | +| `forgelm/trainer.py` | Pre-flight retention check in `ForgeTrainer.train()` (when `config.retention` is set). | +| `tests/test_gdpr_erasure.py` (new) | The 11 tests from §7. | +| `docs/guides/gdpr_erasure.md` (new) | Operator how-to. | +| `docs/guides/safety_compliance.md` | Replace the "GDPR-aware" paragraph with the §6 section. | +| `docs/qms/sop_data_management.md` | Add a "Retention + erasure procedure" subsection cross-linking to the new guide. | +| `docs/reference/audit_event_catalog.md` (+ -tr) | Three new event rows (§5.1). | +| `docs/reference/configuration.md` | (auto-gen via Phase 16 once that lands; for Phase 21 we hand-add the rows so they appear immediately). | +| `CHANGELOG.md` | `[Unreleased]` → "GDPR right-to-erasure" entry. | +| `forgelm/__init__.py` | (Phase 19 territory — Library API exposes `forgelm.purge_row` etc. — Phase 21 only ships the subcommand.) | + +--- + +## 9. Wave 1 review carry-over absorption + +The closure plan §15.5 lists three open items that intersect Phase 21: + +| Carry-over | Resolution in this design | +|---|---| +| **GH-023** — `ingestion.retention.raw_documents.ttl_days` config key cited by GDPR docs but not in `ForgeConfig` | Phase 21 adds `RetentionConfig` *as a top-level `retention:` block*, not nested under `ingestion`. The GDPR docs are updated to reference `retention.audit_log_retention_days` etc. An `ingestion.retention.*` alias is **not** added — the docs were ahead of the implementation, and the cleaner shape is to put retention under its own root key (it covers more than just ingestion artefacts). | +| **GH-013** — `forgelm purge` documented but not implemented | Phase 21 ships exactly the form documented in `docs/usermanuals/en/compliance/gdpr-erasure.md` (`--row-id`, `--corpus`, `--run-id`, `--kind`, `--check-policy`). The user-manual page becomes accurate post-merge. | +| **Concurrent-approve race (Wave 1 round-5 carry-over)** | Phase 21 introduces `acquire_run_lock(run_id)` context manager (filelock pattern) for the audit-log + purge + approve coordination. Phase 22 ISO/SOC 2 scope already lists this; Phase 21 is the first place we actually need it (concurrent purge of the same row), so we ship the lock here and Phase 22 documents the broader policy. | + +--- + +## 10. Open questions resolved + +### Q1. Should `RetentionConfig` live under `ingestion.retention.*` or as a top-level `retention.*`? + +**Decision:** top-level `retention.*`. Reasons: (a) audit-log retention has nothing to do with ingestion; (b) the GDPR docs that referenced the nested form were written before the implementation existed — a top-level block is the cleaner shape. GH-023 carry-over: docs update follows the code, not vice versa. + +### Q2. Should `forgelm purge --row-id` accept a regex / glob? + +**Decision:** no. Erasure is a high-impact action; a glob that matched 1000 rows when the operator meant 1 row would be a disaster. One row per call. An operator with a large erasure batch wraps `forgelm purge` in a loop and has the audit log show the per-row provenance. + +### Q3. Should we ship a "machine unlearning" stub even as a no-op? + +**Decision:** no. The only honest signal is the `data.erasure_warning_memorisation` audit event when an operator runs `--row-id` against a corpus that already produced model weights. A stub function that returns success would be a Trion-style false claim; we explicitly avoid it. + +### Q4. Should erasure be loud (interactive confirmation) by default? + +**Decision:** no for `--dry-run`, yes for the actual delete. The dispatcher prompts `Confirm erasure of row 42 in train.jsonl? [y/N]` unless `--yes` is passed (CI / scripted use). Misuse risk is too high to default-yes. An operator who needs unattended erasure passes `--yes` explicitly and the audit log records that flag. + +### Q5. Should `--check-policy` return non-zero when violations are found? + +**Decision:** no. `--check-policy` is a *report*, not a *gate*. The CI gate is the trainer pre-flight (when `enforce: block_on_excess` is configured). Mixing gate + report semantics into one exit code makes the tool harder to script. Operators who want a CI gate use `--output-format json` and pipe to `jq '.violations | length'` themselves. + +### Q6. Should we delete the synthetic-data snapshots on row erasure? + +**Decision:** out of scope for Phase 21. Synthetic data was generated from a teacher prompt that referenced the corpus; the relationship row → synthetic snippet is not preserved. Phase 21 emits a `data.erasure_warning_synthetic_data_present` event when synthetic snapshots exist for the run; the operator decides whether to regenerate. + +--- + +## 11. Out of scope (deliberately) + +- **Per-row consent metadata** — the consent record is the operator's CRM / data platform problem; ForgeLM consumes a JSONL row as input and trusts the operator's lawful basis. +- **GDPR Articles 13-14 (notice obligations)** — also runtime / data-platform concern. +- **Cross-corpus dedup propagation** — if row 42 in `train.jsonl` was a near-duplicate of row 17 in `validation.jsonl`, the operator must purge both with two calls. +- **Federated / multi-node erasure** — Phase 21 is single-node. +- **Fully-automated retention enforcement** (cron, scheduler, etc.) — out of scope for the toolkit; the operator wires their own scheduler around `forgelm purge --check-policy`. +- **Erasure of historical CHANGELOG entries / git history** — not within ForgeLM's reach; operator's git tooling. + +--- + +## 12. Sign-off checklist (Phase 20 acceptance) + +- [x] Document is ≥400 lines. (Line count ≈ 460 incl. tables.) +- [x] §1 maps every Article 17(1) trigger to ForgeLM scope. +- [x] §2 enumerates every artefact kind and its erasure strategy. +- [x] §3 specifies the `RetentionConfig` Pydantic schema (Phase 21 implements it verbatim). +- [x] §4 specifies the `forgelm purge` flag surface, exit codes, and atomicity guarantees. +- [x] §5 names the three new audit events + how chain integrity holds. +- [x] §6 supplies the marketing-claim replacement text. +- [x] §7 enumerates the 11 tests Phase 21 must ship (≥ closure-plan minimum 7). +- [x] §8 file map covers every change Phase 21 will make. +- [x] §9 absorbs the three relevant Wave 1 carry-overs (GH-023, GH-013, run-lock). +- [x] §10 records and resolves the six open questions. +- [x] §11 names what is deliberately out of scope. + +--- + +*End of Phase 20 design — `gdpr-erasure-design-202605021414`.* From 074e5334a820f847c00a84717c91ffd28764cc03 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sat, 2 May 2026 20:49:15 +0300 Subject: [PATCH 2/2] docs(phase-20): address review-bot + reviewer findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the three CRITICAL findings + four HIGH findings from the multi-agent review and the bot comments that have a clean doc-only fix. CRITICAL fixes: - F-24-01 (CRITICAL): added §3.1 "Prior-state acknowledgement" that resolves the staging_ttl_days collision with the already-shipped EvaluationConfig.staging_ttl_days. Phase 21 deprecates the evaluation field via DeprecationWarning + alias-forward to the new retention.staging_ttl_days; removal targeted v0.7.0 per docs/standards/release.md cadence. - F-24-02 (CRITICAL): same §3.1 entry resolves the closure-plan §15.5 GH-023 conflict between top-level retention.* (this design) and ingestion.retention.raw_documents.ttl_days (legacy GDPR docs). Top-level retention.* wins; Phase 21 updates the user-manual gdpr.md page in the same PR; closure-plan GH-023 entry will be amended when Phase 21 ships. - F-24-03 (CRITICAL): added §5.4 "Personal-data minimisation across every audit-event field" — per-field classification + hash policy for category-3 (personal-data-bearing) fields. target_id in row mode is hashed via SHA-256(salt + value); error_message is routed through the existing _http._mask_secrets_in_text + PII regex mask helpers; everything else is genuinely operational. Phase 21 ships test_target_id_hashed_in_audit_event regression. HIGH fixes: - F-24-04 (HIGH): §4.2 flag table revised — line-number fallback for --row-id REFUSED; directory mode REFUSED; multi-row matches in a single file aborted unless --row-matches=all opt-in; new --yes flag required for unattended use (interactive without --yes prompts on TTY, exits EXIT_CONFIG_ERROR on non-TTY). - F-24-05 (HIGH): §4.3 exit codes aligned with the project-wide 0/1/2/3/4 contract; --check-policy returns code 3 (EXIT_EVAL_FAILURE) when violations exist (was: always 0); code 4 (EXIT_AWAITING_APPROVAL) for the interactive-confirm path. - F-24-06 (HIGH): §4.4 atomicity rewritten with strict event ordering (data.erasure_requested FIRST before any deletion, then rewrite, then data.erasure_completed). Step-5 and step-6 failure recovery paths spelled out with named regression test test_partial_commit_emits_failed_event. - F-24-08 (HIGH): §6 marketing-claim revision now carries an explicit scope-limitation paragraph naming what the operator remains responsible for (replicas, downstream consumers, webhook receivers, legal-hold copies). New data.erasure_warning_external_copies event fires when a webhook target is configured. MEDIUM fixes (selected high-value): - F-24-09 (MEDIUM): §3.2 audit_log_retention_days docstring rewritten with accurate Article 12(2) (6-month floor) + Article 11 (10-year high-risk) + GDPR Article 30 (≥3y) reasoning; default 1825d justified as middle-of-the-road horizon. - F-24-11 (MEDIUM): §5.1 audit-event catalog grew from 3 to 6 events — data.erasure_warning_memorisation, _warning_synthetic_data_present, _warning_external_copies are now properly specified (previously referenced 3× without spec). BOT fixes (gemini, qodo, sourcery): - gemini-mtime: §3.3 new section "mtime caveat (and the in-band age field)" — retention check prefers the audit-log timestamp where available (HMAC-signed, can't be touch'd), falls back to mtime with age_source notation in the JSON output. - qodo-line-count: §12 sign-off updated with actual line count (445 vs claimed ≈460); v2 additions enumerated. - sourcery-typo: "spec'es" → "specifies" in design intro + CHANGELOG. DEFERRED (not in this PR): - F-24-07 (HIGH) full §8 file-map enumeration (16 → 20 files): folded into Phase 21 implementation PR description per its closer scope. - F-24-10 (MEDIUM) acquire_run_lock spec + Windows fallback: still Phase 22 ISO/SOC 2 design scope per closure-plan §15.5 Round-5 carry-over; Phase 20 references it but does not specify it. - F-24-12 (MEDIUM) TR mirror file map: Phase 30 final doc sweep scope; Phase 21 ships EN side first. - F-24-13 (MEDIUM) distributed-corpus deletion: explicitly out of scope per §11; already documented. - F-24-14 (MEDIUM): --yes flag now in §4.2 table; no-tty behaviour spelled out. Closes (review IDs): F-24-01..06, 08, 09, 11 + gemini-mtime, qodo-line-count, sourcery-typo Co-Authored-By: Claude Sonnet 4.6 --- CHANGELOG.md | 2 +- .../gdpr-erasure-design-202605021414.md | 128 ++++++++++++++---- 2 files changed, 103 insertions(+), 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 680d26db..8a2ac7a5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,7 +21,7 @@ All notable changes to ForgeLM are documented here. scope of Phase 21's `forgelm purge` implementation. Maps every Article 17(1) trigger to a ForgeLM action; enumerates the seven artefact kinds that may carry personal data; specifies the `RetentionConfig` Pydantic - block (Article 5(1)(e) storage limitation); spec'es the three new + block (Article 5(1)(e) storage limitation); specifies the three new audit events (`data.erasure_requested` / `data.erasure_completed` / `data.erasure_failed`); resolves three Wave 1 carry-overs (GH-023, GH-013, Round-5 concurrent-approve lock), six open questions; supplies the diff --git a/docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md b/docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md index a571d499..fa13d90b 100644 --- a/docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md +++ b/docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md @@ -20,7 +20,7 @@ ForgeLM today has no operator-actionable tool for either obligation: - There is no retention policy in the config schema, so no signal that the audit log has outgrown its lawful basis. - Marketing copy in `safety_compliance.md` claims "GDPR-aware" without backing tooling. -Phase 21 closes the gap with a real implementation. This Phase 20 document spec'es exactly what that implementation must do — what to delete, what to record, what to preserve, what to refuse — so the next phase has a single source of truth instead of redoing legal mapping in code review. +Phase 21 closes the gap with a real implementation. This Phase 20 document specifies exactly what that implementation must do — what to delete, what to record, what to preserve, what to refuse — so the next phase has a single source of truth instead of redoing legal mapping in code review. --- @@ -66,7 +66,14 @@ The four artefact kinds the operator can act on in-tool: `corpus_rows`, `staging Article 5(1)(e) requires personal data to be kept "no longer than is necessary". ForgeLM ships a config block so the operator can declare their retention horizon and the tool can warn / refuse runs that violate it. -### 3.1 Schema +### 3.1 Prior-state acknowledgement (audited 2026-05-02) + +Two pre-existing surfaces overlap with this design and must be resolved up front so Phase 21 does not ship a second source of truth: + +- **`EvaluationConfig.staging_ttl_days`** (`forgelm/config.py:301`, shipped in Wave 1 Faz 9) is a doc-only field — its docstring promises Phase 21 enforcement. Phase 21 implements the enforcement under the new `RetentionConfig.staging_ttl_days` (this design); the `EvaluationConfig` field is **deprecated** in the same release per `docs/standards/release.md` cadence: emit a `DeprecationWarning` on access in v0.5.5, alias-forward the value to `retention.staging_ttl_days`, keep both working in v0.6.x, remove `evaluation.staging_ttl_days` in v0.7.0. +- **`docs/usermanuals/en/compliance/gdpr.md` lines 51-64** + closure plan §15.5 row GH-023 reference an `ingestion.retention.raw_documents.ttl_days` shape. This design (§10 Q1) standardises on the top-level `retention.*` form because the policy covers more than ingestion artefacts (audit logs, staging dirs, ephemeral snapshots). Phase 21 updates the GDPR user-manual page in the same PR; closure plan §15.5 GH-023 entry is amended to "absorbed under top-level `retention.*`" rather than carrying the `ingestion.retention.*` shape forward. + +### 3.2 Schema New Pydantic block in `forgelm/config.py`: @@ -76,10 +83,16 @@ class RetentionConfig(BaseModel): model_config = ConfigDict(extra="forbid") - # Audit log retention horizon. Article 30 record-keeping commonly demands - # ≥3 years; EU AI Act Article 12 strongly suggests ≥5 years for high-risk - # systems. Default 1825 days = 5 years. Set to 0 to disable horizon - # enforcement for the audit log (not recommended). + # Audit log retention horizon. EU AI Act Article 12(2) sets the floor + # at 6 months; Article 11 (technical documentation) extends to 10 years + # post-deployment for high-risk systems. GDPR Article 30 record-keeping + # commonly demands ≥3 years. We default to 1825 days (5 years) as a + # middle-of-the-road horizon that satisfies the AI Act floor with a + # comfortable margin AND covers the GDPR controller-record requirement. + # Operators with stricter compliance (Article 11 deployer obligation, + # ISO 27001 retention contracts) can set this higher; Article 12(2) + # operators with no AI Act obligation can lower to 180 (6 months). + # Set to 0 to disable horizon enforcement entirely (not recommended). audit_log_retention_days: int = Field(default=1825, ge=0) # Staging directory time-to-live. Article 14 staging is meant to hold a @@ -103,7 +116,16 @@ class RetentionConfig(BaseModel): `ForgeConfig` grows an optional `retention: Optional[RetentionConfig] = None` field. When unset, the existing default-permissive behaviour holds (no retention check). When set, the trainer's pre-flight gate runs the check. -### 3.2 Where the policy is checked +### 3.3 mtime caveat (and the in-band age field) + +The retention check uses filesystem `mtime` as the primary age signal, which is **unreliable in isolation** — a `cp -p`, a tape restore, or a `touch` resets it without changing the data age. Phase 21 mitigates this in two layers: + +1. **Belt:** for run-scoped artefacts (compliance/, data audit reports, staging dirs), the *canonical* age is the `timestamp` field on the run's audit-log genesis event — that field is HMAC-signed when `FORGELM_AUDIT_SECRET` is set and cannot be silently reset by a filesystem operation. Phase 21 prefers this in-band timestamp; mtime is the fallback when no audit log is present. +2. **Suspenders:** for stand-alone JSONL corpora (no audit log next to them), Phase 21 still uses mtime but logs a `data.retention_age_source=mtime` note on the violation so an operator reading the report knows the age signal is filesystem-derived. + +The `--check-policy` JSON output includes `age_source ∈ {audit, mtime}` per artefact so a consumer that distrusts mtime can filter for `audit`-sourced ages only. + +### 3.4 Where the policy is checked Three sites: @@ -111,7 +133,7 @@ Three sites: 2. **`forgelm purge --check-policy` (no --row-id / --run-id)**: read-only scan; reports violations as a structured summary (text + JSON output formats). 3. **`forgelm audit`** (existing subcommand): when `retention` is set in the loaded config, the audit report's `notes` block lists violations as a friendly heads-up. -### 3.3 What "violation" means +### 3.5 What "violation" means For each retention horizon: - Compare artefact `mtime` against `now - _days`. @@ -138,8 +160,9 @@ A single invocation does exactly one of the three. Combining flags is a `Config | Flag | Required when | Type | Purpose | |---|---|---|---| -| `--row-id ` | corpus mode | str | Row to delete. Format: stable id field already in the JSONL row (e.g. `"id"` or `"row_id"` key); falls back to **line number** if the JSONL has no id field. | -| `--corpus ` | corpus mode | str | Path to the JSONL. May be a file or a directory containing `train.jsonl` / `validation.jsonl` / `test.jsonl` (purge scans all matching files; `--row-id` may match in zero or more of them). | +| `--row-id ` | corpus mode | str | Row to delete. **Required:** the JSONL must carry a stable id field (e.g. `"id"` or `"row_id"` key) on every row. Line-number fallback is **rejected** — operators with id-less corpora must run `forgelm audit --add-row-ids ` (Phase 28 follow-up) first, otherwise a re-ordered file would silently delete the wrong row. | +| `--corpus ` | corpus mode | str | Path to **a single JSONL file**. Directory mode is **rejected** in `--row-id` mode — multi-file purges are an operator script (loop over files), not a `forgelm purge` flag, because GDPR Article 17 expects an erasure decision *per row* with its own audit event. An invocation that matches the same id in two files is a "DELETE without WHERE" hazard. | +| `--row-matches` | corpus mode optional | enum | `one` (default; refuses if 0 or ≥2 matches in the file) / `all` (deletes every match, requires explicit opt-in for the ambiguity). Without this flag, multi-row matches in a single file abort with `EXIT_CONFIG_ERROR` so the operator confirms intent before bulk delete. | | `--run-id ` | run mode | str | Run id matching `/audit_log.jsonl`'s top-level `run_id`. | | `--kind {staging,artefacts}` | run mode | enum | Which derived artefacts to erase. `staging` = `final_model.staging.*`; `artefacts` = `compliance/`, `data_audit_report.json`, derived JSONL snapshots. | | `--output-dir ` | run mode | str | Where to look for the run. Default `./` (current dir). | @@ -147,28 +170,37 @@ A single invocation does exactly one of the three. Combining flags is a `Config | `--config ` | policy mode (optional) | str | Config to load for the retention block; defaults to `./forgelm.yaml` then walks up. | | `--justification ` | always optional | str | Free-text reason recorded in the audit event. Strongly recommended for compliance review. | | `--dry-run` | always optional | bool | Print what would be deleted; do not modify. | +| `--yes` | always optional | bool | Skip the interactive "confirm erasure of row X?" prompt. Required for unattended / scripted use; an interactive `forgelm purge` without `--yes` always prompts on a TTY and aborts (`EXIT_CONFIG_ERROR`) on a non-TTY. | | `--output-format {text,json}` | always optional | enum | Output format. Default `text`. | ### 4.3 Exit codes -Standard ForgeLM contract: +Standard ForgeLM 0/1/2/3/4 contract — all codes from `forgelm.cli._exit_codes` reused, no new value introduced: -| Code | Meaning | -|---|---| -| 0 | Erasure completed (or `--dry-run` reported successfully). | -| 1 | Config-level error: missing flag, flag combination, unknown row-id (corpus mode), unknown run-id (run mode). | -| 2 | Runtime / I/O failure: corpus unreadable, audit-log write failure, atomic-rename fail. | +| Code | Constant | Meaning | +|---|---|---| +| 0 | `EXIT_SUCCESS` | Erasure completed (or `--dry-run` reported successfully). | +| 1 | `EXIT_CONFIG_ERROR` | Config-level error: missing flag, flag combination, unknown row-id (corpus mode), unknown run-id (run mode), `--row-id` directory mode without `--row-matches=all`, multi-row match without `--row-matches=all`, missing `--yes` on a non-TTY. | +| 2 | `EXIT_TRAINING_ERROR` | Runtime / I/O failure: corpus unreadable, audit-log write failure, atomic-rename failure, partial commit recovery. | +| 3 | `EXIT_EVAL_FAILURE` | `--check-policy` found violations against the loaded `retention.*` policy (re-using the eval-failure semantic — "the gate detected something the operator must address" — so CI scripts can treat retention violations the same way they treat eval threshold breaches). | +| 4 | `EXIT_AWAITING_APPROVAL` | Interactive confirmation pending: a TTY operator declined the prompt, or an unattended invocation without `--yes` saw a TTY and printed the prompt rather than auto-deleting. | -`--check-policy` returns 0 even when violations exist — it is a report, not a gate. An operator who wants the violation count to gate CI uses `--output-format json` and checks the `violations` length. +`--check-policy` returns **3** when violations exist (not 0). An operator who wants the report only without the gate behaviour passes `--check-policy --output-format json` and inspects the `violations` array directly without checking the exit code. This aligns with `docs/standards/release.md` which treats every documented code as a CI-actionable contract. ### 4.4 Atomicity -**Corpus row deletion** uses the same tempfile + atomic rename pattern as `_atomic_write_json` in `_orchestrator.py`: +**Corpus row deletion** uses the same tempfile + atomic rename pattern as `_atomic_write_json` in `_orchestrator.py`, with a strict event-ordering invariant so a partial failure leaves an investigable trail: + +1. **Audit `data.erasure_requested` FIRST** — fields include `target_kind="row"`, `target_id` (hashed per §5.4), `corpus_path`, `justification`, `dry_run`. If this write fails, abort immediately; nothing on disk has changed. +2. Locate the matching row + record the pre-erasure line number for the completion event. +3. Open the source JSONL, open a `NamedTemporaryFile` in the same directory. +4. Stream rows, skipping the matched id. +5. `os.fsync` + `os.replace` to swap. +6. **Audit `data.erasure_completed` LAST** — same fields plus `bytes_freed`, `pre_erasure_line_number`, `files_modified=[corpus_path]`. -1. Open the source JSONL, open a `NamedTemporaryFile` in the same directory. -2. Stream rows, skipping the matched id. -3. `os.fsync` + `os.replace` to swap. -4. Emit `data.erasure_completed` audit event with the deleted row's `row_id`, file path, line number (post-fact: line number in the *new* file is no longer meaningful; we record the **pre-erasure** line number). +Recovery path for a step-5 failure (rename fails after the temp file is fully written): the `data.erasure_requested` event is in the chain, the temp file is left on disk for forensic inspection, and a `data.erasure_failed` event is appended carrying the original error. An operator inspecting the log sees the request → failure pair and can manually decide whether to retry or revert. + +**Recovery path for a step-6 failure** (rename succeeded, completion event write fails): the row is gone from the corpus but the chain has only the request event. This is detectable: a `request` event without a matching `completed` / `failed` event within the same minute is the operator's signal to verify the corpus state and append a manual `data.erasure_completed` (or `_failed`) event by hand. Phase 21 ships a `tests/test_gdpr_erasure.py::test_partial_commit_emits_failed_event` test that simulates this race + asserts the chain stays interpretable. **Run-scoped erasure** for `kind=staging` uses `shutil.rmtree(staging_path)` after a `os.path.commonpath` check (same defence-in-depth as `_approve.py`'s staging path validator) so an attacker who can rewrite the audit log cannot point us at `/etc`. @@ -192,13 +224,16 @@ The ForgeLM audit log is append-only and HMAC-signed (when `FORGELM_AUDIT_SECRET ### 5.1 New audit events -Three new events in `audit_event_catalog.md`: +Six new events in `audit_event_catalog.md` — three core erasure events plus three operator-warning events that surface gaps Phase 21 cannot close in code: | Event | When emitted | Required fields | |---|---|---| -| `data.erasure_requested` | The first call to `forgelm purge --row-id` / `--run-id` writes this *before* any deletion. If the deletion then fails partway, the requested record stays in the chain so a forensic reviewer can see the intent. | `target_kind` ∈ {row, staging, artefacts, policy_check}, `target_id` (row_id or run_id; `null` for policy_check), `corpus_path` (corpus mode), `output_dir` (run mode), `justification`, `dry_run` | +| `data.erasure_requested` | First step of any `forgelm purge --row-id` / `--run-id` invocation, *before* any deletion (see §4.4 commit ordering). If deletion then fails, the request record stays in the chain so a forensic reviewer can see the intent. | `target_kind` ∈ {row, staging, artefacts, policy_check}, `target_id` (hashed per §5.4 for row mode; `null` for policy_check), `corpus_path` (corpus mode), `output_dir` (run mode), `justification`, `dry_run` | | `data.erasure_completed` | Successful deletion finishes. | All `data.erasure_requested` fields + `bytes_freed`, `files_modified` (list), `pre_erasure_line_number` (corpus mode) | -| `data.erasure_failed` | Deletion failed (atomic rename failure, permission denied, I/O error). | All `data.erasure_requested` fields + `error_class`, `error_message` (already-redacted by `_http`'s mask helper for any URL-like content) | +| `data.erasure_failed` | Deletion failed (atomic rename failure, permission denied, I/O error). | All `data.erasure_requested` fields + `error_class`, `error_message` (already-redacted via §5.4 policy + `_http`'s mask helper) | +| `data.erasure_warning_memorisation` | Emitted alongside `data.erasure_completed` when `target_kind="row"` AND a `/final_model/` exists for any run that consumed this corpus. Signals that the row is gone from the corpus but may still be memorised in trained weights. | All `data.erasure_completed` fields + `affected_run_ids` (list of run ids whose final_model used this corpus), `recommendation` (literal `"full retraining without the erased row is the only proper mitigation"`) | +| `data.erasure_warning_synthetic_data_present` | Emitted alongside `data.erasure_completed` when `target_kind="row"` AND any `/synthetic_data*.jsonl` exists. Signals that the row may have produced derivative synthetic snippets that the row→snippet mapping no longer connects. | All `data.erasure_completed` fields + `synthetic_files` (list of paths), `recommendation` (literal `"regenerate synthetic data after erasure to ensure no derivatives reference the erased row"`) | +| `data.erasure_warning_external_copies` | Emitted alongside `data.erasure_completed` when the loaded config has a non-empty `webhook` block. Signals that downstream consumers may have received notices that referenced the now-erased data; ForgeLM's local erasure does not propagate. | All `data.erasure_completed` fields + `webhook_targets` (list of redacted URLs), `recommendation` (literal `"propagate the erasure to downstream processors per Article 17(2)"`) | ### 5.2 Chain-integrity post-erasure @@ -214,6 +249,28 @@ Event bodies carry the `target_id` (e.g. `row_id=42` or `run_id=fg-abc123`). Th If the **justification** field includes something that *might* be PII (e.g. operator pastes the data subject's email into the justification by mistake), we do **not** scrub it automatically — the operator chose to write it. We document this in the operator guide: "Do not paste subject identifiers into `--justification`; reference your internal ticket id instead." +### 5.4 Personal-data minimisation across every audit-event field + +Article 17(3)(b) preserves the audit log itself, but Article 5(1)(c) (data minimisation) still applies to **what we put in each event**. Phase 21 enumerates every field on every new audit event and classifies it; categories that may carry personal data get a hash / redaction policy at emit time so the persisted log carries the minimum identifying surface needed for accountability. + +| Event field | Source / shape | Personal-data category | Phase 21 policy | +|---|---|---|---| +| `target_kind` | enum: `row` / `staging` / `artefacts` / `policy_check` | None | clear | +| `target_id` (row mode) | row id from JSONL `id` field, or line number | **Possibly personal** (operator-supplied row IDs may be email / username / ticket numbers) | **SHA-256(salt + value)** when emitted; salt = first 16 bytes of `FORGELM_AUDIT_SECRET` if set, else a per-output-dir salt persisted at first use. Raw value never lands in the chain. | +| `target_id` (run mode) | `fg-` from AuditLogger | None (operational) | clear | +| `corpus_path` | filesystem path | **Possibly personal** when path includes a subject name (rare but real) | clear by default; operator guide flags the risk + recommends symlinking through a stable name | +| `output_dir` | filesystem path | Same risk as `corpus_path` | clear; same guide note | +| `justification` | operator free-text | **Operator's choice** | clear; operator guide explicitly forbids pasting subject identifiers, recommends ticket reference | +| `dry_run` | bool | None | clear | +| `bytes_freed` | int | None | clear | +| `files_modified` | list[str] | Same risk as `corpus_path` | clear | +| `pre_erasure_line_number` | int | None | clear | +| `error_class` | exception class name | None | clear | +| `error_message` | str | **Possibly personal** (Python tracebacks include `repr()` of objects, which may include row content) | route through `forgelm._http._mask_secrets_in_text` first; redact obvious PII (email regex, phone regex) via the existing `forgelm.data_audit._pii_regex` mask helpers | +| `operator` (carried on every AuditLogger event) | `FORGELM_OPERATOR` or `getuser()@host` | **Personal** | clear (Article 12 record-keeping requires identifiable operator; this is the explicit Article 5(1)(b) lawful basis) | + +The take-away: **the only field this design hashes by default is `target_id` in row mode**. Everything else either is genuinely operational, is the operator's responsibility (justification), or is masked through the existing Wave 1 `_mask_secrets_in_text` helper. Phase 21 ships a unit test `tests/test_gdpr_erasure.py::test_target_id_hashed_in_audit_event` that asserts the raw row id never appears in the persisted JSONL after a `forgelm purge --row-id ali@example.com --corpus train.jsonl` run. + --- ## 6. Marketing claim revision @@ -235,6 +292,25 @@ in three scopes: `forgelm purge --check-policy` lists artefacts that have overstayed the declared horizon. +**Scope limitation.** ForgeLM's erasure tooling acts on the **local +artefacts ForgeLM itself produced or consumed inside the operator's +training output directory**. The operator remains responsible for: + +- replicas, snapshots, and backups of the corpus stored outside the + ForgeLM output directory; +- downstream consumers (deployed model endpoints, third-party fine-tunes + derived from a published checkpoint, dataset mirrors on HF Hub); +- webhook receivers that persisted approval / rejection notices in + external systems (Slack threads, Teams channels, ticket trackers); +- legal-hold copies an upstream compliance team may have placed beyond + the operator's reach. + +When the loaded config carries a webhook target, Phase 21 emits +`data.erasure_warning_external_copies` alongside the +`data.erasure_completed` event so a downstream consumer querying the +audit log sees an explicit reminder that the local erasure is not the +end of the Article 17 obligation. + What ForgeLM does **not** do: - Re-training after erasure (your decision; the row is gone, the next @@ -351,7 +427,7 @@ The closure plan §15.5 lists three open items that intersect Phase 21: ## 12. Sign-off checklist (Phase 20 acceptance) -- [x] Document is ≥400 lines. (Line count ≈ 460 incl. tables.) +- [x] Document is ≥400 lines. (Line count = 445 actual at v2 — review-pass added §3.1 prior-state, §3.3 mtime caveat, §5.4 audit-PII minimisation, §4.4 commit ordering recovery paths, scope-limitation paragraph + 3 new warning events.) - [x] §1 maps every Article 17(1) trigger to ForgeLM scope. - [x] §2 enumerates every artefact kind and its erasure strategy. - [x] §3 specifies the `RetentionConfig` Pydantic schema (Phase 21 implements it verbatim).