Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 19 additions & 19 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ documentation pass.
**Faz 22 — ISO 27001 / SOC 2 alignment design**

- New design document at
[`docs/analysis/code_reviews/iso27001-soc2-alignment-202605052315.md`](docs/analysis/code_reviews/iso27001-soc2-alignment-202605052315.md)
[`docs/design/iso27001_soc2_alignment.md`](docs/design/iso27001_soc2_alignment.md)
(~865 lines). Maps every ISO 27001:2022 Annex A control (93
controls across A.5 + A.6 + A.7 + A.8) and every SOC 2 Trust
Services Criterion (Common Criteria CC1.x–CC9.x mandatory + 4
Expand Down Expand Up @@ -845,7 +845,7 @@ enough that the move can land alongside Phase 21's first commit.

**Docs:**

- `docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md` §5.1
- `docs/design/gdpr_erasure.md` §5.1
event catalog row for `data.erasure_requested` now lists `salt_source`
(row mode only). Wave 2a Round-5 F-R5-05 introduced the field in
§5.4 (per-event personal-data table) but did not propagate it to
Expand Down Expand Up @@ -984,7 +984,7 @@ in this revision. Full delta:
**Design-doc fixes** (no behavioural change to v0.5.5; tightens Phase 19
/ 21 implementer guidance):

- Library API design (`library-api-design-202605021414.md`): CI trigger
- Library API design (`library_api.md`): CI trigger
fixed (was `release-*` branch which release.md:259 forbids; now
`pull_request` to main + `workflow_dispatch`); `[evaluation]` extra
name fixed to `[eval]`; lazy-import test #9 promise corrected
Expand All @@ -993,7 +993,7 @@ in this revision. Full delta:
attribute access fixed to `["pairs"]` dict key access; tier table
10th test row added (`test_config_from_dict`); `__dir__()` recipe
alignment; `forgelm.ingestion` correctly labelled "the module".
- GDPR erasure design (`gdpr-erasure-design-202605021414.md`):
- GDPR erasure design (`gdpr_erasure.md`):
`staging_ttl_days` dual-set semantics resolved (ConfigError on
conflict + tracking issue per release.md:95); §4.3 vs §10 Q5
contradiction resolved (`--check-policy` always exits 0; report-not-
Expand Down Expand Up @@ -1059,7 +1059,7 @@ shipped a fix. Three were rejected with rationale:

Fixed:

- `docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md` —
- `docs/design/gdpr_erasure.md` —
unescaped `|` in markdown table cell (`compliance/*.json|yaml` →
`compliance/*.{json,yaml}`); outdated hard-coded line count "445" in
§12 sign-off replaced with a "recheck with `wc -l`" note that won't
Expand Down Expand Up @@ -1153,17 +1153,17 @@ Fixed:
- *qodo-code-review L682 ("doctor uses non-secret env-vars")* — same
rejection as the round-1 entry above; doctor mirrors what
`compliance.py::AuditLogger` and `huggingface_hub` upstream read.
- *CodeRabbit "Count erasure events consistently" (gdpr-erasure-design
- *CodeRabbit "Count erasure events consistently" (gdpr_erasure design
L388 file map + L454 sign-off)* — file map already says six events at
L389; the sign-off line at L454 says "six new audit events (three
core erasure events + three operator-warning events)"; the L449 "+ 3
new warning events" parenthetical refers to what Round-2 *added* on
top of Round-1's three core events, not the document's total.
- *CodeRabbit "--check-policy exit code unambiguous"* — already
resolved at gdpr-erasure-design L196 (exit-code table cross-references
resolved at gdpr_erasure design L196 (exit-code table cross-references
§10 Q5) + L199 (paragraph: "**`--check-policy` always exits 0**").
- *CodeRabbit "ForgeTrainer access must stay lazy"* — the test at
library-api-design L349 already pins lazy behaviour
library_api design L349 already pins lazy behaviour
(`assert "torch" not in sys.modules` after `_ = forgelm.ForgeTrainer`);
the bot misread it as the opposite.
- *CodeRabbit TR installation L109 "`distributed` → `deepspeed` rename"* —
Expand Down Expand Up @@ -1200,24 +1200,24 @@ Fixed:
CI/CD scripts that branch on exit code. The
`_SIGINT_GUARDED_SUBCOMMANDS` constant was deleted because the new
policy is uniform across every subcommand.
- `docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md`
- `docs/design/gdpr_erasure.md`
§5.4 `target_id` row-mode source claim aligned with §4.2 — removed
the "or line number" fallback from the table cell (it was already
rejected at the CLI per L174); the cell now explicitly cites the
Phase 28 `forgelm audit --add-row-ids` follow-up that operators with
id-less corpora must run first.
- `docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md`
- `docs/design/gdpr_erasure.md`
§3.1 dual-set conflict-resolution bullets — varied the leading
phrase across the four bullets ("When only X is set", "When only Y
is set", "In the case where both are set with identical values",
"If both are set with different values") so the LanguageTool /
prose-style lint stops flagging the four-in-a-row "If" pattern.
Behaviour spec (alias-forward / canonical / DeprecationWarning /
ConfigError) is unchanged.
- `docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md`
- `docs/design/gdpr_erasure.md`
§3.3 mtime-distrust sentence reworded to remove the "consumer that
distrusts mtime" person/object ambiguity.
- `docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md`
- `docs/design/gdpr_erasure.md`
§5.3 wording: "by mistake" → "accidentally" for concision.
- `docs/usermanuals/tr/reference/cli.md`: `--output DIR` row "ya da"
→ "veya" (more formal Turkish for the docs register); `forgelm
Expand Down Expand Up @@ -1297,20 +1297,20 @@ OSError at ERROR level which `--quiet`'s WARNING floor surfaces).
claimed the symbol was a re-export for tests. (F-R5-08)
- `CHANGELOG.md` — inserted blank line before the
`### Added — Wave 2a / Phase 34` heading (MD022 fix). (Inline A)
- `docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md`
- `docs/design/gdpr_erasure.md`
§5.3 reconciled with §5.4: clarified that `target_id` handling
depends on `target_kind` (run mode = clear, row mode = SHA-256
hash), and that the `row_id=42` examples in operator-facing prose
show what the operator typed on the CLI rather than what lands in
the audit chain. (Inline B)
- `docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md`
- `docs/design/gdpr_erasure.md`
§3.4 reconciled with §4.1: `compliance/*.json` IS deletable via
`forgelm purge --run-id <id> --kind artefacts`; the previous
"deleting them requires deleting the whole `<output_dir>`" claim
contradicted the purge-mode table. Whole-`<output_dir>` deletion
is now framed as the operator's manual escalation when the entire
run is being scrubbed. (Inline C)
- `docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md`
- `docs/design/gdpr_erasure.md`
§5.4 — `target_id` salt-fallback semantics tightened: the
per-output-dir salt is **persistent** (written at
`<output_dir>/.forgelm_audit_salt`, mode 0600, on first emission)
Expand All @@ -1328,7 +1328,7 @@ OSError at ERROR level which `--quiet`'s WARNING floor surfaces).
### Added — Wave 2a — Phase 18 Library API design + Phase 20 GDPR erasure design

- **Phase 18 — Library API analysis & design** —
`docs/analysis/code_reviews/library-api-design-202605021414.md` (525 lines).
`docs/design/library_api.md` (525 lines).
12 sections + 16-row task plan that pin the public Python surface for
downstream consumers. Resolves the stable / experimental / internal tier
split; documents the lazy-import invariant (`import forgelm` does not pull
Expand All @@ -1340,7 +1340,7 @@ OSError at ERROR level which `--quiet`'s WARNING floor surfaces).
`__getattr__` migration for `forgelm/cli/__init__.py`); specifies the
`tests/test_library_api.py` integration suite Phase 19 must ship.
- **Phase 20 — GDPR Article 17 erasure analysis & design** —
`docs/analysis/code_reviews/gdpr-erasure-design-202605021414.md`. 12
`docs/design/gdpr_erasure.md`. 12
sections + 11-test plan + file map that pin the scope of Phase 21's
`forgelm purge` implementation. Maps every Article 17(1) trigger to a
ForgeLM action; enumerates the seven artefact kinds that may carry
Expand Down Expand Up @@ -1590,7 +1590,7 @@ Both are design-only PRs — Phase 19 + Phase 21 implementations follow.
[master-review-opus-202604300906.md](docs/analysis/code_reviews/master-review-opus-202604300906.md)
(175 findings).
- `data_audit/` + `cli/` package split design:
[split-design-data_audit-cli-202604300906.md](docs/analysis/code_reviews/split-design-data_audit-cli-202604300906.md)
[data_audit_cli_split.md](docs/design/data_audit_cli_split.md)
(Faz 14-15 forward-looking).

### Deprecated
Expand Down Expand Up @@ -1624,7 +1624,7 @@ Both are design-only PRs — Phase 19 + Phase 21 implementations follow.
so external callers (`forgelm.ingestion`, `forgelm.wizard`) and the
test suite keep working without code changes. Closes F-code-103 (Major).
See
[split-design-data_audit-cli-202604300906.md](docs/analysis/code_reviews/split-design-data_audit-cli-202604300906.md)
[data_audit_cli_split.md](docs/design/data_audit_cli_split.md)
§1 for the design.
- **`forgelm/cli.py` → `forgelm/cli/` package (Faz 15)** — the ~2300-line
monolith was split into a 24-module package (subcommands/, `_dispatch`,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
# Split Design — `forgelm/data_audit.py` and `forgelm/cli.py`

**Created:** 2026-04-30
**Commit:** `6b515ed912f8f22304194c1b3f55ed07a26f519c`
**Branch:** `main`
**Companion to:** [`master-review-opus-202604300906.md`](./master-review-opus-202604300906.md) (§3 Theme γ, §5.5, §8.3) and [`closure-plan-202604300906.md`](./closure-plan-202604300906.md)
**Author:** Plan agent (read-only design pass; full file reads of both modules + every cross-import in `tests/` and `forgelm/`)
> **Scope:** Design specification for the sub-package split of two
> historically-monolithic modules: `forgelm/data_audit.py` (was 3098
> lines) → `forgelm/data_audit/` package, and `forgelm/cli.py` (was
> 1756 lines) → `forgelm/cli/` package. Records the chosen module
> boundary, public-symbol re-export shape, and lazy-import contract
> rationale. Living spec — kept in sync with the implementation under
> `forgelm/data_audit/__init__.py` and `forgelm/cli/__init__.py`.

**Status:** Implemented in v0.5.5 (Phase 14 + Phase 15 splits). See
`CHANGELOG.md` for the user-facing summary.

---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
# ForgeLM GDPR Right-to-Erasure — Analysis & Design
# ForgeLM GDPR Right-to-Erasure — Design

**Document ID:** `gdpr-erasure-design-202605021414`
**Status:** Draft (Phase 20 deliverable)
**Author:** Closure Wave 2a
**Companion:** [`closure-plan-202604300906.md §8 Phase 20`](./closure-plan-202604300906.md)
**Implements (next phase):** Phase 21 — GDPR right-to-erasure implementation
**Base commit:** `0ffdfd6` (post-Wave-1 merge into `development`); Round-1 + Round-2 review fixes absorbed in the same wave (PR #28 `closure/wave2a-integration`).
**Regulatory anchor:** EU GDPR Article 17 ("right to erasure" / "right to be forgotten") + Article 5(1)(e) ("storage limitation")
> **Scope:** Design specification for ForgeLM's GDPR Article 17
> right-to-erasure tool surface — the `forgelm purge` subcommand
> semantics, the `RetentionConfig` Pydantic schema, and the audit-chain
> integration that records each erasure / refusal event. Living spec —
> kept in sync with the implementation under
> `forgelm/cli/subcommands/_purge.py` and `forgelm/config.py`
> (`RetentionConfig`).

**Regulatory anchor:** EU GDPR Article 17 ("right to erasure" / "right
to be forgotten") + Article 5(1)(e) ("storage limitation").

**Status:** Implemented in v0.5.5. See `CHANGELOG.md` and
`docs/guides/gdpr_erasure.md` for the user-facing surface.

---

Expand Down Expand Up @@ -342,7 +348,7 @@ What ForgeLM does **not** do:
- Pushing erasure notices to downstream processors (Article 17(2)) — that
is your runtime layer's responsibility, not the training toolkit's.

See [`docs/guides/gdpr_erasure.md`](../../guides/gdpr_erasure.md) for the operator how-to
See [`docs/guides/gdpr_erasure.md`](../guides/gdpr_erasure.md) for the operator how-to
and the audit-event reference (Phase 21 deliverable; the file is added by
the same PR that ships `forgelm purge`).
```
Expand Down Expand Up @@ -405,7 +411,7 @@ The closure plan §15.5 lists three open items that intersect Phase 21:
|---|---|
| **GH-023** — `ingestion.retention.raw_documents.ttl_days` config key cited by GDPR docs but not in `ForgeConfig` | Phase 21 adds `RetentionConfig` *as a top-level `retention:` block*, not nested under `ingestion`. The GDPR docs are updated to reference `retention.audit_log_retention_days` etc. An `ingestion.retention.*` alias is **not** added — the docs were ahead of the implementation, and the cleaner shape is to put retention under its own root key (it covers more than just ingestion artefacts). |
| **GH-013** — `forgelm purge` documented but not implemented | Phase 21 ships exactly the form documented in `docs/usermanuals/en/compliance/gdpr-erasure.md` (`--row-id`, `--corpus`, `--run-id`, `--kind`, `--check-policy`). The user-manual page becomes accurate post-merge. |
| **Concurrent-approve race (Wave 1 round-5 carry-over)** | Phase 21 introduces `acquire_run_lock(run_id)` context manager (filelock pattern) for the audit-log + purge + approve coordination. Phase 22 ISO/SOC 2 scope already lists this; Phase 21 is the first place we actually need it (concurrent purge of the same row), so we ship the lock here and Phase 22 documents the broader policy. |
| **Concurrent-approve race** | Phase 21 introduces `acquire_run_lock(run_id)` context manager (filelock pattern) for the audit-log + purge + approve coordination. Phase 22 ISO/SOC 2 scope already lists this; Phase 21 is the first place we actually need it (concurrent purge of the same row), so we ship the lock here and Phase 22 documents the broader policy. |

---

Expand Down Expand Up @@ -448,21 +454,12 @@ The closure plan §15.5 lists three open items that intersect Phase 21:

---

## 12. Sign-off checklist (Phase 20 acceptance)

- [x] Document is ≥400 lines. (Concrete line count drifts as later review passes amend in place — recheck with `wc -l` rather than relying on a hard-coded count. Sections added through Round-1 + Round-2 review: §3.1 prior-state, §3.3 mtime caveat, §5.4 audit-PII minimisation, §4.4 commit-ordering recovery paths, scope-limitation paragraph + 3 new warning events.)
- [x] §1 maps every Article 17(1) trigger to ForgeLM scope.
- [x] §2 enumerates every artefact kind and its erasure strategy.
- [x] §3 specifies the `RetentionConfig` Pydantic schema (Phase 21 implements it verbatim).
- [x] §4 specifies the `forgelm purge` flag surface, exit codes, and atomicity guarantees.
- [x] §5 names the six new audit events (three core erasure events + three operator-warning events) + how chain integrity holds.
- [x] §6 supplies the marketing-claim replacement text.
- [x] §7 enumerates the 11 tests Phase 21 must ship (≥ closure-plan minimum 7).
- [x] §8 file map covers every change Phase 21 will make.
- [x] §9 absorbs the three relevant Wave 1 carry-overs (GH-023, GH-013, run-lock).
- [x] §10 records and resolves the six open questions.
- [x] §11 names what is deliberately out of scope.
## 12. Implementation status

---
The design above shipped end-to-end in v0.5.5:

*End of Phase 20 design — `gdpr-erasure-design-202605021414`.*
- **`RetentionConfig`** (§3): pinned in `forgelm/config.py` as a top-level `retention:` block with five fields (`audit_log_retention_days`, `staging_ttl_days`, `ephemeral_artefact_retention_days`, `raw_documents_retention_days`, `enforce`). `EvaluationConfig.staging_ttl_days` deprecated with `DeprecationWarning` + alias-forwarding to `retention.staging_ttl_days`.
- **`forgelm purge` subcommand** (§4): pinned in `forgelm/cli/subcommands/_purge.py` with `--row-id`, `--corpus`, `--run-id`, `--kind`, `--check-policy`, atomic-rewrite + recovery semantics.
- **Audit events** (§5): six new events live in `audit_event_catalog.md` — `data.erasure_requested`, `data.erasure_completed`, `data.erasure_failed`, plus three operator-warning events.
- **Tests** (§7): `tests/test_gdpr_erasure.py` ships the regression coverage (unknown row id, multi-match refusal, partial-commit recovery, retention horizon enforcement, audit-PII redaction, etc.).
- **User-facing docs** (§6, §8): `docs/guides/gdpr_erasure{,-tr}.md` + `docs/usermanuals/{en,tr}/compliance/gdpr-erasure.md` reference the implemented surface.
Loading
Loading