Repository navigation
Conversation
…, and document the real memorisation check The purge reference and the user manual both said that the implicit `--output-dir` fallback "emits a WARNING naming the resolved dir". It did not: nothing was logged. The fallback is the corpus's directory, and the memorisation check (`data.erasure_warning_memorisation`) looks for `<output_dir>/final_model/` there, a place a training run rarely writes its model. A default `forgelm purge --row-id X --corpus data/train.jsonl` therefore ended with a completed-erasure record and no memorisation event, so a reader of the audit chain alone would conclude that no model had consumed the row. `--row-id` without `--output-dir` now logs one warning naming the resolved directory and the `final_model/` it will look for, which makes the documented behaviour true. The audit-event catalogue, the purge reference, the GDPR erasure guide, the user manual and the risk-treatment plan (English and Turkish) described the condition as "a final_model/ exists for any run that consumed the corpus"; the code checks one literal directory and cannot tell which runs consumed the corpus. They now say that. Reading `training.output_dir` from `--config` and scanning `final_model*` / `checkpoint-*` is the larger fix and stays open on the issue; it changes the content of an erasure audit event (additively) and wants its own review. Refs #102 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…tput-dir warning The warning that `forgelm purge --row-id` now logs when --output-dir is omitted adds eight lines to a module on the deferred-split list, and the budget has to equal the measured size. It is a raise rather than a split for the reason the previous raise gave (C-18): the message has to sit where the default is resolved. The history note says why. Refs #102 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Sorry @cemililik, you've used your own review budget of 250,000 diff characters for the last 7 days. You can request another review in 12 hours and 58 minutes by commenting |
|
Warning Review limit reachedEnable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. Next included review available in 1 minute. View limit details
Comment |
Reviewer's GuideThis PR makes the documented implicit --output-dir behavior real by logging the corpus-directory fallback and the exact final_model path inspected, adds focused regression tests, and updates English/Turkish operational documentation to clarify that memorisation detection is limited to a literal final_model directory directly under the selected output directory. Sequence diagram for purge output directory fallback and memorisation checksequenceDiagram
participant Operator
participant Purge as forgelm purge
participant Filesystem
participant Audit as Audit log
Operator->>Purge: _run_purge_row_id(args, output_format)
Purge->>Purge: Resolve output_dir from corpus directory
Purge-->>Operator: logger.warning(resolved output_dir and final_model path)
Purge->>Filesystem: os.makedirs(output_dir, exist_ok=True)
Purge->>Filesystem: Check output_dir/final_model
alt final_model exists
Purge->>Audit: data.erasure_warning_memorisation
end
Purge->>Audit: data.erasure_completed
Flow diagram for purge memorisation detection scopeflowchart TD
A[forgelm purge --row-id] --> B{--output-dir given?}
B -- No --> C[Use corpus parent directory]
C --> D[Log WARNING with resolved directory and final_model path]
B -- Yes --> E[Use selected output directory]
D --> F[Check exact output_dir/final_model directory]
E --> F
F --> G{final_model exists directly under output_dir?}
G -- Yes --> H[Emit data.erasure_warning_memorisation]
G -- No --> I[No memorisation warning]
H --> J[Complete erasure audit]
I --> J
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Superseded by #999: the wave-0 fixes are combined by theme into five pull requests to cut the review load. This branch's commits are included there unchanged apart from the |
…n.enforce, transformers CVE floor, FSDP strategies, purge output dir (#999) ## Summary Six code fixes combined into one pull request, replacing #984, #985, #987, #989, #990 and #992 (to be closed as superseded). Each fixes a case where the code failed, or a documented option did nothing, without saying so. Each was mutation-checked on its own; together they pass the full gauntlet. ## Related issues Refs #103, #95, #92, #98, #109, #104, #102. Not closed by this change: the top-of-range problems named in #109 (`warmup_ratio` on transformers 5.15+, the string `fsdp` form removed in 5.20), the FSDP2 migration (#104), and reading `training.output_dir` for the purge memorisation check (#102). ## Changes | # | Fix | Where | |---|---|---| | #984 | `training.rope_scaling` crashed every model load on transformers 5 (`KeyError: 'rope_type'`, or `rope_theta` lost). The requested keys are merged **over** the model's `rope_parameters` and the finished config is loaded; `sliding_window_attention` travels in the same config object. | `forgelm/model.py` | | #985 | The CLI sets `HF_HUB_DISABLE_TELEMETRY=1` (via `setdefault`) before `huggingface_hub` is imported, so TRL's trainer no longer sends Hugging Face's anonymous usage ping the privacy page says does not exist. Library use is not covered; the air-gap page says so. | `cli/_config_load.py`, `cli/_dispatch.py` | | #987 | `retention.enforce` was documented as a gate (`warn_on_excess`, `block_on_excess` → `EXIT_EVAL_FAILURE`) but no command reads it. Docs, field description, both wizards say it is reserved; the CLI warns when a config sets either of the last two. | `config.py` (in place), `cli/_config_load.py`, `wizard/_collectors.py` | | #989 | `transformers>=5.3.0` was called the CVE floor but admitted CVE-2026-5241 and CVE-2026-9856 (a crafted `chat_template` name writes outside the save directory). Floor is now `>=5.10.1` (5.10.0 is yanked), `huggingface_hub>=1.5.0`, the nightly floor job pins both, and tests keep it from drifting. | `pyproject.toml`, `nightly.yml`, `tests/` | | #990 | `fsdp_strategy: no_shard` / `hybrid_shard` passed `--dry-run` and exited 2 after the model loaded (FSDP2 rejects them). They are now rejected when the config loads (exit 1), and two knobs FSDP2 ignores (`fsdp_auto_wrap: false`, `backward_post`) log a warning. | `config.py` (in place), `cli/_config_load.py` | | #992 | `forgelm purge --row-id` without `--output-dir` silently pointed the memorisation check at the corpus directory. It now warns and names the path. The purge module's line budget is raised by 8 with a `budget_history` note (the project's documented option). | `cli/subcommands/_purge.py`, `tools/check_module_size.py` | `config.py` (0 lines of headroom on the module-size ratchet) and `trainer.py` change only by in-place string edits; new warnings live in `cli/_config_load.py`. ### Merge-conflict handling The six branches touched `CHANGELOG.md`, `tests/test_cli.py` (two appended test classes) and the import line in `cli/_dispatch.py`; all three were resolved by keeping both sides (the import line now imports `_disable_hub_telemetry` and `_warn_inert_fsdp_options`). The affected test modules pass together (621 tests). ## Type - [x] Bug fix - [x] Dependency ## Testing - The full `CLAUDE.md` gauntlet from a clean worktree of this branch: exit 0. - Per-fix tests as in the original pull requests, including: a real tiny Llama on CPU for `rope_scaling` (linear, dynamic, yarn); the traversal reproduction (**fails on transformers 5.3.0**, passes on 5.10.1); all 32 accepted FSDP variants building real `TrainingArguments`. - transformers-facing test modules pass on a 5.10.1 / `huggingface_hub` 1.5.0 overlay (443 tests). The nightly minimum-versions job runs on its schedule after merge; I did not dispatch it. ## Checklist - [x] English and Turkish mirrors updated together - [x] CHANGELOG entries under `[Unreleased]` - [x] Tests added 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by Sourcery Harden CLI behavior and dependency compatibility across model loading, privacy, retention, distributed training, and purge workflows. Bug Fixes: - Restore Transformers 5 compatibility for RoPE scaling and sliding-window model configuration. - Disable Hugging Face Hub telemetry by default for CLI runs while preserving explicit operator opt-in. - Warn when reserved retention enforcement modes are configured instead of implying unsupported enforcement behavior. - Reject unsupported FSDP1-only strategies during configuration and warn about FSDP2 options that have no effect. - Warn when purge uses the corpus directory as its implicit output directory and clarify the memorisation-check scope. Enhancements: - Document the actual retention, FSDP, telemetry, and purge behavior in English and Turkish user-facing documentation. Build: - Raise the minimum runtime dependencies to transformers>=5.10.1 and huggingface_hub>=1.5.0 to address relevant security advisories. - Update the nightly minimum-version job and packaging checks to enforce the new dependency floors. Tests: - Add coverage for telemetry initialization, reserved retention warnings, FSDP validation, purge output-directory warnings, Transformers 5 RoPE handling, and chat-template path traversal protection. Chores: - Increase the purge module size budget for the new output-directory warning. --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Summary
The purge reference and the user manual said that the implicit
--output-dirfallback "emits a WARNING naming the resolved dir". It did not: nothing was logged. The fallback is the corpus's directory, and the memorisation check (data.erasure_warning_memorisation) looks for<output_dir>/final_model/there, a place a training run rarely writes its model. A defaultforgelm purge --row-id X --corpus data/train.jsonltherefore ended with a completed-erasure record and no memorisation event, so a reader of the audit chain alone would conclude that no model had consumed the row.--row-idwithout--output-dirnow logs one warning naming the resolved directory and thefinal_model/path it will look for, which makes the documented behaviour true. The documentation also described the condition as "afinal_model/exists for any run that consumed the corpus"; the code checks one literal directory and cannot tell which runs consumed the corpus. It now says that.Related issues
Refs #102 — the warning and the documentation. Reading
training.output_dirfrom--configand scanningfinal_model*/checkpoint-*is the larger fix; it changes the content of an erasure audit event (additively) and wants its own review, so it stays open.Changes
forgelm/cli/subcommands/_purge.py: when--output-diris omitted, aWARNINGnames the corpus directory used for the audit log, the salt and the memorisation check, thefinal_model/path that check looks for, and tells the operator to pass--output-dir <training output dir>(also needed to correlate withforgelm reverse-pii).tools/check_module_size.py: the purge module's budget goes from 1216 to 1224 lines (+8) with a history note. This follows the project's documented option ("land the split or raise the budget with abudget_historynote") and the previous raise's reasoning (C-18): the message has to sit where the default is resolved.purgereference, the GDPR erasure guide and user-manual page, and the risk-treatment plan.Type
Testing
tests/test_gdpr_erasure.py::TestImplicitOutputDirWarning: the warning fires without--output-dir, names both paths, and is absent when--output-diris given.tests/test_check_module_size.py(the budget); that is fixed by the second commit. The fullCLAUDE.mdgauntlet from a clean worktree of the final state: exit 0.Checklist
[Unreleased]🤖 Generated with Claude Code
Summary by Sourcery
Make implicit purge output-directory behavior visible and align memorisation-check documentation with the implementation.
Bug Fixes:
purge --row-idimplicitly uses the corpus directory as--output-dir, including the exactfinal_model/path checked for memorisation.Enhancements:
final_model/directly under--output-dirand cannot identify which training runs consumed the corpus.Build:
Documentation:
Tests: