diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d44dd02..58631870 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,11 @@ All notable changes to ForgeLM are documented here. ### Fixed +- `retention.enforce` is documented for what it is: reserved. No command reads it, so `warn_on_excess` and + `block_on_excess` never warned or blocked anything and nothing was deleted automatically; retention is only reported, + read-only, by `forgelm purge --check-policy` (always exit 0). The config description, the wizard prompt, the + configuration reference, the GDPR erasure guide and `config_template.yaml` now say so (English and Turkish), and + the CLI logs a warning when a config sets a non-default `enforce` instead of staying silent (#92, #98). - Fix the KTO bullet on the Alignment overview user-manual page (English and Turkish): it named the fields `(prompt, response, good?)`, but the KTO loader reads `prompt`, `completion` and a boolean `label` (#965). diff --git a/config_template.yaml b/config_template.yaml index 8aec1398..d364836e 100644 --- a/config_template.yaml +++ b/config_template.yaml @@ -276,7 +276,7 @@ data: # staging_ttl_days: 7 # final_model.staging./ TTL after `forgelm reject` # ephemeral_artefact_retention_days: 90 # compliance bundles + audit reports # raw_documents_retention_days: 90 # ingested raw docs (PDF/DOCX/EPUB/TXT/MD) -# enforce: "log_only" # "log_only" | "warn_on_excess" | "block_on_excess" +# enforce: "log_only" # reserved: no command acts on it yet ("log_only" | "warn_on_excess" | "block_on_excess") # Phase 14 — Multi-stage training pipeline chains (SFT → DPO → GRPO). # Uncomment the ``pipeline:`` block below to run multiple training stages diff --git a/docs/guides/gdpr_erasure-tr.md b/docs/guides/gdpr_erasure-tr.md index 64a1bb16..6f22323e 100644 --- a/docs/guides/gdpr_erasure-tr.md +++ b/docs/guides/gdpr_erasure-tr.md @@ -78,10 +78,10 @@ retention: staging_ttl_days: 7 # `forgelm reject` üzerine aksiyon almak için bir iş haftası ephemeral_artefact_retention_days: 90 # üç-aylık review cadence'i raw_documents_retention_days: 90 # data audit yeniden çalıştırmadan önce ingestion penceresi - enforce: log_only # log_only / warn_on_excess / block_on_excess + enforce: log_only # ayrılmış: henüz hiçbir komut onu kullanmaz (log_only / warn_on_excess / block_on_excess) ``` -Herhangi bir horizon'u `0` olarak ayarlamak o artefact kind'ı için policy'yi devre dışı bırakır (sınırsız tut). `enforce`, trainer pre-flight gate'inin ihlallere nasıl tepki vereceğini kontrol eder. +Herhangi bir horizon'u `0` olarak ayarlamak o artefact kind'ı için policy'yi devre dışı bırakır (sınırsız tut). Horizon'lar `forgelm purge --check-policy` ile raporlanır (salt-okunur, her zaman 0 ile çıkar); hiçbir şey otomatik silinmez ya da engellenmez. `enforce` ayrılmıştır: henüz hiçbir komut onu kullanmaz, bu yüzden üç değer de `log_only` gibi davranır. ## Kaldırıldı: `evaluation.staging_ttl_days` diff --git a/docs/guides/gdpr_erasure.md b/docs/guides/gdpr_erasure.md index 24941879..b5d5890e 100644 --- a/docs/guides/gdpr_erasure.md +++ b/docs/guides/gdpr_erasure.md @@ -78,10 +78,10 @@ retention: staging_ttl_days: 7 # one work-week to act on a `forgelm reject` ephemeral_artefact_retention_days: 90 # quarterly review cadence raw_documents_retention_days: 90 # ingestion-window before re-running data audit - enforce: log_only # log_only / warn_on_excess / block_on_excess + enforce: log_only # reserved: no command acts on it yet (log_only / warn_on_excess / block_on_excess) ``` -Setting any horizon to `0` disables the policy for that artefact kind (retain indefinitely). `enforce` controls how the trainer pre-flight gate reacts to violations. +Setting any horizon to `0` disables the policy for that artefact kind (retain indefinitely). The horizons are reported by `forgelm purge --check-policy` (read-only, always exits 0); nothing is deleted or blocked automatically. `enforce` is reserved: no command acts on it yet, so all three values behave like `log_only`. ## Removed: `evaluation.staging_ttl_days` diff --git a/docs/reference/configuration-tr.md b/docs/reference/configuration-tr.md index b78fa07f..38e6cbe6 100644 --- a/docs/reference/configuration-tr.md +++ b/docs/reference/configuration-tr.md @@ -270,10 +270,10 @@ training: Uyumluluk, eğitim ve değerlendirme artefaktları için saklama ufuklarını belirler. Ufuklar GDPR Madde 5(1)(e) "saklama sınırlaması" ve Madde 17 -"silme hakkı" tarihlerini onurlandırır. `enforce` anahtarı yalnız-loglama, -uyarı ve sert-engelleme modları arasında geçiş yaparak regüle edilen bir CI -kapısının saklama ufkunu eski bir çalışma alanını yeniden kullanarak sessizce -uzatmasını engeller. +"silme hakkı" tarihlerini onurlandırır. Ufuklar **raporlanır**, zorlanmaz: +`forgelm purge --check-policy` ufku aşanları listeler ve her zaman 0 ile çıkar; +hiçbir şey otomatik silinmez, engellenmez ya da zamanlanmaz. `enforce` anahtarı +ayrılmıştır: henüz hiçbir komut onu okumaz. | Alan | Tip | Varsayılan | Açıklama | |------|-----|-----------|----------| @@ -281,7 +281,7 @@ uzatmasını engeller. | `staging_ttl_days` | int | `7` | `forgelm reject` kararından sonra `final_model.staging./` dizininin planlı temizlenmeden önce saklanacağı gün sayısı. `0` süresiz saklama anlamına gelir. v0.8.0'da kaldırılan `evaluation.staging_ttl_days` yerine geçer. | | `ephemeral_artefact_retention_days` | int | `90` | Uyumluluk paketleri, veri denetim raporları ve diğer çalışma kapsamlı türetilmiş artefaktların saklanma süresi (gün). `0` süresiz saklama. | | `raw_documents_retention_days` | int | `90` | İngest edilmiş ham belgelerin (PDF / DOCX / EPUB / TXT / Markdown) operatörün ingestion-output dizininde saklanma süresi (gün). `0` süresiz saklama. | -| `enforce` | string | `"log_only"` | Politika uygulama modu: `"log_only"` (yalnızca audit log), `"warn_on_excess"` (stderr'e yapılandırılmış uyarı), `"block_on_excess"` (`EXIT_EVAL_FAILURE` = 3 ile trainer ön-kontrolünü iptal eder). | +| `enforce` | string | `"log_only"` | Ayrılmış politika modu. Henüz hiçbir komut onu okumaz: `"log_only"`, `"warn_on_excess"` ve `"block_on_excess"` kabul edilir ve aynı davranır; bir config son ikisinden birini ayarlarsa CLI uyarır. | > **Kaldırıldı:** `evaluation.staging_ttl_days` (v0.5.5 itibarıyla kullanımdan > kaldırılmıştı) v0.8.0'da kaldırılmıştır. Artık geçerli tek form diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 0f83d16e..0762173e 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -270,9 +270,10 @@ across retries. Each retry attempt is logged to the audit trail. Defines maximum retention horizons for compliance, training, and evaluation artefacts. Horizons honour GDPR Article 5(1)(e) "storage limitation" and -Article 17 "right to erasure" deadlines. The `enforce` knob switches between -log-only, warning, and hard-block modes so a regulated CI gate cannot -silently extend the retention horizon by re-using a stale workspace. +Article 17 "right to erasure" deadlines. The horizons are **reported**, not +enforced: `forgelm purge --check-policy` lists what is past them and always +exits 0, and nothing is deleted, blocked or scheduled automatically. The +`enforce` knob is reserved: no command reads it yet. | Field | Type | Default | Description | |-------|------|---------|-------------| @@ -280,7 +281,7 @@ silently extend the retention horizon by re-using a stale workspace. | `staging_ttl_days` | int | `7` | Days to retain `final_model.staging./` after a `forgelm reject` decision before scheduled cleanup. Set to `0` to retain indefinitely. Replaces the removed `evaluation.staging_ttl_days` (removed in v0.8.0). | | `ephemeral_artefact_retention_days` | int | `90` | Days to retain compliance bundles, data audit reports, and other run-scoped derived artefacts. Set to `0` to retain indefinitely. | | `raw_documents_retention_days` | int | `90` | Days to retain ingested raw documents (PDF / DOCX / EPUB / TXT / Markdown) under the operator's ingestion-output directory. Set to `0` to retain indefinitely. | -| `enforce` | string | `"log_only"` | Policy enforcement mode: `"log_only"` (audit-log only), `"warn_on_excess"` (structured stderr warning), `"block_on_excess"` (abort trainer pre-flight with `EXIT_EVAL_FAILURE` = 3). | +| `enforce` | string | `"log_only"` | Reserved policy mode. No command reads it yet: `"log_only"`, `"warn_on_excess"` and `"block_on_excess"` are all accepted and behave the same, and the CLI warns when a config sets either of the last two. | > **Removed:** `evaluation.staging_ttl_days` (deprecated as of v0.5.5) was > removed in v0.8.0. `retention.staging_ttl_days` is now the only accepted form. diff --git a/docs/usermanuals/en/reference/configuration.md b/docs/usermanuals/en/reference/configuration.md index c4ba73c2..4d3b8f69 100644 --- a/docs/usermanuals/en/reference/configuration.md +++ b/docs/usermanuals/en/reference/configuration.md @@ -363,7 +363,7 @@ retention: staging_ttl_days: 7 # days to retain staging model after forgelm reject ephemeral_artefact_retention_days: 90 # compliance bundles, audit reports raw_documents_retention_days: 90 # ingested PDF/DOCX/EPUB/TXT/Markdown - enforce: "log_only" # log_only | warn_on_excess | block_on_excess + enforce: "log_only" # reserved: no command acts on it yet (log_only | warn_on_excess | block_on_excess) ``` ## `pipeline:` diff --git a/docs/usermanuals/tr/reference/configuration.md b/docs/usermanuals/tr/reference/configuration.md index 39f8e44b..d94545c9 100644 --- a/docs/usermanuals/tr/reference/configuration.md +++ b/docs/usermanuals/tr/reference/configuration.md @@ -371,7 +371,7 @@ retention: staging_ttl_days: 7 # forgelm reject sonrası staging modelini saklama süresi ephemeral_artefact_retention_days: 90 # uyumluluk paketleri, denetim raporları raw_documents_retention_days: 90 # ingest edilen PDF/DOCX/EPUB/TXT/Markdown - enforce: "log_only" # log_only | warn_on_excess | block_on_excess + enforce: "log_only" # ayrılmış: henüz hiçbir komut onu kullanmaz (log_only | warn_on_excess | block_on_excess) ``` ## `pipeline:` diff --git a/forgelm/cli/_config_load.py b/forgelm/cli/_config_load.py index 4e5ac477..111b4932 100644 --- a/forgelm/cli/_config_load.py +++ b/forgelm/cli/_config_load.py @@ -14,6 +14,22 @@ from ._logging import logger +def _warn_reserved_retention_enforce(config) -> None: + """``retention.enforce`` is validated and offered by both wizards, but no command reads it. + + A config that sets ``warn_on_excess`` or ``block_on_excess`` is asking for a gate that does not exist; say so, + rather than accept it silently. (This lives here, not in the schema, to keep ``config.py`` within its + deferred-split line budget.) + """ + mode = getattr(getattr(config, "retention", None), "enforce", "log_only") + if mode != "log_only": + logger.warning( + "retention.enforce=%r is accepted but no command acts on it yet: retention horizons are only reported " + "by `forgelm purge --check-policy`, and the trainer is not warned or blocked.", + mode, + ) + + def _load_config_or_exit(config_path: str, json_output: bool): """Load config and translate exceptions into the right exit code. @@ -25,7 +41,9 @@ def _load_config_or_exit(config_path: str, json_output: bool): """ try: logger.info("Loading configuration from %s...", config_path) - return load_config(config_path) + config = load_config(config_path) + _warn_reserved_retention_enforce(config) + return config except FileNotFoundError as e: msg = f"Config file not found: {e}" except ConfigError as e: diff --git a/forgelm/cli/subcommands/_audit.py b/forgelm/cli/subcommands/_audit.py index d9b4f4a1..b4e17ff7 100644 --- a/forgelm/cli/subcommands/_audit.py +++ b/forgelm/cli/subcommands/_audit.py @@ -55,8 +55,7 @@ def _exit_on_critical_secrets(verdict: dict, *, allow_secrets: bool, defer_exit: chain, GGUF sidecar); training data was never signed. - ``3`` fits exactly: an evaluation gate examined an artefact and said no. This is the same semantics as ``forgelm safety-eval``'s - non-passing branch (``subcommands/_safety_eval.py``) and the - retention ``block_on_excess`` gate. A new public exit code is a + non-passing branch (``subcommands/_safety_eval.py``). A new public exit code is a permanent contract addition and this incident does not need one. ``allow_secrets`` is an explicit per-invocation escape hatch for the diff --git a/forgelm/config.py b/forgelm/config.py index 4114f2df..17876b9b 100644 --- a/forgelm/config.py +++ b/forgelm/config.py @@ -1598,12 +1598,12 @@ class RetentionConfig(BaseModel): days (typical ingestion-window before re-running data audit). Setting any horizon to ``0`` disables the policy for that artefact - kind (retain indefinitely). ``enforce`` controls how the trainer - pre-flight gate reacts to violations: ``log_only`` records a - notice, ``warn_on_excess`` emits a structured warning, and - ``block_on_excess`` aborts training with EXIT_EVAL_FAILURE so a - regulated CI cannot accidentally extend the retention horizon by - re-using a stale workspace. + kind (retain indefinitely). The horizons are *reported*, not enforced: + ``forgelm purge --check-policy`` lists artefacts past them and always + exits 0; nothing is deleted, blocked or scheduled automatically. + ``enforce`` is reserved: no command acts on it yet, so + ``warn_on_excess`` and ``block_on_excess`` behave like ``log_only`` + (the CLI logs a warning when a config sets either). """ model_config = ConfigDict(extra="forbid") @@ -1645,9 +1645,9 @@ class RetentionConfig(BaseModel): enforce: Literal["log_only", "warn_on_excess", "block_on_excess"] = Field( default="log_only", description=( - "Policy enforcement mode. `log_only` records violations in the audit log without operator-visible output; " - "`warn_on_excess` adds a structured warning to stderr; `block_on_excess` aborts the trainer pre-flight with " - "EXIT_EVAL_FAILURE (3) so a regulated CI gate does not silently extend the retention horizon." + "Reserved: no command reads it yet. Retention horizons are only reported, read-only, by `forgelm purge " + "--check-policy` (always exit 0); nothing is deleted, warned about or blocked automatically. " + "`warn_on_excess` and `block_on_excess` are accepted but behave like `log_only`; the CLI warns when set." ), ) diff --git a/forgelm/wizard/_collectors.py b/forgelm/wizard/_collectors.py index 83c0039a..d7574192 100644 --- a/forgelm/wizard/_collectors.py +++ b/forgelm/wizard/_collectors.py @@ -855,11 +855,11 @@ def _collect_retention() -> Optional[Dict[str, Any]]: ) raw_docs = _prompt_int("raw_documents_retention_days (0 = retain indefinitely)", 90, min_val=0, max_val=3650) enforce = _prompt_choice( - "enforcement mode:", + "enforcement mode (reserved: no command acts on it yet; `forgelm purge --check-policy` only reports):", [ - "log_only (record violations only)", - "warn_on_excess (stderr warning, run continues)", - "block_on_excess (abort trainer with EXIT_EVAL_FAILURE)", + "log_only (default; behaves the same as the other two for now)", + "warn_on_excess (accepted, behaves like log_only)", + "block_on_excess (accepted, behaves like log_only)", ], default=1, ) diff --git a/tests/test_cli.py b/tests/test_cli.py index a9765895..85310503 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -319,3 +319,45 @@ def test_facade_reexports_every_subcommand_registrar(self): ] for name in registrars: assert callable(getattr(cli, name, None)), f"{name} is not re-exported from forgelm.cli" + + +class TestReservedRetentionEnforceWarning: + """`retention.enforce` is accepted but no command reads it; the CLI says so (#92).""" + + @staticmethod + def _config(enforce): + from forgelm.config import ForgeConfig + from tests._helpers.factories import minimal_config + + return ForgeConfig(**minimal_config(retention={"enforce": enforce})) + + def test_the_default_is_silent(self, caplog): + import logging + + from forgelm.cli._config_load import _warn_reserved_retention_enforce + + with caplog.at_level(logging.WARNING): + _warn_reserved_retention_enforce(self._config("log_only")) + assert not [r for r in caplog.records if "retention.enforce" in r.getMessage()] + + @pytest.mark.parametrize("mode", ["warn_on_excess", "block_on_excess"]) + def test_a_gate_that_does_not_exist_is_called_out(self, caplog, mode): + import logging + + from forgelm.cli._config_load import _warn_reserved_retention_enforce + + with caplog.at_level(logging.WARNING): + _warn_reserved_retention_enforce(self._config(mode)) + warned = [r.getMessage() for r in caplog.records if "retention.enforce" in r.getMessage()] + assert len(warned) == 1 and mode in warned[0] and "no command acts on it" in warned[0] + + def test_loading_a_config_file_triggers_it(self, tmp_path, caplog, minimal_config): + import logging + + from forgelm.cli._config_load import _load_config_or_exit + + path = tmp_path / "c.yaml" + path.write_text(yaml.dump(minimal_config(retention={"enforce": "block_on_excess"}))) + with caplog.at_level(logging.WARNING): + _load_config_or_exit(str(path), json_output=False) + assert any("retention.enforce" in r.getMessage() for r in caplog.records) diff --git a/tests/test_config.py b/tests/test_config.py index 10b81fd1..70296912 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -935,6 +935,17 @@ def test_legacy_evaluation_staging_ttl_days_is_rejected(self): with pytest.raises(ValidationError, match="staging_ttl_days"): ForgeConfig(**minimal_config(evaluation={"staging_ttl_days": 14})) + +class TestRetentionConfig: + """`retention` horizons are reported, not enforced; `enforce` is reserved (#92).""" + + def test_the_description_does_not_promise_a_gate(self): + from forgelm.config import RetentionConfig + + description = RetentionConfig.model_fields["enforce"].description or "" + assert "no command reads it" in description + assert "EXIT_EVAL_FAILURE" not in description and "aborts" not in description + def test_canonical_retention_staging_ttl_days_still_works(self): cfg = ForgeConfig(**minimal_config(retention={"staging_ttl_days": 30})) assert cfg.retention.staging_ttl_days == 30