Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,11 @@ All notable changes to ForgeLM are documented here.

### Fixed

- `retention.enforce` is documented for what it is: reserved. No command reads it, so `warn_on_excess` and
`block_on_excess` never warned or blocked anything and nothing was deleted automatically; retention is only reported,
read-only, by `forgelm purge --check-policy` (always exit 0). The config description, the wizard prompt, the
configuration reference, the GDPR erasure guide and `config_template.yaml` now say so (English and Turkish), and
the CLI logs a warning when a config sets a non-default `enforce` instead of staying silent (#92, #98).
- Fix the KTO bullet on the Alignment overview user-manual page (English and Turkish): it named the
fields `(prompt, response, good?)`, but the KTO loader reads `prompt`, `completion` and a boolean
`label` (#965).
Expand Down
2 changes: 1 addition & 1 deletion config_template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,7 @@ data:
# staging_ttl_days: 7 # final_model.staging.<run_id>/ TTL after `forgelm reject`
# ephemeral_artefact_retention_days: 90 # compliance bundles + audit reports
# raw_documents_retention_days: 90 # ingested raw docs (PDF/DOCX/EPUB/TXT/MD)
# enforce: "log_only" # "log_only" | "warn_on_excess" | "block_on_excess"
# enforce: "log_only" # reserved: no command acts on it yet ("log_only" | "warn_on_excess" | "block_on_excess")

# Phase 14 — Multi-stage training pipeline chains (SFT → DPO → GRPO).
# Uncomment the ``pipeline:`` block below to run multiple training stages
Expand Down
4 changes: 2 additions & 2 deletions docs/guides/gdpr_erasure-tr.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,10 @@ retention:
staging_ttl_days: 7 # `forgelm reject` üzerine aksiyon almak için bir iş haftası
ephemeral_artefact_retention_days: 90 # üç-aylık review cadence'i
raw_documents_retention_days: 90 # data audit yeniden çalıştırmadan önce ingestion penceresi
enforce: log_only # log_only / warn_on_excess / block_on_excess
enforce: log_only # ayrılmış: henüz hiçbir komut onu kullanmaz (log_only / warn_on_excess / block_on_excess)
```

Herhangi bir horizon'u `0` olarak ayarlamak o artefact kind'ı için policy'yi devre dışı bırakır (sınırsız tut). `enforce`, trainer pre-flight gate'inin ihlallere nasıl tepki vereceğini kontrol eder.
Herhangi bir horizon'u `0` olarak ayarlamak o artefact kind'ı için policy'yi devre dışı bırakır (sınırsız tut). Horizon'lar `forgelm purge --check-policy` ile raporlanır (salt-okunur, her zaman 0 ile çıkar); hiçbir şey otomatik silinmez ya da engellenmez. `enforce` ayrılmıştır: henüz hiçbir komut onu kullanmaz, bu yüzden üç değer de `log_only` gibi davranır.

## Kaldırıldı: `evaluation.staging_ttl_days`

Expand Down
4 changes: 2 additions & 2 deletions docs/guides/gdpr_erasure.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,10 @@ retention:
staging_ttl_days: 7 # one work-week to act on a `forgelm reject`
ephemeral_artefact_retention_days: 90 # quarterly review cadence
raw_documents_retention_days: 90 # ingestion-window before re-running data audit
enforce: log_only # log_only / warn_on_excess / block_on_excess
enforce: log_only # reserved: no command acts on it yet (log_only / warn_on_excess / block_on_excess)
```

Setting any horizon to `0` disables the policy for that artefact kind (retain indefinitely). `enforce` controls how the trainer pre-flight gate reacts to violations.
Setting any horizon to `0` disables the policy for that artefact kind (retain indefinitely). The horizons are reported by `forgelm purge --check-policy` (read-only, always exits 0); nothing is deleted or blocked automatically. `enforce` is reserved: no command acts on it yet, so all three values behave like `log_only`.

## Removed: `evaluation.staging_ttl_days`

Expand Down
10 changes: 5 additions & 5 deletions docs/reference/configuration-tr.md
Original file line number Diff line number Diff line change
Expand Up @@ -270,18 +270,18 @@ training:

Uyumluluk, eğitim ve değerlendirme artefaktları için saklama ufuklarını
belirler. Ufuklar GDPR Madde 5(1)(e) "saklama sınırlaması" ve Madde 17
"silme hakkı" tarihlerini onurlandırır. `enforce` anahtarı yalnız-loglama,
uyarı ve sert-engelleme modları arasında geçiş yaparak regüle edilen bir CI
kapısının saklama ufkunu eski bir çalışma alanını yeniden kullanarak sessizce
uzatmasını engeller.
"silme hakkı" tarihlerini onurlandırır. Ufuklar **raporlanır**, zorlanmaz:
`forgelm purge --check-policy` ufku aşanları listeler ve her zaman 0 ile çıkar;
hiçbir şey otomatik silinmez, engellenmez ya da zamanlanmaz. `enforce` anahtarı
ayrılmıştır: henüz hiçbir komut onu okumaz.

| Alan | Tip | Varsayılan | Açıklama |
|------|-----|-----------|----------|
| `audit_log_retention_days` | int | `1825` (~5 yıl) | `audit_log.jsonl` dosyasının Madde 5(1)(e) kapsamında "geciken" olarak işaretlenmeden önce saklanacağı gün sayısı. `0` süresiz saklamayı belirtir (Madde 17(3)(b) savunması). |
| `staging_ttl_days` | int | `7` | `forgelm reject` kararından sonra `final_model.staging.<run_id>/` dizininin planlı temizlenmeden önce saklanacağı gün sayısı. `0` süresiz saklama anlamına gelir. v0.8.0'da kaldırılan `evaluation.staging_ttl_days` yerine geçer. |
| `ephemeral_artefact_retention_days` | int | `90` | Uyumluluk paketleri, veri denetim raporları ve diğer çalışma kapsamlı türetilmiş artefaktların saklanma süresi (gün). `0` süresiz saklama. |
| `raw_documents_retention_days` | int | `90` | İngest edilmiş ham belgelerin (PDF / DOCX / EPUB / TXT / Markdown) operatörün ingestion-output dizininde saklanma süresi (gün). `0` süresiz saklama. |
| `enforce` | string | `"log_only"` | Politika uygulama modu: `"log_only"` (yalnızca audit log), `"warn_on_excess"` (stderr'e yapılandırılmış uyarı), `"block_on_excess"` (`EXIT_EVAL_FAILURE` = 3 ile trainer ön-kontrolünü iptal eder). |
| `enforce` | string | `"log_only"` | Ayrılmış politika modu. Henüz hiçbir komut onu okumaz: `"log_only"`, `"warn_on_excess"` ve `"block_on_excess"` kabul edilir ve aynı davranır; bir config son ikisinden birini ayarlarsa CLI uyarır. |

> **Kaldırıldı:** `evaluation.staging_ttl_days` (v0.5.5 itibarıyla kullanımdan
> kaldırılmıştı) v0.8.0'da kaldırılmıştır. Artık geçerli tek form
Expand Down
9 changes: 5 additions & 4 deletions docs/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -270,17 +270,18 @@ across retries. Each retry attempt is logged to the audit trail.

Defines maximum retention horizons for compliance, training, and evaluation
artefacts. Horizons honour GDPR Article 5(1)(e) "storage limitation" and
Article 17 "right to erasure" deadlines. The `enforce` knob switches between
log-only, warning, and hard-block modes so a regulated CI gate cannot
silently extend the retention horizon by re-using a stale workspace.
Article 17 "right to erasure" deadlines. The horizons are **reported**, not
enforced: `forgelm purge --check-policy` lists what is past them and always
exits 0, and nothing is deleted, blocked or scheduled automatically. The
`enforce` knob is reserved: no command reads it yet.

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `audit_log_retention_days` | int | `1825` (~5 years) | Days to retain `audit_log.jsonl` before flagging it as overdue under Article 5(1)(e). Set to `0` to retain indefinitely (Article 17(3)(b) defence). |
| `staging_ttl_days` | int | `7` | Days to retain `final_model.staging.<run_id>/` after a `forgelm reject` decision before scheduled cleanup. Set to `0` to retain indefinitely. Replaces the removed `evaluation.staging_ttl_days` (removed in v0.8.0). |
| `ephemeral_artefact_retention_days` | int | `90` | Days to retain compliance bundles, data audit reports, and other run-scoped derived artefacts. Set to `0` to retain indefinitely. |
| `raw_documents_retention_days` | int | `90` | Days to retain ingested raw documents (PDF / DOCX / EPUB / TXT / Markdown) under the operator's ingestion-output directory. Set to `0` to retain indefinitely. |
| `enforce` | string | `"log_only"` | Policy enforcement mode: `"log_only"` (audit-log only), `"warn_on_excess"` (structured stderr warning), `"block_on_excess"` (abort trainer pre-flight with `EXIT_EVAL_FAILURE` = 3). |
| `enforce` | string | `"log_only"` | Reserved policy mode. No command reads it yet: `"log_only"`, `"warn_on_excess"` and `"block_on_excess"` are all accepted and behave the same, and the CLI warns when a config sets either of the last two. |

> **Removed:** `evaluation.staging_ttl_days` (deprecated as of v0.5.5) was
> removed in v0.8.0. `retention.staging_ttl_days` is now the only accepted form.
Expand Down
2 changes: 1 addition & 1 deletion docs/usermanuals/en/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -363,7 +363,7 @@ retention:
staging_ttl_days: 7 # days to retain staging model after forgelm reject
ephemeral_artefact_retention_days: 90 # compliance bundles, audit reports
raw_documents_retention_days: 90 # ingested PDF/DOCX/EPUB/TXT/Markdown
enforce: "log_only" # log_only | warn_on_excess | block_on_excess
enforce: "log_only" # reserved: no command acts on it yet (log_only | warn_on_excess | block_on_excess)
```

## `pipeline:`
Expand Down
2 changes: 1 addition & 1 deletion docs/usermanuals/tr/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -371,7 +371,7 @@ retention:
staging_ttl_days: 7 # forgelm reject sonrası staging modelini saklama süresi
ephemeral_artefact_retention_days: 90 # uyumluluk paketleri, denetim raporları
raw_documents_retention_days: 90 # ingest edilen PDF/DOCX/EPUB/TXT/Markdown
enforce: "log_only" # log_only | warn_on_excess | block_on_excess
enforce: "log_only" # ayrılmış: henüz hiçbir komut onu kullanmaz (log_only | warn_on_excess | block_on_excess)
```

## `pipeline:`
Expand Down
20 changes: 19 additions & 1 deletion forgelm/cli/_config_load.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,22 @@
from ._logging import logger


def _warn_reserved_retention_enforce(config) -> None:
"""``retention.enforce`` is validated and offered by both wizards, but no command reads it.

A config that sets ``warn_on_excess`` or ``block_on_excess`` is asking for a gate that does not exist; say so,
rather than accept it silently. (This lives here, not in the schema, to keep ``config.py`` within its
deferred-split line budget.)
"""
mode = getattr(getattr(config, "retention", None), "enforce", "log_only")
if mode != "log_only":
logger.warning(
"retention.enforce=%r is accepted but no command acts on it yet: retention horizons are only reported "
"by `forgelm purge --check-policy`, and the trainer is not warned or blocked.",
mode,
)


def _load_config_or_exit(config_path: str, json_output: bool):
"""Load config and translate exceptions into the right exit code.

Expand All @@ -25,7 +41,9 @@ def _load_config_or_exit(config_path: str, json_output: bool):
"""
try:
logger.info("Loading configuration from %s...", config_path)
return load_config(config_path)
config = load_config(config_path)
_warn_reserved_retention_enforce(config)
return config
except FileNotFoundError as e:
msg = f"Config file not found: {e}"
except ConfigError as e:
Expand Down
3 changes: 1 addition & 2 deletions forgelm/cli/subcommands/_audit.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,7 @@ def _exit_on_critical_secrets(verdict: dict, *, allow_secrets: bool, defer_exit:
chain, GGUF sidecar); training data was never signed.
- ``3`` fits exactly: an evaluation gate examined an artefact and said
no. This is the same semantics as ``forgelm safety-eval``'s
non-passing branch (``subcommands/_safety_eval.py``) and the
retention ``block_on_excess`` gate. A new public exit code is a
non-passing branch (``subcommands/_safety_eval.py``). A new public exit code is a
permanent contract addition and this incident does not need one.

``allow_secrets`` is an explicit per-invocation escape hatch for the
Expand Down
18 changes: 9 additions & 9 deletions forgelm/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -1598,12 +1598,12 @@ class RetentionConfig(BaseModel):
days (typical ingestion-window before re-running data audit).

Setting any horizon to ``0`` disables the policy for that artefact
kind (retain indefinitely). ``enforce`` controls how the trainer
pre-flight gate reacts to violations: ``log_only`` records a
notice, ``warn_on_excess`` emits a structured warning, and
``block_on_excess`` aborts training with EXIT_EVAL_FAILURE so a
regulated CI cannot accidentally extend the retention horizon by
re-using a stale workspace.
kind (retain indefinitely). The horizons are *reported*, not enforced:
``forgelm purge --check-policy`` lists artefacts past them and always
exits 0; nothing is deleted, blocked or scheduled automatically.
``enforce`` is reserved: no command acts on it yet, so
``warn_on_excess`` and ``block_on_excess`` behave like ``log_only``
(the CLI logs a warning when a config sets either).
"""

model_config = ConfigDict(extra="forbid")
Expand Down Expand Up @@ -1645,9 +1645,9 @@ class RetentionConfig(BaseModel):
enforce: Literal["log_only", "warn_on_excess", "block_on_excess"] = Field(
default="log_only",
description=(
"Policy enforcement mode. `log_only` records violations in the audit log without operator-visible output; "
"`warn_on_excess` adds a structured warning to stderr; `block_on_excess` aborts the trainer pre-flight with "
"EXIT_EVAL_FAILURE (3) so a regulated CI gate does not silently extend the retention horizon."
"Reserved: no command reads it yet. Retention horizons are only reported, read-only, by `forgelm purge "
"--check-policy` (always exit 0); nothing is deleted, warned about or blocked automatically. "
"`warn_on_excess` and `block_on_excess` are accepted but behave like `log_only`; the CLI warns when set."
),
)

Expand Down
8 changes: 4 additions & 4 deletions forgelm/wizard/_collectors.py
Original file line number Diff line number Diff line change
Expand Up @@ -855,11 +855,11 @@ def _collect_retention() -> Optional[Dict[str, Any]]:
)
raw_docs = _prompt_int("raw_documents_retention_days (0 = retain indefinitely)", 90, min_val=0, max_val=3650)
enforce = _prompt_choice(
"enforcement mode:",
"enforcement mode (reserved: no command acts on it yet; `forgelm purge --check-policy` only reports):",
[
"log_only (record violations only)",
"warn_on_excess (stderr warning, run continues)",
"block_on_excess (abort trainer with EXIT_EVAL_FAILURE)",
"log_only (default; behaves the same as the other two for now)",
"warn_on_excess (accepted, behaves like log_only)",
"block_on_excess (accepted, behaves like log_only)",
],
default=1,
)
Expand Down
42 changes: 42 additions & 0 deletions tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -319,3 +319,45 @@ def test_facade_reexports_every_subcommand_registrar(self):
]
for name in registrars:
assert callable(getattr(cli, name, None)), f"{name} is not re-exported from forgelm.cli"


class TestReservedRetentionEnforceWarning:
"""`retention.enforce` is accepted but no command reads it; the CLI says so (#92)."""

@staticmethod
def _config(enforce):
from forgelm.config import ForgeConfig
from tests._helpers.factories import minimal_config

return ForgeConfig(**minimal_config(retention={"enforce": enforce}))

def test_the_default_is_silent(self, caplog):
import logging

from forgelm.cli._config_load import _warn_reserved_retention_enforce

with caplog.at_level(logging.WARNING):
_warn_reserved_retention_enforce(self._config("log_only"))
assert not [r for r in caplog.records if "retention.enforce" in r.getMessage()]

@pytest.mark.parametrize("mode", ["warn_on_excess", "block_on_excess"])
def test_a_gate_that_does_not_exist_is_called_out(self, caplog, mode):
import logging

from forgelm.cli._config_load import _warn_reserved_retention_enforce

with caplog.at_level(logging.WARNING):
_warn_reserved_retention_enforce(self._config(mode))
warned = [r.getMessage() for r in caplog.records if "retention.enforce" in r.getMessage()]
assert len(warned) == 1 and mode in warned[0] and "no command acts on it" in warned[0]

def test_loading_a_config_file_triggers_it(self, tmp_path, caplog, minimal_config):
import logging

from forgelm.cli._config_load import _load_config_or_exit

path = tmp_path / "c.yaml"
path.write_text(yaml.dump(minimal_config(retention={"enforce": "block_on_excess"})))
with caplog.at_level(logging.WARNING):
_load_config_or_exit(str(path), json_output=False)
assert any("retention.enforce" in r.getMessage() for r in caplog.records)
11 changes: 11 additions & 0 deletions tests/test_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -935,6 +935,17 @@ def test_legacy_evaluation_staging_ttl_days_is_rejected(self):
with pytest.raises(ValidationError, match="staging_ttl_days"):
ForgeConfig(**minimal_config(evaluation={"staging_ttl_days": 14}))


class TestRetentionConfig:
"""`retention` horizons are reported, not enforced; `enforce` is reserved (#92)."""

def test_the_description_does_not_promise_a_gate(self):
from forgelm.config import RetentionConfig

description = RetentionConfig.model_fields["enforce"].description or ""
assert "no command reads it" in description
assert "EXIT_EVAL_FAILURE" not in description and "aborts" not in description

def test_canonical_retention_staging_ttl_days_still_works(self):
cfg = ForgeConfig(**minimal_config(retention={"staging_ttl_days": 30}))
assert cfg.retention.staging_ttl_days == 30
Expand Down
Loading