Skip to content

Wizard presents retention.enforce and monitoring.* as active enforcement and hooks, but no runtime code reads them (CR-A18-003) #98

Open

Description

@cemililik

Wave 0 · theme D0-5: Documented controls that are not implemented · highest severity High · 1 finding(s) · effort small

Summary

The retention accordion is labelled 'Enforce automatic deletion', with the hint 'When set, ForgeLM auto-deletes artefacts past their horizon' and an option 'block_on_excess — abort trainer (regulated CI)' (translations.js:1819-1823; wizard.js:2912-2936). A repo-wide search finds no reader of retention.enforce: the only hit is the CLI wizard writing it (wizard/_collectors.py:877). The retention horizons are read only by the read-only forgelm purge --check-policy report (_purge.py:1141-1221), and nothing auto-deletes or blocks the trainer. The monitoring block is marked locked and 'Required for high-risk', and its copy says ForgeLM 'wires the right hooks into the training run' and will 'Emit a webhook alert when the drift detector flags a regression' (translations.js:1786, ~1796). In fact monitoring.* is only copied into the training manifest (compliance.py:1582-1589). There is no metrics export, no endpoint call and no drift detector. A regulated CI that picks block_on_excess in the wizard gets no gate. This breaks STD-020 (no 'Ready' claims for unimplemented paths).

Primary finding

CR-A18-003 — Wizard presents retention.enforce and monitoring.* as active enforcement and hooks, but no runtime code reads them

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace)

What happens. The retention accordion is labelled 'Enforce automatic deletion', with the hint 'When set, ForgeLM auto-deletes artefacts past their horizon' and an option 'block_on_excess — abort trainer (regulated CI)' (translations.js:1819-1823; wizard.js:2912-2936). A repo-wide search finds no reader of retention.enforce: the only hit is the CLI wizard writing it (wizard/_collectors.py:877). The retention horizons are read only by the read-only forgelm purge --check-policy report (_purge.py:1141-1221), and nothing auto-deletes or blocks the trainer. The monitoring block is marked locked and 'Required for high-risk', and its copy says ForgeLM 'wires the right hooks into the training run' and will 'Emit a webhook alert when the drift detector flags a regression' (translations.js:1786, ~1796). In fact monitoring.* is only copied into the training manifest (compliance.py:1582-1589). There is no metrics export, no endpoint call and no drift detector. A regulated CI that picks block_on_excess in the wizard gets no gate. This breaks STD-020 (no 'Ready' claims for unimplemented paths).

Where.

Evidence.

  • rg '.enforce\b|"enforce"' forgelm/ -> only wizard/_collectors.py:877. rg alert_on_drift|metrics_export|check_interval_hours|endpoint_env outside config.py/wizard -> only compliance.py:1582-1589.

Recommended fix. Reword the wizard copy to describe these fields as recorded metadata and report-only policy, or implement the behaviour (trainer pre-flight on enforce; monitoring hooks) before advertising it.

Acceptance criteria

  • A regression test reproduces the failure scenario and asserts the corrected behaviour.
  • Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
  • Every finding in this issue is resolved or explicitly closed with a reason.
  • The release note names the contract this fix restores.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: siteCode area: sitebugSomething isn't workingseverity: highWrong behaviour or a false claim users rely onsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 0Roadmap wave 0: immediate (patch release)

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions