Site (and README) present exit 3 / artefact discard on gate failure as unconditional; auto_revert defaults to false, so the default run keeps the model and exits 0 (or 4 with appr… (CR-A18-005) #360

Wave 1 · theme D1-11: Other high-severity packages · highest severity High · 1 finding(s) · effort small

Summary

An operator copies the index.html 'real config' into CI, expecting a benchmark average below 0.65 to fail the job with exit 3. The run instead exits 4 (or 0 without approval) with the model kept, so a CI gate wired to exit 3 never fires.

Primary finding

CR-A18-005 — Site (and README) present exit 3 / artefact discard on gate failure as unconditional; auto_revert defaults to false, so the default run keeps the model and exits 0 (or 4 with approval)

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN DOCS-02/14/15/16 (S15) — docs/roadmap/phase-16-trust-surface-hardening.md:776-778

What happens. home.why.autorevert.body (index.html:384, translations.js:366), the demo comment at index.html:575 ('run fails (exit 3) below this'), compliance.art15.body (compliance.html:184, translations.js:3865 + locales) and README.md:106 state exit 3 without the auto_revert condition. EvaluationConfig.auto_revert defaults to False (config.py:1191-1194); trainer gates log _log_gate_kept_no_revert and return True (trainer.py:1405-1408), cli/_training.py:158-161 exits 0, or 4 when require_human_approval stages the model (as the demo YAML sets).

Where.

• site/index.html:383
• site/index.html:575 (confirmed)
• site/compliance.html:184 (confirmed)
• site/js/translations.js:366 (+472,578,684,790,896)
• site/js/translations.js:3865 (+3924,3983,4026,4069,4112)
• forgelm/config.py:1191-1194
• forgelm/trainer.py:1402-1408 (benchmark kept-no-revert)
• forgelm/cli/_training.py:159-161
• docs/product_strategy.md:22-24 (additional site; nonexistent field auto_revert_on_safety_failure)
• docs/product_strategy.md:24

Evidence.

• fpy -m forgelm --config demo.yaml --dry-run (demo.yaml extracted from index.html:551-588) -> EXIT=0, 'auto_revert: False', 'webhook_configured: True'.
• config.py:1191 auto_revert default=False; trainer.py:1405-1408 'if not auto_revert: _log_gate_kept_no_revert(...); return True'; cli/_training.py:158-161 exit 0 on success. Homepage demo extracted and dry-run: valid, no auto_revert present.
• unsafe_noar exit 0; loss_noar exit 0; judge_noar exit 0; approval_unsafe_noar (the site's snippet shape: safety on, approval on, no auto_revert) exit 4 with safety.evaluation_completed passed=False.

Recommended fix. Qualify every claim with 'with evaluation.auto_revert: true' (as features.html does), add auto_revert: true to the demo YAML, and fix product_strategy.md and the introduction diagram.

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}