Troubleshooting page points to a nonexistent evaluation.safety.baseline and describes the safety gate as baseline-relative (CR-A01-017) #426

Wave 1 · theme D1-8: Docs describing features that do not exist (doc side of code findings) · highest severity High · 1 finding(s) · effort trivial

Summary

An operator whose base model already fails many probes reads this advice, expects the gate to tolerate an unchanged absolute unsafe rate, and adds evaluation.safety.baseline. The config is rejected (exit 1). Without the key, the gate fails or reverts on the absolute ratio, which is the opposite of what the page told them.

Primary finding

CR-A01-017 — Troubleshooting page points to a nonexistent evaluation.safety.baseline and describes the safety gate as baseline-relative

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN C31-DOC-SCHEMA-PROSE — docs/roadmap/phase-16-trust-surface-hardening.md:785-786 (S15 open)

What happens. troubleshooting.md:130 says the safety check that matters is 'regression vs baseline' and that evaluation.safety.baseline should point at the pre-train baseline. No such field exists and no pre-training safety measurement is taken. max_safety_regression is an absolute unsafe-ratio ceiling, as forgelm/safety/init.py:7-11 and _gates.py:257-259 state. The doc misstates the gate's semantics, and the advised key fails validation.

Where.

• docs/usermanuals/en/operations/troubleshooting.md:130
• docs/usermanuals/tr/operations/troubleshooting.md:130
• forgelm/config.py:908-910 (SafetyConfig, extra=forbid, no baseline field)
• forgelm/safety/_gates.py:256-261 (absolute gate, comment states no before/after comparison)
• forgelm/safety/_gates.py:~254-261
• forgelm/safety/__init__.py:7-11
• forgelm/cli/_parser.py:925
• tests/test_safety_advanced.py:203-209
• forgelm/config.py:941-945 (max_safety_regression: absolute unsafe-ratio ceiling)
• docs/usermanuals/_meta.yaml:465 (troubleshooting page is in the shipped manual navigation)

Evidence.

• The dry-run gives EXIT=1, 'evaluation.safety.baseline Extra inputs are not permitted'. tests/test_safety_advanced.py:203-209 asserts that no baseline function exists.

Recommended fix. Rewrite the 'Llama Guard always reports high severity' fix to describe the absolute max_safety_regression and severity_thresholds semantics. Drop the baseline key in EN and TR.

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}