Repository navigation
[Triage] docs/usermanuals/en/evaluation/safety.md: false denial — Guard-verdict scoring is unbatched ... Under classifier_mode: generation (the default) (DOC-D3B08-096) #833
Description
Needs triage. Reviewers disagreed on this finding, so it is not counted in the backlog. Decide whether it is a real defect: if yes, set severity and wave; if not, close it with the reason.
DOC-D3B08-096 — docs/usermanuals/en/evaluation/safety.md: false denial — Guard-verdict scoring is unbatched ... Under classifier_mode: generation (the default)
severity High · confidence disputed
Where. docs/usermanuals/en/evaluation/safety.md:231 · type: false denial
The document says. Guard-verdict scoring is unbatched ... Under classifier_mode: generation (the default)
What the code does. Scoring is indeed sequential per prompt (forgelm/safety/_score_generation.py:287-300). The default classifier_mode is 'auto', though, not 'generation' (config.py:917).
Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.
Activity
cemililik commentedon Oct 20, 2026
The factual part is correct: the guard pass runs one model.generate call per prompt (forgelm/safety/_score_generation.py:286-300) and batch_size only batches the probe-response generation. The only imprecision is calling
classifier_mode: generation'the default': the field default isauto, which resolves to generation for the default meta-llama/Llama-Guard-3-8B checkpoint.The same wording (lines 47, 188, 190, 207, 214 and 231, plus the Turkish page) is already tracked in #761 as DOC-D3B08-087; this finding (DOC-D3B08-096) is recorded there too. Closing as a duplicate of #761.
Triage round 1, checked against
developmenton 2026-10-20.