Skip to content

docs/roadmap/completed-phases.md: false claim — The per-turn Llama Guard screen is planned; 'today operators get the same screening by enabling safety: enabled: true in the YAML config the chat REP… (DOC-D2B07-001) #91

Open

Description

@cemililik

Wave 0 · theme D0-5: Documented controls that are not implemented · highest severity Critical · 1 finding(s) · effort docs fix

Summary

Operators who demo or hand off a model through forgelm chat expect Llama Guard screening on every turn. No output is screened.

Primary finding

DOC-D2B07-001 — docs/roadmap/completed-phases.md: false claim — The per-turn Llama Guard screen is planned; 'today operators get the same screening by enabling safety: enabled: true in the YAML config the chat REP…

severity Critical

Where. docs/roadmap/completed-phases.md:669-675 · type: false claim

The document says. The per-turn Llama Guard screen is planned; 'today operators get the same screening by enabling safety: enabled: true in the YAML config the chat REPL reads.'

What the code does. forgelm chat takes a positional model_path plus flags (--adapter, --system, --temperature, --max-new-tokens, --no-stream, --load-in-4bit/8bit, --trust-remote-code, --backend). It has no --config and reads no YAML. forgelm/chat.py contains no safety or classifier code, so chat replies are never screened.

Code references. forgelm/cli/_parser.py:35-67, forgelm/chat.py:150-156, forgelm/chat.py:253, forgelm/chat.py:358

User impact. Operators who demo or hand off a model through forgelm chat expect Llama Guard screening on every turn. No output is screened.

Proposed correction. Replace the sentence with: "forgelm chat performs no safety screening today; it does not read a YAML config. Screen a model with forgelm safety-eval --model <path> --default-probes (or the evaluation.safety block during training) before exposing it."

Acceptance criteria

  • Each statement is re-checked against the code at fix time.
  • Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
  • Every finding in this issue is resolved or explicitly closed with a reason.
  • The release note names the contract this fix restores.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationseverity: criticalBreaks a stated security or safety contractsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 0Roadmap wave 0: immediate (patch release)

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions