safety-eval for GGUF artifacts: evaluate the quantized file that is actually deployed #1031

Embeddable engine · EMB-04 · Kind: code + docs · Size L · background: #1027

Summary

forgelm safety-eval refuses a .gguf file, and the training-time safety gate evaluates the Hugging Face checkpoint before export. The quantized GGUF that people actually run in llama.cpp, Ollama or LM Studio therefore never passes a ForgeLM safety gate, and ForgeLM can attest its integrity (verify-gguf, SHA-256) but not its behaviour. This issue adds a generation backend for GGUF files so the same probes, classifier and gates can evaluate the artefact that is actually deployed.

Why evaluating the deployed artefact matters

Published results show that quantization can change refusal behaviour, and that this depends on the model and on the build. Both papers are reported as their authors describe them; the findings were not reproduced for this issue.

• Quality Is Not a Safety Proxy Under Quantization (arXiv:2606.10154, 6 models from 4 families, a 7-level GGUF ladder plus AWQ/GPTQ INT4): rows that kept stable or better quality while refusal fell by 12 to 68 percentage points; the authors conclude that "retained quality cannot waive direct safety evaluation".
• The Effect of Quantization on Clinical Benchmarks (arXiv:2609.22216, five 7–8B models): INT8 was safe in their tests, INT4 safety effects were substantial and model-dependent (for example −17.8% and −28.3% on two models).

Two ForgeLM facts make this concrete. First, forgelm export automates only f16 and q8_0; K-quants such as q4_k_m need a manual llama-quantize step (export.py#L37-L53), so most deployed GGUFs are produced outside ForgeLM and the checker must accept any GGUF, not only ones ForgeLM wrote. Second, the project's compliance claim for Article 15 (accuracy and robustness) is about the deployed model.

What exists today (2059b59)

• _load_model_for_safety, cli/subcommands/_safety_eval.py#L100-L180 refuses .gguf with exit 1 (L119-L127). The check is case-sensitive (CR-A06-071 in [Wave 4] Low/info — Module sweep: Evaluation gates: safety, LLM-as-judge, benchmarks, safety-eval (part 2/3) #677), and the message and safety_eval_subcommand.md#L67 promise a "Phase 36+ extension" that is not on the roadmap.
• forgelm.inference.load_model loads only Hugging Face and Unsloth models; forgelm chat has the same limit (docs/usermanuals/en/deployment/chat.md: false claim — forgelm chat accepts 'A GGUF file: ./model.gguf (uses llama.cpp under the hood)'. (DOC-D3B08-003) #258).
• run_safety_evaluation(model, tokenizer, …) takes live Hugging Face objects, and generation is hard-wired to them: tokenizer(batch, …) and model.generate(…, do_sample=False) (safety/_generate.py#L56-L75). There is no backend abstraction. The harm classifier is a separate Hugging Face model and can stay as it is.
• Prompt construction. The probes are tokenised as raw text with no chat template (same lines). A GGUF backend must send the same raw text, or the Hugging Face-versus-GGUF comparison would measure template differences instead of quantization. Whether probes should be wrapped in the chat template is a separate, pre-existing question; do not change the Hugging Face behaviour here without its own decision record.

Loader options (measure first, then choose)

Proposal. (1) A spike on two small models comparing A and C: agreement of refusal verdicts with a llama.cpp reference run, peak memory, wall time. (2) Introduce a small GenerationBackend protocol in a new module (forgelm/safety/_backends.py): generate(prompts, max_new_tokens) -> list[GeneratedResponse] and close(), with the Hugging Face path as the default implementation, behaviour-neutral. (3) Implement the backend the spike supports; A first if it passes (no dependency), C as a follow-up.

Output and evidence

Reuse the existing per-category breakdown and gates. Add model_format: "gguf" and gguf: {path, sha256, architecture, quant_type} (read from the GGUF header) to the result and to the safety.evaluation_completed event so the evidence binds to the exact file; extend docs/reference/audit_event_catalog.md (a CI guard checks it in both directions). Run the structural verify_gguf check first. Exit codes keep their meaning: an unsupported architecture is a config error (1) with an actionable message, a load failure is 2, a failed gate is 3.

Out of scope

forgelm chat on GGUF (#258, same loader, separate issue); changing the Hugging Face prompt policy; evaluating the classifier as GGUF; running GGUF inside the training-time gate (the training gate keeps evaluating the checkpoint it just trained).

Acceptance criteria

• A spike note (committed as a dated decision row) reports A versus C on at least two models: verdict agreement, memory, time, and the supported architecture list; it names the chosen backend.
• GenerationBackend exists in a new module; the Hugging Face implementation produces byte-identical results to today (existing safety tests pass untouched).
• The chosen GGUF backend runs forgelm safety-eval --model x.gguf and the library run_safety_evaluation path; raw probe text is used on both backends and a test pins it.
• The .gguf refusal is replaced by real support, matched case-insensitively (closes CR-A06-071 in [Wave 4] Low/info — Module sweep: Evaluation gates: safety, LLM-as-judge, benchmarks, safety-eval (part 2/3) #677), and the "Phase 36+" wording is gone from the message and the documentation.
• Results and the audit event record model_format, the file's SHA-256, architecture and quantization type; the catalog and the JSON output reference describe them.
• An unsupported architecture, a corrupt file and an out-of-memory load each map to the documented exit code with an actionable message.
• Tests are CPU-only and need no download: a tiny GGUF fixture is generated at test time (the gguf package can write one) or an existing tiny model is converted; the test module follows the one-test-file-per-module rule.
• Documentation in English and Turkish: safety_eval_subcommand.md, the safety usermanual page, the JSON output reference and a CHANGELOG Added entry.

Sequencing and dependencies

The spike can start now (it changes nothing in the repository); implement in Stage C of #1027, after Phase 16 S5 (#891), because S5 redefines verification outcomes and the GGUF result must use the same vocabulary. The GenerationBackend refactor touches forgelm/safety/, which no remaining step owns.

Related

#744 (documentation: safety-eval reference), #677 (CR-A06-071), #258 (chat and GGUF), #891 (S5), #676 and #677 (module sweeps for the evaluation gates).

_{Raised on 2026-10-20 as part of the embeddable-engine decision. Code links point at 2059b59; line numbers may have moved. Package facts (transformers GGUF loader, llama-cpp-python distribution) were checked against the vendor documentation and PyPI on 2026-10-20. Tracking ID: EMB-04.}