docs/usermanuals/en/deployment/model-merging.md: broken example — tasks: ["hellaswag", "humaneval", "gsm8k"] # mix of skills from each specialist (with min_score: 0.5) (DOC-BLEGAL-029) #136

Wave 1 · theme D1-10: Documentation-only fixes · highest severity High · 1 finding(s) · effort docs fix

Summary

A merged model that loses the math or code specialist's skill passes the gate, so the gate verdict silently measures one task.

Primary finding

DOC-BLEGAL-029 — docs/usermanuals/en/deployment/model-merging.md: broken example — tasks: ["hellaswag", "humaneval", "gsm8k"] # mix of skills from each specialist (with min_score: 0.5)

severity High

Where. docs/usermanuals/en/deployment/model-merging.md:88 (Turkish mirror: docs/usermanuals/tr/deployment/model-merging.md:88) · type: broken example

The document says. tasks: ["hellaswag", "humaneval", "gsm8k"] # mix of skills from each specialist (with min_score: 0.5)

What the code does. The lm-eval v0.4.13 result keys are ',': gsm8k gives exact_match,strict-match and exact_match,flexible-extract, and humaneval gives pass@1 with filter create_test and declares unsafe_code: true. None contains 'acc', so ForgeLM drops both tasks (forgelm/benchmark.py:39-62, 83-84) and the merge gate checks min_score against hellaswag alone.

User impact. A merged model that loses the math or code specialist's skill passes the gate, so the gate verdict silently measures one task.

Proposed correction. Use accuracy-reporting tasks (e.g. arc_challenge, mmlu subsets), or state that gsm8k and humaneval are not counted by the min_score gate. TR: aynı uyarı.

Turkish mirror. the Turkish page has the same error; fix both in the same PR

Acceptance criteria

• Each statement is re-checked against the code at fix time.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}