output_format=prompt_response output cannot be trained with SFT (exit 2, KeyError); the error message points users to GRPO; the manual's headline {prompt, completion} SFT format f… (CR-A08-006) #406

Wave 1 · theme D1-5: Training, merging and export correctness · highest severity High · 1 finding(s) · effort small

Summary

An operator follows synthetic-data.md, picks output_format: prompt_response (or writes SFT data in the manual's {prompt, completion} 'instructions' format), and runs training. The model loads, then the run aborts with exit 2 and a message that says the data is prompt-only and suggests GRPO. The corpus is unusable for SFT without hand-renaming columns.

Primary finding

CR-A08-006 — output_format=prompt_response output cannot be trained with SFT (exit 2, KeyError); the error message points users to GRPO; the manual's headline {prompt, completion} SFT format fails the same way

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end

What happens. synthetic.py:410-412 writes {prompt, response} for output_format=prompt_response. SFT routing never accepts 'prompt' as the user column. process_batch (data.py:483-488) sends rows without text or messages columns to _process_user_assistant_format. That function accepts only 'User' or 'instruction' as the user key (data.py:437, 452), although it does accept 'response' as the assistant key (data.py:438, 453). So has_user is False and data.py:444-450 raises KeyError. The message builds on _detect_dataset_format, whose prompt-only branch (data.py:40-41) comes before the instruction branch (data.py:42-45), so it reports 'Detected format: prompt-only format. Suggested trainer: grpo'. That steers the operator to a trainer that throws the teacher responses away (see the GRPO finding). Nothing in the schema or docs warns about this. config.py:1364-1373 lists prompt_response next to the trainable formats and adds a trainability note only for chatml. EN/TR synthetic-data.md:35, reference/configuration.md:392 (TR :344), usermanuals en/tr reference/configuration.md:188/189 and config_template.yaml:115 all list it without a caveat. The audit package even treats ('prompt','response') as an instruction pair and cites 'the loaders in forgelm.data' (data_audit/_types.py:80-93), so forgelm audit accepts the shape that training rejects. The same root cause breaks the user manual's first SFT format: usermanuals/en/concepts/data-formats.md:14,22-27 (TR :14,22-26) documents 'instructions' = {prompt, completion} for SFT, and :149 says such a row auto-detects as instructions. On the same tiny model that shape also exits 2 with the same KeyError (Found: ['prompt', 'completion']). data_preparation.md:29 does not list 'prompt' as a user column either. The failure is loud, but it comes only afte…

Where.

• forgelm/synthetic.py:411-412
• forgelm/data.py:444-450
• forgelm/data.py:452-453
• forgelm/data.py:39-44
• forgelm/data.py:483-488
• forgelm/config.py:1364-1373
• docs/usermanuals/en/data/synthetic-data.md:35
• docs/usermanuals/tr/data/synthetic-data.md:35
• docs/usermanuals/en/concepts/data-formats.md:22-27
• docs/usermanuals/en/concepts/data-formats.md:25

Evidence.

• Private workdir, snapshot cab2947, tiny-qwen, teacher_backend=file, offline. fpy -m forgelm --config cfg/gen_pr.yaml --generate-data -> exit 0, 12/12 rows {prompt,response}, Turkish row intact. fpy -m forgelm --config cfg/a_sft.yaml --dry-run -> exit 0. fpy -m forgelm --config cfg/a_sft.yaml -> exit 2, log '[ERROR] forgelm.cli: Training pipeline failed.' plus traceback through data.py:733 -> 681 (datasets.map worker): KeyError: "Dataset must contain 'User'/'instruction' and 'Assistant'/'o…

Recommended fix. Accept 'prompt' as a user key and 'completion' as an assistant key in _process_user_assistant_format (and in its error text), and move the prompt/response (prompt/completion) detection ahead of the prompt-only branch in _detect_dataset_format. Alternatively, drop prompt_response or document it as GRPO-only. Add a test that feeds every _format_entry output through prepare_dataset.

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}