Wizard-generated synthetic block can never run: teacher_backend/api_base are never emitted, so every --generate-data call fails (CR-A18-012) #411

Wave 1 · theme D1-5: Training, merging and export correctness · highest severity High · 2 finding(s) · effort small

Summary

When synthetic is enabled, buildYaml emits only enabled/teacher_model/seed_file/api_key_env (wizard.js:844-851). teacher_backend therefore defaults to 'api' and api_base to '' (config.py:1333-1343). _call_api_teacher raises 'synthetic.api_base is required for API teacher backend.' for every prompt (synthetic.py:218-222). The config validator does not check api_base, so --dry-run passes. The hints offer 'API model name ... or local model path' for the teacher (translations.js:1803) and say 'Empty = local teacher model' for the key env (translations.js:1808), but the wizard never switches teacher_backend to local, and it rejects an empty key env (wizard.js:461-463).

Primary finding

CR-A18-012 — Wizard-generated synthetic block can never run: teacher_backend/api_base are never emitted, so every --generate-data call fails

severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end

What happens. When synthetic is enabled, buildYaml emits only enabled/teacher_model/seed_file/api_key_env (wizard.js:844-851). teacher_backend therefore defaults to 'api' and api_base to '' (config.py:1333-1343). _call_api_teacher raises 'synthetic.api_base is required for API teacher backend.' for every prompt (synthetic.py:218-222). The config validator does not check api_base, so --dry-run passes. The hints offer 'API model name ... or local model path' for the teacher (translations.js:1803) and say 'Empty = local teacher model' for the key env (translations.js:1808), but the wizard never switches teacher_backend to local, and it rejects an empty key env (wizard.js:461-463).

Where.

• site/js/wizard.js:844-851
• site/js/wizard.js:461-463 (empty api_key_env rejection)
• forgelm/synthetic.py:221-222 (the raise itself)
• site/js/translations.js:1808
• forgelm/config.py:1333-1343 (teacher_backend and api_base defaults)
• forgelm/config.py:1419-1441
• forgelm/cli/_no_train_modes.py:180-189

Evidence.

• From h/dry with seeds.jsonl: fpy -m forgelm --config ../yamls/synthetic.yaml --generate-data -> 'Generation failed for prompt 0/1: synthetic.api_base is required for API teacher backend.' ... 'produced no usable examples.' EXIT=2. Loading the same YAML with load_config succeeds.

Recommended fix. Expose teacher_backend and api_base (default https://api.openai.com/v1 for the api backend), and emit teacher_backend: local when a local path is chosen.

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

DOC-D4B09-011 — site/js/translations.js: broken example — Teacher/judge 'API model name (gpt-4o, claude-opus-4-7) or local model path'; synthetic api_key_env 'Empty = local teacher model'; judge 'Empty = local judge…

severity High

Where. site/js/translations.js:1801-1808 (wizard.synthetic.enabled.detail/.teacher_model.hint/.api_key_env.hint), :1747-1749 (wizard.judge.model.hint/.api_key_env.hint) · type: broken example

The document says. Teacher/judge 'API model name (gpt-4o, claude-opus-4-7) or local model path'; synthetic api_key_env 'Empty = local teacher model'; judge 'Empty = local judge model'

What the code does. (1) The wizard emits synthetic: enabled, teacher_model, seed_file, api_key_env (site/js/wizard.js:844-850) but no api_base. teacher_backend defaults to 'api' (config.py:1333) and the API teacher raises ValueError 'synthetic.api_base is required' for every prompt (synthetic.py:221-222), which is caught per prompt, so the run produces no data. The config still passes --dry-run. (2) An empty api_key_env does not select a local teacher; local needs teacher_backend: local, which the wizard never emits. The wizard also rejects an empty key (wizard.js:461-462). (3) Judge and teacher API calls speak the OpenAI chat-completions format (judge.py:223-233; synthetic.py:208-246). A claude-* model name is sent to api.openai.com unless judge_api_base or synthetic.api_base points at an OpenAI-compatible gateway, and the wizard exposes neither. (4) The judge hint 'Empty = local judge model' conflicts with wizard.js:441-442, which makes it required.

Code references. site/js/wizard.js:844-850, forgelm/synthetic.py:208-246, forgelm/config.py:1333,1343, forgelm/judge.py:223-233

User impact. The synthetic-data step of the wizard produces a YAML that yields nothing. Users who follow the hint and enter a Claude model name get API errors.

Proposed correction. Write: 'Model name served by an OpenAI-compatible /chat/completions endpoint (set the endpoint in synthetic.api_base / evaluation.llm_judge.judge_api_base). Leave the API-key variable empty only if you set teacher_backend: local in the YAML.' Delete the 'Empty = local' sentences or make them true.

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}