[Wave 3] Medium — Module sweep: Trainer core, OOM recovery, auto-revert & GRPO rewards (part 2/2) #570

Wave 3 · 14 finding(s) in 14 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

• CR-A02-061 (code, Medium) — A run whose Art. 11 / Annex IV export, model_integrity.json or deployer instructions failed still exits 0 with success:true and nothing in the JSON envelope says so
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. Tiny-qwen SFT config with a compliance: block (limited-risk), forgelm --config cfg.yaml --output-format json. (1) Semantic: out/compliance/ pre-created with mode 555, so tempfile.mkdtemp raises PermissionError. Result: EXIT=0, envelope {success:true, reverted:false, awaiting_approval:false, ...} with no compliance key, compliance/ empty. Audit: ..., model.integrity_verified, compliance.artifacts_export_failed{reason:'[Errno 13] Permission denied ...'}, pipeline.completed. (2) Injected OSError…
  Where. forgelm/trainer.py:2088, forgelm/trainer.py:2148-2155, forgelm/trainer.py:2183-2189, forgelm/trainer.py:1496-1506
  Recommended fix. Carry the compliance status into TrainResult and the envelope, e.g. compliance: {manifest_exported, annex_iv_written, integrity_written, deployer_instructions_written, governance_ok, error}. When a compliance: block or a strict risk tier is configured, treat a failed Art. 11 / Annex IV export as a run failure: exit 2 or a dedicated documented code, success:false and an error string. Log at ERROR…

• CR-A02-062 (code, Medium) — Pipeline per-stage webhook payloads carry no stage name; chained stages all report run_name 'final_model_finetune'
  severity Medium · reachability documented-opt-in · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. u229 2-stage pipeline: the DPO stage's training.start and training.success payloads both have run_name 'final_model_finetune' and no stage field. A 3-stage SFT->DPO->GRPO chain would send two indistinguishable 'final_model_finetune' streams.
  Where. forgelm/trainer.py:347, forgelm/webhook.py:297-306, docs/guides/pipeline.md:111
  Recommended fix. Pass the stage name (or the pipeline run id plus stage) into per-stage payloads through a registered extra key, or derive run_name from the stage. Normalise trailing separators with os.path.basename(os.path.normpath(...)). Otherwise correct pipeline.md:111.

• CR-A02-063 (code, Medium) — Audit-log user manual example shows a sequence the code never emits
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace)
  Failure scenario. An auditor following the documented example accepts a log with model.reverted followed by pipeline.completed as normal, or writes a parser that expects 'sha256:'-prefixed prev_hash values.
  Where. docs/usermanuals/en/compliance/audit-log.md:14-19, docs/usermanuals/en/compliance/audit-log.md:103-109, forgelm/trainer.py:1621-1657, docs/usermanuals/en/compliance/audit-log.md:40
  Recommended fix. Replace the example with a real sequence (for example training.started → … → model.reverted with no pipeline.completed), use bare-hex prev_hash values, and mirror the change to TR.

• CR-A02-064 (code, Medium) — DPO guide says auto-revert rolls back to the pre-DPO checkpoint; it only deletes
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN DOCS-04 — docs/roadmap/phase-16-trust-surface-hardening.md:760-765,778-780 (S15 open; auto-revert sweep; the roadmap does not say whether this is DOCS-03 or DO…
  Failure scenario. An operator who relies on dpo.md enables auto_revert expecting a usable rolled-back model. After a gate failure, final_model is deleted, no pre-DPO artefact is promoted, and the job exits 3 with no model to serve.
  Where. forgelm/trainer.py:960-1007, docs/usermanuals/en/training/dpo.md:203, docs/usermanuals/tr/training/dpo.md:189, docs/usermanuals/en/reference/yaml-templates.md:104 (same wrong comment, not in the candidate list)
  Recommended fix. Reword dpo.md:187 and :203 and the TR mirror to 'deletes the DPO artefacts (nothing is restored)'.

• CR-A02-066 (code, Medium) — Benchmark scores are never sent to the experiment-tracking backend
  severity Medium · reachability documented-opt-in · intent contract-violation · confidence confirmed · independently confirmed · reproduced with stubs
  Failure scenario. A team compares fine-tune runs on a W&B dashboard and expects arc_easy/hellaswag scores there as documented. The panels stay empty, and only the local results file or the model card has the numbers.
  Where. forgelm/trainer.py:1393-1401, docs/usermanuals/en/operations/experiment-tracking.md:19, docs/usermanuals/tr/operations/experiment-tracking.md:19, forgelm/trainer.py:1641
  Recommended fix. Either remove 'and benchmark scores' from the EN/TR text, or log the benchmark/* metrics through the trainer's callback handler (e.g. trainer.log) before the backend run is closed.

• CR-A02-067 (code, Medium) — Safety notebook reads safety_results.json from final_model/, but ForgeLM writes it to <output_dir>/safety/
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced with stubs
  Failure scenario. A user runs the notebook end to end; the safety gate passes and writes results. Step 7 prints 'No safety_results.json found ... Model was auto-reverted due to safety failure' and shows no results.
  Where. notebooks/safety_evaluation.ipynb (cell 9, json line 316), forgelm/trainer.py:1931, forgelm/safety/_results.py:106
  Recommended fix. Change results_path to './safety_checkpoints/safety/safety_results.json' and add a notebook-path check to CI.

• CR-A02-068 (code, Medium) — Deferring GRPO beta to TRL's default means no KL penalty and no reference model, but the manual implies both
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator reads the manual's VRAM table and reward-hacking warning, assumes GRPO is KL-anchored to a reference model, and runs it. The policy is unregularised and no YAML knob can change that.
  Where. forgelm/trainer.py:754-780, docs/usermanuals/en/training/grpo.md:59, docs/usermanuals/en/training/grpo.md:99, docs/usermanuals/tr/training/grpo.md:96
  Recommended fix. State in grpo.md that the effective KL coefficient is 0 (no reference model) under TRL defaults, or expose a grpo_beta field.

• CR-A02-069 (code, Medium) — ForgeLM calls merge_and_unload on a bitsandbytes 4-bit/8-bit base, so peft re-quantises the merged weights and largely erases the adapter
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced with stubs
  Where. forgelm/trainer.py:1734-1740 (log 'Merging adapters and saving full model', merge_and_unload at 1737, save_pretrained at 1740), forgelm/trainer.py:1737, forgelm/trainer.py:1740, forgelm/trainer.py:1587-1588 (eval_loss metrics measured on the unmerged model before the merge)
  Recommended fix. Never merge into a quantised layer. For quantised bases, reload the base unquantised in bf16 or fp16 (CPU is fine), apply the saved adapter with PeftModel.from_pretrained, then merge_and_unload and save, as export._merge_adapter already does (export.py:155-177). Otherwise refuse merge_adapters together with load_in_4bit at config validation. Re-measure eval_loss on the artefact actually saved, an…

• CR-A02-071 (code, Medium) — On --resume the recorded train_loss is the resumed session's loss sum divided by the whole run's global_step, about half the true loss, and it is propagated unannotated into the envelope, manifest, A…
  severity Medium · reachability documented-opt-in · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator resumes an interrupted run. The model card and Annex IV metrics show a training loss about 2x lower than reality, which an auditor or a dashboard reads as a much better fit.
  Where. forgelm/trainer.py:1579-1580 (confirmed), forgelm/compliance.py:1859, forgelm/model_card.py:285
  Recommended fix. After a resume, recompute train_loss from trainer.state.log_history (the mean over all logged steps), or drop it and record the resumed-from step. At minimum, annotate the metric as session-scoped in the manifest and model card.

• CR-A02-073 (code, Medium) — No verifier binds model_integrity.json to the HMAC-keyed audit log: the 'model.integrity_verified' event carries only a file count, and GGUF sidecars are unkeyed whatever FORGELM_AUDIT_SECRET is
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. Same as the README finding: a model-dir writer substitutes the weights and regenerates the manifest. Nothing in ForgeLM can notice, even when the operator holds the HMAC key, because the keyed log never recorded what the manifest should hash to.
  Where. forgelm/trainer.py:2156-2168 (event emitted at 2167), forgelm/verify/_model_integrity.py:79-86, forgelm/verify/_pipeline_evidence.py:520-536, forgelm/verify/_gguf.py:103-151
  Recommended fix. Log sha256(model_integrity.json) (and optionally per-file digests) in the HMAC-tagged event. Add verify-integrity --audit-log <path>, which requires the secret and corroborates the manifest digest against the event, routing a mismatch to 6. Do the same for export (event with GGUF digest).

• CR-A02-074 (code, Medium) — Model card and deployer instructions do not change when a gate fails: a kept benchmark, safety or judge failure ships a README.md that matches a passing run
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator enables the benchmark, safety and judge gates and keeps the shipped default auto_revert:false. The benchmark average is 0.5 against min_score 0.7, every safety probe is judged unsafe, and the judge average is 8.0 against min_score 8.5. The run exits 0 and the model is promoted. final_model/README.md, the Art. 13 artefact later uploaded to the Hub, reads 'safety-evaluated', 'Average 0.5000', 'Safety score: 0.0000' and says nothing about the judge. A downstream deployer, or an approve…
  Where. forgelm/trainer.py:1789-1802, forgelm/model_card.py:264-272, forgelm/model_card.py:162-176, forgelm/trainer.py:1402-1405
  Recommended fix. Pass the gate outcomes into generate_model_card and generate_deployer_instructions: benchmark_passed, safety_passed, safety evaluation_completed, a judge passed flag, the thresholds, and the list of kept failure reasons. Render a per-gate 'Evaluation gates' table (gate, threshold, measured, outcome PASSED / FAILED (kept, auto_revert=false) / NOT EVALUATED). Add a front-matter tag such as 'gate-fa…

• CR-A02-075 (code, Medium) — Statically knowable gate inputs (judge eval_dataset, safety probes, classifier/judge model, benchmark tasks) are checked only after training
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator enables evaluation.llm_judge with a typo in eval_dataset (or the default judge_model 'gpt-4o' without judge_api_key_env, or an unknown lm-eval task) and sets auto_revert:true. The whole training run executes (hours of GPU on a real model), then the gate fails on an input that was wrong before step 1. _revert_model deletes final_model and the run exits 3 ('do not deploy'). The same config with safety.test_prompts mistyped exits 0 with no safety evidence. In every case --dry-run said…
  Where. forgelm/trainer.py:401-421 (_validate_evaluation_config: lm-eval import probe, then warnings only), forgelm/trainer.py:373-394, forgelm/trainer.py:396-411, forgelm/config.py:888-905
  Recommended fix. Extend the init preflight (and ideally --dry-run) to cover every input the gates will read. Check that safety.test_prompts and llm_judge.eval_dataset exist and are readable, and that they parse to at least one usable prompt (reuse _load_safety_prompts/_load_eval_prompts). Run _reject_generation_only_classifier when classifier_mode='classification'. When model.offline is true, check that class…

• CR-A02-076 (code, Medium) — SOC 2 / ISO mapping rows name artefacts and payload fields ForgeLM does not produce: pipeline.failed error_class/error_message, model_card.md, an 'Annex IV public summary', an Annex IV bundle co-loca…
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An auditor testing CC7.3 looks for error_class in pipeline.failed and finds only free text. An auditor asks for model_card.md or the 'Annex IV public summary' and nothing by that name exists. Reading PI1.5, the auditor collects compliance/ as the traceable bundle and gets no audit log and no integrity manifest. Evidence requests built from these pages fail or need manual reinterpretation.
  Where. docs/reference/soc2_trust_criteria_mapping.md:55, docs/reference/soc2_trust_criteria_mapping.md:34, docs/reference/soc2_trust_criteria_mapping.md:83, docs/reference/soc2_trust_criteria_mapping.md:82
  Recommended fix. Correct the rows. CC7.3: pipeline.failed carries error (or add error_class to the emitter and the catalog). PI1.3 and A.5.13: README.md (model card). CC2.2: drop 'Annex IV public summary' or name annex_iv_metadata.json. PI1.5: name the actual locations of the audit log and the integrity manifest. PI1.4: config_hash is a top-level key.

• CR-A02-077 (code, Medium) — neftune_noise_alpha is forwarded to every trainer type, so the NEFTune hook perturbs non-SFT forward passes (GRPO rollouts, DPO/KTO reference passes)
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced (plumbing only)
  Failure scenario. A GRPO config with neftune_noise_alpha: 5.0 (the documented 'typical' value; the wizard default) trains on rollouts sampled from a heavily noised model. Completions and rewards are degraded or lower quality in ways the operator cannot see, and the scoring pass uses a different noise scale from sampling. Reported metrics look normal.
  Where. forgelm/trainer.py:524, forgelm/trainer.py:755-780 (confirmed), forgelm/wizard/_collectors.py:605-609 (confirmed), forgelm/wizard/_orchestrator.py:567,594-595
  Recommended fix. Reject or warn on neftune_noise_alpha for trainer_type grpo (and document it as SFT-only), or do not forward it to GRPOConfig. Do not offer the wizard prompt for non-SFT trainers.

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}