Skip to content

[Wave 3] Medium — Docs: docs/usermanuals/en/training/ #617

Open

Description

@cemililik

Wave 3 · 10 finding(s) in 10 fix package(s). Documents with fewer than five findings each in this wave, grouped by directory.

Findings

  • DOC-D3B05-038 (docs, Medium) — docs/usermanuals/en/training/kto.md: false claim — The data loader rejects integer labels for KTO
    severity Medium
    Where. docs/usermanuals/en/training/kto.md:85 (Turkish mirror: docs/usermanuals/tr/training/kto.md:85) · type: false claim
    The document says. The data loader rejects integer labels for KTO
    What the code does. _validate_trainer_columns checks only that completion and label columns exist (data.py:584-615). A KTO run with "label": 1/0 trained to completion with exit 0 (repro).
    Code references. forgelm/data.py:584-615
    Proposed correction. Say: "Use JSON booleans; ForgeLM does not validate the label type, so integers are passed to TRL unchanged."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B05-040 (docs, Medium) — docs/usermanuals/en/training/sft.md: false claim — Re-run forgelm audit (against the new tokenizer) and check the rendered samples / rendered-sample preview in the audit report
    severity Medium
    Where. docs/usermanuals/en/training/sft.md:113,117 (Turkish mirror: docs/usermanuals/tr/training/sft.md:113,117) · type: false claim
    The document says. Re-run forgelm audit (against the new tokenizer) and check the rendered samples / rendered-sample preview in the audit report
    What the code does. forgelm audit takes no model or tokenizer argument (audit --help) and renders no samples. The report keys are generated_at, source_path, source_input, total_samples, splits, cross_split_overlap, pii_*, near_duplicate_summary, secrets_summary, quality_summary, croissant and notes. Training formatting does not use the chat template anyway (data.py:354-411).
    Code references. forgelm/cli/_parser.py (audit subparser), forgelm/data_audit/_orchestrator.py, forgelm/data.py:354-411
    Proposed correction. Remove the audit reference. Suggest inspecting prepare_dataset output or a few formatted rows directly, and note that training rows are rendered as [ROLE] blocks.
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B05-041 (docs, Medium) — docs/usermanuals/en/training/sft.md: wrong detail — On disk: checkpoints/sft/{adapter_model.safetensors, README.md, config_snapshot.yaml, compliance/ (only when the compliance: block is present)}
    severity Medium
    Where. docs/usermanuals/en/training/sft.md:124-134 (Turkish mirror: docs/usermanuals/tr/training/sft.md:128-134) · type: wrong detail
    The document says. On disk: checkpoints/sft/{adapter_model.safetensors, README.md, config_snapshot.yaml, compliance/ (only when the compliance: block is present)}
    What the code does. The adapter, README.md, model_integrity.json and deployer_instructions.md land in <output_dir>/final_model/ (training.final_model_dir default 'final_model', config.py:454-456). No config_snapshot.yaml is written; compliance/training_manifest.yaml is the config record. compliance/ (compliance_report.json, data_governance_report.json, data_provenance.json, training_manifest.yaml) was produced by…
    Code references. forgelm/config.py:451-456
    Proposed correction. Show checkpoints/sft/final_model/{adapter_model.safetensors, adapter_config.json, README.md, model_integrity.json, deployer_instructions.md}, checkpoints/sft/compliance/{compliance_report.json, training_manifest.yaml, data_provenance.json, data_governance_report.json} (always written) and audit_log.jsonl.
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B05-042 (docs, Medium) — docs/usermanuals/en/training/simpo.md: misleading — If --fit-check reports OOM for DPO, switch to SimPO
    severity Medium
    Where. docs/usermanuals/en/training/simpo.md:19 (Turkish mirror: docs/usermanuals/tr/training/simpo.md:19) · type: misleading
    The document says. If --fit-check reports OOM for DPO, switch to SimPO
    What the code does. estimate_vram ignores trainer_type (fit_check.py:340-470), so DPO and SimPO configs get the same verdict (checked: identical estimates for sft/dpo/simpo/grpo).
    Code references. forgelm/fit_check.py:340-470
    Proposed correction. Say that --fit-check is trainer-agnostic and the DPO→SimPO saving is not reflected in its estimate.
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B06-024 (docs, Medium) — docs/usermanuals/en/training/distributed.md: false claim — "If your architecture isn't supported, ForgeLM logs a warning and falls back to the standard backend."
    severity Medium
    Where. docs/usermanuals/en/training/distributed.md:59 (Turkish mirror: docs/usermanuals/tr/training/distributed.md:59) · type: false claim
    The document says. "If your architecture isn't supported, ForgeLM logs a warning and falls back to the standard backend."
    What the code does. _load_unsloth has no fallback. It raises ImportError when unsloth is missing and otherwise calls FastLanguageModel.from_pretrained directly; any failure propagates (forgelm/model.py:290-372). It also raises ConfigError for method: pissa (model.py:343-351). Unsloth combined with distributed training is rejected at config load (config.py:1940-1944).
    Code references. forgelm/model.py:290-372, forgelm/config.py:1940-1944
    Proposed correction. "If Unsloth cannot load your architecture the run fails; set model.backend: transformers. Unsloth is Linux+CUDA only, rejects lora.method: pissa, and cannot be combined with distributed.strategy."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B06-025 (docs, Medium) — docs/usermanuals/en/training/distributed.md: false claim — "ForgeLM raises an error early if [batch sizes are] mismatched … --dry-run from the launching node is sufficient."
    severity Medium
    Where. docs/usermanuals/en/training/distributed.md:145 (Turkish mirror: docs/usermanuals/tr/training/distributed.md:145) · type: false claim
    The document says. "ForgeLM raises an error early if [batch sizes are] mismatched … --dry-run from the launching node is sufficient."
    What the code does. ForgeLM has no cross-node consistency check. --dry-run only validates and echoes the local config (forgelm/cli/_dry_run.py:40-89) and contacts no other node.
    Code references. forgelm/cli/_dry_run.py:40-89
    Proposed correction. "ForgeLM does not compare configs across nodes; make sure every node launches with the identical YAML."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B06-027 (docs, Medium) — docs/usermanuals/en/training/lora.md: misleading — The decision tree offers "Full fine-tuning" as a ForgeLM outcome ("No, can melt the base").
    severity Medium
    Where. docs/usermanuals/en/training/lora.md:19-26 (Turkish mirror: docs/usermanuals/tr/training/lora.md:19) · type: misleading
    The document says. The decision tree offers "Full fine-tuning" as a ForgeLM outcome ("No, can melt the base").
    What the code does. ForgeLM always wraps the model with PEFT: lora is a required block (forgelm/config.py:1608), and get_peft_model(model, _build_lora_config(config)) runs unconditionally (forgelm/model.py:609). No config path trains all base weights.
    Code references. forgelm/config.py:1608, forgelm/model.py:609
    Proposed correction. Replace the "Full fine-tuning" node with "Full fine-tuning (not supported by ForgeLM — every run trains a LoRA-family adapter)", or remove the branch.
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B06-028 (docs, Medium) — docs/usermanuals/en/training/lora.md: wrong detail — Rank 16 is the Default.
    severity Medium
    Where. docs/usermanuals/en/training/lora.md:84 (Turkish mirror: docs/usermanuals/tr/training/lora.md:84) · type: wrong detail
    The document says. Rank 16 is the Default.
    What the code does. lora.r defaults to 8 and lora.alpha to 16 (forgelm/config.py:347-358). lora.dropout defaults to 0.1, not the 0.05 used in the example.
    Code references. forgelm/config.py:347-365
    Proposed correction. Mark row 16 as "Recommended starting point" and state: "The schema default is r: 8, alpha: 16, dropout: 0.1."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B06-029 (docs, Medium) — docs/usermanuals/en/training/lora.md: wrong detail — "The default (Q/K/V/O projections) is the right balance for most tasks."
    severity Medium
    Where. docs/usermanuals/en/training/lora.md:104 (Turkish mirror: docs/usermanuals/tr/training/lora.md:102) · type: wrong detail
    The document says. "The default (Q/K/V/O projections) is the right balance for most tasks."
    What the code does. The default lora.target_modules is ["q_proj", "v_proj"] (forgelm/config.py:384-386), which covers Q and V only.
    Code references. forgelm/config.py:384-386
    Proposed correction. "The default is ["q_proj", "v_proj"]. Add k_proj/o_proj (or the MLP projections) explicitly for more capacity."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B06-030 (docs, Medium) — docs/usermanuals/en/training/pipelines.md: stale — "compliance/pipeline_manifest.json indexes every per-stage training_manifest.json."
    severity Medium
    Where. docs/usermanuals/en/training/pipelines.md:122 (Turkish mirror: docs/usermanuals/tr/training/pipelines.md:122) · type: stale
    The document says. "compliance/pipeline_manifest.json indexes every per-stage training_manifest.json."
    What the code does. Each stage's evidence pointer is <stage output_dir>/compliance/annex_iv_metadata.json (ANNEX_IV_ARTEFACT_BASENAME, forgelm/compliance.py:93; forgelm/cli/_pipeline.py:1642-1644). The code comments say training_manifest.json is "a filename no ForgeLM version has ever written" (_pipeline.py:1635-1641).
    Code references. forgelm/compliance.py:72-93, forgelm/cli/_pipeline.py:1631-1644
    Proposed correction. "…indexes every stage's compliance/annex_iv_metadata.json …"
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationseverity: mediumReal defect with limited reach or a workaroundsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 3Roadmap wave 3: medium severity, module sweeps

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions