Skip to content

[Wave 4] Low/info — Docs: docs/usermanuals/en/getting-started/ #788

Open

Description

@cemililik

Wave 4 · 14 finding(s) in 14 fix package(s). Documents with fewer than five findings each in this wave, grouped by directory.

Findings

  • DOC-D3B04B-022 (docs, Low) — docs/usermanuals/en/getting-started/first-run.md: misleading — customer-support is an SFT-only run, but the sample training log shows 'DPO preference pass', benchmark and Llama Guard lines and an Ann…
    severity Low
    Where. docs/usermanuals/en/getting-started/first-run.md:83,140-149 · type: misleading
    The document says. customer-support is an SFT-only run, but the sample training log shows 'DPO preference pass', benchmark and Llama Guard lines and an Annex IV path under artifacts/
    What the code does. The template config enables SFT only, with no benchmark, safety or compliance block.
    Code references. forgelm/templates/customer-support/config.yaml
    Proposed correction. Replace with a real SFT log excerpt, or label it 'full-featured example'.
    Turkish mirror. the Turkish page has no such statement

  • DOC-D3B04B-023 (docs, Low) — docs/usermanuals/en/getting-started/first-run.md: wrong detail — customer-support is 'the smallest' template.
    severity Low
    Where. docs/usermanuals/en/getting-started/first-run.md:83 (Turkish mirror: docs/usermanuals/tr/getting-started/first-run.md:83) · type: wrong detail
    The document says. customer-support is 'the smallest' template.
    What the code does. Bundled seeds: customer-support 58 rows, medical-qa-tr 49, grpo-math 40 (forgelm/templates/*/data.jsonl). medical-qa-tr has the same ~15 min estimate.
    Code references. forgelm/templates/customer-support/data.jsonl, forgelm/quickstart.py
    Proposed correction. pick customer-support — a small, general-purpose SFT seed that finishes in roughly 15 minutes on the primary model.
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04B-024 (docs, Low) — docs/usermanuals/en/getting-started/first-run.md: wrong detail — fit-check prints FITS est. peak 11.4 GB / 12 GB available.
    severity Low
    Where. docs/usermanuals/en/getting-started/first-run.md:122-125 (Turkish mirror: docs/usermanuals/tr/getting-started/first-run.md:124) · type: wrong detail
    The document says. fit-check prints FITS est. peak 11.4 GB / 12 GB available.
    What the code does. The output is a '=== VRAM Fit Check ===' block with GPU available, Estimated peak, Verdict, Breakdown and Recommendations (forgelm/fit_check.py:476-512). FITS needs an estimate at or below 85% of available VRAM. 11.4/12 is 95%, which is TIGHT (forgelm/fit_check.py:27-28, 330-334).
    Code references. forgelm/fit_check.py:27-28, forgelm/fit_check.py:330-334, forgelm/fit_check.py:476-512
    Proposed correction. Show the real block, e.g. 'GPU available: 12.0 GB / Estimated peak: 9.8 GB / Verdict: ✅ FITS (2.2 GB headroom)', and state FITS ≤85%, TIGHT ≤95%, OOM >95%.
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04B-025 (docs, Low) — docs/usermanuals/en/getting-started/first-run.md: wrong detail — Sample '--fit-check' prints 'FITS est. peak 11.4 GB / 12 GB'; UNKNOWN = 'Architecture not in the GPU profile database'
    severity Low
    Where. docs/usermanuals/en/getting-started/first-run.md:124,134 · type: wrong detail
    The document says. Sample '--fit-check' prints 'FITS est. peak 11.4 GB / 12 GB'; UNKNOWN = 'Architecture not in the GPU profile database'
    What the code does. UNKNOWN is returned when no GPU is detected (hypothetical estimate); the output is a multi-line block, and 11.4/12 = 0.95 would not be FITS (thresholds 0.85 TIGHT, 0.95 OOM).
    Code references. forgelm/fit_check.py:27-28,59,327-334
    Proposed correction. Align with quickstart.md:157-166 and the real output.
    Turkish mirror. the Turkish page has no such statement

  • DOC-D3B04B-026 (docs, Low) — docs/usermanuals/en/getting-started/installation.md: misleading — The base install gives 'evaluation' (with the trainer, safety scoring and compliance), enough to run end-to-end.
    severity Low
    Where. docs/usermanuals/en/getting-started/installation.md:34 (Turkish mirror: docs/usermanuals/tr/getting-started/installation.md:34) · type: misleading
    The document says. The base install gives 'evaluation' (with the trainer, safety scoring and compliance), enough to run end-to-end.
    What the code does. lm-evaluation-harness benchmarks need the [eval] extra (pyproject.toml:80-82; the doctor's extras.eval row). Only loss, safety and judge evaluation run on the base install.
    Code references. pyproject.toml:80-82, forgelm/cli/subcommands/_doctor.py:102
    Proposed correction. ...evaluation (loss, Llama Guard safety, LLM-as-judge; lm-eval benchmarks need forgelm[eval])...
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04B-027 (docs, Low) — docs/usermanuals/en/getting-started/installation.md: stale — python -m forgelm --version prints ForgeLM 0.9.1rc1.
    severity Low
    Where. docs/usermanuals/en/getting-started/installation.md:47 (Turkish mirror: docs/usermanuals/tr/getting-started/installation.md:47) · type: stale
    The document says. python -m forgelm --version prints ForgeLM 0.9.1rc1.
    What the code does. pyproject.toml:19 version is 0.11.1rc1, and the command prints 'ForgeLM 0.11.1rc1' (run against the snapshot).
    Code references. pyproject.toml:19
    Proposed correction. Use a version-agnostic sample, e.g. ForgeLM 0.11.1rc1 or ForgeLM <version>.
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04B-028 (docs, Low) — docs/usermanuals/en/getting-started/installation.md: wrong detail — The [export] extra 'pulls in the gguf writer and supporting libraries'.
    severity Low · already tracked: KNOWN-OPEN CLI-02|CLI-17|CLI-18|CLI-09|CLI-12|CLI-13 docs/roadmap/risks-and-decisions.md:191
    Where. docs/usermanuals/en/getting-started/installation.md:78 (Turkish mirror: docs/usermanuals/tr/getting-started/installation.md:78) · type: wrong detail
    The document says. The [export] extra 'pulls in the gguf writer and supporting libraries'.
    What the code does. export = ['llama-cpp-python>=0.2.90; sys_platform != "win32"'] (pyproject.toml:126-132). Conversion uses llama-cpp-python's convert_hf_to_gguf.py. The extra is skipped on Windows.
    Code references. pyproject.toml:126-132
    Proposed correction. Pulls in llama-cpp-python (its bundled convert_hf_to_gguf.py); not installed on Windows.
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04B-029 (docs, Low) — docs/usermanuals/en/getting-started/installation.md: wrong detail — Combining extras would pull 'DeepSpeed / bitsandbytes / Presidio / sentence-transformers'.
    severity Low
    Where. docs/usermanuals/en/getting-started/installation.md:96 (Turkish mirror: docs/usermanuals/tr/getting-started/installation.md:96) · type: wrong detail
    The document says. Combining extras would pull 'DeepSpeed / bitsandbytes / Presidio / sentence-transformers'.
    What the code does. No extra declares sentence-transformers (pyproject.toml:70-235).
    Code references. pyproject.toml:70-235
    Proposed correction. Drop 'sentence-transformers' (e.g. 'DeepSpeed / bitsandbytes / Presidio / mlflow').
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04B-032 (docs, Low) — docs/usermanuals/en/getting-started/introduction.md: misleading — The flowchart draws Auto-revert looping back to training (G --> E), implying that a regression triggers retraining.
    severity Low
    Where. docs/usermanuals/en/getting-started/introduction.md:32-33 (Turkish mirror: docs/usermanuals/tr/getting-started/introduction.md) · type: misleading
    The document says. The flowchart draws Auto-revert looping back to training (G --> E), implying that a regression triggers retraining.
    What the code does. Auto-revert deletes the saved model and marks the run reverted. It does not start training again (forgelm/trainer.py:1490-1493 calls _revert_model and _mark_reverted, then returns False).
    Code references. forgelm/trainer.py:1486-1494
    Proposed correction. Point the Auto-revert node to a terminal 'run marked reverted (model deleted)' node instead of back to training, or label the edge 'operator fixes config and re-runs'.
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04B-033 (docs, Low) — docs/usermanuals/en/getting-started/introduction.md: misleading — Every capability listed below is opt-in via YAML.
    severity Low
    Where. docs/usermanuals/en/getting-started/introduction.md:51 (Turkish mirror: docs/usermanuals/tr/getting-started/introduction.md) · type: misleading
    The document says. Every capability listed below is opt-in via YAML.
    What the code does. Ingestion, audit, GGUF export and deploy are CLI subcommands driven by flags, not YAML keys. ForgeConfig has no ingest/audit/export section (top-level fields: model, lora, training, data, auth, evaluation, webhook, distributed, merge, compliance, risk_assessment, monitoring, synthetic, retention, pipeline; introspection of forgelm.config.ForgeConfig).
    Code references. forgelm/config.py, forgelm/cli/_parser.py:530
    Proposed correction. Every capability below is opt-in, either through a YAML block (training, evaluation, compliance) or through its own subcommand (ingest, audit, export, deploy).
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04B-034 (docs, Low) — docs/usermanuals/en/getting-started/project-layout.md: false claim — ForgeLM uses a project-local .forgelm/ cache directory.
    severity Low
    Where. docs/usermanuals/en/getting-started/project-layout.md:30,59,71 (Turkish mirror: docs/usermanuals/tr/getting-started/project-layout.md:30,59,71) · type: false claim
    The document says. ForgeLM uses a project-local .forgelm/ cache directory.
    What the code does. No code reads or writes a .forgelm/ directory. ForgeLM's own state uses $XDG_CACHE_HOME/forgelm/ (wizard state, forgelm/wizard/_state.py:174-178) and .forgelm_audit_salt in the output dir (forgelm/cli/subcommands/_purge.py:79). Model caches follow HF_HOME / HF_HUB_CACHE.
    Code references. forgelm/wizard/_state.py:174-178, forgelm/cli/subcommands/_purge.py:79
    Proposed correction. Remove .forgelm/ from the tree and the gitignore, or replace it with a note on $XDG_CACHE_HOME/forgelm/ (wizard state).
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04B-035 (docs, Low) — docs/usermanuals/en/getting-started/project-layout.md: false claim — forgelm deploy --output writes 'Modelfile, K8s manifest, etc.'
    severity Low
    Where. docs/usermanuals/en/getting-started/project-layout.md:41 (Turkish mirror: docs/usermanuals/tr/getting-started/project-layout.md:41) · type: false claim
    The document says. forgelm deploy --output writes 'Modelfile, K8s manifest, etc.'
    What the code does. The deploy targets produce a Modelfile (ollama), vllm_config.yaml, a docker-compose.yaml (tgi) or hf_endpoint.json. There is no Kubernetes manifest (forgelm/deploy.py:36-41).
    Code references. forgelm/deploy.py:36-41
    Proposed correction. --output (Modelfile, vllm_config.yaml, TGI docker-compose.yaml, or hf_endpoint.json)
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04B-036 (docs, Low) — docs/usermanuals/en/getting-started/project-layout.md: wrong detail — Training output default is ./checkpoints/<name>/.
    severity Low
    Where. docs/usermanuals/en/getting-started/project-layout.md:82 (Turkish mirror: docs/usermanuals/tr/getting-started/project-layout.md:80) · type: wrong detail
    The document says. Training output default is ./checkpoints/<name>/.
    What the code does. TrainingConfig.output_dir defaults to './checkpoints', with no per-run name. The final model goes to <output_dir>/final_model (config introspection; forgelm/config.py TrainingConfig).
    Code references. forgelm/config.py:443
    Proposed correction. Training output | ./checkpoints/ (set a per-project training.output_dir, e.g. ./checkpoints/<name>/) | training.output_dir: in YAML
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04B-037 (docs, Low) — docs/usermanuals/en/getting-started/project-layout.md: misleading — To avoid duplication between prod and dev configs, use a YAML anchor (&base / <<: *base) in a single file.
    severity Low
    Where. docs/usermanuals/en/getting-started/project-layout.md:96 (Turkish mirror: docs/usermanuals/tr/getting-started/project-layout.md) · type: misleading
    The document says. To avoid duplication between prod and dev configs, use a YAML anchor (&base / <<: *base) in a single file.
    What the code does. Each config file is one ForgeConfig, and every model uses extra='forbid'. A top-level holder key for the anchored base (anything outside model/lora/training/data/...) is rejected; verified that an unknown top-level key such as extends fails validation. One file cannot hold both a prod and a dev config.
    Code references. forgelm/config.py:122, forgelm/config.py:1549
    Proposed correction. Use YAML anchors only for repeated sub-blocks inside one config; to share between prod and dev, generate both from a template in CI (e.g. yq/envsubst).
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backlog: knownAlready tracked in docs/roadmapdocumentationImprovements or additions to documentationseverity: lowMinor defect, inaccuracy or rough edgesource: review-2026-10Found by the October 2026 code and documentation reviewwave: 4Roadmap wave 4: low / info / watch, when touching the module

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions