[Wave 4] Low/info — Docs: notebooks/ (part 1/2) #793

Wave 4 · 20 finding(s) in 20 fix package(s). Documents with fewer than five findings each in this wave, grouped by directory.

Findings

• DOC-BYAML-016 (docs, Low) — notebooks/post_training_workflow.ipynb: stale — Fallback fetches general_safety.jsonl from the v0.5.7 tag
  severity Low · already tracked: KNOWN-OPEN DOCS-13/DOCS-17 (S16) — docs/roadmap/phase-16-trust-surface-hardening.md:815-816
  Where. notebooks/safety_evaluation.ipynb cell5 · type: stale
  The document says. Fallback fetches general_safety.jsonl from the v0.5.7 tag
  What the code does. Fixed old tag, and the inline fallback creates only 3 prompts with categories S1/S2/S5.
  Code references. notebooks/safety_evaluation.ipynb cell5
  Proposed correction. Pin to the current release tag.

• DOC-BYAML-017 (docs, Low) — notebooks/post_training_workflow.ipynb: stale — config_template.yaml fetched from the v0.9.0 tag 'so the YAML matches the wheel above'
  severity Low · already tracked: KNOWN-OPEN DOCS-13/DOCS-17 (S16) — docs/roadmap/phase-16-trust-surface-hardening.md:815-816
  Where. notebooks/post_training_workflow.ipynb cell1 (lines 40-43) · type: stale
  The document says. config_template.yaml fetched from the v0.9.0 tag 'so the YAML matches the wheel above'
  What the code does. The wheel is pinned to forgelm==0.11.0 (pyproject is 0.11.1rc1) while the template is from v0.9.0; the comment's rationale no longer holds. I could not test the v0.9.0 file offline.
  Code references. notebooks/post_training_workflow.ipynb:40-43
  Proposed correction. Bump the tag with the pin (or bundle the template).

• DOC-D4B02-020 (docs, Low) — notebooks/safety_evaluation.ipynb: misleading — ForgeLM includes a built-in library of 140 adversarial prompts across 6 categories.
  severity Low
  Where. notebooks/safety_evaluation.ipynb:11 (cell 0) · type: misleading
  The document says. ForgeLM includes a built-in library of 140 adversarial prompts across 6 categories.
  What the code does. The 140 prompts (6 files) live in the repository at configs/safety_prompts/ and are not shipped in the pip wheel: package-data covers only forgelm.safety_prompts/*.jsonl. Cell 5 itself has to git-clone them. The wheel does bundle a different set, default_probes.jsonl with 51 prompts, used by forgelm safety-eval --default-probes.
  Code references. pyproject.toml:250-270, configs/safety_prompts/, forgelm/cli/subcommands/_safety_eval.py:50-54
  Proposed correction. The ForgeLM repository ships a library of 140 adversarial prompts in 6 files under configs/safety_prompts/ (not included in the pip wheel — Step 3 fetches it). The wheel bundles a separate 51-prompt set used by forgelm safety-eval --default-probes.

• DOC-D4B02-021 (docs, Low) — notebooks/safety_evaluation.ipynb: misleading — Requirements: GPU runtime required (... T4 GPU)
  severity Low
  Where. notebooks/safety_evaluation.ipynb:13 (cell 0) · type: misleading
  The document says. Requirements: GPU runtime required (... T4 GPU)
  What the code does. The default classifier loads in float16 with device_map="auto" (~16 GB). A T4 exposes less than 16 GB, so cell 4's own check (vram_gb < 16) always warns on a T4, and cell 3 recommends A100/V100. The stated requirement contradicts the notebook's own pre-flight.
  Code references. forgelm/safety/_classifier.py:296-306
  Proposed correction. Requirements: GPU runtime required; the default Llama-Guard-3-8B classifier needs ~16 GB VRAM (A100/V100 recommended — a T4 is at the edge, see the pre-flight below).

• DOC-D4B02-022 (docs, Low) — notebooks/safety_evaluation.ipynb: wrong detail — PEFT adapter + tokenizer files, ~30-100 MB total
  severity Low
  Where. notebooks/safety_evaluation.ipynb:367 (cell 10) · type: wrong detail
  The document says. PEFT adapter + tokenizer files, ~30-100 MB total
  What the code does. Estimate, not measured: SmolLM2-135M with r=16 on q_proj/v_proj has roughly 0.9M LoRA parameters (~4 MB in fp32), plus a few MB of tokenizer files. That is an order of magnitude below 30-100 MB. The adapter is written by save_final_model (adapter-only by default).
  Code references. forgelm/trainer.py:1711-1712
  Proposed correction. (adapter_config.json + adapter_model.safetensors + tokenizer files — a few MB for this 135M model)

• DOC-D4B02-023 (docs, Low) — notebooks/safety_evaluation.ipynb: misleading — confidence_weighted | Classifier confidence score weighted into safety score
  severity Low
  Where. notebooks/safety_evaluation.ipynb:440 (cell 12) · type: misleading
  The document says. confidence_weighted | Classifier confidence score weighted into safety score
  What the code does. Under generation mode, which the default meta-llama/Llama-Guard-3-8B resolves to with classifier_mode auto, each verdict is categorical (safe=1.0 / unsafe=0.0), so confidence_weighted degenerates to the binary safe ratio. A true probability exists only on the classification pipeline path.
  Code references. forgelm/config.py:946-955, forgelm/safety/_classifier.py:15-35
  Proposed correction. Add to the row: "(only on the classification pipeline path; with the default generative Llama-Guard it equals the binary safe ratio)"

• DOC-D4B05-007 (docs, Low) — notebooks/dpo_alignment.ipynb: wrong detail — Adapter plus tokenizer is about 30-100 MB
  severity Low
  Where. notebooks/dpo_alignment.ipynb:177 (Step 8 text '~30-100 MB total') · type: wrong detail
  The document says. Adapter plus tokenizer is about 30-100 MB
  What the code does. About 5-10 MB for this model (estimate).
  Proposed correction. Write '~5-10 MB for this demo'.

• DOC-D4B05-008 (docs, Low) — notebooks/galore_memory_optimization.ipynb: wrong detail — The lora block is required because of a 'Phase 19 library-API invariant'
  severity Low
  Where. notebooks/galore_memory_optimization.ipynb:20 ('ForgeLM requires a lora: block in every config (Phase 19 library-API invariant)') · type: wrong detail
  The document says. The lora block is required because of a 'Phase 19 library-API invariant'
  What the code does. The lora block is required because it is a non-optional field of ForgeConfig (config.py:1608, 'PEFT / LoRA configuration block (required)'), and lora.r must be >= 1 (config.py:347-352). Nothing in the code ties it to a Phase 19 or library-API invariant. The requirement itself is true.
  Code references. forgelm/config.py:1608, forgelm/config.py:347-352
  Proposed correction. Write: 'ForgeLM's config schema requires a lora: block in every config (lora.r >= 1), so this notebook trains a LoRA adapter and uses GaLore as its optimizer.'

• DOC-D4B05-009 (docs, Low) — notebooks/galore_memory_optimization.ipynb: misleading — GaLore applied to the LoRA adapter parameters gives low-rank optimizer-state savings, and rank 64 is a meaningful setting
  severity Low
  Where. notebooks/galore_memory_optimization.ipynb:99 and :9-11 (galore_rank: 64 '# lower rank = less memory'; 'GaLore's optimizer-state savings' with LoRA) (Turkish mirror: forgelm/config.py:539-541 describes galore_enabled as '(alternative to LoRA)', which contradicts both this notebook and the code (lora block is required); also regis…) · type: misleading
  The document says. GaLore applied to the LoRA adapter parameters gives low-rank optimizer-state savings, and rank 64 is a meaningful setting
  What the code does. Verified: with ForgeLM's target regexes (.*.attn.*, .*.mlp.*, trainer.py:548) GaLore is applied to exactly the trainable LoRA A/B weights (8 trainable tensors for 2 layers in my test; the frozen base weights sit in the group but have no grads). However the LoRA matrices here have a smaller side of r=16 (lora.r in the same YAML), so projecting with rank 64 >= 16 compresses nothing: only the 8-…
  Code references. forgelm/trainer.py:544-562, forgelm/config.py:553-555 (galore_rank default 128)
  Proposed correction. Add after the rank comment: '# with LoRA r=16 the projected matrices are already rank <= 16, so galore_rank >= r gives no extra projection saving; the saving here comes from the 8-bit optimizer'. In the intro replace 'GaLore's optimizer-state savings' with 'the 8-bit optimizer-state savings'.

• DOC-D4B05-010 (docs, Low) — notebooks/galore_memory_optimization.ipynb: wrong detail — Adapter plus tokenizer is about 30-100 MB
  severity Low
  Where. notebooks/galore_memory_optimization.ipynb:167 (Step 8 text '~30-100 MB total') · type: wrong detail
  The document says. Adapter plus tokenizer is about 30-100 MB
  What the code does. About 5-10 MB for this model (estimate).
  Proposed correction. Write '~5-10 MB for this demo'.

• DOC-D4B05-011 (docs, Low) — notebooks/galore_memory_optimization.ipynb: wrong detail — The LoRA adapter is about 0.2% of the model's parameters
  severity Low
  Where. notebooks/galore_memory_optimization.ipynb:244 (comparison table: LoRA adapter matrices '~0.2% of params') · type: wrong detail
  The document says. The LoRA adapter is about 0.2% of the model's parameters
  What the code does. For this notebook's model and config (SmolLM2-135M, r=16, q_proj+v_proj; architecture from memory, not available offline) the adapter has about 0.92M parameters, about 0.7% of 135M. 0.2% is a plausible figure for a multi-billion-parameter model, not for this demo.
  Code references. notebooks/galore_memory_optimization.ipynb cell 3 YAML
  Proposed correction. Write '(under 1% of params; ~0.7% for SmolLM2-135M at r=16)'.

• DOC-D4B05-012 (docs, Low) — notebooks/grpo_reasoning.ipynb: wrong detail — The PEFT adapter plus tokenizer files are about 30-100 MB
  severity Low
  Where. notebooks/grpo_reasoning.ipynb:215 (Step 9 text '~30-100 MB total') (Turkish mirror: same sentence in quickstart_sft (line 201), dpo_alignment (177), kto_binary_feedback (227), galore_memory_optimization (167), multi_dataset (262)) · type: wrong detail
  The document says. The PEFT adapter plus tokenizer files are about 30-100 MB
  What the code does. The notebook trains LoRA r=16 on q_proj and v_proj of SmolLM2-135M. From SmolLM2-135M's public architecture (30 layers, hidden 576, 3 KV heads, from memory, not available offline) that is about 0.92M trainable parameters, roughly 3.7 MB in fp32, plus tokenizer files of a few MB, so about 5-8 MB. Measured on the tiny-qwen fixture (r=16, q/v, 2 layers) final_model was 11 MB, almost all tokenizer. T…
  Proposed correction. Write '~5-10 MB total for this SmolLM2-135M demo; larger models and higher ranks produce proportionally larger adapters.'

• DOC-D4B05-013 (docs, Low) — notebooks/grpo_reasoning.ipynb: stale — The download cell runs after Steps 6 and 7 (train/verify)
  severity Low
  Where. notebooks/grpo_reasoning.ipynb:241 (ZIP cell comment 'Run this cell after Step 6+7') (Turkish mirror: notebooks/kto_binary_feedback.ipynb:253, notebooks/multi_dataset.ipynb:288 (same stale numbering)) · type: stale
  The document says. The download cell runs after Steps 6 and 7 (train/verify)
  What the code does. In this notebook Step 6 is Train, Step 7 is Verify output and Step 8 is the model comparison. The cell that defines model_path for the ZIP cell is Step 7/8.
  Code references. notebooks/grpo_reasoning.ipynb: cells 6-8
  Proposed correction. Change the comment to 'Run this cell after Step 7+8.'

• DOC-D4B05-014 (docs, Low) — notebooks/grpo_reasoning.ipynb: stale — The reward stack, including combined_format_length_reward, is defined in forgelm/trainer.py
  severity Low
  Where. notebooks/grpo_reasoning.ipynb:292 (How GRPO Works: 'defined in forgelm/trainer.py') · type: stale
  The document says. The reward stack, including combined_format_length_reward, is defined in forgelm/trainer.py
  What the code does. combined_format_length_reward is defined in forgelm/grpo_rewards.py:207 and only imported by trainer.py (trainer.py:1098). Only _math_reward_fn (trainer.py:216) lives in trainer.py. The cell-3 comment naming trainer.py::_math_reward_fn is correct.
  Code references. forgelm/grpo_rewards.py:207, forgelm/trainer.py:216, forgelm/trainer.py:1098
  Proposed correction. Write: 'This demo's reward stack (combined_format_length_reward in forgelm/grpo_rewards.py, _math_reward_fn in forgelm/trainer.py):'

• DOC-D4B05-015 (docs, Low) — notebooks/grpo_reasoning.ipynb: misleading — combined_format_length_reward rewards outputs that follow the 'Answer: ...' format and are 'within a reasonable length'
  severity Low
  Where. notebooks/grpo_reasoning.ipynb:295 (reward stack item 1) · type: misleading
  The document says. combined_format_length_reward rewards outputs that follow the 'Answer: ...' format and are 'within a reasonable length'
  What the code does. The length term is min(len(completion)/200, 1.0) (grpo_rewards.py:156, 193-204, saturating at 200 characters). It rewards longer output up to 200 characters and has no upper bound or penalty, so nothing checks for 'a reasonable length'. The combined value is 0.8 x format + 0.2 x length. The format term requires the completion to END with 'Answer: '.
  Code references. forgelm/grpo_rewards.py:156, forgelm/grpo_rewards.py:193-204, forgelm/grpo_rewards.py:207-220
  Proposed correction. Write: 'rewards completions that END with "Answer: " (weight 0.8) plus a small length bonus that grows up to 200 characters (weight 0.2); long outputs are not penalised.'

• DOC-D4B05-016 (docs, Low) — notebooks/kto_binary_feedback.ipynb: wrong detail — Adapter plus tokenizer is about 30-100 MB
  severity Low
  Where. notebooks/kto_binary_feedback.ipynb:227 (Step 9 text '~30-100 MB total') · type: wrong detail
  The document says. Adapter plus tokenizer is about 30-100 MB
  What the code does. About 5-10 MB for this model (estimate). The KTO final_model also contains a ref/ subdirectory.
  Proposed correction. Write '~5-10 MB for this demo'.

• DOC-D4B05-017 (docs, Low) — notebooks/kto_binary_feedback.ipynb: stale — Step numbering
  severity Low
  Where. notebooks/kto_binary_feedback.ipynb:253 (ZIP cell comment 'Run this cell after Step 6+7') · type: stale
  The document says. Step numbering
  What the code does. In this notebook Step 6 is Train, Step 7 is Verify, Step 8 is Compare.
  Code references. notebooks/kto_binary_feedback.ipynb: cells 6-8
  Proposed correction. Change to 'Run this cell after Step 7+8.'

• DOC-D4B05-018 (docs, Low) — notebooks/multi_dataset.ipynb: wrong detail — Adapter plus tokenizer is about 30-100 MB
  severity Low
  Where. notebooks/multi_dataset.ipynb:262 (Step 9 text '~30-100 MB total') · type: wrong detail
  The document says. Adapter plus tokenizer is about 30-100 MB
  What the code does. See the same item in grpo_reasoning.ipynb: about 5-10 MB for LoRA r=16 q/v on SmolLM2-135M (estimate from the model's public architecture, not measured).
  Proposed correction. Write '~5-10 MB for this demo'.

• DOC-D4B05-019 (docs, Low) — notebooks/multi_dataset.ipynb: stale — Step numbering for the ZIP cell
  severity Low
  Where. notebooks/multi_dataset.ipynb:288 (ZIP cell comment 'Run this cell after Step 6+7') · type: stale
  The document says. Step numbering for the ZIP cell
  What the code does. In this notebook Step 6 is Train, Step 7 is Verify, Step 8 is Compare; same stale comment as in the GRPO and KTO notebooks.
  Code references. notebooks/multi_dataset.ipynb: cells 6-8
  Proposed correction. Change to 'Run this cell after Step 7+8.'

• DOC-D4B05-020 (docs, Low) — notebooks/quickstart_sft.ipynb: wrong detail — Adapter plus tokenizer is about 30-100 MB
  severity Low
  Where. notebooks/quickstart_sft.ipynb:201 (Step 8 text '~30-100 MB total') · type: wrong detail
  The document says. Adapter plus tokenizer is about 30-100 MB
  What the code does. About 5-10 MB for this model (see grpo_reasoning item); 11 MB measured on the tiny-qwen fixture, which has a larger tokenizer.
  Proposed correction. Write '~5-10 MB for this demo'.

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}