Repository navigation
[Wave 4] Low/info — Docs: notebooks/ (part 2/2) #794
Description
Wave 4 · 11 finding(s) in 11 fix package(s). Documents with fewer than five findings each in this wave, grouped by directory.
Findings
-
DOC-D4B06-010 (docs, Low) — notebooks/data_curation.ipynb: wrong detail — Pass
--no-quality-filter... (thequality_summaryblock is omitted entirely).
severity Low
Where.notebooks/data_curation.ipynbcell 12 (markdown, section 5) · type: wrong detail
The document says. Pass--no-quality-filter... (thequality_summaryblock is omitted entirely).
What the code does. With --no-quality-filter the report still contains the keyquality_summary, as an empty object {} (AuditReport.quality_summary defaults to {}: forgelm/data_audit/_types.py:147; observed in audit JSON). ingestion_playground.ipynb cell 10 states this correctly.
Code references.forgelm/data_audit/_types.py:147,forgelm/data_audit/_orchestrator.py:427-436
Proposed correction. '(thequality_summarykey is present but empty:{})'. -
DOC-D4B06-011 (docs, Low) — notebooks/data_curation.ipynb: misleading — Pin
num_permfor cross-run determinism.
severity Low
Where.notebooks/data_curation.ipynbcell 15 (markdown) and cell 17 (code label 'num_perm (pin this)') · type: misleading
The document says. Pinnum_permfor cross-run determinism.
What the code does. The CLI has no num_perm flag;forgelm audit --dedup-method minhashalways uses DEFAULT_MINHASH_NUM_PERM=128. num_perm is settable only through the library kwarg minhash_num_perm (forgelm/data_audit/_aggregator.py:323, _orchestrator.py:304).
Code references.forgelm/cli/_parser.py:498-679,forgelm/data_audit/_types.py,forgelm/data_audit/_aggregator.py:323
Proposed correction. 'num_perm is fixed at 128 on the CLI (settable only via audit_dataset(minhash_num_perm=...)); it is recorded in near_duplicate_summary.num_perm.' -
DOC-D4B06-012 (docs, Low) — notebooks/data_curation.ipynb: wrong detail — Three realistic document types, each deliberately seeded with a near-duplicate, an embedded PII span, and a credential leak.
severity Low
Where.notebooks/data_curation.ipynbcell 2 (markdown) · type: wrong detail
The document says. Three realistic document types, each deliberately seeded with a near-duplicate, an embedded PII span, and a credential leak.
What the code does. The files are two .md and one .txt (two document types). Only deploy_runbook.txt carries a credential; handbook_v2.md contains no PII or credential; only handbook.md and the runbook carry PII.
Code references.notebooks/data_curation.ipynbcell 3
Proposed correction. 'Three small files (two Markdown, one plain text): a near-duplicate pair, PII in the handbook and runbook, and credentials in the runbook.' -
DOC-D4B06-013 (docs, Low) — notebooks/data_curation.ipynb: wrong detail — ingestion extra = PDF/DOCX/EPUB/Markdown parsers + langdetect.
severity Low
Where.notebooks/data_curation.ipynbcell 1 (code comment) · type: wrong detail
The document says. ingestion extra = PDF/DOCX/EPUB/Markdown parsers + langdetect.
What the code does. pyproject.toml ingestion extra lists pypdf, python-docx, ebooklib, beautifulsoup4, langdetect, xxhash; Markdown/TXT need no parser (stdlib).
Code references.pyproject.toml:141-148
Proposed correction. '(PDF/DOCX/EPUB parsers + langdetect + xxhash)'. -
DOC-D4B06-014 (docs, Low) — notebooks/ingestion_playground.ipynb: misleading — Paragraph is the safe default for mixed corpora; Markdown is heading-aware (implicitly safe for .md corpora).
severity Low
Where.notebooks/ingestion_playground.ipynbcell 4 · type: misleading
The document says. Paragraph is the safe default for mixed corpora; Markdown is heading-aware (implicitly safe for .md corpora).
What the code does. Same fallback behaviour as data_curation: --strategy markdown on a heading-less file emits it as one over-cap chunk (forgelm/ingestion.py:1722-1757). The table does not warn about this; paragraph advice is correct.
Code references.forgelm/ingestion.py:1722-1757
Proposed correction. Add: 'markdown emits heading-less files as one chunk; keep paragraph for mixed corpora.' -
DOC-D4B06-015 (docs, Low) — notebooks/ingestion_playground.ipynb: stale — swap
--chunk-size Nfor--chunk-tokens N --tokenizer HF_IDin Step 3.
severity Low
Where.notebooks/ingestion_playground.ipynbcell 13 (markdown, What's next) · type: stale
The document says. swap--chunk-size Nfor--chunk-tokens N --tokenizer HF_IDin Step 3.
What the code does. The ingest command is configured in Step 2 (cell 5: CHUNK_TOKENS / TOKENIZER variables) and executed in Step 3; the notebook already offers variables for this, so the instruction to edit the command by hand is stale.
Code references.notebooks/ingestion_playground.ipynbcell 5
Proposed correction. 'set CHUNK_TOKENS and TOKENIZER in Step 2.' -
DOC-D4B06-016 (docs, Low) — notebooks/post_training_workflow.ipynb: wrong detail — estimation_caveat notes that FlashAttention/SDPA, MoE expert counts, and sliding-window attention aren't modeled.
severity Low
Where.notebooks/post_training_workflow.ipynbcell 2 (markdown) · type: wrong detail
The document says. estimation_caveat notes that FlashAttention/SDPA, MoE expert counts, and sliding-window attention aren't modeled.
What the code does. The emitted caveat text is 'heuristic estimate, +/-30% typical; FlashAttention/SDPA, MoE expert count, and attention type are not modeled' (forgelm/fit_check.py:396-399); sliding-window appears only in a code comment.
Code references.forgelm/fit_check.py:392-400
Proposed correction. '... FlashAttention/SDPA, MoE expert count and attention type are not modeled (typical error +/-30%)'. -
DOC-D4B06-017 (docs, Low) — notebooks/post_training_workflow.ipynb: misleading — walks through the four CLI commands that run after training: 1.
forgelm --fit-check...
severity Low
Where.notebooks/post_training_workflow.ipynbcell 0 (markdown) · type: misleading
The document says. walks through the four CLI commands that run after training: 1.forgelm --fit-check...
What the code does. --fit-check is a flag mode of the training CLI (forgelm/cli/_parser.py:1352-1358) that runs before training (the notebook says so in cell 2); chat/export/deploy are the three post-training subcommands.
Code references.forgelm/cli/_parser.py:1352-1358,forgelm/cli/_no_train_modes.py:269-271
Proposed correction. 'ForgeLM's toolchain around training: one pre-training check (--fit-check) and three post-training commands (chat, export, deploy)'. -
DOC-D4B06-018 (docs, Low) — notebooks/synthetic_data_training.ipynb: misleading — API backend - OpenAI, Anthropic, or any OpenAI-compatible endpoint.
severity Low
Where.notebooks/synthetic_data_training.ipynbcell 0 (markdown) · type: misleading
The document says. API backend - OpenAI, Anthropic, or any OpenAI-compatible endpoint.
What the code does. The api backend only POSTs{api_base}/chat/completionswith a Bearer token and an OpenAI-shaped payload (forgelm/synthetic.py:208-246). There is no native Anthropic Messages API support (different path and x-api-key header); Anthropic is reachable only via an OpenAI-compatible endpoint.
Code references.forgelm/synthetic.py:208-246,forgelm/config.py:1343-1349
Proposed correction. 'API - any OpenAI-compatible chat/completions endpoint (OpenAI, vLLM, or a compatibility gateway in front of Claude)'. -
DOC-D4B06-019 (docs, Low) — notebooks/synthetic_data_training.ipynb: wrong detail — PEFT adapter (...) ~30-100 MB total.
severity Low
Where.notebooks/synthetic_data_training.ipynbcell 9 (markdown) · type: wrong detail
The document says. PEFT adapter (...) ~30-100 MB total.
What the code does. Not measured. For this notebook's config (SmolLM2-135M, r=16, q_proj+v_proj only) the adapter has roughly 1M parameters (about 4 MB in fp32) plus tokenizer files of a few MB, i.e. well below 30 MB. Figure copied from the larger-model notebook.
Code references.notebooks/synthetic_data_training.ipynbcell 4
Proposed correction. '(adapter_config.json + adapter_model.safetensors + tokenizer files, roughly 5-10 MB for this tiny model)'. -
DOC-D4B06-020 (docs, Low) — notebooks/synthetic_data_training.ipynb: stale — Run this cell after Step 6+7.
severity Low
Where.notebooks/synthetic_data_training.ipynbcell 10 (code comment) · type: stale
The document says. Run this cell after Step 6+7.
What the code does. Training is Step 7 and the compare cell that defines model_path is Step 8; the comment misnumbers them.
Code references.notebooks/synthetic_data_training.ipynbcells 7-8
Proposed correction. 'Run this cell after Steps 7 and 8.'
Done when
- Every finding above is fixed or closed with a reason (tick it here).
- Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.
Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity