Skip to content

[Wave 4] Low/info — Docs: notebooks/ (part 2/2) #794

Open

Description

@cemililik

Wave 4 · 11 finding(s) in 11 fix package(s). Documents with fewer than five findings each in this wave, grouped by directory.

Findings

  • DOC-D4B06-010 (docs, Low) — notebooks/data_curation.ipynb: wrong detail — Pass --no-quality-filter ... (the quality_summary block is omitted entirely).
    severity Low
    Where. notebooks/data_curation.ipynb cell 12 (markdown, section 5) · type: wrong detail
    The document says. Pass --no-quality-filter ... (the quality_summary block is omitted entirely).
    What the code does. With --no-quality-filter the report still contains the key quality_summary, as an empty object {} (AuditReport.quality_summary defaults to {}: forgelm/data_audit/_types.py:147; observed in audit JSON). ingestion_playground.ipynb cell 10 states this correctly.
    Code references. forgelm/data_audit/_types.py:147, forgelm/data_audit/_orchestrator.py:427-436
    Proposed correction. '(the quality_summary key is present but empty: {})'.

  • DOC-D4B06-011 (docs, Low) — notebooks/data_curation.ipynb: misleading — Pin num_perm for cross-run determinism.
    severity Low
    Where. notebooks/data_curation.ipynb cell 15 (markdown) and cell 17 (code label 'num_perm (pin this)') · type: misleading
    The document says. Pin num_perm for cross-run determinism.
    What the code does. The CLI has no num_perm flag; forgelm audit --dedup-method minhash always uses DEFAULT_MINHASH_NUM_PERM=128. num_perm is settable only through the library kwarg minhash_num_perm (forgelm/data_audit/_aggregator.py:323, _orchestrator.py:304).
    Code references. forgelm/cli/_parser.py:498-679, forgelm/data_audit/_types.py, forgelm/data_audit/_aggregator.py:323
    Proposed correction. 'num_perm is fixed at 128 on the CLI (settable only via audit_dataset(minhash_num_perm=...)); it is recorded in near_duplicate_summary.num_perm.'

  • DOC-D4B06-012 (docs, Low) — notebooks/data_curation.ipynb: wrong detail — Three realistic document types, each deliberately seeded with a near-duplicate, an embedded PII span, and a credential leak.
    severity Low
    Where. notebooks/data_curation.ipynb cell 2 (markdown) · type: wrong detail
    The document says. Three realistic document types, each deliberately seeded with a near-duplicate, an embedded PII span, and a credential leak.
    What the code does. The files are two .md and one .txt (two document types). Only deploy_runbook.txt carries a credential; handbook_v2.md contains no PII or credential; only handbook.md and the runbook carry PII.
    Code references. notebooks/data_curation.ipynb cell 3
    Proposed correction. 'Three small files (two Markdown, one plain text): a near-duplicate pair, PII in the handbook and runbook, and credentials in the runbook.'

  • DOC-D4B06-013 (docs, Low) — notebooks/data_curation.ipynb: wrong detail — ingestion extra = PDF/DOCX/EPUB/Markdown parsers + langdetect.
    severity Low
    Where. notebooks/data_curation.ipynb cell 1 (code comment) · type: wrong detail
    The document says. ingestion extra = PDF/DOCX/EPUB/Markdown parsers + langdetect.
    What the code does. pyproject.toml ingestion extra lists pypdf, python-docx, ebooklib, beautifulsoup4, langdetect, xxhash; Markdown/TXT need no parser (stdlib).
    Code references. pyproject.toml:141-148
    Proposed correction. '(PDF/DOCX/EPUB parsers + langdetect + xxhash)'.

  • DOC-D4B06-014 (docs, Low) — notebooks/ingestion_playground.ipynb: misleading — Paragraph is the safe default for mixed corpora; Markdown is heading-aware (implicitly safe for .md corpora).
    severity Low
    Where. notebooks/ingestion_playground.ipynb cell 4 · type: misleading
    The document says. Paragraph is the safe default for mixed corpora; Markdown is heading-aware (implicitly safe for .md corpora).
    What the code does. Same fallback behaviour as data_curation: --strategy markdown on a heading-less file emits it as one over-cap chunk (forgelm/ingestion.py:1722-1757). The table does not warn about this; paragraph advice is correct.
    Code references. forgelm/ingestion.py:1722-1757
    Proposed correction. Add: 'markdown emits heading-less files as one chunk; keep paragraph for mixed corpora.'

  • DOC-D4B06-015 (docs, Low) — notebooks/ingestion_playground.ipynb: stale — swap --chunk-size N for --chunk-tokens N --tokenizer HF_ID in Step 3.
    severity Low
    Where. notebooks/ingestion_playground.ipynb cell 13 (markdown, What's next) · type: stale
    The document says. swap --chunk-size N for --chunk-tokens N --tokenizer HF_ID in Step 3.
    What the code does. The ingest command is configured in Step 2 (cell 5: CHUNK_TOKENS / TOKENIZER variables) and executed in Step 3; the notebook already offers variables for this, so the instruction to edit the command by hand is stale.
    Code references. notebooks/ingestion_playground.ipynb cell 5
    Proposed correction. 'set CHUNK_TOKENS and TOKENIZER in Step 2.'

  • DOC-D4B06-016 (docs, Low) — notebooks/post_training_workflow.ipynb: wrong detail — estimation_caveat notes that FlashAttention/SDPA, MoE expert counts, and sliding-window attention aren't modeled.
    severity Low
    Where. notebooks/post_training_workflow.ipynb cell 2 (markdown) · type: wrong detail
    The document says. estimation_caveat notes that FlashAttention/SDPA, MoE expert counts, and sliding-window attention aren't modeled.
    What the code does. The emitted caveat text is 'heuristic estimate, +/-30% typical; FlashAttention/SDPA, MoE expert count, and attention type are not modeled' (forgelm/fit_check.py:396-399); sliding-window appears only in a code comment.
    Code references. forgelm/fit_check.py:392-400
    Proposed correction. '... FlashAttention/SDPA, MoE expert count and attention type are not modeled (typical error +/-30%)'.

  • DOC-D4B06-017 (docs, Low) — notebooks/post_training_workflow.ipynb: misleading — walks through the four CLI commands that run after training: 1. forgelm --fit-check ...
    severity Low
    Where. notebooks/post_training_workflow.ipynb cell 0 (markdown) · type: misleading
    The document says. walks through the four CLI commands that run after training: 1. forgelm --fit-check ...
    What the code does. --fit-check is a flag mode of the training CLI (forgelm/cli/_parser.py:1352-1358) that runs before training (the notebook says so in cell 2); chat/export/deploy are the three post-training subcommands.
    Code references. forgelm/cli/_parser.py:1352-1358, forgelm/cli/_no_train_modes.py:269-271
    Proposed correction. 'ForgeLM's toolchain around training: one pre-training check (--fit-check) and three post-training commands (chat, export, deploy)'.

  • DOC-D4B06-018 (docs, Low) — notebooks/synthetic_data_training.ipynb: misleading — API backend - OpenAI, Anthropic, or any OpenAI-compatible endpoint.
    severity Low
    Where. notebooks/synthetic_data_training.ipynb cell 0 (markdown) · type: misleading
    The document says. API backend - OpenAI, Anthropic, or any OpenAI-compatible endpoint.
    What the code does. The api backend only POSTs {api_base}/chat/completions with a Bearer token and an OpenAI-shaped payload (forgelm/synthetic.py:208-246). There is no native Anthropic Messages API support (different path and x-api-key header); Anthropic is reachable only via an OpenAI-compatible endpoint.
    Code references. forgelm/synthetic.py:208-246, forgelm/config.py:1343-1349
    Proposed correction. 'API - any OpenAI-compatible chat/completions endpoint (OpenAI, vLLM, or a compatibility gateway in front of Claude)'.

  • DOC-D4B06-019 (docs, Low) — notebooks/synthetic_data_training.ipynb: wrong detail — PEFT adapter (...) ~30-100 MB total.
    severity Low
    Where. notebooks/synthetic_data_training.ipynb cell 9 (markdown) · type: wrong detail
    The document says. PEFT adapter (...) ~30-100 MB total.
    What the code does. Not measured. For this notebook's config (SmolLM2-135M, r=16, q_proj+v_proj only) the adapter has roughly 1M parameters (about 4 MB in fp32) plus tokenizer files of a few MB, i.e. well below 30 MB. Figure copied from the larger-model notebook.
    Code references. notebooks/synthetic_data_training.ipynb cell 4
    Proposed correction. '(adapter_config.json + adapter_model.safetensors + tokenizer files, roughly 5-10 MB for this tiny model)'.

  • DOC-D4B06-020 (docs, Low) — notebooks/synthetic_data_training.ipynb: stale — Run this cell after Step 6+7.
    severity Low
    Where. notebooks/synthetic_data_training.ipynb cell 10 (code comment) · type: stale
    The document says. Run this cell after Step 6+7.
    What the code does. Training is Step 7 and the compare cell that defines model_path is Step 8; the comment misnumbers them.
    Code references. notebooks/synthetic_data_training.ipynb cells 7-8
    Proposed correction. 'Run this cell after Steps 7 and 8.'

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationseverity: lowMinor defect, inaccuracy or rough edgesource: review-2026-10Found by the October 2026 code and documentation reviewwave: 4Roadmap wave 4: low / info / watch, when touching the module

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions