Skip to content

[Wave 3] Medium — Docs: site/ #623

Open

Description

@cemililik

Wave 3 · 9 finding(s) in 9 fix package(s). Documents with fewer than five findings each in this wave, grouped by directory.

Findings

  • DOC-BLEGAL-158 (docs, Medium) — site/features.html: misleading — GDPR Article 15 access — forgelm reverse-pii scans masked JSONL corpora … that surface every row in which a data-subject identifier appears.
    severity Medium
    Where. site/features.html:422-423 (Turkish mirror: site/js/translations.js:1142-1143) · type: misleading
    The document says. GDPR Article 15 access — forgelm reverse-pii scans masked JSONL corpora … that surface every row in which a data-subject identifier appears.
    What the code does. Art. 15(1) requires confirmation, access and the (a)-(h) information (purposes, categories, recipients, retention, rights, complaint, source, automated decisions). Art. 15(3) requires 'a copy of the personal data undergoing processing', which CJEU C-487/21 defines as a faithful and intelligible reproduction. Locating rows in a corpus is a search primitive for that workflow, not Art. 15 access.
    Proposed correction. Title 'GDPR Art. 15 search primitive'. Append to the body: 'Supports locating data for an access request; the Art. 15(1) information and copy remain the controller's response.' Fix translations.js:1037-1038 (EN), 1142-1143 (TR), 4470-4471 (DE), 4516-4517 (FR), 4562-4563 (ES) and 4608-4609 (ZH) too.

  • DOC-D4B07-011 (docs, Medium) — site/features.html: wrong detail — GRPO ... Built-in format/length shaping reward, or your own reward function.
    severity Medium
    Where. site/js/translations.js:958; site/js/translations.js:407 · type: wrong detail
    The document says. GRPO ... Built-in format/length shaping reward, or your own reward function.
    What the code does. The only user-supplied reward source is training.grpo_reward_model, an HF classifier model path (config.py:522-531). _resolve_grpo_reward_funcs builds either that classifier reward or the built-in format/length reward, plus a math reward when gold_answer is present (trainer.py:1074-1092). No config or API hook accepts a custom Python reward function.
    Code references. forgelm/config.py:522-531, forgelm/trainer.py:1074-1092
    Proposed correction. "Group Relative Policy Optimization for reasoning RL. Built-in format/length shaping reward (plus a correctness reward when your data carries gold_answer), or your own reward model via training.grpo_reward_model."

  • DOC-D4B07-012 (docs, Medium) — site/features.html: misleading — Exit codes: 0 / 1 / 2 / 3 / 4 / 5 / 6 — a public contract for CI/CD. Fail-loud is the default; no silent || true.
    severity Medium
    Where. site/js/translations.js:980 · type: misleading
    The document says. Exit codes: 0 / 1 / 2 / 3 / 4 / 5 / 6 — a public contract for CI/CD. Fail-loud is the default; no silent || true.
    What the code does. The 0-6 contract is accurate (cli/_exit_codes.py:10-51). But evaluation.auto_revert defaults to False (config.py:1191-1193), and with it off a failed benchmark, safety or judge gate is logged as a WARNING and the run exits 0. tests/test_trainer.py:286-309 asserts 'continue → run still exits 0'. Failing loudly is opt-in, not the default.
    Code references. forgelm/cli/_exit_codes.py:10-51, forgelm/config.py:1191-1193, forgelm/trainer.py:1405-1408, tests/test_trainer.py:286-309
    Proposed correction. "0 / 1 / 2 / 3 / 4 / 5 / 6 — a public contract for CI/CD. Set evaluation.auto_revert: true to make a failed quality or safety gate exit 3; by default a failed gate is logged and recorded but the run exits 0."

  • DOC-D4B08-009 (docs, Medium) — site/README.md: wrong detail — No build step; any static server or file:// works; GitHub Pages: Settings → Pages → source: main branch, folder /site.
    severity Medium
    Where. site/README.md:3, site/README.md:7-15, site/README.md:90-98 · type: wrong detail
    The document says. No build step; any static server or file:// works; GitHub Pages: Settings → Pages → source: main branch, folder /site.
    What the code does. guide.html needs generated, gitignored bundles site/js/usermanuals/*.js (site/guide.html:38-39; .gitignore:12). They are built by tools/build_usermanuals.py, and the real deployment is the GitHub Actions workflow (site-deploy.yml:14 'Source: "GitHub Actions"', :59-66). These files are missing from the snapshot, so a plain static serve or a branch-folder Pages deploy breaks the Guide page.
    Code references. site/guide.html:38-39, .gitignore:12, .github/workflows/site-deploy.yml:14, .github/workflows/site-deploy.yml:59-66
    Proposed correction. Add: 'guide.html needs generated bundles: run python3 tools/build_usermanuals.py first (CI does this in .github/workflows/site-deploy.yml; Pages source must be "GitHub Actions").'

  • DOC-D4B08-010 (docs, Medium) — site/README.md: false claim — No tracking — no analytics, no pixels, no third-party JS beyond Google Fonts and (optionally) the Formspree endpoint.
    severity Medium
    Where. site/README.md:49 · type: false claim
    The document says. No tracking — no analytics, no pixels, no third-party JS beyond Google Fonts and (optionally) the Formspree endpoint.
    What the code does. guide.html loads third-party JavaScript from jsDelivr: <script src="https://cdn.jsdelivr.net/npm/mermaid@10.9.4/dist/mermaid.min.js"> (site/guide.html:40). The site's own privacy text discloses Mermaid/jsDelivr as a third request (translations.js:4345). Google Fonts is CSS and font files, not JS.
    Code references. site/guide.html:40, site/js/translations.js:4345
    Proposed correction. 'No tracking — no analytics, no pixels. Third-party requests: Google Fonts (CSS/fonts, every page), Mermaid from jsDelivr (guide.html only, SRI-pinned), Formspree (contact form submit only).'

  • DOC-D4B08-019 (docs, Medium) — site/quickstart.html: false denial — Python 3.10+ on Linux or macOS.
    severity Medium
    Where. site/quickstart.html:206; site/js/translations.js:1479 · type: false denial
    The document says. Python 3.10+ on Linux or macOS.
    What the code does. Windows is supported: the publish matrix tests windows-latest on 3.10-3.13 (.github/workflows/publish.yml:47-48), pyproject.toml:46-51 states 'Apple Silicon, Linux and Windows are unaffected', and the classifier is 'OS Independent' (:35). Intel (x86_64) macOS is NOT supported (pyproject.toml:46-51). The page's own JSON-LD says 'Linux, macOS, Windows'.
    Code references. pyproject.toml:35, pyproject.toml:46-51, .github/workflows/publish.yml:47-48
    Proposed correction. 'Python 3.10–3.13 on Linux, Windows or Apple-Silicon macOS (Intel Macs are not supported; QLoRA/Unsloth need Linux + CUDA).'

  • DOC-D4B08-021 (docs, Medium) — site/quickstart.html: wrong detail — ✔ finished, exit 0 — checkpoints/customer-support/ (staging if human_approval set)
    severity Medium
    Where. site/quickstart.html:305 · type: wrong detail
    The document says. ✔ finished, exit 0 — checkpoints/customer-support/ (staging if human_approval set)
    What the code does. When evaluation.require_human_approval is set and the gate fires, the run exits 4 (EXIT_AWAITING_APPROVAL), not 0 (forgelm/cli/_training.py:159-160; config.py:1220-1223 'stages model ... and exits 4').
    Code references. forgelm/cli/_training.py:159-161, forgelm/cli/_exit_codes.py:14, forgelm/config.py:1220-1223
    Proposed correction. '✔ finished, exit 0 — checkpoints/customer-support/final_model/ (exit 4 + final_model.staging.<run_id>/ when evaluation.require_human_approval: true)'

  • DOC-D4B08-022 (docs, Medium) — site/quickstart.html: broken example — verify the audit chain + approve the staged checkpoint ... '✓ audit_log.jsonl chain valid (47 events, HMAC ok)' ... '✔ promoted to ./checkpoints/customer-suppor…
    severity Medium
    Where. site/quickstart.html:307-315 · type: broken example
    The document says. verify the audit chain + approve the staged checkpoint ... '✓ audit_log.jsonl chain valid (47 events, HMAC ok)' ... '✔ promoted to ./checkpoints/customer-support/'
    What the code does. The customer-support template sets no evaluation.require_human_approval (checked in the generated YAML), so nothing is staged and forgelm approve fails ('Staging directory not found', _approve.py:562-566). When staging exists, approve promotes to final_model/ (_approve.py:541-556, 660), not to the output-dir root. verify-audit prints 'OK: N entries verified' and adds '(HMAC validated)' o…
    Code references. forgelm/cli/subcommands/_approve.py:541-566, forgelm/cli/subcommands/_approve.py:660, forgelm/cli/subcommands/_verify_audit.py:258-260
    Proposed correction. Before the approval commands, add: '# only if you added evaluation.require_human_approval: true (run exits 4)'. Show real output: 'OK: 47 entries verified (HMAC validated) # HMAC only with FORGELM_AUDIT_SECRET set' and 'Approved run ; final model promoted to ./checkpoints/customer-support/final_model.'

  • DOC-D4B08-023 (docs, Medium) — site/quickstart.html: wrong detail — forgelm export ... --output model.gguf --quant q4_k_m produces a q4_k_m GGUF
    severity Medium
    Where. site/quickstart.html:336-337 · type: wrong detail
    The document says. forgelm export ... --output model.gguf --quant q4_k_m produces a q4_k_m GGUF
    What the code does. K-quants are not produced directly. For q4_k_m the converter writes an intermediate f16 GGUF to model.f16.gguf and logs a warning to run llama-quantize yourself (forgelm/export.py:36-52, 266-292; pyproject.toml:126-129).
    Code references. forgelm/export.py:36-52, forgelm/export.py:266-292
    Proposed correction. Use --quant q8_0 (direct), or add a comment: '# K-quants: writes model.f16.gguf, then run llama-quantize model.f16.gguf model.gguf Q4_K_M'.

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationseverity: mediumReal defect with limited reach or a workaroundsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 3Roadmap wave 3: medium severity, module sweeps

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions