Skip to content

[Wave 3] Medium — Docs: docs/usermanuals/en/reference/ #616

Open

Description

@cemililik

Wave 3 · 8 finding(s) in 8 fix package(s). Documents with fewer than five findings each in this wave, grouped by directory.

Findings

  • DOC-D3B03-019 (docs, Medium) — docs/usermanuals/en/reference/library-api.md: misleading — Anything the CLI does, the library can do — minus the exit-code mapping.
    severity Medium
    Where. docs/usermanuals/en/reference/library-api.md:8 (Turkish mirror: docs/usermanuals/tr/reference/library-api.md:8) · type: misleading
    The document says. Anything the CLI does, the library can do — minus the exit-code mapping.
    What the code does. forgelm.all (32 names) has no public symbol for purge, reverse-pii, approve/reject/approvals, ingest (ingest_path is internal), export, deploy, chat, quickstart, standalone safety-eval (run_safety_evaluation is internal), verify-annex-iv --pipeline, doctor, or cache-models/cache-tasks. These are reachable only through internal modules, which the same page says carry no stability guarantee.
    Code references. forgelm/__init__.py:77-119, forgelm/__init__.py:142-177
    Proposed correction. Replace with: "The public library API covers training, configuration, dataset audit, PII/secrets utilities and the artefact verifiers. Other CLI features (purge, reverse-pii, approvals, ingest, export, deploy, chat, quickstart, doctor, cache-*) have no public, stability-tiered library entry point."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04A-010 (docs, Medium) — docs/usermanuals/en/reference/cli.md: wrong detail — approvals/approve/reject exit codes: '0 … 1 = unknown run_id / config error, 2 = argparse usage error (a missing --output-dir lands here),…
    severity Medium
    Where. docs/usermanuals/en/reference/cli.md:205 (Turkish mirror: docs/usermanuals/tr/reference/cli.md:205) · type: wrong detail
    The document says. approvals/approve/reject exit codes: '0 … 1 = unknown run_id / config error, 2 = argparse usage error (a missing --output-dir lands here), 4 (training mode only)'.
    What the code does. These commands also exit 2 (EXIT_TRAINING_ERROR) on runtime I/O failures. This covers a failed staging promotion, a failed write of human_approval.granted after the model was already promoted (an audit gap), a failed write of human_approval.rejected, and a mid-read failure of the audit-log scan in approvals --pending.
    Code references. forgelm/cli/subcommands/_approve.py:588-595, forgelm/cli/subcommands/_approve.py:614-629, forgelm/cli/subcommands/_approve.py:721-728, forgelm/cli/subcommands/_approvals.py:337-348
    Proposed correction. "2 = argparse usage error (a missing --output-dir lands here) or a runtime I/O failure — the staging promotion failed, the approval/rejection audit event could not be written, or the audit log could not be read through."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04A-011 (docs, Medium) — docs/usermanuals/en/reference/configuration.md: misleading — 'Every YAML field ForgeLM understands' / 'the canonical reference for every YAML field ForgeLM accepts'.
    severity Medium
    Where. docs/usermanuals/en/reference/configuration.md:3, 8 (Turkish mirror: docs/usermanuals/tr/reference/configuration.md:3, 8) · type: misleading
    The document says. 'Every YAML field ForgeLM understands' / 'the canonical reference for every YAML field ForgeLM accepts'.
    What the code does. Several accepted fields are documented nowhere on the page: model.moe.{quantize_experts, experts_to_train}, model.multimodal.{enabled, image_column, text_column}, lora.task_type, and evaluation.llm_judge.min_valid_fraction (default 0.8, the judge evidence floor). The ModelConfig note at :52 lists the fields that do not exist but omits moe/multimodal.
    Code references. forgelm/config.py:119-137, forgelm/config.py:249-343, forgelm/config.py:344-442, forgelm/config.py:1080-1186
    Proposed correction. Add the missing fields: model.moe: {quantize_experts: false, experts_to_train: "all"}, model.multimodal: {enabled: false, image_column: "image", text_column: "text"}, lora.task_type: "CAUSAL_LM", and evaluation.llm_judge.min_valid_fraction: 0.8 (minimum fraction of judge responses that must parse), each with a one-line comment.
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04A-012 (docs, Medium) — docs/usermanuals/en/reference/configuration.md: wrong detail — On a page that promises 'types, defaults, and notes', the model/lora blocks show max_length: 4096, load_in_4bit: false, bnb_4bit_co… severity **Medium** **Where.** [docs/usermanuals/en/reference/configuration.md:39-44](https://github.com/HodeTech/ForgeLM/blob/cab2947/docs/usermanuals/en/reference/configuration.md#L39-L44), 58-63 (Turkish mirror: [docs/usermanuals/tr/reference/configuration.md:39-44](https://github.com/HodeTech/ForgeLM/blob/cab2947/docs/usermanuals/tr/reference/configuration.md#L39-L44), 58-63) · type: wrong detail **The document says.** On a page that promises 'types, defaults, and notes', the model/lora blocks show max_length: 4096, load_in_4bit: false, bnb_4bit_compute_dtype: "bfloat16", r: 16, alpha: 32, dropout: 0.05, target_modules: [q,k,v,o]. The training/evaluation blocks show the real defaults. **What the code does.** Defaults are max_length=2048, load_in_4bit=True, bnb_4bit_compute_dtype='auto', r=8, alpha=16, dropout=0.1, target_modules=['q_proj','v_proj']. yaml-templates.md:205 correctly states that load_in_4bit defaults to true. **Code references.** [forgelm/config.py:249-343](https://github.com/HodeTech/ForgeLM/blob/cab2947/forgelm/config.py#L249-L343), [forgelm/config.py:344-442](https://github.com/HodeTech/ForgeLM/blob/cab2947/forgelm/config.py#L344-L442) **Proposed correction.** Either show the defaults (max_length: 2048, load_in_4bit: true, bnb_4bit_compute_dtype: "auto", r: 8, alpha: 16, dropout: 0.1, target_modules: ["q_proj", "v_proj"]`) or mark each changed value "# example; default: …".
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04A-013 (docs, Medium) — docs/usermanuals/en/reference/configuration.md: wrong detail — 'The Annex IV bundle is generated automatically once compliance: is present and risk_classification resolves to high-risk or unac… severity **Medium** **Where.** [docs/usermanuals/en/reference/configuration.md:246](https://github.com/HodeTech/ForgeLM/blob/cab2947/docs/usermanuals/en/reference/configuration.md#L246) (Turkish mirror: [docs/usermanuals/tr/reference/configuration.md:247](https://github.com/HodeTech/ForgeLM/blob/cab2947/docs/usermanuals/tr/reference/configuration.md#L247)) · type: wrong detail **The document says.** 'The Annex IV bundle is generated automatically once compliance:is present andrisk_classificationresolves tohigh-riskorunacceptable'. **What the code does.** annex_iv_metadata.json is written whenever a compliance: block is present and at least one of provider_name/system_name/intended_purpose is non-blank, whatever the risk tier. Reproduced: a minimal-risk config exported annex_iv_metadata.json. Conversely, it is skipped even for high-risk when all three identity fields are blank. **Code references.** [forgelm/compliance.py:1551-1561](https://github.com/HodeTech/ForgeLM/blob/cab2947/forgelm/compliance.py#L1551-L1561), [forgelm/compliance.py:1803-1816](https://github.com/HodeTech/ForgeLM/blob/cab2947/forgelm/compliance.py#L1803-L1816), [forgelm/compliance.py:2215-2236](https://github.com/HodeTech/ForgeLM/blob/cab2947/forgelm/compliance.py#L2215-L2236) **Proposed correction.** "compliance/annex_iv_metadata.jsonis written whenever thecompliance:block is present and at least one ofprovider_name, system_nameorintended_purposeis non-empty — for every risk tier. With all three blank the file is skipped, even forhigh-risk. There is no separate annex_iv: true` toggle."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04A-015 (docs, Medium) — docs/usermanuals/en/reference/yaml-templates.md: misleading — auto_revert: true # rolls back on regression.
    severity Medium · already tracked: KNOWN-OPEN DOCS-04 docs/roadmap/phase-16-trust-surface-hardening.md:760-765,778-780
    Where. docs/usermanuals/en/reference/yaml-templates.md:104 (Turkish mirror: docs/usermanuals/tr/reference/yaml-templates.md:105) · type: misleading
    The document says. auto_revert: true # rolls back on regression.
    What the code does. auto_revert deletes the saved model directory on a regression. Nothing is restored or rolled back to a previous model.
    Code references. forgelm/config.py:1191-1194, forgelm/trainer.py:785-899
    Proposed correction. auto_revert: true # DELETES the saved model dir on regression (nothing is restored)
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04A-016 (docs, Medium) — docs/usermanuals/en/reference/yaml-templates.md: wrong detail — 'The bundled Annex IV bundle (compliance/) is generated automatically when the compliance: block is present and risk_classificatio… severity **Medium** **Where.** [docs/usermanuals/en/reference/yaml-templates.md:134-136](https://github.com/HodeTech/ForgeLM/blob/cab2947/docs/usermanuals/en/reference/yaml-templates.md#L134-L136) (Turkish mirror: [docs/usermanuals/tr/reference/yaml-templates.md:135-137](https://github.com/HodeTech/ForgeLM/blob/cab2947/docs/usermanuals/tr/reference/yaml-templates.md#L135-L137)) · type: wrong detail **The document says.** 'The bundled Annex IV bundle (compliance/) is generated automatically when the compliance:block is present andrisk_classificationishigh-riskorunacceptable.' **What the code does.** annex_iv_metadata.json is written whenever compliance: is present with any of provider_name/system_name/intended_purpose non-blank, for every risk tier. It is skipped when all three are blank, even for high-risk. **Code references.** [forgelm/compliance.py:1551-1561](https://github.com/HodeTech/ForgeLM/blob/cab2947/forgelm/compliance.py#L1551-L1561), [forgelm/compliance.py:1803-1816](https://github.com/HodeTech/ForgeLM/blob/cab2947/forgelm/compliance.py#L1803-L1816), [forgelm/compliance.py:2215-2236](https://github.com/HodeTech/ForgeLM/blob/cab2947/forgelm/compliance.py#L2215-L2236) **Proposed correction.** "The Annex IV artefact (compliance/annex_iv_metadata.json) is generated whenever the compliance:` block is present and names the provider, system or intended purpose — at every risk tier."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D3B04A-017 (docs, Medium) — docs/usermanuals/en/reference/yaml-templates.md: broken example — grpo_reward_model: "my_reward.score" # optional — falls back to built-in format/length shaping.
    severity Medium
    Where. docs/usermanuals/en/reference/yaml-templates.md:166 (Turkish mirror: docs/usermanuals/tr/reference/yaml-templates.md:167) · type: broken example
    The document says. grpo_reward_model: "my_reward.score" # optional — falls back to built-in format/length shaping.
    What the code does. grpo_reward_model is an HF model id or path. The trainer loads it with AutoTokenizer and AutoModelForSequenceClassification.from_pretrained; it is not a Python callable path. 'my_reward.score' would fail to load at training time (dry-run accepts it).
    Code references. forgelm/config.py:522-530, forgelm/trainer.py:1086-1094, forgelm/trainer.py:1155-1174
    Proposed correction. grpo_reward_model: "OpenAssistant/reward-model-deberta-v3-large-v2" # optional HF sequence-classification reward model; omit to use the built-in rewards
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backlog: knownAlready tracked in docs/roadmapdocumentationImprovements or additions to documentationseverity: mediumReal defect with limited reach or a workaroundsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 3Roadmap wave 3: medium severity, module sweeps

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions