Skip to content

[Wave 3] Medium — Docs: docs/guides/iso_soc2_deployer_guide.md #576

Open

Description

@cemililik

Wave 3 · 7 finding(s) in 7 fix package(s). One pass over this document; keep the Turkish mirror in sync.

Findings

  • DOC-D2B03-027 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: misleading — Out of the box: 'audit_log.jsonl per training run, append-only with HMAC + SHA-256 hash chain + genesis manifest sidecar.'
    severity Medium
    Where. docs/guides/iso_soc2_deployer_guide.md:23-25 (Turkish mirror: docs/guides/iso_soc2_deployer_guide-tr.md:23-25) · type: misleading
    The document says. Out of the box: 'audit_log.jsonl per training run, append-only with HMAC + SHA-256 hash chain + genesis manifest sidecar.'
    What the code does. Per-line HMAC is written only when FORGELM_AUDIT_SECRET is set. Without it the log is unkeyed, and an edit to the last line leaves the chain self-consistent (the README says so). The log lives per output directory and is shared by every run and command that writes there, not per training run.
    Code references. forgelm/compliance.py:230-260, forgelm/compliance.py:575-581, forgelm/compliance.py:171-176, forgelm/trainer.py:345-354
    Proposed correction. "audit_log.jsonl per output directory, append-only with a SHA-256 hash chain and a genesis manifest sidecar; a per-line HMAC is added only when FORGELM_AUDIT_SECRET is set (see the checklist)."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D2B03-028 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: misleading — 'Enable auto_revert: true ... so a quality regression rolls back to the baseline model automatically.'
    severity Medium · already tracked: KNOWN-OPEN DOCS-03/DOCS-04 (S15 auto-revert) — docs/roadmap/phase-16-trust-surface-hardening.md:760-765
    Where. docs/guides/iso_soc2_deployer_guide.md:69-71 (Turkish mirror: docs/guides/iso_soc2_deployer_guide-tr.md:71-72) · type: misleading
    The document says. 'Enable auto_revert: true ... so a quality regression rolls back to the baseline model automatically.'
    What the code does. The key is evaluation.auto_revert. On a breach it deletes the saved model directory, logs model.reverted and exits 3; nothing is rolled back or swapped to a baseline. It is also withheld when the safety evaluation produced no evidence.
    Code references. forgelm/config.py:1191, forgelm/trainer.py:960-1007, forgelm/trainer.py:1405-1411, forgelm/trainer.py:1458-1466
    Proposed correction. "Enable evaluation.auto_revert: true for production training so a quality regression deletes the fine-tuned output (model.reverted, exit 3) instead of leaving it deployable. ForgeLM does not restore a previous model; keep your last-good model in your registry."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D2B03-029 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: misleading — 'Subscribe nightly SBOM diffs for every release you depend on.'
    severity Medium
    Where. docs/guides/iso_soc2_deployer_guide.md:87-88 (Turkish mirror: docs/guides/iso_soc2_deployer_guide-tr.md:88-89) · type: misleading
    The document says. 'Subscribe nightly SBOM diffs for every release you depend on.'
    What the code does. No nightly SBOM generation or diff exists. nightly.yml runs tests, wheel smoke, min-deps and the pip-audit+bandit scan only; SBOMs are produced only by the tag-triggered publish workflow.
    Code references. .github/workflows/nightly.yml:264-347, .github/workflows/publish.yml:104-111
    Proposed correction. "Archive the per-release SBOM artifacts for every ForgeLM version you depend on and diff them on each upgrade (see Q4)."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D2B03-030 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: misleading — 'Two distinct operator IDs prove segregation of duties (ISO A.5.3, SOC 2 CC1.5).'
    severity Medium
    Where. docs/guides/iso_soc2_deployer_guide.md:158-159 (Turkish mirror: docs/guides/iso_soc2_deployer_guide-tr.md:158-159) · type: misleading
    The document says. 'Two distinct operator IDs prove segregation of duties (ISO A.5.3, SOC 2 CC1.5).'
    What the code does. operator is FORGELM_OPERATOR copied verbatim when set: an unauthenticated, self-asserted environment string. One person can set two different values, so distinct strings do not prove separation without the IdP cross-reference described in Q5.
    Code references. forgelm/compliance.py:192-194, forgelm/cli/subcommands/_approve.py:151-153, docs/qms/access_control.md:198-200
    Proposed correction. "Two distinct operator IDs are consistent with segregation of duties; prove it by cross-referencing both IDs with your IdP audit log (Q5). ForgeLM does not authenticate FORGELM_OPERATOR."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D2B03-031 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: wrong detail — The hf_revision_source table lists three values: loaded, unverified and unresolved.
    severity Medium
    Where. docs/guides/iso_soc2_deployer_guide.md:183-189 (Turkish mirror: docs/guides/iso_soc2_deployer_guide-tr.md:185-189) · type: wrong detail
    The document says. The hf_revision_source table lists three values: loaded, unverified and unresolved.
    What the code does. A fourth value, local_path, is written for every local file or directory corpus.
    Code references. forgelm/compliance.py:880-892, forgelm/compliance.py:1105-1113
    Proposed correction. Add a row: "local_path | The corpus is files on disk; no Hub commit exists and none was sought. Evidence is sha256 (single files only; directories carry no content hash). | Yes for a single file via sha256; no for a directory."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D2B03-032 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: wrong detail — 'forgelm compliance-only always produces this' (unverified).
    severity Medium
    Where. docs/guides/iso_soc2_deployer_guide.md:188 (Turkish mirror: docs/guides/iso_soc2_deployer_guide-tr.md:188) · type: wrong detail
    The document says. 'forgelm compliance-only always produces this' (unverified).
    What the code does. There is no compliance-only subcommand; the CLI rejects it with exit 2. The no-load export is forgelm --config <cfg> --compliance-export <dir>.
    Code references. forgelm/cli/_no_train_modes.py:250-254, forgelm/compliance.py:926-935
    Proposed correction. "forgelm --config <cfg> --compliance-export <dir> always produces this, because it writes a bundle without ever reading the corpus."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

  • DOC-D2B03-033 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: misleading — Article 17: forgelm purge --row-id "alice@example.com" --corpus data/2026Q2.jsonl --output-dir ./outputs erases the subject's data.
    severity Medium
    Where. docs/guides/iso_soc2_deployer_guide.md:275-284 (Turkish mirror: docs/guides/iso_soc2_deployer_guide-tr.md:275-277) · type: misleading
    The document says. Article 17: forgelm purge --row-id "alice@example.com" --corpus data/2026Q2.jsonl --output-dir ./outputs erases the subject's data.
    What the code does. --row-id erases only rows whose id or row_id field equals the value exactly. It is not a search for the identifier. Reproduced on a corpus containing the email in its text: exit 1, 'No row with id=… Refusing to delete', and data.erasure_failed.
    Code references. forgelm/cli/subcommands/_purge.py:87, forgelm/cli/subcommands/_purge.py:272-295
    Proposed correction. "First locate the subject's rows with forgelm reverse-pii (the Article 15 step above), then erase each one by its row id: forgelm purge --row-id <row id> --corpus data/2026Q2.jsonl --output-dir ./outputs (add --row-matches all when several rows share the id)."
    Turkish mirror. the Turkish page has the same error; fix both in the same PR

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backlog: knownAlready tracked in docs/roadmapdocumentationImprovements or additions to documentationseverity: mediumReal defect with limited reach or a workaroundsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 3Roadmap wave 3: medium severity, module sweeps

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions