Repository navigation
[Wave 3] Medium — Docs: docs/guides/iso_soc2_deployer_guide.md #576
Description
Wave 3 · 7 finding(s) in 7 fix package(s). One pass over this document; keep the Turkish mirror in sync.
Findings
-
DOC-D2B03-027 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: misleading — Out of the box: '
audit_log.jsonlper training run, append-only with HMAC + SHA-256 hash chain + genesis manifest sidecar.'
severity Medium
Where.docs/guides/iso_soc2_deployer_guide.md:23-25(Turkish mirror:docs/guides/iso_soc2_deployer_guide-tr.md:23-25) · type: misleading
The document says. Out of the box: 'audit_log.jsonlper training run, append-only with HMAC + SHA-256 hash chain + genesis manifest sidecar.'
What the code does. Per-line HMAC is written only when FORGELM_AUDIT_SECRET is set. Without it the log is unkeyed, and an edit to the last line leaves the chain self-consistent (the README says so). The log lives per output directory and is shared by every run and command that writes there, not per training run.
Code references.forgelm/compliance.py:230-260,forgelm/compliance.py:575-581,forgelm/compliance.py:171-176,forgelm/trainer.py:345-354
Proposed correction. "audit_log.jsonlper output directory, append-only with a SHA-256 hash chain and a genesis manifest sidecar; a per-line HMAC is added only whenFORGELM_AUDIT_SECRETis set (see the checklist)."
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
DOC-D2B03-028 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: misleading — 'Enable
auto_revert: true... so a quality regression rolls back to the baseline model automatically.'
severity Medium · already tracked: KNOWN-OPEN DOCS-03/DOCS-04 (S15 auto-revert) — docs/roadmap/phase-16-trust-surface-hardening.md:760-765
Where.docs/guides/iso_soc2_deployer_guide.md:69-71(Turkish mirror:docs/guides/iso_soc2_deployer_guide-tr.md:71-72) · type: misleading
The document says. 'Enableauto_revert: true... so a quality regression rolls back to the baseline model automatically.'
What the code does. The key isevaluation.auto_revert. On a breach it deletes the saved model directory, logs model.reverted and exits 3; nothing is rolled back or swapped to a baseline. It is also withheld when the safety evaluation produced no evidence.
Code references.forgelm/config.py:1191,forgelm/trainer.py:960-1007,forgelm/trainer.py:1405-1411,forgelm/trainer.py:1458-1466
Proposed correction. "Enableevaluation.auto_revert: truefor production training so a quality regression deletes the fine-tuned output (model.reverted, exit 3) instead of leaving it deployable. ForgeLM does not restore a previous model; keep your last-good model in your registry."
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
DOC-D2B03-029 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: misleading — 'Subscribe nightly SBOM diffs for every release you depend on.'
severity Medium
Where.docs/guides/iso_soc2_deployer_guide.md:87-88(Turkish mirror:docs/guides/iso_soc2_deployer_guide-tr.md:88-89) · type: misleading
The document says. 'Subscribe nightly SBOM diffs for every release you depend on.'
What the code does. No nightly SBOM generation or diff exists. nightly.yml runs tests, wheel smoke, min-deps and the pip-audit+bandit scan only; SBOMs are produced only by the tag-triggered publish workflow.
Code references..github/workflows/nightly.yml:264-347,.github/workflows/publish.yml:104-111
Proposed correction. "Archive the per-release SBOM artifacts for every ForgeLM version you depend on and diff them on each upgrade (see Q4)."
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
DOC-D2B03-030 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: misleading — 'Two distinct operator IDs prove segregation of duties (ISO A.5.3, SOC 2 CC1.5).'
severity Medium
Where.docs/guides/iso_soc2_deployer_guide.md:158-159(Turkish mirror:docs/guides/iso_soc2_deployer_guide-tr.md:158-159) · type: misleading
The document says. 'Two distinct operator IDs prove segregation of duties (ISO A.5.3, SOC 2 CC1.5).'
What the code does.operatoris FORGELM_OPERATOR copied verbatim when set: an unauthenticated, self-asserted environment string. One person can set two different values, so distinct strings do not prove separation without the IdP cross-reference described in Q5.
Code references.forgelm/compliance.py:192-194,forgelm/cli/subcommands/_approve.py:151-153,docs/qms/access_control.md:198-200
Proposed correction. "Two distinct operator IDs are consistent with segregation of duties; prove it by cross-referencing both IDs with your IdP audit log (Q5). ForgeLM does not authenticateFORGELM_OPERATOR."
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
DOC-D2B03-031 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: wrong detail — The
hf_revision_sourcetable lists three values: loaded, unverified and unresolved.
severity Medium
Where.docs/guides/iso_soc2_deployer_guide.md:183-189(Turkish mirror:docs/guides/iso_soc2_deployer_guide-tr.md:185-189) · type: wrong detail
The document says. Thehf_revision_sourcetable lists three values: loaded, unverified and unresolved.
What the code does. A fourth value,local_path, is written for every local file or directory corpus.
Code references.forgelm/compliance.py:880-892,forgelm/compliance.py:1105-1113
Proposed correction. Add a row: "local_path| The corpus is files on disk; no Hub commit exists and none was sought. Evidence issha256(single files only; directories carry no content hash). | Yes for a single file viasha256; no for a directory."
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
DOC-D2B03-032 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: wrong detail — '
forgelm compliance-onlyalways produces this' (unverified).
severity Medium
Where.docs/guides/iso_soc2_deployer_guide.md:188(Turkish mirror:docs/guides/iso_soc2_deployer_guide-tr.md:188) · type: wrong detail
The document says. 'forgelm compliance-onlyalways produces this' (unverified).
What the code does. There is nocompliance-onlysubcommand; the CLI rejects it with exit 2. The no-load export isforgelm --config <cfg> --compliance-export <dir>.
Code references.forgelm/cli/_no_train_modes.py:250-254,forgelm/compliance.py:926-935
Proposed correction. "forgelm --config <cfg> --compliance-export <dir>always produces this, because it writes a bundle without ever reading the corpus."
Turkish mirror. the Turkish page has the same error; fix both in the same PR -
DOC-D2B03-033 (docs, Medium) — docs/guides/iso_soc2_deployer_guide.md: misleading — Article 17:
forgelm purge --row-id "alice@example.com" --corpus data/2026Q2.jsonl --output-dir ./outputserases the subject's data.
severity Medium
Where.docs/guides/iso_soc2_deployer_guide.md:275-284(Turkish mirror:docs/guides/iso_soc2_deployer_guide-tr.md:275-277) · type: misleading
The document says. Article 17:forgelm purge --row-id "alice@example.com" --corpus data/2026Q2.jsonl --output-dir ./outputserases the subject's data.
What the code does. --row-id erases only rows whoseidorrow_idfield equals the value exactly. It is not a search for the identifier. Reproduced on a corpus containing the email in its text: exit 1, 'No row with id=… Refusing to delete', and data.erasure_failed.
Code references.forgelm/cli/subcommands/_purge.py:87,forgelm/cli/subcommands/_purge.py:272-295
Proposed correction. "First locate the subject's rows withforgelm reverse-pii(the Article 15 step above), then erase each one by its rowid:forgelm purge --row-id <row id> --corpus data/2026Q2.jsonl --output-dir ./outputs(add--row-matches allwhen several rows share the id)."
Turkish mirror. the Turkish page has the same error; fix both in the same PR
Done when
- Every finding above is fixed or closed with a reason (tick it here).
- Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.
Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity