From 07e03bba8872bb43eb0a22081670def874f94f2a Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 10:33:51 +0300 Subject: [PATCH] docs(supply-chain): say that SBOMs are expiring workflow artifacts, not release assets The supply-chain reference, the deployer guide, the change-management SOP and the user-manual pages told an auditor to run `gh release download ... --pattern 'sbom-*'` and to re-emit the SBOM from the git tag and diff it against the one attached to the release. publish.yml only uploads the twelve files as workflow-run artifacts (90-day retention), so there is nothing on a Release page, and the SBOM is the CI runner's `pip list` taken after pytest and, on Linux, three extras were installed: test tooling is listed, there is no dependency graph and no licence data (checked on the real v0.11.0 SBOM: 130 components, pytest and pytest-cov among them). It cannot be reproduced from the tag, because dependencies resolve from PyPI at build time. The pages now give commands that work (`gh run list --workflow publish.yml --branch v --status success`, then `gh run download`; the layout was checked against the v0.11.0 run), say what the file contains, and drop "license extraction in SBOM" from ISO A.5.32, SOC 2 CC9.2 and the statement of applicability. The nightly pip-audit is described as querying PyPI's vulnerability data (its default service; the workflow passes no --vulnerability-service), and "exit 1" in the manual now includes UNKNOWN-severity findings, which the reference page already said. Attaching SBOMs to the release and generating them from a clean runtime-only environment are code changes and stay open. Refs #114 Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/nightly.yml | 2 +- CHANGELOG.md | 10 ++++++ docs/guides/iso_soc2_deployer_guide-tr.md | 26 +++++++++------ docs/guides/iso_soc2_deployer_guide.md | 25 ++++++++++----- docs/qms/sop_change_management-tr.md | 9 ++++-- docs/qms/sop_change_management.md | 9 ++++-- docs/qms/statement_of_applicability-tr.md | 2 +- docs/qms/statement_of_applicability.md | 2 +- docs/reference/iso27001_control_mapping-tr.md | 2 +- docs/reference/iso27001_control_mapping.md | 2 +- .../soc2_trust_criteria_mapping-tr.md | 2 +- docs/reference/soc2_trust_criteria_mapping.md | 2 +- docs/reference/supply_chain_security-tr.md | 32 +++++++++++++------ docs/reference/supply_chain_security.md | 30 +++++++++++------ .../en/operations/iso-soc2-deployer.md | 4 +-- .../usermanuals/en/operations/supply-chain.md | 17 ++++++---- .../tr/operations/iso-soc2-deployer.md | 4 +-- .../usermanuals/tr/operations/supply-chain.md | 17 ++++++---- tools/generate_sbom.py | 5 +-- 19 files changed, 134 insertions(+), 68 deletions(-) diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index ae7d4821..4a6a592c 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -270,7 +270,7 @@ jobs: # the moment a dependency advisory drops, even before the team # re-bumps versions: # - # - pip-audit (transitive CVEs against the OSV / GHSA databases) + # - pip-audit (transitive CVEs against PyPI's vulnerability data, its default service) # - bandit (static-analysis on forgelm/ production code only) # # Severity policy (per tools/check_pip_audit.py + tools/check_bandit.py): diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d44dd02..6f065810 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,16 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Say where the per-release SBOMs actually are and what they contain. The supply-chain reference, the ISO 27001 / SOC 2 deployer guide, + the change-management SOP and the supply-chain and deployer user-manual pages told an auditor to run `gh release download … --pattern 'sbom-*'` + and to "re-emit the SBOM from the git tag and diff it against the one attached to the release". Nothing attaches an SBOM to a + GitHub Release: `publish.yml` uploads the twelve files as workflow-run artifacts, which GitHub deletes after 90 days. The docs now give + the working commands (`gh run list --workflow publish.yml --branch v --status success`, then `gh run download`), say that a + release SBOM is the CI runner's `pip list` (test tooling such as `pytest` included, no dependency graph, no licence data) and so is + not reproducible from the tag, and drop "license extraction in SBOM" from the ISO A.5.32, SOC 2 CC9.2 and statement-of-applicability + rows. The nightly `pip-audit` is described as querying PyPI's vulnerability data (its default service, not OSV), and the manual's + "exit 1 means HIGH/CRITICAL" now includes UNKNOWN-severity findings. English and Turkish. Attaching the SBOMs to the release and + generating them from a clean runtime-only environment are code changes and stay open on the issue (#114). - Fix the KTO bullet on the Alignment overview user-manual page (English and Turkish): it named the fields `(prompt, response, good?)`, but the KTO loader reads `prompt`, `completion` and a boolean `label` (#965). diff --git a/docs/guides/iso_soc2_deployer_guide-tr.md b/docs/guides/iso_soc2_deployer_guide-tr.md index fd52e63e..3d4a2f8e 100644 --- a/docs/guides/iso_soc2_deployer_guide-tr.md +++ b/docs/guides/iso_soc2_deployer_guide-tr.md @@ -34,8 +34,10 @@ ForgeLM kanıtına sahip: (collection_method, annotation_process, known_biases, personal_data_included, dpia_completed). 4. **Supply chain** — yayın etiketi başına yayın matrisinin her (OS - × Python-version) hücresi için emit edilen CycloneDX 1.5 SBOM. - Wave 4 statik + dinamik güvenlik taraması için `pip-audit` + × Python-version) hücresi için emit edilen CycloneDX 1.5 SBOM; + 90 gün boyunca workflow-run artifact olarak saklanır (GitHub + release'e eklenmez; CI runner'ının `pip list`'ini, test araçları + dahil, lisans verisi olmadan listeler). Wave 4 statik + dinamik güvenlik taraması için `pip-audit` nightly + `bandit` CI ekler. ## Denetim gözlem dönemi öncesi setup checklist'i @@ -198,19 +200,25 @@ eğittiğinin kanıtı saymayın. ### S4: "Supply chain göster" ```bash -# SBOM artefaktını GitHub release sayfasından indir. -gh release download v0.5.5 --pattern 'sbom-*' - -# Veya istek üzerine yeniden üret. +# Tag'in publish çalıştırmasından bir SBOM indir. Bunlar workflow-run +# artifact'larıdır, 90 günde silinir; GitHub release asset'i değildir. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v0.11.0 --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --name sbom-ubuntu-latest-py3.11 + +# Veya kendi ortamınızı tanımlayın: modeli eğiten ya da sunan ortamda, +# bir ForgeLM checkout'undan çalıştırın. python3 tools/generate_sbom.py > sbom.json # Son yayınla diff al. diff <(jq -S . sbom-prev.json) <(jq -S . sbom.json) ``` -CycloneDX 1.5 JSON her transitive bağımlılığı purl -(`pkg:pypi/...`) ve sürümle listeler. Dependency-Track CVE -korelasyonu için bunu doğal olarak içe alır. +CycloneDX 1.5 JSON kurulu her paketi purl (`pkg:pypi/...`) ve +sürümle listeler; bağımlılık grafiği ve lisans verisi yoktur. +Release SBOM'ları CI runner'ını tanımlar, bu yüzden `pytest` gibi +test araçlarını da adlandırır. Dependency-Track dosyayı CVE +korelasyonu için doğal olarak içe alır. ### S5: "Erişim kontrolleri göster — yalnız yetkili reviewer'ların model onayladığını nasıl ispatlıyorsunuz?" diff --git a/docs/guides/iso_soc2_deployer_guide.md b/docs/guides/iso_soc2_deployer_guide.md index 3132ef96..26c0e313 100644 --- a/docs/guides/iso_soc2_deployer_guide.md +++ b/docs/guides/iso_soc2_deployer_guide.md @@ -33,7 +33,10 @@ direct ForgeLM evidence: (collection_method, annotation_process, known_biases, personal_data_included, dpia_completed). 4. **Supply chain** — CycloneDX 1.5 SBOM emitted per release tag for - every (OS × Python-version) cell of the publish matrix. Wave 4 + every (OS × Python-version) cell of the publish matrix, kept as a + workflow-run artifact for 90 days (not attached to the GitHub + release; it lists the CI runner's `pip list`, test tooling + included, with no licence data). Wave 4 adds `pip-audit` nightly + `bandit` CI for static + dynamic security scanning. @@ -198,19 +201,25 @@ corpus trained the model. ### Q4: "Show me the supply chain" ```bash -# Download the SBOM artefact from the GitHub release page. -gh release download v0.5.5 --pattern 'sbom-*' - -# Or regenerate on demand. +# Download one SBOM from the tag's publish run. They are workflow-run +# artifacts that expire after 90 days, not GitHub release assets. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v0.11.0 --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --name sbom-ubuntu-latest-py3.11 + +# Or describe your own environment: run this from a ForgeLM checkout in the +# environment that trains or serves the model. python3 tools/generate_sbom.py > sbom.json # Diff against last release. diff <(jq -S . sbom-prev.json) <(jq -S . sbom.json) ``` -The CycloneDX 1.5 JSON lists every transitive dependency with its -purl (`pkg:pypi/...`) and version. Dependency-Track ingests this -natively for CVE correlation. +The CycloneDX 1.5 JSON lists every installed package with its purl +(`pkg:pypi/...`) and version; it has no dependency graph and no licence +data. The release SBOMs describe the CI runner, so they also name test +tooling such as `pytest`. Dependency-Track ingests the file natively for +CVE correlation. ### Q5: "Show me the access controls — how do you prove only authorised reviewers approve models?" diff --git a/docs/qms/sop_change_management-tr.md b/docs/qms/sop_change_management-tr.md index c010fc36..82657489 100644 --- a/docs/qms/sop_change_management-tr.md +++ b/docs/qms/sop_change_management-tr.md @@ -134,9 +134,12 @@ olarak atfeder. 1.5 SBOM üretir. Wave 4 bir determinism contract testi ekler (`tests/test_supply_chain_security.py::TestGenerateSbomDeterministic`). -Drift tespiti: bir release'in SBOM'u `git tag`'inden tekrarlanabilir. -Bir denetçi bunu istek üzerine yeniden emit edip GitHub release'e -ekli artefakta diff alabilir. +Drift tespiti: SBOM, release'in build edildiği ortamın kaydıdır (matris runner'ında +`pip list`); tag'in `publish.yml` çalıştırmasının workflow-run artifact'ı olarak 90 gün +saklanır, GitHub release'e eklenmez. Bağımlılıklar build anında PyPI'dan çözüldüğü için +yalnızca `git tag`'den tekrarlanamaz. İki release'in SBOM'unu `diff` ile karşılaştırın; +dağıtılmış bir ortamı kanıtlamak için orada `python3 tools/generate_sbom.py` ile yeni +bir tane üretin. ## 5. Versiyon İzleme diff --git a/docs/qms/sop_change_management.md b/docs/qms/sop_change_management.md index 79336c3d..6752ff35 100644 --- a/docs/qms/sop_change_management.md +++ b/docs/qms/sop_change_management.md @@ -129,9 +129,12 @@ CycloneDX 1.5 SBOM per (OS × Python-version) on every release tag (`publish.yml`). Wave 4 adds a determinism contract test (`tests/test_supply_chain_security.py::TestGenerateSbomDeterministic`). -Drift detection: a release's SBOM is reproducible from its `git tag`. -An auditor can re-emit it on demand and diff against the artefact -attached to the GitHub release. +Drift detection: the SBOM is a record of the environment the release was built in +(`pip list` on the matrix runner), kept as a workflow-run artifact of the tag's +`publish.yml` run for 90 days; it is not attached to the GitHub release. It is not +reproducible from the `git tag` alone, because dependencies resolve from PyPI at build +time. Compare two releases' SBOMs with `diff`; to evidence a deployed environment, emit +a fresh one there with `python3 tools/generate_sbom.py`. ## 5. Version Tracking diff --git a/docs/qms/statement_of_applicability-tr.md b/docs/qms/statement_of_applicability-tr.md index ac90f3c5..673d620e 100644 --- a/docs/qms/statement_of_applicability-tr.md +++ b/docs/qms/statement_of_applicability-tr.md @@ -59,7 +59,7 @@ operatör-tarafı eylem. | A.5.29 Aksaklık sırasında bilgi güvenliği | YES | `auto_revert` baseline-flip; `model.reverted` event | Base-model retention dokümante et | | A.5.30 İş sürekliliği için ICT hazırlığı | YES | DR planlama | Benimse | | A.5.31 Yasal, kanuni, düzenleyici ve sözleşmesel gerekliliklerin tanımlanması | YES | EU AI Act + GDPR mapping'ler; Annex IV bundle | Kural değişikliklerini izle | -| A.5.32 Fikri mülkiyet hakları | YES | SBOM'da lisans çıkarımı; HF model-card metadata | Per-model lisans review | +| A.5.32 Fikri mülkiyet hakları | YES | HF model-card metadata (SBOM lisans verisi taşımaz) | Per-model lisans review | | A.5.33 Kayıtların korunması | YES | Append-only + HMAC + manifest sidecar; off-site replica | Off-site backup | | A.5.34 Gizlilik ve PII korunması | YES | `forgelm reverse-pii` Md. 15; `forgelm purge` Md. 17; `forgelm audit` PII tespiti + maskeleme | DSAR workflow + DPIA | | A.5.35 Bilgi güvenliğinin bağımsız incelenmesi | YES | Yıllık external denetim | Benimse | diff --git a/docs/qms/statement_of_applicability.md b/docs/qms/statement_of_applicability.md index b78bc887..00a401e7 100644 --- a/docs/qms/statement_of_applicability.md +++ b/docs/qms/statement_of_applicability.md @@ -58,7 +58,7 @@ contribution → deployer-side action. | A.5.29 Information security during disruption | YES | `auto_revert` baseline-flip; `model.reverted` event | Document base-model retention | | A.5.30 ICT readiness for business continuity | YES | DR planning | Adopt | | A.5.31 Identification of legal, statutory, regulatory and contractual requirements | YES | EU AI Act + GDPR mappings; Annex IV bundle | Track rule changes | -| A.5.32 Intellectual property rights | YES | License extraction in SBOM; HF model-card metadata | Per-model license review | +| A.5.32 Intellectual property rights | YES | HF model-card metadata (the SBOM carries no licence data) | Per-model license review | | A.5.33 Protection of records | YES | Append-only + HMAC + manifest; off-site replica | Off-site backup | | A.5.34 Privacy and protection of PII | YES | `forgelm reverse-pii` Article 15; `forgelm purge` Article 17; `forgelm audit` PII detection + masking | DSAR workflow + DPIA | | A.5.35 Independent review of information security | YES | Annual external audit | Adopt | diff --git a/docs/reference/iso27001_control_mapping-tr.md b/docs/reference/iso27001_control_mapping-tr.md index 57e78d96..0c8913b1 100644 --- a/docs/reference/iso27001_control_mapping-tr.md +++ b/docs/reference/iso27001_control_mapping-tr.md @@ -52,7 +52,7 @@ | A.5.29 Aksaklık sırasında bilgi güvenliği | FL-helps | `auto_revert` baseline-flip; `model.reverted` olayı | | A.5.30 İş sürekliliği için ICT hazırlığı | OOS | — | | A.5.31 Yasal, kanuni, düzenleyici ve sözleşmesel gerekliliklerin tanımlanması | FL-helps | EU AI Act + GDPR mappings; Annex IV bundle | -| A.5.32 Fikri mülkiyet hakları | FL-helps | SBOM'da lisans çıkarımı; HF model-card metadata | +| A.5.32 Fikri mülkiyet hakları | FL-helps | HF model-card metadata (SBOM lisans verisi taşımaz) | | A.5.33 Kayıtların korunması | FL | Append-only + HMAC + manifest sidecar | | A.5.34 Gizlilik ve PII korunması | FL | `forgelm reverse-pii` Madde 15; `forgelm purge` Madde 17 (ayrıca bkz. A.8.3) | | A.5.35 Bilgi güvenliğinin bağımsız incelenmesi | OOS | — | diff --git a/docs/reference/iso27001_control_mapping.md b/docs/reference/iso27001_control_mapping.md index 5f1885a2..093062be 100644 --- a/docs/reference/iso27001_control_mapping.md +++ b/docs/reference/iso27001_control_mapping.md @@ -53,7 +53,7 @@ | A.5.29 Information security during disruption | FL-helps | `auto_revert` baseline-flip; `model.reverted` event | | A.5.30 ICT readiness for business continuity | OOS | — | | A.5.31 Identification of legal, statutory, regulatory and contractual requirements | FL-helps | EU AI Act + GDPR mappings; Annex IV bundle | -| A.5.32 Intellectual property rights | FL-helps | License extraction in SBOM; HF model-card metadata | +| A.5.32 Intellectual property rights | FL-helps | HF model-card metadata (the SBOM carries no licence data) | | A.5.33 Protection of records | FL | Append-only + HMAC + manifest sidecar | | A.5.34 Privacy and protection of PII | FL | `forgelm reverse-pii` Article 15; `forgelm purge` Article 17 (see also A.8.3) | | A.5.35 Independent review of information security | OOS | — | diff --git a/docs/reference/soc2_trust_criteria_mapping-tr.md b/docs/reference/soc2_trust_criteria_mapping-tr.md index 07c43f47..7ea6c67e 100644 --- a/docs/reference/soc2_trust_criteria_mapping-tr.md +++ b/docs/reference/soc2_trust_criteria_mapping-tr.md @@ -57,7 +57,7 @@ kategoriler engagement-bazında scoplanır. | CC7.5 | Düzeltici eylemleri tanımlar, geliştirir | `human_approval.rejected`; `sop_change_management.md` | | CC8.1 | Değişimleri yetkilendirir | `forgelm approve` Madde 14 gate; staging dir | | CC9.1 | Riskleri tanımlar, yönetir | `risk_assessment` config + safety eval; `risk_treatment_plan.md` | -| CC9.2 | Tedarikçi + iş ortağı riskini yönetir | SBOM; `loaded` / `unverified` / `local_path` / `unresolved` olarak derecelendirilen dataset Hub commit SHA'sı; temel-model pin'i (`model.revision` → `model_lineage.base_model_revision`); lisans çıkarımı. `model_lineage.component_revisions` altında **sabitlenen ve kaydedilen:** güvenlik sınıflandırıcısı, LLM judge, GRPO ödül modeli, sentetik teacher. **Kapsanmayan:** merge kaynakları ve `forgelm safety-eval` (`--config` almaz) | +| CC9.2 | Tedarikçi + iş ortağı riskini yönetir | SBOM; `loaded` / `unverified` / `local_path` / `unresolved` olarak derecelendirilen dataset Hub commit SHA'sı; temel-model pin'i (`model.revision` → `model_lineage.base_model_revision`). `model_lineage.component_revisions` altında **sabitlenen ve kaydedilen:** güvenlik sınıflandırıcısı, LLM judge, GRPO ödül modeli, sentetik teacher. **Kapsanmayan:** merge kaynakları ve `forgelm safety-eval` (`--config` almaz) | ## Availability (A1.x) diff --git a/docs/reference/soc2_trust_criteria_mapping.md b/docs/reference/soc2_trust_criteria_mapping.md index 57ce25ed..9e2bb753 100644 --- a/docs/reference/soc2_trust_criteria_mapping.md +++ b/docs/reference/soc2_trust_criteria_mapping.md @@ -57,7 +57,7 @@ categories are scoped per-engagement. | CC7.5 | Identifies, develops corrective actions | `human_approval.rejected`; `sop_change_management.md` | | CC8.1 | Authorises changes | `forgelm approve` Article 14 gate; staging dir | | CC9.1 | Identifies, manages risks | `risk_assessment` config + safety eval; `risk_treatment_plan.md` | -| CC9.2 | Manages vendor + business-partner risk | SBOM; dataset Hub commit SHA graded `loaded` / `unverified` / `local_path` / `unresolved`; base-model pin (`model.revision` → `model_lineage.base_model_revision`); license extraction. **Pinned and recorded** under `model_lineage.component_revisions`: safety classifier, LLM judge, GRPO reward model, synthetic teacher. **Not covered:** merge sources, and `forgelm safety-eval` (takes no `--config`) | +| CC9.2 | Manages vendor + business-partner risk | SBOM; dataset Hub commit SHA graded `loaded` / `unverified` / `local_path` / `unresolved`; base-model pin (`model.revision` → `model_lineage.base_model_revision`). **Pinned and recorded** under `model_lineage.component_revisions`: safety classifier, LLM judge, GRPO reward model, synthetic teacher. **Not covered:** merge sources, and `forgelm safety-eval` (takes no `--config`) | ## Availability (A1.x) diff --git a/docs/reference/supply_chain_security-tr.md b/docs/reference/supply_chain_security-tr.md index 0b74c5a2..e6f5c193 100644 --- a/docs/reference/supply_chain_security-tr.md +++ b/docs/reference/supply_chain_security-tr.md @@ -39,7 +39,17 @@ içinde çalışır: ``` Cross-OS test matrisinin her (OS × Python-version) hücresi için bir -SBOM. Artefaktlar GitHub release sayfasından indirilebilir. +SBOM. + +SBOM'lar, release tag'i için çalışan `publish.yml` çalıştırmasının **workflow-run artifact**'larıdır. +GitHub Release sayfasına eklenmezler ve GitHub, deponun saklama süresi (90 gün) dolunca run +artifact'larını siler; ihtiyacınız olanı çalıştırma hâlâ duruyorken indirin ve kayıtlarınızla +birlikte arşivleyin. + +Bir SBOM'un listelediği: matris runner'ında paketlenmiş wheel kurulduktan sonraki `pip list` +çıktısı (bağımlılıkları o anda PyPI'dan çözülür, sabitlenmemiştir), Linux'ta `qlora`, `ingestion` +ve `eval` extras'ı ve `pytest` / `pytest-cov`. Bu yüzden runtime kapanışının yanında test araçlarını +da adlandırır, bağımlılık grafiği taşımaz ve **lisans verisi taşımaz**. ### Determinism contract @@ -52,17 +62,21 @@ Wave 4 / Faz 23 bir determinism testi ekler - `tests/test_supply_chain_security.py::TestGenerateSbomDeterministic::test_two_invocations_produce_same_content` CI'da bunu pin'ler. -Bir denetçi karşılık gelen `git tag`'den SBOM'u yeniden emit edip -GitHub release'e ekli artefaktla diff alabilir; trivial olmayan -herhangi bir fark build ortamında dependency-resolution drift'ine -işaret eder. +Determinizm, aynı ortama özgü bir özelliktir. Aynı `git tag`'den sonradan yapılan yeniden +emit'in bir release'in SBOM'uyla eşleşmesi beklenmez; çünkü o release bağımlılıklarını +build edildiği anda PyPI'dan çözdü. **Sizin** dağıttığınızı kaydetmek için, modeli eğiten ya da +sunan ortamda bir ForgeLM checkout'undan `python3 tools/generate_sbom.py` çalıştırın. ### SBOM'u tüketme ```bash -# Bir release için tüm SBOM'ları indir. v'yi CI'da pinlediğiniz -# tag ile değiştirin; SBOM artefakt formatı release'ler arasında stabildir. -gh release download v --pattern 'sbom-*' +# CI'da pinlediğiniz tag için başarılı publish çalıştırmasını bulun (bir tag'in +# daha önce başarısız olmuş bir çalıştırması olabilir), sonra ondan bir SBOM indirin. +# v'yi tag ile değiştirin. Run artifact'ları 90 günde silinir; on ikisi için +# --pattern 'sbom-*' kullanın. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --name sbom-ubuntu-latest-py3.11 # İnsan inceleme için pretty-print. jq . sbom-ubuntu-latest-py3.11.json | less @@ -110,7 +124,7 @@ Wave 4 / Faz 23 nightly workflow'a `pip-audit` ekler. Davranış: Belirli bir CVE'yi kabul etmek için opt-in ignore dosyasıyla dokümante edin (aşağıdaki Suppression'a bakın); yeşil kalmak için UNKNOWN kovasına güvenmeyin. -- OSV / GHSA veritabanlarını kullanır (pip-audit varsayılanı). +- PyPI'ın güvenlik açığı verisini sorgular (pip-audit'in varsayılan `pypi` servisi; nightly `--vulnerability-service` geçmez). Operatörler aynı tooling'i lokalde kurar: diff --git a/docs/reference/supply_chain_security.md b/docs/reference/supply_chain_security.md index 13cac6c0..527f4799 100644 --- a/docs/reference/supply_chain_security.md +++ b/docs/reference/supply_chain_security.md @@ -39,7 +39,16 @@ publish workflow: ``` One SBOM per (OS × Python-version) cell of the cross-OS test matrix. -The artefacts are downloadable from the GitHub release page. + +The SBOMs are **workflow-run artifacts** of the `publish.yml` run for the release tag. They are +not attached to the GitHub Release page, and GitHub deletes run artifacts when the repository's +retention period ends (90 days), so download what you need while the run is still available and +archive it with your records. + +What an SBOM lists: the output of `pip list` on the matrix runner after it installed the packaged +wheel (its dependencies resolve from PyPI at that moment, unpinned), on Linux the `qlora`, +`ingestion` and `eval` extras, and `pytest` / `pytest-cov`. It therefore names test tooling +beside the runtime closure, carries no dependency graph, and carries **no licence data**. ### Determinism contract @@ -52,17 +61,20 @@ Wave 4 / Faz 23 adds a determinism test (`tests/test_supply_chain_security.py`): - `tests/test_supply_chain_security.py::TestGenerateSbomDeterministic::test_two_invocations_produce_same_content` pins this in CI. -An auditor can re-emit the SBOM from the corresponding `git tag` -and diff against the artefact attached to the GitHub release; any -non-trivial difference indicates dependency-resolution drift in the -build environment. +Determinism is a same-environment property. A later re-emission from the same `git tag` is not +expected to match a release's SBOM, because that release resolved its dependencies from PyPI when +it was built. To record what **you** deploy, run `python3 tools/generate_sbom.py` from a ForgeLM +checkout in the environment that trains or serves the model. ### Consuming the SBOM ```bash -# Download all SBOMs for a release. Replace v with the tag you -# pinned in your CI; the SBOM artefact format is stable across releases. -gh release download v --pattern 'sbom-*' +# Find the successful publish run for the tag you pinned in your CI (a tag can have an +# earlier failed run), then download one SBOM from it. Replace v with the tag. +# Run artifacts expire after 90 days; use --pattern 'sbom-*' for all twelve. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --name sbom-ubuntu-latest-py3.11 # Pretty-print for human review. jq . sbom-ubuntu-latest-py3.11.json | less @@ -108,7 +120,7 @@ Wave 4 / Faz 23 adds `pip-audit` to the nightly workflow. Behaviour: To accept a specific CVE, document it via the opt-in ignore file (see Suppression below); do not rely on the UNKNOWN bucket to stay green. -- Uses the OSV / GHSA databases (pip-audit's default). +- Queries PyPI's vulnerability data (pip-audit's default `pypi` service; the nightly does not pass `--vulnerability-service`). Operators install the same tooling locally: diff --git a/docs/usermanuals/en/operations/iso-soc2-deployer.md b/docs/usermanuals/en/operations/iso-soc2-deployer.md index 862c2748..5e308784 100644 --- a/docs/usermanuals/en/operations/iso-soc2-deployer.md +++ b/docs/usermanuals/en/operations/iso-soc2-deployer.md @@ -14,7 +14,7 @@ Four pillars an ISO / SOC 2 auditor cares about most have direct ForgeLM evidenc 1. **Audit trail** — `audit_log.jsonl` per training run, append-only with HMAC + SHA-256 hash chain + genesis manifest sidecar. `forgelm verify-audit` validates the chain end-to-end. 2. **Change control** — Article 14 staging gate (`forgelm approve` / `reject`) + `human_approval.required/granted/rejected` audit events + `config_hash` (per-run manifest sidecar field) stamped per run. 3. **Data lineage** — `data_provenance.json` (SHA-256 fingerprint + size + mtime, plus the dataset's HF Hub commit SHA **graded by `hf_revision_source`**) + `data_governance_report.json` (collection_method, annotation_process, known_biases, personal_data_included, dpia_completed). Read the grade before the SHA: only `hf_revision_source: loaded` is evidence of the corpus that trained the model. -4. **Supply chain** — CycloneDX 1.5 SBOM emitted per release, `pip-audit` nightly, `bandit` CI for static + dynamic security scanning. +4. **Supply chain** — CycloneDX 1.5 SBOM emitted per release (a workflow-run artifact kept 90 days, not a GitHub release asset), `pip-audit` nightly, `bandit` CI for static + dynamic security scanning. ## The eight audit-floor questions @@ -23,7 +23,7 @@ The deep guide answers each with the exact `jq` / CLI command + the artefact it 1. "Show me the audit trail for every model promotion in the past 90 days" → `forgelm verify-audit` + `jq 'select(.event == "human_approval.granted")'`. 2. "Show me the change-control evidence — who approved this model?" → cross-reference `training.started` + `human_approval.granted` events; two distinct operator IDs prove segregation of duties (ISO A.5.3, SOC 2 CC1.5). 3. "Show me the data lineage" → `data_provenance.json`; `sha256` + `hf_revision` pin the corpus — but only when `hf_revision_source` reads `loaded`. `unverified` means the SHA is a Hub lookup made when the bundle was written, with no coupling to the load; `unresolved` means no SHA was obtained and none is claimed. -4. "Show me the supply chain" → `gh release download "$(gh release view --json tagName -q .tagName)" --pattern 'sbom-*'`; CycloneDX 1.5 JSON. +4. "Show me the supply chain" → `gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success`, then `gh run download --repo HodeTech/ForgeLM --pattern 'sbom-*'` within 90 days; CycloneDX 1.5 JSON (the CI runner's package list, test tooling included, no licence data). 5. "Show me the access controls" → IdP audit log + `FORGELM_OPERATOR` cross-reference. 6. "Show me the encryption posture" → deployer-side substrate (KMS audit log + `data_governance_report.json`). 7. "Show me the incident response" → `audit.classifier_load_failed` + F-compliance-110 strict gate. diff --git a/docs/usermanuals/en/operations/supply-chain.md b/docs/usermanuals/en/operations/supply-chain.md index 7ec468ae..47c50c00 100644 --- a/docs/usermanuals/en/operations/supply-chain.md +++ b/docs/usermanuals/en/operations/supply-chain.md @@ -5,21 +5,24 @@ description: Operator surface for ForgeLM's SBOM + pip-audit + bandit pipeline # Supply Chain -ForgeLM emits a CycloneDX 1.5 SBOM per release tag, and runs `pip-audit` + `bandit` nightly via the `nightly.yml` workflow (also re-run on every release-tag publish). This page is the operator's mental model: what runs when, where to get the artefacts, and how to mirror the same checks locally. +ForgeLM emits a CycloneDX 1.5 SBOM per release tag (as a workflow-run artifact, not a GitHub release asset), and runs `pip-audit` + `bandit` nightly via the `nightly.yml` workflow (also re-run on every release-tag publish). This page is the operator's mental model: what runs when, where to get the artefacts, and how to mirror the same checks locally. ## What runs when | Trigger | Tool | Result | Failure policy | |---|---|---|---| -| Release tag (`v*`) | `tools/generate_sbom.py` | One CycloneDX 1.5 JSON per (OS × Python-version) cell, attached to the GitHub release | SBOM step is pure-stdlib; cannot silently degrade a green release matrix | -| Nightly 03:00 UTC | `pip-audit` (via `tools/check_pip_audit.py`) | OSV / GHSA scan against installed deps | HIGH/CRITICAL → exit 1 + GitHub issue; MEDIUM → `::warning::`; LOW → silent | +| Release tag (`v*`) | `tools/generate_sbom.py` | One CycloneDX 1.5 JSON per (OS × Python-version) cell, uploaded as a workflow-run artifact of the tag's publish run (kept 90 days; not attached to the GitHub release) | SBOM step is pure-stdlib; cannot silently degrade a green release matrix | +| Nightly 03:00 UTC | `pip-audit` (via `tools/check_pip_audit.py`) | PyPI vulnerability-data scan against installed deps | HIGH/CRITICAL or UNKNOWN severity → exit 1 + GitHub issue; MEDIUM → `::warning::`; LOW → silent | | Nightly 03:00 UTC + release tag | `bandit` (via `tools/check_bandit.py`) | Static security scan of `forgelm/` (excludes `tests/`) | HIGH → exit 1; MEDIUM → `::warning::`; LOW → silent | ## Where to get the SBOM ```bash -# All SBOMs for a release. -gh release download "$(gh release view --json tagName -q .tagName)" --pattern 'sbom-*' +# Find the successful publish run for a tag, then download its SBOMs +# (one directory per artifact). Run artifacts expire after 90 days. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --pattern 'sbom-*' # Pretty-print one. jq . sbom-ubuntu-latest-py3.11.json | less @@ -34,7 +37,7 @@ curl -X POST -H "X-Api-Key: $DT_KEY" -H "Content-Type: application/octet-stream" https://deptrack.example.com/api/v1/bom ``` -The SBOM is determinism-pinned — two consecutive emissions on the same Python environment produce content-identical JSON (modulo `serialNumber` and `metadata.timestamp`, both intentionally varying per CycloneDX semantics). +The SBOM is determinism-pinned — two consecutive emissions on the same Python environment produce content-identical JSON (modulo `serialNumber` and `metadata.timestamp`, both intentionally varying per CycloneDX semantics). That is a same-environment property: a release SBOM is the CI runner's `pip list` (test tooling such as `pytest` included, no dependency graph, no licence data), so re-emitting it later from the same tag will not reproduce it. To describe your own environment, run `python3 tools/generate_sbom.py` from a ForgeLM checkout there. ## Mirror the nightly check locally @@ -46,7 +49,7 @@ pip-audit --strict --format json --output /tmp/pip-audit.json python3 tools/check_pip_audit.py /tmp/pip-audit.json ``` -Exit 0 means the same severity policy ForgeLM CI applies passed. Exit 1 means a HIGH/CRITICAL vulnerability needs addressing before the nightly fires. +Exit 0 means the same severity policy ForgeLM CI applies passed. Exit 1 means a HIGH/CRITICAL vulnerability, or a finding whose severity pip-audit does not report (UNKNOWN, which is most of them), needs addressing before the nightly fires. ## Mirror the bandit check locally diff --git a/docs/usermanuals/tr/operations/iso-soc2-deployer.md b/docs/usermanuals/tr/operations/iso-soc2-deployer.md index d96a3898..43957d5f 100644 --- a/docs/usermanuals/tr/operations/iso-soc2-deployer.md +++ b/docs/usermanuals/tr/operations/iso-soc2-deployer.md @@ -14,7 +14,7 @@ ISO / SOC 2 denetçisinin en çok önem verdiği dört sütun, doğrudan ForgeLM 1. **Denetim izi** — eğitim çalışması başına `audit_log.jsonl`, HMAC + SHA-256 hash chain + genesis manifest sidecar ile append-only. `forgelm verify-audit` zinciri uçtan uca doğrular. 2. **Değişiklik kontrolü** — Article 14 staging gate'i (`forgelm approve` / `reject`) + `human_approval.required/granted/rejected` audit olayları + run başına damgalanmış `config_hash` (per-run manifest sidecar field). 3. **Veri lineage** — `data_provenance.json` (SHA-256 fingerprint + size + mtime, ayrıca **`hf_revision_source` ile derecelendirilmiş** dataset HF Hub commit SHA'sı) + `data_governance_report.json` (collection_method, annotation_process, known_biases, personal_data_included, dpia_completed). Dereceyi SHA'dan önce okuyun: modeli eğiten korpusun kanıtı yalnızca `hf_revision_source: loaded`'dır. -4. **Tedarik zinciri** — sürüm başına yayınlanan CycloneDX 1.5 SBOM, gecelik `pip-audit`, statik + dinamik güvenlik tarama için CI'da `bandit`. +4. **Tedarik zinciri** — sürüm başına üretilen CycloneDX 1.5 SBOM (90 gün saklanan bir workflow-run artifact'ı, GitHub release asset'i değil), gecelik `pip-audit`, statik + dinamik güvenlik tarama için CI'da `bandit`. ## Sekiz denetim katı sorusu @@ -23,7 +23,7 @@ Derin rehber her birini tam `jq` / CLI komutu + döndürdüğü artefakt ile yan 1. "Son 90 gündeki her model promotion için audit trail'i göster" → `forgelm verify-audit` + `jq 'select(.event == "human_approval.granted")'`. 2. "Bana change-control kanıtını göster — bu modeli kim onayladı?" → `training.started` + `human_approval.granted` olaylarını çapraz referans alın; iki farklı operator ID, görev ayrılığını (segregation of duties) kanıtlar (ISO A.5.3, SOC 2 CC1.5). 3. "Veri lineage'ını göster" → `data_provenance.json`; `sha256` + `hf_revision` corpus'u pinler — ancak yalnızca `hf_revision_source` `loaded` olduğunda. `unverified`, SHA'nın paket yazılırken yapılan ve yüklemeyle hiçbir bağı olmayan bir Hub sorgusu olduğu anlamına gelir; `unresolved` ise hiçbir SHA alınmadığı ve hiçbir iddiada bulunulmadığı anlamına gelir. -4. "Tedarik zincirini göster" → `gh release download "$(gh release view --json tagName -q .tagName)" --pattern 'sbom-*'`; CycloneDX 1.5 JSON. +4. "Tedarik zincirini göster" → `gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success`, ardından 90 gün içinde `gh run download --repo HodeTech/ForgeLM --pattern 'sbom-*'`; CycloneDX 1.5 JSON (CI runner'ının paket listesi, test araçları dahil, lisans verisi yok). 5. "Erişim kontrollerini göster" → IdP audit log + `FORGELM_OPERATOR` çapraz referansı. 6. "Şifreleme duruşunu göster" → operatör-tarafı substrate (KMS audit log + `data_governance_report.json`). 7. "Olay müdahalesini göster" → `audit.classifier_load_failed` + F-compliance-110 strict gate. diff --git a/docs/usermanuals/tr/operations/supply-chain.md b/docs/usermanuals/tr/operations/supply-chain.md index 00a82beb..a2d13134 100644 --- a/docs/usermanuals/tr/operations/supply-chain.md +++ b/docs/usermanuals/tr/operations/supply-chain.md @@ -5,21 +5,24 @@ description: ForgeLM SBOM + pip-audit + bandit pipeline'ı için operatör yüze # Tedarik Zinciri -ForgeLM sürüm tag'i başına bir CycloneDX 1.5 SBOM yayınlar ve `pip-audit` + `bandit`'i `nightly.yml` workflow'u üzerinden gecelik çalıştırır (her sürüm-tag yayınında da yeniden koşar). Bu sayfa operatörün zihinsel modelidir: ne zaman ne çalışır, artefaktlar nereden alınır ve aynı kontroller lokalde nasıl yansıtılır. +ForgeLM sürüm tag'i başına bir CycloneDX 1.5 SBOM üretir (workflow-run artifact'ı olarak, GitHub release asset'i değil) ve `pip-audit` + `bandit`'i `nightly.yml` workflow'u üzerinden gecelik çalıştırır (her sürüm-tag yayınında da yeniden koşar). Bu sayfa operatörün zihinsel modelidir: ne zaman ne çalışır, artefaktlar nereden alınır ve aynı kontroller lokalde nasıl yansıtılır. ## Ne zaman ne çalışır | Tetikleyici | Araç | Sonuç | Hata politikası | |---|---|---|---| -| Sürüm tag'i (`v*`) | `tools/generate_sbom.py` | (OS × Python-sürüm) hücresi başına bir CycloneDX 1.5 JSON, GitHub release'e iliştirilir | SBOM adımı pure-stdlib; yeşil sürüm matrisini sessizce bozamaz | -| Gecelik 03:00 UTC | `pip-audit` (`tools/check_pip_audit.py` üzerinden) | Kurulu bağımlılıklara karşı OSV / GHSA taraması | HIGH/CRITICAL → exit 1 + GitHub issue; MEDIUM → `::warning::`; LOW → sessiz | +| Sürüm tag'i (`v*`) | `tools/generate_sbom.py` | (OS × Python-sürüm) hücresi başına bir CycloneDX 1.5 JSON, tag'in publish çalıştırmasının workflow-run artifact'ı olarak yüklenir (90 gün saklanır; GitHub release'e iliştirilmez) | SBOM adımı pure-stdlib; yeşil sürüm matrisini sessizce bozamaz | +| Gecelik 03:00 UTC | `pip-audit` (`tools/check_pip_audit.py` üzerinden) | Kurulu bağımlılıklara karşı PyPI güvenlik açığı verisi taraması | HIGH/CRITICAL ya da UNKNOWN severity → exit 1 + GitHub issue; MEDIUM → `::warning::`; LOW → sessiz | | Gecelik 03:00 UTC + sürüm tag'i | `bandit` (`tools/check_bandit.py` üzerinden) | `forgelm/` üzerinde statik güvenlik taraması (`tests/` hariç) | HIGH → exit 1; MEDIUM → `::warning::`; LOW → sessiz | ## SBOM nereden alınır ```bash -# Bir sürüm için tüm SBOM'lar. -gh release download "$(gh release view --json tagName -q .tagName)" --pattern 'sbom-*' +# Bir tag için başarılı publish çalıştırmasını bulun, sonra SBOM'larını indirin +# (artifact başına bir dizin). Run artifact'ları 90 günde silinir. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --pattern 'sbom-*' # Birini güzel-yazdır. jq . sbom-ubuntu-latest-py3.11.json | less @@ -34,7 +37,7 @@ curl -X POST -H "X-Api-Key: $DT_KEY" -H "Content-Type: application/octet-stream" https://deptrack.example.com/api/v1/bom ``` -SBOM determinizm-pinli — aynı Python ortamında ardışık iki yayın content-identical JSON üretir (CycloneDX semantiği gereği kasıtlı olarak değişen `serialNumber` ve `metadata.timestamp` hariç). +SBOM determinizm-pinli — aynı Python ortamında ardışık iki yayın content-identical JSON üretir (CycloneDX semantiği gereği kasıtlı olarak değişen `serialNumber` ve `metadata.timestamp` hariç). Bu, aynı ortama özgü bir özelliktir: release SBOM'u CI runner'ının `pip list`'idir (`pytest` gibi test araçları dahil, bağımlılık grafiği ve lisans verisi yok); aynı tag'den sonradan yeniden emit etmek onu tekrarlamaz. Kendi ortamınızı tanımlamak için orada bir ForgeLM checkout'undan `python3 tools/generate_sbom.py` çalıştırın. ## Gecelik kontrolü lokalde yansıtın @@ -46,7 +49,7 @@ pip-audit --strict --format json --output /tmp/pip-audit.json python3 tools/check_pip_audit.py /tmp/pip-audit.json ``` -Exit 0, ForgeLM CI'nın uyguladığı aynı severity politikasının geçtiği anlamına gelir. Exit 1, gecelik ateşlenmeden önce bir HIGH/CRITICAL açığın ele alınması gerektiği anlamına gelir. +Exit 0, ForgeLM CI'nın uyguladığı aynı severity politikasının geçtiği anlamına gelir. Exit 1, gecelik ateşlenmeden önce bir HIGH/CRITICAL açığın ya da pip-audit'in severity bildirmediği (UNKNOWN; çoğu bulgu böyledir) bir bulgunun ele alınması gerektiği anlamına gelir. ## Bandit kontrolünü lokalde yansıtın diff --git a/tools/generate_sbom.py b/tools/generate_sbom.py index 47b5b5ce..1b32f9dd 100644 --- a/tools/generate_sbom.py +++ b/tools/generate_sbom.py @@ -3,8 +3,9 @@ Reads the installed package set via ``pip list --format=json`` and renders a CycloneDX 1.5 JSON document on stdout. Pure stdlib, cross-platform — used by -the ``publish.yml`` cross-OS matrix to attach a per-(os, python-version) SBOM -artifact to every tagged release. +the ``publish.yml`` cross-OS matrix, which uploads a per-(os, python-version) SBOM +as a workflow-run artifact of every tag's publish run (it is not attached to the +GitHub release, and run artifacts expire after the repository's retention period). This is intentionally a hand-rolled emitter rather than the optional ``cyclonedx-bom`` PyPI package: keeping the dependency footprint at zero means