From 3dec5721312205aaa597cfedca68ce4224c9278c Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 10:36:59 +0300 Subject: [PATCH 1/5] docs(webhooks): say that Teams and Discord need a relay, and fix three wrong webhook statements The webhooks user-manual page (English and Turkish) said "Slack, Teams and Discord all accept the payload directly". The body is a Slack-style `attachments` block with no top-level text, content or embeds. Discord requires content or embeds and answers HTTP 400; Office 365 Connectors for Teams were retired in May 2026 and Teams Workflows webhooks expect an Adaptive Card. The page now shows the real payload (copied from the schema page, which was already right), says Slack accepts the attachments block and that Teams and Discord need a relay, and says a rejected delivery is only a WARNING. Three neighbouring statements were wrong too: - `url_env` was said to override `url`; `_resolve_url` reads `url_env` only when `url` is empty, so `url` wins. - The `notify_on_failure` description said "any non-zero exit"; it gates training.failure and training.reverted. - The access-control SOP listed five webhook events (there are eight), said every payload carries FORGELM_OPERATOR (no payload field does) and named a "Teams connector token". The field descriptions in config.py are reworded in place, so the module's line budget is unchanged. Refs #112 Co-Authored-By: Claude Sonnet 5.5 --- CHANGELOG.md | 7 ++ docs/qms/access_control-tr.md | 31 +++++---- docs/qms/access_control.md | 32 ++++----- docs/reference/configuration-tr.md | 6 +- docs/reference/configuration.md | 6 +- docs/usermanuals/en/operations/webhooks.md | 67 ++++++++++++------- .../usermanuals/en/reference/configuration.md | 4 +- docs/usermanuals/tr/operations/webhooks.md | 66 +++++++++++------- .../usermanuals/tr/reference/configuration.md | 4 +- forgelm/config.py | 8 +-- 10 files changed, 139 insertions(+), 92 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d44dd02..c8e65785 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,13 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Stop saying that Microsoft Teams and Discord accept the webhook payload directly. The user manual (English and Turkish) and the + `webhook` field descriptions said all three ingest it; the body is a Slack-style `attachments` block with no top-level `text`, + `content` or `embeds`. Discord needs `content` or `embeds` and answers HTTP 400; Teams retired Office 365 Connectors in May 2026 and + its Workflows webhooks expect an Adaptive Card. Both need a relay. The page now shows the real payload, and the access-control SOP + no longer claims that a payload carries `FORGELM_OPERATOR` (it does not) or lists five events (there are eight). The `url_env` + description said it overrides `url`; it is read only when `url` is empty, so `url` wins. A formatter for Teams and Discord is a + separate, larger change and stays open on the issue (#112). - Fix the KTO bullet on the Alignment overview user-manual page (English and Turkish): it named the fields `(prompt, response, good?)`, but the KTO loader reads `prompt`, `completion` and a boolean `label` (#965). diff --git a/docs/qms/access_control-tr.md b/docs/qms/access_control-tr.md index 13341c3a..4ad7f9fb 100644 --- a/docs/qms/access_control-tr.md +++ b/docs/qms/access_control-tr.md @@ -230,20 +230,23 @@ webhook: ForgeLM webhook gövdelerini HMAC ile **imzalamaz** — `WebhookConfig` üzerinde `webhook.secret_env` alanı yoktur (bkz. -`forgelm/config.py::WebhookConfig`). Hedef-tarafı atıf (a) HTTPS + -`webhook.url_env` üzerinden URL-gizliliğine, (b) curated payload -içinde taşınan `FORGELM_OPERATOR` kimliğine, ve (c) alıcı sistemin -kendi bearer-token / imzalı-istek kontrollerine (Slack signing -secret, Teams connector token) düşer. - -Webhook wire-format olay sözlüğü (5 olay): `training.start`, -`training.success`, `training.failure`, `training.reverted`, -`approval.required` — Phase 8. In-process notifier metot adları -`notify_*` öneki kullanır (`notify_start`, `notify_success`, -`notify_failure`, `notify_reverted`, `notify_awaiting_approval`) -ve bu beş wire olayına dispatch eder. Her payload `FORGELM_OPERATOR` -kimliğini taşır böylece alıcı sistem (Slack, Teams, custom -incident-management) bildirimi attribute edebilir. +`forgelm/config.py::WebhookConfig`). Hedef-tarafı özgünlük (a) HTTPS + +`webhook.url_env` üzerinden URL-gizliliğine ve (b) alıcı sistemin +kendi bearer-token / imzalı-istek kontrollerine (örneğin Slack'in +signing secret'ı) düşer. Webhook payload'u `FORGELM_OPERATOR` +kimliğini **taşımaz**; bir modeli kimin çalıştırdığı veya onayladığı +bildirimden değil, audit log'dan okunur. + +Webhook wire-format olay sözlüğü sekiz olaydır: beş tek-aşamalı +yaşam-döngüsü olayı `training.start`, `training.success`, +`training.failure`, `training.reverted` ve `approval.required`, ayrıca +çok-aşamalı bir koşunun bunlarla birlikte yaydığı üç `pipeline.*` +olayı (`pipeline.started`, `pipeline.completed`, +`pipeline.stage_reverted`). In-process notifier metotları `notify_*` +öneki kullanır (`notify_start`, `notify_success`, `notify_failure`, +`notify_reverted`, `notify_awaiting_approval`, +`notify_pipeline_started`, `notify_pipeline_completed`, +`notify_pipeline_reverted`) ve bu wire olaylarına dispatch eder. ## 8. Doğrulama checklist'i diff --git a/docs/qms/access_control.md b/docs/qms/access_control.md index e3cb1aed..ad5d97c1 100644 --- a/docs/qms/access_control.md +++ b/docs/qms/access_control.md @@ -228,21 +228,23 @@ webhook: ForgeLM does **not** HMAC-sign webhook bodies — there is no `webhook.secret_env` field on `WebhookConfig` (see -`forgelm/config.py::WebhookConfig`). Destination-side attribution falls to (a) -HTTPS + URL-secrecy via `webhook.url_env`, (b) the -`FORGELM_OPERATOR` identity carried inside the curated payload, and -(c) the receiving system's own bearer-token / signed-request -controls (Slack signing secret, Teams connector token). - -The webhook wire-format event vocabulary (5 events) is -`training.start`, `training.success`, `training.failure`, -`training.reverted`, and `approval.required` — Phase 8. The -in-process notifier method names use the `notify_*` prefix -(`notify_start`, `notify_success`, `notify_failure`, -`notify_reverted`, `notify_awaiting_approval`) and dispatch to those -five wire events. Every payload carries the `FORGELM_OPERATOR` -identity so the receiving system (Slack, Teams, custom -incident-management) can attribute the notification. +`forgelm/config.py::WebhookConfig`). Destination-side authenticity falls to +(a) HTTPS + URL-secrecy via `webhook.url_env` and (b) the receiving +system's own bearer-token / signed-request controls (for example Slack's +signing secret). A webhook payload does **not** carry the +`FORGELM_OPERATOR` identity; attribution of who ran or approved a model +comes from the audit log, not from the notification. + +The webhook wire-format event vocabulary has eight events: the five +single-stage lifecycle events `training.start`, `training.success`, +`training.failure`, `training.reverted` and `approval.required`, plus the +three `pipeline.*` events (`pipeline.started`, `pipeline.completed`, +`pipeline.stage_reverted`) that a multi-stage run emits alongside them. The +in-process notifier methods use the `notify_*` prefix (`notify_start`, +`notify_success`, `notify_failure`, `notify_reverted`, +`notify_awaiting_approval`, `notify_pipeline_started`, +`notify_pipeline_completed`, `notify_pipeline_reverted`) and dispatch to +those wire events. ## 8. Verification checklist diff --git a/docs/reference/configuration-tr.md b/docs/reference/configuration-tr.md index b78fa07f..3535ecb0 100644 --- a/docs/reference/configuration-tr.md +++ b/docs/reference/configuration-tr.md @@ -433,11 +433,11 @@ kaynağı kaydedilmemiştir. | Alan | Tip | Varsayılan | Açıklama | |------|-----|-----------|----------| -| `url` | string | `null` | Webhook hedef URL | -| `url_env` | string | `null` | URL'yi içeren ortam değişkeni adı | +| `url` | string | `null` | Webhook hedef URL'si: bir Slack incoming webhook'u ya da bir relay (Teams ve Discord payload'u doğrudan reddeder) | +| `url_env` | string | `null` | URL'yi içeren ortam değişkeni adı; yalnızca `url` boşken okunur | | `notify_on_start` | bool | `true` | Eğitim başlangıcında bildir | | `notify_on_success` | bool | `true` | Başarıda bildir | -| `notify_on_failure` | bool | `true` | Hata durumunda bildir | +| `notify_on_failure` | bool | `true` | Eğitim hata verdiğinde ya da eğitim sonrası bir kapı modeli geri aldığında bildir | | `timeout` | int | `10` | HTTP istek zaman aşımı (saniye). Notifier ≥ 1s'ye clamp'ler. v0.5.5'te varsayılan 10s'ye çıkarıldı (önceden 5s'di) — Slack/Teams gateway gecikme atışları production'da düzenli olarak 5s'yi aşıyor ve bir webhook zaman aşımı audit chain'i sessizce zayıflatıyor (webhook arızası best-effort). | | `allow_private_destinations` | bool | `false` | RFC1918 / loopback / link-local hedeflere webhook gönderimine izin verir (cluster içi Slack proxy, on-prem Teams gateway gibi). Varsayılan yalnızca genel internet — SSRF koruması | | `require_https` | bool | `false` | TLS-only zorlama. `true`, plaintext bir `http://` URL'ini reddeder (SSRF chokepoint raise eder; POST atlanır), warn-and-send yerine. Varsayılan `false`, warn-then-send davranışını korur | diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 0f83d16e..a8880b2a 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -292,11 +292,11 @@ silently extend the retention horizon by re-using a stale workspace. | Field | Type | Default | Description | |-------|------|---------|-------------| -| `url` | string | `null` | Webhook destination URL | -| `url_env` | string | `null` | Env var name containing URL | +| `url` | string | `null` | Webhook destination URL: a Slack incoming webhook, or a relay (Teams and Discord reject the payload directly) | +| `url_env` | string | `null` | Env var name containing the URL; read only when `url` is empty | | `notify_on_start` | bool | `true` | Notify on training start | | `notify_on_success` | bool | `true` | Notify on success | -| `notify_on_failure` | bool | `true` | Notify on failure | +| `notify_on_failure` | bool | `true` | Notify when training raises or a post-training gate reverts the model | | `timeout` | int | `10` | HTTP request timeout (seconds). Clamped to ≥ 1s by the notifier. Default raised to 10s in v0.5.5 (was 5s) — Slack/Teams gateway latency spikes regularly cross 5s in production, and a webhook timeout silently degrades the audit chain (webhook failure is best-effort). | | `allow_private_destinations` | bool | `false` | Opt in to webhooks pointing at RFC1918 / loopback / link-local hosts (in-cluster Slack proxy, on-prem Teams gateway). Defaults to public-internet only — SSRF guard | | `require_https` | bool | `false` | TLS-only enforcement. `true` refuses a plaintext `http://` URL (the SSRF chokepoint raises; the POST is skipped) instead of warn-and-send. Default `false` preserves warn-then-send | diff --git a/docs/usermanuals/en/operations/webhooks.md b/docs/usermanuals/en/operations/webhooks.md index d7cf7107..52e76eb3 100644 --- a/docs/usermanuals/en/operations/webhooks.md +++ b/docs/usermanuals/en/operations/webhooks.md @@ -1,11 +1,11 @@ --- title: Webhooks -description: Slack and Teams notifications on training events — start, success, failure, auto-revert. +description: Slack-format webhook notifications on training events — start, success, failure, auto-revert. Teams and Discord need a relay. --- # Webhooks -ForgeLM fires structured webhooks on training milestones. Wire them into Slack, Teams, or any incident tool that accepts JSON payloads — get the right context to the right humans without anyone watching a log. +ForgeLM fires structured webhooks on training milestones. Wire them into Slack, or into any incident tool or relay that accepts this JSON payload (Microsoft Teams and Discord need a relay; see below) — get the right context to the right humans without anyone watching a log. ## Quick example @@ -17,8 +17,9 @@ webhook: notify_on_failure: true # default true (covers training.failure + training.reverted) ``` -The notifier emits a generic JSON payload — Slack and Teams ingest it directly via -their incoming-webhook endpoints. Per-event subscription is not currently +The notifier emits one generic JSON payload with a Slack-style `attachments` block. +Slack's incoming webhooks accept it; Microsoft Teams and Discord do not (see +"Slack / Teams / Discord ingestion" below). Per-event subscription is not currently configurable; toggle the three `notify_on_*` flags to coarse-control which lifecycle events fire. @@ -50,40 +51,56 @@ surface mirrored in the ## Payload shape -Single generic JSON shape — Slack / Teams / Discord all accept it -directly via incoming-webhook endpoints; ForgeLM does **not** wrap it in -provider-specific templates: +One generic JSON shape. Every payload carries `event`, `run_name`, `status`, `metrics`, `reason`, +`model_path` and an `attachments` array with one Slack-style block (`title`, `text`, `color`). +ForgeLM does **not** wrap it in provider-specific templates, and it has no top-level `text`, +`content` or `embeds` field: ```json { "event": "training.reverted", - "run_name": "customer-support-v1.2.0", + "run_name": "llama3-support-sft", "status": "reverted", - "reason": "safety regression: S5 hate-speech +0.18 over baseline" + "metrics": {}, + "reason": "Safety gate failed: unsafe_rate 0.08 exceeds max_unsafe_rate 0.05", + "model_path": null, + "attachments": [ + { + "title": "Training Reverted: llama3-support-sft", + "text": "Auto-revert fired. Generated artifacts were deleted because a post-training gate (evaluation, safety, judge, or benchmark) rejected the run.\n\nReason: Safety gate failed: unsafe_rate 0.08 exceeds max_unsafe_rate 0.05", + "color": "#ff9900" + } + ] } ``` -Payload keys vary by event; the full per-event field list is in the -[Audit Event Catalog on GitHub](https://github.com/HodeTech/ForgeLM/blob/main/docs/reference/audit_event_catalog.md) -under the *Webhook lifecycle events* table. +The `pipeline.*` events add a few fields. The full per-event field list is in the +[Webhook Payload Schema on GitHub](https://github.com/HodeTech/ForgeLM/blob/main/docs/reference/webhook_schema.md). ## Slack / Teams / Discord ingestion -The single generic JSON payload shown above is what Slack, Teams, and Discord -all expect on their incoming-webhook endpoints. There are **no per-provider +The payload above is a Slack-style body. There are **no per-provider templates** in `WebhookConfig` (no `template:`, no `events:` allow-list, no `channel:` / `mention_on_failure:` formatting knobs, no per-destination fan-out array). Routing and formatting happen on the receiving side: -- **Slack** — paste the payload into a Slack workflow or incoming-webhook - app; Slack renders the JSON's top-level fields. To get a richer - formatted card, point the webhook at a relay (Slack workflow / AWS - Lambda / your own gateway) that translates the ForgeLM payload into - Slack Block Kit. -- **Microsoft Teams** — similar pattern. Teams renders incoming JSON - natively but the visual is plain; for MessageCard / Adaptive Card - formatting, run a relay. -- **Discord** — accepts the JSON directly via the bot/webhook URL. +- **Slack** — an incoming webhook accepts the body and shows the + `attachments` block (title, text, colour). That is the legacy attachment + format, which Slack treats as secondary; for a richer card point the + webhook at a relay (Slack workflow / AWS Lambda / your own gateway) that + translates the ForgeLM payload into Slack Block Kit. +- **Microsoft Teams** — not accepted directly. Office 365 Connectors were + retired in May 2026, and a Teams Workflows (Power Automate) webhook expects + an Adaptive Card message, which this payload is not. Run a relay that wraps + the payload in an Adaptive Card. +- **Discord** — not accepted directly. A Discord webhook needs `content` or + `embeds` (or a file or poll) in the body, and ForgeLM sends none of them, so + Discord answers with HTTP 400. Run a relay that maps `attachments[0]` to + `content` or an embed. + +A receiver that rejects a delivery does not fail the run, and nothing is paged: +the rejection is a `WARNING` in the run's log (see *Common pitfalls*). Send a +test event to a new destination before relying on it. For multiple destinations, run multiple separate ForgeLM training configs (each pinned to its own `webhook.url_env`) or fan out from a @@ -97,7 +114,7 @@ Real `WebhookConfig` (see `forgelm/config.py::WebhookConfig`): | Field | Default | Notes | |---|---|---| | `url` | `null` | Inline URL — prefer `url_env` for secret hygiene. | -| `url_env` | `null` | Env-var name carrying the URL. Overrides `url` when set. | +| `url_env` | `null` | Env-var name carrying the URL. Read only when `url` is empty; when both are set, `url` wins. | | `notify_on_start` | `true` | Gates `training.start` AND `pipeline.started`. | | `notify_on_success` | `true` | Gates `training.success`, `approval.required`, AND a successful `pipeline.completed`. | | `notify_on_failure` | `true` | Gates `training.failure`, `training.reverted`, `pipeline.stage_reverted`, AND a failing `pipeline.completed`. | @@ -117,7 +134,7 @@ live outside ForgeLM. - **TLS strongly recommended.** ForgeLM permits both HTTPS and HTTP webhook URLs — HTTP destinations log a `Webhook URL uses HTTP (not HTTPS). Data will be sent unencrypted.` warning but are not rejected by default (see `forgelm/webhook.py` `_send`). On a regulated estate set `webhook.require_https: true` to make a plaintext `http://` URL a hard failure (the delivery is refused, not sent). Pin `https://` URLs in production. - **Curated payload.** ForgeLM never includes raw training data, full configs, or unredacted PII in webhook payloads. The notifier wraps a fixed-shape JSON; there is no `webhook.redact` toggle because there's nothing user-controllable to redact. - **Server-Side Request Forgery (SSRF) guard.** ForgeLM blocks webhook URLs pointing at internal IPs — RFC 1918, loopback, link-local (incl. cloud IMDS at `169.254.169.254`), RFC 6598 Shared Address Space / Carrier-Grade-NAT (`100.64.0.0/10`, which also covers Alibaba Cloud ECS's IMDS at `100.100.100.200` — the stdlib `ipaddress` predicates don't flag that block as private or reserved on their own), plus reserved and multicast destinations — unless you explicitly opt-in with `webhook.allow_private_destinations: true`. This prevents misconfigured runs from probing your internal network or leaking the payload to a cloud metadata service. -- **No HMAC body signing.** ForgeLM does not sign webhook bodies — destination-side authenticity falls to TLS + URL secrecy via `url_env` plus the receiving system's bearer-token / signed-request controls (Slack signing secret, Teams connector token). +- **No HMAC body signing.** ForgeLM does not sign webhook bodies — destination-side authenticity falls to TLS + URL secrecy via `url_env` plus the receiving system's bearer-token / signed-request controls (for example Slack's signing secret). ## Common pitfalls diff --git a/docs/usermanuals/en/reference/configuration.md b/docs/usermanuals/en/reference/configuration.md index c4ba73c2..d14da1e9 100644 --- a/docs/usermanuals/en/reference/configuration.md +++ b/docs/usermanuals/en/reference/configuration.md @@ -262,8 +262,8 @@ compliance: ```yaml webhook: - url: null # Slack / Teams / Discord / custom; prefer url_env - url_env: null # env var carrying the webhook URL + url: null # Slack incoming webhook, or a relay for Teams / Discord; prefer url_env + url_env: null # env var carrying the webhook URL; read only when url is empty notify_on_start: true notify_on_success: true notify_on_failure: true diff --git a/docs/usermanuals/tr/operations/webhooks.md b/docs/usermanuals/tr/operations/webhooks.md index 6f80a377..efe79dab 100644 --- a/docs/usermanuals/tr/operations/webhooks.md +++ b/docs/usermanuals/tr/operations/webhooks.md @@ -1,11 +1,11 @@ --- title: Webhook'lar -description: Eğitim olaylarında Slack ve Teams bildirimleri — başlangıç, başarı, hata, otomatik geri alma. +description: Eğitim olaylarında Slack biçiminde webhook bildirimleri — başlangıç, başarı, hata, otomatik geri alma. Teams ve Discord için relay gerekir. --- # Webhook'lar -ForgeLM eğitim dönüm noktalarında yapılandırılmış webhook fırlatır. Bunları Slack, Teams veya JSON payload kabul eden herhangi bir incident aracına bağlayın — hiç kimsenin log izlemesine gerek kalmadan doğru bağlamı doğru insanlara getirin. +ForgeLM eğitim dönüm noktalarında yapılandırılmış webhook fırlatır. Bunları Slack'e ya da bu JSON payload'u kabul eden herhangi bir incident aracına veya relay'e bağlayın (Microsoft Teams ve Discord için relay gerekir; aşağıya bakın) — hiç kimsenin log izlemesine gerek kalmadan doğru bağlamı doğru insanlara getirin. ## Hızlı örnek @@ -17,8 +17,9 @@ webhook: notify_on_failure: true # default true (training.failure + training.reverted'i kapsar) ``` -Notifier generic JSON payload yayar — Slack ve Teams bunu incoming-webhook -uçlarından doğrudan kabul eder. Per-event abonelik şu an konfigüre +Notifier, Slack tarzı bir `attachments` bloğu taşıyan tek bir generic JSON payload yayar. +Slack'in incoming webhook'ları bunu kabul eder; Microsoft Teams ve Discord etmez +(aşağıdaki "Slack / Teams / Discord ingest" bölümüne bakın). Per-event abonelik şu an konfigüre edilemiyor; üç `notify_on_*` flag'iyle hangi yaşam-döngüsü olaylarının fırlatılacağı kaba ayarlanır. @@ -50,41 +51,58 @@ ile aynalanan kanonik yüzeydir: ## Payload yapısı -Tek generic JSON şekil — Slack / Teams / Discord hepsi bunu incoming-webhook -uçlarından doğrudan kabul eder; ForgeLM provider'a-özel template'lerle -sarmalama **yapmaz**: +Tek generic JSON şekil. Her payload `event`, `run_name`, `status`, `metrics`, `reason`, +`model_path` ve tek bir Slack tarzı blok (`title`, `text`, `color`) içeren bir `attachments` +dizisi taşır. ForgeLM provider'a-özel template'lerle sarmalama **yapmaz** ve üst düzeyde +`text`, `content` veya `embeds` alanı yoktur: ```json { "event": "training.reverted", - "run_name": "customer-support-v1.2.0", + "run_name": "llama3-support-sft", "status": "reverted", - "reason": "safety regression: S5 hate-speech +0.18 over baseline" + "metrics": {}, + "reason": "Safety gate failed: unsafe_rate 0.08 exceeds max_unsafe_rate 0.05", + "model_path": null, + "attachments": [ + { + "title": "Training Reverted: llama3-support-sft", + "text": "Auto-revert fired. Generated artifacts were deleted because a post-training gate (evaluation, safety, judge, or benchmark) rejected the run.\n\nReason: Safety gate failed: unsafe_rate 0.08 exceeds max_unsafe_rate 0.05", + "color": "#ff9900" + } + ] } ``` -Payload anahtarları event'e göre değişir; tam per-event alan listesi -[GitHub'daki Audit Event Kataloğu](https://github.com/HodeTech/ForgeLM/blob/main/docs/reference/audit_event_catalog.md) -*Webhook lifecycle events* tablosundadır. +`pipeline.*` event'leri birkaç alan ekler. Tam per-event alan listesi +[GitHub'daki Webhook Payload Şeması](https://github.com/HodeTech/ForgeLM/blob/main/docs/reference/webhook_schema.md) +sayfasındadır. ## Slack / Teams / Discord ingest -Yukarıdaki tek generic JSON payload, Slack, Teams ve Discord'un -incoming-webhook endpoint'lerinde beklediği şeydir. `WebhookConfig`'te +Yukarıdaki payload Slack tarzı bir gövdedir. `WebhookConfig`'te **provider başına template yoktur** (no `template:`, no `events:` allow-list, no `channel:` / `mention_on_failure:` formatlama knob'ları, no per-destination fan-out array). Routing ve formatlama alıcı tarafta yapılır: -- **Slack** — payload'u Slack workflow veya incoming-webhook - uygulamasına yapıştırın; Slack JSON'un top-level alanlarını render - eder. Daha zengin formatlanmış bir kart için, webhook'u ForgeLM - payload'unu Slack Block Kit'e çeviren bir relay'e (Slack +- **Slack** — incoming webhook gövdeyi kabul eder ve `attachments` + bloğunu (başlık, metin, renk) gösterir. Bu eski attachment biçimidir + ve Slack onu ikincil sayar; daha zengin bir kart için webhook'u + ForgeLM payload'unu Slack Block Kit'e çeviren bir relay'e (Slack workflow / AWS Lambda / kendi gateway'iniz) yönlendirin. -- **Microsoft Teams** — benzer pattern. Teams gelen JSON'u natively - render eder ama görsel düz; MessageCard / Adaptive Card - formatlama için bir relay çalıştırın. -- **Discord** — JSON'u doğrudan bot/webhook URL üzerinden kabul eder. +- **Microsoft Teams** — doğrudan kabul edilmez. Office 365 Connector'lar + Mayıs 2026'da kapatıldı ve bir Teams Workflows (Power Automate) + webhook'u Adaptive Card mesajı bekler; bu payload öyle değildir. + Payload'u Adaptive Card'a saran bir relay çalıştırın. +- **Discord** — doğrudan kabul edilmez. Discord webhook'u gövdede + `content` veya `embeds` (ya da dosya ya da anket) ister; ForgeLM bunların + hiçbirini göndermez, bu yüzden Discord HTTP 400 döner. `attachments[0]`'ı + `content`'e veya bir embed'e eşleyen bir relay çalıştırın. + +Teslimatı reddeden bir alıcı run'ı başarısız etmez ve kimseye alarm gitmez: +ret, run log'unda bir `WARNING`'dir (bkz. *Sık hatalar*). Yeni bir hedefe güvenmeden +önce ona bir test event'i gönderin. Birden çok hedef için, her biri kendi `webhook.url_env`'ine pinlenmiş birden çok ayrı ForgeLM training config'i çalıştırın @@ -99,7 +117,7 @@ Gerçek `WebhookConfig` (bkz. `forgelm/config.py::WebhookConfig`): | Alan | Vars. | Notlar | |---|---|---| | `url` | `null` | Inline URL — secret hijyeni için `url_env`'i tercih edin. | -| `url_env` | `null` | URL'i taşıyan env var adı. Set edildiğinde `url`'i override eder. | +| `url_env` | `null` | URL'i taşıyan env var adı. Yalnızca `url` boşken okunur; ikisi de set ise `url` kazanır. | | `notify_on_start` | `true` | `training.start` VE `pipeline.started`'ı gate'ler. | | `notify_on_success` | `true` | `training.success`, `approval.required` VE başarılı bir `pipeline.completed`'ı gate'ler. | | `notify_on_failure` | `true` | `training.failure`, `training.reverted`, `pipeline.stage_reverted` VE başarısız bir `pipeline.completed`'ı gate'ler. | @@ -118,7 +136,7 @@ payload zaten curated), ve routing hepsi ForgeLM'in dışında yaşar. - **TLS şiddetle önerilir.** ForgeLM hem HTTPS hem HTTP webhook URL'lerine izin verir — HTTP hedefleri `Webhook URL uses HTTP (not HTTPS). Data will be sent unencrypted.` uyarısı loglar ama varsayılan olarak reddedilmez (bkz. `forgelm/webhook.py` `_send`). Regüle bir ortamda `webhook.require_https: true` ile plaintext bir `http://` URL'ini hard failure yapın (teslimat reddedilir, gönderilmez). Üretimde `https://` URL'leri pinleyin. - **Curated payload.** ForgeLM webhook payload'larına asla raw eğitim verisi, tam config'ler veya unredacted PII koymaz. Notifier sabit-şekilli bir JSON sarar; `webhook.redact` toggle'ı yoktur çünkü kullanıcı-kontrollü redakte edilecek bir şey yok. - **SSRF guard.** ForgeLM iç IP'lere işaret eden webhook URL'lerini engeller — RFC 1918, loopback, link-local (bulut IMDS `169.254.169.254` dahil), RFC 6598 Shared Address Space / Carrier-Grade-NAT (`100.64.0.0/10`; bu blok, Alibaba Cloud ECS'in IMDS'ini de kapsar — `100.100.100.200` — stdlib `ipaddress` predicate'leri bu bloğu kendiliğinden private veya reserved olarak işaretlemez), ayrıca reserved ve multicast hedefler — `webhook.allow_private_destinations: true` ile açıkça opt-in olmadıkça. Yanlış konfigüre koşuların iç ağınızı sondalamasını veya payload'ı bir bulut metadata servisine sızdırmasını önler. -- **HMAC body imzalama yok.** ForgeLM webhook gövdelerini imzalamaz — hedef-tarafı authenticity TLS + `url_env` üzerinden URL gizliliği artı alıcı sistemin bearer-token / signed-request kontrollerine (Slack signing secret, Teams connector token) düşer. +- **HMAC body imzalama yok.** ForgeLM webhook gövdelerini imzalamaz — hedef-tarafı authenticity TLS + `url_env` üzerinden URL gizliliği artı alıcı sistemin bearer-token / signed-request kontrollerine (örneğin Slack signing secret) düşer. ## Sık hatalar diff --git a/docs/usermanuals/tr/reference/configuration.md b/docs/usermanuals/tr/reference/configuration.md index 39f8e44b..4a7902fd 100644 --- a/docs/usermanuals/tr/reference/configuration.md +++ b/docs/usermanuals/tr/reference/configuration.md @@ -264,8 +264,8 @@ compliance: ```yaml webhook: - url: null # Slack / Teams / Discord / özel; url_env tercihli - url_env: null # webhook URL'sini taşıyan env değişkeni + url: null # Slack incoming webhook'u ya da Teams / Discord için relay; url_env tercihli + url_env: null # webhook URL'sini taşıyan env değişkeni; yalnızca url boşken okunur notify_on_start: true notify_on_success: true notify_on_failure: true diff --git a/forgelm/config.py b/forgelm/config.py index 4114f2df..f3f7ba74 100644 --- a/forgelm/config.py +++ b/forgelm/config.py @@ -1523,17 +1523,17 @@ class WebhookConfig(BaseModel): url: Optional[str] = Field( default=None, - description="Webhook URL (Slack / Teams / Discord / custom). Use `url_env` to read from env to avoid committing secrets.", + description="Webhook URL (a Slack incoming webhook, or a relay that forwards to Teams / Discord / custom). Use `url_env` to avoid committing secrets.", ) url_env: Optional[str] = Field( - default=None, description="Env var name carrying the webhook URL (overrides `url` when set)." + default=None, description="Env var name carrying the webhook URL; read only when `url` is empty." ) notify_on_start: bool = Field(default=True, description="POST a `notify_start` event when training begins.") notify_on_success: bool = Field( default=True, description="POST a `notify_success` event when training completes successfully." ) notify_on_failure: bool = Field( - default=True, description="POST a `notify_failure` event when training fails (any non-zero exit)." + default=True, description="POST a failure event when training raises or a post-training gate reverts the model." ) timeout: int = Field( default=10, @@ -1677,7 +1677,7 @@ class ForgeConfig(BaseModel): description="Post-training evaluation block (loss / benchmark / safety / judge / human-approval gate).", ) webhook: Optional[WebhookConfig] = Field( - default=None, description="Webhook notification block (Slack / Teams / Discord / custom)." + default=None, description="Webhook notification block (Slack-format JSON; Teams and Discord need a relay)." ) distributed: Optional[DistributedConfig] = Field( default=None, description="DeepSpeed / FSDP multi-GPU configuration block." From 16ba7247fcbfcab546daab8ba3b9175c30a04aa4 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 10:38:43 +0300 Subject: [PATCH 2/5] docs(deepspeed): say that the preset names work only from a source checkout distributed.deepspeed_config accepts zero2, zero3 and zero3_offload, resolved to configs/deepspeed/.json beside the package or in the current directory. That folder is in the repository and the sdist but not in the wheel or the Docker image, so on a pip install the names raise a ConfigError, including the default (zero2, used when deepspeed_config is unset). The configuration and distributed-training references, the troubleshooting entry (whose sample error was the wrong exception type; it now shows the real message), the enterprise guide, the user manuals, config_template.yaml and the field description said "built-in" or "preset" without that condition. They now say it and point at a JSON path that works on every install. The unverified "zero3_offload fits 70B on 8x24 GB" sentence is removed, the zero2 preset's own failure (allgather_bucket_size "auto" is not filled in) is noted with its issue, and the template's launch line is `-m forgelm`, the only form that runs the checkout. Packaging the presets and a test that resolves one from an installed wheel are code changes and stay open. Refs #110, #111, #115, #410 Co-Authored-By: Claude Sonnet 5.5 --- CHANGELOG.md | 9 +++++++++ config_template.yaml | 4 ++-- docs/guides/enterprise_deployment.md | 2 +- docs/guides/troubleshooting-tr.md | 7 ++++--- docs/guides/troubleshooting.md | 6 ++++-- docs/reference/configuration-tr.md | 2 +- docs/reference/configuration.md | 2 +- docs/reference/distributed_training-tr.md | 15 ++++++++++++--- docs/reference/distributed_training.md | 15 ++++++++++++--- docs/usermanuals/en/reference/configuration.md | 2 +- docs/usermanuals/en/training/distributed.md | 4 ++-- docs/usermanuals/tr/reference/configuration.md | 2 +- docs/usermanuals/tr/training/distributed.md | 4 ++-- forgelm/config.py | 2 +- 14 files changed, 53 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d44dd02..4f1765d2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,15 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Say that the DeepSpeed preset names (`zero2`, `zero3`, `zero3_offload`) work only from a source checkout. They resolve to + `configs/deepspeed/.json` beside the package or in the current directory, and neither the wheel nor the Docker image ships + that folder, so on a `pip install` the names fail with a configuration error (and the unset-`deepspeed_config` default is + `zero2`). The configuration and distributed-training references, the troubleshooting entry (its sample error is now the real + message), the enterprise guide, the user manuals, `config_template.yaml` and the field description now say so and point at a + JSON path; the template's `torchrun … forgelm` launch line is `-m forgelm`, the only form that runs the checkout. The + unverified "`zero3_offload` fits 70B on 8×24 GB" sentence is gone, and the `zero2` preset's own start-up failure is noted + (#410). Shipping the presets in the package, and a test that resolves one from an installed wheel, are code changes and stay + open on the issues (#110, #111, #115). - Fix the KTO bullet on the Alignment overview user-manual page (English and Turkish): it named the fields `(prompt, response, good?)`, but the KTO loader reads `prompt`, `completion` and a boolean `label` (#965). diff --git a/config_template.yaml b/config_template.yaml index 8aec1398..e06e01f9 100644 --- a/config_template.yaml +++ b/config_template.yaml @@ -162,10 +162,10 @@ data: # dare_seed: 42 # DARE: RNG seed for the random drop mask # Distributed training (multi-GPU). Requires: pip install forgelm[distributed] -# Launch with: torchrun --nproc_per_node=N forgelm --config my_config.yaml +# Launch with: torchrun --nproc_per_node=N -m forgelm --config my_config.yaml # distributed: # strategy: "deepspeed" # "deepspeed" or "fsdp" -# deepspeed_config: "zero2" # preset: "zero2", "zero3", "zero3_offload" or path to JSON +# deepspeed_config: "ds.json" # path to a DeepSpeed JSON (the presets "zero2", "zero3", "zero3_offload" need a source checkout) # # --- FSDP options (when strategy: "fsdp") --- # # fsdp_strategy: "full_shard" # "full_shard", "shard_grad_op", "no_shard", "hybrid_shard" # # fsdp_auto_wrap: true diff --git a/docs/guides/enterprise_deployment.md b/docs/guides/enterprise_deployment.md index f9c70e60..c56d6f2d 100644 --- a/docs/guides/enterprise_deployment.md +++ b/docs/guides/enterprise_deployment.md @@ -121,7 +121,7 @@ ForgeLM sets `HF_HUB_OFFLINE=1`, `TRANSFORMERS_OFFLINE=1`, `HF_DATASETS_OFFLINE= ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # or "zero3", "zero3_offload" + deepspeed_config: "/etc/forgelm/ds_zero2.json" # your own DeepSpeed JSON; the zero2/zero3/zero3_offload preset names need a source checkout ``` ```bash diff --git a/docs/guides/troubleshooting-tr.md b/docs/guides/troubleshooting-tr.md index c4cb2408..29e6331c 100644 --- a/docs/guides/troubleshooting-tr.md +++ b/docs/guides/troubleshooting-tr.md @@ -363,11 +363,12 @@ nvidia-smi ### DeepSpeed Config bulunamadı ```text -FileNotFoundError: DeepSpeed preset 'zero2' not found +DeepSpeed preset 'zero2' not found at /configs/deepspeed/zero2.json ``` -ForgeLM proje kökünden çalıştırdığınızdan emin olun ya da mutlak yol -kullanın: +Ön ayar adları yalnızca bir kaynak checkout'unda çözülür (paketin yanındaki ya da geçerli dizindeki bir +`configs/deepspeed/` dizini); `pip install` veya Docker imajı bunları içermez. ForgeLM proje kökünden çalıştırın ya da +bir DeepSpeed JSON dosyasının yolunu kullanın: ```yaml distributed: diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.md index 7533eedb..9549875b 100644 --- a/docs/guides/troubleshooting.md +++ b/docs/guides/troubleshooting.md @@ -344,10 +344,12 @@ nvidia-smi ### DeepSpeed Config Not Found ```text -FileNotFoundError: DeepSpeed preset 'zero2' not found +DeepSpeed preset 'zero2' not found at /configs/deepspeed/zero2.json ``` -Ensure you're running from the ForgeLM project root, or use an absolute path: +The preset names resolve only from a source checkout (a `configs/deepspeed/` directory beside the package or in the +current directory); a `pip install` or the Docker image does not contain them. Run from the ForgeLM project root, or +use the path of a DeepSpeed JSON file: ```yaml distributed: diff --git a/docs/reference/configuration-tr.md b/docs/reference/configuration-tr.md index b78fa07f..1c512eaa 100644 --- a/docs/reference/configuration-tr.md +++ b/docs/reference/configuration-tr.md @@ -397,7 +397,7 @@ kaynağı kaydedilmemiştir. | Alan | Tip | Varsayılan | Açıklama | |------|-----|-----------|----------| | `strategy` | string | `null` | `"deepspeed"` veya `"fsdp"` (null = tek GPU) | -| `deepspeed_config` | string | `null` | Ön ayar (`"zero2"`, `"zero3"`, `"zero3_offload"`) veya JSON yolu | +| `deepspeed_config` | string | `null` | JSON yolu ya da yalnızca kaynak checkout'unda çözülen (pip veya Docker kurulumunda çözülmeyen) bir ön ayar adı (`"zero2"`, `"zero3"`, `"zero3_offload"`) | | `fsdp_strategy` | string | `"full_shard"` | `"full_shard"`, `"shard_grad_op"`, `"hybrid_shard"`, `"no_shard"` | | `fsdp_auto_wrap` | bool | `true` | Transformer katmanlarını otomatik sar | | `fsdp_offload` | bool | `false` | Parametreleri CPU'ya taşı | diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 0f83d16e..fa533638 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -309,7 +309,7 @@ silently extend the retention horizon by re-using a stale workspace. | Field | Type | Default | Description | |-------|------|---------|-------------| | `strategy` | string | `null` | `"deepspeed"` or `"fsdp"` (null = single GPU) | -| `deepspeed_config` | string | `null` | Preset (`"zero2"`, `"zero3"`, `"zero3_offload"`) or JSON path | +| `deepspeed_config` | string | `null` | JSON path, or a preset name (`"zero2"`, `"zero3"`, `"zero3_offload"`) that resolves only from a source checkout, not from a pip or Docker install | | `fsdp_strategy` | string | `"full_shard"` | `"full_shard"`, `"shard_grad_op"`, `"hybrid_shard"`, `"no_shard"` | | `fsdp_auto_wrap` | bool | `true` | Auto-wrap transformer layers | | `fsdp_offload` | bool | `false` | Offload parameters to CPU | diff --git a/docs/reference/distributed_training-tr.md b/docs/reference/distributed_training-tr.md index 32e6378f..bee3d2ef 100644 --- a/docs/reference/distributed_training-tr.md +++ b/docs/reference/distributed_training-tr.md @@ -49,7 +49,16 @@ torchrun --nproc_per_node=4 -m forgelm.cli --config my_config.yaml ## DeepSpeed Yapılandırması -ForgeLM, üç yerleşik DeepSpeed ön ayarı sunar. `deepspeed_config` parametresine bu isimlerden birini veya kendi JSON dosyanızın yolunu verin. +Depo, `configs/deepspeed/` altında üç DeepSpeed ön ayarı barındırır. `deepspeed_config` parametresine bunların adlarından birini veya kendi JSON dosyanızın yolunu verin. + +**Ön ayar adları yalnızca bir kaynak checkout'unda çalışır.** Kurulu paketin yanındaki ya da geçerli dizindeki +`configs/deepspeed/.json` dosyasına çözülürler. `pip install forgelm` wheel'i ve Docker imajı bu dosyaları içermez; +orada bir ön ayar adı yapılandırma hatasıyla (exit 1) başarısız olur. Deponun `configs/deepspeed/` klasöründeki JSON +dosyalarını kopyalayın ya da kendinizinkini yazın ve `deepspeed_config`'i onun yoluna ayarlayın. + +**Bilinen sorun ([#410](https://github.com/HodeTech/ForgeLM/issues/410)):** `deepspeed_config` ayarlanmadığında da +kullanılan `zero2` ön ayarı `allgather_bucket_size` değerini `"auto"` yapar; transformers'ın DeepSpeed entegrasyonu bu +değeri doldurmaz, bu yüzden onunla bir ZeRO-2 koşusu durur. Düzelene kadar kendi ZeRO-2 JSON'unuzu kullanın. ### Ön Ayarlar (Presets) @@ -64,8 +73,8 @@ ForgeLM, üç yerleşik DeepSpeed ön ayarı sunar. `deepspeed_config` parametre ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # Ön ayar adı - # deepspeed_config: "./my_ds_config.json" # Veya özel dosya yolu + deepspeed_config: "./my_ds_config.json" # Kendi JSON'unuzun yolu (her kurulumda çalışır) + # deepspeed_config: "zero2" # Ön ayar adı: yalnızca kaynak checkout'u ``` ### ZeRO Aşama Karşılaştırması diff --git a/docs/reference/distributed_training.md b/docs/reference/distributed_training.md index b83b29f9..8d9f6f29 100644 --- a/docs/reference/distributed_training.md +++ b/docs/reference/distributed_training.md @@ -49,7 +49,16 @@ torchrun --nproc_per_node=4 -m forgelm.cli --config my_config.yaml ## DeepSpeed Configuration -ForgeLM provides three built-in DeepSpeed presets. Set `deepspeed_config` to one of these names, or provide a path to your own JSON file. +The repository ships three DeepSpeed presets under `configs/deepspeed/`. Set `deepspeed_config` to one of their names, or provide a path to your own JSON file. + +**The preset names work only from a source checkout.** They resolve to `configs/deepspeed/.json` beside the +installed package or in the current directory. A `pip install forgelm` wheel and the Docker image do not contain those +files, so there a preset name fails with a configuration error (exit 1). Copy the JSON files from the repository's +`configs/deepspeed/` folder, or write your own, and set `deepspeed_config` to its path. + +**Known issue ([#410](https://github.com/HodeTech/ForgeLM/issues/410)):** the `zero2` preset, which is also used when +`deepspeed_config` is unset, sets `allgather_bucket_size` to `"auto"`, a value transformers' DeepSpeed integration does +not fill in, so a ZeRO-2 run with it aborts. Use your own ZeRO-2 JSON until that is fixed. ### Presets @@ -64,8 +73,8 @@ ForgeLM provides three built-in DeepSpeed presets. Set `deepspeed_config` to one ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # Preset name - # deepspeed_config: "./my_ds_config.json" # Or custom path + deepspeed_config: "./my_ds_config.json" # Path to your own JSON (works on every install) + # deepspeed_config: "zero2" # Preset name: source checkout only ``` ### ZeRO Stage Comparison diff --git a/docs/usermanuals/en/reference/configuration.md b/docs/usermanuals/en/reference/configuration.md index c4ba73c2..82e25c66 100644 --- a/docs/usermanuals/en/reference/configuration.md +++ b/docs/usermanuals/en/reference/configuration.md @@ -228,7 +228,7 @@ The merge field is `method` (not `algorithm`), and there is no `dare_ties` choic ```yaml distributed: strategy: null # null (single-GPU, no distributed wrapping) | deepspeed | fsdp - deepspeed_config: null # path to a DeepSpeed JSON, or preset name: zero2 | zero3 | zero3_offload + deepspeed_config: null # path to a DeepSpeed JSON (preset names zero2 | zero3 | zero3_offload need a source checkout) fsdp_strategy: "full_shard" # full_shard | shard_grad_op | no_shard | hybrid_shard fsdp_auto_wrap: true # auto-wrap transformer layers (recommended) fsdp_offload: false # offload FSDP parameters to CPU between forward/backward diff --git a/docs/usermanuals/en/training/distributed.md b/docs/usermanuals/en/training/distributed.md index 7aa622ff..41f23b79 100644 --- a/docs/usermanuals/en/training/distributed.md +++ b/docs/usermanuals/en/training/distributed.md @@ -66,7 +66,7 @@ ZeRO-2 shards the optimiser state (the heaviest VRAM component for adaptive opti ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # preset name (or a filesystem path to a DeepSpeed JSON) + deepspeed_config: "./ds_zero2.json" # a filesystem path to a DeepSpeed JSON (preset names such as "zero2" need a source checkout) training: gradient_accumulation_steps: 4 @@ -95,7 +95,7 @@ training: gradient_accumulation_steps: 8 ``` -The `zero3_offload` preset fits 70B on 8×24 GB by offloading optimiser state and parameters to CPU. ZeRO-Infinity's NVMe offload isn't a built-in preset — point `deepspeed_config` at a custom DeepSpeed JSON with `offload_param`/`offload_optimizer` set to `device: nvme` for that case. +The `zero3_offload` preset offloads optimiser state and parameters to CPU, trading speed for GPU memory; whether a given model fits depends on your GPU memory and CPU RAM. The preset names `zero2`, `zero3` and `zero3_offload` resolve only from a source checkout (the JSON files live in the repository's `configs/deepspeed/` folder, not in the pip wheel or the Docker image), and the `zero2` preset is currently broken ([#410](https://github.com/HodeTech/ForgeLM/issues/410)), so give `deepspeed_config` the path of a JSON file you control. ZeRO-Infinity's NVMe offload isn't a preset — point `deepspeed_config` at a custom DeepSpeed JSON with `offload_param`/`offload_optimizer` set to `device: nvme` for that case. | Model | GPUs | ZeRO-3 + offload? | |---|---|---| diff --git a/docs/usermanuals/tr/reference/configuration.md b/docs/usermanuals/tr/reference/configuration.md index 39f8e44b..4ac4d99c 100644 --- a/docs/usermanuals/tr/reference/configuration.md +++ b/docs/usermanuals/tr/reference/configuration.md @@ -230,7 +230,7 @@ Merge alanı `algorithm` değil `method`'dur, `dare_ties` seçeneği, `base_mode ```yaml distributed: strategy: null # null (tek GPU, dağıtık sarmalama yok) | deepspeed | fsdp - deepspeed_config: null # bir DeepSpeed JSON'a yol, veya preset adı: zero2 | zero3 | zero3_offload + deepspeed_config: null # bir DeepSpeed JSON'a yol (preset adları zero2 | zero3 | zero3_offload kaynak checkout'u ister) fsdp_strategy: "full_shard" # full_shard | shard_grad_op | no_shard | hybrid_shard fsdp_auto_wrap: true # transformer katmanlarını otomatik sarmala (önerilir) fsdp_offload: false # forward/backward arasında FSDP parametrelerini CPU'ya offload et diff --git a/docs/usermanuals/tr/training/distributed.md b/docs/usermanuals/tr/training/distributed.md index 4dcfd239..53d2ae78 100644 --- a/docs/usermanuals/tr/training/distributed.md +++ b/docs/usermanuals/tr/training/distributed.md @@ -66,7 +66,7 @@ ZeRO-2 optimizer state'i sharder (Adam gibi adaptif optimizer'larda en ağır VR ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # preset adı (veya bir DeepSpeed JSON dosya yolu) + deepspeed_config: "./ds_zero2.json" # bir DeepSpeed JSON dosya yolu ("zero2" gibi preset adları kaynak checkout'u ister) training: gradient_accumulation_steps: 4 @@ -95,7 +95,7 @@ training: gradient_accumulation_steps: 8 ``` -`zero3_offload` preset'i, optimizer state ve parametreleri CPU'ya boşaltarak 70B'i 8×24 GB'a sığdırır. ZeRO-Infinity'nin NVMe offload'u hazır bir preset değildir — bunun için `deepspeed_config`'i, `offload_param`/`offload_optimizer` alanları `device: nvme` olarak ayarlanmış özel bir DeepSpeed JSON dosyasına yönlendirin. +`zero3_offload` preset'i optimizer state ve parametreleri CPU'ya boşaltır, hızdan GPU belleği kazandırır; belirli bir modelin sığıp sığmayacağı GPU belleğinize ve CPU RAM'inize bağlıdır. `zero2`, `zero3` ve `zero3_offload` preset adları yalnızca kaynak checkout'unda çözülür (JSON dosyaları pip wheel'inde ya da Docker imajında değil, deponun `configs/deepspeed/` klasöründedir) ve `zero2` preset'i şu an bozuktur ([#410](https://github.com/HodeTech/ForgeLM/issues/410)); bu yüzden `deepspeed_config`'e kontrolünüzdeki bir JSON dosyasının yolunu verin. ZeRO-Infinity'nin NVMe offload'u bir preset değildir — bunun için `deepspeed_config`'i, `offload_param`/`offload_optimizer` alanları `device: nvme` olarak ayarlanmış özel bir DeepSpeed JSON dosyasına yönlendirin. | Model | GPU | ZeRO-3 + offload? | |---|---|---| diff --git a/forgelm/config.py b/forgelm/config.py index 4114f2df..135a6deb 100644 --- a/forgelm/config.py +++ b/forgelm/config.py @@ -768,7 +768,7 @@ class DistributedConfig(BaseModel): ) deepspeed_config: Optional[str] = Field( default=None, - description="DeepSpeed config: filesystem path to a DS JSON OR preset name (`zero2`, `zero3`, `zero3_offload`).", + description="DeepSpeed config: filesystem path to a DS JSON. The preset names (`zero2`, `zero3`, `zero3_offload`) resolve only from a source checkout, not a pip or Docker install.", ) fsdp_strategy: Literal["full_shard", "shard_grad_op", "no_shard", "hybrid_shard"] = Field( default="full_shard", From afd726aa8b31307b2c9b799bd5161835e2997762 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 10:45:13 +0300 Subject: [PATCH 3/5] docs(usermanuals): show the real output directory on the first-run, layout and approval pages The first-run page told a new user that a run produces checkpoints//artifacts/ with a manifest.json that hashes "every file inside ... for tamper-evidence", a config_snapshot.yaml, a safety_report.json and a data_audit_report.json; the project-layout page repeated the artifacts/ directory and said that merging runs "breaks the SHA-256 manifest". None of those files exist. A run writes (checked in trainer.py and compliance.py): - audit_log.jsonl and audit_log.jsonl.manifest.json at the top of training.output_dir; - compliance/: compliance_report.json, training_manifest.yaml, data_provenance.json, data_governance_report.json, and annex_iv_metadata.json only when the YAML sets compliance.provider_name, system_name or intended_purpose; - safety/ and benchmark/ only when those gates are enabled; - final_model/: the weights, README.md (model card), deployer_instructions.md and model_integrity.json (or final_model.staging./ until forgelm approve, when approval is required). Only the audit log (verify-audit) and final_model/ (verify-integrity) are tamper-checked; nothing hashes compliance/, and the pages now say so. The invented training log (a DPO pass in an SFT-only template, "Llama Guard S1-S14: clean") is replaced by a description of the stages the template really runs, the approval-gate sample shows the two log lines the trainer prints, and the doctor sample uses the renderer's real format (ASCII header, [+ pass] markers, "discoverable" wording, abridged rows). English and Turkish. Whether to add a manifest over compliance/ is a code decision and stays open. Refs #96, #534 Co-Authored-By: Claude Sonnet 5.5 --- CHANGELOG.md | 10 +++ .../en/compliance/human-oversight.md | 8 +- .../en/getting-started/first-run.md | 76 ++++++++++--------- .../en/getting-started/project-layout.md | 15 ++-- .../tr/compliance/human-oversight.md | 8 +- .../tr/getting-started/first-run.md | 76 ++++++++++--------- .../tr/getting-started/project-layout.md | 15 ++-- 7 files changed, 110 insertions(+), 98 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d44dd02..e0b24da2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,16 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Show the real output directory on the first-run, project-layout and human-oversight pages (English and Turkish). They described a + `checkpoints//artifacts/` bundle with a `manifest.json` that hashes "every file inside … for tamper-evidence", a + `config_snapshot.yaml`, a `safety_report.json` and a `data_audit_report.json`. None of those exist: a run writes `audit_log.jsonl` with its + `.manifest.json` genesis pin, a `compliance/` directory (`compliance_report.json`, `training_manifest.yaml`, `data_provenance.json`, + `data_governance_report.json`, and `annex_iv_metadata.json` only when the YAML names a provider, system or purpose), `safety/` and + `benchmark/` only when enabled, and `final_model/` with the weights, model card, `deployer_instructions.md` and `model_integrity.json`. + Only the audit log and `final_model/` are tamper-checked; nothing hashes `compliance/`, and the pages now say so. The invented training log + (a DPO pass in an SFT-only template, `Llama Guard S1-S14: clean`) is replaced by a description of the real stages, the approval-gate + sample shows the two log lines the trainer prints, and the `forgelm doctor` sample uses the real renderer's format (ASCII header and + `[+ pass]` markers, "discoverable" wording, an abridged row list) (#96, #534). - Fix the KTO bullet on the Alignment overview user-manual page (English and Turkish): it named the fields `(prompt, response, good?)`, but the KTO loader reads `prompt`, `completion` and a boolean `label` (#965). diff --git a/docs/usermanuals/en/compliance/human-oversight.md b/docs/usermanuals/en/compliance/human-oversight.md index 373f7db2..61e05a72 100644 --- a/docs/usermanuals/en/compliance/human-oversight.md +++ b/docs/usermanuals/en/compliance/human-oversight.md @@ -48,12 +48,8 @@ There is **no** `compliance.human_approval` field, no `approval.*` block, no `si The trainer halts after eval and prints: ```text -[2026-04-29 14:33:10] Human approval required. - Run ID: abc123 - Bundle: checkpoints/run/artifacts/ - - To approve: forgelm approve abc123 --output-dir checkpoints/run --comment "..." - To reject: forgelm reject abc123 --output-dir checkpoints/run --comment "..." +2026-04-29 14:33:10 [INFO] forgelm.trainer: Human approval required. Model staged at: checkpoints/run/final_model.staging.abc123 +2026-04-29 14:33:10 [INFO] forgelm.trainer: Review results in checkpoints/run/compliance/ and run `forgelm approve abc123 --output-dir checkpoints/run` to promote, or `forgelm reject abc123 --output-dir checkpoints/run` to preserve for forensic review. ``` The reviewer runs the approval command from any machine with access to the audit-log + staging directory. ForgeLM resolves their identity from `FORGELM_OPERATOR`, HMAC-chains the decision event, and (on `approve`) renames the staging directory to the canonical `final_model/` path. diff --git a/docs/usermanuals/en/getting-started/first-run.md b/docs/usermanuals/en/getting-started/first-run.md index 99a2b6ee..5b09a35c 100644 --- a/docs/usermanuals/en/getting-started/first-run.md +++ b/docs/usermanuals/en/getting-started/first-run.md @@ -29,22 +29,23 @@ flowchart LR Before anything else, run `forgelm doctor` to check that Python, PyTorch, CUDA, and any optional dependencies are wired up correctly: ```shell -$ forgelm doctor -forgelm doctor — environment check - - [✓ pass] python.version Python 3.11.4 (CPython). - [✓ pass] torch.cuda torch 2.4.0 with CUDA 12.4. - [✓ pass] gpu.inventory 1 GPU(s) — GPU0: NVIDIA RTX 4090 (24.0 GiB). - [✓ pass] extras.qlora Installed (module bitsandbytes, purpose: 4-bit / 8-bit QLoRA training (Linux GPUs only)). - [✓ pass] extras.unsloth Installed (module unsloth, purpose: Unsloth-accelerated training (Linux GPUs only)). - [! warn] extras.distributed Optional extra missing — install with: pip install 'forgelm[distributed]' (purpose: DeepSpeed ZeRO + offload distributed training (Linux + macOS only)). - [✓ pass] hf_hub.reachable HuggingFace Hub reachable (HTTP 200). - [✓ pass] disk.workspace Workspace /home/me/forgelm — 387.0 GiB free of 500.0 GiB. - [! warn] operator.identity FORGELM_OPERATOR not set; audit events will fall back to 'me@workstation'. Pin FORGELM_OPERATOR= for CI / pipeline runs. - -Summary: 7 pass, 2 warn, 0 fail. +$ forgelm doctor --offline +forgelm doctor - environment check + + [+ pass] python.version Python 3.11.4 (CPython). + [! warn] torch.cuda torch 2.4.0 installed but CUDA is unavailable. CPU-only runs are supported but training will be very slow; expect to use this only for tiny experiments or smoke tests. + [+ pass] numpy.torch_abi torch 2.4.0 + numpy 2.1.0 are ABI-compatible. + [! warn] gpu.inventory No CUDA devices visible; CPU-only mode. + [! warn] extras.distributed Optional extra missing — install with: pip install 'forgelm[distributed]' (purpose: DeepSpeed ZeRO + offload distributed training (Linux + macOS only)). + [+ pass] disk.workspace Workspace /home/me/forgelm — 387.0 GiB free of 500.0 GiB. + [! warn] operator.identity FORGELM_OPERATOR not set; audit events will fall back to 'me@workstation'. Pin FORGELM_OPERATOR= for CI / pipeline runs so the audit log identifies a stable identity. + ... + +Summary: 7 pass, 12 warn, 0 fail. ``` +This is an excerpt: the real output has around 19 rows (it also checks the ForgeLM install, every optional extra, and the HF cache or Hub), and the rows, values and counts depend on your machine. A `pass` on an extra means the module is discoverable, not that it was import-tested; `[+ pass]`, `[! warn]` and `[x fail]` are the status markers. + `--output-format json` returns a structured envelope (`{"success": bool, "checks": [...], "summary": {...}}`) so CI can filter on individual probe results without parsing the table. Pass `--offline` to skip the HF Hub network probe and inspect the local cache instead — useful for air-gapped deployments. :::tip @@ -137,17 +138,10 @@ Possible verdicts: ```shell $ forgelm --config configs/quickstart-customer-support.yaml -[2026-04-28 14:01:32] config validated -[2026-04-28 14:01:33] auditing data/customer-support.jsonl (12,400 rows, 3 splits) -[2026-04-28 14:01:35] PII flags: 0 critical, 5 medium · cross-split overlap: 0 -[2026-04-28 14:01:37] SFT epoch 1/3 · loss=2.31 → 1.42 -[2026-04-28 14:18:55] DPO preference pass · β=0.1 · KL=4.2 -[2026-04-28 14:32:11] benchmark hellaswag=0.62 truthfulqa=0.48 -[2026-04-28 14:33:02] Llama Guard S1-S14: clean -[2026-04-28 14:33:04] Annex IV → checkpoints/customer-support/artifacts/annex_iv_metadata.json -[2026-04-28 14:33:04] ✔ finished, exit 0 ``` +The output is a stream of timestamped `[INFO]` lines from the `forgelm.*` loggers, with the progress bar and the per-step loss lines that transformers and TRL print. The `customer-support` template is an SFT run with no `evaluation:` block, so the stages you get are: config validation, training, then the files below being written. Stages such as the benchmark, the safety evaluation, the judge and the human-approval gate run only when the YAML enables them. The command ends with exit code `0` on success (see [Exit codes](#/reference/exit-codes)). + ## 6. Try the model ```shell @@ -160,20 +154,32 @@ continues until the end of the current billing period… ## What you got on disk ```text -checkpoints/customer-support/ -├── artifacts/ -│ ├── annex_iv_metadata.json ← Article 11 technical documentation -│ ├── audit_log.jsonl ← Article 12 append-only event log -│ ├── data_audit_report.json ← Article 10 data governance evidence -│ ├── safety_report.json ← Llama Guard verdict -│ ├── benchmark_results.json ← Per-task accuracy -│ └── manifest.json ← SHA-256 over every artifact -├── README.md ← Article 13 model card -├── config_snapshot.yaml ← The exact YAML this run used -└── adapter_model.safetensors ← LoRA weights (or merged checkpoint) +checkpoints/customer-support/ ← training.output_dir +├── audit_log.jsonl ← Article 12 append-only event log, hash-chained +├── audit_log.jsonl.manifest.json ← genesis pin: lets verify-audit detect truncation +├── compliance/ +│ ├── compliance_report.json ← Article 11 manifest (the canonical machine-readable bundle) +│ ├── training_manifest.yaml ← operator-readable summary of the run +│ ├── data_provenance.json ← Article 10 dataset fingerprint +│ └── data_governance_report.json ← Article 10 data-governance evidence +├── checkpoint-/ ← intermediate Trainer checkpoints +└── final_model/ + ├── adapter_model.safetensors ← LoRA weights (or merged checkpoint) + ├── README.md ← Article 13 model card + ├── deployer_instructions.md ← notes for whoever deploys the model + └── model_integrity.json ← SHA-256 checksums of the files in final_model/ ``` -That `artifacts/` directory is the deliverable for compliance reviews. Every file inside is hashed in `manifest.json` for tamper-evidence. +Some files appear only when the YAML asks for them: + +| File | Appears when | +|---|---| +| `compliance/annex_iv_metadata.json` | `compliance.provider_name`, `system_name` or `intended_purpose` is set | +| `safety/safety_results.json` | the safety evaluation is enabled | +| `benchmark/benchmark_results.json` | the benchmark gate is enabled | +| `final_model.staging./` instead of `final_model/` | `evaluation.require_human_approval` is on, until `forgelm approve` promotes it | + +There is no `artifacts/` directory and no single manifest over the whole bundle. Two things are tamper-checked: the audit log by its hash chain (`forgelm verify-audit`) and `final_model/` by `model_integrity.json` (`forgelm verify-integrity`). Nothing hashes `compliance/`, so keep those files where an edit would show, for example in version control or write-once storage. ## Next steps diff --git a/docs/usermanuals/en/getting-started/project-layout.md b/docs/usermanuals/en/getting-started/project-layout.md index 05b26f00..6883382c 100644 --- a/docs/usermanuals/en/getting-started/project-layout.md +++ b/docs/usermanuals/en/getting-started/project-layout.md @@ -22,10 +22,9 @@ my-finetune/ │ └── data_audit_report.json ├── checkpoints/ — model outputs (gitignored) │ └── customer-support/ -│ ├── adapter_model.safetensors -│ ├── README.md — model card (Article 13) -│ ├── config_snapshot.yaml — frozen copy of the config used -│ └── artifacts/ — compliance evidence bundle +│ ├── audit_log.jsonl — append-only, hash-chained event log +│ ├── compliance/ — Article 10/11 evidence (JSON + YAML) +│ └── final_model/ — weights, model card (README.md), model_integrity.json ├── ingested/ — raw documents → JSONL via `forgelm ingest` └── .forgelm/ — local cache (also gitignored) ``` @@ -50,11 +49,11 @@ ForgeLM's defaults play well with version control if you commit the right things **Commit:** - `configs/*.yaml` — your runs are configurations; the YAML is the source of truth. - `audit/*.json` — small, machine-readable, useful in PR reviews ("did the audit numbers regress?"). -- `checkpoints/*/README.md` and `checkpoints/*/artifacts/` (model card and audit bundle, both small). +- `checkpoints/*/final_model/README.md` and `checkpoints/*/compliance/` (model card and compliance evidence, both small). **Gitignore:** - `data/` — usually too large; track via DVC or your preferred dataset registry instead. -- `checkpoints/*/adapter_model.safetensors` and other weight files — too large for git, push to HuggingFace Hub or model registry. +- `checkpoints/*/final_model/*.safetensors`, the `checkpoint-*` directories and other weight files — too large for git, push to HuggingFace Hub or model registry. - `ingested/` — re-buildable from raw documents. - `.forgelm/` — local cache. @@ -125,10 +124,10 @@ Run the dev variant during development, the full one for release. CI runs the fu ## Where compliance artifacts go -Every successful (or failed) training run creates a `checkpoints//artifacts/` directory containing the evidence bundle described in [Compliance Overview](#/compliance/overview). This is the directory you hand to a regulator or pin in your CI artifacts. +A training run writes its evidence into `training.output_dir`: `audit_log.jsonl` (with its `.manifest.json` genesis pin) at the top level and a `compliance/` directory next to `final_model/`; [Your First Run](#/getting-started/first-run) shows the full tree and which files depend on the YAML. There is no `artifacts/` directory. Hand `compliance/` and `audit_log.jsonl` to a reviewer, or pin them in your CI artifacts. :::warn -**Don't merge the artifacts/ directory across runs.** Each run is a separate evidence bundle; mixing them breaks the SHA-256 manifest and undermines tamper-evidence. +**Keep each run's output directory separate.** Each run has its own audit chain and run id; copying two runs' files into one directory mixes them. Only the audit log (`forgelm verify-audit`) and `final_model/` (`forgelm verify-integrity`) are tamper-checked; nothing hashes `compliance/`. ::: ## See also diff --git a/docs/usermanuals/tr/compliance/human-oversight.md b/docs/usermanuals/tr/compliance/human-oversight.md index 65a1f336..4761258d 100644 --- a/docs/usermanuals/tr/compliance/human-oversight.md +++ b/docs/usermanuals/tr/compliance/human-oversight.md @@ -48,12 +48,8 @@ webhook: Trainer eval'den sonra durur ve yazdırır: ```text -[2026-04-29 14:33:10] Human approval required. - Run ID: abc123 - Bundle: checkpoints/run/artifacts/ - - To approve: forgelm approve abc123 --output-dir checkpoints/run --comment "..." - To reject: forgelm reject abc123 --output-dir checkpoints/run --comment "..." +2026-04-29 14:33:10 [INFO] forgelm.trainer: Human approval required. Model staged at: checkpoints/run/final_model.staging.abc123 +2026-04-29 14:33:10 [INFO] forgelm.trainer: Review results in checkpoints/run/compliance/ and run `forgelm approve abc123 --output-dir checkpoints/run` to promote, or `forgelm reject abc123 --output-dir checkpoints/run` to preserve for forensic review. ``` Reviewer audit-log + staging dizinine erişimi olan herhangi bir makineden onay komutunu çalıştırır. ForgeLM kimliği `FORGELM_OPERATOR`'dan resolve eder, kararı HMAC ile zincirler ve (`approve`'da) staging dizinini kanonik `final_model/` yoluna rename eder. diff --git a/docs/usermanuals/tr/getting-started/first-run.md b/docs/usermanuals/tr/getting-started/first-run.md index 1f65a3eb..46b1cd20 100644 --- a/docs/usermanuals/tr/getting-started/first-run.md +++ b/docs/usermanuals/tr/getting-started/first-run.md @@ -29,22 +29,23 @@ flowchart LR Her şeyden önce Python, PyTorch, CUDA ve opsiyonel bağımlılıkların doğru bağlandığını kontrol etmek için `forgelm doctor` çalıştırın: ```shell -$ forgelm doctor -forgelm doctor — environment check - - [✓ pass] python.version Python 3.11.4 (CPython). - [✓ pass] torch.cuda torch 2.4.0 with CUDA 12.4. - [✓ pass] gpu.inventory 1 GPU(s) — GPU0: NVIDIA RTX 4090 (24.0 GiB). - [✓ pass] extras.qlora Installed (module bitsandbytes, purpose: 4-bit / 8-bit QLoRA training (Linux GPUs only)). - [✓ pass] extras.unsloth Installed (module unsloth, purpose: Unsloth-accelerated training (Linux GPUs only)). - [! warn] extras.distributed Optional extra missing — install with: pip install 'forgelm[distributed]' (purpose: DeepSpeed ZeRO + offload distributed training (Linux + macOS only)). - [✓ pass] hf_hub.reachable HuggingFace Hub reachable (HTTP 200). - [✓ pass] disk.workspace Workspace /home/me/forgelm — 387.0 GiB free of 500.0 GiB. - [! warn] operator.identity FORGELM_OPERATOR not set; audit events will fall back to 'me@workstation'. Pin FORGELM_OPERATOR= for CI / pipeline runs. - -Summary: 7 pass, 2 warn, 0 fail. +$ forgelm doctor --offline +forgelm doctor - environment check + + [+ pass] python.version Python 3.11.4 (CPython). + [! warn] torch.cuda torch 2.4.0 installed but CUDA is unavailable. CPU-only runs are supported but training will be very slow; expect to use this only for tiny experiments or smoke tests. + [+ pass] numpy.torch_abi torch 2.4.0 + numpy 2.1.0 are ABI-compatible. + [! warn] gpu.inventory No CUDA devices visible; CPU-only mode. + [! warn] extras.distributed Optional extra missing — install with: pip install 'forgelm[distributed]' (purpose: DeepSpeed ZeRO + offload distributed training (Linux + macOS only)). + [+ pass] disk.workspace Workspace /home/me/forgelm — 387.0 GiB free of 500.0 GiB. + [! warn] operator.identity FORGELM_OPERATOR not set; audit events will fall back to 'me@workstation'. Pin FORGELM_OPERATOR= for CI / pipeline runs so the audit log identifies a stable identity. + ... + +Summary: 7 pass, 12 warn, 0 fail. ``` +Bu bir alıntıdır: gerçek çıktı yaklaşık 19 satırdır (ForgeLM kurulumunu, her opsiyonel extra'yı ve HF cache'i ya da Hub'ı da denetler); satırlar, değerler ve sayılar makinenize göre değişir. Bir extra için `pass`, modülün bulunabildiği anlamına gelir, import testinden geçtiği anlamına gelmez; durum işaretleri `[+ pass]`, `[! warn]` ve `[x fail]`'dir. + `--output-format json` yapısal bir zarf döner (`{"success": bool, "checks": [...], "summary": {...}}`); CI ayrı ayrı probe sonuçlarını tabloyu parse etmeden filtreleyebilir. `--offline` HF Hub ağ probe'unu atlar ve onun yerine yerel cache'i inceler — air-gap dağıtımları için faydalı. :::tip @@ -137,17 +138,10 @@ Olası verdict'ler: ```shell $ forgelm --config configs/quickstart-customer-support.yaml -[2026-04-29 14:01:32] config validated -[2026-04-29 14:01:33] auditing data/customer-support.jsonl (12,400 rows, 3 splits) -[2026-04-29 14:01:35] PII flags: 0 critical, 5 medium · cross-split overlap: 0 -[2026-04-29 14:01:37] SFT epoch 1/3 · loss=2.31 → 1.42 -[2026-04-29 14:18:55] DPO preference pass · β=0.1 · KL=4.2 -[2026-04-29 14:32:11] benchmark hellaswag=0.62 truthfulqa=0.48 -[2026-04-29 14:33:02] Llama Guard S1-S14: clean -[2026-04-29 14:33:04] Annex IV → checkpoints/customer-support/artifacts/annex_iv_metadata.json -[2026-04-29 14:33:04] ✔ finished, exit 0 ``` +Çıktı, `forgelm.*` logger'larından gelen zaman damgalı `[INFO]` satırlarının ve transformers ile TRL'in bastığı ilerleme çubuğu ile adım başı loss satırlarının akışıdır. `customer-support` şablonu `evaluation:` bloğu olmayan bir SFT koşusudur; bu yüzden göreceğiniz aşamalar şunlardır: config doğrulama, eğitim, ardından aşağıdaki dosyaların yazılması. Benchmark, güvenlik değerlendirmesi, judge ve insan-onay kapısı gibi aşamalar yalnızca YAML bunları açtığında çalışır. Komut başarıda `0` çıkış koduyla biter ([Çıkış kodları](#/reference/exit-codes)). + ## 6. Modeli dene ```shell @@ -160,20 +154,32 @@ Erişiminiz mevcut faturalama döneminin sonuna kadar devam eder… ## Diskte ne var ```text -checkpoints/customer-support/ -├── artifacts/ -│ ├── annex_iv_metadata.json ← Article 11 teknik dokümantasyon -│ ├── audit_log.jsonl ← Article 12 append-only event log -│ ├── data_audit_report.json ← Article 10 veri yönetimi kanıtı -│ ├── safety_report.json ← Llama Guard verdict -│ ├── benchmark_results.json ← Görev başı doğruluk -│ └── manifest.json ← Her artifact üzerinde SHA-256 -├── README.md ← Article 13 model card -├── config_snapshot.yaml ← Bu koşunun kullandığı tam YAML -└── adapter_model.safetensors ← LoRA ağırlıkları (veya birleştirilmiş checkpoint) +checkpoints/customer-support/ ← training.output_dir +├── audit_log.jsonl ← Article 12 append-only event log, hash zincirli +├── audit_log.jsonl.manifest.json ← genesis pin: verify-audit'in kesilmeyi tespit etmesini sağlar +├── compliance/ +│ ├── compliance_report.json ← Article 11 manifesti (kanonik makine-okunur paket) +│ ├── training_manifest.yaml ← koşunun operatör-okunur özeti +│ ├── data_provenance.json ← Article 10 veri seti parmak izi +│ └── data_governance_report.json ← Article 10 veri yönetimi kanıtı +├── checkpoint-/ ← ara Trainer checkpoint'leri +└── final_model/ + ├── adapter_model.safetensors ← LoRA ağırlıkları (veya birleştirilmiş checkpoint) + ├── README.md ← Article 13 model card + ├── deployer_instructions.md ← modeli dağıtacak kişi için notlar + └── model_integrity.json ← final_model/ içindeki dosyaların SHA-256 özetleri ``` -O `artifacts/` dizini compliance incelemeleri için teslim edilebilirdir. İçindeki her dosya `manifest.json`'da tamper-evidence için hashlenmiştir. +Bazı dosyalar yalnızca YAML istediğinde oluşur: + +| Dosya | Ne zaman oluşur | +|---|---| +| `compliance/annex_iv_metadata.json` | `compliance.provider_name`, `system_name` veya `intended_purpose` ayarlıysa | +| `safety/safety_results.json` | güvenlik değerlendirmesi açıksa | +| `benchmark/benchmark_results.json` | benchmark kapısı açıksa | +| `final_model/` yerine `final_model.staging./` | `evaluation.require_human_approval` açıksa, `forgelm approve` onu terfi ettirene kadar | + +`artifacts/` dizini ve tüm paketi kapsayan tek bir manifest yoktur. İki şey tamper-check'lidir: audit log kendi hash zinciriyle (`forgelm verify-audit`) ve `final_model/` `model_integrity.json` ile (`forgelm verify-integrity`). `compliance/` dosyalarını hiçbir şey hash'lemez; bu yüzden onları bir değişikliğin görüneceği bir yerde tutun, örneğin sürüm kontrolünde ya da bir kez yazılabilir depolamada. ## Sonraki adımlar diff --git a/docs/usermanuals/tr/getting-started/project-layout.md b/docs/usermanuals/tr/getting-started/project-layout.md index e24bf6ef..568043f9 100644 --- a/docs/usermanuals/tr/getting-started/project-layout.md +++ b/docs/usermanuals/tr/getting-started/project-layout.md @@ -22,10 +22,9 @@ my-finetune/ │ └── data_audit_report.json ├── checkpoints/ — model çıktıları (gitignored) │ └── customer-support/ -│ ├── adapter_model.safetensors -│ ├── README.md — model card (Article 13) -│ ├── config_snapshot.yaml — kullanılan config'in dondurulmuş kopyası -│ └── artifacts/ — compliance kanıt paketi +│ ├── audit_log.jsonl — append-only, hash zincirli olay günlüğü +│ ├── compliance/ — Article 10/11 kanıtı (JSON + YAML) +│ └── final_model/ — ağırlıklar, model card (README.md), model_integrity.json ├── ingested/ — `forgelm ingest` üzerinden ham döküman → JSONL └── .forgelm/ — yerel cache (gitignored) ``` @@ -50,11 +49,11 @@ ForgeLM varsayılanları doğru şeyleri commit'leyip gerisini ignore'larsanız **Commit:** - `configs/*.yaml` — koşularınız konfigürasyon; YAML kaynak doğrudur. - `audit/*.json` — küçük, makine-okunur, PR review'larında faydalı ("audit numaraları geriledi mi?"). -- `checkpoints/*/README.md` ve `checkpoints/*/artifacts/` (model card ve audit paketi, ikisi de küçük). +- `checkpoints/*/final_model/README.md` ve `checkpoints/*/compliance/` (model card ve compliance kanıtı, ikisi de küçük). **Gitignore:** - `data/` — genelde fazla büyük; DVC veya tercih ettiğiniz dataset registry'sine alın. -- `checkpoints/*/adapter_model.safetensors` ve diğer ağırlık dosyaları — git için fazla büyük; HuggingFace Hub veya model registry'ye gönderin. +- `checkpoints/*/final_model/*.safetensors`, `checkpoint-*` dizinleri ve diğer ağırlık dosyaları — git için fazla büyük; HuggingFace Hub veya model registry'ye gönderin. - `ingested/` — ham dokümanlardan yeniden inşa edilebilir. - `.forgelm/` — yerel cache. @@ -123,10 +122,10 @@ Geliştirme sırasında dev varyantı, release için tam olanını koşturun. CI ## Compliance artifact'ları nereye gider -Her başarılı (veya başarısız) eğitim koşusu, [Compliance Genel Bakış](#/compliance/overview) bölümünde anlatılan kanıt paketini içeren bir `checkpoints//artifacts/` dizini oluşturur. Regülatöre veya CI artifact'larında pinleyeceğiniz dizin budur. +Bir eğitim koşusu kanıtını `training.output_dir` içine yazar: en üstte `audit_log.jsonl` (`.manifest.json` genesis pin'iyle) ve `final_model/` yanında bir `compliance/` dizini; tam ağacı ve hangi dosyaların YAML'a bağlı olduğunu [İlk Koşunuz](#/getting-started/first-run) gösterir. `artifacts/` dizini yoktur. `compliance/` ve `audit_log.jsonl`'i bir incelemeciye teslim edin ya da CI artifact'larında pinleyin. :::warn -**`artifacts/` dizinini koşular arasında birleştirmeyin.** Her koşu ayrı bir kanıt paketidir; karıştırmak SHA-256 manifesti bozar ve tamper-evidence'ı baltalar. +**Her koşunun çıktı dizinini ayrı tutun.** Her koşunun kendi audit zinciri ve run id'si vardır; iki koşunun dosyalarını tek dizine kopyalamak onları karıştırır. Yalnızca audit log (`forgelm verify-audit`) ve `final_model/` (`forgelm verify-integrity`) tamper-check'lidir; `compliance/`'ı hiçbir şey hash'lemez. ::: ## Bkz. From f91e586f0d623796e6c814db7c9cf27218e41318 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 10:47:44 +0300 Subject: [PATCH 4/5] docs(offline): describe --offline as what it does --offline (and model.offline) sets HF_HUB_OFFLINE, TRANSFORMERS_OFFLINE and HF_DATASETS_OFFLINE. Nothing else reads it: webhook delivery, the LLM-judge and synthetic-data API clients, and W&B / MLflow are unaffected (the air-gap user-manual page already said so after an earlier correction). The --help text, the model.offline field description, the configuration and CLI references (English and Turkish) and the enterprise deployment guide still said "Air-gapped mode" and "Prevents any network calls", which is how an operator who relies on the flag as an egress control would be misled. They now say which variables are set and what is not stopped, and the enterprise guide names the settings that finish the job (omit webhook:, use local judge/teacher models, report_to: none). The air-gap table no longer lists Comet, which training.report_to (tensorboard, wandb, mlflow, none) does not accept. All edits are in place, so the parser and config.py line budgets are unchanged. Gating those clients behind the flag is a code change and stays open. Refs #74, #100, #893 Co-Authored-By: Claude Sonnet 5.5 --- CHANGELOG.md | 5 +++++ docs/guides/enterprise_deployment.md | 2 +- docs/reference/configuration-tr.md | 2 +- docs/reference/configuration.md | 2 +- docs/usermanuals/en/operations/air-gap.md | 2 +- docs/usermanuals/en/reference/cli.md | 2 +- docs/usermanuals/tr/operations/air-gap.md | 2 +- docs/usermanuals/tr/reference/cli.md | 2 +- forgelm/cli/_parser.py | 2 +- forgelm/config.py | 2 +- 10 files changed, 14 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d44dd02..d7fa7b17 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,11 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Describe `--offline` / `model.offline` as what it does. It sets `HF_HUB_OFFLINE`, `TRANSFORMERS_OFFLINE` and `HF_DATASETS_OFFLINE`; it does not + stop webhooks, the LLM-judge and synthetic-data API clients, or W&B / MLflow. The `--offline` help text, the `model.offline` field + description, the configuration and CLI references (English and Turkish) and the enterprise deployment guide said "Air-gapped mode" and + "prevents any network calls"; the air-gap page no longer lists Comet, which `training.report_to` does not accept. Gating those clients + behind the flag is a code change and stays open (#74, #100, #893). - Fix the KTO bullet on the Alignment overview user-manual page (English and Turkish): it named the fields `(prompt, response, good?)`, but the KTO loader reads `prompt`, `completion` and a boolean `label` (#965). diff --git a/docs/guides/enterprise_deployment.md b/docs/guides/enterprise_deployment.md index f9c70e60..0a433df6 100644 --- a/docs/guides/enterprise_deployment.md +++ b/docs/guides/enterprise_deployment.md @@ -152,7 +152,7 @@ See the [Distributed Training Guide](../reference/distributed_training.md) for d | `trust_remote_code` | `false` | Prevents arbitrary code execution from model repos | | `auth.hf_token` | Use `HUGGINGFACE_TOKEN` env var | Don't hardcode tokens in YAML | | `webhook.url` | Always use `url_env` with environment variable | Webhook tokens excluded from model cards (v0.3.1rc1+), but avoid direct URLs for credential hygiene | -| `offline` | `true` for air-gapped | Prevents any network calls | +| `offline` | `true` for air-gapped | Blocks Hugging Face Hub, transformers and datasets network calls (it sets `HF_HUB_OFFLINE`, `TRANSFORMERS_OFFLINE`, `HF_DATASETS_OFFLINE`). It does **not** stop webhooks, judge or synthetic-data API clients, or W&B / MLflow: disable those (omit `webhook:`, use local judge and teacher models, set `report_to: none`) for a true air gap | --- diff --git a/docs/reference/configuration-tr.md b/docs/reference/configuration-tr.md index b78fa07f..8c930656 100644 --- a/docs/reference/configuration-tr.md +++ b/docs/reference/configuration-tr.md @@ -15,7 +15,7 @@ Tam açıklamalı örnek için `config_template.yaml` dosyasına bakın. | `load_in_4bit` | bool | `true` | QLoRA 4-bit NF4 kuantizasyon | | `backend` | string | `"transformers"` | `"transformers"` veya `"unsloth"` (desteklenen mimarilerde daha hızlı, Linux) | | `trust_remote_code` | bool | `false` | Model depolarından özel kod çalıştırma. **Güvenlik riski** | -| `offline` | bool | `false` | İzole mod: HF Hub çağrısı yok. Modeller/veri setleri yerel olmalı | +| `offline` | bool | `false` | HF offline değişkenlerini ayarlar; Hub, transformers ve datasets ağ çağrısı yapmaz. Modeller/veri setleri yerel olmalı. Webhook'ları, judge veya sentetik-veri API istemcilerini ve W&B / MLflow'u durdurmaz | | `revision` | string | `null` | Temel modeli + tokenizer'ı bir HF Hub commit SHA'sına (40-hex) veya branch/tag'e sabitle. **Bugün uygulanıyor.** Bkz. [Hub revision pinleme](#hub-revision-pinleme) | | `bnb_4bit_use_double_quant` | bool | `true` | Ekstra VRAM tasarrufu için çift kuantizasyon | | `bnb_4bit_quant_type` | string | `"nf4"` | Kuantizasyon tipi (`"nf4"` veya `"fp4"`) | diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 0f83d16e..c7719ba6 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -15,7 +15,7 @@ See `config_template.yaml` for a complete annotated example. | `load_in_4bit` | bool | `true` | Enable QLoRA 4-bit NF4 quantization | | `backend` | string | `"transformers"` | `"transformers"` or `"unsloth"` (faster on supported architectures, Linux only) | | `trust_remote_code` | bool | `false` | Allow custom code from model repos. **Security risk** — only enable for models that require it | -| `offline` | bool | `false` | Air-gapped mode: no HF Hub calls. Models/datasets must be local | +| `offline` | bool | `false` | Sets the HF offline variables, so Hub, transformers and datasets make no network calls. Models/datasets must be local. Does not stop webhooks, judge or synthetic-data API clients, or W&B / MLflow | | `revision` | string | `null` | Pin the base model + tokenizer to an HF Hub commit SHA (40-hex) or a branch/tag. **Honoured today.** See [Hub revision pinning](#hub-revision-pinning) | | `bnb_4bit_use_double_quant` | bool | `true` | Double quantization for extra VRAM savings | | `bnb_4bit_quant_type` | string | `"nf4"` | Quantization type (`"nf4"` or `"fp4"`) | diff --git a/docs/usermanuals/en/operations/air-gap.md b/docs/usermanuals/en/operations/air-gap.md index aacf64e5..398d6a02 100644 --- a/docs/usermanuals/en/operations/air-gap.md +++ b/docs/usermanuals/en/operations/air-gap.md @@ -59,7 +59,7 @@ ERROR: model "meta-llama/Llama-Guard-3-8B" not in local cache. | Subsystem | Flag effect or required operator preparation | |---|---| | HuggingFace Hub downloads | `--offline` sets the HF offline flags; pre-cache the required artifacts. | -| W&B / MLflow / Comet | Disable reporting or configure local-only MLflow with a file URI; `--offline` does not disable these integrations. | +| W&B / MLflow | Set `training.report_to` to `none` (or `tensorboard`), or configure local-only MLflow with a file URI; `--offline` does not disable these integrations. | | OpenAI / Anthropic judges | Disable API-backed evaluation or use a local judge; `--offline` does not disable API clients. | | Webhooks | `--offline` does not disable configured delivery. Omit the `webhook:` block, or leave `url` / `url_env` unset, to disable it; `allow_private_destinations` permits private destinations in addition to public ones. | | pip install (during runtime) | Install required dependencies before disconnecting; `--offline` does not control pip. | diff --git a/docs/usermanuals/en/reference/cli.md b/docs/usermanuals/en/reference/cli.md index a9423316..af59db43 100644 --- a/docs/usermanuals/en/reference/cli.md +++ b/docs/usermanuals/en/reference/cli.md @@ -44,7 +44,7 @@ Run `forgelm --help` for any of these. | `--dry-run` | Validate configuration and check model/dataset access; no training. | | `--fit-check` | Estimate peak training VRAM; no model load. Requires `--config`. | | `--resume [PATH]` | Resume training. Bare `--resume` auto-detects last checkpoint; `--resume PATH` resumes from a specific one. | -| `--offline` | Air-gapped mode: disable all HF Hub network calls. Models and datasets must be available locally. | +| `--offline` | Offline mode: set the Hugging Face offline variables so Hub, transformers and datasets make no network calls. Models and datasets must be available locally. Does not stop webhooks, judge or synthetic-data API clients, or experiment trackers. | | `--benchmark-only MODEL_PATH` | Run benchmark evaluation on an existing model without training. Requires `evaluation.benchmark` config. | | `--merge` | Run model merging from the `merge:` config block. No training. | | `--stage NAME` | Multi-stage pipelines only: run a single named stage in isolation. Non-first stages need the previous stage's on-disk output, or `--input-model`. | diff --git a/docs/usermanuals/tr/operations/air-gap.md b/docs/usermanuals/tr/operations/air-gap.md index 58dc233e..904e95eb 100644 --- a/docs/usermanuals/tr/operations/air-gap.md +++ b/docs/usermanuals/tr/operations/air-gap.md @@ -59,7 +59,7 @@ ERROR: model "meta-llama/Llama-Guard-3-8B" not in local cache. | Alt sistem | Bayrağın etkisi veya operatörün yapması gereken hazırlık | |---|---| | HuggingFace Hub indirmeleri | `--offline`, HF çevrimdışı ayarlarını yapar; gerekli artefaktları önceden önbelleğe alın. | -| W&B / MLflow / Comet | Raporlamayı kapatın veya MLflow'u file URI ile yalnızca yerel çalışacak şekilde yapılandırın; `--offline` bu entegrasyonları kapatmaz. | +| W&B / MLflow | `training.report_to`'yu `none` (veya `tensorboard`) yapın ya da MLflow'u file URI ile yalnızca yerel çalışacak şekilde yapılandırın; `--offline` bu entegrasyonları kapatmaz. | | OpenAI / Anthropic judge'lar | API tabanlı değerlendirmeyi kapatın veya yerel judge kullanın; `--offline` API istemcilerini kapatmaz. | | Webhook'lar | `--offline`, yapılandırılmış teslimatları kapatmaz. Kapatmak için `webhook:` bloğunu çıkarın veya `url` / `url_env` değerlerini boş bırakın; `allow_private_destinations`, genel hedeflerin yanında özel hedeflere de izin verir. | | Runtime'da pip install | Gerekli bağımlılıkları bağlantıyı kesmeden kurun; `--offline`, pip'i yönetmez. | diff --git a/docs/usermanuals/tr/reference/cli.md b/docs/usermanuals/tr/reference/cli.md index e1da7e23..9f679ae2 100644 --- a/docs/usermanuals/tr/reference/cli.md +++ b/docs/usermanuals/tr/reference/cli.md @@ -44,7 +44,7 @@ Bunlardan herhangi biri için `forgelm --help`. | `--dry-run` | Config'i ve model/dataset erişimini doğrula; eğitim yok. | | `--fit-check` | Eğitim VRAM tahmini; model yüklenmez. `--config` gerektirir. | | `--resume [PATH]` | Eğitime kaldığı yerden devam. Çıplak `--resume` son checkpoint'i otomatik bulur; `--resume PATH` belirli bir yerden. | -| `--offline` | Air-gap modu: tüm HF Hub ağ çağrılarını kapat. Modeller ve dataset'ler yerel olarak mevcut olmalı. | +| `--offline` | Offline mod: Hugging Face offline değişkenlerini ayarlar; Hub, transformers ve datasets ağ çağrısı yapmaz. Modeller ve dataset'ler yerel olarak mevcut olmalı. Webhook'ları, judge veya sentetik-veri API istemcilerini ve deney izleyicilerini durdurmaz. | | `--benchmark-only MODEL_PATH` | Mevcut bir model üzerinde benchmark koştur, eğitim yok. `evaluation.benchmark` config'i gerektirir. | | `--merge` | Config'in `merge:` bloğundan model birleştirmeyi koştur. Eğitim yok. | | `--stage NAME` | Yalnızca çok aşamalı pipeline'lar: adı verilen tek bir stage'i izole koştur. İlk olmayan stage'ler önceki stage'in on-disk çıktısına ya da `--input-model` override'ına ihtiyaç duyar. | diff --git a/forgelm/cli/_parser.py b/forgelm/cli/_parser.py index 562c6343..b76d985b 100644 --- a/forgelm/cli/_parser.py +++ b/forgelm/cli/_parser.py @@ -1368,7 +1368,7 @@ def parse_args() -> argparse.Namespace: parser.add_argument( "--offline", action="store_true", - help="Air-gapped mode: disable all HF Hub network calls. Models and datasets must be available locally.", + help="Offline mode: set the Hugging Face offline variables so Hub, transformers and datasets make no network calls. Models and datasets must be available locally. Does not stop webhooks, judge or synthetic-data API clients, or experiment trackers.", ) mode_group.add_argument( "--benchmark-only", diff --git a/forgelm/config.py b/forgelm/config.py index 4114f2df..534a4874 100644 --- a/forgelm/config.py +++ b/forgelm/config.py @@ -269,7 +269,7 @@ class ModelConfig(BaseModel): ) offline: bool = Field( default=False, - description="Air-gapped mode: refuse HF Hub network calls. Models/datasets/extras must be available locally.", + description="Offline mode: set the HF offline variables (Hub, transformers, datasets). Models/datasets/extras must be available locally. Webhooks, judge/synthetic API clients and trackers are not stopped.", ) revision: Optional[str] = Field( default=None, From 784ead2bd7c11b77e3672e7044b2e773c1c6d67e Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 12:21:29 +0300 Subject: [PATCH 5/5] docs: fix two review findings (Slack signing secret, preset availability) CodeRabbit: Slack's signing secret verifies requests from Slack, not requests to an incoming-webhook URL, so it is not a control for receiving ForgeLM webhooks; ForgeLM sends no signature, so a custom relay needs its own authentication. Corrected in the access-control SOP and both webhook pages, English and Turkish. And a DeepSpeed preset name works whenever configs/deepspeed/.json is available beside the package or in the working directory; a source checkout is merely what provides it. Refs #112, #110 Co-Authored-By: Claude Sonnet 5.5 --- docs/qms/access_control-tr.md | 9 ++++++--- docs/qms/access_control.md | 9 ++++++--- docs/reference/distributed_training-tr.md | 9 +++++---- docs/reference/distributed_training.md | 9 +++++---- docs/usermanuals/en/operations/webhooks.md | 2 +- docs/usermanuals/tr/operations/webhooks.md | 2 +- 6 files changed, 24 insertions(+), 16 deletions(-) diff --git a/docs/qms/access_control-tr.md b/docs/qms/access_control-tr.md index 4ad7f9fb..67f3ea60 100644 --- a/docs/qms/access_control-tr.md +++ b/docs/qms/access_control-tr.md @@ -231,9 +231,12 @@ webhook: ForgeLM webhook gövdelerini HMAC ile **imzalamaz** — `WebhookConfig` üzerinde `webhook.secret_env` alanı yoktur (bkz. `forgelm/config.py::WebhookConfig`). Hedef-tarafı özgünlük (a) HTTPS + -`webhook.url_env` üzerinden URL-gizliliğine ve (b) alıcı sistemin -kendi bearer-token / imzalı-istek kontrollerine (örneğin Slack'in -signing secret'ı) düşer. Webhook payload'u `FORGELM_OPERATOR` +`webhook.url_env` üzerinden URL-gizliliğine ve (b) alıcı tarafın kendi +uyguladığı kimlik doğrulamaya düşer. Bir Slack incoming webhook'unun gizli +URL'sinden başka bir doğrulaması yoktur (Slack'in signing secret'ı Slack'ten +*gelen* istekleri doğrular, bir webhook URL'sine gelenleri değil); bu yüzden +özel bir relay kendi kimlik doğrulamasını istemelidir, çünkü ForgeLM imza +göndermez. Webhook payload'u `FORGELM_OPERATOR` kimliğini **taşımaz**; bir modeli kimin çalıştırdığı veya onayladığı bildirimden değil, audit log'dan okunur. diff --git a/docs/qms/access_control.md b/docs/qms/access_control.md index ad5d97c1..9e5f344a 100644 --- a/docs/qms/access_control.md +++ b/docs/qms/access_control.md @@ -229,9 +229,12 @@ webhook: ForgeLM does **not** HMAC-sign webhook bodies — there is no `webhook.secret_env` field on `WebhookConfig` (see `forgelm/config.py::WebhookConfig`). Destination-side authenticity falls to -(a) HTTPS + URL-secrecy via `webhook.url_env` and (b) the receiving -system's own bearer-token / signed-request controls (for example Slack's -signing secret). A webhook payload does **not** carry the +(a) HTTPS + URL-secrecy via `webhook.url_env` and (b) authentication +the receiving side applies itself. A Slack incoming webhook has none +beyond its secret URL (Slack's signing secret verifies requests *from* +Slack, not requests to a webhook URL), so a custom relay should require +its own authentication, because ForgeLM sends no signature. A webhook +payload does **not** carry the `FORGELM_OPERATOR` identity; attribution of who ran or approved a model comes from the audit log, not from the notification. diff --git a/docs/reference/distributed_training-tr.md b/docs/reference/distributed_training-tr.md index 0295e962..6054462b 100644 --- a/docs/reference/distributed_training-tr.md +++ b/docs/reference/distributed_training-tr.md @@ -51,10 +51,11 @@ torchrun --nproc_per_node=4 -m forgelm.cli --config my_config.yaml Depo, `configs/deepspeed/` altında üç DeepSpeed ön ayarı barındırır. `deepspeed_config` parametresine bunların adlarından birini veya kendi JSON dosyanızın yolunu verin. -**Ön ayar adları yalnızca bir kaynak checkout'unda çalışır.** Kurulu paketin yanındaki ya da geçerli dizindeki -`configs/deepspeed/.json` dosyasına çözülürler. `pip install forgelm` wheel'i ve Docker imajı bu dosyaları içermez; -orada bir ön ayar adı yapılandırma hatasıyla (exit 1) başarısız olur. Deponun `configs/deepspeed/` klasöründeki JSON -dosyalarını kopyalayın ya da kendinizinkini yazın ve `deepspeed_config`'i onun yoluna ayarlayın. +**Bir ön ayar adı yalnızca `configs/deepspeed/.json` kurulu paketin yanında ya da geçerli dizinde mevcutsa çalışır.** +Kaynak checkout'u bu dosyaları sağlar; `pip install forgelm` wheel'i ve Docker imajı sağlamaz, orada bir ön ayar adı +yapılandırma hatasıyla (exit 1) başarısız olur. Deponun `configs/deepspeed/` klasöründeki JSON dosyalarını kopyalayın +(çalışma dizinindeki bir `configs/deepspeed/` dizinine ya da `deepspeed_config`'i yoluna yönelttiğiniz herhangi bir yere) +veya kendinizinkini yazın. **Bilinen sorun ([#410](https://github.com/HodeTech/ForgeLM/issues/410)):** `deepspeed_config` ayarlanmadığında da kullanılan `zero2` ön ayarı `allgather_bucket_size` değerini `"auto"` yapar; transformers'ın DeepSpeed entegrasyonu bu diff --git a/docs/reference/distributed_training.md b/docs/reference/distributed_training.md index 75b1f53a..95becdf9 100644 --- a/docs/reference/distributed_training.md +++ b/docs/reference/distributed_training.md @@ -51,10 +51,11 @@ torchrun --nproc_per_node=4 -m forgelm.cli --config my_config.yaml The repository ships three DeepSpeed presets under `configs/deepspeed/`. Set `deepspeed_config` to one of their names, or provide a path to your own JSON file. -**The preset names work only from a source checkout.** They resolve to `configs/deepspeed/.json` beside the -installed package or in the current directory. A `pip install forgelm` wheel and the Docker image do not contain those -files, so there a preset name fails with a configuration error (exit 1). Copy the JSON files from the repository's -`configs/deepspeed/` folder, or write your own, and set `deepspeed_config` to its path. +**A preset name works only when `configs/deepspeed/.json` is available beside the installed package or in the +current directory.** A source checkout provides those files; a `pip install forgelm` wheel and the Docker image do not, +so there a preset name fails with a configuration error (exit 1). Copy the JSON files from the repository's +`configs/deepspeed/` folder (into a `configs/deepspeed/` directory in the working directory, or anywhere if you point +`deepspeed_config` at the path), or write your own. **Known issue ([#410](https://github.com/HodeTech/ForgeLM/issues/410)):** the `zero2` preset, which is also used when `deepspeed_config` is unset, sets `allgather_bucket_size` to `"auto"`, a value transformers' DeepSpeed integration does diff --git a/docs/usermanuals/en/operations/webhooks.md b/docs/usermanuals/en/operations/webhooks.md index 52e76eb3..ffccf7bf 100644 --- a/docs/usermanuals/en/operations/webhooks.md +++ b/docs/usermanuals/en/operations/webhooks.md @@ -134,7 +134,7 @@ live outside ForgeLM. - **TLS strongly recommended.** ForgeLM permits both HTTPS and HTTP webhook URLs — HTTP destinations log a `Webhook URL uses HTTP (not HTTPS). Data will be sent unencrypted.` warning but are not rejected by default (see `forgelm/webhook.py` `_send`). On a regulated estate set `webhook.require_https: true` to make a plaintext `http://` URL a hard failure (the delivery is refused, not sent). Pin `https://` URLs in production. - **Curated payload.** ForgeLM never includes raw training data, full configs, or unredacted PII in webhook payloads. The notifier wraps a fixed-shape JSON; there is no `webhook.redact` toggle because there's nothing user-controllable to redact. - **Server-Side Request Forgery (SSRF) guard.** ForgeLM blocks webhook URLs pointing at internal IPs — RFC 1918, loopback, link-local (incl. cloud IMDS at `169.254.169.254`), RFC 6598 Shared Address Space / Carrier-Grade-NAT (`100.64.0.0/10`, which also covers Alibaba Cloud ECS's IMDS at `100.100.100.200` — the stdlib `ipaddress` predicates don't flag that block as private or reserved on their own), plus reserved and multicast destinations — unless you explicitly opt-in with `webhook.allow_private_destinations: true`. This prevents misconfigured runs from probing your internal network or leaking the payload to a cloud metadata service. -- **No HMAC body signing.** ForgeLM does not sign webhook bodies — destination-side authenticity falls to TLS + URL secrecy via `url_env` plus the receiving system's bearer-token / signed-request controls (for example Slack's signing secret). +- **No HMAC body signing.** ForgeLM does not sign webhook bodies — destination-side authenticity falls to TLS + URL secrecy via `url_env` plus whatever authentication the receiver applies itself. A Slack incoming webhook has none beyond its secret URL (Slack's signing secret verifies requests *from* Slack, not requests to a webhook URL), so put your own authentication in front of a custom relay: ForgeLM sends no signature. ## Common pitfalls diff --git a/docs/usermanuals/tr/operations/webhooks.md b/docs/usermanuals/tr/operations/webhooks.md index efe79dab..722e15d5 100644 --- a/docs/usermanuals/tr/operations/webhooks.md +++ b/docs/usermanuals/tr/operations/webhooks.md @@ -136,7 +136,7 @@ payload zaten curated), ve routing hepsi ForgeLM'in dışında yaşar. - **TLS şiddetle önerilir.** ForgeLM hem HTTPS hem HTTP webhook URL'lerine izin verir — HTTP hedefleri `Webhook URL uses HTTP (not HTTPS). Data will be sent unencrypted.` uyarısı loglar ama varsayılan olarak reddedilmez (bkz. `forgelm/webhook.py` `_send`). Regüle bir ortamda `webhook.require_https: true` ile plaintext bir `http://` URL'ini hard failure yapın (teslimat reddedilir, gönderilmez). Üretimde `https://` URL'leri pinleyin. - **Curated payload.** ForgeLM webhook payload'larına asla raw eğitim verisi, tam config'ler veya unredacted PII koymaz. Notifier sabit-şekilli bir JSON sarar; `webhook.redact` toggle'ı yoktur çünkü kullanıcı-kontrollü redakte edilecek bir şey yok. - **SSRF guard.** ForgeLM iç IP'lere işaret eden webhook URL'lerini engeller — RFC 1918, loopback, link-local (bulut IMDS `169.254.169.254` dahil), RFC 6598 Shared Address Space / Carrier-Grade-NAT (`100.64.0.0/10`; bu blok, Alibaba Cloud ECS'in IMDS'ini de kapsar — `100.100.100.200` — stdlib `ipaddress` predicate'leri bu bloğu kendiliğinden private veya reserved olarak işaretlemez), ayrıca reserved ve multicast hedefler — `webhook.allow_private_destinations: true` ile açıkça opt-in olmadıkça. Yanlış konfigüre koşuların iç ağınızı sondalamasını veya payload'ı bir bulut metadata servisine sızdırmasını önler. -- **HMAC body imzalama yok.** ForgeLM webhook gövdelerini imzalamaz — hedef-tarafı authenticity TLS + `url_env` üzerinden URL gizliliği artı alıcı sistemin bearer-token / signed-request kontrollerine (örneğin Slack signing secret) düşer. +- **HMAC body imzalama yok.** ForgeLM webhook gövdelerini imzalamaz — hedef-tarafı authenticity TLS + `url_env` üzerinden URL gizliliği artı alıcının kendi uyguladığı kimlik doğrulamaya düşer. Bir Slack incoming webhook'unun gizli URL'sinden başka doğrulaması yoktur (Slack'in signing secret'ı Slack'ten *gelen* istekleri doğrular, bir webhook URL'sine gelenleri değil); bu yüzden özel bir relay'in önüne kendi kimlik doğrulamanızı koyun: ForgeLM imza göndermez. ## Sık hatalar