From 2ba095fbcacdd491d6e9c74486ae0ff46fd9eaa5 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Thu, 7 May 2026 23:28:49 +0300 Subject: [PATCH 01/14] docs(wave6-ab): canonical-name drift sweep across guides/qms/reference/usermanuals MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes Wave A (mechanical) + Wave B (canonical-name drift) of the 2026-05-07 docs audit plan. ≈240 of the original ≈270 audit findings remain after this sweep — the deferred bulk is P1 (parallel-universe YAML schema) which needs the Wave D render-from-code pipeline. Mechanical (Wave A): - pipeline.reverted → model.reverted across 12 docs (21 lines) - ForgeLM 0.5.2 → 0.5.5 in 4 usermanual artefacts - README.md:59 forgelm purge --subject-id → --row-id --corpus - CHANGELOG bilingual-pair count 39 → 40 (verified _PAIRS = 40) - CLAUDE.md / CONTRIBUTING.md / docs/standards/testing.md test-module count 68 → 70 (verified find tests/test_*.py | wc -l = 70) - TR typos: 'Sıraki' → 'Sonraki' in installation + first-run Canonical-name drift (Wave B): - P2: compliance.human_approval → evaluation.require_human_approval (10 user-facing files; ComplianceMetadataConfig has no human_approval, EvaluationConfig.require_human_approval is the real field per forgelm/config.py:480) - P5: forgelm audit --add-row-ids reframed as Phase 28 backlog item (5 files); flag does not exist in current parser - P7: webhook event vocabulary synced to canonical 5-event surface (training.start/success/failure/reverted, approval.required) in 4 user-facing tables; webhook lifecycle method names corrected (notify_started/succeeded/failed → notify_start/success/failure) in 6 QMS/design/reference docs - P8: model.integrity_verified row added to audit_event_catalog (EN+TR); was emitted at trainer.py:1450 but undocumented Ghost-field cleanup: - webhook.secret_env removed from 5 files (no such field in WebhookConfig; webhook.py does not HMAC-sign bodies) - encryption_at_rest: true|false claim removed from 4 files (DataGovernanceConfig has no encryption field) QMS reconciliation: - statement_of_applicability column header semantics clarified in EN+TR (operator-side vs ForgeLM-evidence distinction); resolves the apparent SoA ↔ iso27001_control_mapping conflict on 10 A.5.x controls TR mirror parity: - operations/cicd-tr.md GitLab stages: added missing - deploy - data/audit-tr.md JSON schema expanded ({...} → full) - concepts/data-formats-tr.md KTO JSON formatted to match EN Earlier review-comment fixes folded in (from inline review verification): - site/css/style.css:2060,2328 currentColor → currentcolor (lowercase per CSS spec / Stylelint canonical form) - site/quickstart.html:363-365 wizard modal architectural comment reflects actual implementation (hidden + sibling aria-hidden; no inert) Validated: bilingual_parity ✓, anchor_resolution ✓, cli_help_consistency ✓ (all --strict). Deferred to Wave D: - P1 parallel-universe YAML schema (~22 CRITICAL + 27 HIGH pages) — needs render-from-code via tools/regenerate_yaml_templates_doc.py - compliance.config_hash reframe (>10 files; nuanced rewrite) - usermanuals training/{kto,orpo,grpo}-tr.md output: blocks (EN itself uses ghost output: block; full canonical rewrite belongs in P1 lift) Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 4 +- CLAUDE.md | 2 +- CONTRIBUTING.md | 2 +- README.md | 2 +- docs/design/gdpr_erasure.md | 4 +- docs/design/iso27001_soc2_alignment.md | 16 ++--- docs/guides/gdpr_erasure-tr.md | 2 +- docs/guides/gdpr_erasure.md | 2 +- docs/guides/iso_soc2_deployer_guide-tr.md | 18 +++--- docs/guides/iso_soc2_deployer_guide.md | 18 +++--- docs/qms/access_control-tr.md | 4 +- docs/qms/access_control.md | 4 +- docs/qms/encryption_at_rest-tr.md | 9 ++- docs/qms/encryption_at_rest.md | 8 ++- docs/qms/risk_treatment_plan-tr.md | 6 +- docs/qms/risk_treatment_plan.md | 6 +- docs/qms/sop_incident_response-tr.md | 5 +- docs/qms/sop_incident_response.md | 5 +- docs/qms/statement_of_applicability-tr.md | 6 +- docs/qms/statement_of_applicability.md | 6 +- docs/reference/audit_event_catalog-tr.md | 1 + docs/reference/audit_event_catalog.md | 1 + docs/reference/compliance_summary.md | 4 +- docs/reference/iso27001_control_mapping-tr.md | 4 +- docs/reference/iso27001_control_mapping.md | 4 +- docs/reference/purge_subcommand-tr.md | 2 +- docs/reference/purge_subcommand.md | 2 +- .../soc2_trust_criteria_mapping-tr.md | 2 +- docs/reference/soc2_trust_criteria_mapping.md | 2 +- docs/standards/testing.md | 2 +- docs/usermanuals/en/compliance/audit-log.md | 28 +++++---- .../en/compliance/human-oversight.md | 2 +- docs/usermanuals/en/compliance/model-card.md | 4 +- docs/usermanuals/en/compliance/overview.md | 2 +- .../en/deployment/deploy-targets.md | 2 +- docs/usermanuals/en/operations/webhooks.md | 61 ++++++++++--------- docs/usermanuals/en/reference/exit-codes.md | 4 +- docs/usermanuals/tr/compliance/audit-log.md | 30 +++++---- .../tr/compliance/human-oversight.md | 2 +- docs/usermanuals/tr/compliance/model-card.md | 4 +- docs/usermanuals/tr/compliance/overview.md | 2 +- docs/usermanuals/tr/concepts/data-formats.md | 12 +++- docs/usermanuals/tr/data/audit.md | 12 +++- .../tr/deployment/deploy-targets.md | 2 +- .../tr/getting-started/first-run.md | 2 +- .../tr/getting-started/installation.md | 2 +- docs/usermanuals/tr/operations/cicd.md | 1 + docs/usermanuals/tr/operations/webhooks.md | 61 ++++++++++--------- docs/usermanuals/tr/reference/exit-codes.md | 4 +- site/css/style.css | 4 +- site/quickstart.html | 8 ++- 51 files changed, 230 insertions(+), 172 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b7d21be5..c54450f9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,8 +48,8 @@ triplets (50 new doc files):** - `docs/usermanuals/_meta.yaml` registers the 8 new pages with full 6-language title sets (EN+TR mandatory; DE/FR/ES/ZH fallback to EN per Task L policy). -- `tools/check_bilingual_parity.py::_PAIRS` grew from 23 to 39 - pairs (16 new pairs across guides + reference). +- `tools/check_bilingual_parity.py::_PAIRS` grew from 23 to 40 + pairs (17 new pairs across guides + reference). **Faz 30 Task J — `tools/check_cli_help_consistency.py` (new guard):** diff --git a/CLAUDE.md b/CLAUDE.md index 05611940..8e13a50e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -71,7 +71,7 @@ ForgeLM/ │ └── ... # benchmark, judge, merging, synthetic, wizard, │ # quickstart, model_card, fit_check, deploy, chat, │ # export, inference, results, utils -├── tests/ # 68 test modules; count grows over time (run `pytest --collect-only -q` for current) +├── tests/ # 70 test modules; count grows over time (run `pytest --collect-only -q` for current) ├── tools/ # CI guards: check_anchor_resolution, │ # check_bilingual_parity, check_cli_help_consistency, │ # check_field_descriptions, check_pip_audit, diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b153b972..25b87af2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -81,7 +81,7 @@ Push your branch and open a Pull Request against `main`. ForgeLM is a single-package layout: a mix of single-file modules and two focused sub-packages (`forgelm/cli/` post-Phase-15 split and -`forgelm/data_audit/` post-Phase-14 split) under `forgelm/`, ~68 test files +`forgelm/data_audit/` post-Phase-14 split) under `forgelm/`, ~70 test files under `tests/` (collected-test count grows over time — run `pytest --collect-only -q` for current), plus `configs/`, `docs/`, `tools/` (CI guards), and `notebooks/`. For the authoritative module-by-module map diff --git a/README.md b/README.md index 569b9ed7..7198150b 100644 --- a/README.md +++ b/README.md @@ -56,7 +56,7 @@ - **EU AI Act Compliance**: Auto-generated audit trails, data provenance (SHA-256), training manifests - **ISO 27001 / SOC 2 Type II Alignment**: Software cannot be ISO/SOC 2 *certified* — only organisations can — but ForgeLM produces the audit-trail, change-management, data-lineage, and supply-chain evidence the deployer's auditor explicitly asks for. CycloneDX 1.5 SBOM per release, `pip-audit` nightly, `bandit` CI, append-only HMAC-chained audit log (HMAC integrity is active **only when** `FORGELM_AUDIT_SECRET` is set + KMS-managed + rotated between output-dir lifecycles — see [`docs/qms/access_control.md`](docs/qms/access_control.md) §3.4), Article 14 staging gate, Article 15/17 GDPR tooling. See [`docs/guides/iso_soc2_deployer_guide.md`](docs/guides/iso_soc2_deployer_guide.md) for the deployer audit cookbook + 93-control coverage map. - **Library API**: `from forgelm import ForgeTrainer, audit_dataset, verify_audit_log, verify_annex_iv_artifact, verify_gguf, mask_pii, mask_secrets, ...` — every CLI surface has a stable Python entry point under `forgelm.__all__`, version-pinned via `forgelm.__api_version__` (decoupled from `__version__`). Run `python -c "import forgelm; print(sorted(forgelm.__all__))"` for the full surface. -- **GDPR Tooling**: `forgelm purge --subject-id ` (Article 17 right-to-erasure: redacts data + adapters in place; the audit log itself is append-only — Article 17(3)(b) preservation — and the erasure is recorded by appending six compensating events (`data.erasure_requested`, `data.erasure_completed`, etc.) so the hash chain remains intact) and `forgelm reverse-pii --query ` (Article 15 right-of-access: locates PII matches across artefacts without re-loading raw data). +- **GDPR Tooling**: `forgelm purge --row-id --corpus ` (Article 17 right-to-erasure: redacts data + adapters in place; the audit log itself is append-only — Article 17(3)(b) preservation — and the erasure is recorded by appending six compensating events (`data.erasure_requested`, `data.erasure_completed`, etc.) so the hash chain remains intact) and `forgelm reverse-pii --query ` (Article 15 right-of-access: locates PII matches across artefacts without re-loading raw data). - **Operational Subcommands**: `forgelm doctor` (env / GPU / CUDA / extras pre-flight), `forgelm cache-models` + `forgelm cache-tasks` (air-gap pre-cache for HF models + lm-eval tasks), `forgelm safety-eval` (standalone Llama Guard run), `forgelm verify-audit` / `verify-annex-iv` / `verify-gguf` (compliance + artefact verification toolbelt), `forgelm approvals` (list pending Article 14 staging-gate runs). - **Docker**: Official Dockerfile and docker-compose for portable deployment - **Offline / Air-Gapped**: Full operation without internet for regulated industries diff --git a/docs/design/gdpr_erasure.md b/docs/design/gdpr_erasure.md index 3d63955e..cb6c7509 100644 --- a/docs/design/gdpr_erasure.md +++ b/docs/design/gdpr_erasure.md @@ -177,7 +177,7 @@ A single invocation does exactly one of the three. Combining flags is a `Config | Flag | Required when | Type | Purpose | |---|---|---|---| -| `--row-id ` | corpus mode | str | Row to delete. **Required:** the JSONL must carry a stable id field (e.g. `"id"` or `"row_id"` key) on every row. Line-number fallback is **rejected** — operators with id-less corpora must run `forgelm audit --add-row-ids ` (Phase 28 follow-up) first, otherwise a re-ordered file would silently delete the wrong row. | +| `--row-id ` | corpus mode | str | Row to delete. **Required:** the JSONL must carry a stable id field (e.g. `"id"` or `"row_id"` key) on every row. Line-number fallback is **rejected** — operators with id-less corpora must pre-populate ids via an operator-side script (a `forgelm audit --add-row-ids` helper is on the Phase 28 backlog) first, otherwise a re-ordered file would silently delete the wrong row. | | `--corpus ` | corpus mode | str | Path to **a single JSONL file**. Directory mode is **rejected** in `--row-id` mode — multi-file purges are an operator script (loop over files), not a `forgelm purge` flag, because GDPR Article 17 expects an erasure decision *per row* with its own audit event. An invocation that matches the same id in two files is a "DELETE without WHERE" hazard. | | `--row-matches` | corpus mode optional | enum | `one` (default; refuses if 0 or ≥2 matches in the file) / `all` (deletes every match, requires explicit opt-in for the ambiguity). Without this flag, multi-row matches in a single file abort with `EXIT_CONFIG_ERROR` so the operator confirms intent before bulk delete. | | `--run-id ` | run mode | str | Run id matching `/audit_log.jsonl`'s top-level `run_id`. | @@ -276,7 +276,7 @@ Article 17(3)(b) preserves the audit log itself, but Article 5(1)(c) (data minim | Event field | Source / shape | Personal-data category | Phase 21 policy | |---|---|---|---| | `target_kind` | enum: `row` / `staging` / `artefacts` / `policy_check` | None | clear | -| `target_id` (row mode) | Row id from the JSONL `id` (or `row_id`) field — canonical and only source. Line-number fallback is **rejected** at the CLI per §4.2; operators with id-less corpora must run `forgelm audit --add-row-ids ` (Phase 28 follow-up) before invoking `forgelm purge --row-id`. | **Possibly personal** (operator-supplied row IDs may be email / username / ticket numbers) | **SHA-256(salt + value)** when emitted. Salt = first 16 bytes of `FORGELM_AUDIT_SECRET` if set, else a per-output-dir salt persisted at `/.forgelm_audit_salt` (16 random bytes, mode 0600, written on first emission). | +| `target_id` (row mode) | Row id from the JSONL `id` (or `row_id`) field — canonical and only source. Line-number fallback is **rejected** at the CLI per §4.2; operators with id-less corpora must pre-populate ids via an operator-side script (a `forgelm audit --add-row-ids` helper is on the Phase 28 backlog) before invoking `forgelm purge --row-id`. | **Possibly personal** (operator-supplied row IDs may be email / username / ticket numbers) | **SHA-256(salt + value)** when emitted. Salt = first 16 bytes of `FORGELM_AUDIT_SECRET` if set, else a per-output-dir salt persisted at `/.forgelm_audit_salt` (16 random bytes, mode 0600, written on first emission). | | `salt_source` (row mode, all erasure events) | Resolved salt source for the row's `target_id` hash | None (operational metadata) | One of `"env_var"` / `"per_dir"`. Persisted on every `data.erasure_requested` / `data.erasure_completed` / `data.erasure_failed` event in row mode so a compliance reviewer reconciling two erasure events for the same subject can detect a salt-source change. Salt-source mismatch across events for the same operator-supplied row id IS a hash discontinuity (different salt → different hash); the field exists so the discontinuity is visible in the chain instead of silently producing a "different subject" misreading. | | `target_id` (run mode) | `fg-` from AuditLogger | None (operational) | clear | | `corpus_path` | filesystem path | **Possibly personal** when path includes a subject name (rare but real) | clear by default; operator guide flags the risk + recommends symlinking through a stable name | diff --git a/docs/design/iso27001_soc2_alignment.md b/docs/design/iso27001_soc2_alignment.md index 4831554a..da431f79 100644 --- a/docs/design/iso27001_soc2_alignment.md +++ b/docs/design/iso27001_soc2_alignment.md @@ -117,8 +117,8 @@ auditable evidence. Cited symbols are real (verified against | Annex IV bundle verification | `forgelm verify-annex-iv` (Phase 36) | manifest-hash check, missing-artefact detection | | GGUF integrity | `forgelm verify-gguf` (Phase 36) | magic header + `.sha256` sidecar | | Risk classification gate | `_warn_high_risk_compliance` (Faz 28 strict gate) | `ConfigError` for high-risk + safety-disabled | -| Auto-revert | `evaluation.auto_revert` + safety-eval | `pipeline.reverted` audit event | -| Webhook lifecycle | `notify_started/succeeded/failed/reverted/awaiting_approval` (Phase 8) | webhook event payload + audit trail | +| Auto-revert | `evaluation.auto_revert` + safety-eval | `model.reverted` audit event | +| Webhook lifecycle | `notify_start/succeeded/failed/reverted/awaiting_approval` (Phase 8) | webhook event payload + audit trail | | HTTP discipline | `safe_post` (Phase 7) | SSRF guard, TLS-only, no-redirect, header masking | | Data audit pipeline | `forgelm audit` (Phase 11) | `data_audit_report.json` (PII / secrets / dedup) | | Annex IV §8 governance | `_data_governance_block` (compliance.py) | `data_governance_report.json` Article 10 | @@ -175,9 +175,9 @@ Each row classifies coverage as one of: | A.5.24 Information security incident management planning and preparation | FL-helps | `sop_incident_response.md` QMS template; audit chain preserves incident state | Establish IR team, on-call rotation | Org structure deployer-side | | A.5.25 Assessment and decision on information security events | FL-helps | `data.erasure_failed`, `pipeline.failed`, `audit.classifier_load_failed` events with `error_class`+`error_message` | Triage runbook tied to event class | Triage process deployer-side | | A.5.26 Response to information security incidents | FL-helps | Audit chain preserves before/after state through HMAC chain | Document runbook actions per event class | Runbook deployer-side | -| A.5.27 Learning from information security incidents | FL-helps | `pipeline.reverted` events accumulate post-mortem evidence | Weekly post-mortem cadence | Process deployer-side | +| A.5.27 Learning from information security incidents | FL-helps | `model.reverted` events accumulate post-mortem evidence | Weekly post-mortem cadence | Process deployer-side | | A.5.28 Collection of evidence | FL | `audit_log.jsonl` is forensic-grade (HMAC + hash chain + manifest sidecar); `forgelm verify-audit` validates | Ship to write-once storage (S3 Object Lock, Azure Immutable Blob) | Storage substrate deployer-side | -| A.5.29 Information security during disruption | FL-helps | `auto_revert` flips to baseline model on safety regression; `pipeline.reverted` event | Document base-model retention; multi-region replicas | DR infra deployer-side | +| A.5.29 Information security during disruption | FL-helps | `auto_revert` flips to baseline model on safety regression; `model.reverted` event | Document base-model retention; multi-region replicas | DR infra deployer-side | | A.5.30 ICT readiness for business continuity | OOS | — | DR planning | Out of scope | | A.5.31 Identification of legal, statutory, regulatory and contractual requirements | FL-helps | EU AI Act mapping in `compliance.py`; GDPR Article 15/17 handling; Annex IV bundle | Track jurisdictional rule changes (e.g. UK Online Safety Act updates) | Legal monitoring deployer-side | | A.5.32 Intellectual property rights | FL-helps | License extraction in SBOM; HF Hub model-card metadata recorded in manifest | Track per-model license terms | Legal review deployer-side | @@ -316,7 +316,7 @@ categories are scoped per-engagement. | CC7.1 | Detects vulnerabilities | `pip-audit` nightly (Faz 23); CVE feed | Vuln-mgmt cadence | | CC7.2 | Monitors system components | `forgelm verify-audit`; `forgelm verify-gguf`; `safety_trend.jsonl` | SIEM dashboards | | CC7.3 | Evaluates security events | `data.erasure_failed`, `pipeline.failed` events with `error_class`+`error_message` | Triage runbook | -| CC7.4 | Responds to security events | `auto_revert`, `pipeline.reverted` event; `data.erasure_failed` event | IR runbook | +| CC7.4 | Responds to security events | `auto_revert`, `model.reverted` event; `data.erasure_failed` event | IR runbook | | CC7.5 | Identifies, develops corrective actions| `human_approval.rejected` event; `sop_change_management.md` | CAPA cadence | | CC8.1 | Authorises changes | `forgelm approve` Article 14 gate; staging dir | Change Advisory Board | | CC9.1 | Identifies, manages risks | `risk_assessment` config + safety eval; `risk_treatment_plan.md` (Faz 23) | Risk register | @@ -602,8 +602,10 @@ produces harmful output, accuracy drop, formatting errors). severity. Runbook: pin to safe version, rebuild SBOM, regenerate dependent artefacts, notify deployer if model already shipped. 4. **Webhook target compromised** — webhook recipient confirms breach. - Runbook: rotate `webhook.secret_env`; re-emit lifecycle events from - audit chain to confirm attacker did not splice events. + Runbook: rotate the webhook URL (read via `webhook.url_env`) and + the destination-side bearer token; re-emit lifecycle events from + the audit chain to confirm attacker did not splice events. (HMAC + body signing is not yet implemented in ForgeLM — Phase 28+ backlog.) 5. **PII subject access request (Article 15)** — `forgelm reverse-pii` workflow + DSR response template. 6. **PII erasure request (Article 17)** — `forgelm purge` workflow + diff --git a/docs/guides/gdpr_erasure-tr.md b/docs/guides/gdpr_erasure-tr.md index 63609bfa..40d7b016 100644 --- a/docs/guides/gdpr_erasure-tr.md +++ b/docs/guides/gdpr_erasure-tr.md @@ -40,7 +40,7 @@ Tool şunları yapar: 1. Per-output-dir salt'ı `/.forgelm_audit_salt`'ta çözer (ilk kullanımda oluşturur; mode `0600`; `FORGELM_AUDIT_SECRET[:16]` set'liyse XOR'lar). **Not:** bu XOR yalnızca **identifier hashing**'i besler — audit-chain HMAC anahtarı bağımsız olarak `SHA-256(FORGELM_AUDIT_SECRET ‖ run_id)` şeklinde türetilir (bkz. [`docs/qms/access_control.md`](../qms/access_control.md) §3.4 / `forgelm/compliance.py:104-114`). İki primitif kasıtlı olarak ayrıdır. 2. Audit log'a `data.erasure_requested` yazar — **hash'lenmiş** target_id (`SHA-256(salt + value)`) ve operatör-sağlanan justification ile. -3. Eşleşen JSONL satırını `id` (veya `row_id`) field'ı ile bulur — re-order edilmiş bir dosyada sessiz yanlış-satır silmesini engellemek için satır-numarası fallback'i **reddedilir**. Id'siz corpus'lara sahip operatörler önce `forgelm audit --add-row-ids ` (Phase 28 follow-up) çalıştırmalı. +3. Eşleşen JSONL satırını `id` (veya `row_id`) field'ı ile bulur — re-order edilmiş bir dosyada sessiz yanlış-satır silmesini engellemek için satır-numarası fallback'i **reddedilir**. ForgeLM şu an bir id-doldurma yardımcısı sunmuyor (Phase 28 backlog'unda `forgelm audit --add-row-ids` flag'i bekliyor); id'siz corpus'lara sahip operatörler purge'den önce id'leri operatör-tarafı bir script ile (örn. `jq -c 'to_entries | with_entries(...)'` ya da tek-seferlik bir Python döngüsü) pre-populate etmeli. 4. Corpus'u atomic olarak yeniden yazar (kardeş bir temp dosyaya yazar + `os.replace`); operatörler ya tam silme-öncesi dosyayı ya da tam silme-sonrası dosyayı görür, asla kısmi state'i değil. 5. Uyarı koşullarını tespit eder ve eşleşen event'leri yayar (memorization, synthetic-data varlığı, dış kopyalar). 6. Disk operasyonu raise ederse `data.erasure_failed`, başarılıysa `data.erasure_completed` yazar; zincir nihai state'i yansıtır. diff --git a/docs/guides/gdpr_erasure.md b/docs/guides/gdpr_erasure.md index 2d82ef93..bc1c2e6f 100644 --- a/docs/guides/gdpr_erasure.md +++ b/docs/guides/gdpr_erasure.md @@ -40,7 +40,7 @@ The tool: 1. Resolves the per-output-dir salt at `/.forgelm_audit_salt` (creates it on first use; mode `0600`; XOR'd with `FORGELM_AUDIT_SECRET[:16]` if the env var is set). **Note:** this XOR feeds **identifier hashing only** — the audit-chain HMAC key is derived independently as `SHA-256(FORGELM_AUDIT_SECRET ‖ run_id)` (see [`docs/qms/access_control.md`](../qms/access_control.md) §3.4 / `forgelm/compliance.py:104-114`). The two primitives are intentionally separate. 2. Writes `data.erasure_requested` to the audit log with the **hashed** target_id (`SHA-256(salt + value)`) and the operator-supplied justification. -3. Locates the matching JSONL row by its `id` (or `row_id`) field — line-number fallback is **rejected** to defend against silent wrong-row deletion on a re-ordered file. Operators with id-less corpora must run `forgelm audit --add-row-ids ` (Phase 28 follow-up) first. +3. Locates the matching JSONL row by its `id` (or `row_id`) field — line-number fallback is **rejected** to defend against silent wrong-row deletion on a re-ordered file. ForgeLM does not currently provide an id-population helper (a `forgelm audit --add-row-ids` flag is on the Phase 28 backlog); operators with id-less corpora must pre-populate ids with an operator-side script (e.g. `jq -c 'to_entries | with_entries(...)'` or a one-shot Python loop) before invoking purge. 4. Atomically rewrites the corpus (writes a sibling temp file + `os.replace`); operators get either the full pre-erasure file or the full post-erasure file, never a partial state. 5. Detects warning conditions and emits the matching events (memorisation, synthetic-data presence, external copies). 6. Writes `data.erasure_completed` (or `data.erasure_failed` if the disk operation raised) so the chain reflects the final state. diff --git a/docs/guides/iso_soc2_deployer_guide-tr.md b/docs/guides/iso_soc2_deployer_guide-tr.md index 6d657cbc..77a5c5e5 100644 --- a/docs/guides/iso_soc2_deployer_guide-tr.md +++ b/docs/guides/iso_soc2_deployer_guide-tr.md @@ -196,8 +196,9 @@ substrate şifreler. Referans: - `docs/qms/encryption_at_rest.md` — kurum içi politikanız ForgeLM artefakt sınıflarına eşlenmiş. - KMS audit log — encryption-in-use'un substrate-side kanıtı. -- ForgeLM `data_governance_report.json` — config block'unuz operatör - beyanı başına `encryption_at_rest: true|false` kaydeder. +- Operatör-tarafı şifreleme tanıklığı — ForgeLM config-seviyesi bir + `encryption_at_rest` field'ı yüzeyleyene kadar (Phase 28+ backlog'u) + QMS evidence bundle'ınızda out-of-band olarak kaydedilir. ### S7: "Incident response göster — koşum ortasında safety classifier crash ederse ne olur?" @@ -250,10 +251,13 @@ Operatörlerin ilk denetimlerinde yanlış yaptığı şeyler: `FORGELM_OPERATOR=ci` yaptık" — audit chain yorumlanamaz hale gelir çünkü her giriş aynı string'e attribute olur. Pipeline başına + koşum başına namespaced identifier kullanın. -2. **Webhook secret'larını YAML'de saklamak.** - `webhook.secret_env` → KMS'den runtime'da resolve. Bir secret - rotate edilse bile version control'deki plaintext-secret YAML - bir bulgudur (denetçi `git log`'da tarihsel exposure görür). +2. **Webhook URL'lerini YAML'de saklamak.** `webhook.url_env: + SLACK_WEBHOOK` → secret manager'dan runtime'da resolve; plaintext + webhook URL'sini asla version control'e commit'leme. Rotate edilse + bile denetçi `git log`'da tarihsel exposure görür. (Not: ForgeLM şu + an webhook gövdelerini **imzalamıyor** — gateway-düzeyi bütünlük + mTLS ya da hedef-tarafı kontrollerine bağlı; HMAC body imzası Phase + 28+ backlog'unda.) 3. **CI'da `forgelm verify-audit` atlamak.** "Audit chain'e zımnen güveniyoruz" savunulabilir bir pozisyon değildir. Haftalık cron + sıfır-olmayan exit'e alarm zamanlayın; alarm geçmişini @@ -270,7 +274,7 @@ Operatörlerin ilk denetimlerinde yanlış yaptığı şeyler: 5. **Üretim eğitiminde `auto_revert` yok.** Her zaman yeşil eğitime bahis koyuyorsanız, regulator-reportable incident'a bir safety-classifier degradation'ı uzaktasınız. `auto_revert: - true` etkinleştirin ve `pipeline.reverted` event'lerinin + true` etkinleştirin ve `model.reverted` event'lerinin çalışan safeguard kanıtı olarak birikmesine izin verin. 6. **Üretimde `FORGELM_ALLOW_ANONYMOUS_OPERATOR=1` ile ForgeLM çalıştırmak.** O env var yalnız kısa-ömürlü test koşumları diff --git a/docs/guides/iso_soc2_deployer_guide.md b/docs/guides/iso_soc2_deployer_guide.md index ffbeb029..05a6e3b0 100644 --- a/docs/guides/iso_soc2_deployer_guide.md +++ b/docs/guides/iso_soc2_deployer_guide.md @@ -195,8 +195,9 @@ substrate does. Reference: - `docs/qms/encryption_at_rest.md` — your in-house policy mapped to ForgeLM artefact classes. - KMS audit log — substrate-side evidence of encryption-in-use. -- ForgeLM `data_governance_report.json` — your config block records - `encryption_at_rest: true|false` per the operator's declaration. +- Operator-side encryption attestation — recorded out-of-band in your + QMS evidence bundle until ForgeLM exposes a config-level + `encryption_at_rest` field (Phase 28+ backlog). ### Q7: "Show me the incident response — what happens if the safety classifier crashes mid-run?" @@ -251,10 +252,13 @@ Things deployers get wrong on their first audit: `FORGELM_OPERATOR=ci`" — the audit chain becomes uninterpretable because every entry attributes to the same string. Use namespaced identifiers per pipeline + per run. -2. **Storing webhook secrets in YAML.** Use `webhook.secret_env` → - resolve from KMS at runtime. A YAML in version control with a - plaintext secret is a finding even if the secret has been rotated - (the auditor sees historical exposure in `git log`). +2. **Storing webhook URLs in YAML.** Use `webhook.url_env: SLACK_WEBHOOK` + → resolve from your secret manager at runtime; never commit a + plaintext webhook URL to version control. The auditor sees + historical exposure in `git log` even after rotation. (Note: ForgeLM + does **not** sign webhook bodies today — gateway-level integrity + relies on mTLS or destination-side checks; HMAC body signing is on + the Phase 28+ backlog.) 3. **Skipping `forgelm verify-audit` in CI.** "We trust the audit chain implicitly" is not a defensible position. Schedule the weekly cron + alert on non-zero exit; have the alert history to @@ -271,7 +275,7 @@ Things deployers get wrong on their first audit: 5. **No `auto_revert` on production training.** If you're betting on always-green training, you're a single safety-classifier degradation away from a regulator-reportable incident. Enable - `auto_revert: true` and let `pipeline.reverted` events accumulate + `auto_revert: true` and let `model.reverted` events accumulate as evidence of working safeguards. 6. **Running ForgeLM with `FORGELM_ALLOW_ANONYMOUS_OPERATOR=1` in production.** That env var exists for short-lived test runs only. diff --git a/docs/qms/access_control-tr.md b/docs/qms/access_control-tr.md index 0bd36de7..7ace5168 100644 --- a/docs/qms/access_control-tr.md +++ b/docs/qms/access_control-tr.md @@ -230,8 +230,8 @@ webhook: timeout: 10 ``` -Webhook lifecycle olayları (`notify_started`, `notify_succeeded`, -`notify_failed`, `notify_reverted`, `notify_awaiting_approval` — +Webhook lifecycle olayları (`notify_start`, `notify_success`, +`notify_failure`, `notify_reverted`, `notify_awaiting_approval` — Phase 8 vocabulary) `FORGELM_OPERATOR` kimliğini webhook payload'a taşır böylece alıcı sistem (Slack, Teams, custom incident-management) bildirimi attribute edebilir. diff --git a/docs/qms/access_control.md b/docs/qms/access_control.md index a745e4d1..5236fea6 100644 --- a/docs/qms/access_control.md +++ b/docs/qms/access_control.md @@ -227,8 +227,8 @@ webhook: timeout: 10 ``` -The webhook lifecycle events (`notify_started`, `notify_succeeded`, -`notify_failed`, `notify_reverted`, `notify_awaiting_approval` — +The webhook lifecycle events (`notify_start`, `notify_success`, +`notify_failure`, `notify_reverted`, `notify_awaiting_approval` — Phase 8 vocabulary) carry the `FORGELM_OPERATOR` identity into the webhook payload so the receiving system (Slack, Teams, custom incident-management) can attribute the notification. diff --git a/docs/qms/encryption_at_rest-tr.md b/docs/qms/encryption_at_rest-tr.md index 145f2199..75907f18 100644 --- a/docs/qms/encryption_at_rest-tr.md +++ b/docs/qms/encryption_at_rest-tr.md @@ -206,9 +206,12 @@ ForgeLM kendisi şifreleme uygulamaz ama ŞUNLARI YAPAR: 1. **Neyi şifreleyeceğini tespit et.** `forgelm audit` eğitim verisindeki PII, secrets ve credential'ları flag'ler ki operatör eğitim öncesi şifreleyebilir veya maskeleyebilir. -2. **Neyin şifrelendiğini rapor et.** `data_governance_report.json` - operatörün config block'u başına `encryption_at_rest: true|false` - kaydeder. +2. **Neyin şifrelendiğini rapor et.** ForgeLM şu anda + `data_governance_report.json`'da substrate-seviyesi bir + `encryption_at_rest` bayrağı yüzeylemiyor — operatör bu gerçeği + out-of-band kayda alır (örn. cloud provider'ın KMS attestation'ı + ile birlikte QMS evidence bundle'ında) ta ki config-seviyesi bir + field eklenene kadar (Phase 28+ backlog'u). 3. **Şifreleme-sonrası bütünlüğü doğrula.** `forgelm verify-audit` chain-after-decryption'ın chain-as-emitted ile eşleştiğini teyit eder; substrate-seviyesi herhangi bir bozulma tespit edilebilirdir. diff --git a/docs/qms/encryption_at_rest.md b/docs/qms/encryption_at_rest.md index 4803ee87..a4697b4d 100644 --- a/docs/qms/encryption_at_rest.md +++ b/docs/qms/encryption_at_rest.md @@ -197,8 +197,12 @@ ForgeLM does not implement encryption itself but DOES: 1. **Detect what to encrypt.** `forgelm audit` flags PII, secrets, and credentials in training data so the operator can encrypt or mask before training. -2. **Report what is encrypted.** `data_governance_report.json` records - `encryption_at_rest: true|false` per the operator's config block. +2. **Report what is encrypted.** ForgeLM does not currently surface a + substrate-level `encryption_at_rest` flag in + `data_governance_report.json` — the operator records this fact + out-of-band (e.g. in the QMS evidence bundle alongside the cloud + provider's KMS attestation) until a config-level field is added + (Phase 28+ backlog). 3. **Verify post-encryption integrity.** `forgelm verify-audit` confirms the chain-after-decryption matches the chain-as-emitted; any substrate-level corruption is detectable. diff --git a/docs/qms/risk_treatment_plan-tr.md b/docs/qms/risk_treatment_plan-tr.md index 0a91ce8f..f214fb39 100644 --- a/docs/qms/risk_treatment_plan-tr.md +++ b/docs/qms/risk_treatment_plan-tr.md @@ -112,9 +112,9 @@ ekler. | Alan | Değer | |---|---| -| Açıklama | `evaluation.auto_revert: true` geçici bir eval regression'ında tetiklenir; sonuç `pipeline.reverted` downstream denetçiye safety failure gibi görünür | +| Açıklama | `evaluation.auto_revert: true` geçici bir eval regression'ında tetiklenir; sonuç `model.reverted` downstream denetçiye safety failure gibi görünür | | L × I (inherent) | Med × Low = LOW | -| Treatment | `pipeline.reverted` audit event regression delta + threshold taşır; `safety_trend.jsonl` cross-run context sağlar; operatör dashboard "gerçek" revert'leri threshold-tuning sorunlarından ayırır | +| Treatment | `model.reverted` audit event regression delta + threshold taşır; `safety_trend.jsonl` cross-run context sağlar; operatör dashboard "gerçek" revert'leri threshold-tuning sorunlarından ayırır | | Residual L × I | Low × Low = LOW | | Sahip | ML Lead | | Review cadence | Haftalık trend review | @@ -136,7 +136,7 @@ ekler. | Alan | Değer | |---|---| -| Açıklama | Operatörün Slack / Teams workspace'i ele geçirilir; saldırgan notify_started payload'larını okur ve model-deployment cadence'ini öğrenir | +| Açıklama | Operatörün Slack / Teams workspace'i ele geçirilir; saldırgan notify_start payload'larını okur ve model-deployment cadence'ini öğrenir | | L × I (inherent) | Low × Low = LOW | | Treatment | Webhook payload curation asla raw eğitim verisi veya unredacted PII taşımaz; `FORGELM_AUDIT_SECRET`-imzalı payload'lar splicing'i tespit eder; operatör olayda webhook secret'ı rotate eder | | Residual L × I | Low × Low = LOW | diff --git a/docs/qms/risk_treatment_plan.md b/docs/qms/risk_treatment_plan.md index a92b8407..2050ab19 100644 --- a/docs/qms/risk_treatment_plan.md +++ b/docs/qms/risk_treatment_plan.md @@ -111,9 +111,9 @@ section §4. | Field | Value | |---|---| -| Description | `evaluation.auto_revert: true` triggers on a transient eval regression; the resulting `pipeline.reverted` looks like a safety failure to a downstream auditor | +| Description | `evaluation.auto_revert: true` triggers on a transient eval regression; the resulting `model.reverted` looks like a safety failure to a downstream auditor | | L × I (inherent) | Med × Low = LOW | -| Treatment | `pipeline.reverted` audit event carries the regression delta + threshold; `safety_trend.jsonl` provides cross-run context; deployer dashboard distinguishes "real" reverts from threshold-tuning issues | +| Treatment | `model.reverted` audit event carries the regression delta + threshold; `safety_trend.jsonl` provides cross-run context; deployer dashboard distinguishes "real" reverts from threshold-tuning issues | | Residual L × I | Low × Low = LOW | | Owner | ML Lead | | Review cadence | Weekly trend review | @@ -135,7 +135,7 @@ section §4. | Field | Value | |---|---| -| Description | Operator's Slack / Teams workspace is compromised; attacker reads notify_started payloads and learns model-deployment cadence | +| Description | Operator's Slack / Teams workspace is compromised; attacker reads notify_start payloads and learns model-deployment cadence | | L × I (inherent) | Low × Low = LOW | | Treatment | Webhook payload curation never carries raw training data or unredacted PII; `FORGELM_AUDIT_SECRET`-signed payloads detect splicing; deployer rotates webhook secret on incident | | Residual L × I | Low × Low = LOW | diff --git a/docs/qms/sop_incident_response-tr.md b/docs/qms/sop_incident_response-tr.md index 8358ad4a..c574ff8f 100644 --- a/docs/qms/sop_incident_response-tr.md +++ b/docs/qms/sop_incident_response-tr.md @@ -158,7 +158,10 @@ olabilir. **Runbook:** -1. [ ] **`webhook.secret_env`'i hemen rotate et**. +1. [ ] **Webhook URL'sini ve destination-tarafı bearer token'ını + hemen rotate et** (URL'yi `webhook.url_env` ile resolve + ediyorsan, secret manager'da yeni değeri set'le; ForgeLM şu + an gövdeleri HMAC ile imzalamıyor). 2. [ ] **Audit chain'i yürü** saldırganın olayları recipient'a splice etmediğini teyit için: `audit_log.jsonl`'ı event sınıfına göre filtrele (`jq 'select(.event | startswith("notify_"))'`) ve diff --git a/docs/qms/sop_incident_response.md b/docs/qms/sop_incident_response.md index d00de374..5ae1af2e 100644 --- a/docs/qms/sop_incident_response.md +++ b/docs/qms/sop_incident_response.md @@ -160,7 +160,10 @@ that an attacker may have observed. **Runbook:** -1. [ ] **Rotate `webhook.secret_env`** immediately. +1. [ ] **Rotate the webhook URL and destination-side bearer token** + immediately (URL is resolved via `webhook.url_env` from your + secret manager; ForgeLM does not currently HMAC-sign webhook + bodies). 2. [ ] **Walk the audit chain** to confirm the attacker did not splice events into the recipient: filter `audit_log.jsonl` by event class diff --git a/docs/qms/statement_of_applicability-tr.md b/docs/qms/statement_of_applicability-tr.md index 486ceb5f..d40c8a05 100644 --- a/docs/qms/statement_of_applicability-tr.md +++ b/docs/qms/statement_of_applicability-tr.md @@ -26,7 +26,7 @@ operatör-tarafı eylem. ### 2.1 A.5 Organisational controls (37) -| Kontrol | Applicable? | Gerekçe / ForgeLM kanıtı | Implementation status | +| Kontrol | Applicable? | Uygulanabilirlik gerekçesi (uygun olduğunda ForgeLM kanıtı; aksi halde operatör-tarafı kapsam) | Implementation status | |---|---|---|---| | A.5.1 Bilgi güvenliği için politikalar | YES | EU AI Act Md. 17 QMS gerektirir; ForgeLM audit-event vocabulary neyin loglandığını dokümante eder | ForgeLM audit log'u referans veren kurum-genelinde ISMS politikası benimse | | A.5.2 Bilgi güvenliği rolleri ve sorumlulukları | YES | `roles_responsibilities.md` QMS şablonu AI Officer / ML Lead / Data Steward / DPO tanımlar | Rol tanımlarını benimse | @@ -54,9 +54,9 @@ operatör-tarafı eylem. | A.5.24 Bilgi güvenliği olay yönetimi planlaması ve hazırlığı | YES | `sop_incident_response.md`; audit chain durumu korur | IR ekibi kur | | A.5.25 Bilgi güvenliği olaylarının değerlendirilmesi ve karara bağlanması | YES | `data.erasure_failed`, `pipeline.failed`, `audit.classifier_load_failed` event'ler `error_class` + `error_message` ile | Triage runbook | | A.5.26 Bilgi güvenliği olaylarına yanıt | YES | Audit chain HMAC öncesi/sonrası korur | Runbook dokümante et | -| A.5.27 Bilgi güvenliği olaylarından öğrenme | YES | `pipeline.reverted` event'leri post-mortem kanıtı biriktirir | Haftalık post-mortem cadence | +| A.5.27 Bilgi güvenliği olaylarından öğrenme | YES | `model.reverted` event'leri post-mortem kanıtı biriktirir | Haftalık post-mortem cadence | | A.5.28 Kanıt toplama | YES | `audit_log.jsonl` forensic-grade; `forgelm verify-audit` doğrular | Write-once depolamaya gönder | -| A.5.29 Aksaklık sırasında bilgi güvenliği | YES | `auto_revert` baseline-flip; `pipeline.reverted` event | Base-model retention dokümante et | +| A.5.29 Aksaklık sırasında bilgi güvenliği | YES | `auto_revert` baseline-flip; `model.reverted` event | Base-model retention dokümante et | | A.5.30 İş sürekliliği için ICT hazırlığı | YES | DR planlama | Benimse | | A.5.31 Yasal, kanuni, düzenleyici ve sözleşmesel gerekliliklerin tanımlanması | YES | EU AI Act + GDPR mapping'ler; Annex IV bundle | Kural değişikliklerini izle | | A.5.32 Fikri mülkiyet hakları | YES | SBOM'da lisans çıkarımı; HF model-card metadata | Per-model lisans review | diff --git a/docs/qms/statement_of_applicability.md b/docs/qms/statement_of_applicability.md index a6c7e22b..975e501a 100644 --- a/docs/qms/statement_of_applicability.md +++ b/docs/qms/statement_of_applicability.md @@ -25,7 +25,7 @@ contribution → deployer-side action. ### 2.1 A.5 Organisational controls (37) -| Control | Applicable? | Justification / ForgeLM evidence | Implementation status | +| Control | Applicable? | Applicability rationale (ForgeLM evidence where applicable; otherwise operator-side scope) | Implementation status | |---|---|---|---| | A.5.1 Policies for information security | YES | EU AI Act Art. 17 mandates QMS; ForgeLM audit-event vocabulary documents what's logged | Adopt org-wide ISMS policy referencing ForgeLM audit log | | A.5.2 Information security roles and responsibilities | YES | `roles_responsibilities.md` QMS template defines AI Officer / ML Lead / Data Steward / DPO | Adopt role definitions | @@ -53,9 +53,9 @@ contribution → deployer-side action. | A.5.24 Information security incident management planning and preparation | YES | `sop_incident_response.md`; audit chain preserves state | Establish IR team | | A.5.25 Assessment and decision on information security events | YES | `data.erasure_failed`, `pipeline.failed`, `audit.classifier_load_failed` events with `error_class` + `error_message` | Triage runbook | | A.5.26 Response to information security incidents | YES | Audit chain HMAC preserves before/after | Document runbook | -| A.5.27 Learning from information security incidents | YES | `pipeline.reverted` events accumulate post-mortem evidence | Weekly post-mortem cadence | +| A.5.27 Learning from information security incidents | YES | `model.reverted` events accumulate post-mortem evidence | Weekly post-mortem cadence | | A.5.28 Collection of evidence | YES | `audit_log.jsonl` forensic-grade; `forgelm verify-audit` validates | Ship to write-once storage | -| A.5.29 Information security during disruption | YES | `auto_revert` baseline-flip; `pipeline.reverted` event | Document base-model retention | +| A.5.29 Information security during disruption | YES | `auto_revert` baseline-flip; `model.reverted` event | Document base-model retention | | A.5.30 ICT readiness for business continuity | YES | DR planning | Adopt | | A.5.31 Identification of legal, statutory, regulatory and contractual requirements | YES | EU AI Act + GDPR mappings; Annex IV bundle | Track rule changes | | A.5.32 Intellectual property rights | YES | License extraction in SBOM; HF model-card metadata | Per-model license review | diff --git a/docs/reference/audit_event_catalog-tr.md b/docs/reference/audit_event_catalog-tr.md index c650ecfe..e766bcec 100644 --- a/docs/reference/audit_event_catalog-tr.md +++ b/docs/reference/audit_event_catalog-tr.md @@ -44,6 +44,7 @@ Hash zinciri, satır diske düştükten (`flush` + `fsync`) sonra ilerler; kirli | Event | Ne zaman emit edilir | Payload | Madde | |--------------------------------|--------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------|-------| | `model.reverted` | Auto-revert kalite regresyonu sonrası önceki bir checkpoint'i geri yükledi. _(Faz 8 — webhook bağlantılı.)_ | `from_checkpoint`, `to_checkpoint`, `reason`, `metrics_delta` | 15 | +| `model.integrity_verified` | Eğitim sonrası nihai-model bütünlük manifesto'su (`model_integrity.json`) yazıldı ve başarıyla yeniden hash'lendi. | `artifacts` (yeniden-hash'lenen dosya sayısı) | 15 | | `audit.classifier_load_failed` | Güvenlik sınıflandırıcısı (örn. Llama Guard) yüklenemedi. Koşu yine `passed=False` kaydeder. | `classifier`, `reason` | 15 | ### Madde 11 + Ek IV — Uyumluluk artefaktları diff --git a/docs/reference/audit_event_catalog.md b/docs/reference/audit_event_catalog.md index 25e98298..ac82edf3 100644 --- a/docs/reference/audit_event_catalog.md +++ b/docs/reference/audit_event_catalog.md @@ -44,6 +44,7 @@ The hash chain advances after the line lands on disk (`flush` + `fsync`), so an | Event | When emitted | Payload | Article | |--------------------------------|-----------------------------------------------------------------------------------------------------------|---------------------------------------------------------------|---------| | `model.reverted` | Auto-revert restored a previous checkpoint after a quality regression. _(Faz 8 — webhook-coupled.)_ | `from_checkpoint`, `to_checkpoint`, `reason`, `metrics_delta` | 15 | +| `model.integrity_verified` | Final-model integrity manifest (`model_integrity.json`) was written and re-hashed successfully after training. | `artifacts` (count of files re-hashed) | 15 | | `audit.classifier_load_failed` | Safety classifier (e.g., Llama Guard) failed to load. The run still records `passed=False`. | `classifier`, `reason` | 15 | ### Article 11 + Annex IV — Compliance artefacts diff --git a/docs/reference/compliance_summary.md b/docs/reference/compliance_summary.md index 4e2e9aa9..13608cc6 100644 --- a/docs/reference/compliance_summary.md +++ b/docs/reference/compliance_summary.md @@ -61,7 +61,7 @@ What the regulator asks vs. how ForgeLM answers: - Implementation: `forgelm.trainer` post-training evaluation chain; `auto_revert` flag triggers fall-back to baseline on regression. - Evidence: `safety_results.json` (per-prompt classification); - `pipeline.reverted` audit event with regression delta. + `model.reverted` audit event with regression delta. - Configuration: `evaluation.safety.enabled`, `evaluation.auto_revert`, `evaluation.safety.scoring`, `evaluation.safety.min_safety_score`. @@ -187,7 +187,7 @@ name; this survives refactors that the prior `#L33` form did not. `forgelm.cli.subcommands._reject`, `forgelm.cli.subcommands._approvals`. - **Webhook lifecycle**: `forgelm.webhook` (search for - `notify_started`, `notify_succeeded`, `notify_failed`, + `notify_start`, `notify_success`, `notify_failure`, `notify_reverted`, `notify_awaiting_approval`). - **HTTP discipline**: `forgelm._http` (search for `safe_post`, `safe_get`). diff --git a/docs/reference/iso27001_control_mapping-tr.md b/docs/reference/iso27001_control_mapping-tr.md index a126ae26..c0bb3d45 100644 --- a/docs/reference/iso27001_control_mapping-tr.md +++ b/docs/reference/iso27001_control_mapping-tr.md @@ -47,9 +47,9 @@ | A.5.24 Bilgi güvenliği olay yönetimi planlaması ve hazırlığı | FL-helps | `sop_incident_response.md`; audit chain durumu korur | | A.5.25 Bilgi güvenliği olaylarının değerlendirilmesi ve karara bağlanması | FL-helps | `data.erasure_failed`, `pipeline.failed`, `audit.classifier_load_failed` | | A.5.26 Bilgi güvenliği olaylarına yanıt | FL-helps | Audit chain HMAC öncesi/sonrası durumu korur | -| A.5.27 Bilgi güvenliği olaylarından öğrenme | FL-helps | `pipeline.reverted` olayları post-mortem kanıtı biriktirir | +| A.5.27 Bilgi güvenliği olaylarından öğrenme | FL-helps | `model.reverted` olayları post-mortem kanıtı biriktirir | | A.5.28 Kanıt toplama | FL | `audit_log.jsonl` forensic-grade; `forgelm verify-audit` doğrular | -| A.5.29 Aksaklık sırasında bilgi güvenliği | FL-helps | `auto_revert` baseline-flip; `pipeline.reverted` olayı | +| A.5.29 Aksaklık sırasında bilgi güvenliği | FL-helps | `auto_revert` baseline-flip; `model.reverted` olayı | | A.5.30 İş sürekliliği için ICT hazırlığı | OOS | — | | A.5.31 Yasal, kanuni, düzenleyici ve sözleşmesel gerekliliklerin tanımlanması | FL-helps | EU AI Act + GDPR mappings; Annex IV bundle | | A.5.32 Fikri mülkiyet hakları | FL-helps | SBOM'da lisans çıkarımı; HF model-card metadata | diff --git a/docs/reference/iso27001_control_mapping.md b/docs/reference/iso27001_control_mapping.md index 7e43b110..a02171ae 100644 --- a/docs/reference/iso27001_control_mapping.md +++ b/docs/reference/iso27001_control_mapping.md @@ -48,9 +48,9 @@ | A.5.24 Information security incident management planning and preparation | FL-helps | `sop_incident_response.md`; audit chain preserves state | | A.5.25 Assessment and decision on information security events | FL-helps | `data.erasure_failed`, `pipeline.failed`, `audit.classifier_load_failed` | | A.5.26 Response to information security incidents | FL-helps | Audit chain HMAC preserves before/after | -| A.5.27 Learning from information security incidents | FL-helps | `pipeline.reverted` events accumulate post-mortem evidence | +| A.5.27 Learning from information security incidents | FL-helps | `model.reverted` events accumulate post-mortem evidence | | A.5.28 Collection of evidence | FL | `audit_log.jsonl` forensic-grade; `forgelm verify-audit` validates | -| A.5.29 Information security during disruption | FL-helps | `auto_revert` baseline-flip; `pipeline.reverted` event | +| A.5.29 Information security during disruption | FL-helps | `auto_revert` baseline-flip; `model.reverted` event | | A.5.30 ICT readiness for business continuity | OOS | — | | A.5.31 Identification of legal, statutory, regulatory and contractual requirements | FL-helps | EU AI Act + GDPR mappings; Annex IV bundle | | A.5.32 Intellectual property rights | FL-helps | License extraction in SBOM; HF model-card metadata | diff --git a/docs/reference/purge_subcommand-tr.md b/docs/reference/purge_subcommand-tr.md index 226110f1..41acd337 100644 --- a/docs/reference/purge_subcommand-tr.md +++ b/docs/reference/purge_subcommand-tr.md @@ -41,7 +41,7 @@ JSONL eğitim corpus'undan `id` (veya `row_id`) field'ı ile tanımlanan tek bir **Atomik yazma sözleşmesi.** Corpus, kardeş bir temp dosya + `os.replace` ile yeniden yazılır; kesilen bir purge ya tam silme-öncesi dosyayı ya da tam silme-sonrası dosyayı bırakır — asla kısmi state'i değil. -**Satır-numarası fallback'i reddedilir** (design §4.2). Id'siz corpus'lara sahip operatörler önce `forgelm audit --add-row-ids` çalıştırarak id'leri pre-populate etmelidir. +**Satır-numarası fallback'i reddedilir** (design §4.2). ForgeLM şu an bir id-doldurma yardımcısı sunmuyor (`forgelm audit --add-row-ids` Phase 28 backlog'unda); id'siz corpus'lara sahip operatörler `forgelm purge --row-id` çalıştırmadan önce id'leri operatör-tarafı bir script ile pre-populate etmelidir. ### Koşum-kapsamlı artefact silme (`--run-id` + `--kind`) diff --git a/docs/reference/purge_subcommand.md b/docs/reference/purge_subcommand.md index 3502a3fc..2f424959 100644 --- a/docs/reference/purge_subcommand.md +++ b/docs/reference/purge_subcommand.md @@ -41,7 +41,7 @@ Erases a single row identified by its `id` (or `row_id`) field from a JSONL trai **Atomic write contract.** The corpus is rewritten via a sibling temp file + `os.replace`, so an interrupted purge leaves either the full pre-erasure file or the full post-erasure file — never a partial state. -**Line-number fallback is rejected** (design §4.2). Operators with id-less corpora must pre-populate ids via `forgelm audit --add-row-ids`. +**Line-number fallback is rejected** (design §4.2). ForgeLM does not currently provide an id-population helper (`forgelm audit --add-row-ids` is on the Phase 28 backlog); operators with id-less corpora must pre-populate ids with an operator-side script before invoking `forgelm purge --row-id`. ### Run-scoped artefact erasure (`--run-id` + `--kind`) diff --git a/docs/reference/soc2_trust_criteria_mapping-tr.md b/docs/reference/soc2_trust_criteria_mapping-tr.md index c00e1947..5b7fba66 100644 --- a/docs/reference/soc2_trust_criteria_mapping-tr.md +++ b/docs/reference/soc2_trust_criteria_mapping-tr.md @@ -53,7 +53,7 @@ kategoriler engagement-bazında scoplanır. | CC7.1 | Zafiyetleri tespit eder | `pip-audit` nightly; CVE feed | | CC7.2 | Sistem bileşenlerini izler | `forgelm verify-audit`; `forgelm verify-gguf`; `safety_trend.jsonl` | | CC7.3 | Güvenlik olaylarını değerlendirir | `data.erasure_failed`, `pipeline.failed` olayları `error_class` + `error_message` ile | -| CC7.4 | Güvenlik olaylarına yanıt verir | `auto_revert`; `pipeline.reverted` olayı | +| CC7.4 | Güvenlik olaylarına yanıt verir | `auto_revert`; `model.reverted` olayı | | CC7.5 | Düzeltici eylemleri tanımlar, geliştirir | `human_approval.rejected`; `sop_change_management.md` | | CC8.1 | Değişimleri yetkilendirir | `forgelm approve` Madde 14 gate; staging dir | | CC9.1 | Riskleri tanımlar, yönetir | `risk_assessment` config + safety eval; `risk_treatment_plan.md` | diff --git a/docs/reference/soc2_trust_criteria_mapping.md b/docs/reference/soc2_trust_criteria_mapping.md index 0c8c32a2..dd8d6071 100644 --- a/docs/reference/soc2_trust_criteria_mapping.md +++ b/docs/reference/soc2_trust_criteria_mapping.md @@ -53,7 +53,7 @@ categories are scoped per-engagement. | CC7.1 | Detects vulnerabilities | `pip-audit` nightly; CVE feed | | CC7.2 | Monitors system components | `forgelm verify-audit`; `forgelm verify-gguf`; `safety_trend.jsonl` | | CC7.3 | Evaluates security events | `data.erasure_failed`, `pipeline.failed` events with `error_class` + `error_message` | -| CC7.4 | Responds to security events | `auto_revert`; `pipeline.reverted` event | +| CC7.4 | Responds to security events | `auto_revert`; `model.reverted` event | | CC7.5 | Identifies, develops corrective actions | `human_approval.rejected`; `sop_change_management.md` | | CC8.1 | Authorises changes | `forgelm approve` Article 14 gate; staging dir | | CC9.1 | Identifies, manages risks | `risk_assessment` config + safety eval; `risk_treatment_plan.md` | diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 4ff27b35..b973f221 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -5,7 +5,7 @@ ## Layout -Current structure (post Wave 5 / Phase 12.6 closure cycle: **~68 test +Current structure (post Wave 5 / Phase 12.6 closure cycle: **~70 test modules**, one per feature area; the collected-test count grows over time — run `pytest --collect-only -q` for current). The tree below is a **representative subset** — see `git ls-files tests/` for the full diff --git a/docs/usermanuals/en/compliance/audit-log.md b/docs/usermanuals/en/compliance/audit-log.md index b4284fb8..446c735b 100644 --- a/docs/usermanuals/en/compliance/audit-log.md +++ b/docs/usermanuals/en/compliance/audit-log.md @@ -32,18 +32,22 @@ Every entry has: | Event | When emitted | |---|---| | `run_start` | At the start of every `forgelm` invocation. | -| `config_validated` | After `--dry-run` passes. | -| `data_audit_complete` | After `forgelm audit` runs (whether ad hoc or as part of a training run). | -| `training_epoch_complete` | After each training epoch. | -| `training_step_milestone` | At configurable step intervals (default 1000). | -| `benchmark_complete` | After the benchmark suite runs. | -| `safety_eval_complete` | After Llama Guard scoring. | -| `auto_revert` | When auto-revert triggers. | -| `human_approval_request` | When `compliance.human_approval` blocks. | -| `human_approval_granted` | When approval is signed. | -| `model_exported` | After `forgelm export` writes a deployable artefact. | -| `run_complete` | At successful exit. | -| `run_failed` | At non-zero exit. | +| `training.started` | Trainer enters fine-tuning. | +| `pipeline.completed` | End-to-end CLI run returned exit code 0. | +| `pipeline.failed` | Pipeline aborted with an error. | +| `model.reverted` | Auto-revert restored a previous checkpoint after a quality regression. | +| `human_approval.required` | `evaluation.require_human_approval=true` paused the run for an operator decision. | +| `human_approval.granted` | Operator approved a paused gate via `forgelm approve`. | +| `human_approval.rejected` | Operator rejected a paused gate via `forgelm reject`. | +| `audit.classifier_load_failed` | Safety classifier (e.g. Llama Guard) failed to load. | +| `compliance.governance_exported` | EU AI Act Article 10 governance report written. | +| `compliance.artifacts_exported` | Annex IV bundle (manifest + model card + audit zip) written. | +| `data.erasure_*` | Six-event family covering `forgelm purge` lifecycle (Article 17). | +| `data.access_request_query` | `forgelm reverse-pii` invocation (GDPR Article 15). | +| `cli.legacy_flag_invoked` | A deprecated CLI flag was used. | + +The full event catalog (with payload schema and emitting site) lives in +[`docs/reference/audit_event_catalog.md`](#/reference/audit-event-catalog). ## Append-only by design diff --git a/docs/usermanuals/en/compliance/human-oversight.md b/docs/usermanuals/en/compliance/human-oversight.md index 444d4fc7..bac92ad1 100644 --- a/docs/usermanuals/en/compliance/human-oversight.md +++ b/docs/usermanuals/en/compliance/human-oversight.md @@ -5,7 +5,7 @@ description: Block model promotion until a human reviews and signs off — Artic # Human Oversight -EU AI Act Article 14 requires high-risk AI systems to provide for human oversight. ForgeLM implements this as an optional config gate: when `compliance.human_approval: true`, model promotion blocks until a human signs an approval. +EU AI Act Article 14 requires high-risk AI systems to provide for human oversight. ForgeLM implements this as an optional config gate: when `evaluation.require_human_approval: true`, model promotion blocks until a human signs an approval. ## How the gate works diff --git a/docs/usermanuals/en/compliance/model-card.md b/docs/usermanuals/en/compliance/model-card.md index 592cb4dc..63808231 100644 --- a/docs/usermanuals/en/compliance/model-card.md +++ b/docs/usermanuals/en/compliance/model-card.md @@ -29,7 +29,7 @@ The output is a Markdown file at `checkpoints/run/README.md` ready to upload to ```markdown # Acme Customer Support v1.2.0 -Customer-support assistant fine-tuned from Qwen2.5-7B-Instruct using ForgeLM 0.5.2. +Customer-support assistant fine-tuned from Qwen2.5-7B-Instruct using ForgeLM 0.5.5. ## Model details @@ -112,7 +112,7 @@ If you use this model, please cite: title = {Acme Customer Support v1.2.0}, author = {Acme Corp}, year = {2026}, - note = {Fine-tuned with ForgeLM 0.5.2} + note = {Fine-tuned with ForgeLM 0.5.5} } ``` ``` diff --git a/docs/usermanuals/en/compliance/overview.md b/docs/usermanuals/en/compliance/overview.md index 690047c2..bc8f7f59 100644 --- a/docs/usermanuals/en/compliance/overview.md +++ b/docs/usermanuals/en/compliance/overview.md @@ -57,7 +57,7 @@ This bundle is the deliverable for compliance reviews. Every file is hashed in ` | **11** | Technical documentation | `annex_iv_metadata.json` is a populated Annex IV. | | **12** | Record-keeping | Append-only `audit_log.jsonl` covering training start, eval gates, revert decisions. | | **13** | Transparency | Auto-generated model card listing capabilities, limitations, training summary. | -| **14** | Human oversight | Optional `compliance.human_approval: true` blocks promotion until a human signs off. | +| **14** | Human oversight | Optional `evaluation.require_human_approval: true` blocks promotion until a human signs off. | | **15** | Accuracy & robustness | Benchmark gates + safety eval + cybersecurity (PII / secrets at ingest). | | **16-17** | Conformity & QMS | Declaration scaffold + QMS SOPs in `docs/qms/`. | diff --git a/docs/usermanuals/en/deployment/deploy-targets.md b/docs/usermanuals/en/deployment/deploy-targets.md index 788a0282..f45deb5c 100644 --- a/docs/usermanuals/en/deployment/deploy-targets.md +++ b/docs/usermanuals/en/deployment/deploy-targets.md @@ -47,7 +47,7 @@ PARAMETER temperature 0.7 PARAMETER top_p 0.9 PARAMETER stop "<|im_end|>" -# Generated by ForgeLM 0.5.2 from checkpoint customer-support v1.2.0 +# Generated by ForgeLM 0.5.5 from checkpoint customer-support v1.2.0 ``` ForgeLM picks the right `PARAMETER` lines based on the model's tokeniser (chat templates, stop tokens) — no manual configuration needed. diff --git a/docs/usermanuals/en/operations/webhooks.md b/docs/usermanuals/en/operations/webhooks.md index 1d273172..a3ab3de9 100644 --- a/docs/usermanuals/en/operations/webhooks.md +++ b/docs/usermanuals/en/operations/webhooks.md @@ -10,13 +10,18 @@ ForgeLM fires structured webhooks on training milestones. Wire them into Slack, ## Quick example ```yaml -output: - webhook: - url: "${SLACK_WEBHOOK}" - events: ["run_start", "auto_revert", "run_complete", "run_failed"] - template: "slack" # or teams, generic +webhook: + url_env: "SLACK_WEBHOOK" # reads URL from $SLACK_WEBHOOK at runtime + notify_on_start: true # default true + notify_on_success: true # default true + notify_on_failure: true # default true (covers training.failure + training.reverted) ``` +The notifier emits a generic JSON payload — Slack and Teams ingest it directly via +their incoming-webhook endpoints. Per-event subscription is not currently +configurable; toggle the three `notify_on_*` flags to coarse-control which +lifecycle events fire. + ForgeLM picks up `${SLACK_WEBHOOK}` from environment variables. Common pattern: ```shell @@ -24,41 +29,39 @@ $ export SLACK_WEBHOOK="https://hooks.slack.com/services/T.../B.../..." $ forgelm --config configs/run.yaml ``` -## Events you can subscribe to +## Wire-format events -| Event | When fired | -|---|---| -| `run_start` | Training kicks off. | -| `data_audit_complete` | After `forgelm audit`. | -| `training_epoch_complete` | After each epoch. (verbose; usually skipped) | -| `benchmark_complete` | After eval suite. | -| `safety_eval_complete` | After Llama Guard scoring. | -| `auto_revert` | When auto-revert triggers. | -| `human_approval_request` | When `compliance.human_approval` blocks. | -| `human_approval_granted` | When approval is signed. | -| `model_exported` | After `forgelm export`. | -| `run_complete` | Successful exit. | -| `run_failed` | Non-zero exit. | +ForgeLM emits exactly **five** webhook events. The table below is the +canonical surface mirrored in +[`docs/reference/audit_event_catalog.md`](#/reference/audit-event-catalog): -Subscribe selectively — too-frequent webhooks become spam. +| Event | When fired | Gated by | +|---|---|---| +| `training.start` | `train()` enters, before model load. | `webhook.notify_on_start` | +| `training.success` | All gates pass; no human-approval requirement. | `webhook.notify_on_success` | +| `training.failure` | Training raised (OOM, dataset error, unhandled exception). | `webhook.notify_on_failure` | +| `training.reverted` | A post-training gate (eval / safety / judge / benchmark) rejected the run and `_revert_model` rolled adapters back. | `webhook.notify_on_failure` | +| `approval.required` | Run succeeded, `evaluation.require_human_approval=true` is set, model staged for review (EU AI Act Article 14). | `webhook.notify_on_success` | ## Payload shape -Generic format (the `slack` and `teams` templates wrap this in their respective formats): +Single generic JSON shape — Slack / Teams / Discord all accept it +directly via incoming-webhook endpoints; ForgeLM does **not** wrap it in +provider-specific templates: ```json { - "event": "auto_revert", - "ts": "2026-04-29T14:33:04Z", - "run_id": "abc123", - "config_path": "configs/customer-support.yaml", - "trigger": "safety_regression", - "regressed_categories": ["S5"], - "details": {...}, - "artifacts_url": "https://compliance-store.example/abc123/" + "event": "training.reverted", + "run_name": "customer-support-v1.2.0", + "status": "reverted", + "reason": "safety regression: S5 hate-speech +0.18 over baseline" } ``` +Payload keys vary by event; the full per-event field list is in +[`docs/reference/audit_event_catalog.md`](#/reference/audit-event-catalog) +under the *Webhook lifecycle events* table. + ## Slack template ```yaml diff --git a/docs/usermanuals/en/reference/exit-codes.md b/docs/usermanuals/en/reference/exit-codes.md index 4c6d724d..b06536f1 100644 --- a/docs/usermanuals/en/reference/exit-codes.md +++ b/docs/usermanuals/en/reference/exit-codes.md @@ -15,7 +15,7 @@ ForgeLM's exit codes are a public contract. CI/CD pipelines, schedulers, and das | **1** | `EXIT_CONFIG_ERROR` | YAML invalid, file missing, env var unset, or argument malformed. | Fail fast | | **2** | `EXIT_TRAINING_ERROR` | Training-time runtime error (any unhandled exception that isn't a config or eval-gate failure: data load, OOM, NaN loss, audit `--strict` failure, I/O). | Investigate; surface logs | | **3** | `EXIT_EVAL_FAILURE` | Benchmark or safety gate failed; auto-reverted if configured. | Investigate; do NOT promote | -| **4** | `EXIT_AWAITING_APPROVAL` | `compliance.human_approval: true` blocking. | Hold pipeline; trigger reviewer | +| **4** | `EXIT_AWAITING_APPROVAL` | `evaluation.require_human_approval: true` blocking. | Hold pipeline; trigger reviewer | These five integers are the entire public contract — see [`forgelm/cli/_exit_codes.py`](https://github.com/cemililik/ForgeLM/blob/main/forgelm/cli/_exit_codes.py) for the canonical definition. Any other non-zero value (including signal-derived 128+N codes) is clamped to `EXIT_TRAINING_ERROR` (2) before the process exits. @@ -84,7 +84,7 @@ stage('Train') { | Final loss is NaN / OOM / I/O failure mid-training | 2 | | DPO run, Llama Guard S5 regressed beyond tolerance | 3 | | Benchmark hellaswag dropped below floor | 3 | -| `compliance.human_approval: true` and no approval signed | 4 | +| `evaluation.require_human_approval: true` and no approval signed | 4 | | User Ctrl+C (signal-derived 128+N) | 2 (clamped) | ## Programmatic determination diff --git a/docs/usermanuals/tr/compliance/audit-log.md b/docs/usermanuals/tr/compliance/audit-log.md index 4f932c8b..cab228e8 100644 --- a/docs/usermanuals/tr/compliance/audit-log.md +++ b/docs/usermanuals/tr/compliance/audit-log.md @@ -31,19 +31,23 @@ Her kayıtta: | Olay | Ne zaman | |---|---| -| `run_start` | Her `forgelm` çağrısının başında. | -| `config_validated` | `--dry-run` geçtikten sonra. | -| `data_audit_complete` | `forgelm audit` koşturulduktan sonra. | -| `training_epoch_complete` | Her eğitim epoch'undan sonra. | -| `training_step_milestone` | Konfigüre adım aralıklarında (varsayılan 1000). | -| `benchmark_complete` | Benchmark suite koşturulduktan sonra. | -| `safety_eval_complete` | Llama Guard skorlamasından sonra. | -| `auto_revert` | Otomatik geri alma tetiklendiğinde. | -| `human_approval_request` | `compliance.human_approval` engellediğinde. | -| `human_approval_granted` | Onay imzalandığında. | -| `model_exported` | `forgelm export` deploy edilebilir artifact yazdığında. | -| `run_complete` | Başarılı çıkışta. | -| `run_failed` | Sıfır olmayan çıkışta. | +| `training.started` | Trainer fine-tuning'e başladığında. | +| `pipeline.completed` | Uçtan-uca CLI çalıştırması exit kod 0 ile bittiğinde. | +| `pipeline.failed` | Pipeline bir hata ile abort olduğunda. | +| `model.reverted` | Auto-revert kalite regresyonundan sonra önceki checkpoint'i geri yüklediğinde. | +| `human_approval.required` | `evaluation.require_human_approval=true` koşumu operatör kararı için duraklattığında. | +| `human_approval.granted` | Operatör `forgelm approve` ile duraklatılmış gate'i onayladığında. | +| `human_approval.rejected` | Operatör `forgelm reject` ile duraklatılmış gate'i reddettiğinde. | +| `audit.classifier_load_failed` | Safety classifier (örn. Llama Guard) yüklenemediğinde. | +| `compliance.governance_exported` | EU AI Act Madde 10 yönetişim raporu yazıldığında. | +| `compliance.artifacts_exported` | Annex IV bundle'ı (manifest + model card + audit zip) yazıldığında. | +| `data.erasure_*` | `forgelm purge` yaşam döngüsünü kapsayan altı-event ailesi (Madde 17). | +| `data.access_request_query` | `forgelm reverse-pii` çağrısı (GDPR Madde 15). | +| `cli.legacy_flag_invoked` | Deprecated bir CLI flag'i kullanıldığında. | + +Tam event kataloğu (payload şeması ve emit yeri ile) +[`docs/reference/audit_event_catalog.md`](#/reference/audit-event-catalog) +altındadır. ## Tasarım gereği append-only diff --git a/docs/usermanuals/tr/compliance/human-oversight.md b/docs/usermanuals/tr/compliance/human-oversight.md index 2d0ed2b4..d7a38f38 100644 --- a/docs/usermanuals/tr/compliance/human-oversight.md +++ b/docs/usermanuals/tr/compliance/human-oversight.md @@ -5,7 +5,7 @@ description: Bir insan inceleyip imzalayana kadar model terfisini engelleyin — # İnsan Gözetimi -EU AI Act Madde 14, yüksek-riskli AI sistemlerinin insan gözetimi imkânı sağlamasını gerektirir. ForgeLM bunu opsiyonel bir config kapısı olarak uygular: `compliance.human_approval: true` olduğunda model terfisi bir insan onay imzalayana kadar engellenir. +EU AI Act Madde 14, yüksek-riskli AI sistemlerinin insan gözetimi imkânı sağlamasını gerektirir. ForgeLM bunu opsiyonel bir config kapısı olarak uygular: `evaluation.require_human_approval: true` olduğunda model terfisi bir insan onay imzalayana kadar engellenir. ## Kapı nasıl çalışır diff --git a/docs/usermanuals/tr/compliance/model-card.md b/docs/usermanuals/tr/compliance/model-card.md index ebc1fd5c..4acf9b05 100644 --- a/docs/usermanuals/tr/compliance/model-card.md +++ b/docs/usermanuals/tr/compliance/model-card.md @@ -29,7 +29,7 @@ EU AI Act Madde 13 şeffaflığı zorunlu kılar: deploy edilen AI sistemleri ne ```markdown # Acme Customer Support v1.2.0 -ForgeLM 0.5.2 kullanılarak Qwen2.5-7B-Instruct'tan fine-tune edilmiş +ForgeLM 0.5.5 kullanılarak Qwen2.5-7B-Instruct'tan fine-tune edilmiş müşteri-destek asistanı. ## Model detayları @@ -113,7 +113,7 @@ Bu modeli kullanırsanız lütfen şöyle atıf yapın: title = {Acme Customer Support v1.2.0}, author = {Acme Corp}, year = {2026}, - note = {ForgeLM 0.5.2 ile fine-tune edildi} + note = {ForgeLM 0.5.5 ile fine-tune edildi} } ``` ``` diff --git a/docs/usermanuals/tr/compliance/overview.md b/docs/usermanuals/tr/compliance/overview.md index c99d98cc..a6d2c7fa 100644 --- a/docs/usermanuals/tr/compliance/overview.md +++ b/docs/usermanuals/tr/compliance/overview.md @@ -57,7 +57,7 @@ Bu paket compliance incelemeleri için teslim edilebilir. İçindeki her dosya ` | **11** | Teknik dokümantasyon | `annex_iv_metadata.json` dolu Annex IV. | | **12** | Kayıt tutma | Eğitim başlangıcı, eval kapıları, geri alma kararlarını kapsayan append-only `audit_log.jsonl`. | | **13** | Şeffaflık | Otomatik üretilen model card; yetenekleri, sınırları, eğitim özetini listeler. | -| **14** | İnsan gözetimi | Opsiyonel `compliance.human_approval: true` insan imzalayana kadar terfi engeller. | +| **14** | İnsan gözetimi | Opsiyonel `evaluation.require_human_approval: true` insan imzalayana kadar terfi engeller. | | **15** | Doğruluk ve sağlamlık | Benchmark kapıları + güvenlik eval + cybersec (ingest'te PII / sırlar). | | **16-17** | Uygunluk ve QMS | Beyan iskeleti + `docs/qms/`'deki QMS SOP'ları. | diff --git a/docs/usermanuals/tr/concepts/data-formats.md b/docs/usermanuals/tr/concepts/data-formats.md index cd9dfb43..09df389d 100644 --- a/docs/usermanuals/tr/concepts/data-formats.md +++ b/docs/usermanuals/tr/concepts/data-formats.md @@ -73,8 +73,16 @@ Audit (`forgelm audit`) `chosen == rejected` olan satırları flagler — prefer Tek yanıt + thumbs-up/down. Eşli tercihten daha kolay toplanır. ```json -{"prompt": "Aboneliği nasıl iptal ederim?", "response": "Sadece ödemeyi durdur.", "label": false} -{"prompt": "Aboneliği nasıl iptal ederim?", "response": "Ayarlar → Faturalandırma...", "label": true} +{ + "prompt": "Aboneliği nasıl iptal ederim?", + "response": "Sadece ödemeyi durdur.", + "label": false +} +{ + "prompt": "Aboneliği nasıl iptal ederim?", + "response": "Ayarlar → Faturalandırma → Aboneliği iptal et…", + "label": true +} ``` Anlamlar: diff --git a/docs/usermanuals/tr/data/audit.md b/docs/usermanuals/tr/data/audit.md index 0bcb177b..a14c4184 100644 --- a/docs/usermanuals/tr/data/audit.md +++ b/docs/usermanuals/tr/data/audit.md @@ -115,9 +115,17 @@ Train vs validation vs test satırları arasında kesin ve near-duplicate eşle "secrets_summary": {"total": 0}, "near_duplicate_pairs": 47, "cross_split_overlap": 0, - "quality_flags": {...}, + "quality_flags": { + "short_response": 24, + "repeated_lines": 0, + "abnormal_word_length": 12 + }, "language_distribution": {"tr": 0.992, "en": 0.008}, - "preference_specific": {"identical_chosen_rejected": 12, ...}, + "preference_specific": { + "identical_chosen_rejected": 12, + "empty_chosen": 0, + "swapped_likely": 0 + }, "verdict": "warnings" } ``` diff --git a/docs/usermanuals/tr/deployment/deploy-targets.md b/docs/usermanuals/tr/deployment/deploy-targets.md index f0a0584a..2904fbf9 100644 --- a/docs/usermanuals/tr/deployment/deploy-targets.md +++ b/docs/usermanuals/tr/deployment/deploy-targets.md @@ -47,7 +47,7 @@ PARAMETER temperature 0.7 PARAMETER top_p 0.9 PARAMETER stop "<|im_end|>" -# customer-support v1.2.0 checkpoint'inden ForgeLM 0.5.2 ile üretildi +# customer-support v1.2.0 checkpoint'inden ForgeLM 0.5.5 ile üretildi ``` ForgeLM modelin tokenizer'ına dayanarak doğru `PARAMETER` satırlarını seçer (chat template, stop token) — manuel konfigürasyon gerekmez. diff --git a/docs/usermanuals/tr/getting-started/first-run.md b/docs/usermanuals/tr/getting-started/first-run.md index a0f8d5b4..e7da3012 100644 --- a/docs/usermanuals/tr/getting-started/first-run.md +++ b/docs/usermanuals/tr/getting-started/first-run.md @@ -149,7 +149,7 @@ checkpoints/customer-support/ O `artifacts/` dizini compliance incelemeleri için teslim edilebilirdir. İçindeki her dosya `manifest.json`'da tamper-evidence için hashlenmiştir. -## Sıraki adımlar +## Sonraki adımlar - [Proje Yapısı](#/getting-started/project-layout) — ForgeLM şeyleri nereye ve neden koyar. - [Trainer Seçimi](#/concepts/choosing-trainer) — `customer-support`'tan kendi kullanım senaryonuza geçiş. diff --git a/docs/usermanuals/tr/getting-started/installation.md b/docs/usermanuals/tr/getting-started/installation.md index b1060b33..349c2a95 100644 --- a/docs/usermanuals/tr/getting-started/installation.md +++ b/docs/usermanuals/tr/getting-started/installation.md @@ -131,6 +131,6 @@ Yeni bir ortamda `forgelm --config ...` koşmadan *önce* `forgelm doctor` çal **`OSError: HuggingFace token not found`.** Bazı modeller (ör. Llama 3, Llama Guard) HuggingFace access token gerektirir. `huggingface-cli login` ile veya `HF_TOKEN` environment variable üzerinden ayarlayın. Tüm auth env var'ları için bkz. [CLI Referansı](#/reference/cli). ::: -## Sıraki adımlar +## Sonraki adımlar ForgeLM kurulu olduğuna göre uçtan uca eğitim turu için [İlk Koşunuz](#/getting-started/first-run)'a geçin — yaklaşık 5 dakikalık okuma ve 30 dakikalık GPU süresinde fine-tuned bir modeliniz olacak. diff --git a/docs/usermanuals/tr/operations/cicd.md b/docs/usermanuals/tr/operations/cicd.md index 33c7ac98..8b86111c 100644 --- a/docs/usermanuals/tr/operations/cicd.md +++ b/docs/usermanuals/tr/operations/cicd.md @@ -75,6 +75,7 @@ Artifact upload'taki `if: always()` önemli — başarısızlıkta bile audit lo stages: - audit - train + - deploy audit: stage: audit diff --git a/docs/usermanuals/tr/operations/webhooks.md b/docs/usermanuals/tr/operations/webhooks.md index 1cc2e88a..eb8b66d6 100644 --- a/docs/usermanuals/tr/operations/webhooks.md +++ b/docs/usermanuals/tr/operations/webhooks.md @@ -10,13 +10,18 @@ ForgeLM eğitim dönüm noktalarında yapılandırılmış webhook fırlatır. B ## Hızlı örnek ```yaml -output: - webhook: - url: "${SLACK_WEBHOOK}" - events: ["run_start", "auto_revert", "run_complete", "run_failed"] - template: "slack" # veya teams, generic +webhook: + url_env: "SLACK_WEBHOOK" # URL'yi $SLACK_WEBHOOK'tan runtime'da okur + notify_on_start: true # default true + notify_on_success: true # default true + notify_on_failure: true # default true (training.failure + training.reverted'i kapsar) ``` +Notifier generic JSON payload yayar — Slack ve Teams bunu incoming-webhook +uçlarından doğrudan kabul eder. Per-event abonelik şu an konfigüre +edilemiyor; üç `notify_on_*` flag'iyle hangi yaşam-döngüsü olaylarının +fırlatılacağı kaba ayarlanır. + ForgeLM `${SLACK_WEBHOOK}`'u environment variable'dan okur. Yaygın pattern: ```shell @@ -24,41 +29,39 @@ $ export SLACK_WEBHOOK="https://hooks.slack.com/services/T.../B.../..." $ forgelm --config configs/run.yaml ``` -## Abone olabileceğiniz olaylar +## Wire-format event'ler -| Olay | Ne zaman | -|---|---| -| `run_start` | Eğitim başlar. | -| `data_audit_complete` | `forgelm audit` sonrası. | -| `training_epoch_complete` | Her epoch sonrası. (gürültülü; genelde atlanır) | -| `benchmark_complete` | Eval suite sonrası. | -| `safety_eval_complete` | Llama Guard skorlama sonrası. | -| `auto_revert` | Otomatik geri alma tetiklendiğinde. | -| `human_approval_request` | `compliance.human_approval` engellediğinde. | -| `human_approval_granted` | Onay imzalandığında. | -| `model_exported` | `forgelm export` sonrası. | -| `run_complete` | Başarılı çıkış. | -| `run_failed` | Sıfır olmayan çıkış. | +ForgeLM tam **beş** webhook event'i yayar. Aşağıdaki tablo +[`docs/reference/audit_event_catalog.md`](#/reference/audit-event-catalog) +ile aynalanan kanonik yüzeydir: -Seçici abone olun — çok-sık webhook spam olur. +| Event | Ne zaman fırlar | Gate | +|---|---|---| +| `training.start` | `train()` başlar, model yüklenmeden önce. | `webhook.notify_on_start` | +| `training.success` | Tüm gate'ler geçer; insan-onay gereksinimi yok. | `webhook.notify_on_success` | +| `training.failure` | Eğitim raise ediyor (OOM, dataset hatası, yakalanmamış istisna). | `webhook.notify_on_failure` | +| `training.reverted` | Eğitim-sonrası bir gate (eval / safety / judge / benchmark) koşumu reddetti ve `_revert_model` adapter'ları geri aldı. | `webhook.notify_on_failure` | +| `approval.required` | Koşum başarılı, `evaluation.require_human_approval=true` set, model review için staged (EU AI Act Madde 14). | `webhook.notify_on_success` | ## Payload yapısı -Generic format (`slack` ve `teams` template'leri bunu kendi formatlarına sarar): +Tek generic JSON şekil — Slack / Teams / Discord hepsi bunu incoming-webhook +uçlarından doğrudan kabul eder; ForgeLM provider'a-özel template'lerle +sarmalama **yapmaz**: ```json { - "event": "auto_revert", - "ts": "2026-04-29T14:33:04Z", - "run_id": "abc123", - "config_path": "configs/customer-support.yaml", - "trigger": "safety_regression", - "regressed_categories": ["S5"], - "details": {...}, - "artifacts_url": "https://compliance-store.example/abc123/" + "event": "training.reverted", + "run_name": "customer-support-v1.2.0", + "status": "reverted", + "reason": "safety regression: S5 hate-speech +0.18 over baseline" } ``` +Payload anahtarları event'e göre değişir; tam per-event alan listesi +[`docs/reference/audit_event_catalog.md`](#/reference/audit-event-catalog) +*Webhook lifecycle events* tablosundadır. + ## Slack template ```yaml diff --git a/docs/usermanuals/tr/reference/exit-codes.md b/docs/usermanuals/tr/reference/exit-codes.md index 5c5b9668..12c92a30 100644 --- a/docs/usermanuals/tr/reference/exit-codes.md +++ b/docs/usermanuals/tr/reference/exit-codes.md @@ -15,7 +15,7 @@ ForgeLM'in exit kodları kamuya açık bir kontrattır. CI/CD hatları, schedule | **1** | `EXIT_CONFIG_ERROR` | YAML geçersiz, dosya yok, env var ayarsız veya argüman bozuk. | Hızlı başarısız | | **2** | `EXIT_TRAINING_ERROR` | Eğitim sırasında runtime hatası (config veya değerlendirme kapısı dışı her ele alınmamış istisna: data yükleme, OOM, NaN loss, audit `--strict` ihlali, I/O). | İncele; logları yüzeyle | | **3** | `EXIT_EVAL_FAILURE` | Benchmark veya güvenlik kapısı geçemedi; konfigüre edilmişse geri alındı. | İncele; terfi ETTİRME | -| **4** | `EXIT_AWAITING_APPROVAL` | `compliance.human_approval: true` engelliyor. | Hattı tut; reviewer'ı tetikle | +| **4** | `EXIT_AWAITING_APPROVAL` | `evaluation.require_human_approval: true` engelliyor. | Hattı tut; reviewer'ı tetikle | Bu beş tam sayı tüm kamuya açık kontratı oluşturur — kanonik tanım için bkz. [`forgelm/cli/_exit_codes.py`](https://github.com/cemililik/ForgeLM/blob/main/forgelm/cli/_exit_codes.py). Diğer her sıfır olmayan değer (sinyal kaynaklı 128+N kodları dahil) süreç çıkmadan önce `EXIT_TRAINING_ERROR` (2) değerine sıkıştırılır. @@ -84,7 +84,7 @@ stage('Train') { | Eğitim ortasında final loss NaN / OOM / I/O hatası | 2 | | DPO koşusu, Llama Guard S5 toleransı aştı | 3 | | Benchmark hellaswag floor altına düştü | 3 | -| `compliance.human_approval: true` ve onay imzalanmamış | 4 | +| `evaluation.require_human_approval: true` ve onay imzalanmamış | 4 | | Kullanıcı Ctrl+C (sinyal kaynaklı 128+N) | 2 (sıkıştırılır) | ## Programatik tespit diff --git a/site/css/style.css b/site/css/style.css index cf3e7da2..e2faa084 100644 --- a/site/css/style.css +++ b/site/css/style.css @@ -2057,7 +2057,7 @@ section { padding: 5rem 0; } height: 14px; margin-right: 0.5rem; vertical-align: -2px; - background: currentColor; + background: currentcolor; -webkit-mask: url("data:image/svg+xml;utf8,") center/contain no-repeat; mask: url("data:image/svg+xml;utf8,") center/contain no-repeat; opacity: 0.85; @@ -2325,7 +2325,7 @@ section { padding: 5rem 0; } } .wizard-resume-banner button { background: transparent; - border: 1px solid currentColor; + border: 1px solid currentcolor; border-radius: 999px; color: inherit; padding: 0.25rem 0.7rem; diff --git a/site/quickstart.html b/site/quickstart.html index 634e1549..fa74993e 100644 --- a/site/quickstart.html +++ b/site/quickstart.html @@ -360,9 +360,11 @@

Prefer a gu + CTA). When closed, the modal element gets the `hidden` attribute; + when open, every sibling of the modal under
gets + `aria-hidden="true"` so keyboard / AT users stay focused on the + wizard. State persists to localStorage so a refresh resumes the + same step. -->