Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/add-test/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Do **not** use for:
|---|---|
| Pydantic config field or validator | `test_config.py` |
| A new alignment trainer (DPO-style) | `test_alignment.py` |
| A full pipeline scenario | `test_integration_smoke.py` |
| A full pipeline scenario | `test_integration.py` |
| CLI argument parsing, exit codes | `test_cli.py` or `test_cli_subcommands.py` |
| Compliance artifact generation | `test_compliance.py` or `test_eu_ai_act.py` |
| Safety evaluation | `test_safety_advanced.py` |
Expand Down
2 changes: 1 addition & 1 deletion .claude/skills/add-trainer-feature/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Every trainer-level feature touches roughly the same list of files. Tick them as

- [ ] **`tests/test_<feature>.py`** — unit tests for the new module
- [ ] **`tests/test_config.py`** — validation tests for new config
- [ ] **`tests/test_integration_smoke.py`** — dry-run that exercises the feature
- [ ] **`tests/test_integration.py`** — dry-run that exercises the feature
- [ ] **`tests/conftest.py`** — new fixture if reusable across tests

### Config
Expand Down
36 changes: 10 additions & 26 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -194,32 +194,16 @@ jobs:
# tools/check_site_claims.py for what's checked and why.
run: python3 tools/check_site_claims.py --strict

- name: Bilingual doc H2 parity check
run: |
python -c "
import re, sys
pairs = [
('docs/reference/configuration.md', 'docs/reference/configuration-tr.md'),
('docs/reference/usage.md', 'docs/reference/usage-tr.md'),
('docs/reference/distributed_training.md','docs/reference/distributed_training-tr.md'),
('docs/reference/data_preparation.md', 'docs/reference/data_preparation-tr.md'),
('docs/reference/architecture.md', 'docs/reference/architecture-tr.md'),
('docs/guides/ingestion.md', 'docs/guides/ingestion-tr.md'),
('docs/guides/data_audit.md', 'docs/guides/data_audit-tr.md'),
]
failed = False
for en, tr in pairs:
with open(en) as f:
en_count = sum(1 for line in f if re.match(r'^## ', line))
with open(tr) as f:
tr_count = sum(1 for line in f if re.match(r'^## ', line))
status = 'OK' if en_count == tr_count else 'FAIL'
print(f'{status}: {en} ({en_count} H2) vs {tr} ({tr_count} H2)')
if en_count != tr_count:
failed = True
if failed:
sys.exit(1)
"
- name: Bilingual doc H2/H3/H4 parity check (strict)
# Phase 24: extended structural parity guard. Where the prior
# inline check counted H2 only and missed reordered / demoted
# sections, ``tools/check_bilingual_parity.py`` walks every
# registered EN/TR pair and compares the full H2 + H3 + H4
# spine. Translated text differs by definition; structural
# depth + ordering must not. Add a new pair: register it in
# ``tools/check_bilingual_parity.py::_PAIRS``; CI picks it up
# automatically the next run.
run: python3 tools/check_bilingual_parity.py --strict

- name: License check
run: |
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ docs/analysis/code_reviews/*
!docs/analysis/code_reviews/master-review-opus-202604300906.md
!docs/analysis/code_reviews/split-design-data_audit-cli-202604300906.md
!docs/analysis/code_reviews/wave2b-final-merge-review-prompt.md
!docs/analysis/code_reviews/wave3-review-prompt.md
!docs/analysis/code_reviews/wave3-followup-review-prompt.md

# Generated by tools/build_usermanuals.py — produced from
# docs/usermanuals/<lang>/**/*.md during the CI deploy. Not committed;
Expand Down
124 changes: 124 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,130 @@ All notable changes to ForgeLM are documented here.

## [Unreleased]

### Wave 3 — Faz 24 + 28 + 38 (`closure/wave3-integration`)

Single integration branch covering three closure-plan phases:

**Faz 38 — `forgelm reverse-pii` (GDPR Article 15 right-of-access)**

- New CLI subcommand: `forgelm reverse-pii --query VALUE [--type
literal|email|phone|tr_id|us_ssn|iban|credit_card|custom]
[--salt-source per_dir|env_var] JSONL_GLOB...`. Walks JSONL
corpora, reports every line where the supplied identifier appears.
Two scan modes: *plaintext residual* (mask-leak detection) and
*hash-mask* (reuses `forgelm purge`'s per-output-dir salt to
re-derive the digest, so a purge → reverse-pii cycle for the same
subject yields matching digests). Snippets are centred on the
matched span and capped at 160 chars so the operator can always
eyeball the hit.
- New audit event `data.access_request_query` (catalogued bilingually).
The identifier is **salted-and-hashed before audit emission**,
reusing the same per-output-dir salt that purge uses for
`target_id` — Article 15 access requests must not themselves leak
the subject's data into the audit log, AND a wordlist attack
against the audit chain requires the operator's salt file. The
`salt_source` field is recorded in every event so a compliance
reviewer can correlate Article 17 + Article 15 events for the same
subject.
- Default `--type` is `literal` (not `custom`) — a stray
`--query alice@example.com` matches the literal e-mail substring,
not the regex shape `alice@exampleXcom`. Operators wanting raw
regex pass `--type custom` explicitly; on POSIX a 30s SIGALRM
budget guards against ReDoS hangs.
- Audit fail-closed: AuditLogger init failure on any non-`ConfigError`
exception class (per Wave-3-followup F-W3FU-03 absorption — was
narrowed to `(OSError, ValueError)` only, now bare `Exception`)
refuses the run with `EXIT_TRAINING_ERROR`, naming the audit-dir.
`ConfigError` (operator-identity unavailable) still skips with a
WARNING — the only "best-effort" branch.
- Audit-dir default: same as `--output-dir` (matching `forgelm purge`
so `verify-audit` correlates Article 17 + Article 15 events for the
same subject in one chain — Wave-3-followup F-W3FU-01 reverted the
intermediate `<output_dir>/audit/` move that broke cross-tool
correlation).
- Mid-scan UTF-8 / I/O failures and ReDoS timeouts emit a failure-
flavoured `data.access_request_query` event before exit — same
no-leak invariant as the success path.
- 28 regression tests (was 18) covering: literal-default no
false-positives, salted audit hash, purge ↔ reverse-pii digest
correlation, failure-path no-leak, multi-byte UTF-8 truncation,
malformed UTF-8 corpus, explicit-audit-dir fail-closed,
per_dir-with-env-var symmetric refusal, overlapping-glob dedupe,
directory-arg diagnostic, no-salt-side-effect on regex parse error.

**Faz 24 — Bilingual TR mirror sweep + parity CI guard**

- `tools/check_bilingual_parity.py` (new): replaces the inline
H2-only check in `ci.yml` with an extended H2 + H3 + H4 structural
diff. Detects missing sections, depth changes, and reorders.
AST-free; runs in the lint job. 16 regression tests; live-repo
smoke test pins the canonical pair set passes `--strict`.
- 4 doc pairs brought to parity with their EN originals:
- `docs/guides/ingestion-tr.md` — added "Markdown-aware splitter"
+ "DOCX table preservation" H3 sections (Phase 12 features).
- `docs/reference/architecture-tr.md` — added 4 missing module H3s
(`results.py`, `benchmark.py`, `judge.py`, `model_card.py`).
- `docs/reference/distributed_training-tr.md` — added 3 missing H3s
(Custom DeepSpeed Config, "When to choose FSDP over DeepSpeed",
LoRA + Distributed) plus reordering Multi-Node ↔ Docker.
- `docs/reference/configuration-tr.md` — added missing
`model.multimodal` H4 block; reordered `evaluation.benchmark`
before `evaluation.safety` to match EN.
- 4 user-manual H2 drift fixes:
- `tr/training/sft.md` "Diskte ne elde edersiniz" added.
- `tr/training/simpo.md` "Veri formatı" added.
- `tr/compliance/overview.md` "Annex IV neyi içerir" added.
- `tr/concepts/data-formats.md` "Verinizi doğrulama" added.
- `docs/guides/alignment.md:230` "v0.5.1 (Phase 14)" → phase-number
reference (no version anchor — pipeline chains slated for v0.6.0+).
- CI integration: `tools/check_bilingual_parity.py --strict` replaces
the inline H2 check in `ci.yml` validate job.

**Faz 28 — Curated cleanup**

- `forgelm/config.py` (F-compliance-110 — **breaking**): high-risk /
unacceptable risk classification now **raises `ConfigError`** when
`evaluation.safety.enabled: false`. Was a warning; EU AI Act
Article 9 risk-management evidence cannot be derived from a
disabled safety eval. Operators with sandboxed runs must lower
the risk_classification or enable safety.
- `forgelm/config.py` (F-compliance-106): `WebhookConfig.timeout`
default raised 5s → 10s. Slack/Teams gateway latency spikes
regularly cross 5s; webhook failure is best-effort but a timeout
silently degrades the audit chain.
- `forgelm/compliance.py` (F-compliance-111): `_maybe_inline_audit_report`
missing-file branch escalated `INFO → WARNING`. A missing
`data_audit_report.json` is a real Article 10 compliance gap
(governance bundle ships without its data-quality section); the
signal must be visible in operator log dashboards.
- `forgelm/compliance.py` (M-204): added `_sanitize_md_list` helper +
migrated the `foreseeable_misuse` bullet build to use it.
- `forgelm/deploy.py::_ollama_modelfile` (M-205): SYSTEM line now
escapes newlines (`\n`/`\r`) so multi-line operator-supplied system
prompts don't break the Modelfile parser.
- `forgelm/webhook.py` (C-54): dropped `_is_private_destination`
re-export from `__all__`. The Phase 7 split moved the helper to
`forgelm._http`; no downstream importer of the webhook-side
re-export was found at the time of removal (clean drop).
- `forgelm/trainer.py` (C-57): GRPO reward token list now carries
an explicit "GSM8K + MATH-tuned" docstring caveat. Operators
training other math domains should write a custom reward callable
via `training.grpo_reward_model` rather than expecting this
stripper to generalise.
- `tests/test_integration_smoke.py` → `tests/test_integration.py`
(F-test-011): rename — the file is an integration test, not a
smoke test.

**Validation:**

- `ruff format` + `ruff check` clean
- `pytest`: 1333 passed / 14 skipped (was 1298 → **+35 net**: +18
reverse-pii, +16 parity tool, +1 high-risk-raise regression).
- `forgelm --config config_template.yaml --dry-run` green
- `forgelm reverse-pii --help` + dispatch round-trip via main CLI
- `tools/check_bilingual_parity.py --strict`: 8 / 8 doc pairs at
parity.

### Wave 2b inline review absorption (round 2)

A second inline review pass surfaced 6 valid defects + 3 actionable
Expand Down
19 changes: 11 additions & 8 deletions docs/analysis/code_reviews/closure-plan-202604300906.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,11 +93,11 @@ Her faz bir veya birden çok PR'la kapanır. Sıralama dependency-driven; parale
| **21** | **GDPR right-to-erasure — Implementation** | Tek PR | 1 | Faz 20 | ✅ Wave 2b (PR #30) |
| **22** | **ISO 27001 / SOC 2 Type II — Analysis & gap assessment** | Tek PR | 1 | Faz 3, 6, 7, 8, 9, 21 (security + compliance baseline) | ⏳ PENDING |
| **23** | **ISO 27001 / SOC 2 Type II — Implementation** | Tek PR (büyük) | 1 | Faz 22 | ⏳ PENDING |
| **24** | Bilingual TR mirror sweep (configuration-tr, distributed-tr, ingestion-tr, EN/TR user-manual H2 drift) | Tek PR | 1 | Faz 16 (description= migration ile config docs auto-gen) | ⏳ PENDING |
| **24** | Bilingual TR mirror sweep (configuration-tr, distributed-tr, ingestion-tr, EN/TR user-manual H2 drift) | Tek PR | 1 | Faz 16 (description= migration ile config docs auto-gen) | ✅ Wave 3 (PR pending) |
| **25** | Site picker honesty + site-as-tested-surface CI guard | Tek PR | 1 | Faz 1 (site honesty fixes önce) | ✅ DONE |
| **26** | QMS bilingual EN+TR mirror + compliance docs cleanup | Tek PR | 1 | Faz 23 (ISO/SOC 2 dokümanları QMS ile lockstep) | ⏳ PENDING |
| **27** | Silent `except Exception:` sweep (Theme D residual) | Tek PR | 1 | — | ✅ DONE |
| **28** | Remaining Major + Minor cleanup | Tek PR | 1 | Hepsi öncesindekilere bağlı | ⏳ PENDING |
| **28** | Remaining Major + Minor cleanup | Tek PR | 1 | Hepsi öncesindekilere bağlı | ✅ Wave 3 (PR pending; tier 1+2 absorbed) |
| **29** | Nit sweep (40 madde mass-edit) | Tek PR | 1 | — | ✅ DONE |
| **30** | **Final documentation + site finalization (EN+TR only)** — tüm `docs/` + `docs/usermanuals/{en,tr}/` + `site/*` v0.5.5 nihai durumuna senkron | Tek PR | 1 | **Faz 1-29 + 34-38** (tüm fonksiyonel iş bitmiş olmalı) | ⏳ PENDING |
| **31** | **Cross-OS release-tag matrix workflow** | Tek PR | 1 | — (release öncesi her şeyden) | ✅ DONE |
Expand All @@ -106,7 +106,7 @@ Her faz bir veya birden çok PR'la kapanır. Sıralama dependency-driven; parale
| **35** | **Air-gap pre-cache subcommands** (`cache-models` + `cache-tasks`) (GH-002, GH-003) | Tek PR | 1 | Faz 15 + Faz 34 (doctor `--offline` paydaş) | ✅ Wave 2b (PR #30) |
| **36** | **Compliance verification toolbelt** (`safety-eval` standalone, `verify-annex-iv`, `verify-gguf`) (GH-004, GH-005, GH-009) | Tek PR | 1 | Faz 6 (verify-audit pattern), Faz 9 (compliance baseline) | ✅ Wave 2b (PR #30) |
| **37** | **`forgelm approvals` listing subcommand** — Faz 9 follow-up (GH-007) | Tek PR | 1 | Faz 9 (`approve` + `reject` + staging directory) | ✅ Wave 2a (PR #28) |
| **38** | **`forgelm reverse-pii` GDPR Article 15 subcommand** (GH-014) | Tek PR | 1 | Faz 21 (GDPR erasure baseline) | ⏳ PENDING |
| **38** | **`forgelm reverse-pii` GDPR Article 15 subcommand** (GH-014) | Tek PR | 1 | Faz 21 (GDPR erasure baseline) | ✅ Wave 3 (PR pending) |
| **33** | **v0.5.5 RELEASE** (CHANGELOG, version bump, tag, PyPI) | Tek PR | 1 | **Tüm önceki fazlar (1-32, 34-38)** | ⏳ POST-WAVE |

**Toplam:** 38 faz, ~52 PR (multi-PR seri'ler dahil).
Expand All @@ -115,7 +115,8 @@ Her faz bir veya birden çok PR'la kapanır. Sıralama dependency-driven; parale
- ✅ DONE — Foundation / Wave 1 (PR #19 → main, 2026-04-30; PR #21 → development, 2026-05-02); faz tamamlandı.
- ✅ Wave 2a (PR #28) — `closure/wave2a-integration` → development, 2026-05-04; faz tamamlandı.
- ✅ Wave 2b (PR #30) — `closure/wave2b-integration` → development, 2026-05-05 (merge commit `b05edb5`); faz tamamlandı.
- ⏳ PENDING — henüz başlanmadı; bağımlılıkları sağlanan ilk batch sıradaki Wave 3+'da açılır.
- ✅ Wave 3 (PR pending) — `closure/wave3-integration` → development, 2026-05-05; Faz 24 + Faz 28 (curated tier 1+2) + Faz 38 absorption tamamlandı; PR review aşamasında.
- ⏳ PENDING — henüz başlanmadı; bağımlılıkları sağlanan ilk batch sıradaki Wave 4+'da açılır.
- ⏳ POST-WAVE — release fazı (33), tüm fonksiyonel iş bittikten sonra son.

**Wave 1 kapanış özeti (post-merge):** 12 faz tamamlandı (Faz 9-15, 25, 27, 29, 31, 32). Foundation (Faz 1-8) Wave 0'da PR #19 ile merge edilmişti.
Expand All @@ -124,7 +125,9 @@ Her faz bir veya birden çok PR'la kapanır. Sıralama dependency-driven; parale

**Wave 2b kapanış özeti (post-merge, 2026-05-05):** 5 faz tamamlandı (Faz 16 Pydantic `description=` migration, Faz 19 Library API implementation, Faz 21 GDPR erasure implementation, Faz 35 air-gap pre-cache, Faz 36 compliance verification toolbelt). PR #30 → development (merge commit `b05edb5`). 6 round absorption + 4-agent final review + 4 followup absorption commits; suite 1160 → 1298 tests (+138 net).

**Kalan iş (Wave 3+):** 7 faz + 1 release. Pending: Faz 22 (ISO 27001/SOC 2 design), Faz 23 (ISO 27001/SOC 2 impl), Faz 24 (bilingual TR mirror sweep), Faz 26 (QMS bilingual mirror), Faz 28 (Major+Minor cleanup), Faz 30 (final docs + site finalization), Faz 38 (`forgelm reverse-pii` GDPR Art. 15). Release: Faz 33 (v0.5.5 PyPI publish).
**Wave 3 kapanış özeti (2026-05-05, PR review aşamasında):** 3 faz tamamlandı (Faz 24 bilingual TR mirror sweep + `tools/check_bilingual_parity.py` CI guard, Faz 28 curated tier 1+2 cleanup, Faz 38 `forgelm reverse-pii` GDPR Article 15 subcommand). Branch: `closure/wave3-integration`. Yeni: 1 CLI subcommand (`reverse-pii`), 1 audit event (`data.access_request_query`), 1 yeni CI guard tool. Behaviour change: high-risk + safety-disabled artık `ConfigError` raise eder (F-compliance-110). Suite 1298 → 1333 tests (+35 net).

**Kalan iş (Wave 3+):** 4 faz + 1 release. Pending: Faz 22 (ISO 27001/SOC 2 design), Faz 23 (ISO 27001/SOC 2 impl), Faz 26 (QMS bilingual mirror), Faz 30 (final docs + site finalization). Release: Faz 33 (v0.5.5 PyPI publish). (Faz 24 + Faz 28 + Faz 38 Wave 3 ile tamamlandı; bkz. yukarıdaki kapanış özeti.)

---

Expand Down Expand Up @@ -812,7 +815,7 @@ Yeni dosya: `docs/analysis/code_reviews/iso27001-soc2-alignment-202604YYMMHHMM.m

## 9. Polish + remaining cleanup fazları (Faz 24-29)

### Faz 24 — Bilingual TR mirror sweep (configuration-tr, distributed-tr, ingestion-tr, EN+TR user-manual H2 drift) (Tek PR)
### Faz 24 — Bilingual TR mirror sweep (configuration-tr, distributed-tr, ingestion-tr, EN+TR user-manual H2 drift) (Tek PR) ✅ TAMAMLANDI — Wave 3 (PR pending)

**Closes:** M-DOC-002, M-DOC-007, M-DOC-012, M-DOC-013, M-DOC-014, M-DOC-015, F-loc-005, F-loc-006, F-loc-007, F-loc-008

Expand Down Expand Up @@ -922,7 +925,7 @@ Yeni dosya: `docs/analysis/code_reviews/iso27001-soc2-alignment-202604YYMMHHMM.m

---

### Faz 28 — Remaining Major + Minor + Nit cleanup (Tek PR)
### Faz 28 — Remaining Major + Minor + Nit cleanup (Tek PR) ✅ TAMAMLANDI — Wave 3 (PR pending; tier 1+2 absorbed, large items deferred to v0.6.0)

**Closes:** Master review tablosundan Faz 1-27'ye girmeyen tüm Major + Minor + Nit (~70+ madde).

Expand Down Expand Up @@ -1156,7 +1159,7 @@ Yeni dosya: `docs/analysis/code_reviews/iso27001-soc2-alignment-202604YYMMHHMM.m

---

### Faz 38 — `forgelm reverse-pii` GDPR Article 15 subcommand (Tek PR) ⏳ PENDING
### Faz 38 — `forgelm reverse-pii` GDPR Article 15 subcommand (Tek PR) ✅ TAMAMLANDI — Wave 3 (PR pending)

**Closes:** GH-014 (Medium — GDPR Article 15 right of access; data subject "benim verim eğitim setinde mi?" sorgusuna yanıt veren tooling).

Expand Down
Loading