Skip to content

Wave 2a: Phase 17/18/20/34/37 — consolidated integration (supersedes #23-#27) #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cemililik merged 23 commits into from
May 4, 2026
Merged

Wave 2a: Phase 17/18/20/34/37 — consolidated integration (supersedes #23-#27) #28

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b7a1f06
docs(phase-18): library API analysis & design
cemililik May 2, 2026
774d029
docs(phase-20): GDPR right-to-erasure analysis & design
cemililik May 2, 2026
37c33e2
feat(phase-37): forgelm approvals listing subcommand
cemililik May 2, 2026
a724e36
feat(phase-17): forgelm audit --workers N + determinism contract
cemililik May 2, 2026
49d8fa3
feat(phase-34): forgelm doctor env-check subcommand
cemililik May 2, 2026
93fd9ad
docs(phase-18): address review-bot + reviewer findings
cemililik May 2, 2026
074e533
docs(phase-20): address review-bot + reviewer findings
cemililik May 2, 2026
6d09e0b
fix(phase-37): address review-bot + reviewer findings
cemililik May 2, 2026
e537151
fix(phase-17): address review-bot + reviewer findings
cemililik May 2, 2026
9511e1d
fix(phase-34): address review-bot + reviewer findings
cemililik May 2, 2026
25bb52c
Merge Phase 18: Library API design (was PR #23)
cemililik May 4, 2026
d6ebd1a
Merge Phase 20: GDPR right-to-erasure design (was PR #24)
cemililik May 4, 2026
49e79fd
Merge Phase 17: forgelm audit --workers N + determinism contract (was…
cemililik May 4, 2026
29ae8a6
Merge Phase 37: forgelm approvals listing + shared audit_log_reader (…
cemililik May 4, 2026
246d6f7
Merge Phase 34: forgelm doctor subcommand (was PR #27)
cemililik May 4, 2026
5a57286
style: ruff format pass on integration branch
cemililik May 4, 2026
4fdeb17
docs(wave-2a): add Round-2 multi-agent review prompt for PR #28
cemililik May 4, 2026
59f8d8c
fix(wave-2a-round2): absorb 4-agent review + 9 inline comments
cemililik May 4, 2026
2148a49
fix(wave-2a-round2): absorb 4-agent review + 9 inline comments
cemililik May 4, 2026
5a40883
fix(wave-2a-round3): SonarCloud + CodeRabbit follow-ups on PR #28
cemililik May 4, 2026
a2428ca
fix(wave-2a-round4): polish prose + SIGINT exit-code contract
cemililik May 4, 2026
843d1ab
fix(wave-2a-round5): pre-merge audit absorption (8 findings + 4 inlin…
cemililik May 4, 2026
59d5f2e
refactor(wave-2a-round5): drop reject complexity below SonarCloud S37…
cemililik May 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,27 @@ jobs:
- name: Ruff format check
run: ruff format --check .

# Wave 2a Round-2 F-XPR-02-03: project-wide HTTP discipline gate
# (per docs/standards/architecture.md "HTTP discipline"). Every
# outbound HTTP call must go through forgelm/_http.py (safe_post /
# safe_get). This grep guard fails CI if any new module reaches
# for requests / urllib / httpx directly.
- name: HTTP discipline guard
run: |
set +e
# Match ACTUAL CALLS only (the function name followed by an
# opening paren) so docstring prose mentioning the same names
# is not flagged.
violations=$(grep -rn -E '(requests\.(get|post|put|delete|patch|request|head)\(|urllib\.request\.urlopen\(|httpx\.[a-z]+\()' forgelm/ --include='*.py' | grep -v 'forgelm/_http.py')
if [ -n "$violations" ]; then
echo "::error::Found undisciplined HTTP call(s) outside forgelm/_http.py:"
echo "$violations"
echo ""
echo "Route through forgelm._http.safe_post / safe_get instead."
echo "See docs/standards/architecture.md 'HTTP discipline' section."
exit 1
fi

# --- Job 2: Test (matrix across Python versions) ---
test:
runs-on: ubuntu-latest
Expand Down
8 changes: 8 additions & 0 deletions .markdownlint.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"//": "ForgeLM markdownlint policy. Two rules disabled with rationale:",
"//1": "MD051 (link-fragments-valid): docs use SPA hash-router routes like `#/data/audit` for in-app navigation; markdownlint's anchor-resolution doesn't understand SPA conventions and would flag every cross-reference.",
"//2": "MD014 (commands-show-output): shell examples in operator docs deliberately show only the command (`$ forgelm audit ...`) without sample output — output snippets rot quickly across versions and would force a doc update on every CLI banner change.",
"default": true,
"MD051": false,
"MD014": false
}
569 changes: 569 additions & 0 deletions CHANGELOG.md
View file
Open in desktop

Large diffs are not rendered by default.

9 changes: 5 additions & 4 deletions docs/analysis/code_reviews/closure-plan-202604300906.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -697,7 +697,7 @@ Yeni dosya: `docs/analysis/code_reviews/gdpr-erasure-design-202604YYMMHHMM.md`.
- `--row-id`, `--corpus`, `--run-id`, `--kind`, `--check-policy` flag'leri
- Atomic operations
3. `forgelm/compliance.py` — `record_erasure_event(target_kind, target_id, justification)` helper.
4. Audit event vocabulary'sine ekle: `data.erasure_requested`, `data.erasure_completed`, `data.erasure_failed`.
4. Audit event vocabulary'sine ekle (Faz 20 design §5.1 ile hizalanmış, 6 olay): `data.erasure_requested`, `data.erasure_completed`, `data.erasure_failed`, `data.erasure_warning_memorisation` (row erasure × var-olan final_model), `data.erasure_warning_synthetic_data_present` (row erasure × var-olan synthetic_data*.jsonl), `data.erasure_warning_external_copies` (webhook block configured → downstream propagation reminder).
5. `docs/guides/gdpr_erasure.md` + `-tr.md` mirror — operator-facing how-to.
6. `docs/qms/sop_data_management.md` — retention + erasure procedure güncelleme (Faz 1'deki rewrite üstüne).
7. Tests `tests/test_gdpr_erasure.py` (yeni):
Expand Down Expand Up @@ -1916,14 +1916,14 @@ Plan §8 Faz 22 "ISO 27001 / SOC 2 Type II — Analysis & gap assessment" design
| GH-020 (ingest `--format/--language/--include/--exclude/--pii-locale`) | Medium | **Doc-only — kaldır + flag rename'leri** | **Faz 30 Task O** |
| GH-021 (chat slash `/load//top_p//max_tokens//safety`) | Low | **Doc-only — kaldır** | **Faz 30 Task O** |
| GH-022 (export quant `q6_k`/`fp16`) | Low | **Doc-only — `q6_k` kaldır, `fp16` → `f16`** | **Faz 30 Task O** |
| GH-023 (`ingestion.retention.raw_documents.ttl_days` config) | Medium | **Faz 21'in `RetentionConfig`'ine extend** — `ingestion` top-level key + `retention.raw_documents.ttl_days` field | Faz 21 (genişletme) |
| GH-023 (`ingestion.retention.raw_documents.ttl_days` config) | Medium | **Top-level `retention.*` altında absorbe** (Faz 20 design §10 Q1 kararı) — nested `ingestion.retention.*` shape'i REDDEDİLDİ; policy ingestion artefact'larının ötesini kapsadığı için top-level. Faz 21 `RetentionConfig`'ine `raw_documents_retention_days` field'ı eklenir + `docs/usermanuals/en/compliance/gdpr.md` YAML örneği yeniden yazılır | Faz 21 (genişletme) |
| GH-024 (`FORGELM_RESUME_TOKEN`) | Low | **Doc-only — kaldır** (kodda kullanılmıyor) | **Faz 30 Task O** |
| GH-025 (`FORGELM_CACHE_DIR`) | Low | **Doc-only — kaldır + `HF_HOME` canonical işaretle** | **Faz 30 Task O** |
| GH-026 (`ingestion.retention` config; duplicate of GH-023) | Medium | (GH-023 ile aynı — Faz 21 extend) | Faz 21 |
| GH-026 (`ingestion.retention` config; duplicate of GH-023) | Medium | (GH-023 ile aynı — top-level `retention.*` altında absorbe) | Faz 21 |
| GH-027 (alias of GH-024) | Low | (GH-024 ile aynı) | Faz 30 Task O |
| GH-028 (alias of GH-025) | Low | (GH-025 ile aynı) | Faz 30 Task O |

**Ek görev:** Faz 21 kapsamı **GH-023 absorption** ile genişletilir — `ForgeConfig`'a `ingestion: IngestionConfig` top-level key + `IngestionConfig.retention.raw_documents.ttl_days` Pydantic field eklenir; `_purge --kind raw_documents` flag'ı bu retention policy'yi enforce eder. Faz 21 acceptance'a yeni satır: "Config: `ingestion.retention.raw_documents.ttl_days: int = 90` field çalışır + Pydantic Literal sweep tarafından kapsanır + GDPR docs'taki YAML örneği geçerli."
**Ek görev:** Faz 21 kapsamı **GH-023 absorption** ile genişletilir — Faz 20 design §10 Q1 kararına göre `ForgeConfig`'a top-level `retention: Optional[RetentionConfig] = None` field eklenir (nested `ingestion.retention.*` shape REDDEDİLDİ; gerekçe: policy ingestion artefact'larının ötesini — audit log + staging + ephemeral artefact — kapsar). `RetentionConfig` field'ları: `audit_log_retention_days: int = 1825`, `staging_ttl_days: int = 7`, `ephemeral_artefact_retention_days: int = 90`, `raw_documents_retention_days: int = 90` (GH-023'ün koruduğu eski `ingestion.retention.raw_documents.ttl_days` semantiği). `_purge --kind {raw_documents,staging,artefacts}` flag'ı policy'yi enforce eder. **Aynı PR'da:** `docs/usermanuals/en/compliance/gdpr.md` lines 51-64 YAML örneği yeniden yazılır + TR mirror — eski `ingestion.retention.raw_documents.ttl_days` shape `retention.raw_documents_retention_days`'e değiştirilir; `EvaluationConfig.staging_ttl_days` (v0.5.5'te shipped) deprecation cadence'ı uygulanır (alias-forward to `retention.staging_ttl_days`, v0.7.0'da kaldır). Faz 21 acceptance'a yeni satırlar: "Config: top-level `retention.*` block çalışır + 4 field Pydantic-validate; `evaluation.staging_ttl_days` DeprecationWarning + alias-forward; GDPR docs YAML örneği yeni shape'i gösteriyor + TR mirror eşleşiyor."

Comment on lines +1919 to 1927

@coderabbitai coderabbitai Bot May 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Resolve the Phase‑21 contract drift on retention schema and purge kinds.

These lines add raw_documents_retention_days and --kind raw_documents, but the Phase 20 design doc in this PR still specifies a narrower purge-kind surface and retention schema. Please unify both docs to one canonical contract before implementation starts, otherwise Phase 21 acceptance/test scope is ambiguous.

As per coding guidelines, "Documentation must follow docs/standards/documentation.md and docs/standards/localization.md standards".

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@docs/analysis/code_reviews/closure-plan-202604300906.md` around lines 1919 -
1927, The PR introduces Phase‑21 retention fields
(`raw_documents_retention_days`) and purge kinds but conflicts with the Phase‑20
design; reconcile by adding a top-level RetentionConfig and wiring it through
ForgeConfig and CLI, i.e., add ForgeConfig.retention: Optional[RetentionConfig]
= None, implement class RetentionConfig with fields
audit_log_retention_days:int=1825, staging_ttl_days:int=7,
ephemeral_artefact_retention_days:int=90, raw_documents_retention_days:int=90,
update purge CLI to accept --kind {raw_documents,staging,artefacts}, deprecate
EvaluationConfig.staging_ttl_days by alias‑forwarding it to
retention.staging_ttl_days with a DeprecationWarning (keep alias until v0.7.0),
and update the YAML example in docs/usermanuals/en/compliance/gdpr.md (lines
~51–64) and its TR mirror to use retention.raw_documents_retention_days; ensure
related acceptance tests/assertions reflect the new top‑level schema and purge
kinds.

@cemililik cemililik May 4, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rejected — Phase 21 scope leak. raw_documents_retention_days and --kind raw_documents are spec'd in Phase 20 design (this PR, design-only) and scheduled for Phase 21 implementation (separate PR). Implementing them in the integration PR would leapfrog Phase 21 and break the design-only / implementation-only split this wave was structured around.

The "contract drift" the bot perceives is actually the spec describing what Phase 21 will implement; the closure-plan §15.5 v2.5 entry (this same line range) explicitly calls out the absorption decision and aligns with the Phase 20 design. No change needed in this PR.

@coderabbitai coderabbitai Bot May 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cemililik, understood and acknowledged. You're right — the closure-plan §15.5 v2.5 entry is spec language describing what Phase 21 will implement; it does not represent implementation happening in this PR. The design-only / implementation-only split is intentional per this Wave's structure, and flagging the spec description as "contract drift" was incorrect on my part. I'll retract the concern.

✏️ Learnings added 
Learnt from: cemililik
Repo: cemililik/ForgeLM PR: 28
File: docs/analysis/code_reviews/closure-plan-202604300906.md:1919-1927
Timestamp: 2026-05-04T13:05:51.890Z
Learning: In cemililik/ForgeLM, the closure-plan document (`docs/analysis/code_reviews/closure-plan-202604300906.md`) is a living design/planning document. When its §15.5 or phase-table entries describe retention fields, purge kinds, config shapes, or acceptance criteria that a *future* phase (e.g., Phase 21) will implement, this is intentional spec language — not "contract drift" or "scope leak". Each Wave is structured with design-only PRs (e.g., Phase 20) describing what implementation PRs (e.g., Phase 21, separate PR) will deliver. Do not flag closure-plan language about future phase scope as needing immediate implementation or as a doc/code inconsistency.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

🧠 Learnings used 
Learnt from: cemililik
Repo: cemililik/ForgeLM PR: 28
File: docs/usermanuals/tr/getting-started/installation.md:109-109
Timestamp: 2026-05-04T13:05:13.484Z
Learning: In the cemililik/ForgeLM repo documentation, use the correct pip optional-extra name for distributed training: `distributed` (e.g., `pip install 'forgelm[distributed]'`). Do not suggest renaming `distributed` to `deepspeed` in any pip install commands or extras lists. Document `deepspeed>=0.14.0` only as the underlying PyPI dependency that is included inside the `distributed` extra (as defined in `pyproject.toml`). If `forgelm doctor` shows an `extras.deepspeed` label, treat it as a probe/display name rather than the pip extra name.

### Foundation PR closure-coverage özeti

Expand All @@ -1937,6 +1937,7 @@ Wave 1 PR (closure/wave1-faz9-32) review fazında ek olarak **28 ghost entry** t

Bu plan v2 (revised 2026-04-30 kullanıcı kararları sonrası).
**v2.1 (2026-04-30):** §15.5 Foundation PR carry-over registry eklendi.
**v2.5 (2026-05-04 — Wave 2a Round-2 review F-18-04 fix):** §15.5 GH-023 + GH-026 satırları + ek görev paragrafı, Faz 20 design (`gdpr-erasure-design-202605021414.md` §10 Q1) kararıyla hizalandı. Önceki "nested `ingestion.retention.*` shape" yönlendirmesi REDDEDİLDİ; `RetentionConfig` top-level `retention.*` olarak konuşlanır (audit log + staging + ephemeral + raw_documents horizon'larını tek block'ta toplar). Faz 21 acceptance'a `EvaluationConfig.staging_ttl_days` deprecation alias-forward + GDPR docs YAML rewrite (EN+TR) satırları eklendi. Wave 2a Round-2 design-doc agent F-18-04 finding'i kapatıldı.
**v2.4 (2026-05-02 — post-merge audit closure):** Wave 1 post-merge audit raporundaki açık iş'ler kapatıldı. Kod tarafı: (a) Faz 7 — synthetic.py / judge.py / export.py'da 6 `from_pretrained` site'a explicit `trust_remote_code=False` eklendi; (b) Faz 10 — `risk_classification` + `risk_category` Literal'ları 3-değer'den 5-değer EU AI Act tier set'ine genişletildi (default unchanged; backward compatible) + 8 yeni test; (c) Faz 27 — `except Exception as e:` form'undaki ~22 site context-aware BLE001 + justification ile kapatıldı, gate sıkılaştırıldı; (d) Faz 1 — `docs/roadmap.md` "Tür" → "Type" header drift fix. Doc tarafı: Faz 25 acceptance'a "site picker reverted to 6 languages (commit e57aaf8)" notu, Faz 15 acceptance'a `_parser.py` 662-satır kabul gerekçesi.
**v2.3 (2026-05-02 — post-merge):** Wave 1 PR (`closure/wave1-faz9-32`) main'e merge edildi. Status değişiklikleri: 12 faz `✅ Wave 1 PR` → `✅ DONE` (Faz 9, 10, 11, 12, 13, 14, 15, 25, 27, 29, 31, 32); 17 faz + Faz 30 + Faz 34-38 `🔄 Wave 1 IP` → `⏳ PENDING`. Wave 2 başlangıcı için 7 immediate-batch faz dependency-free durumda (§14 güncel haritada).
**v2.2 (2026-05-02):** Ghost feature analysis absorption — Wave 1 PR review aşamasında tespit edilen 28 ghost entry plan'a entegre edildi. Yeni Faz 34-38 açıldı (`forgelm doctor`, `cache-models` + `cache-tasks`, compliance verification toolbelt, `approvals` listing, `reverse-pii`); Faz 30 Task O ghost feature naming/flag drift sweep eklendi (10 doc-only Tier 1 fix); Phase 13 + Phase 14 deferred row'ları Faz 30 Task A'ya eklendi; Faz 21 GH-023 absorption ile `ingestion.retention` extend edildi. Toplam faz sayısı 33 → 38; toplam ~52 PR. Wave 1 IN PROGRESS işaretleri Faz 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 28, 30, 34, 35, 36, 37, 38 satırlarına uygulandı.
Expand Down
Loading