From c78040b87cd3e5d28bbfc9b3a9687f6eea506032 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Wed, 29 Apr 2026 01:54:28 +0300 Subject: [PATCH 1/2] docs(post-merge): refresh roadmap + README + CHANGELOG to match merged-on-main state MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PR #13 (Phase 12 — v0.5.2) is merged on `main`. This sweep aligns the narrative-doc state with that fact and consolidates the post-merge review-cycle fixes under a single follow-up section. roadmap.md / roadmap-tr.md - Phase 11.5 + 12 status descriptions: "merged to development" → "merged to main (PR #11/#12 / #13)" with the actual merge date. - "Latest release on PyPI" line now correctly says v0.4.5 (PyPI state) instead of v0.5.0. Adds explicit clarification that v0.5.0 / v0.5.1 / v0.5.2 are merged to main but not yet tagged + published. - Phase-count matches v0.5.3 plan: "3 phases (12.5, 13, 14) planned". - TR mirror status icons updated to "🟡 Birleşti" matching the EN "🟡 Merged" — were previously prematurely marked "✅ Tamam". - Mermaid diagram in roadmap-tr.md gains the missing P125 node and bumps merged-but-not-published phases to a lighter green. - New cross-link to issue #14 (webhook SSRF DNS-rebinding TOCTOU) surfaces in both the v0.5.2 row and the v0.5.3 "Next" line as the tracked v0.5.3 hardening follow-up. roadmap/phase-14-pipeline-chains.md / roadmap/releases.md - "starts after Phase 12 lands" → concrete release event (`v0.5.2` PyPI tag is published). - v0.5.1 + v0.5.2 release-page status both moved off "Merged on development" to "Merged on main … remaining release-engineering steps" with the issue #14 hardening reference on v0.5.2. README.md - Always-on secrets-scan list updated: was 7 families with the wrong "private-key headers" wording; now lists all nine families per `forgelm.data_audit.SECRET_TYPES` (full BEGIN…END for OpenSSH / PGP). CHANGELOG.md - New "Fixed — post-PR-#13 review-cycle batches (rounds 8-12)" block in [Unreleased] consolidating the post-merge inline-comment batches: audit log hardening, deployer-instructions Markdown injection, quality-filter denominator, numpy fast-path bits guard, sliding-overlap clamp, batch-tokenizer narrow-except, webhook redirect/secret-fallback safety, trainer governance failure visibility, compliance-manifest exception narrowing, strict messages-format validation, wizard ASCII regex flag, GGUF converter case-insensitive validation, markdown-chunker complexity refactor, bidirectional MinHash extraction, streaming length digest, and the doc/test-count drift sweep. Pointers to issue #14 for the remaining DNS-rebinding follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 109 +++++++++++++++++++++++ README.md | 2 +- docs/roadmap-tr.md | 29 +++--- docs/roadmap.md | 14 +-- docs/roadmap/phase-14-pipeline-chains.md | 2 +- docs/roadmap/releases.md | 6 +- 6 files changed, 137 insertions(+), 25 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 43b366ff..87c973d2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,115 @@ All notable changes to ForgeLM are documented here. ## [Unreleased] +### Fixed — post-PR-#13 review-cycle batches (rounds 8-12) + +Inline-comment batches landing on top of PR #13 (now merged to `main`). +Same review surface as rounds 4-7; further hardening on top of the +`v0.5.2` content. The `v0.5.2` git tag + PyPI publish are the +remaining release-engineering steps before this section can be +frozen into a release. + +- **Audit log hardening** (`forgelm/compliance.py`) — HMAC `_hmac` field is now + emitted only when `FORGELM_AUDIT_SECRET` is set; without a secret, a key + derived solely from the public `run_id` would be forgeable, so we no longer + claim tamper-evidence we cannot deliver. `log_event` re-reads the chain head + from disk under the same `flock` so two writers sharing the same log can no + longer fork the chain. `_read_chain_head` refuses to derive a head from a + tail that does not end with `\n` (truncated last record). The oversize- + final-entry case is recovered by re-reading from `seek_start` without + skipping the partial first line. +- **Deployer-instructions Markdown injection** (`forgelm/compliance.py::generate_deployer_instructions`) — + config-derived strings (`system_name`, `model.name_or_path`, fine-tuning + method, model location, foreseeable-misuse bullets, metric names) now go + through `_sanitize_md` before template substitution; pipes / backticks / + link syntax in any of those can no longer break out of table cells or + bullets in the generated Article 13 document. +- **Quality-filter denominator** (`forgelm/data_audit.py::_build_quality_summary`) — + `overall_quality_score` now divides by the number of rows the filter + actually evaluated (text-bearing dict rows) instead of `total_samples`. + A corpus that's 50 % null but 100 % clean on the rest now reads `1.0` + instead of `0.5`. +- **Numpy-fast-path bits guard** (`forgelm/data_audit.py::compute_simhash`) — + the `_compute_simhash_numpy` dispatch now also gates on `bits <= 64`; + without it, `np.uint64` would silently truncate digests wider than 64 + bits. +- **Sliding-overlap clamp** (`forgelm/ingestion.py::ingest_path`) — when + `--overlap` is not passed and the strategy is `sliding`, the implicit + `DEFAULT_SLIDING_OVERLAP` (200) is now clamped to `chunk_size // 2`. A + small `--chunk-size 300` used to trip `_chunk_sliding`'s + "overlap > chunk_size // 2" guard with the default overlap — surfacing + as a confusing error for a knob the user did not set. +- **Batch-tokenizer narrow except** (`forgelm/ingestion.py::_count_section_tokens`) — + the bare `except Exception` around the batched `tokenizer(blocks)` call + is now narrowed to `(TypeError, ValueError)` (the documented + unsupported-batch signal); the returned `BatchEncoding` is shape- + validated before its `input_ids` is consumed. Real bugs (corrupted + input, OOM, etc.) no longer mask behind the slow per-block fallback. +- **Webhook secret-fallback safety** (`forgelm/webhook.py`) — + `requests.post` now passes `allow_redirects=False` (an SSRF-pre-validated + URL cannot be redirected to a private destination) and the + `mask_secrets` `ImportError` fallback emits `[REDACTED — secrets + masker unavailable]` instead of the raw 512-char reason prefix. + See [#14](https://github.com/cemililik/ForgeLM/issues/14) for the + remaining DNS-rebinding TOCTOU follow-up tracked for `v0.5.3`. +- **Trainer governance failure visibility** (`forgelm/trainer.py`) — the + `data_governance_report.json` export try/except now catches the full + `Exception` set (was `OSError` only) so non-IO failures (`TypeError`, + `ValueError`, `AttributeError`) still surface as + `compliance.governance_failed` audit events instead of crashing the + surrounding compliance flow. The rollup `compliance.artifacts_exported` + event is gated on a `governance_ok` flag so the audit chain truthfully + reflects which artefacts are actually on disk. +- **Compliance manifest exception narrowing** (`forgelm/compliance.py`) — + the broad `except Exception` around the HF Hub `load_dataset_builder` + fingerprint fetch is now a tuple of realistic failure modes + (`ImportError`, `FileNotFoundError`, `ValueError`, `AttributeError`, + `ConnectionError`, `TimeoutError`). +- **Strict messages-format validation** (`forgelm/data.py`) — + `_process_messages_format` now explicitly checks `isinstance(role, str)` + and `isinstance(content, str)` before formatting; non-string content + (dicts, ints) used to be silently coerced via f-string `__format__` + and slip through into training. +- **Wizard ASCII regex flag** (`forgelm/wizard.py`) — `_HF_HUB_ID_RE` + now compiles with `re.ASCII` so the `\w` class means + `[A-Za-z0-9_]`. HF Hub IDs are ASCII-only, and Unicode-aware `\w` + would otherwise accept characters the Hub itself rejects. +- **GGUF converter case-insensitive validation** (`forgelm/export.py`) — + the `FORGELM_GGUF_CONVERTER` `.py` suffix check now uses + `casefold()` (cross-platform: HFS+/NTFS), and `export_model()`'s + catch widened from `(ImportError, FileNotFoundError)` to also + include `ValueError` so a non-`.py` env override produces an + `ExportResult` instead of crashing the caller. +- **Markdown chunker complexity refactor** (`forgelm/ingestion.py`) — + `_chunk_markdown_tokens` split into `_build_markdown_section_blocks` + (render breadcrumb + body), `_count_section_tokens` (batch tokenizer + call with per-block fallback), and the main chunker (greedy packing). + Cognitive complexity drops from 16 → ~8. +- **Bidirectional MinHash extraction** (`forgelm/data_audit.py`) — the + two near-identical `a→lsh_b` / `b→lsh_a` query loops in + `_count_leaked_rows_minhash_bidirectional` were extracted into one + `_count_leaks_against_index` helper. Complexity drops from 24 → ~5; + the SonarCloud duplication metric on this file goes away. +- **Streaming length digest** (`forgelm/data_audit.py`) — the per-split + text-length distribution is now accumulated via a bounded + `_LengthDigest` (Algorithm R reservoir, 100K cap) instead of an + unbounded `List[int]`. Audit memory on multi-million-row splits is + now O(1) instead of O(n). +- **Documentation drift sweep (round N)** — five compliance-summary + links repointed to `../../forgelm/...`, two missing FSDP knobs + (`fsdp_backward_prefetch`, `fsdp_state_dict_type`) and two webhook + knobs (`allow_private_destinations`, `tls_ca_bundle`) added to both + EN and TR `configuration` reference; Pro CLI section added to + `README.md`; CI now runs a bilingual H2 parity check across seven + EN/TR doc pairs (`configuration`, `usage`, `distributed_training`, + `data_preparation`, `architecture`, `ingestion`, `data_audit`); test + count refreshed to 47 in `CONTRIBUTING.md` to match + `architecture.md`; secrets list aligned to the full nine families + in `forgelm.data_audit.SECRET_TYPES` (was missing two private-key + splits + Azure storage in some prose). +- **Phase 12 fenced log block** in both `usage.md` and `usage-tr.md` + now uses ```` ```text ```` so markdownlint MD040 stops flagging it. + ### Fixed — multi-agent master review (rounds 4-7) Multi-dimension review (business, code, compliance, documentation, performance, security) surfaced a cluster of correctness, claim/evidence, and silent-failure issues that have been swept in batches. diff --git a/README.md b/README.md index 4418f034..3a6fd97a 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,7 @@ - **Chunking Strategies**: `paragraph` (default; preserves boundaries), `sliding` (fixed window with overlap), or `markdown` (v0.5.2 — heading-aware splitter that keeps fenced code blocks atomic and inlines a heading breadcrumb at the top of each chunk for SFT context). A `semantic` strategy is reserved for a follow-up phase — the implementation in `forgelm.ingestion` raises `NotImplementedError` today and the CLI hides it from `--strategy` choices. Token-aware mode in v0.5.1: `--chunk-tokens 1024 --tokenizer Qwen/Qwen2.5-7B-Instruct` sizes chunks against your model's actual vocabulary. - **PII Masking on Ingest**: `--pii-mask` redacts emails, phones, credit cards (Luhn-validated), IBAN, and national IDs (TR / DE / FR / US-SSN) before chunks land in the JSONL. - **PDF Page Header/Footer Dedup (v0.5.1)**: Lines that recur on ≥ 70 % of PDF pages (watermarks, page numbers, copyright lines) are stripped automatically — the audit's near-duplicate counts stop misfiring on long policy / book PDFs. -- **Dataset Audit**: `forgelm audit data/sft.jsonl --output ./audit/` — produces `data_audit_report.json` with sample count, length distribution, top-3 language detection, LSH-banded near-duplicate rate (`O(n × k)` typical case, exact recall at the default Hamming threshold; v0.5.2 added optional **MinHash LSH** via `--dedup-method minhash` for >50K-row corpora), cross-split leakage check, null/empty rate, and PII flag counts with **severity tiers** (critical / high / medium / low + worst-tier verdict). v0.5.2 also added an always-on **secrets/credential scan** (AWS / GitHub / Slack / OpenAI / Google / JWT / private-key headers) and an opt-in **heuristic quality filter** (`--quality-filter`). CPU-only; streaming JSONL reader keeps memory bounded on multi-million-row splits; feeds EU AI Act Article 10 governance artifact automatically when present at training time. Legacy `--data-audit` flag still works as a deprecation alias. +- **Dataset Audit**: `forgelm audit data/sft.jsonl --output ./audit/` — produces `data_audit_report.json` with sample count, length distribution, top-3 language detection, LSH-banded near-duplicate rate (`O(n × k)` typical case, exact recall at the default Hamming threshold; v0.5.2 added optional **MinHash LSH** via `--dedup-method minhash` for >50K-row corpora), cross-split leakage check, null/empty rate, and PII flag counts with **severity tiers** (critical / high / medium / low + worst-tier verdict). v0.5.2 also added an always-on **secrets/credential scan** (nine families per `forgelm.data_audit.SECRET_TYPES`: AWS access keys, GitHub tokens, Slack tokens, OpenAI / Google API keys, JWTs, full OpenSSH/PGP private-key blocks, Azure storage connection strings) and an opt-in **heuristic quality filter** (`--quality-filter`). CPU-only; streaming JSONL reader keeps memory bounded on multi-million-row splits; feeds EU AI Act Article 10 governance artifact automatically when present at training time. Legacy `--data-audit` flag still works as a deprecation alias. - **Secrets-Aware Ingest (v0.5.2)**: `forgelm ingest … --secrets-mask` scrubs credentials before chunks land in the JSONL — fine-tuning on text containing real API keys memorises them at training time. Pairs with `--pii-mask`; secrets run first so combined detectors don't double-count overlapping spans. ### Quickstart Layer (v0.4.5) diff --git a/docs/roadmap-tr.md b/docs/roadmap-tr.md index 323027c1..aa4cef5c 100644 --- a/docs/roadmap-tr.md +++ b/docs/roadmap-tr.md @@ -10,22 +10,23 @@ | ✅ Tamam | [Faz 10 — Post-Training Tamamlama](roadmap/phase-10-post-training.md) | `inference.py`, `chat`, `export` (GGUF), `--fit-check`, `deploy` — `v0.4.0` | | ✅ Tamam | [Faz 10.5 — Quickstart Katmanı ve Onboarding](roadmap/phase-10-5-quickstart.md) | `forgelm quickstart