feat(wave-5): Faz 30 full sweep — final pre-release documentation + tooling closure

.github/workflows/ci.yml

@@ -205,6 +205,28 @@ jobs:
         # automatically the next run.
         run: python3 tools/check_bilingual_parity.py --strict
 
+      - name: Markdown anchor resolution check (strict)
+        # Wave 5 / Faz 30 Task N: every Markdown anchor + relative-link
+        # under docs/ must resolve.  Catches renamed/removed targets,
+        # slug-case drift, and forward-refs to pages that never landed
+        # (the GH-014/017/019 bug class).  Strict mode wired up after
+        # the Wave 5 cleanup zeroed the live drift count from 36 → 0;
+        # the tool itself remains usable in advisory mode (no flag) for
+        # local feedback during a docs PR.  See tools/check_anchor_resolution.py
+        # for what's checked and the GFM slug approximation used.
+        run: python3 tools/check_anchor_resolution.py --strict
+
+      - name: CLI / docs help-consistency drift check (strict)
+        # Wave 5 / Faz 30 Task J: catches docs that cite ghost flags
+        # or non-existent subcommands (the GH-008/011/016/018/020
+        # bug class).  Strict mode wired up after the Wave 5 cleanup
+        # commit zeroed the live drift count from 40 → 0 (matches the
+        # check_anchor_resolution.py precedent: gate lands advisory,
+        # baseline cleanup follows, later commit flips to --strict).
+        # The tool itself remains usable in advisory mode (no flag)
+        # for local feedback during a docs PR.
+        run: python3 tools/check_cli_help_consistency.py --strict
+
       - name: License check
         run: |
           test -f LICENSE || (echo "LICENSE file missing" && exit 1)

.gitignore

@@ -27,6 +27,7 @@ docs/analysis/code_reviews/*
 !docs/analysis/code_reviews/iso27001-soc2-alignment-202605052315.md
 !docs/analysis/code_reviews/wave4-review-prompt.md
 !docs/analysis/code_reviews/wave4-followup-review-prompt.md
+!docs/analysis/code_reviews/wave5-review-prompt.md
 
 # Generated by tools/build_usermanuals.py — produced from
 # docs/usermanuals/<lang>/**/*.md during the CI deploy. Not committed;

CHANGELOG.md

@@ -4,6 +4,158 @@ All notable changes to ForgeLM are documented here.
 
 ## [Unreleased]
 
+### Wave 5 — Faz 30 full sweep (`closure/wave5-integration`)
+
+Final pre-release documentation + tooling closure. Wave 4 partial
+landed Faz 30 Tier 1 ghost-feature drift + stat blocks; Wave 5
+completes Faz 30 Tasks A through O end-to-end.
+
+**Faz 30 Task A — feature × {guide, reference, usermanual} EN+TR
+triplets (50 new doc files):**
+
+- 11 new reference docs: `docs/reference/{verify_audit,
+  verify_annex_iv_subcommand, verify_gguf_subcommand,
+  purge_subcommand, reverse_pii_subcommand, approve_subcommand,
+  approvals_subcommand, doctor_subcommand, cache_subcommands,
+  safety_eval_subcommand, library_api_reference}.md` (+ TR
+  mirrors). Each carries the live parser surface (flags + exit
+  codes + audit events emitted), worked examples, and stable
+  symbol citations.
+- 5 new guides: `docs/guides/{getting-started, air_gap_deployment,
+  human_approval_gate, library_api, performance}.md` (+ TR
+  mirrors). `getting-started.md` is the v0.5.5 onboarding canonical
+  (opens with `forgelm doctor`); `air_gap_deployment.md` is the
+  deep deployer cookbook for `cache-models` / `cache-tasks`;
+  `human_approval_gate.md` cross-links access_control.md §6's
+  `jq -rs` segregation-of-duties pipeline.
+- 9 new usermanual pages (`docs/usermanuals/{en,tr}/...`):
+  `compliance/{verify-audit, annex-iv, gdpr-erasure,
+  human-approval-gate}`, `deployment/verify-gguf`,
+  `operations/{iso-soc2-deployer, supply-chain}`,
+  `reference/{library-api, performance}`.
+- `docs/usermanuals/_meta.yaml` registers the 8 new pages with
+  full 6-language title sets (EN+TR mandatory; DE/FR/ES/ZH
+  fallback to EN per Task L policy).
+- `tools/check_bilingual_parity.py::_PAIRS` grew from 23 to 39
+  pairs (16 new pairs across guides + reference).
+
+**Faz 30 Task J — `tools/check_cli_help_consistency.py` (new
+guard):**
+
+- 749 LOC + 15 pinned tests (`tests/test_check_cli_help_consistency.py`).
+- Discovers the live parser surface by spawning
+  `forgelm <subcommand> --help` for every shipped subcommand;
+  walks docs / README for fenced bash/shell `forgelm` invocations;
+  reports three drift classes (subcommand not in parser; flag not
+  in parser for that subcommand; flag value not in parser's
+  `choices` list). Skip patterns honour forward-reference framing
+  (`planned`/`roadmap`/`future`/`not in v0.5.5`) and `Wrong:` /
+  `Don't:` / `Anti-pattern:` / `Legacy:` / `Yanlış:`-tagged code
+  blocks.
+- `.github/workflows/ci.yml` runs the gate in `--strict` mode in
+  the validate job (advisory in commit `c7bedc9`, flipped to
+  strict in commit `fbb082d` after baseline cleanup).
+
+**Faz 30 Task N — `tools/check_anchor_resolution.py` strict CI
+flip:**
+
+- Drift baseline cleanup: 36 → 0 broken anchor / relative-link
+  references. Fixes cluster in: `forgelm/data_audit.py` /
+  `forgelm/cli.py` references (Faz 14 / 15 package splits — paths
+  updated to `forgelm/data_audit/<sub>.py` /
+  `forgelm/cli/<sub>.py` per symbol); slug-case mismatches
+  (double-dash collapse: `#v050--document-ingestion-...` →
+  `#v050-document-ingestion-...`); Turkish-letter slug drift
+  (`#dil-secimi` → `#dil-seçimi` Unicode-preserving canonical);
+  documentation.md self-illustrative `[Other doc 1](other.md)`
+  examples rephrased as plain prose.
+- `.github/workflows/ci.yml` runs the gate in `--strict` mode.
+
+**Faz 30 Task O — Tier 1 ghost-feature drift residuals:**
+
+- GH-011: `docs/usermanuals/{en,tr}/evaluation/benchmarks.md`
+  rewritten — `forgelm benchmark --model "..."` (subcommand form
+  that doesn't exist) → `forgelm --config <yaml>
+  --benchmark-only <path>` (the shipped flag form).
+- GH-016: `--export-bundle` → `--compliance-export` rename in
+  `docs/qms/sop_model_training.md`,
+  `docs/qms/roles_responsibilities.md`,
+  `docs/roadmap/completed-phases.md`.
+- GH-018: `kserve` / `triton` rows + dedicated example sections
+  removed from `docs/usermanuals/{en,tr}/deployment/deploy-targets.md`
+  (parser only ships `{ollama,vllm,tgi,hf-endpoints}`); replaced
+  with hand-authored manifest pointer note.
+- GH-020: ingest flag drift in
+  `docs/usermanuals/{en,tr}/data/{ingestion, pii-masking,
+  language-detection, synthetic-data}.md` — `--max-tokens` →
+  `--chunk-tokens`; removed non-existent `--language` /
+  `--include` / `--exclude` / `--format` / `--pii-locale` flags;
+  `--strategy` choices clarified to `{sliding,paragraph,markdown}`
+  (no `tokens` / `sentence`); `--pii-locale` example replaced
+  with the shipped `forgelm audit --pii-ml --pii-ml-language LANG`
+  invocation.
+
+**Faz 30 Tasks B + C + D + E + F + G:**
+
+- `docs/usermanuals/_meta.yaml` +8 page entries (Task B); JS
+  bundle rebuild via `tools/build_usermanuals.py` 56 → 64 pages
+  EN+TR (Task C; artefacts gitignored, deployed by CI).
+- `site/{index,features,compliance,quickstart,privacy}.html`
+  brought to v0.5.5 final state with new Wave 2-5 feature blocks
+  (Library API, GDPR rights tooling, doctor, air-gap, ISO/SOC 2
+  alignment, supply-chain security, etc.); `site/js/translations.js`
+  6-language i18n updates (Task D).
+- `README.md` + `CONTRIBUTING.md` + `CLAUDE.md` final pass: stat
+  block refreshed (~72 modules / 1428 tests); feature list
+  reflects Wave 2-5 deliverables; CI guard list updated for the
+  two new strict gates (Task E).
+- `docs/roadmap.md` + `-tr.md`: Phase 12.6 closure cycle ✅ Done;
+  v0.5.5 next-up; `docs/roadmap/releases.md` v0.5.5 row added;
+  `docs/roadmap/risks-and-decisions.md` Wave 5 cycle decisions
+  appended; new `docs/roadmap/phase-12-6-closure-cycle.md`
+  summary (Task F).
+- 10 standards files final pass: `architecture.md` reflects
+  package splits; `documentation.md` cites the new CI guards;
+  `localization.md` formalises EN+TR mandatory + DE/FR/ES/ZH
+  deferred; `release.md` documents v0.5.5 release sequence;
+  `testing.md` test count refreshed (Task G).
+
+**Faz 30 Tasks H + I + K + L + M:** all green / closed.
+
+- H (parity strict CI): 39/39 OK.
+- I (site-claim CI strict): all green (5 templates, 16 GPU
+  profiles, version 0.5.5).
+- K (config doc auto-gen): closed without building the
+  speculatively-named `tools/regenerate_config_doc.py`. The
+  description→doc-source-of-truth contract is gated by
+  `check_field_descriptions.py --strict` +
+  `check_bilingual_parity.py --strict` +
+  `check_cli_help_consistency.py --strict` together;
+  configuration.md + `-tr.md` are hand-maintained.
+- L (locale policy): formalised in `docs/standards/localization.md`
+  via Task G.
+- M (diagram / asset sweep): user-facing tree image-free; no
+  drift.
+
+**Migration notes (Wave 5):**
+
+- **`pyproject.toml` version `0.5.1rc1` → `0.5.5`.** Brought
+  forward from Faz 33 because `tools/check_site_claims.py
+  --strict` enforces site/code version match (site cites
+  `pip install forgelm==0.5.5`). The Faz 33 release commit
+  pushes the tag + PyPI publish; v0.5.5 will be installable from
+  PyPI within minutes of that release.
+- **Two new strict CI gates land.** Operators running ForgeLM
+  CI workflows on top of the project's existing `ci.yml` gain
+  `tools/check_anchor_resolution.py --strict` (Wave 4 / Faz 26
+  tool, advisory until this commit) and
+  `tools/check_cli_help_consistency.py --strict` (Wave 5 / Faz
+  30 Task J new tool). Both fail-the-build on drift; baselines
+  are at zero before the strict flip lands.
+- **8 new usermanual pages + 16 new bilingual doc pairs.** No
+  user-facing behaviour change; the navigation surface grows
+  but the underlying capabilities all shipped in Wave 1-4.
+
 ### Wave 4 — Faz 22 + 23 + 26 + 30 (`closure/wave4-integration`)
 
 ISO 27001 / SOC 2 Type II alignment + QMS bilingual sweep + final

CLAUDE.md

@@ -46,26 +46,46 @@ Each skill's `SKILL.md` has the full checklist. Follow it; don't skip steps to s
 
 ```
 ForgeLM/
-├── forgelm/                 # Source code (17 single-file modules)
-│   ├── cli.py               # Entry point
+├── forgelm/                 # Source code: ~22 single-file modules + 2 sub-packages
+│   ├── cli/                 # CLI package (Phase 15 split): _parser, _dispatch,
+│   │                        # _exit_codes, subcommands/{ingest, audit, chat,
+│   │                        # export, deploy, quickstart, doctor, cache,
+│   │                        # purge, reverse_pii, approve, approvals,
+│   │                        # safety_eval, verify_audit, verify_annex_iv,
+│   │                        # verify_gguf, ...}
+│   ├── data_audit/          # Audit package (Phase 14 split): _orchestrator,
+│   │                        # _aggregator, _streaming, _simhash, _minhash,
+│   │                        # _pii_regex, _pii_ml, _secrets, _quality,
+│   │                        # _croissant, _summary, _splits
 │   ├── config.py            # Pydantic schemas (19 models)
 │   ├── trainer.py           # TRL wrapper (SFT/DPO/SimPO/KTO/ORPO/GRPO)
 │   ├── model.py             # HF + PEFT model loading
 │   ├── data.py              # Dataset loading + format detection
+│   ├── ingestion.py         # Raw docs → SFT JSONL (`forgelm ingest`)
 │   ├── safety.py            # Llama Guard + harm categories + auto-revert
-│   ├── compliance.py        # EU AI Act Articles 9-17 + Annex IV
-│   ├── webhook.py           # Slack/Teams notifications
+│   ├── compliance.py        # EU AI Act Articles 9-17 + Annex IV + GDPR purge / reverse-pii primitives
+│   ├── webhook.py           # Slack/Teams notifications (5-event vocabulary)
 │   ├── grpo_rewards.py      # Built-in GRPO format/length shaping reward fallback
-│   └── ...                  # benchmark, judge, merging, synthetic, wizard, ...
-├── tests/                   # pytest test modules
+│   ├── _http.py             # SSRF-guarded HTTP chokepoint (safe_post / safe_get)
+│   ├── _version.py          # `__version__` + `__api_version__` (decoupled)
+│   └── ...                  # benchmark, judge, merging, synthetic, wizard,
+│                            # quickstart, model_card, fit_check, deploy, chat,
+│                            # export, inference, results, utils
+├── tests/                   # 68 test modules, ~1442 tests
+├── tools/                   # CI guards: check_anchor_resolution,
+│                            # check_bilingual_parity, check_cli_help_consistency,
+│                            # check_field_descriptions, check_pip_audit,
+│                            # check_bandit, check_site_claims,
+│                            # generate_sbom, build_usermanuals
 ├── docs/
 │   ├── roadmap.md           # Public roadmap (short index)
 │   ├── roadmap/             # Detailed phase files + archive
 │   ├── reference/           # User-facing API/config reference
 │   ├── guides/              # User-facing tutorials
+│   ├── usermanuals/{en,tr}/ # 4-section user manual (training, eval, deploy, ref)
 │   ├── design/              # Design specs (internal)
 │   ├── standards/           # Engineering standards (this project's rulebook)
-│   ├── qms/                 # Quality management SOPs (EU AI Act Art. 17)
+│   ├── qms/                 # Quality management SOPs (EU AI Act Art. 17, EN+TR)
 │   ├── analysis/            # Research, code reviews, external repo analyses
 │   └── marketing/           # Local-only (gitignored): marketing + strategy
 ├── config_template.yaml     # Canonical YAML example — CI dry-runs this
@@ -127,10 +147,16 @@ Default workflow for a non-trivial change:
 
    ```bash
    ruff format . && ruff check . && pytest tests/ && \
-     forgelm --config config_template.yaml --dry-run
+     forgelm --config config_template.yaml --dry-run && \
+     python3 tools/check_bilingual_parity.py --strict && \
+     python3 tools/check_anchor_resolution.py --strict && \
+     python3 tools/check_cli_help_consistency.py --strict
    ```
 
-   All four must pass.
+   All seven must pass. The first four are the historical gauntlet; the
+   three doc guards (Wave 3 / Wave 4 / Wave 5 additions) catch bilingual
+   structural drift, broken markdown anchors, and CLI ↔ docs help-text
+   drift before the PR opens.
 
 ## Etiquette when communicating with the user
 

CONTRIBUTING.md

@@ -38,19 +38,31 @@ Branch naming: `feat/`, `fix/`, `docs/`, `test/`, `chore/` + short description.
 
 ### 4. Make your changes
 
-Edit the code, then verify:
+Edit the code, then run the full validation gauntlet (every guard CI also
+enforces — passing locally means CI will too):
 
 ```bash
-# Run tests
-pytest tests/ -q
+# 1. Lint + format
+ruff format . && ruff check .
 
-# Run linter + format check
-ruff check . && ruff format --check .
+# 2. Test suite
+pytest tests/ -q
 
-# Quick smoke test
+# 3. Config dry-run smoke test
 forgelm --config config_template.yaml --dry-run
+
+# 4. Doc-side guards (Wave 4 / Wave 5 additions)
+python3 tools/check_bilingual_parity.py --strict
+python3 tools/check_anchor_resolution.py --strict
+python3 tools/check_cli_help_consistency.py --strict
 ```
 
+The four-tool sequence at the top of the gauntlet (`ruff` + `pytest` +
+`--dry-run`) is the historical "self-review" command from
+[`docs/standards/code-review.md`](docs/standards/code-review.md). The three
+doc guards landed in Waves 3-5 and run on every PR via `.github/workflows/`;
+running them locally before pushing avoids CI round-trips.
+
 ### 5. Submit a PR
 
 Push your branch and open a Pull Request against `main`.
@@ -59,10 +71,13 @@ Push your branch and open a Pull Request against `main`.
 
 ### Project Structure
 
-ForgeLM is a single-package layout: ~26 single-file modules under `forgelm/`,
-~47 test files under `tests/`, plus `configs/`, `docs/`, and `notebooks/`. For
-the authoritative module-by-module map (purpose, public surface, dependency
-arrows), see [`docs/reference/architecture.md`](docs/reference/architecture.md).
+ForgeLM is a single-package layout: a mix of single-file modules and two
+focused sub-packages (`forgelm/cli/` post-Phase-15 split and
+`forgelm/data_audit/` post-Phase-14 split) under `forgelm/`, ~68 test files /
+~1442 tests under `tests/`, plus `configs/`, `docs/`, `tools/` (CI guards),
+and `notebooks/`. For the authoritative module-by-module map (purpose, public
+surface, dependency arrows), see
+[`docs/reference/architecture.md`](docs/reference/architecture.md).
 
 ### Running Tests