Skip to content

docs(wave6): canonical-name drift sweep + 5 new CI guards (audit closure) - #39

Merged
cemililik merged 14 commits into
mainfrom
development
May 8, 2026
Merged

cemililik merged 14 commits into
mainfrom
development

Conversation

@cemililik

@cemililik cemililik commented May 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Closes the 2026-05-07 docs audit (≈270 findings) across 7 commits:
canonical-name drift sweep, reference subcommand pages, P1 schema
rewrites, full safety_compliance-tr translation, 5 new defence-layer
CI guards, and post-audit residual cleanup. All 8 doc guards green.

Scope: 130 files / +4784 / -1510. Docs-only — no forgelm/ source
changes.

Wave breakdown (commit log)

Commit Wave Focus
2ba095f A + B Mechanical drift sweep — pipeline.reverted → model.reverted (12 docs), compliance.human_approval → evaluation.require_human_approval (10 docs), --add-row-ids reframe (5 docs), webhook event vocabulary sync, version stamps 0.5.2 → 0.5.5, ghost field cleanup
f7f5de0 C Reference subcommand pages — JSON envelopes (approve, approvals, purge, reverse_pii, safety_eval, verify_audit), exit-code tables (verify_audit 0/1 surface), 11 ghost audit flags removed from cli.md
9788f49 D1 P1 parallel-universe YAML schema rewrite — 6 trainer pages (sft, dpo, simpo, kto, orpo, grpo) EN+TR, KTO format response→completion, secrets 11→9 families, judge-tr.md identifier revert
0d48266 D2 yaml-templates.md rewrite (canonical schema + ghost-field deprecation table); compliance.config_hash reframe across 15 files
7ff9b35 D3 safety_compliance-tr.md full translation — 38 sections, 170→547 lines (was a stub)
0c4692c E 5 new defence-layer CI guards: check_yaml_snippets.py (333 yaml blocks, 23 ForgeConfig validate), check_audit_event_catalog.py (36↔36), check_library_api_doc.py (30↔30 symbols), check_doc_numerical_claims.py (9/6/5/5 counts), check_bilingual_code_blocks.py (45 pairs)
c9998ff F Residual cleanup R1-R33 — 30 fixes + 5 new TR mirrors (compliance_summary-tr, alignment-tr, cicd_pipeline-tr, quickstart-tr, troubleshooting-tr); architecture.md flat→package; library_api Example #3 unrunnable; configuration.md RoPE/webhook drift; design docs status banner + 8-step wizard rewrite

Audit findings closure

Status Count Notes
Closed ≈260 All 9 systemic patterns (P1-P9) addressed; 22+27 P1 trainer/evaluation pages either rewritten or covered by render-pointer
Audit hallucinations (won't-fix) 2 P9 prompt corpus (140×6 is correct); evaluation.approval_retention_days (canonical alias is staging_ttl_days)
Phase 28+ backlog 9 --add-row-ids impl, HMAC body sign, encryption_at_rest config field, additional secret patterns, KTO weight surfaces, GRPO beta flat field, compliance.config_hash payload emit
v0.6.x scope ~30 pages Usermanuals P1-residual (training/{lora,galore,distributed,long-context}, evaluation/, deployment/) — best handled via render-from-code pipeline rather than hand-rewrites

Provenance: docs/analysis/code_reviews/2026-05-07_docs_audit.md (audit) + 2026-05-07_docs_audit_plan.md (verification + execution plan). Both gitignored under docs/analysis/ per project convention.

CI guard surface

3 pre-existing + 5 new = 8 doc guards, all green:

$ python3 tools/check_bilingual_parity.py --strict
OK: all 45 bilingual doc pair(s) carry the same H2/H3/H4 spine.
$ python3 tools/check_anchor_resolution.py --strict
OK: 273 markdown file(s) under docs/ have all anchors + relative links resolved.
$ python3 tools/check_cli_help_consistency.py --strict
OK: 439 forgelm invocation(s) across docs, README.md all match the live parser surface.
$ python3 tools/check_yaml_snippets.py --strict
OK: 23 ForgeConfig-shaped YAML snippet(s) validated across 357 total yaml blocks under docs/.
$ python3 tools/check_audit_event_catalog.py --strict
OK: 36 unique audit-event(s) in forgelm/ all match the 36 catalog row(s).
$ python3 tools/check_library_api_doc.py --strict
OK: 30 symbols in forgelm.__all__ all match the 30 top-level row(s) (+ 2 method rows).
$ python3 tools/check_doc_numerical_claims.py --strict
OK: every numerical doc claim matches canonical scrapes (secret_families=9, trainer_types=6, templates=5, webhook_events=5).
$ python3 tools/check_bilingual_code_blocks.py --strict
OK: all 45 bilingual pair(s) have matching fenced-block counts AND matching YAML-key sets per block.

The new gates caught real regressions during the wave runs — Wave D's lora:-block omission in 5 trainer YAML snippets, and Wave F's accidental forgelm wizard (not a real subcommand — the canonical form is forgelm --wizard flag) in 3 docs. Both were fixed before commit; the gates would have blocked them at PR time.

Test plan

  • python3 tools/check_bilingual_parity.py --strict — H2/H3/H4 spine across 45 EN+TR pairs.
  • python3 tools/check_anchor_resolution.py --strict — 273 markdown files, all anchors resolve.
  • python3 tools/check_cli_help_consistency.py --strict — 439 forgelm invocations match parser surface.
  • python3 tools/check_yaml_snippets.py --strict — 23 ForgeConfig-shaped snippets validate against live Pydantic schema.
  • python3 tools/check_audit_event_catalog.py --strict — 36 emitted events match 36 catalog rows.
  • python3 tools/check_library_api_doc.py --strict — forgelm.__all__ ↔ library_api_reference.md rows match.
  • python3 tools/check_doc_numerical_claims.py --strict — secret families / trainer types / templates / webhook events claims match canonical scrapes.
  • python3 tools/check_bilingual_code_blocks.py --strict — fenced-block counts + YAML-key sets match across pairs.
  • pytest tests/ — no regressions (docs-only PR, but verify CI green).
  • forgelm --config config_template.yaml --dry-run — Pydantic validation passes.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • GDPR CLI: row-id corpus purge and reverse-PII query; interactive setup wizard and revamped quickstart; human-approval gate moved under evaluation.require_human_approval.
    • Safety & synthetic data: expanded safety probe set (51 prompts / 18 categories) and improved synthetic-data controls.
  • Documentation

    • Major docs refresh (including Turkish translations): audit-event vocabulary, lifecycle/webhook behavior, audit manifest naming, CLI JSON envelopes and exit-code semantics, trainer/config examples, quickstart and guides.
  • Tools

    • Added docs validation tools for audit-event parity, bilingual/code-block parity, numerical-claim checks, and YAML snippet validation.

cemililik and others added 7 commits May 7, 2026 23:28
…e/usermanuals

Closes Wave A (mechanical) + Wave B (canonical-name drift) of the
2026-05-07 docs audit plan.  ≈240 of the original ≈270 audit findings
remain after this sweep — the deferred bulk is P1 (parallel-universe
YAML schema) which needs the Wave D render-from-code pipeline.

Mechanical (Wave A):
- pipeline.reverted → model.reverted across 12 docs (21 lines)
- ForgeLM 0.5.2 → 0.5.5 in 4 usermanual artefacts
- README.md:59 forgelm purge --subject-id → --row-id --corpus
- CHANGELOG bilingual-pair count 39 → 40 (verified _PAIRS = 40)
- CLAUDE.md / CONTRIBUTING.md / docs/standards/testing.md test-module
  count 68 → 70 (verified find tests/test_*.py | wc -l = 70)
- TR typos: 'Sıraki' → 'Sonraki' in installation + first-run

Canonical-name drift (Wave B):
- P2: compliance.human_approval → evaluation.require_human_approval
  (10 user-facing files; ComplianceMetadataConfig has no
  human_approval, EvaluationConfig.require_human_approval is the
  real field per forgelm/config.py:480)
- P5: forgelm audit --add-row-ids reframed as Phase 28 backlog item
  (5 files); flag does not exist in current parser
- P7: webhook event vocabulary synced to canonical 5-event surface
  (training.start/success/failure/reverted, approval.required) in
  4 user-facing tables; webhook lifecycle method names corrected
  (notify_started/succeeded/failed → notify_start/success/failure)
  in 6 QMS/design/reference docs
- P8: model.integrity_verified row added to audit_event_catalog
  (EN+TR); was emitted at trainer.py:1450 but undocumented

Ghost-field cleanup:
- webhook.secret_env removed from 5 files (no such field in
  WebhookConfig; webhook.py does not HMAC-sign bodies)
- encryption_at_rest: true|false claim removed from 4 files
  (DataGovernanceConfig has no encryption field)

QMS reconciliation:
- statement_of_applicability column header semantics clarified in
  EN+TR (operator-side vs ForgeLM-evidence distinction); resolves
  the apparent SoA ↔ iso27001_control_mapping conflict on 10 A.5.x
  controls

TR mirror parity:
- operations/cicd-tr.md GitLab stages: added missing - deploy
- data/audit-tr.md JSON schema expanded ({...} → full)
- concepts/data-formats-tr.md KTO JSON formatted to match EN

Earlier review-comment fixes folded in (from inline review
verification):
- site/css/style.css:2060,2328 currentColor → currentcolor
  (lowercase per CSS spec / Stylelint canonical form)
- site/quickstart.html:363-365 wizard modal architectural comment
  reflects actual implementation (hidden + sibling aria-hidden;
  no inert)

Validated: bilingual_parity ✓, anchor_resolution ✓,
cli_help_consistency ✓ (all --strict).

Deferred to Wave D:
- P1 parallel-universe YAML schema (~22 CRITICAL + 27 HIGH pages) —
  needs render-from-code via tools/regenerate_yaml_templates_doc.py
- compliance.config_hash reframe (>10 files; nuanced rewrite)
- usermanuals training/{kto,orpo,grpo}-tr.md output: blocks (EN
  itself uses ghost output: block; full canonical rewrite belongs
  in P1 lift)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wave C of the 2026-05-07 docs audit plan. Rewrites every CRITICAL
finding in `docs/reference/*_subcommand*.md` and the related
`usermanuals/{en,tr}/reference/{cli,exit-codes}.md` against the live
parser + handler surface.

Per-subcommand JSON envelope corrections (EN + TR):

- **verify_audit**: doc claimed exit 2 for `--require-hmac` env-var
  unset; code returns 1 (`EXIT_CONFIG_ERROR` covers both option errors
  and integrity failures during the v0.5.5 stabilisation cycle —
  `_verify_audit.py:18-27`). Exit-code table reduced to 0/1; v0.6.x
  deprecation note added.

- **safety_eval**: bundled probe set advertised as "50 prompts × 14
  categories" — actual is 51 × 18 (verified via wc -l +
  category dedup over `forgelm/safety_prompts/default_probes.jsonl`).
  Output artefact filenames `safety_report.json` / `safety_examples.jsonl`
  / `safety_run.log` were fabricated; real names are
  `safety_results.json` (`forgelm/safety.py:417`) and
  `safety_trend.jsonl` (`forgelm/safety.py:686`). Phantom `[eval]`
  extra dependency on exit-1 path removed (transformers ImportError
  routes to EXIT_TRAINING_ERROR per `_safety_eval.py:154-155`).

- **approve / approvals**: envelope keys re-aligned to
  `_approve.py:468-477` and `_approvals.py:240-249,440-444`
  respectively. `promoted_to` -> `final_model_path`;
  `events`/`staging` -> `chain`/`staging_contents`;
  `age` (string)/`staging` (string) -> `age_seconds` (int)/`staging_exists`
  (bool). Asymmetric missing-log behaviour documented:
  `--pending` returns 0 on missing log, `--show` returns 1.

- **purge**: three-mode envelope shape now matches code (row-mode
  doesn't carry an explicit `success` field — `mode` is the success
  discriminator; run-mode `deleted` is a list, not the row-mode
  string). Violation key renamed `kind` -> `artefact_kind` to match
  `_purge.py:1067-1073`.

- **reverse_pii**: match keys re-aligned (`path`/`preview` ->
  `file`/`snippet`); `files_scanned` is a list of `{path, match_count}`
  objects, not an integer. Exit-code table updated: explicit
  `--audit-dir` unwritable returns EXIT_TRAINING_ERROR (2), not 1
  (`_reverse_pii.py:537,573`).

Usermanuals reference page corrections (EN + TR):

- **data/audit.md**: 11+ ghost flags removed (`--strict`, `--skip-pii`,
  `--skip-secrets`, `--skip-quality`, `--skip-leakage`,
  `--remove-duplicates`, `--remove-cross-split-overlap`,
  `--output-clean`, `--show-leakage`, `--minhash-jaccard`,
  `--minhash-num-perm`, `--dedup-algo`, `--dedup-threshold`,
  `--sample-rate`, `--add-row-ids`); table rebuilt from
  `_parser.py:335-468`. The note about removed flags points operators
  at the canonical alternatives.

- **reference/cli.md**: same ghost-flag scrub on the `forgelm audit`
  synopsis block; full per-flag table delegated to the data/audit
  page so a single source-of-truth.

- **reference/exit-codes.md**: ghost `audit --strict` exit-2 row
  removed; `verify-audit` chain-break behaviour (exit 1 during v0.5.5
  cycle) added; "exit 0 guarantees" closing event renamed
  `run_complete` -> `pipeline.completed` (canonical).

P8 catalog row already added in Wave B (commit 2ba095f).
yaml-templates.md deferred to Wave D where it merges with the P1
parallel-universe-YAML rewrite.

All three doc guards green:
- `tools/check_bilingual_parity.py --strict` (40 EN+TR pairs)
- `tools/check_anchor_resolution.py --strict` (268 markdown files)
- `tools/check_cli_help_consistency.py --strict` (423 forgelm invocations)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wave D Part 1 of the 2026-05-07 docs audit plan. Closes the
parallel-universe YAML schema (P1) on the trainer-spec pages and the
two highest-impact data/concept pages, plus the TR judge.md identifier
revert flagged by the audit.

Trainer pages (sft, dpo, simpo, kto, orpo, grpo — EN + TR ×6):

- Top-level `datasets:[{path, format}]` array → singular `data: {
  dataset_name_or_path: ... }` (matches `ForgeConfig.data:` block).
- `training.trainer` → `training.trainer_type` (canonical literal).
- `training.epochs` / `training.batch_size` →
  `training.num_train_epochs` / `training.per_device_train_batch_size`.
- Nested `training.{dpo, simpo, kto, orpo, grpo}: { ... }` blocks →
  flat `training.{dpo_beta, simpo_beta, simpo_gamma, kto_beta,
  orpo_beta, grpo_num_generations, grpo_max_completion_length,
  grpo_reward_model}` fields (matches `forgelm/config.py:163-218`).
- Top-level `output: { dir: ... }` block → `training.output_dir: ...`
  scalar (no `output:` block exists).
- Removed `model.load_in_4bit: true` ghost (replaced by `model.backend:
  "transformers"|"unsloth"`).
- Removed `training.scheduler: "cosine"` and
  `training.loss_on_completions_only` ghosts (no such fields in
  TrainingConfig; HF TRL handles these defaults internally).
- DPO worked example: `compliance.human_approval` →
  `evaluation.require_human_approval`; `safety.model:` →
  `safety.classifier:`; `safety.block_categories: [list]` →
  `safety.track_categories: bool` + `safety.severity_thresholds:
  {dict}`; `evaluation.benchmark.floors: {per-task dict}` →
  `evaluation.benchmark.min_score: scalar`.
- KTO note added: `desirable_weight` / `undesirable_weight` are TRL-side
  defaults, not surfaced as ForgeLM config fields (Phase 28+ backlog).
- GRPO note added: TRL-side `beta` not surfaced as flat field; built-in
  format/length reward shaping is always-on fallback
  (`forgelm/grpo_rewards.py`).
- KTO dataset format: `prompt + response + label` →
  `prompt + completion + label` (matches `_detect_dataset_format` at
  `forgelm/data.py:13-29`; also fixed in concepts/data-formats EN+TR).

concepts/data-formats.md (EN + TR):
- KTO format auto-detection field: `response` → `completion`.

data/secrets.md (EN + TR):
- 11-category claim → real 9 families (`aws_access_key`,
  `github_token`, `slack_token`, `openai_api_key`, `google_api_key`,
  `jwt`, `openssh_private_key`, `pgp_private_key`,
  `azure_storage_key`). Anthropic / Stripe / SendGrid / Twilio
  references removed (no per-vendor patterns in
  `forgelm/data_audit/_secrets.py:_SECRET_PATTERNS`); operators with
  those traffic types extend out-of-tree.
- Redaction string `[REDACTED-PEM-BLOCK]` claim removed (real:
  `[REDACTED-SECRET]` per `_secrets.py:106`).

evaluation/judge-tr.md:
- Reverted translated rubric identifiers (`yardımseverlik` / `ton` /
  `doğruluk`) to canonical `helpfulness` / `tone` / `factual_accuracy`
  per docs/standards/localization.md (CLI / code / config-key names
  are not translated). Added `rubric_distributions` to the JSON
  example envelope.

Validated: bilingual_parity ✓, anchor_resolution ✓,
cli_help_consistency ✓.

Wave D Part 2 (yaml-templates.md, evaluation pages, config_hash
reframe) and Part 3 (safety_compliance-tr 38-section translate) follow
in subsequent commits.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wave D Part 2 of the 2026-05-07 docs audit plan.

usermanuals/{en,tr}/reference/yaml-templates.md (full rewrite):
- Refocused as a thin layer over the canonical
  `forgelm/templates/<name>/config.yaml` files; readers are pointed at
  `forgelm quickstart <name>` to materialise the source-of-truth.
- Three worked examples in canonical schema (customer-support SFT,
  SFT→DPO chain with Article 14 + safety/benchmark gates, GRPO math).
- Added a "removed templates / fields" table mapping every old ghost
  key to its canonical replacement, to defend against operators
  copying from cached / older versions of the page.

compliance.config_hash reframe (15 user-facing files):
- Replaced `compliance.config_hash` (presented as an audit-event
  payload field) with `config_hash` (per-run manifest sidecar field)
  across docs/{design,guides,qms,reference,usermanuals}/. No event
  emitter currently writes a `config_hash` field on its payload — the
  legacy claim was a doc-only fabrication. The Approvals subcommand
  passively forwards it from upstream events when present
  (`_approvals.py:248`), and the per-run manifest sidecar carries the
  hash in code paths that have it; until a producer is wired Phase
  28+, the doc no longer claims it ships on the wire.

Files touched: yaml-templates.md (EN+TR rewrite); design/iso27001…,
guides/iso_soc2_deployer_guide{,.tr}, qms/{encryption_at_rest{,.tr},
sop_incident_response{,.tr}, statement_of_applicability{,.tr}},
reference/{iso27001_control_mapping{,.tr},soc2_trust_criteria_mapping{,.tr}},
usermanuals/{en,tr}/operations/iso-soc2-deployer (config_hash sed).

Validated: bilingual_parity ✓, anchor_resolution ✓,
cli_help_consistency ✓ (425 forgelm invocations).

Wave D Part 3 (safety_compliance-tr 38-section translate) and the
remaining evaluation pages (benchmark, judge full rewrite, safety,
auto-revert, trend-tracking) still pending.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wave D Part 3 of the 2026-05-07 docs audit plan.

docs/guides/safety_compliance-tr.md is no longer a 38-section stub
("Tam içerik için EN'e bakın"). Translates the full 493-line EN
content into idiomatic Turkish (547 lines after natural Turkish
wrap), preserving:

- Section structure (5 H2 + 33 H3/H4 — bilingual_parity guard verifies
  same spine).
- Code blocks, YAML keys, JSON fields, audit-event names, exit codes
  in English (per docs/standards/localization.md: CLI / code /
  config-key names are not translated).
- Probe-set counts updated to canonical 30 / 25 / 25 (matching
  configs/safety_prompts/{general_safety,bias_discrimination,
  jailbreak_attempts}.jsonl as verified during this audit).
- Llama Guard S1-S14 category labels translated to Turkish
  prose, codes preserved.

The "translate eventually" pointer to the EN file is retired; the TR
mirror now carries the same operational depth as EN.

Validated: bilingual_parity ✓ (40 pairs, full content match on
heading spine), anchor_resolution ✓ (268 files), cli_help_consistency
✓ (427 forgelm invocations).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wave E of the 2026-05-07 docs audit plan. Closes the post-audit
"prevention layer" — five new tools/check_*.py guards that catch every
drift family the audit identified, before it ships. Each guard mirrors
an audit finding pattern and is wired to be added to ci.yml alongside
the existing bilingual_parity / anchor_resolution / cli_help_consistency
gauntlet.

New guards (all support `--strict --quiet`):

1. **`tools/check_yaml_snippets.py`** — extracts every fenced `yaml`
   block from `docs/`, validates the complete (model+training+data)
   ones against `forgelm.config.ForgeConfig`. Skips fragments and
   intentional sketches (`...` placeholders). Catches P1 — the
   parallel-universe YAML schema. Today: 20 ForgeConfig-shaped
   snippets validated across 333 total yaml blocks.

2. **`tools/check_audit_event_catalog.py`** — scans forgelm/ for
   dotted-namespace event literals (catches both direct
   `event="foo.bar"` and constant-indirection
   `_EVT_FOO = "foo.bar"`), diffs against
   `docs/reference/audit_event_catalog.md` rows in both directions.
   Filters filename-shaped matches via
   `_NON_EVENT_SECOND_SEGMENTS`. Catches P8 (missing-from-catalog)
   AND ghost rows (`audit.cross_run_continuity`). Today: 36
   emitted ↔ 36 catalogued.

3. **`tools/check_library_api_doc.py`** — diffs `forgelm.__all__`
   against `forgelm.X` rows in `library_api_reference.md`.
   Catches the audit's `compute_minhash` missing-row finding.
   Method-style rows (`forgelm.ForgeTrainer.train`) are checked
   against the parent class. Today: 30 ↔ 30 + 2 method rows.

4. **`tools/check_doc_numerical_claims.py`** — scrapes four canonical
   counts from code (secret families, trainer types, templates,
   webhook events) and diffs against doc claims via qualifier-anchored
   regexes ("9 secret families", "5 webhook events", …). Patterns
   tightened to require qualifier so generic numbers don't false-
   positive (e.g. "Six events" without "webhook" qualifier doesn't
   match the webhook rule). Today: 9 / 6 / 5 / 5.

5. **`tools/check_bilingual_code_blocks.py`** — extends the existing
   `check_bilingual_parity.py` (which only checks H2/H3/H4 heading
   spine) to:
   - Same fenced-block count per pair.
   - For yaml blocks at the same ordinal position, same set of
     top-level keys.
   Catches the audit's S3 pattern (TR collapsing multi-line YAML to
   flow-style) and the translated-identifier pattern (S2). Today:
   40 pairs, all matching.

Doc fixes folded in (caught by the new guards on first run):

- `library_api_reference.md` (EN+TR): added missing
  `forgelm.compute_minhash` row.
- `audit_event_catalog.md` (EN+TR): added 5 missing rows
  (`pipeline.initialized`, `training.oom_recovery`,
  `benchmark.evaluation_completed`, `safety.evaluation_completed`,
  `judge.evaluation_completed`); removed ghost
  `audit.cross_run_continuity` row.
- `usermanuals/{en,tr}/training/{simpo,kto,orpo,grpo}.md`: added
  back the `lora:` block I'd dropped in Wave D (regression caught
  by gate 1 — the YAML snippets failed validation against the
  required `lora` field in ForgeConfig).

All 8 guards (3 pre-existing + 5 new) pass `--strict`:

- `check_bilingual_parity.py` ✓ (40 pairs, H2/H3/H4 spine)
- `check_anchor_resolution.py` ✓ (268 markdown files)
- `check_cli_help_consistency.py` ✓ (427 forgelm invocations)
- `check_yaml_snippets.py` ✓ (20 ForgeConfig snippets validated)
- `check_audit_event_catalog.py` ✓ (36/36 events)
- `check_library_api_doc.py` ✓ (30/30 symbols)
- `check_doc_numerical_claims.py` ✓ (9/6/5/5 counts)
- `check_bilingual_code_blocks.py` ✓ (40 pairs, code-blocks + yaml keys)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wave F of the 2026-05-07 docs audit plan. Closes the residual R1-R33
inventory after Wave A-E left ≈33 unaddressed items, and the user-
identified gap of 3 missing TR mirrors (cicd_pipeline, quickstart,
troubleshooting).

48 files changed: 43 modified + 5 new TR mirrors. All 8 doc guards
green (3 pre-existing + 5 Wave-E gates).

Group A — mechanical sed/text fixes:
- pii-ml.md (EN+TR) line 126 v0.5.0 → v0.5.5 anchor.
- CHANGELOG.md "8 new pages" / "16 new bilingual" → "9 new pages" /
  "17 new bilingual" (matches the actual _PAIRS registry post-Wave-F).
- ingestion.md "≈10 prefix-anchored patterns" → "9".
- safety_compliance.md (EN) probe count `general_safety: 20 / bias: 15
  / jailbreak: 15` → canonical 30 / 25 / 25 (matches
  configs/safety_prompts/*.jsonl wc -l verified during audit).
- quickstart.md `forgelm --data-audit ...` → canonical `forgelm audit
  ...` subcommand.

Group B — existing-doc edits (R1, R4, R5, R7-R15, R18-R22):
- architecture.md (EN+TR): flat-module diagram + directory tree
  rewritten for Phase 14/15 split (cli/ + data_audit/ packages); 21
  Pydantic models (was 19); ~70 test modules (was 800+); 18 subcommand
  list replaces the ghost "13 CLI flags" claim.
- library_api_reference.md (EN+TR) Example #3 unrunnable: model.name
  → name_or_path; top-level dataset → data; num_epochs →
  num_train_epochs; batch_size → per_device_train_batch_size; added
  required `lora:` block. Added explanatory note on which keys are
  Pydantic-required.
- guides/library_api.md (EN+TR): notify_training_start →
  notify_start; report.duplicate_count → report.near_duplicate_summary
  ["pairs"]; report.pii_findings → report.pii_summary; config.dataset.path
  → config.data.dataset_name_or_path; config.output_dir →
  config.training.output_dir.
- configuration.md (EN+TR): `rope_scaling` type fixed (string/2-value
  → Optional[Dict]/4-value: linear, dynamic, yarn, longrope);
  `webhook.timeout` default 5 → 10 with v0.5.5 raise rationale.
- performance.md (EN+TR) top-level `ingestion:` YAML block (no such
  key in ForgeConfig) replaced with the canonical CLI flag form.
- design/blackwell_optimized.md: "Status: Not yet implemented (Phase
  28+ backlog)" banner so the doc no longer reads as fait accompli.
- design/wizard_mode.md: rewritten to reflect the shipped 8-step flow
  (was 6-step sketch); pure-stdlib `input()` (was "rich/questionary");
  documented the 0.5.5 implementation as source of truth.
- design/gdpr_erasure.md: §3.2 RetentionConfig 4 → 5 fields (added
  `raw_documents_retention_days`); §4.2 `--yes` flag re-framed as
  Phase 28+ backlog (the shipped purge is non-interactive); §279
  api.py → _version.py.
- design/iso27001_soc2_alignment.md: §13 Faz 23 acceptance checkboxes
  marked complete (`[x]` × 30; previously stale `[ ]` despite §17
  saying "Implemented in v0.5.5"); A.8.29 "1300+ tests" → "~70 test
  modules".
- design/library_api.md: `forgelm/api.py` references → `forgelm/_version.py`
  (the implementation chose the latter; design doc was stale).
- distributed_training.md (EN+TR): `zero3_offload` "CPU only" → "optimizer
  state + parameters → CPU"; `fsdp_offload` "Offload parameters" → "Offload
  parameters AND gradients (between forward/backward)".
- iso27001_control_mapping-tr.md: section headers translated to Turkish
  (A.5 Organizasyonel kontroller, A.6 İnsan kaynağı, A.7 Fiziksel,
  A.8 Teknolojik) — was English in TR file.
- supply_chain_security.md (EN+TR): `gh release download v0.5.5` →
  `v<X.Y.Z>` placeholder so the example doesn't drift on every release.
- qms/risk_treatment_plan.md (EN+TR) R-12 row F-compliance-110
  misattribution: clarified that F-compliance-110 enforces
  evaluation.safety.enabled (not require_human_approval — see R-04).
- guides/gdpr_erasure.md (EN+TR): row-mode-only qualifier added to
  `data.erasure_warning_memorisation` and `..._synthetic_data_present`
  rows; `..._external_copies` already covers both modes.
- reference/data_preparation.md (EN+TR): added scope-pointer block
  redirecting operators to the modern pipeline (ingestion guide,
  audit guide, dataset-formats reference, prepare_dataset Python API).

Group C — new TR mirrors (R2, R3 of the plan):
- compliance_summary-tr.md (NEW) — 207 lines, full mirror of EN
  compliance_summary; ISO control count corrected to "59 of 93
  (FL 11 + FL-helps 48)" matching the canonical mapping.
- alignment-tr.md (NEW) — 240 lines, full mirror of EN alignment.

Group D — usermanuals operations + getting-started ghost cleanup:
- operations/experiment-tracking.md (EN+TR): `report_to` is a single
  Literal, not a list; per-backend nested config blocks
  (`wandb: { project: ... }`, `mlflow: { tracking_uri: ... }`,
  `tensorboard: { log_dir: ... }`, `comet_ml: { api_key: ... }`) all
  removed — backends configured via env vars per HF Transformers
  convention; `training.tags` and `log_grad_norm` ghost fields removed.
- operations/vram-fit-check.md (EN+TR): `--gpus N` ghost flag removed
  (estimator probes nvidia-smi); `--fit-check-strict` ghost flag
  reframed as calibration-training pattern.
- operations/gpu-cost.md (EN+TR): `--estimate-cost` ghost flag
  reframed as calibration-then-extrapolate pattern using the per-run
  compliance_report.json `gpu_hours`.
- getting-started/project-layout.md (EN+TR): `extends:` YAML
  inheritance ghost (no such ForgeConfig field) replaced with YAML
  anchor (`&base` / `<<: *base`) recommendation.

Group E — user-identified missing TR mirrors:
- cicd_pipeline-tr.md (NEW) — 240 lines, full TR mirror of EN
  cicd_pipeline; webhook payload "training.awaiting_approval" ghost
  fixed to canonical `approval.required` in BOTH EN and TR (P7).
- quickstart-tr.md (NEW) — 290 lines, full TR mirror; synthetic ghost
  fields (`teacher_api_key_env`, `num_samples`) replaced with canonical
  `api_key_env` + `output_format` in BOTH EN and TR (P6).
- troubleshooting-tr.md (NEW) — 360 lines, full TR mirror; synthetic
  ghost timeout/retry/batch fields fixed to canonical `api_timeout`,
  `api_delay`, `max_new_tokens` in BOTH EN and TR (P6).

`tools/check_bilingual_parity.py::_PAIRS` grew from 40 → 45 (+5 new
mirrors registered).

CLI consistency caught a regression my Wave D wizard_mode.md rewrite
introduced: `forgelm wizard` is not a registered subcommand — the
wizard is invoked via the top-level `--wizard` flag. Fixed in
wizard_mode.md, alignment-tr.md, quickstart-tr.md.

All 8 doc guards green:
- check_bilingual_parity --strict       ✓ 45 pairs
- check_anchor_resolution --strict      ✓ 273 markdown files
- check_cli_help_consistency --strict   ✓ 439 forgelm invocations
- check_yaml_snippets --strict          ✓ 23 ForgeConfig snippets
- check_audit_event_catalog --strict    ✓ 36 ↔ 36 events
- check_library_api_doc --strict        ✓ 30 ↔ 30 symbols
- check_doc_numerical_claims --strict   ✓ 9/6/5/5 counts
- check_bilingual_code_blocks --strict  ✓ 45 pairs

Items legitimately deferred (intentional, NOT in this commit):
- P9 false positive (audit hallucination — 140×6 corpus is correct).
- evaluation.approval_retention_days false positive (audit hallucination).
- --add-row-ids flag implementation (Phase 28+).
- HMAC body sign (Phase 28+).
- encryption_at_rest config field (Phase 28+).
- Anthropic / Stripe / SendGrid / Twilio secret patterns (Phase 28+).
- KTO desirable_weight / undesirable_weight surface (Phase 28+).
- GRPO beta flat field (Phase 28+).
- compliance.config_hash audit-event payload emission (Phase 28+).
- usermanuals P1-residual: full rewrite of training/{lora, galore,
  distributed, long-context}, evaluation/{benchmark, safety,
  auto-revert, trend-tracking}, deployment/* (~30 pages) — deferred
  to render-from-code pipeline (v0.6.x).
- Codacy 100 issues (web-UI access required).
- Sonar 8.8% duplication (intentional — marketing site chrome).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@qodo-code-review

Copy link
Copy Markdown
ⓘ You've reached your Qodo monthly free-tier limit. Reviews pause until next month — upgrade your plan to continue now, or link your paid account if you already have one.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @cemililik, your pull request is larger than the review limit of 150000 diff characters

@codacy-production

codacy-production Bot commented May 7, 2026 •

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 100 minor

Alerts:
⚠ 100 issues (≤ 0 issues of at least minor severity)

Results:
100 new issues

Category Results
CodeStyle 100 minor

View in Codacy

🟢 Metrics 160 complexity · 2 duplication

Metric Results
Complexity 160
Duplication 2

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@coderabbitai

coderabbitai Bot commented May 7, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Rate limit exceeded

@cemililik has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 42 minutes and 54 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7935d5ac-e515-4990-9b36-137139d453f4

📥 Commits

Reviewing files that changed from the base of the PR and between b275e18 and 755eaf7.

📒 Files selected for processing (10)
  • docs/guides/safety_compliance-tr.md
  • docs/guides/safety_compliance.md
  • docs/guides/troubleshooting-tr.md
  • docs/guides/troubleshooting.md
  • docs/usermanuals/en/data/audit.md
  • docs/usermanuals/en/data/secrets.md
  • docs/usermanuals/en/operations/experiment-tracking.md
  • docs/usermanuals/tr/compliance/human-oversight.md
  • docs/usermanuals/tr/data/secrets.md
  • docs/usermanuals/tr/operations/experiment-tracking.md
📝 Walkthrough

Walkthrough

Bulk v0.5.5 update: EN/TR documentation overhaul, standardized audit-event names and manifest naming, revised CLI JSON envelopes and exit codes, added docs-check tooling, and small site/CSS/metadata edits.

Changes

v0.5.5 Documentation and Tooling Sweep

Layer / File(s) Summary
All edits
docs/**, docs/usermanuals/**, docs/reference/**
Wide-ranging content updates and additions: quickstart, guides, safety/judge, GDPR purge/reverse-pii, CI/CD, training guides, ISO/SOC2/QMS mappings, and many Turkish translations.
CLI / Reference
docs/reference/*_subcommand*.md, README.md
Revised CLI examples and JSON stdout envelopes for purge, approve(s), reverse-pii, verify-audit, safety-eval; README CLI usage updated (--row-id --corpus).
Audit / Compliance
docs/reference/audit_event_catalog.md, docs/design/gdpr_erasure.md, docs/qms/*
Standardized event names (e.g., model.reverted, pipeline.completed), renamed manifest to audit_log.jsonl.manifest.json, aligned config_hash usage and verify-audit exit semantics.
Tools / Checks
tools/check_*.py
Added/updated tools: audit-event catalog checker, bilingual code-block checker, YAML-snippet validator, numerical-claims checker, library-API doc guard; expanded bilingual _PAIRS registry.
Site & metadata
site/*, CHANGELOG.md, CONTRIBUTING.md, CLAUDE.md, site/css/style.css
Minor site text and CSS casing fixes, CHANGELOG counts adjusted, CONTRIBUTING/CLAUDE test counts updated.
New Turkish guides
docs/guides/*-tr.md, docs/usermanuals/tr/**
Multiple TR guide additions/expansions (quickstart, troubleshooting, CI/CD, compliance translations).

Estimated code review effort
🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

"A rabbit flips through pages new,
Aligns the logs, the webhooks too.
With tools that sniff out drift and doubt,
It tidies docs inside and out.
Then thumps the ground, “v0.5.5!” 🥕"

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch development

CI's `ruff check .` step caught 5 lint findings on the Wave E CI
guard tools (commit 0c4692c):

- I001 (un-sorted imports) on tools/check_audit_event_catalog.py,
  tools/check_doc_numerical_claims.py, tools/check_library_api_doc.py,
  tools/check_yaml_snippets.py.
- F541 (f-string without placeholders) on
  tools/check_audit_event_catalog.py:189.

`ruff check . --fix` resolves all five; `ruff format` then applied
consistent formatting to 4 of the new tools (line-wrap + trailing
comma normalisation).

All 8 doc guards remain green after the format pass:
- check_yaml_snippets        ✓ 23/357
- check_audit_event_catalog  ✓ 36/36
- check_library_api_doc      ✓ 30/30
- check_doc_numerical_claims ✓ 9/6/5/5
- check_bilingual_code_blocks ✓ 45 pairs

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a comprehensive documentation overhaul for ForgeLM v0.5.5, focusing on bilingual parity and technical synchronization. Key changes include the translation of several core guides into Turkish, the alignment of event names and configuration schemas with the current implementation, and the introduction of a suite of CI tools designed to automate the detection of documentation drift. The reviewer pointed out that hardcoding line numbers in the reference documentation is a brittle practice that leads to stale information, recommending that these be replaced with symbolic references to functions or classes to improve maintainability.

Comment thread docs/reference/reverse_pii_subcommand.md
Comment thread docs/reference/verify_audit.md Outdated
Comment thread docs/reference/reverse_pii_subcommand-tr.md
Comment thread docs/reference/verify_audit-tr.md Outdated
Absorbs gemini-code-assist's 4 inline review comments on PR #39 (same
finding × 4 files): hardcoded line numbers in subcommand reference docs
are brittle and rot when code evolves. Replaces them with function /
symbol-name references — the convention compliance_summary.md's Wave 4
cleanup formalised ("each link points at the file root, not a line
anchor; the auditor opens the file and greps the cited symbol name").

Replacements (10 docs, EN+TR pairs ×5 subcommands):

| Was | Now |
|---|---|
| `_verify_audit.py:41,45,56,63` | `_run_verify_audit_cmd` |
| `_verify_audit.py:18-27` (docstring) | `_run_verify_audit_cmd` docstring |
| `_purge.py:609-617` | `_perform_row_erasure_and_audit` |
| `_purge.py:805-811` | `_run_purge_run_id` |
| `_purge.py:993` | `_run_purge_check_policy` |
| `_purge.py:101` | `_output_error_and_exit` |
| `_reverse_pii.py:537,573` | `_maybe_audit_logger` |
| `_reverse_pii.py:769-777` | `_run_reverse_pii_cmd` |
| `_reverse_pii.py:107` | `_output_error_and_exit` |
| `_approve.py:468-477` | `_run_approve_cmd` |
| `_approve.py:542-551` | `_run_reject_cmd` |
| `_approvals.py:240-249` | `_summarise_pending` |
| `_approvals.py:440-444` | `_emit_show_json` |
| `_approvals.py:319-327` | `_run_approvals_list_pending` |
| `_approvals.py:458-463` | `_run_approvals_show` |

Symbol-name refs survive code refactors (rename detection makes them
self-correcting); line numbers don't.

Files: verify_audit.md, reverse_pii_subcommand.md, purge_subcommand.md,
approve_subcommand.md, approvals_subcommand.md (each EN + TR).

Validated: bilingual_parity ✓ 45 pairs, anchor_resolution ✓ 273 files,
cli_help_consistency ✓ 439 invocations, ruff check ✓ all passed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cemililik

Copy link
Copy Markdown
Collaborator Author

Review absorption — PR #39

Snapshot of all 8 review comments and bot reports:

Inline reviews (4)

All 4 are the same gemini-code-assist finding repeated for EN + TR pairs of verify_audit.md and reverse_pii_subcommand.md: hardcoded line numbers (module.py:N) are brittle and drift when code evolves. Fixed in 694bf12 — converted line refs to symbol/function-name refs across all affected subcommand docs (verify_audit, reverse_pii_subcommand, purge_subcommand, approve_subcommand, approvals_subcommand × EN/TR = 10 files).

The convention is the one compliance_summary.md's Wave 4 cleanup formalised: "each link points at the file root, not a line anchor; the auditor opens the file and greps the cited symbol name; this survives refactors that the prior #L33 form did not." Symbol-name refs are self-correcting under rename detection.

Bot reports (4)

Bot Status
sonarqubecloud ✅ Quality Gate PASSED (9 new issues, informational; PR #38 had 8.8% duplication failure — that's gone)
codacy-production ⚠️ 100 issues (1 medium + 99 minor) — BestPractice + CodeStyle category. PR #38 had 100 high-severity; this is a major improvement. The minor ones are likely doc-style nits not caught by ruff. Cannot fetch the per-issue list via gh api (Codacy uses its own webhook, not the GitHub Issues API); will inspect in the Codacy web UI as a follow-up pass.
qodo-code-review Free-tier rate-limited (informational)
coderabbitai Out of credits / rate-limited (informational)

Closure

All actionable feedback (the 4 gemini comments) is now in. Bot reports have either passed (sonar) or are informational/rate-limited (qodo, coderabbit). Codacy 100 minor issues will be inspected separately — they don't block the merge gate.

Sweep verified findings from docs/analysis/code_reviews/2026-05-08_full_docs_audit.md
(7 sub-auditors, ~253 findings, 27 CRITICAL — all verified zero false positives
against current code state).

Reference docs (configuration / library_api / audit_event_catalog / approvals /
iso27001 / soc2 / compliance_summary / architecture / usage / human_approval_gate
+ TR mirrors):

  - configuration.md: galore_optim default 8bit→adamw; galore_target_modules
    default list→null (regex fallback); synthetic block fully rewritten from
    SyntheticConfig (api_base/api_key_env/max_new_tokens/api_delay/api_timeout
    flat fields, output_format Literal messages|instruction|chatml|prompt_response).
  - usage.md / usage-tr.md: rope_scaling bare-string → dict form
    {type, factor} per Optional[Dict[str,Any]] schema.
  - audit_event_catalog: 8 event payload field-name corrections; audit_log.jsonl
    path corrected from compliance/ → top-level.
  - library_api_reference: 14 signature/dataclass corrections (TrainResult 18-field,
    AuditReport canonical fields, VerifyResult, VerifyAnnexIVResult, VerifyGgufResult,
    run_benchmark, BenchmarkResult, setup_authentication, manage_checkpoints,
    prepare_dataset, compute_minhash, detect_pii/secrets returns).
  - library_api guide: result.output_dir/run_id → config.training.output_dir;
    annex-iv-bundle.zip → annex_iv_metadata.json.
  - human_approval_gate: compliance.human_approval → evaluation.require_human_approval.
  - approvals_subcommand: config_hash documented as forward-compat read with null
    surface (no producer emits it currently).
  - iso27001 / soc2: config_hash phantom evidence replaced with real evidence
    (pipeline.training_started SHAs); 1370+ tests → 1493; webhook HMAC clarified.
  - compliance_summary: verify-audit exits 0/1/2/3 → 0|1; manifest filename
    audit_log.manifest.json → audit_log.jsonl.manifest.json; 140-prompt corpus
    line replaced with canonical 51×18 list.
  - architecture: subcommand inline list 17→18 (added reject).

QMS (4 CRITICAL phantom claims — EN+TR):

  - risk_treatment_plan: forgelm audit --secrets removed (no such flag — secrets
    scan is always-on); FORGELM_AUDIT_SECRET-signed webhook payloads removed
    (forgelm/webhook.py posts unsigned curated JSON); R-12 see-R-04 → see-R-06;
    trainer ≠ approver clarified as record (not enforce).
  - sop_incident_response: .sha256 sidecar removed (only .manifest.json exists);
    --secrets phantom flag removed.
  - access_control: webhook.secret_env phantom field removed (no such field on
    WebhookConfig); webhook event vocabulary clarified (5 wire events vs 5
    notifier method names); audit_log.manifest.json → audit_log.jsonl.manifest.json.
  - encryption_at_rest: .sha256 sidecar removed.
  - statement_of_applicability: 1370+→1493; webhook HMAC qualified
    (audit-log chain HMAC ≠ webhook body HMAC); A.5.3 record-not-enforce.

Usermanual deep cleanup (EN+TR):

  - evaluation/auto-revert: phantom auto_revert sub-block + last_good_checkpoint
    + notify_on_revert + guards.<name> registry → real flat schema (auto_revert: bool,
    benchmark/safety/llm_judge sub-blocks).
  - evaluation/safety: phantom block_categories / severity_threshold / model
    / regression_tolerance → real classifier / severity_thresholds dict /
    track_categories / max_safety_regression / min_safety_score; default probe
    set 50×~14 → 51×18.
  - evaluation/judge: full rewrite from JudgeConfig — block name judge → llm_judge,
    no pairwise/ELO/multi-rubric/budget_usd/rate_limit (all phantom).
  - data/synthetic-data: phantom synthetic.teacher.{provider,model,api_key} block
    → flat teacher_model/teacher_backend/api_base/api_key_env; phantom budget_usd
    + rate_limit removed.
  - compliance/overview: phantom compliance.annex_iv: true / human_approval /
    deployment_geographies / responsible_party / data_audit_artifact removed
    (none exist on ComplianceMetadataConfig); model_card.md → README.md;
    conformity_declaration.md not produced clarified.
  - operations/webhooks: phantom output.webhook block + template/events
    arrays removed (WebhookConfig has no template/events/headers/retries
    /channel/mention_on_failure fields); --webhook-test flag never existed.

Site:

  - site/compliance.html line 287: model_card.md → README.md (HuggingFace-compatible).

Validation: all 8 doc CI guards green (bilingual_parity, anchor_resolution,
cli_help_consistency, yaml_snippets, audit_event_catalog, library_api_doc,
doc_numerical_claims, bilingual_code_blocks); ruff check clean; usermanuals
regenerated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Note

Due to the large number of review comments, Critical severity comments were prioritized as inline comments.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (12)
docs/usermanuals/tr/operations/cicd.md (1)

75-99: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

deploy aşaması tanımlanmış ama örnekte deploy job’ı yok

Line 78’de deploy stage’i eklenmiş, ancak aşağıda buna karşılık gelen bir deploy: job bloğu yok. Kopyala-yapıştır yapan kullanıcılar için örnek eksik/yanıltıcı kalıyor. Ya deploy stage’ini kaldırın ya da minimal bir deploy job ekleyin.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/operations/cicd.md` around lines 75 - 99, The CI config
declares a deploy stage in the stages list but does not define a corresponding
deploy job, which is confusing for users; either remove "deploy" from the stages
array or add a minimal "deploy:" job block that references stage: deploy (e.g.,
a simple image and a script that runs a harmless deploy/echo step and preserves
artifacts if needed). Update the file so the "stages:" list and the "deploy:"
job (or the removal of "deploy" from stages) are consistent.
docs/reference/distributed_training-tr.md (1)

131-131: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Missing bilingual mirror update for FSDP offload clarification.

The English version (distributed_training.md:131) added a clarifying comment about FSDP offload behavior, but the Turkish file is missing the equivalent update. Line 131 should include a Turkish translation of the comment explaining that fsdp_offload offloads parameters AND gradients to CPU between forward/backward passes.

🌍 Proposed fix to restore bilingual parity
-  fsdp_offload: false
+  fsdp_offload: false           # Parametreleri VE gradyanları (forward/backward arası) CPU'ya taşı
+  fsdp_backward_prefetch: "backward_pre"  # Prefetch stratejisi
+  fsdp_state_dict_type: "FULL_STATE_DICT" # State dict yönetimi

As per coding guidelines, bilingual docs (EN ↔ TR mirrors) must maintain parity for user-facing documentation.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/reference/distributed_training-tr.md` at line 131, The Turkish doc is
missing the bilingual clarification about FSDP offload: add a Turkish comment
next to the fsdp_strategy: "full_shard" entry that mirrors the English note
about fsdp_offload, explicitly stating that fsdp_offload moves both model
parameters AND gradients to CPU between forward and backward passes; locate the
block containing fsdp_strategy and insert a concise Turkish sentence that
conveys “fsdp_offload offloads parameters AND gradients to CPU between
forward/backward passes” using the term fsdp_offload so the link to the original
comment is clear.
docs/usermanuals/en/evaluation/auto-revert.md (1)

47-47: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Inconsistent documentation: guards registry.

Line 47 lists "Custom guard fails" as a trigger with configuration via evaluation.guards.<name>, but lines 105-109 explicitly state "There is no evaluation.guards.<name>: plug-in registry." Remove the table row at line 47 or clarify that custom guards are implemented outside the config schema.

🔧 Suggested fix

Remove the inconsistent table row:

 | Final loss > starting loss | Always | not configurable |
 | Final loss is NaN/Inf | Always | not configurable |
-| Custom guard fails | User-supplied callable | `evaluation.guards.<name>` |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/evaluation/auto-revert.md` at line 47, The table row that
lists "Custom guard fails" with configuration key `evaluation.guards.<name>` is
inconsistent with the later statement that "There is no
`evaluation.guards.<name>:` plug-in registry"; either remove that table row or
change its text to clarify that custom guards are not configured via the
`evaluation` config schema but are invoked as user-supplied callables
implemented outside the config registry—update the row for "Custom guard fails"
or the surrounding explanation so it references "user-supplied callable" (not
`evaluation.guards.<name>`) and points readers to the external implementation
approach.
docs/usermanuals/en/compliance/human-oversight.md (1)

33-42: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Stale YAML config block contradicts the updated description.

Line 8 now documents evaluation.require_human_approval: true as the canonical activation key, but the Configuration section still shows the old compliance.human_approval: true. A user following this page will copy the wrong key.

The approval.request_webhook field also appears stale — human_approval_gate-tr.md §3 documents the webhook as webhook.url_env, not approval.request_webhook.

🐛 Proposed minimum fix — align YAML block with canonical key
 ```yaml
-compliance:
-  human_approval: true
-  approval:
-    request_webhook: "${SLACK_WEBHOOK}"      # optional notification
-    signature_method: "cli"                   # cli | webhook | api
-    timeout_hours: 48                         # auto-fail after this
-    require_role: "ml-compliance-lead"        # who can approve
-    quorum: 1                                 # required approvers
+evaluation:
+  require_human_approval: true
+
+# optional: approval notifications and policy
+webhook:
+  url_env: "SLACK_WEBHOOK"                    # fires notify_awaiting_approval
+approval:
+  signature_method: "cli"                     # cli | webhook | api
+  timeout_hours: 48                           # auto-fail after this (v0.6.0+)
+  require_role: "ml-compliance-lead"          # who can approve
+  quorum: 1                                   # required approvers
</details>

Please also verify the sub-option canonical paths (`require_role`, `quorum`, `timeout_hours`) against `forgelm/config.py` — this PR's AI summary does not mention their canonical parent key.

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @docs/usermanuals/en/compliance/human-oversight.md around lines 33 - 42,
Replace the stale top-level key compliance.human_approval with
evaluation.require_human_approval and move webhook/approval fields to their
canonical parents: use webhook.url_env (env name string) for the webhook, and
approval.signature_method, approval.timeout_hours, approval.require_role and
approval.quorum under approval; update any inline comments (e.g., timeout_hours
v0.6.0+) to match the new layout and names shown in the proposed diff, and then
verify each sub-option canonical path (require_role, quorum, timeout_hours)
against the definitions in forgelm/config.py to ensure parent keys and names
match the code.


</details>

</blockquote></details>
<details>
<summary>docs/guides/ingestion.md (1)</summary><blockquote>

`117-135`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_

**Prose enumeration (8 items) doesn't match the updated count (9 patterns) — add `PGP private-key` as a separate entry.**

The inline list at lines 117–121 groups `OpenSSH/RSA/DSA/EC/PGP private-key block headers` as a single item (8 total), but line 135 now says "9 prefix-anchored patterns". Per the canonical catalog in `usage.md` (`openssh_private_key` and `pgp_private_key` are separate families), the count of 9 is correct — but a reader reconciling the prose enumeration against the stated count will come up short by one.

<details>
<summary>📝 Proposed fix</summary>

```diff
-Detected categories: AWS access
-keys (`AKIA…`), GitHub PATs (`ghp_…` and friends), Slack tokens, OpenAI
-API keys (`sk-…` / `sk-proj-…`), Google API keys (`AIza…`), JWTs,
-OpenSSH/RSA/DSA/EC/PGP private-key block headers, Azure storage
-connection strings.
+Detected categories: AWS access
+keys (`AKIA…`), GitHub PATs (`ghp_…` and friends), Slack tokens, OpenAI
+API keys (`sk-…` / `sk-proj-…`), Google API keys (`AIza…`), JWTs,
+OpenSSH/RSA/DSA/EC/ECDSA private-key block headers, PGP private-key
+block headers, Azure storage connection strings.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/guides/ingestion.md` around lines 117 - 135, Update the prose
enumeration of detected secret categories so it matches the stated count of 9
patterns: split the combined "OpenSSH/RSA/DSA/EC/PGP private-key block headers"
entry into two distinct items (one for OpenSSH/RSA/DSA/EC private-key block
headers and a second for PGP private-key block headers), and ensure the text
around ingest_path(), forgelm.data_audit.mask_secrets, and the reference to
data_audit.md reflects the two separate families (see usage.md:
openssh_private_key and pgp_private_key) so the list and the "9 prefix-anchored
patterns" statement are consistent.
docs/usermanuals/tr/training/simpo.md (1)

39-61: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Parameters table still uses old beta/gamma key names — update to match the new simpo_beta/simpo_gamma YAML keys.

The YAML example (lines 43–44) now uses simpo_beta and simpo_gamma, but the "Parametreler" table at lines 55–61 documents beta and gamma as the backtick-quoted config keys, and the ## \beta` ve `gamma`section (line 72) and its combination table (lines 73–78) also use the old names. A user reading the table will usetraining.beta/training.gamma`, which are the wrong field names.

📝 Proposed fix for the parameters table (lines 55–61)
 | Parametre | Tip | Vars. | Açıklama |
 |---|---|---|---|
-| `beta` | float | `2.0` | Uzunluk-normalize ödül ölçeği. Yüksek = güçlü tercih kayması. **DPO beta ile aynı ölçek değil.** |
-| `gamma` | float | `1.0` | Margin — chosen ve rejected log-likelihood arasında SimPO'nun korumaya çalıştığı boşluk. |
+| `simpo_beta` | float | `2.0` | Uzunluk-normalize ödül ölçeği. Yüksek = güçlü tercih kayması. **DPO beta ile aynı ölçek değil.** |
+| `simpo_gamma` | float | `1.0` | Margin — chosen ve rejected log-likelihood arasında SimPO'nun korumaya çalıştığı boşluk. |

The ## \beta` ve `gamma`section heading (line 72) and its combination table (lines 73–78) should likewise be updated tosimpo_beta/simpo_gamma`.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/training/simpo.md` around lines 39 - 61, The parameters
docs still reference old keys `beta`/`gamma`; update all occurrences to the new
YAML keys `simpo_beta` and `simpo_gamma` so they match the example block.
Specifically, change the table rows that currently show `beta` and `gamma` to
`simpo_beta` and `simpo_gamma`, update the heading “`beta` ve `gamma`” to
“`simpo_beta` ve `simpo_gamma`”, and update any related combination/example
tables or text that mention the old names so all config key names use
`simpo_beta`/`simpo_gamma`.
docs/usermanuals/en/training/orpo.md (1)

56-58: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Configuration table doesn't match the updated YAML example.

The "Quick example" at lines 30-46 uses the new flat-field schema (orpo_beta under training), but the "Configuration parameters" table at lines 56-58 still lists the old nested field names (beta, loss_type, sft_weight) without the training.orpo_ or orpo_ prefix.

Update the table's "Parameter" column to match the actual config keys (training.orpo_beta, training.orpo_loss_type, training.orpo_sft_weight) or prefix the table with a note that these are shorthand names and the full keys are training.orpo_*.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/training/orpo.md` around lines 56 - 58, The parameters
table currently lists shorthand names (beta, loss_type, sft_weight) that no
longer match the YAML example; update the "Parameter" column to the actual
config keys used in the example (training.orpo_beta, training.orpo_loss_type,
training.orpo_sft_weight) OR add a single-line note above the table stating
these are shorthand names and the full keys are training.orpo_* so readers can
map beta → training.orpo_beta, loss_type → training.orpo_loss_type, and
sft_weight → training.orpo_sft_weight.
docs/usermanuals/en/training/simpo.md (1)

56-60: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Configuration table doesn't match the updated YAML example.

The "Quick example" at lines 29-46 uses the new flat-field schema (simpo_beta, simpo_gamma under training), but the "Configuration parameters" table at lines 56-60 still lists the old nested field names (beta, gamma, loss_type) without the training.simpo_ or simpo_ prefix.

Update the table's "Parameter" column to match the actual config keys (training.simpo_beta, training.simpo_gamma, training.simpo_loss_type) or prefix the table with a note that these are shorthand names and the full keys are training.simpo_*.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/training/simpo.md` around lines 56 - 60, The
configuration table uses legacy shorthand keys (beta, gamma, loss_type,
length_normalize, label_smoothing) but the actual YAML example uses flat keys
under training (e.g., training.simpo_beta, training.simpo_gamma,
training.simpo_loss_type); update the "Parameter" column to the full keys
(training.simpo_beta, training.simpo_gamma, training.simpo_loss_type,
training.simpo_length_normalize, training.simpo_label_smoothing) or add a
single-line note above the table stating these are shorthand and the real config
keys are training.simpo_* so the table matches the quick example; locate and
edit the table rows that currently list `beta`, `gamma`, `loss_type`,
`length_normalize`, and `label_smoothing`.
docs/usermanuals/tr/data/audit.md (1)

142-145: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Remove --strict from CI YAML example—it's documented as removed in line 110.

Line 110 explicitly states --strict "hiç ship olmadı" (never shipped) and does not exist in the parser. The CI example at line 144 still calls forgelm audit data/train.jsonl --strict, which will fail for any user copying it. The same contradiction appears in the EN mirror (line 150).

The page already directs readers to use --output-format json with a jq-based gate for audit-as-gate behavior. Replace the example with the recommended pattern:

Proposed fix
- name: Audit data
-  run: forgelm audit data/train.jsonl --strict
+  run: |
+    forgelm audit data/train.jsonl --output ./audit/ --output-format json
+    # Exit kodu zaten ciddiyeti yansıtır: 0 temiz, 2 uyarı, 3 hata.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/data/audit.md` around lines 142 - 145, The CI example
currently shows the command "forgelm audit data/train.jsonl --strict" which is
invalid because "--strict" was never shipped; update the example to remove
"--strict" and instead show the recommended pattern using "forgelm audit ...
--output-format json" combined with a jq-based gate as described earlier in the
document (follow the guidance near the previous mention of --output-format json)
so the snippet matches the documented, working audit-as-gate approach.
docs/reference/approvals_subcommand-tr.md (1)

162-162: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix field name in usage example to match JSON schema.

Both the JSON envelope (lines 60, 71) define the field as age_seconds, but the usage example at line 162 incorrectly references age. Change age to age_seconds so a reader's jq '.pending[].age_seconds' resolves correctly. (Note: This same error appears in the EN mirror at approvals_subcommand.md:162 and should be fixed there as well.)

📝 Proposed fix
-Daha zengin bir policy kuran operatörler (örn. "herhangi bir bekleyen karar N saatten eskiyse deploy'u blokla") `age` field'ını parse eder. Text çıktısını yalnızca tavsiye olarak ele alın — JSON zarfı stabil sözleşmedir.
+Daha zengin bir policy kuran operatörler (örn. "herhangi bir bekleyen karar N saatten eskiyse deploy'u blokla") `age_seconds` field'ını parse eder. Text çıktısını yalnızca tavsiye olarak ele alın — JSON zarfı stabil sözleşmedir.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/reference/approvals_subcommand-tr.md` at line 162, Update the usage
example that references the JSON field name so it matches the schema: replace
occurrences of the wrong field name "age" in the jq example (e.g., jq
'.pending[].age') with "age_seconds" so jq '.pending[].age_seconds' returns the
expected values; also apply the same change to the English mirror where the same
example appears.
docs/usermanuals/en/training/sft.md (1)

113-117: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Normalize tokeniser vs tokenizer spelling in the same warning block.

Line 113 uses “tokeniser” while Line 117 uses “tokenizer”. Please standardize on one variant in this page to avoid style drift in user-facing docs.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/training/sft.md` around lines 113 - 117, The two-warning
block uses inconsistent spellings “tokeniser” and “tokenizer” (in the paragraph
under "Loss going up instead of down." and the :::warn block titled "Loss going
up after a tokenizer change."); pick one spelling (preferably "tokenizer") and
update both occurrences so the same variant appears in the rendered sample text
and the warning title/paragraph to maintain consistent user-facing
documentation.
docs/usermanuals/tr/compliance/audit-log.md (1)

15-21: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Event examples still use legacy names and contradict the updated catalog.

You updated the canonical event set at Line 34–46, but examples still show run_start, run_complete, data_audit_complete, training_epoch_complete, and auto_revert. This creates conflicting guidance in the same page.

Suggested direction
-{"ts":"...","event":"run_start",...}
+{"ts":"...","event":"training.started",...}

-{"ts":"...","event":"run_complete","exit_code":0,...}
+{"ts":"...","event":"pipeline.completed","exit_code":0,...}

- `run_start` ve `run_complete`.
+ `training.started` ve `pipeline.completed`.

- [Otomatik Geri Alma](`#/evaluation/auto-revert`) — `auto_revert` olaylarını üretir.
+ [Otomatik Geri Alma](`#/evaluation/auto-revert`) — `model.reverted` olayını üretir.

Also applies to: 77-79, 118-124, 145-146

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/compliance/audit-log.md` around lines 15 - 21, The
examples use legacy event names (run_start, run_complete, data_audit_complete,
training_epoch_complete, auto_revert) that conflict with the updated canonical
event set; update every example JSON where "event": "<legacy_name>" appears to
the corresponding new canonical event names defined in the catalog (replace each
legacy token with its exact new token from the updated canonical set), and
ensure associated fields (e.g., "verdict", "exit_code", "epoch", "loss") match
the catalog's expected schema; apply the same replacements to the other example
blocks noted (the later occurrences of those legacy names) so all examples
consistently reflect the updated event catalog.
🟠 Major comments (23)
docs/usermanuals/tr/operations/experiment-tracking.md-19-19 (1)

19-19: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Şemaya dair mutlak ifade, sayfanın artifact örneğiyle çelişiyor.

Line 19 nested backend bloklarını tamamen dışlıyor; fakat aynı doküman training.wandb.log_artifacts örneğini kullanıyor. Cümleyi “bağlantı/kimlik backend ayarları” ile sınırlandırın, aksi halde geçerli alanlar yanlışlıkla kaldırılabilir.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/operations/experiment-tracking.md` at line 19, Satırdaki
mutlak ifade ("Per-backend nested config blokları (örn. `wandb: { project: ...
}`) şemanın parçası değildir") yanlışı; bunu backend'e özgü
bağlantı/kimlik/credential ayarlarıyla sınırlayın ki örnek
`training.wandb.log_artifacts` geçerli kalmaya devam etsin. Düzenleme yaparken
orijinal metni `Per-backend nested config blokları ...` yerine "Per-backend
sadece bağlantı/kimlik/credential ayarları (örn. `wandb: { project: ... }`)
şemanın dışında tutulur; diğer yapılandırma alanları (örn.
`training.wandb.log_artifacts`) şemada tanımlı olabilir." gibi bir ifadeyle
değiştirin ve `wandb: { project: ... }` ile `training.wandb.log_artifacts`
referanslarını koruyun.
docs/usermanuals/en/operations/experiment-tracking.md-19-19 (1)

19-19: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Absolute “no nested backend config blocks” claim conflicts with this page’s own artifact example.

Line 19 says nested backend blocks are not in schema, but this same doc still documents training.wandb.log_artifacts under Artifact management. Please narrow this sentence (e.g., “for backend connection/auth settings”) or readers will remove valid config and break artifact behavior.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/operations/experiment-tracking.md` at line 19, The doc's
blanket statement "Per-backend nested config blocks (e.g. `wandb: { project: ...
}`) are not part of the schema" conflicts with the documented
`training.wandb.log_artifacts` example; update the sentence to narrow the scope
(e.g., "Per-backend nested config blocks for backend connection/auth settings
(e.g., API keys, project identifiers) are not part of the schema") and
explicitly note that backend-specific runtime flags/features such as
`training.wandb.log_artifacts` are supported in the config; ensure the revision
references `training.wandb.log_artifacts` so readers know artifacts and similar
backend-specific toggles are allowed while connection/auth settings must use
environment variables.
docs/usermanuals/en/training/kto.md-45-46 (1)

45-46: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

KTO config guidance is internally contradictory.

This new note says desirable_weight/undesirable_weight are not exposed in ForgeLM config, but the same page still documents them as supported parameters (and recommends setting them). Please make the page consistent in one direction.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/training/kto.md` around lines 45 - 46, The page currently
contradicts itself about desirable_weight and undesirable_weight: update the KTO
section so it is consistent by choosing one stance — either (A) state clearly
that desirable_weight and undesirable_weight are NOT exposed in ForgeLM
(KTOConfig) today and remove any guidance that recommends setting them, instead
documenting the TRL-side override workflow and linking to KTOConfig as an
advanced workaround; or (B) if you intend to support them, update the earlier
note to say they are exposed, keep the usage examples and default values, and
remove the sentence claiming they are not surfaced. Refer to the identifiers
desirable_weight, undesirable_weight, and KTOConfig when editing so you update
all mentions.
docs/reference/compliance_summary-tr.md-117-117 (1)

117-117: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

approvals command example is missing required --output-dir.

forgelm approvals --pending should include --output-dir <dir> to match the documented CLI contract used across compliance docs.

Suggested fix
-- Listeleme: `forgelm approvals --pending` (Phase 37).
+- Listeleme: `forgelm approvals --pending --output-dir <output_dir>` (Phase 37).

Based on learnings, the official approvals workflow contract requires forgelm approvals --pending --output-dir <dir>.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/reference/compliance_summary-tr.md` at line 117, The example command
"forgelm approvals --pending" is missing the required output directory flag;
update the documented CLI example so the command reads "forgelm approvals
--pending --output-dir <dir>" to match the official approvals workflow contract
and other compliance docs.
docs/guides/iso_soc2_deployer_guide.md-111-112 (1)

111-112: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Clarify config_hash source (event vs sidecar).

Line 111 currently reads as if human_approval.granted itself carries config_hash, while also labeling it as a manifest sidecar field. Please make the source explicit to avoid auditors grepping the wrong artifact.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/guides/iso_soc2_deployer_guide.md` around lines 111 - 112, Clarify that
config_hash is a per-run manifest sidecar field (not embedded in the
human_approval.granted event); update the sentence referencing `config_hash` and
`human_approval.granted` to state that `config_hash` is stored in the run's
manifest sidecar and auditors should compare that sidecar value to the YAML in
`git log`, while `human_approval.granted` is just the approval event (does not
carry `config_hash`).
docs/usermanuals/en/data/audit.md-110-110 (1)

110-110: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Removed-flag note conflicts with examples still in this page.

Line 110 says --strict and --sample-rate never shipped, but this same document still uses --strict in the CI example (Line 150) and discusses --sample-rate in pitfalls (Line 160). Please update those sections in the same PR to avoid copy/paste command failures.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/data/audit.md` at line 110, The removed-flags note
conflicts with examples: update the CI example that uses the removed flag
`--strict` (referenced in the CI example block) to use the supported workflow
(wrap `--output-format json` with a jq-based CI gate or remove `--strict`), and
remove or replace mentions of the removed `--sample-rate` in the "pitfalls"
section with the current supported options or guidance; ensure every occurrence
of `--strict` and `--sample-rate` in the document (CI example and pitfalls
paragraphs) is either deleted or converted to the canonical alternative so the
examples run without the removed flags.
docs/usermanuals/tr/training/kto.md-45-45 (1)

45-45: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

KTO config contract is internally inconsistent in this page.

Line 45 says desirable_weight/undesirable_weight are not exposed in ForgeLM config, but the same page still documents them as tunable parameters and recommends setting them (Lines 60-67). Please align one side; otherwise users will try unsupported fields.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/training/kto.md` at line 45, Sayfa içindeki çelişkiyi
düzelt: KTOConfig içindeki `desirable_weight` / `undesirable_weight` alanlarının
ForgeLM config'e yüzeylenmediğini belirten (Line 45) açıklama ile sonraki
bölümlerde bu alanların ayarlanabileceği yönündeki dokümantasyonu tutarlı hale
getir. Bunu yapmak için ya KTOConfig dokümantasyonundaki tunable parametreler
listesinden `desirable_weight`/`undesirable_weight` öğelerini kaldır (böylece
yalnızca trainer TRL varsayılanları 1.0/1.0 geçerli olur) ya da Line 45
açıklamasını güncelleyip açıkça “bu alanlar ForgeLM config olarak desteklenmez;
asimetrik ağırlıklandırma gerekiyorsa TRL tarafında override script kullanın”
şeklinde yazarak Lines 60-67’deki öneriyi bir workaround olarak belirt (referans
verilecek semboller: KTOConfig, desirable_weight, undesirable_weight, ForgeLM,
trainer TRL).
docs/usermanuals/en/training/grpo.md-59-60 (1)

59-60: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Align the new flat GRPO contract with the rest of this page.

The new guidance on Line 59 (“always-on fallback” + grpo_reward_model) conflicts with later sections that still document the old nested training.grpo.reward_function/format_reward/answer_pattern model. Please update the downstream sections/table to one canonical schema to avoid misconfiguration.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/training/grpo.md` around lines 59 - 60, The docs
currently mix the new flat GRPO contract with the old nested schema, causing
confusion; update the downstream sections and the configuration table to use the
canonical flat schema (referencing grpo_reward_model and the built-in fallback
forgelm/grpo_rewards.py) and remove or replace references to the legacy nested
keys (training.grpo.reward_function, format_reward, answer_pattern), ensuring
any mention of TRL's beta/KL remains consistent with the new flat field
guidance. Locate occurrences of the legacy keys and either rename them to
grpo_reward_model (or document as deprecated with a single-line migrate note)
and adjust examples/tables to show the flat contract and the always-on fallback
behavior from forgelm/grpo_rewards.py.
docs/usermanuals/en/data/secrets.md-12-27 (1)

12-27: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Propagate the new secrets taxonomy/mask token across the whole page.

The updated contract here conflicts with later examples/config (github_pat/pem_block/openai_key) and with the PEM section’s [REDACTED-PEM-BLOCK] token. Please align all examples and output snippets to one canonical key set and one masking-output rule.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/data/secrets.md` around lines 12 - 27, The page currently
mixes different secret taxonomy names and mask tokens; update all later
examples, config snippets and the PEM section to use the canonical keys from
_SECRET_PATTERNS (e.g., github_token, openai_api_key, openssh_private_key,
aws_access_key, etc.) and ensure all masked output uses the single mask value
produced by mask_secrets() — [REDACTED-SECRET] — replacing occurrences of
github_pat, pem_block, openai_key, [REDACTED-PEM-BLOCK], or any other alternate
tokens so the entire document references one consistent key set and one masking
output.
docs/design/gdpr_erasure.md-199-199 (1)

199-199: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Resolve --yes semantics drift in this spec.

Line 199 says --yes is not implemented and purge is non-interactive, but later sections still describe prompt-driven behavior and missing---yes errors. Please normalize the entire document to a single shipped behavior model.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/design/gdpr_erasure.md` at line 199, The spec is inconsistent about the
`--yes` flag and interactive behavior: standardize the document to reflect the
shipped behavior that `forgelm purge` is non-interactive by default (as of
v0.5.5) and that `--yes` is currently unimplemented and retained only on the
Phase 28+ backlog; update all places that describe prompt-driven flows,
missing-`--yes` errors, and interactive examples to instead state that purge
never prompts, CI and TTY both require `--justification` for audit metadata, and
mark `--yes` as reserved/backlog (remove any instructions that treat `--yes` as
required or that expect an interactive confirmation).
tools/check_library_api_doc.py-82-90 (1)

82-90: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Fail closed when forgelm import fails.

For a CI drift guard, returning 0 on import failure can silently pass while the checker never runs. This should return non-zero so CI cannot go green on a skipped validation path.

💡 Suggested fix
     try:
         import forgelm  # type: ignore
     except ImportError as exc:
         print(
             f"check_library_api_doc: cannot import forgelm ({exc}); skipping.",
             file=sys.stderr,
         )
-        return 0
+        return 1
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/check_library_api_doc.py` around lines 82 - 90, The try/except that
imports forgelm currently returns 0 on ImportError which lets CI pass silently;
change the except block in check_library_api_doc.py (the import forgelm
try/except) to exit with a non-zero status (e.g., return 1 or raise
SystemExit(1)) and include the original exception text in the stderr message so
the script fails CI instead of succeeding when forgelm cannot be imported.
docs/usermanuals/tr/training/dpo.md-74-75 (1)

74-75: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Update the parameter table to reference dpo_beta (not just beta).

Line 74 correctly shows dpo_beta: 0.1 as the configuration key, but the parameter table at line 100 documents it as beta, creating confusion. Update the table to either show dpo_beta or add clarification that dpo_beta is the flat-field config key. This prevents users from copy-pasting mismatched config snippets.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/training/dpo.md` around lines 74 - 75, The parameter
table currently documents the KL weight as `beta` but the config uses
`dpo_beta`; update the table entries to reference `dpo_beta` (or add a
parenthetical note that `dpo_beta` is the flat-field config key) so examples and
descriptions match the actual config key; specifically change any `beta`
occurrences in the parameters table to `dpo_beta` and adjust the description to
mention it is the KL strength used by DPO (e.g., "dpo_beta: KL strength (flat
field)").
docs/guides/library_api-tr.md-179-180 (1)

179-180: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Fix invalid Python f-string syntax in code example.

Nested double quotes in f-string expression cause SyntaxError. Change inner dictionary access quotes to single quotes so users can run the code without parse errors.

Proposed fix
-        raise ValueError(f"too many duplicates: {report.near_duplicate_summary["pairs"]}")
+        raise ValueError(f"too many duplicates: {report.near_duplicate_summary['pairs']}")
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/guides/library_api-tr.md` around lines 179 - 180, The f-string in the
raise statement uses nested double quotes and causes a SyntaxError; update the
dict key access to use single quotes (replace
report.near_duplicate_summary["pairs"] with
report.near_duplicate_summary['pairs']) and do the same in the return expression
for consistency so both the raise ValueError(...) and the return {"samples":
report.total_samples, "duplicates": report.near_duplicate_summary["pairs"]} use
report.near_duplicate_summary['pairs'].
docs/usermanuals/tr/training/grpo.md-59-60 (1)

59-60: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Align GRPO schema to flat keys throughout both documentation versions.

The quick example correctly uses flat GRPO keys (grpo_num_generations, grpo_max_completion_length, grpo_reward_model), but the "Built-in format shaping" section and configuration tables still describe a nested schema under training.grpo.* (reward_function, format_reward, answer_pattern). This inconsistency appears in both the English and Turkish versions and can lead to invalid configs. Update the later sections to match the canonical flat key structure shown in the quick example.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/training/grpo.md` around lines 59 - 60, The docs use
nested keys under the "Built-in format shaping" section and configuration tables
but the canonical quick example uses flat GRPO keys; update the "Built-in format
shaping" text and every configuration table in both English and Turkish pages to
use the flat keys (e.g., grpo_num_generations, grpo_max_completion_length,
grpo_reward_model, grpo_reward_function, grpo_format_reward,
grpo_answer_pattern) so examples and config schema match the quick example and
produce valid configs.
docs/usermanuals/en/operations/webhooks.md-90-90 (1)

90-90: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win

Brittle line-number reference — convert to symbol/function name per the PR's drift sweep.

The PR objectives explicitly note that hardcoded line-number references in subcommand docs were converted to symbol/function-name references (commit 694bf12) under the "point at file root and cite symbol names" convention. forgelm/config.py:641 is the same brittle pattern: any insertion above line 641 silently desyncs the doc. Cite the class name (WebhookConfig) instead.

📝 Proposed fix
-Real `WebhookConfig` (see `forgelm/config.py:641`):
+Real `WebhookConfig` (defined as the `WebhookConfig` Pydantic model in `forgelm/config.py`):
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/operations/webhooks.md` at line 90, The doc's brittle
line-number reference "forgelm/config.py:641" should be replaced with a
symbol-based reference to the WebhookConfig class; update the text to point at
the module root and cite the symbol, e.g. "forgelm.config:WebhookConfig" or
"forgelm.config — WebhookConfig", so the docs reference the class name
WebhookConfig rather than a line number.
docs/usermanuals/tr/data/audit.md-93-94 (1)

93-94: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win

Brittle line-range reference — symbol-only is the project convention.

Per the PR objectives, hardcoded line refs in subcommand docs were already converted to symbol/function-name references. (335-468 satırları) defeats that convention: any edit to _parser.py above line 468 silently desyncs this doc. Cite _add_audit_subcommand only and drop the line range.

📝 Proposed fix
-Yetkili kaynak: `forgelm/cli/_parser.py::_add_audit_subcommand` (335-468 satırları).
+Yetkili kaynak: `forgelm/cli/_parser.py` içindeki `_add_audit_subcommand` fonksiyonu.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/data/audit.md` around lines 93 - 94, Remove the hardcoded
line-range fragment from the audit doc line that currently reads "Yetkili
kaynak: `forgelm/cli/_parser.py::_add_audit_subcommand` (335-468 satırları)";
keep only the symbol reference so it becomes "Yetkili kaynak:
`forgelm/cli/_parser.py::_add_audit_subcommand`." Ensure the backtick-wrapped
symbol name `_add_audit_subcommand` remains unchanged.
docs/guides/troubleshooting-tr.md-405-411 (1)

405-411: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Exit code 4 guidance points to a stale path: checkpoints/compliance/.

This path does not exist in the codebase. The canonical staging convention for awaiting-approval runs is <output_dir>/final_model.staging.<run_id>/ (with legacy fallback final_model.staging/). Users following this guidance will be unable to locate staged adapters or complete the approval workflow. Point them to forgelm approvals --show <run_id> to inspect the actual staging directory, then forgelm approve <run_id> to promote it.

📝 Proposed fix
-| `4` | Onay bekleniyor | İnsan incelemesi gerekli — `checkpoints/compliance/`'taki sonuçları inceleyin ve hazır olunca yeniden deploy edin |
+| `4` | Onay bekleniyor | İnsan incelemesi gerekli — `forgelm approvals --show <run_id>` ile staged sonuçları inceleyin (`<output_dir>/final_model.staging.<run_id>/`) ve `forgelm approve <run_id>` ile karar verin |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/guides/troubleshooting-tr.md` around lines 405 - 411, Update the
guidance for exit code 4: replace the stale path reference
`checkpoints/compliance/` with the canonical staging convention
`<output_dir>/final_model.staging.<run_id>/` (and mention the legacy fallback
`final_model.staging/`), and add the exact commands users should run to inspect
and promote staged runs: `forgelm approvals --show <run_id>` to locate the
staging directory and `forgelm approve <run_id>` to promote it; ensure this
change is applied to the text that describes exit code 4 in the troubleshooting
table so readers can find staged adapters and complete the approval workflow.
docs/usermanuals/tr/operations/webhooks.md-128-128 (1)

128-128: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Don’t position --dry-run as an end-to-end webhook smoke test.

Line 128 suggests lifecycle webhook validation via --dry-run, but dry-run is validation-only and does not execute the training lifecycle. That can produce a false “webhooks are fine” result. Recommend changing this to a tiny real run (small dataset/epochs) or synthetic payload replay only.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/operations/webhooks.md` at line 128, Update the guidance
that currently suggests using `--dry-run` as an end-to-end webhook smoke test:
clarify that `--dry-run` only validates config and does not execute the training
lifecycle, and replace the suggested test with either a tiny real run (small
dataset + low `num_train_epochs`) against the lifecycle staging webhook URL or
replaying a curated synthetic payload via `curl`; also keep the note that
ForgeLM's `--webhook-test` flag does not send webhooks so it cannot be used for
E2E verification.
docs/guides/quickstart-tr.md-19-21 (1)

19-21: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Use English for code/CLI snippet text in TR quickstart.

The inline code comments and sample CLI output annotations in these blocks are Turkish. Keep snippet content English-only for bilingual consistency and reliable copy/paste workflows.

As per coding guidelines, bilingual user-facing docs must keep code and CLI output in English only.

Also applies to: 161-164

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/guides/quickstart-tr.md` around lines 19 - 21, The documentation
contains Turkish text inside code/CLI snippet blocks (for example the snippet
header "Önerilen: 4-bit quantization'ı aç (Linux)" near the pip install
example); replace those inline snippet comments and CLI annotations with
English-only equivalents (e.g., "Recommended: enable 4-bit quantization
(Linux)") and update the same pattern at the other occurrence mentioned (around
lines 161-164) so all code blocks and CLI output remain English for copy/paste
and bilingual consistency.
docs/usermanuals/tr/evaluation/judge.md-23-30 (1)

23-30: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Keep code-snippet comments in English in TR user manuals.

Line 23–30 uses Turkish inline comments inside YAML. In user-facing bilingual docs, code/CLI content should stay English-only for copy/paste consistency.

Proposed patch
 evaluation:
-  llm_judge:                            # blok adı `llm_judge`, `judge` değil
+  llm_judge:                            # block name is `llm_judge`, not `judge`
     enabled: true
-    judge_model: "gpt-4o-mini"          # veya yerel yol, ör. "./judges/Qwen2.5-72B-Instruct"
-    judge_api_key_env: OPENAI_API_KEY   # null = yerel model (API çağrısı yok)
-    judge_api_base: null                # Azure OpenAI / vLLM-uyumlu gateway için override
+    judge_model: "gpt-4o-mini"          # or local path, e.g. "./judges/Qwen2.5-72B-Instruct"
+    judge_api_key_env: OPENAI_API_KEY   # null = local model (no API call)
+    judge_api_base: null                # override for Azure OpenAI / vLLM-compatible gateway
     eval_dataset: "data/eval-prompts.jsonl"
-    min_score: 6.5                      # ortalama puan alt sınırı (1-10 skala); altına düşerse revert
-    batch_size: 8                       # round başı skorlanan (prompt, completion) çifti; 1 batching'i kapatır
+    min_score: 6.5                      # minimum mean score (1-10); below threshold triggers revert
+    batch_size: 8                       # (prompt, completion) pairs scored per round; 1 disables batching

As per coding guidelines, bilingual user-facing docs must keep code and CLI output in English only.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/evaluation/judge.md` around lines 23 - 30, The YAML
snippet uses Turkish inline comments; replace all Turkish comments in the
llm_judge block (keys: llm_judge, judge_model, judge_api_key_env,
judge_api_base, eval_dataset, min_score, batch_size) with English equivalents so
the code/CLI lines remain English-only for copy/paste consistency—update each
inline comment to English while preserving the keys and values exactly.
docs/guides/safety_compliance-tr.md-46-75 (1)

46-75: ⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Keep code snippets and CLI output in English-only in TR docs.

YAML/CLI snippets include Turkish comments/output text (for example, Line 53 comment and Line 493 echo). Project rule says code and CLI output must remain English even in bilingual docs.

As per coding guidelines, “docs/**/*.md: Bilingual docs (EN ↔ TR mirrors) required for user-facing documentation; code and CLI output must be English only”.

Also applies to: 488-495

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/guides/safety_compliance-tr.md` around lines 46 - 75, The YAML snippets
under the evaluation.safety block contain Turkish inline comments and CLI/echo
text (e.g., the comment after max_safety_regression and the scoring/severity
comments), violating the English-only code/CLI rule; update all inline comments
and any CLI/echo output in these YAML blocks to English while keeping the keys
and values unchanged (e.g., fields evaluation, safety, classifier, test_prompts,
max_safety_regression, scoring, min_safety_score, min_classifier_confidence,
track_categories, severity_thresholds, critical/high/medium) so the code/CLI
remains English but the surrounding Turkish documentation can stay in Turkish.
docs/usermanuals/tr/training/sft.md-28-48 (1)

28-48: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Keep YAML snippet comments English-only in TR mirror.

At Line 28, Line 33, and Line 47, inline YAML comments are Turkish. In bilingual docs, code/CLI content should remain English-only.

As per coding guidelines, “docs/**/*.md: Bilingual docs (EN ↔ TR mirrors) required for user-facing documentation; code and CLI output must be English only”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/training/sft.md` around lines 28 - 48, The inline YAML
comments in the SFT example are in Turkish; update them to English so code/CLI
content remains English-only. Specifically, replace the comment after backend
("ya da "unsloth" — eski `use_unsloth: true` flag'inin yerini alır") with an
English equivalent, change the lora.method comment ("ya da "dora" / "pissa" /
"rslora"") to English, and change the packing comment ("kısa örnekleri bin-pack
için true") to an English comment; keep all YAML keys (backend, lora.r,
lora.alpha, lora.method, lora.target_modules, data.dataset_name_or_path,
training.trainer_type, training.packing, etc.) unchanged so only the comments
are modified.
tools/check_yaml_snippets.py-53-68 (1)

53-68: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Provide install guidance instead of silently skipping on import failures.

Lines 57 and 68 catch import failures and exit silently with code 0. Per the coding guidelines (docs/standards/error-handling.md), missing optional dependencies should raise ImportError with install hints, not silent success.

Additionally, PyYAML is a declared dependency in pyproject.toml and requirements.txt (not optional), so line 57 should be handled differently than the forgelm import. If this tool is intended to run in lint-only environments where forgelm isn't installed, use explicit --allow-missing-forgelm mode instead of silent exit. The current approach makes it harder to diagnose missing dependencies during actual failures.

Consider the pattern in tools/build_usermanuals.py for reference: it exits 2 with an explicit install message.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/check_yaml_snippets.py` around lines 53 - 68, The current import
handlers silently exit with code 0; instead, for the required dependency PyYAML
(import yaml) change the handler to print a clear install message and exit
non-zero (exit code 2) or re-raise ImportError with an install hint (e.g., "pip
install pyyaml"); for the optional forgelm import (from forgelm.config import
ForgeConfig and pydantic.ValidationError) implement an explicit CLI flag (e.g.,
--allow-missing-forgelm) that, when set, prints a warning and continues,
otherwise prints an error with install instructions for forgelm and exits
non-zero (exit code 2) or raises ImportError—update the two except blocks to
follow this behavior and reference the yaml import and the forgelm.config /
ValidationError imports to locate the code to change.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ca4bc064-ee3b-47ea-ab03-fe0c93146e63

📥 Commits

Reviewing files that changed from the base of the PR and between 33f28d5 and ad32077.

📒 Files selected for processing (142)
  • CHANGELOG.md
  • CLAUDE.md
  • CONTRIBUTING.md
  • README.md
  • docs/design/blackwell_optimized.md
  • docs/design/gdpr_erasure.md
  • docs/design/iso27001_soc2_alignment.md
  • docs/design/library_api.md
  • docs/design/wizard_mode.md
  • docs/guides/alignment-tr.md
  • docs/guides/cicd_pipeline-tr.md
  • docs/guides/cicd_pipeline.md
  • docs/guides/gdpr_erasure-tr.md
  • docs/guides/gdpr_erasure.md
  • docs/guides/human_approval_gate-tr.md
  • docs/guides/human_approval_gate.md
  • docs/guides/ingestion.md
  • docs/guides/iso_soc2_deployer_guide-tr.md
  • docs/guides/iso_soc2_deployer_guide.md
  • docs/guides/library_api-tr.md
  • docs/guides/library_api.md
  • docs/guides/performance-tr.md
  • docs/guides/performance.md
  • docs/guides/quickstart-tr.md
  • docs/guides/quickstart.md
  • docs/guides/safety_compliance-tr.md
  • docs/guides/safety_compliance.md
  • docs/guides/troubleshooting-tr.md
  • docs/guides/troubleshooting.md
  • docs/qms/access_control-tr.md
  • docs/qms/access_control.md
  • docs/qms/encryption_at_rest-tr.md
  • docs/qms/encryption_at_rest.md
  • docs/qms/risk_treatment_plan-tr.md
  • docs/qms/risk_treatment_plan.md
  • docs/qms/sop_incident_response-tr.md
  • docs/qms/sop_incident_response.md
  • docs/qms/statement_of_applicability-tr.md
  • docs/qms/statement_of_applicability.md
  • docs/reference/approvals_subcommand-tr.md
  • docs/reference/approvals_subcommand.md
  • docs/reference/approve_subcommand-tr.md
  • docs/reference/approve_subcommand.md
  • docs/reference/architecture-tr.md
  • docs/reference/architecture.md
  • docs/reference/audit_event_catalog-tr.md
  • docs/reference/audit_event_catalog.md
  • docs/reference/compliance_summary-tr.md
  • docs/reference/compliance_summary.md
  • docs/reference/configuration-tr.md
  • docs/reference/configuration.md
  • docs/reference/data_preparation-tr.md
  • docs/reference/data_preparation.md
  • docs/reference/distributed_training-tr.md
  • docs/reference/distributed_training.md
  • docs/reference/iso27001_control_mapping-tr.md
  • docs/reference/iso27001_control_mapping.md
  • docs/reference/library_api_reference-tr.md
  • docs/reference/library_api_reference.md
  • docs/reference/purge_subcommand-tr.md
  • docs/reference/purge_subcommand.md
  • docs/reference/reverse_pii_subcommand-tr.md
  • docs/reference/reverse_pii_subcommand.md
  • docs/reference/safety_eval_subcommand-tr.md
  • docs/reference/safety_eval_subcommand.md
  • docs/reference/soc2_trust_criteria_mapping-tr.md
  • docs/reference/soc2_trust_criteria_mapping.md
  • docs/reference/supply_chain_security-tr.md
  • docs/reference/supply_chain_security.md
  • docs/reference/usage-tr.md
  • docs/reference/usage.md
  • docs/reference/verify_audit-tr.md
  • docs/reference/verify_audit.md
  • docs/standards/testing.md
  • docs/usermanuals/en/compliance/audit-log.md
  • docs/usermanuals/en/compliance/human-oversight.md
  • docs/usermanuals/en/compliance/model-card.md
  • docs/usermanuals/en/compliance/overview.md
  • docs/usermanuals/en/concepts/data-formats.md
  • docs/usermanuals/en/data/audit.md
  • docs/usermanuals/en/data/pii-ml.md
  • docs/usermanuals/en/data/secrets.md
  • docs/usermanuals/en/data/synthetic-data.md
  • docs/usermanuals/en/deployment/deploy-targets.md
  • docs/usermanuals/en/evaluation/auto-revert.md
  • docs/usermanuals/en/evaluation/judge.md
  • docs/usermanuals/en/evaluation/safety.md
  • docs/usermanuals/en/getting-started/project-layout.md
  • docs/usermanuals/en/operations/experiment-tracking.md
  • docs/usermanuals/en/operations/gpu-cost.md
  • docs/usermanuals/en/operations/iso-soc2-deployer.md
  • docs/usermanuals/en/operations/vram-fit-check.md
  • docs/usermanuals/en/operations/webhooks.md
  • docs/usermanuals/en/reference/cli.md
  • docs/usermanuals/en/reference/exit-codes.md
  • docs/usermanuals/en/reference/yaml-templates.md
  • docs/usermanuals/en/training/dpo.md
  • docs/usermanuals/en/training/grpo.md
  • docs/usermanuals/en/training/kto.md
  • docs/usermanuals/en/training/orpo.md
  • docs/usermanuals/en/training/sft.md
  • docs/usermanuals/en/training/simpo.md
  • docs/usermanuals/tr/compliance/audit-log.md
  • docs/usermanuals/tr/compliance/human-oversight.md
  • docs/usermanuals/tr/compliance/model-card.md
  • docs/usermanuals/tr/compliance/overview.md
  • docs/usermanuals/tr/concepts/data-formats.md
  • docs/usermanuals/tr/data/audit.md
  • docs/usermanuals/tr/data/pii-ml.md
  • docs/usermanuals/tr/data/secrets.md
  • docs/usermanuals/tr/data/synthetic-data.md
  • docs/usermanuals/tr/deployment/deploy-targets.md
  • docs/usermanuals/tr/evaluation/auto-revert.md
  • docs/usermanuals/tr/evaluation/judge.md
  • docs/usermanuals/tr/evaluation/safety.md
  • docs/usermanuals/tr/getting-started/first-run.md
  • docs/usermanuals/tr/getting-started/installation.md
  • docs/usermanuals/tr/getting-started/project-layout.md
  • docs/usermanuals/tr/operations/cicd.md
  • docs/usermanuals/tr/operations/experiment-tracking.md
  • docs/usermanuals/tr/operations/gpu-cost.md
  • docs/usermanuals/tr/operations/iso-soc2-deployer.md
  • docs/usermanuals/tr/operations/vram-fit-check.md
  • docs/usermanuals/tr/operations/webhooks.md
  • docs/usermanuals/tr/reference/cli.md
  • docs/usermanuals/tr/reference/exit-codes.md
  • docs/usermanuals/tr/reference/yaml-templates.md
  • docs/usermanuals/tr/training/dpo.md
  • docs/usermanuals/tr/training/grpo.md
  • docs/usermanuals/tr/training/kto.md
  • docs/usermanuals/tr/training/orpo.md
  • docs/usermanuals/tr/training/sft.md
  • docs/usermanuals/tr/training/simpo.md
  • site/compliance.html
  • site/css/style.css
  • site/quickstart.html
  • tools/check_audit_event_catalog.py
  • tools/check_bilingual_code_blocks.py
  • tools/check_bilingual_parity.py
  • tools/check_doc_numerical_claims.py
  • tools/check_library_api_doc.py
  • tools/check_yaml_snippets.py

Comment thread docs/guides/library_api.md
Comment thread docs/usermanuals/en/operations/webhooks.md Outdated
cemililik and others added 2 commits May 8, 2026 08:36
…tions purge

Address the 35 review comments from gemini-code-assist on PR #39
(12 outside-diff + 23 major). Every finding verified against current
code state before applying the fix; EN+TR bilingual pairs kept in lockstep.

Code bugs surfaced by the doc audit:

  - docs/guides/library_api.md + -tr.md: f-string SyntaxError
    (nested double quotes inside expression). Switched inner dict access
    to single quotes.
  - docs/usermanuals/en/operations/webhooks.md + -tr.md: "ForgeLM rejects
    HTTP webhook URLs" was wrong — `forgelm/webhook.py` calls
    `safe_post(..., allow_insecure_http=True)` and only logs a warning
    on `http://` URLs (`_send`). Rewrote to "TLS strongly recommended".
  - tools/check_library_api_doc.py + check_yaml_snippets.py: silent
    sys.exit(0) on import failure → exit 2 with install hint. CI guards
    no longer pass green when the environment is broken.

Trainer schema purge (sft/dpo/simpo/orpo/kto/grpo, EN+TR):

  - DPO: phantom `loss_type` / `label_smoothing` / `reference_free` /
    `reference_model` / `loss_dpop_lambda` / `pref_chosen_weight` rows
    removed. Real schema: only `training.dpo_beta` is configurable.
  - SimPO: phantom `loss_type` / `length_normalize` / `label_smoothing`
    removed; tables aligned to `simpo_beta` / `simpo_gamma`. Default
    `simpo_gamma` corrected 1.0 → 0.5. β-γ combination tables relabeled.
  - ORPO: phantom `loss_type` / `sft_weight` removed; tables aligned to
    `training.orpo_beta`.
  - KTO: phantom `desirable_weight` / `undesirable_weight` / `loss_type`
    removed; tables + tip section aligned to "only `kto_beta` configurable;
    oversample minority class for imbalance".
  - GRPO: nested `training.grpo: { reward_function, format_reward,
    answer_pattern }` block + phantom `group_size` / `beta` / `temperature`
    /  `max_completion_length` (legacy alias) rows removed. Real schema:
    flat `grpo_num_generations` / `grpo_max_completion_length` /
    `grpo_reward_model`; built-in shaper is in `grpo_rewards.py`, no
    user-configurable weight knob.
  - SFT: tokeniser → tokenizer normalisation (single-page style drift).

Audit page contradictions (EN+TR):

  - `--strict` + `--sample-rate` removal-notice already noted, but CI
    examples (lines 150 / 144) still used `--strict` and pitfalls warned
    about `--sample-rate` as if it shipped. Replaced both with the
    canonical `--output-format json | jq -e '...'` gate.
  - audit-tr.md: `_parser.py` line range "335-468" → symbol-only
    `_add_audit_subcommand`. `forgelm/config.py:566/641` brittle line refs
    in compliance/overview.md + qms/access_control.md(-tr) and
    operations/webhooks.md(-tr) → `::WebhookConfig` /
    `::ComplianceMetadataConfig` symbol refs.

Internal-contradiction sweeps:

  - evaluation/auto-revert.md: "Custom guard fails / `evaluation.guards.<name>`"
    table row removed (already documented as nonexistent). Trigger table
    rewritten to match the four real guard families
    (benchmark.min_score / safety.max_safety_regression /
    safety.severity_thresholds / llm_judge.min_score / max_acceptable_loss).
  - operations/experiment-tracking.md: phantom `training.tags` /
    `training.notes` / `training.wandb.{log_artifacts,artifact_type}`
    blocks replaced with the real env-var convention (WANDB_TAGS,
    WANDB_NOTES, WANDB_LOG_MODEL, MLFLOW_TAGS); absolute "no nested
    backend blocks" claim kept but the conflicting `training.wandb.*`
    artifact example was the source of the contradiction — now removed.
  - compliance/human-oversight.md: phantom `compliance.human_approval` /
    `approval.{request_webhook,signature_method,timeout_hours,
    require_role,quorum,webhook_url}` block + ed25519 signature claim +
    quorum + timeout flow all replaced with the real shipped behaviour
    (only `evaluation.require_human_approval`; HMAC chain; no quorum;
    no timeout — `retention.staging_ttl_days` for stale-cleanup).
  - compliance/audit-log.md: legacy event names (`run_start`,
    `run_complete`, `data_audit_complete`, `training_epoch_complete`,
    `benchmark_complete`, `safety_eval_complete`, `auto_revert`) replaced
    with canonical names from `audit_event_catalog.md`. Phantom
    `compliance.audit_log:` config block + `forward_to:` array removed.

Stale-reference fixes:

  - troubleshooting.md + -tr.md exit-code 4 now points to
    `forgelm approvals --show <run_id>` + `forgelm approve / reject` and
    the canonical `final_model.staging.<run_id>/` path (was: nonexistent
    `checkpoints/compliance/`).
  - approvals_subcommand.md + -tr.md `.age` → `.age_seconds` field name
    (matches the JSON envelope); jq example updated.
  - compliance_summary.md + -tr.md: `forgelm approvals --pending` →
    `forgelm approvals --pending --output-dir <dir>` (the flag is
    required by the dispatcher).
  - iso_soc2_deployer_guide.md + -tr.md: `config_hash` (per-run manifest
    sidecar field) clarified — the field is read forward-compatibly by
    `_approvals.py` but no current producer emits it; the real run-pinning
    evidence lives in `pipeline.training_started` event payload.
  - design/gdpr_erasure.md `--yes` semantics drift: §4.2 exit-code matrix
    + Q4 design rationale now match the shipped non-interactive behaviour
    (consent recorded via `--justification`, no prompt, no `--yes`).
  - distributed_training-tr.md FSDP YAML: bilingual parity restored
    (added `fsdp_backward_prefetch` + `fsdp_state_dict_type` to match EN
    + the FSDP-offload clarifying comment).
  - operations/cicd.md + -tr.md: GitLab pipeline now ships a real
    `deploy:` job (was: stage declared, job missing); audit step uses
    the `--output-format json | jq -e` gate; `checkpoints/run/artifacts/`
    artifact paths corrected to top-level `checkpoints/run/`.

Secrets taxonomy normalisation (EN+TR):

  - data/secrets.md: example output (lines 35-41) used non-canonical
    `github_pat` / `pem_block` / `openai_key` names. Replaced with
    `github_token` / `openssh_private_key` / `openai_api_key` from
    `_SECRET_PATTERNS`.
  - PEM section: phantom `[REDACTED-PEM-BLOCK]` per-family token replaced
    with the real single-replacement contract (`mask_secrets()` ships one
    `replacement="[REDACTED-SECRET]"` constant for every family).
  - `secrets_mask.categories` example list updated to canonical names.
  - `_secrets.py:35` / `:106` line refs → `::_SECRET_PATTERNS` /
    `::mask_secrets` symbol refs.

Webhook smoke-test guidance (EN+TR):

  - The `--dry-run` recommendation was misleading — dry-run is
    config-validation only, doesn't fire the trainer lifecycle, so
    webhooks aren't exercised. Replaced with "real tiny training run
    (small dataset + low num_train_epochs)" or `curl` synthetic POST.

Turkish-in-code violations (per coding standard "code/CLI must be
English-only in bilingual docs"):

  - quickstart-tr.md: install/fit-check shell comments → English.
  - judge-tr.md: YAML inline comments → English.
  - sft-tr.md: backend / method / packing comments → English.
  - safety_compliance-tr.md: safety scoring YAML comments + CI bash echo
    → English.

Validation: 8/8 doc CI guards green; ruff check + format clean;
usermanuals regenerated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ixes

Address every SonarQube finding on PR #39's tooling + site:

site/quickstart.html L361 — Major (Code Smell, "unused"):
  Removed the verbose multi-line wizard-modal documentation comment
  that Sonar flagged as commented-out code. Wizard behaviour is
  documented in site/js/wizard.js comments; the HTML element is
  self-evident.

tools/ cognitive-complexity refactors (Critical, "brain-overload"):
  Each function below split into smaller units; behaviour and exit
  codes preserved (all 8 doc guards still green).

  - check_audit_event_catalog.py main()      19 → 15
      Extracted: _build_arg_parser, _print_code_only, _print_catalog_only.
  - check_bilingual_code_blocks.py _pair_diff() 23 → 15
      Extracted: _yaml_key_diff_line (per-block YAML diff),
      _block_pair_diff (lang + yaml dispatch). _pair_diff is now a
      thin loop over the per-block helper.
  - check_doc_numerical_claims.py
      canonical_secret_families() 18 → 15
        Extracted: _secret_patterns_dict_node — handles AnnAssign +
        Assign uniformly, returns the ast.Dict literal or None.
      search_doc_claims()         19 → 15
        Extracted: _is_indexable_doc (skip analysis/marketing),
        _scan_line_for_mismatches (per-line match → Mismatch list).
  - check_library_api_doc.py main()          20 → 15
      Extracted: _build_arg_parser,
      _import_forgelm_or_explain (returns module or None for fail-closed
      exit 2), _print_drift_section (renders one named drift bucket).
  - check_yaml_snippets.py main()            24 → 15
      Extracted: _build_arg_parser,
      _is_validated_forgelm_snippet (count "passed validation"
      ForgeConfig-shaped snippets without re-running the parser inside
      main's loop), _print_failures.

tools/check_library_api_doc.py L45 — Minor (regex, "concise character class"):
  Replaced `[A-Za-z0-9_]` with `\w` in the `_DOC_ROW_RE` symbol-name
  pattern. Behaviour preserved: `[A-Za-z_]\w*` still rejects leading
  digits, accepts the same trailing characters as before. (re module
  default is ASCII-only for `\w` in str patterns when the regex
  doesn't set re.UNICODE — Python 3.x str regex sets re.UNICODE
  implicitly, but `\w` here is bounded by the leading `[A-Za-z_]` so
  the unicode-letter case can never start a symbol name. Match set
  identical for the realistic input — Python identifiers in the
  library reference markdown.)

Validation: 8/8 doc CI guards green; ruff check + format clean on
tools/. No behavioural change to any guard's output.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cemililik

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 8, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 15

🧹 Nitpick comments (1)
docs/guides/troubleshooting-tr.md (1)

51-51: ⚡ Quick win

Consider fixing markdown formatting for consistency.

Static analysis tools flagged several markdown formatting issues that would improve rendering and standards compliance:

  1. Missing blank lines around code blocks (lines 51, 58, 64, 69, 76, 82, 89, 115, 121, 127, 158, 164, 170, 176, 330): Add blank lines before and after fenced code blocks per MD031.

  2. Missing language specifiers (lines 238, 252, 286, 357): Add language identifiers to error message code blocks for proper syntax highlighting. These appear to be error output, so consider using text or plaintext.

  3. Missing blank line before list (line 97): Add a blank line before the **Sebepler:** list.

Example fix for line 238:

-```
+```text
 Configuration validation failed: 1 validation error for ForgeConfig

Also applies to: 58-58, 64-64, 69-69, 76-76, 82-82, 89-89, 97-97, 115-115, 121-121, 127-127, 158-158, 164-164, 170-170, 176-176, 238-238, 252-252, 286-286, 330-330, 357-357

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/guides/troubleshooting-tr.md` at line 51, The markdown has inconsistent
fenced-code formatting: add a blank line before and after every fenced code
block (e.g. the ```yaml block shown) to satisfy MD031, add language specifiers
for the error output blocks (use `text` or `plaintext` for the blocks at the
flagged positions such as the error examples around lines 238, 252, 286, 357),
and insert a blank line before the list headed "**Sebepler:**". Update each
fenced block accordingly across the document so every triple-backtick block has
a surrounding blank line and appropriate language identifier.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/guides/quickstart-tr.md`:
- Around line 145-152: The fenced code block that begins with
"./checkpoints/final_model/" in the quickstart-tr guide lacks a language tag,
causing markdownlint MD040; update that block by changing the opening fence from
``` to ```text (or another appropriate language) so the code block is explicitly
labeled (e.g., use ```text for the directory tree).
- Around line 300-301: Mevcut metinde dosya satır numarası anchor'ı
(`forgelm/config.py:589-635`) kullanılmış; bunu kırılmaya daha dayanıklı olacak
şekilde sembol/tabanlı referansa çevirin: metindeki satır-anchor'ı kaldırıp
yerine `SyntheticConfig` tanımına yönelik sembol tabanlı referans kullanın (ör.
belgedeki veya repo linkindeki `SyntheticConfig` tanımı/başlığına işaret eden
sembol referansı). Aşağıdaki öğelerde değişiklik yapın: belge metni adlandırılan
referansı içeren cümle, link hedefi ve varsa alt metin açıklaması; referans için
kullanılan sembolün tam adı "SyntheticConfig" olduğundan emin olun.

In `@docs/guides/safety_compliance-tr.md`:
- Around line 317-319: The audit example uses non-canonical event names
("pipeline.started" and "training.completed"); update the snippet to use the
canonical event names used across the PR and in
docs/reference/audit_event_catalog.md (e.g., use "training.started" and
"pipeline.completed") so the example aligns with the contract; search for and
replace these two event strings in the integrity example in
docs/guides/safety_compliance-tr.md and verify consistency with the
audit_event_catalog.md event surface.

In `@docs/reference/compliance_summary-tr.md`:
- Around line 140-143: Update the numeric ISO control counts in the Turkish
paragraph that starts "ForgeLM, 93 ISO 27001 Annex A kontrolünün ~59'una teknik
kanıt sağlar" so they match the English compliance summary's "~61 / ~32" (or, if
the TR text intentionally differs, add a concise parenthetical explaining the
reason for the intentional divergence); ensure the sentence and any adjacent
counts (`~59 / ~34`) are replaced or annotated to exactly reflect the EN numbers
(`~61 / ~32`) or include the explicit justification for the difference so both
versions remain consistent and traceable to the underlying code-backed mapping.

In `@docs/usermanuals/en/compliance/audit-log.md`:
- Around line 70-77: The docs claim the canonical audit log location is
`<output_dir>/audit_log.jsonl` but examples still reference
`checkpoints/run/artifacts/audit_log.jsonl`; update all verify/command snippets
in this document to use the canonical `<output_dir>/audit_log.jsonl` path so
examples match the stated contract, including any uses in "verify" command
examples and the example section that currently points to
`checkpoints/run/artifacts/audit_log.jsonl`; search for the string
`checkpoints/run/artifacts/audit_log.jsonl` (and any variants) and replace them
with `<output_dir>/audit_log.jsonl` so the README and the code contract are
consistent.

In `@docs/usermanuals/en/data/secrets.md`:
- Line 26: Update the sentence that currently states all matches are replaced
with `[REDACTED-SECRET]` to explicitly say this is the default behavior of
mask_secrets() and that category-tagging is an opt-in mode; specifically,
mention that mask_secrets() (`forgelm/data_audit/_secrets.py::mask_secrets`)
replaces matches with `[REDACTED-SECRET]` by default and add a short pointer
that when `tag_by_category: true` the output uses category-specific tags as
described later (see the existing description of tag_by_category); keep the same
function reference to avoid confusion.

In `@docs/usermanuals/en/operations/cicd.md`:
- Line 122: The Jenkins sh command string has mismatched quotes around the jq
filter in the forgelm audit pipeline; fix it by making the outer sh string use
double quotes (or escape the inner quotes) so the jq expression '.verdict !=
"errors" and .pii_summary.severity != "high"' is passed intact to jq; update the
sh invocation line (the sh 'forgelm audit ... | jq -e ...' command) to use
consistent quoting so the shell sees the full jq filter as a single argument.

In `@docs/usermanuals/tr/compliance/audit-log.md`:
- Around line 75-80: Remove the stale sentence that references a project-wide
artifacts/.forgelm/global-audit-log.jsonl and ensure the section consistently
states that there is no global log; instead keep/clarify that each run writes
<output_dir>/audit_log.jsonl (and the genesis-pin sidecar
<output_dir>/audit_log.jsonl.manifest.json), that there is no
compliance.audit_log: YAML knob, and that HMAC chaining is enabled via the
FORGELM_AUDIT_SECRET env var before running the trainer.

In `@docs/usermanuals/tr/compliance/human-oversight.md`:
- Around line 63-64: Update the sequence diagram text to match the
documentation’s statement that only the CLI subcommand pair `forgelm approve` /
`forgelm reject` is supported: locate the diagram label or caption that
currently mentions "CLI / webhook callback" and change it to "CLI only (forgelm
approve / forgelm reject)". Ensure any arrows or participant notes referencing a
webhook callback or approval-resume HTTP endpoint are removed or replaced with
CLI-only wording so the flow in the diagram aligns with the `forgelm approve` /
`forgelm reject` mechanism described in the `human-oversight.md` text.

In `@docs/usermanuals/tr/data/audit.md`:
- Around line 125-135: Replace the incorrect JSON snippet: rename the top-level
"quality_flags" to "quality_summary" and change its internal layout to include
"samples_flagged", "samples_evaluated", "by_check" (an object with counts) and
"overall_quality_score"; inside "by_check" use the real checks low_alpha_ratio,
low_punct_endings, abnormal_mean_word_length, short_paragraphs, and
repeated_lines (replace abnormal_word_length and remove short_response), and
remove the entire "preference_specific" section (identical_chosen_rejected,
empty_chosen, swapped_likely) so the example matches the actual audit output
structure.

In `@docs/usermanuals/tr/operations/experiment-tracking.md`:
- Around line 78-79: Lines showing the sampling interval for metrics
`system/gpu_utilization` and `system/vram_used_gb` use the typographically
awkward token "30s'de"; update those two entries to match the documentation
localization/style guide by changing "30s'de" to "30 s'de" (or your repository's
canonical spacing/format from docs/standards/localization.md) so the wording
reads consistently and follows docs/standards/documentation.md.
- Line 61: Satırdaki yanlış ifade "`tensorboardX` `transformers` ile birlikte
gelir" yanlış; güncelleme yaparak bu iddiayı kaldırın ve yerine TensorBoard
entegrasyonu için kullanıcıların ya `tensorboard` (PyTorch >= 1.4 ile) ya da
`tensorboardX`'i ayrı olarak kurmaları gerektiğini belirten metni koyun
(örneğin: "TensorBoard entegrasyonu için `tensorboard` (PyTorch >= 1.4 ile) veya
`tensorboardX` ayrıca kurulmalıdır."), ilgili metin parçasını (mevcut
'`tensorboardX` `transformers` ile birlikte gelir' ifadesi) bulun ve değiştirin.
- Line 65: The doc incorrectly states TRAINER_REPORT_TO is an HF Transformers
convention; update the text around training.report_to to either remove the
TRAINER_REPORT_TO example or mark it explicitly as a ForgeLM-specific extension
(not part of transformers.TrainingArguments), and keep the valid --report-to CLI
override and reference to transformers.TrainingArguments.report_to; ensure the
sentence clearly distinguishes HF’s supported mechanisms (constructor /
--report-to) from any ForgeLM environment-variable shortcut (TRAINER_REPORT_TO)
if you choose to document it.

In `@tools/check_library_api_doc.py`:
- Line 2: The module docstring contains a non-English token ("Faz"); update the
top-level docstring in tools/check_library_api_doc.py to be English-only (e.g.,
change "Wave 6 / Faz 31 — `forgelm.__all__` ↔ library_api_reference.md
cross-check." to an all-English phrase such as "Wave 6 / Phase 31 —
`forgelm.__all__` ↔ library_api_reference.md cross-check.") so the module
docstring follows repository English-only docstring guidelines.
- Around line 109-111: The current check uses args.doc.exists() only, which
misses directories or unreadable files and lets later code (the open/read at the
early file handling) raise an exception; update the validation around args.doc
to ensure it both exists and is a regular file and is readable (use
args.doc.exists(), args.doc.is_file(), and os.access(args.doc, os.R_OK)) and if
any check fails print a clear stderr message and return 1; additionally wrap the
actual file open/read (the code that currently raises) in a try/except that
catches OSError/IOError, logs the error to stderr via the same message pattern,
and returns a non-zero exit code so failures are handled cleanly (refer to
args.doc, .is_file(), os.access, and the file open call in this module).

---

Nitpick comments:
In `@docs/guides/troubleshooting-tr.md`:
- Line 51: The markdown has inconsistent fenced-code formatting: add a blank
line before and after every fenced code block (e.g. the ```yaml block shown) to
satisfy MD031, add language specifiers for the error output blocks (use `text`
or `plaintext` for the blocks at the flagged positions such as the error
examples around lines 238, 252, 286, 357), and insert a blank line before the
list headed "**Sebepler:**". Update each fenced block accordingly across the
document so every triple-backtick block has a surrounding blank line and
appropriate language identifier.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 77d111ef-5773-4fd6-bf4c-885220e700fa

📥 Commits

Reviewing files that changed from the base of the PR and between ad32077 and 67ca824.

📒 Files selected for processing (53)
  • docs/design/gdpr_erasure.md
  • docs/guides/iso_soc2_deployer_guide-tr.md
  • docs/guides/iso_soc2_deployer_guide.md
  • docs/guides/library_api-tr.md
  • docs/guides/library_api.md
  • docs/guides/quickstart-tr.md
  • docs/guides/safety_compliance-tr.md
  • docs/guides/troubleshooting-tr.md
  • docs/guides/troubleshooting.md
  • docs/qms/access_control-tr.md
  • docs/qms/access_control.md
  • docs/reference/approvals_subcommand-tr.md
  • docs/reference/approvals_subcommand.md
  • docs/reference/compliance_summary-tr.md
  • docs/reference/compliance_summary.md
  • docs/reference/distributed_training-tr.md
  • docs/usermanuals/en/compliance/audit-log.md
  • docs/usermanuals/en/compliance/human-oversight.md
  • docs/usermanuals/en/compliance/overview.md
  • docs/usermanuals/en/data/audit.md
  • docs/usermanuals/en/data/secrets.md
  • docs/usermanuals/en/evaluation/auto-revert.md
  • docs/usermanuals/en/operations/cicd.md
  • docs/usermanuals/en/operations/experiment-tracking.md
  • docs/usermanuals/en/operations/webhooks.md
  • docs/usermanuals/en/training/dpo.md
  • docs/usermanuals/en/training/grpo.md
  • docs/usermanuals/en/training/kto.md
  • docs/usermanuals/en/training/orpo.md
  • docs/usermanuals/en/training/sft.md
  • docs/usermanuals/en/training/simpo.md
  • docs/usermanuals/tr/compliance/audit-log.md
  • docs/usermanuals/tr/compliance/human-oversight.md
  • docs/usermanuals/tr/compliance/overview.md
  • docs/usermanuals/tr/data/audit.md
  • docs/usermanuals/tr/data/secrets.md
  • docs/usermanuals/tr/evaluation/auto-revert.md
  • docs/usermanuals/tr/evaluation/judge.md
  • docs/usermanuals/tr/operations/cicd.md
  • docs/usermanuals/tr/operations/experiment-tracking.md
  • docs/usermanuals/tr/operations/webhooks.md
  • docs/usermanuals/tr/training/dpo.md
  • docs/usermanuals/tr/training/grpo.md
  • docs/usermanuals/tr/training/kto.md
  • docs/usermanuals/tr/training/orpo.md
  • docs/usermanuals/tr/training/sft.md
  • docs/usermanuals/tr/training/simpo.md
  • site/quickstart.html
  • tools/check_audit_event_catalog.py
  • tools/check_bilingual_code_blocks.py
  • tools/check_doc_numerical_claims.py
  • tools/check_library_api_doc.py
  • tools/check_yaml_snippets.py
💤 Files with no reviewable changes (1)
  • site/quickstart.html
✅ Files skipped from review due to trivial changes (2)
  • docs/reference/distributed_training-tr.md
  • docs/usermanuals/en/compliance/overview.md
🚧 Files skipped from review as they are similar to previous changes (28)
  • docs/usermanuals/tr/training/kto.md
  • docs/usermanuals/en/training/orpo.md
  • docs/guides/troubleshooting.md
  • docs/usermanuals/tr/operations/cicd.md
  • docs/usermanuals/en/data/audit.md
  • docs/usermanuals/tr/training/orpo.md
  • docs/guides/iso_soc2_deployer_guide.md
  • docs/usermanuals/en/training/simpo.md
  • docs/qms/access_control.md
  • docs/qms/access_control-tr.md
  • docs/usermanuals/tr/data/secrets.md
  • docs/usermanuals/en/training/dpo.md
  • tools/check_audit_event_catalog.py
  • docs/usermanuals/en/training/kto.md
  • docs/design/gdpr_erasure.md
  • docs/usermanuals/en/operations/experiment-tracking.md
  • docs/usermanuals/en/compliance/human-oversight.md
  • docs/usermanuals/tr/evaluation/auto-revert.md
  • docs/guides/iso_soc2_deployer_guide-tr.md
  • docs/usermanuals/en/evaluation/auto-revert.md
  • docs/usermanuals/tr/training/sft.md
  • tools/check_doc_numerical_claims.py
  • tools/check_bilingual_code_blocks.py
  • docs/usermanuals/tr/compliance/overview.md
  • docs/usermanuals/tr/training/dpo.md
  • docs/usermanuals/tr/evaluation/judge.md
  • docs/usermanuals/en/training/sft.md
  • tools/check_yaml_snippets.py

Comment thread docs/guides/quickstart-tr.md Outdated
Comment thread docs/guides/quickstart-tr.md Outdated
Comment thread docs/guides/safety_compliance-tr.md Outdated
Comment thread docs/reference/compliance_summary-tr.md
Comment thread docs/usermanuals/en/compliance/audit-log.md
Comment thread docs/usermanuals/tr/operations/experiment-tracking.md Outdated
Comment thread docs/usermanuals/tr/operations/experiment-tracking.md Outdated
Comment thread docs/usermanuals/tr/operations/experiment-tracking.md Outdated
Comment thread tools/check_library_api_doc.py Outdated
Comment thread tools/check_library_api_doc.py Outdated
Each finding verified against current code/state before applying;
EN+TR mirrors kept in lockstep. One finding skipped with explicit
reason (see below).

Quickstart (EN + TR):
  - quickstart-tr.md L145 / quickstart.md L133: fenced-block missing
    language tag (markdownlint MD040). Added `text` tag to the
    `final_model/` directory tree. EN parity fix included.
  - quickstart-tr.md L300-301 / quickstart.md L281: brittle
    `forgelm/config.py:589-635` line-anchor link → symbol-based
    reference to the `SyntheticConfig` Pydantic model with a
    repo-search link. EN parity fix included.

Safety compliance (EN + TR):
  - safety_compliance-tr.md L317-318 / safety_compliance.md L279-280:
    JSON example used non-canonical event names. Replaced
    `pipeline.started` → `training.started` and `training.completed` →
    `pipeline.completed` (canonical names per
    `docs/reference/audit_event_catalog.md`). EN parity fix included.

Compliance summary (EN):
  - compliance_summary.md L146-147: ISO control counts (~61/~32) did
    not match the underlying `iso27001_control_mapping.md`. Real
    counts: 11 FL + 48 FL-helps = 59 ForgeLM-evidence + 34 OOS = 93
    (TR was already correct at ~59/~34). Updated EN to ~59/~34 with
    explicit FL/FL-helps/OOS breakdown.

Audit log (EN + TR):
  - audit-log.md L62 / audit-log-tr.md L63: verify-audit example
    used `checkpoints/run/artifacts/audit_log.jsonl` despite the
    canonical contract being `<output_dir>/audit_log.jsonl`. Fixed
    both to the canonical placeholder.
  - audit-log-tr.md L71-73: stale paragraph still claimed a
    project-wide `.forgelm/global-audit-log.jsonl` directly above the
    new "no global log" paragraph. Removed; H2 retitled "Koşu başına"
    to match EN "Per-run".

Human oversight (EN + TR):
  - human-oversight.md / -tr.md L23-25: sequence diagram showed
    "Webhook + structured request" + "Sign approval (CLI / webhook
    callback)", but the page text already documents the CLI subcommand
    pair as the only mechanism (no webhook callback, no JWT resume
    endpoint). Diagram lines updated to "Webhook notification —
    informational only" + "CLI only (forgelm approve / forgelm
    reject)" + "HMAC chain" (replacing the phantom "ed25519 signature"
    framing).

Audit data report (EN + TR):
  - audit.md L116-142 / audit-tr.md L116-138: JSON snippet's
    `quality_flags` block used phantom field names (`short_response`,
    `abnormal_word_length`) and shipped a phantom `preference_specific`
    block (`identical_chosen_rejected`, `empty_chosen`,
    `swapped_likely`). Real schema in `_aggregator.py` +
    `_quality.py`: `quality_summary` with `samples_flagged`,
    `samples_evaluated`, `by_check` (containing the canonical 5 checks
    `low_alpha_ratio`, `low_punct_endings`,
    `abnormal_mean_word_length`, `short_paragraphs`, `repeated_lines`),
    and `overall_quality_score`. `preference_specific` block does not
    exist in code; removed. EN+TR aligned.

Operations / CI/CD (EN + TR):
  - cicd.md L122 / cicd-tr.md L123: Jenkins `sh '...'` quoting was
    broken — outer single quotes terminated on the inner jq filter's
    `'.verdict ...'`. Fixed by switching to triple single-quote
    Groovy strings so the jq filter is passed intact.

Operations / experiment tracking (EN + TR):
  - experiment-tracking-tr.md L78-79: typographic "30s'de" → "30 s'de"
    (TR style; EN already used "30s" as the ASCII-correct form, no
    change needed).
  - experiment-tracking.md L61 / -tr.md L61: "tensorboardX ships with
    transformers" was wrong — neither package is bundled. HF
    Transformers' `is_tensorboard_available` probes for `tensorboard`
    OR `tensorboardX` and `TensorBoardCallback` raises if neither is
    present. Updated to require operators to install `tensorboard`
    (PyTorch ≥ 1.4) or `tensorboardX` separately.
  - experiment-tracking.md L65 / -tr.md L65: `TRAINER_REPORT_TO=...`
    env var was presented as an HF Transformers convention; no such
    env var exists in transformers (verified by grep on the installed
    package), and ForgeLM does not honour it either. Removed the
    phantom example; kept the constructor + `--report-to` CLI override
    as the supported paths.

tools/check_library_api_doc.py:
  - L2: docstring contained the Turkish "Faz" — replaced with
    "Phase" (English-only docstring guideline).
  - L62-…: hardened `--doc` validation. Added `_validate_doc_arg`
    helper that checks `Path.exists()` + `Path.is_file()` +
    `os.access(..., os.R_OK)` and returns a clear stderr message
    mapped to exit 1 by main(). Also wrapped `doc_symbols(args.doc)`
    in `try/except OSError` to handle race conditions (file deleted
    between validation and read). Adds `os` + `Optional` imports.
    (Sonar pre-existing flag against the original validate-only
    pattern — now safe against directories, unreadable files, and
    racy deletes.)

Markdownlint (EN + TR troubleshooting):
  - troubleshooting.md / -tr.md: 4 untagged fenced error-output
    blocks (lines ~227, 239, 268, 337 in EN; corresponding lines in
    TR) were flagged by MD040. Added `text` language tags to all
    four; bilingual_code_blocks guard verifies count + lang stay in
    lockstep across the EN/TR pair.

Secrets reference (EN + TR):
  - data/secrets.md L97-111 / -tr.md L97-110: phantom
    `ingestion.secrets_mask:` config block (with
    `enabled` / `tag_by_category` / `strict` / `categories` sub-fields)
    documented features that do not exist — `secrets_mask` is a
    `bool` argument on `audit_dataset()`. Replaced the phantom YAML
    block with a prose paragraph stating the always-on contract and
    pointing at `_SECRET_PATTERNS` for extension. (This was a related
    upgrade to the L26 finding: the original audit comment asked for
    a "default-vs-opt-in" clarification of `tag_by_category`, but the
    field is phantom — removing the block is the correct fix and
    makes the L26 sentence factually correct without further change.)

Skipped findings:
  - data/secrets.md L26: skipped as written. The audit comment asked
    for a "[REDACTED-SECRET] is the default behaviour, tag_by_category
    is an opt-in mode" clarification. `tag_by_category` doesn't exist
    in the codebase (verified: not in `forgelm/data_audit/_secrets.py`
    or `forgelm/ingestion.py`), so adding a pointer to it would
    document a phantom feature. Resolved by removing the phantom
    config block (above) instead — the surviving L26 sentence is
    already accurate as-is.

Validation: 8/8 doc CI guards green; ruff check + format clean on
tools/; user manuals regenerated. New `_validate_doc_arg` smoke-tested
against `--doc /tmp` (returns "is not a regular file." + exit 1).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cemililik

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 8, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@cemililik

Copy link
Copy Markdown
Collaborator Author

Review absorption — coderabbitai run @ commit b275e18

15 inline findings + 1 nitpick from the latest CodeRabbit review pass; all verified against current code before action. EN+TR mirrors kept in lockstep. One finding skipped with explicit reason; rest landed in commit b275e18.

Fixed (15)

# Finding Verification Fix
1 quickstart-tr.md:152 MD040 missing lang tag EN parity check showed same issue EN+TR: ``` → ```text on final_model/ tree
2 quickstart-tr.md:301 brittle config.py:589-635 line-anchor Confirmed line-anchor pattern; symbol exists EN+TR: replaced with SyntheticConfig symbol ref + repo-search link
3 safety_compliance-tr.md:319 non-canonical events audit_event_catalog.md confirms training.started / pipeline.completed canonical EN+TR: pipeline.started → training.started, training.completed → pipeline.completed
4 compliance_summary-tr.md:143 ISO drift ~59/~34 vs EN ~61/~32 Canonical mapping: 11 FL + 48 FL-helps = 59 evidence + 34 OOS = 93. TR was correct, EN was wrong. Fixed EN to ~59/~34 with explicit FL/FL-helps/OOS breakdown
5 audit-log.md:77 path contract drift Verify-audit example used checkpoints/run/artifacts/audit_log.jsonl EN+TR: <output_dir>/audit_log.jsonl placeholder
6 secrets.md:26 clarify default vs opt-in Verified phantom: tag_by_category does not exist in forgelm/data_audit/_secrets.py or forgelm/ingestion.py. Underlying problem: phantom ingestion.secrets_mask: block at L97-111 with tag_by_category/enabled/strict/categories. Resolved by removing the phantom block — surviving L26 sentence is already accurate. (See "Skipped" below for the literal L26 ask.)
7 cicd.md:122 Jenkins sh '...' quoting Outer single + inner single in jq filter = parser break (Groovy) EN+TR: triple single-quote sh '''…'''
8 audit-log-tr.md:80 "Koşu başı vs proje başı" contradiction Stale .forgelm/global-audit-log.jsonl paragraph above the new "no global log" paragraph TR: removed stale paragraph; H2 retitled "Koşu başına"
9 human-oversight-tr.md:64 diagram mismatches text Page text already says CLI-only; diagram still showed "CLI / webhook callback" EN+TR diagram lines updated to "CLI only (forgelm approve / forgelm reject)" + "Webhook notification — informational only" + "HMAC chain"
10 audit-tr.md:135 JSON schema wrong Real schema verified in _aggregator.py + _quality.py: quality_summary with samples_flagged / samples_evaluated / by_check (5 canonical checks: low_alpha_ratio, low_punct_endings, abnormal_mean_word_length, short_paragraphs, repeated_lines) / overall_quality_score. preference_specific block does NOT exist. EN+TR: replaced phantom quality_flags + preference_specific with canonical quality_summary schema
11 experiment-tracking-tr.md:61 "tensorboardX ships with transformers" Verified false: transformers.integrations.is_tensorboard_available PROBES for either package; TensorBoardCallback raises if neither installed EN+TR: rewrote to "install tensorboard (PyTorch ≥ 1.4) or tensorboardX separately"
12 experiment-tracking-tr.md:65 TRAINER_REPORT_TO env var grep'ed installed transformers package + ForgeLM source: env var does not exist anywhere EN+TR: removed the phantom env-var example; kept --report-to CLI override + constructor as supported paths
13 experiment-tracking-tr.md:79 "30s'de" typography TR locale style preference TR: "30s'de" → "30 s'de" (×2 lines)
14 check_library_api_doc.py:2 "Faz" in docstring Mixed-language token confirmed "Wave 6 / Faz 31" → "Wave 6 / Phase 31"
15 check_library_api_doc.py:111 exists() only Confirmed: directory or unreadable file would crash with traceback Added _validate_doc_arg helper (Path.is_file() + os.access(..., os.R_OK)) + try/except OSError around doc_symbols(args.doc) for race-safety. Smoke-tested with --doc /tmp → "is not a regular file." + exit 1

Skipped (1, with reason)

  • secrets.md:26 literal ask ("clarify [REDACTED-SECRET] is default; tag_by_category is opt-in"): skipped. tag_by_category doesn't exist in the codebase (verified: not in forgelm/data_audit/_secrets.py or forgelm/ingestion.py). Adding a pointer to it would document a phantom feature. Resolved the underlying problem instead by removing the phantom ingestion.secrets_mask: config block at L97-111 (see Improve error message when dataset format doesn't match trainer_type #6). The L26 sentence is already factually correct as-is.

Nitpick (1, partial)

  • troubleshooting-tr.md:51 MD031 / MD032 / MD040 cluster: addressed the MD040 part (4 untagged error-output blocks → ```text, EN parity preserved). MD031 (blank-line-around-fence) and MD032 (blank-line-before-list) are stylistic; deferred — they don't block any of the 8 doc CI guards or markdownlint --config .markdownlint.json (we don't run markdownlint in CI). Happy to land the rest in a separate PR if it's blocking.

Validation

8/8 doc CI guards green: bilingual_parity (45 pairs) · anchor_resolution (273 markdown files) · cli_help_consistency (439 invocations) · yaml_snippets (23 ForgeConfig-shaped, 323 total) · audit_event_catalog (36 events) · library_api_doc (30 symbols + 2 methods) · doc_numerical_claims (4 scrapes) · bilingual_code_blocks (45 pairs).

ruff check tools/ + ruff format --check tools/ clean. User manuals regenerated (tools/build_usermanuals.py).

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 8, 2026

Copy link
Copy Markdown
Contributor

@cemililik: Excellent absorption pass — the documentation and verification trail here is unusually thorough. A few highlights worth calling out:

  • Finding #4 (ISO drift) — good catch that TR was actually correct and EN was the drifted copy. The explicit FL/FL-helps/OOS breakdown makes the fix auditor-friendly.
  • Finding #6 (phantom ingestion.secrets_mask: block) — resolving the root cause (removing the phantom block) rather than papering over the symptom with a pointer to a non-existent feature is exactly the right call. The skip rationale is sound.
  • Finding #10 (canonical quality_summary schema) — verifying against _aggregator.py + _quality.py directly before replacing the phantom quality_flags/preference_specific schema is the correct approach for a compliance-sensitive doc.
  • Finding #15 (_validate_doc_arg helper) — the Path.is_file() + os.access(..., os.R_OK) + try/except OSError layering correctly handles the directory, unreadable-file, and TOCTOU race cases that a bare .exists() misses.

The MD031/MD032 deferral is reasonable given they're not gated by any of the 8 CI guards; a follow-up cleanup PR makes sense there.

Triggering the review now.

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
docs/usermanuals/en/operations/experiment-tracking.md (1)

8-8: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Remove unsupported Comet ML references from line 8.

Line 8 claims Comet ML is "first-class" but the config schema (forgelm/config.py:274) only supports tensorboard, wandb, mlflow, and none. Comet is not implemented. Lines 110–112 are fine (they reference Comet only in the context of a planned comparison-runs feature, not current tracking backend support).

This affects both the English and Turkish mirrors identically.

Fix for line 8 (English and Turkish)

English:

-ForgeLM doesn't reinvent experiment tracking — it integrates with whatever your team already uses via the `training.report_to` field. W&B, MLflow, TensorBoard, and Comet ML are first-class.
+ForgeLM doesn't reinvent experiment tracking — it integrates with whatever your team already uses via the `training.report_to` field. W&B, MLflow, and TensorBoard are first-class.

Turkish:

-ForgeLM deney takibini yeniden icat etmez — ekibinizin zaten kullandığı şeye `training.report_to` alanı üzerinden entegre olur. W&B, MLflow, TensorBoard ve Comet ML birinci-sınıf desteklenir.
+ForgeLM deney takibini yeniden icat etmez — ekibinizin zaten kullandığı şeye `training.report_to` alanı üzerinden entegre olur. W&B, MLflow ve TensorBoard birinci-sınıf desteklenir.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/operations/experiment-tracking.md` at line 8, Update the
docs text on line 8 to remove Comet ML as a "first-class" experiment tracking
backend: edit the sentence referencing training.report_to so it lists only the
supported backends (tensorboard, wandb, mlflow, none) and remove Comet ML from
both the English and Turkish files, and ensure this change matches the actual
config schema in forgelm/config.py (see the training.report_to allowed values)
so the docs and implementation stay consistent.
docs/usermanuals/tr/compliance/human-oversight.md (1)

50-57: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Keep CLI output examples in English-only.

Line 51 localizes terminal output; this breaks the repo rule for CLI/output snippets and can drift from real command output. Keep prose Turkish, but terminal output text in English.

Suggested patch
-[2026-04-29 14:33:10] İnsan onayı gerekli.
+[2026-04-29 14:33:10] Human approval required.

As per coding guidelines: “docs/**/*.md: User-facing documentation must be bilingual (English + Turkish mirrors); code, CLI output, logs, and config keys must be English only”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/tr/compliance/human-oversight.md` around lines 50 - 57, The
CLI output block in human-oversight.md contains localized Turkish terminal
lines; change the terminal output to English-only while keeping surrounding
prose Turkish if desired. Specifically update the quoted output strings—replace
"İnsan onayı gerekli." with "Human approval required.", "Onaylamak için:" with
"To approve:", and "Reddetmek için:" with "To reject:" (leave commands like
"forgelm approve abc123 --output-dir ..." unchanged) so the CLI/log snippet
matches the repo rule that code/CLI output/logs/config keys are English only.
🧹 Nitpick comments (1)
docs/usermanuals/en/data/audit.md (1)

93-93: ⚡ Quick win

Remove hardcoded parser line numbers from the authoritative-source reference.

This will drift on routine parser edits. Prefer file + symbol-only reference, consistent with the doc-link durability convention.

Proposed edit
-Authoritative source: `forgelm/cli/_parser.py::_add_audit_subcommand` (lines 335-468).
+Authoritative source: `forgelm/cli/_parser.py::_add_audit_subcommand`.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/usermanuals/en/data/audit.md` at line 93, Update the
authoritative-source reference in docs/usermanuals/en/data/audit.md by removing
the hardcoded line number range and referencing only the file and symbol;
replace "forgelm/cli/_parser.py::_add_audit_subcommand (lines 335-468)" with
"forgelm/cli/_parser.py::_add_audit_subcommand" so the doc points to the file
and the _add_audit_subcommand symbol only (no line numbers) to avoid drift when
the parser changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/guides/safety_compliance-tr.md`:
- Line 353: The example JSON uses a stale value for "forgelm_version" (currently
set to "0.1.0"); update the example in the compliance_report.json snippet by
replacing "forgelm_version": "0.1.0" with a current version string or a neutral
placeholder like "x.y.z" (or the real latest release), ensuring the key
"forgelm_version" remains unchanged so users aren't misled by an outdated
version in the docs.
- Around line 3-5: Replace the absolute claim "tek açık-kaynak fine-tuning
aracıdır" with non-exclusive wording; locate the sentence containing that phrase
in docs/guides/safety_compliance-tr.md and change it to a qualified statement
such as "açık-kaynak fine-tuning araçlarından biridir" or "mevcut açık-kaynak
araçlar arasında yer alır," or add a citation/benchmark to substantiate
exclusivity if you must keep a stronger claim; ensure the revised sentence
preserves the original meaning about integrated safety assessment and EU AI Act
artifact generation.

In `@docs/guides/troubleshooting-tr.md`:
- Line 200: Replace the incorrect key name used in the Turkish prose: change the
reference from `max_tokens` to `max_new_tokens` so it matches the configuration
block; update the sentence on line 200 in docs/guides/troubleshooting-tr.md to
mention `max_new_tokens` (and, if applicable, ensure any explanatory text around
the teacher model suggestion also refers to `max_new_tokens`).

In `@docs/guides/troubleshooting.md`:
- Line 191: Summary: The doc uses the outdated token key name `max_tokens`
instead of the section's actual config name `max_new_tokens`. Fix: update the
sentence that currently reads “reducing `max_tokens`” to use the exact config
key `max_new_tokens` (or fully-qualified `synthetic.max_new_tokens` where
appropriate) so it matches the section’s config examples and avoids copy/paste
confusion; search for occurrences of `max_tokens` in this troubleshooting
paragraph and replace them with `max_new_tokens` (or `synthetic.max_new_tokens`)
to keep naming consistent with the rest of the document.
- Line 388: Update the documentation line that shows the approval workflow to
include the required --comment flag on both commands: change the examples using
the commands `forgelm approve <run_id> --output-dir <dir>` and `forgelm reject
<run_id> --output-dir <dir>` to `forgelm approve <run_id> --output-dir <dir>
--comment "<your comment>"` and `forgelm reject <run_id> --output-dir <dir>
--comment "<your comment>"` respectively so users following the `Awaiting
approval` guidance won't hit missing-comment errors; keep the staging path
(`<output_dir>/final_model.staging.<run_id>/`) text unchanged.

In `@docs/usermanuals/en/data/secrets.md`:
- Line 26: The docs contradict themselves about Anthropic coverage: update the
"False-positive guards" section so it no longer claims a built-in Anthropic-key
entropy guard if Anthropic patterns are not bundled; specifically edit the
description near the entropy-guard examples to either remove or reword the
Anthropic mention and add a clarifying note that mask_secrets()
(forgelm/data_audit/_secrets.py::mask_secrets) does not ship Anthropic patterns
by default and operators must extend the regex set out-of-tree if they need
Anthropic detection.

---

Outside diff comments:
In `@docs/usermanuals/en/operations/experiment-tracking.md`:
- Line 8: Update the docs text on line 8 to remove Comet ML as a "first-class"
experiment tracking backend: edit the sentence referencing training.report_to so
it lists only the supported backends (tensorboard, wandb, mlflow, none) and
remove Comet ML from both the English and Turkish files, and ensure this change
matches the actual config schema in forgelm/config.py (see the
training.report_to allowed values) so the docs and implementation stay
consistent.

In `@docs/usermanuals/tr/compliance/human-oversight.md`:
- Around line 50-57: The CLI output block in human-oversight.md contains
localized Turkish terminal lines; change the terminal output to English-only
while keeping surrounding prose Turkish if desired. Specifically update the
quoted output strings—replace "İnsan onayı gerekli." with "Human approval
required.", "Onaylamak için:" with "To approve:", and "Reddetmek için:" with "To
reject:" (leave commands like "forgelm approve abc123 --output-dir ..."
unchanged) so the CLI/log snippet matches the repo rule that code/CLI
output/logs/config keys are English only.

---

Nitpick comments:
In `@docs/usermanuals/en/data/audit.md`:
- Line 93: Update the authoritative-source reference in
docs/usermanuals/en/data/audit.md by removing the hardcoded line number range
and referencing only the file and symbol; replace
"forgelm/cli/_parser.py::_add_audit_subcommand (lines 335-468)" with
"forgelm/cli/_parser.py::_add_audit_subcommand" so the doc points to the file
and the _add_audit_subcommand symbol only (no line numbers) to avoid drift when
the parser changes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: bb3efd8f-13e4-4b01-80d9-78f2200a52ff

📥 Commits

Reviewing files that changed from the base of the PR and between 67ca824 and b275e18.

📒 Files selected for processing (20)
  • docs/guides/quickstart-tr.md
  • docs/guides/quickstart.md
  • docs/guides/safety_compliance-tr.md
  • docs/guides/safety_compliance.md
  • docs/guides/troubleshooting-tr.md
  • docs/guides/troubleshooting.md
  • docs/reference/compliance_summary.md
  • docs/usermanuals/en/compliance/audit-log.md
  • docs/usermanuals/en/compliance/human-oversight.md
  • docs/usermanuals/en/data/audit.md
  • docs/usermanuals/en/data/secrets.md
  • docs/usermanuals/en/operations/cicd.md
  • docs/usermanuals/en/operations/experiment-tracking.md
  • docs/usermanuals/tr/compliance/audit-log.md
  • docs/usermanuals/tr/compliance/human-oversight.md
  • docs/usermanuals/tr/data/audit.md
  • docs/usermanuals/tr/data/secrets.md
  • docs/usermanuals/tr/operations/cicd.md
  • docs/usermanuals/tr/operations/experiment-tracking.md
  • tools/check_library_api_doc.py
✅ Files skipped from review due to trivial changes (2)
  • docs/guides/safety_compliance.md
  • docs/usermanuals/tr/data/secrets.md
🚧 Files skipped from review as they are similar to previous changes (6)
  • docs/reference/compliance_summary.md
  • docs/usermanuals/en/operations/cicd.md
  • docs/guides/quickstart.md
  • tools/check_library_api_doc.py
  • docs/usermanuals/en/compliance/human-oversight.md
  • docs/usermanuals/tr/operations/experiment-tracking.md

Comment thread docs/guides/safety_compliance-tr.md Outdated
Comment thread docs/guides/safety_compliance-tr.md Outdated
Comment thread docs/guides/troubleshooting-tr.md Outdated
Comment thread docs/guides/troubleshooting.md Outdated
| `2` | Training error | Check GPU, memory, dependencies |
| `3` | Evaluation failure | Model quality below threshold — adjust thresholds or improve data |
| `4` | Awaiting approval | Human review required — review results in `checkpoints/compliance/` and redeploy when ready |
| `4` | Awaiting approval | Human review required — run `forgelm approvals --show <run_id> --output-dir <dir>` to inspect the staging directory, then `forgelm approve <run_id> --output-dir <dir>` to promote or `forgelm reject <run_id> --output-dir <dir>` to discard. The staging path is `<output_dir>/final_model.staging.<run_id>/`. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Add required --comment to approval commands in exit-code guidance.

Line 388 shows forgelm approve / forgelm reject without --comment; that can fail for users following this exact command path. Please include the required comment flag in both examples.

Suggested patch
-| `4` | Awaiting approval | Human review required — run `forgelm approvals --show <run_id> --output-dir <dir>` to inspect the staging directory, then `forgelm approve <run_id> --output-dir <dir>` to promote or `forgelm reject <run_id> --output-dir <dir>` to discard. The staging path is `<output_dir>/final_model.staging.<run_id>/`. |
+| `4` | Awaiting approval | Human review required — run `forgelm approvals --show <run_id> --output-dir <dir>` to inspect the staging directory, then `forgelm approve <run_id> --output-dir <dir> --comment "<note>"` to promote or `forgelm reject <run_id> --output-dir <dir> --comment "<note>"` to discard. The staging path is `<output_dir>/final_model.staging.<run_id>/`. |
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
| `4` | Awaiting approval | Human review required — run `forgelm approvals --show <run_id> --output-dir <dir>` to inspect the staging directory, then `forgelm approve <run_id> --output-dir <dir>` to promote or `forgelm reject <run_id> --output-dir <dir>` to discard. The staging path is `<output_dir>/final_model.staging.<run_id>/`. |
| `4` | Awaiting approval | Human review required — run `forgelm approvals --show <run_id> --output-dir <dir>` to inspect the staging directory, then `forgelm approve <run_id> --output-dir <dir> --comment "<note>"` to promote or `forgelm reject <run_id> --output-dir <dir> --comment "<note>"` to discard. The staging path is `<output_dir>/final_model.staging.<run_id>/`. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/guides/troubleshooting.md` at line 388, Update the documentation line
that shows the approval workflow to include the required --comment flag on both
commands: change the examples using the commands `forgelm approve <run_id>
--output-dir <dir>` and `forgelm reject <run_id> --output-dir <dir>` to `forgelm
approve <run_id> --output-dir <dir> --comment "<your comment>"` and `forgelm
reject <run_id> --output-dir <dir> --comment "<your comment>"` respectively so
users following the `Awaiting approval` guidance won't hit missing-comment
errors; keep the staging path (`<output_dir>/final_model.staging.<run_id>/`)
text unchanged.

Comment thread docs/usermanuals/en/data/secrets.md
Each finding verified against current code/state before action; EN+TR
mirrors kept in lockstep. One finding skipped with explicit reason.

Fixed:

  - safety_compliance-tr.md L353 / safety_compliance.md L308:
    `forgelm_version` JSON example was stale (`0.1.0`) — pyproject.toml
    declares `0.5.5`. EN+TR updated to current release.

  - safety_compliance-tr.md L3-5 / safety_compliance.md L3:
    "tek açık-kaynak fine-tuning aracıdır" / "the only open-source
    fine-tuning tool" was an unsubstantiated exclusivity claim. EN+TR
    softened to "açık-kaynak fine-tuning araçlarından biridir" /
    "one of the open-source fine-tuning tools that ships ... in the
    same pipeline" — preserves the integrated-safety-+-Annex-IV
    framing without the comparative claim.

  - troubleshooting.md L191 / troubleshooting-tr.md L200: prose
    referenced phantom `max_tokens` knob. Real schema is
    `synthetic.max_new_tokens`
    (`SyntheticConfig.max_new_tokens: int = 1024` in
    `forgelm/config.py`). EN+TR aligned to fully-qualified
    `synthetic.max_new_tokens`.

  - usermanuals/en/data/audit.md L93: removed brittle "(lines 335-468)"
    line-range from the parser source ref; symbol-only
    (`forgelm/cli/_parser.py::_add_audit_subcommand`) is the
    drift-resistant form per project convention. (TR mirror was
    already symbol-only.)

  - usermanuals/{en,tr}/operations/experiment-tracking.md L8:
    "Comet ML is first-class" was wrong — `TrainingConfig.report_to`
    Literal is `"tensorboard" | "wandb" | "mlflow" | "none"` (verified
    in `forgelm/config.py:274`); no Comet ML branch ships. EN+TR
    rewrote to enumerate the four Literal values explicitly with a
    pointer to the schema. (Other "Comet UI" references at L110/L112/
    L138 are about Comet's own UI as a third-party service, not
    ForgeLM integration — left as-is.)

  - usermanuals/{en,tr}/data/secrets.md L91: "Entropy threshold for
    OpenAI / Anthropic keys" contradicted L26 (which states the
    detector does NOT ship Anthropic / Stripe / SendGrid / Twilio
    patterns). Verified `_SECRET_PATTERNS` in
    `forgelm/data_audit/_secrets.py`: only `openai_api_key` regex
    exists; no `anthropic_*` entry. EN+TR scoped the entropy guard
    to OpenAI keys + cross-referenced the L26 statement.

  - usermanuals/tr/compliance/human-oversight.md L51-56: terminal-output
    block contained Turkish phrases ("İnsan onayı gerekli", "Onaylamak
    için", "Reddetmek için") that conflict with the project rule
    "code/CLI output/logs/config keys are English only" (per CLAUDE.md
    + docs/standards/localization.md). Translated the prompt strings
    to English; surrounding TR prose retained.

Skipped (1):

  - troubleshooting.md L388 ("add required --comment to approve/reject
    examples"): verified against
    `forgelm/cli/_parser.py:549-555` and `:981-985` — `--comment` is
    declared with `default=None` and `args.comment or ""` is used at
    `_approve.py:459` / `:533`. Comment is **optional**, not required;
    users following the doc do NOT hit a missing-comment error. The
    reviewer's premise was incorrect, so the example is left as the
    minimal form. (The other human-oversight doc shows the
    `--comment "..."` form for operators who DO want to record one.)

Validation: 8/8 doc CI guards green; ruff check clean; user manuals
regenerated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cemililik

Copy link
Copy Markdown
Collaborator Author

Review absorption — sweep #4 @ commit 755eaf7

8 findings (6 inline + 1 outside-diff + 1 nitpick) verified against current code; 7 fixed, 1 skipped with reason.

Fixed (7)

# Finding Verification Fix
1 safety_compliance-tr.md:353 (+ EN parity) forgelm_version: "0.1.0" stale pyproject.toml declares version = "0.5.5" EN+TR: bumped to "0.5.5"
2 safety_compliance-tr.md:3-5 (+ EN parity) "tek açık-kaynak ... aracıdır" / "the only open-source" Unsubstantiated exclusivity claim EN+TR: softened to "araçlarından biridir" / "one of the open-source tools that ships ... in the same pipeline"
3 troubleshooting.md:191 / troubleshooting-tr.md:200 max_tokens Real schema is synthetic.max_new_tokens (verified SyntheticConfig.max_new_tokens: int = 1024 in forgelm/config.py) EN+TR: → synthetic.max_new_tokens
4 audit.md:93 (lines 335-468) brittle line-range Symbol-only convention is the drift-resistant form EN: removed line range; TR mirror was already symbol-only
5 experiment-tracking.md:8 (+TR) "Comet ML is first-class" TrainingConfig.report_to: Literal["tensorboard","wandb","mlflow","none"] — no Comet ML branch EN+TR: rewrote to enumerate the four real Literals with pointer to forgelm/config.py
6 secrets.md:91 (+TR) "OpenAI / Anthropic entropy threshold" contradicts L26 Verified _SECRET_PATTERNS: only openai_api_key regex; no anthropic_* entry EN+TR: scoped entropy guard to OpenAI + back-reference to L26
7 human-oversight-tr.md:51-56 Turkish in CLI-output block Project rule: code/CLI output English-only TR: translated prompt strings to English; surrounding TR prose retained

Skipped (1, with reason)

  • troubleshooting.md:388 ("add required --comment flag"): verified false premise. --comment is declared with default=None in forgelm/cli/_parser.py:549-555 (approve) and :981-985 (reject); the dispatchers use args.comment or "" at _approve.py:459 / :533. Comment is optional, not required — users following the minimal example do NOT hit a missing-comment error. The other human-oversight doc already shows the --comment "..." form for operators who want to record one. Left example as-is.

Validation

8/8 doc CI guards green; ruff check tools/ clean; user manuals regenerated.

@codacy-production

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 1 critical · 99 minor

Alerts:
⚠ 100 issues (≤ 0 issues of at least minor severity)

Results:
100 new issues

Category Results
Security 1 critical
CodeStyle 99 minor

View in Codacy

🟢 Metrics 160 complexity · 2 duplication

Metric Results
Complexity 160
Duplication 2

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@sonarqubecloud

sonarqubecloud Bot commented May 8, 2026

Copy link
Copy Markdown

@cemililik
cemililik merged commit ba8ffce into main May 8, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant