[Wave 4] Low/info — Docs: docs/usermanuals/en/evaluation/ #787

Wave 4 · 8 finding(s) in 8 fix package(s). Documents with fewer than five findings each in this wave, grouped by directory.

Findings

• DOC-BLEGAL-228 (docs, Low) — docs/usermanuals/en/evaluation/judge.md: stale — gpt-4o ~$5 / 1K judgements; gpt-4o-mini ~$1 / 1K; '200 prompts × $0.005 (gpt-4o-mini) ≈ $1 per run'
  severity Low
  Where. docs/usermanuals/en/evaluation/judge.md:69-70,80 (Turkish mirror: docs/usermanuals/tr/evaluation/judge.md:69-70,80) · type: stale
  The document says. gpt-4o $5 / 1K judgements; gpt-4o-mini $1 / 1K; '200 prompts × $0.005 (gpt-4o-mini) ≈ $1 per run'
  What the code does. Current list prices are gpt-4o $2.50/$10.00 and gpt-4o-mini $0.15/$0.60 per 1M input/output tokens, and Claude Haiku 4.5 $1/$5. At ForgeLM's prompt size this gives roughly $2.0-3.0 per 1K for gpt-4o, $0.12-0.21 per 1K for gpt-4o-mini and $0.8-1.6 per 1K for Haiku 4.5. That is the researcher's estimate. The gpt-4o row is about 2x high, the mini row 5-8x high, and line 80 is far high. Line 17 ($1-5…
  Proposed correction. gpt-4o '$2-3', gpt-4o-mini '$0.15-0.25', line 80 '200 prompts ≈ $0.03-0.05 with gpt-4o-mini', with a note that these are list-price estimates as of 2026-10. TR: aynı rakamları ve tarih notunu ekleyin.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D3B08-084 (docs, Low) — docs/usermanuals/en/evaluation/judge.md: misleading — The gate fails only when the mean drops below the floor. Line 42 quotes the rubric as 'Score this response on a 1-10 scale for helpfulness and co…
  severity Low
  Where. docs/usermanuals/en/evaluation/judge.md:8 (Turkish mirror: docs/usermanuals/tr/evaluation/judge.md:8) · type: misleading
  The document says. The gate fails only when the mean drops below the floor. Line 42 quotes the rubric as 'Score this response on a 1-10 scale for helpfulness and correctness.'
  What the code does. The gate also fails when fewer than min_valid_fraction (default 0.8) of prompts yield a parseable score, and when no score parses at all (config.py:1114-1126; judge.py:810-830). The built-in rubric scores helpfulness, accuracy, clarity and instruction-following, and asks for JSON {score, reason} (judge.py:34-55).
  Code references. forgelm/config.py:1114-1126, forgelm/judge.py:810-830, forgelm/judge.py:34-55
  Proposed correction. Add: "The gate also fails when fewer than min_valid_fraction (default 0.8) of prompts yield a parseable score." Quote the four rubric criteria accurately.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D3B08-085 (docs, Low) — docs/usermanuals/en/evaluation/judge.md: wrong detail — batch_size is '(prompt, completion) pairs scored per round'.
  severity Low
  Where. docs/usermanuals/en/evaluation/judge.md:30 (Turkish mirror: docs/usermanuals/tr/evaluation/judge.md:30) · type: wrong detail
  The document says. batch_size is '(prompt, completion) pairs scored per round'.
  What the code does. batch_size only batches the fine-tuned model's response generation. Judge calls are always one per prompt (judge.py:704-714, 735-738; config.py:1109-1113).
  Code references. forgelm/judge.py:704-738, forgelm/config.py:1109-1113
  Proposed correction. batch_size: 8 # batched generation of the trained model's responses; judge calls stay one per prompt
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D3B08-086 (docs, Low) — docs/usermanuals/en/evaluation/judge.md: wrong detail — '200 prompts × $0.005 (gpt-4o-mini) ≈ $1 per run' versus line 70: gpt-4o-mini ~$1 per 1K judgements.
  severity Low
  Where. docs/usermanuals/en/evaluation/judge.md:80 (Turkish mirror: docs/usermanuals/tr/evaluation/judge.md:80) · type: wrong detail
  The document says. '200 prompts × $0.005 (gpt-4o-mini) ≈ $1 per run' versus line 70: gpt-4o-mini ~$1 per 1K judgements.
  What the code does. The page contradicts itself by 5x: $1/1K is $0.001 per call, not $0.005. The actual prices are external and listed under needs_research.
  Proposed correction. Make the two figures consistent after checking current provider pricing, or drop the dollar figures.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D3B08-091 (docs, Low) — docs/usermanuals/en/evaluation/trend-tracking.md: wrong detail — Trend rows carry timestamps like "2026-04-29T14:33:04Z".
  severity Low
  Where. docs/usermanuals/en/evaluation/trend-tracking.md:15-17 (Turkish mirror: docs/usermanuals/tr/evaluation/trend-tracking.md:15) · type: wrong detail
  The document says. Trend rows carry timestamps like "2026-04-29T14:33:04Z".
  What the code does. Timestamps are datetime.now(timezone.utc).isoformat(), i.e. '...+00:00' with microseconds (_results.py:160). safety.md:195 shows the correct form.
  Code references. forgelm/safety/_results.py:159-164
  Proposed correction. Use "2026-04-29T14:33:04.512331+00:00" style timestamps.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D3B08-092 (docs, Low) — docs/usermanuals/en/evaluation/trend-tracking.md: wrong detail — 'the default training.output_dir is typically per-run'.
  severity Low
  Where. docs/usermanuals/en/evaluation/trend-tracking.md:84 (Turkish mirror: docs/usermanuals/tr/evaluation/trend-tracking.md) · type: wrong detail
  The document says. 'the default training.output_dir is typically per-run'.
  What the code does. The default is the fixed './checkpoints' (config.py:451-453). Runs on the default therefore share one trend file.
  Code references. forgelm/config.py:451-453
  Proposed correction. "The default training.output_dir is the fixed ./checkpoints, so runs on defaults accumulate in one file; runs that set a per-run output_dir do not."
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D3B08-093 (docs, Low) — docs/usermanuals/en/evaluation/trend-tracking.md: wrong detail — Correlate trend rows with audit_log.jsonl's training_started / training_completed events.
  severity Low
  Where. docs/usermanuals/en/evaluation/trend-tracking.md:102 (Turkish mirror: docs/usermanuals/tr/evaluation/trend-tracking.md:102) · type: wrong detail
  The document says. Correlate trend rows with audit_log.jsonl's training_started / training_completed events.
  What the code does. No such events exist. The trainer emits training.started (trainer.py:1574) and pipeline.completed (trainer.py:1657); safety itself emits safety.evaluation_completed (trainer.py:1445).
  Code references. forgelm/trainer.py:1574, forgelm/trainer.py:1657, forgelm/trainer.py:1445
  Proposed correction. "... audit_log.jsonl's training.started / safety.evaluation_completed / pipeline.completed events ..."
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D3B08-094 (docs, Low) — docs/usermanuals/en/evaluation/trend-tracking.md: wrong detail — 'only the current run's max_safety_regression / min_safety_score gates the exit code'.
  severity Low
  Where. docs/usermanuals/en/evaluation/trend-tracking.md:111 (Turkish mirror: docs/usermanuals/tr/evaluation/trend-tracking.md) · type: wrong detail
  The document says. 'only the current run's max_safety_regression / min_safety_score gates the exit code'.
  What the code does. severity_thresholds also gate (_gates.py:269-277). In training a failed safety gate changes the exit code only with auto_revert: true; with the default false the run exits 0 (trainer.py:1461-1463; cli/_training.py:161).
  Code references. forgelm/safety/_gates.py:269-277, forgelm/trainer.py:1461-1466, forgelm/cli/_training.py:161
  Proposed correction. "... only the current run's safety gates (max_safety_regression, min_safety_score, severity_thresholds) decide pass/fail, and in training they change the exit code only with evaluation.auto_revert: true."
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}