Skip to content

gpu-cost.md (EN+TR) says no rate setting exists and hides the real training.gpu_cost_per_hour, and documents a --gpus flag, a run_complete event and a model-card cost section that… (CR-A01-023) #429

Open

Description

@cemililik

Wave 1 · theme D1-8: Docs describing features that do not exist (doc side of code findings) · highest severity High · 1 finding(s) · effort small

Summary

An operator who wants their real cloud rate in the compliance report is told no config exists, so they never set training.gpu_cost_per_hour and the report keeps the built-in averages. Or they try the documented output.cost_tracking and hit exit 1. Those who look for a run_complete audit event or pass --gpus find neither.

Primary finding

CR-A01-023 — gpu-cost.md (EN+TR) says no rate setting exists and hides the real training.gpu_cost_per_hour, and documents a --gpus flag, a run_complete event and a model-card cost section that do not exist

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace)

What happens. Item (2) prose verdict: the claim at :95/:104 that the per-GPU rate is multiplied by the GPU count is accurate as arithmetic. trainer.py:1889 computes gpu_hours = runtime/3600 × gpu_count, and :1895 multiplies by the rate, so 4 × 2 h × $1.10 = $8.80 holds. The rate source the page shows (output.cost_tracking) is rejected at config load, however. The real sources are training.gpu_cost_per_hour (forgelm/config.py:609-613, consumed at trainer.py:1858-1861) or the built-in _GPU_PRICING table. The page never mentions training.gpu_cost_per_hour, and its banner (:8-14) tells readers to 'set hourly rates manually until the YAML surface lands', although that YAML field already ships and appears in config_template.yaml (commented gpu_cost_per_hour) and docs/reference/configuration.md:139. Other claims with no code behind them: :59 shows a run_complete audit event with gpu_profile, rate_per_hour and currency. No such event exists; cost lands only as resource/* keys inside pipeline.completed metrics_summary (trainer.py:1418-1420, :1657). :70-77 shows a model-card 'Training cost' section, but forgelm/model_card.py has no cost text. :144 says 'Use --gpus N'; the CLI has no such flag. :34 says an unrecognised GPU logs a warning; _resolve_cost_per_hour returns None silently. The supported-GPU table (:26-32: RTX 6000 Ada, RTX A6000, RTX 4080, RTX 3080, Apple M-series) does not match the 16 keys of _GPU_PRICING (trainer.py:1813-1833). :20-21 says detection reads nvidia-smi; the code uses torch.cuda.get_device_name (:1841). Earlier reports did not cover these page claims; only the :39 block was reported, under M-A17a-020.

Where.

Evidence.

  • rg over forgelm/ finds no hits for run_complete, cost_tracking or gpu_profiles, and no --gpus parser flag. rg 'cost' forgelm/model_card.py returns nothing. gpu_cost_per_hour appears in config.py:609 and trainer.py:1858 and nowhere in gpu-cost.md (grep on both mirrors).

Recommended fix. Document training.gpu_cost_per_hour as the shipped rate override and correct the banner. Replace the run_complete JSON with the real pipeline.completed metrics_summary keys. Remove the --gpus, model-card and nvidia-smi claims, and generate the GPU table from _GPU_PRICING.

Acceptance criteria

  • A regression test reproduces the failure scenario and asserts the corrected behaviour.
  • Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
  • Every finding in this issue is resolved or explicitly closed with a reason.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: configCode area: configbugSomething isn't workingseverity: highWrong behaviour or a false claim users rely onsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 1Roadmap wave 1: high severity, small effort

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions