Skip to content

[Wave 4] Low/info — Module sweep: Trainer core, OOM recovery, auto-revert & GRPO rewards (part 4/4) #711

Open

Description

@cemililik

Wave 4 · 6 finding(s) in 6 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

  • CR-A02-141 (code, Info) — Convention rules with no guard show drift: bare NOSONAR, DEBUG-only BLE001 handlers, pragma: no cover on defensive branches, library log-and-raise, typing style
    severity Info · reachability unreachable · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end
    Failure scenario. model.py:638: a failed expert recast is visible only at DEBUG, so an operator expecting MoE VRAM savings gets none and no warning. The other items are convention hygiene.
    Where. forgelm/trainer.py:1693-1708, forgelm/model.py:638, forgelm/compliance.py:1005, forgelm/compliance.py:2321
    Recommended fix. Add AST guards for rule-coded NOSONAR, BLE001 handler log level and pragma placement. Fix the DEBUG-only handlers first.

  • CR-A02-142 (code, Info) — The eval_steps/save_steps divisibility rule is correct but is never checked against a real transformers TrainingArguments
    severity Info · reachability default · intent intended-and-acceptable · confidence likely · independently confirmed · reproduced end to end · already tracked: PARTIALLY-FIXED CORE-14 — docs/roadmap/phase-16-trust-surface-hardening.md:292 (S2 done); CHANGELOG.md:21-25
    Failure scenario. A transformers or trl upgrade changes the save/eval constraint, for example requiring the multiple also under eval_strategy='epoch', or relaxing it. ForgeLM's rule and its tests stay green and the operator hits the late exit-2 (or a false exit-1) the check was added to prevent.
    Where. tests/test_trainer.py:1377-1440, forgelm/trainer.py:483-495, tests/runtime_smoke.py:1-109, tests/test_trainer_sft_config.py:69
    Recommended fix. Add the differential test described above, one per supported trainer type, using the real SFTConfig, DPOConfig and so on with use_cpu=True and report_to='none'. The existing 'cadence mismatch passes --dry-run' item is unaffected.

  • CR-A02-143 (code, Info) — ConfigError is raised from the ForgeTrainer constructor, not train(); train() re-raises arbitrary exceptions
    severity Info · reachability default · intent contract-violation · confidence likely · independently confirmed · not reproduced (code trace)
    Failure scenario. A caller wraps only train() in try/except ConfigError and misses constructor failures. Other failure types propagate unmapped.
    Where. docs/reference/library_api_reference.md:264, docs/reference/library_api_reference-tr.md:264, forgelm/trainer.py:366-371, forgelm/trainer.py:1693-1709
    Recommended fix. Move ConfigError to the constructor row. State that train() re-raises the underlying exception after emitting pipeline.failed.

  • CR-A02-144 (code, Info) — model.reverted records the regression only as free text; no structured delta or threshold fields
    severity Info · reachability default · intent intended-and-acceptable · confidence likely · independently confirmed · reproduced end to end
    Failure scenario. An auditor or dashboard parsing model.reverted to decide whether a revert was a real regression or threshold tuning has to regex free text, or join to a different event.
    Where. forgelm/trainer.py:992, forgelm/trainer.py:1406-1409, forgelm/trainer.py:1460-1464, forgelm/trainer.py:1489-1492
    Recommended fix. Emit measured, threshold and delta as structured fields on model.reverted, or amend RTP R-07 to point at the gate *.evaluation_completed events.

  • CR-A02-145 (code, Info) — model.reverted reason vocabulary in the docs does not match the emitters
    severity Info · reachability undocumented · intent contract-violation · confidence likely · independently confirmed · reproduced (plumbing only)
    Failure scenario. An audit consumer matching on the documented reason set misses this revert.
    Where. forgelm/trainer.py:863 (confirmed), forgelm/trainer.py:978-979, docs/usermanuals/en/evaluation/auto-revert.md:102 (confirmed), docs/usermanuals/tr/evaluation/auto-revert.md:102
    Recommended fix. Add invalid_threshold to auto-revert.md (EN+TR) and the docstring.

  • CR-A02-146 (code, Info) — save_final_model before train() reports the missing trainer as a save fallback, raises an unexplained AttributeError, and leaves an empty final_model directory
    severity Info · reachability undocumented · intent intended-but-harmful · confidence likely · independently confirmed · reproduced end to end
    Failure scenario. A caller saves adapters before or without training (for example a resume or export helper). They get a misleading 'fallback' warning, a bare AttributeError and an empty final_model/ that later blocks forgelm approve.
    Where. forgelm/trainer.py:1711-1749, forgelm/trainer.py:1713, forgelm/trainer.py:1720-1729, forgelm/trainer.py:1735
    Recommended fix. Check the precondition before makedirs: if the trainer has not been built, raise a clear RuntimeError, or save from self.model. Keep the missing-attribute case out of the narrow fallback catch.

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: trainerCode area: trainerbacklog: knownAlready tracked in docs/roadmapbugSomething isn't workingseverity: infoHardening or improvement suggestionsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 4Roadmap wave 4: low / info / watch, when touching the module

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions