Apple MPS is never used: training and model loading are pinned to CPU on any non-CUDA host (CR-A02-085) #541

Wave 2 · theme D2-8: Docs tied to medium-effort code fixes · highest severity High · 8 finding(s) · effort medium

Summary

A user on an M-series Mac runs a LoRA SFT. The Mac's GPU stays idle, a run takes several times longer than needed, and the manual's 'MPS fallback' suggests otherwise.

Primary finding

CR-A02-085 — Apple MPS is never used: training and model loading are pinned to CPU on any non-CUDA host

severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end

What happens. use_cpu = not torch.cuda.is_available() in the common TrainingArguments kwargs, and model.py:_device_map_for returns None (CPU) when CUDA is absent. On Apple Silicon, where torch reports MPS available, ForgeLM therefore trains on the CPU. A tiny-model microbenchmark on this host measured about 4.7x lower step time on MPS than on CPU. The CUDA-only extras (fused optimizer, gradient checkpointing, bf16/fp16) are safely gated behind the same CUDA test, so nothing breaks. The user manual lists 'Apple M1/M2/M3 Max (CPU/MPS fallback)' under supported GPU profiles (gpu-cost.md:32), but there is no MPS code path and no Apple entry in _GPU_PRICING.

Where.

• forgelm/trainer.py:518
• forgelm/model.py:406-420
• docs/usermanuals/en/operations/gpu-cost.md:32

Evidence.

• Ran _get_common_training_kwargs on a stub ForgeTrainer on an MPS-capable, CUDA-less host. kwargs were use_cpu=True, optim=adamw_torch, bf16=False, gradient_checkpointing=False, and SFTConfig(**kwargs).device == cpu. TrainingArguments(use_cpu=False) on the same host gives device mps. Microbenchmark (bench.py): tiny-qwen 19.7M params, 8x512 tokens, fp32, fwd+bwd+AdamW: cpu 0.731 s/step, mps 0.157 s/step.

Recommended fix. Either add an explicit opt-in (for example a device field that selects mps when torch.backends.mps.is_available(), with model placement to match) or correct the docs to say CPU-only off CUDA. Expected gain: 3-5x on small and mid models on Apple Silicon. Keep fused optimizer, bf16 and gradient checkpointing off for MPS.

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

• TECH-T02-008 (technology, Next release) — Core Hugging Face stack and runtime dependencies: optim adamw_torch_fused only on CUDA, and use_cpu=not torch.cuda.is_available() (forgelm/trainer.py:514-518): MPS training on Apple Silicon (supersed…
  severity Next release
  Item. optim adamw_torch_fused only on CUDA, and use_cpu=not torch.cuda.is_available() (forgelm/trainer.py:514-518): MPS training on Apple Silicon · verdict: superseded-method · urgency: Next release
  State in ForgeLM. trainer.py:514-518 keys gradient_checkpointing, optim, bf16, fp16 and use_cpu on torch.cuda.is_available(). A host without CUDA always trains on CPU with adamw_torch. forgelm/ has no MPS code path (rg finds no 'mps', backends.mps or mps_available). docs/usermanuals/en/operations/gpu-cost.md:32 lists 'Apple | M1/M2/M3 Max (CPU/MPS fallback)'.
  State of the ecosystem on 2026-10-01. In transformers 5.18.0, the default optimizer is adamw_torch_fused when torch>=2.8 (training_args.py:803-811), although the docstring at :227-231 still calls adamw_torch the 'recommended default'. Device selection picks mps unless use_cpu is set (training_args.py:1886-1889). MIGRATION_GUIDE_V5.md:638 removes use_mps_device because 'mps will be used by default if detected', and the HF Apple Silico…
  Recommendation. Stop forcing use_cpu when CUDA is absent: let transformers choose the device, or add an explicit config switch. Keep adamw_torch_fused on CUDA, which matches upstream, and allow it on MPS. Make bf16 on MPS opt-in. Validate with a TRL SFTTrainer smoke test on MPS before making it the default. Fix gpu-cost.md:32 (EN and TR).
  Sources.

  • Transformers docs: Apple Silicon (MPS detected automatically) (2026-10-01)
  • Transformers v5 migration guide (use_mps_device removed, line 638) (2026-10-01)

• TECH-T13-009 (technology, Next release) — Python, platform, CI, container and GPU prices: Apple Silicon forced to CPU (use_cpu=not torch.cuda.is_available()) vs docs 'CPU/MPS fallback', and bitsandbytes platform statements (Linux-only, no Me…
  severity Next release
  Item. Apple Silicon forced to CPU (use_cpu=not torch.cuda.is_available()) vs docs 'CPU/MPS fallback', and bitsandbytes platform statements (Linux-only, no Metal/MPS) · verdict: incorrect-claim · urgency: Next release
  State in ForgeLM. forgelm/trainer.py:514-518 gates gradient checkpointing, fused AdamW and bf16/fp16 on torch.cuda.is_available() and sets use_cpu=not torch.cuda.is_available(); no 'mps' code path exists in forgelm/. docs/usermanuals/en/operations/gpu-cost.md:32 (and tr:32) lists 'Apple | M1/M2/M3 Max (CPU/MPS fallback)'. bitsandbytes statements: pyproject.toml:72 '(Linux-only for most setups)' and :74 marker sys_…
  State of the ecosystem on 2026-10-01. Transformers docs: 'Trainer detects MPS automatically with torch.backends.mps.is_available and sets the device to mps without any configuration changes'; bf16 on MPS needs macOS 14+, fp16 also supported; caveats are op coverage (PYTORCH_ENABLE_MPS_FALLBACK=1), whole-model-in-unified-memory and graph-cache growth (torch_empty_cache_steps). TrainingArguments: use_cpu help 'If set to False, we will…
  Recommendation. Either implement an explicit, opt-in device policy (cuda -> mps -> cpu; bf16 gated on macOS 14+, fused AdamW on CUDA only) with tests, or change gpu-cost.md:32 (EN+TR) to 'CPU only (MPS not used)'. Rewrite the bitsandbytes notes to say ForgeLM's qlora extra is Linux-only by ForgeLM's choice, fix publish.yml:78 and pyproject.toml:72, and consider widening the marker to win32 after a Windows smoke…
  Sources.

  • Transformers docs: Apple Silicon (Trainer detects MPS automatically) (2026-10-01 (main))
  • transformers training_args.py (use_cpu help; device selection cpu/mps/xpu) (2026-10-01 (main))

• DOC-BNEG-006 (docs, High) — pyproject.toml: false denial — N-30: qlora/unsloth Linux-only; export/distributed not on Windows; a missing backend raises ImportError naming the extra
  severity High
  Where. pyproject.toml:74,79,116,131 · type: false denial
  The document says. N-30: qlora/unsloth Linux-only; export/distributed not on Windows; a missing backend raises ImportError naming the extra
  What the code does. The platform markers hold: qlora/unsloth sys_platform=='linux' (pyproject.toml:74,79) and distributed/export !='win32' (:116,:131). The claim 'raises ImportError naming the extra' holds for unsloth (model.py:291-294, inference.py:141-144), deepspeed (trainer.py:581-586) and export (export.py:123-126). It does NOT hold for qlora: model.py:440-450 builds transformers' BitsAndBytesConfig with no for…

• DOC-BLEGAL-136 (docs, Medium) — docs/usermanuals/en/getting-started/installation.md: stale — bitsandbytes does not currently support Metal/MPS. On macOS, ForgeLM falls back to full-precision training. For 4-bit quantized training…
  severity Medium
  Where. docs/usermanuals/en/getting-started/installation.md:141 (Turkish mirror: docs/usermanuals/tr/getting-started/installation.md:141) · type: stale
  The document says. bitsandbytes does not currently support Metal/MPS. On macOS, ForgeLM falls back to full-precision training. For 4-bit quantized training (QLoRA), you need a Linux host with a CUDA GPU.
  What the code does. bitsandbytes 0.50.0 (2026-07-25) states 'all 4-bit and LLM.int8() configurations now work on MPS', and the MPS path requires torch >= 2.9. 0.50.2 ships a macosx_14_0_arm64 wheel, and only 8-bit optimizers on MPS are pending. The limitation now lies in ForgeLM: the CUDA-only 4-bit gate (forgelm/model.py:440-460) and the Linux-only marker on the extra.
  Proposed correction. 'ForgeLM enables 4-bit loading only when CUDA is available, and installs bitsandbytes only on Linux. On macOS it falls back to full precision even though bitsandbytes >= 0.50 supports 4-bit on MPS (torch >= 2.9). For QLoRA through ForgeLM you need a Linux host with a CUDA GPU.' TR: 'ForgeLM 4-bit yüklemeyi yalnızca CUDA varken etkinleştirir; bitsa…
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-BLEGAL-230 (docs, Low) — docs/usermanuals/en/getting-started/installation.md: stale — [IS-T03-04] 'bitsandbytes does not currently support Metal/MPS. On macOS, ForgeLM falls back to full-precision training. For 4-bit quant…
  severity Low
  Where. docs/usermanuals/en/getting-started/installation.md:141 (Turkish mirror: docs/usermanuals/tr/getting-started/installation.md:141) · type: stale
  The document says. [IS-T03-04] 'bitsandbytes does not currently support Metal/MPS. On macOS, ForgeLM falls back to full-precision training. For 4-bit quantized training (QLoRA), you need a Linux host with a CUDA GPU.'
  What the code does. bitsandbytes 0.50.0 (2026-07-25): 'all 4-bit and LLM.int8() configurations now work on MPS' (torch >= 2.9); 0.50.2 ships macosx_14_0_arm64, win_amd64 and win_arm64 wheels, and its README marks QLoRA 4-bit supported on macOS 14+ CPU and Metal. The real limit is ForgeLM's own Linux-only extra marker and CUDA-only 4-bit gate.
  Proposed correction. On macOS, ForgeLM falls back to full-precision training: its [qlora] extra installs bitsandbytes only on Linux and its 4-bit path requires CUDA. (bitsandbytes ≥ 0.50 itself supports 4-bit on Apple MPS; ForgeLM does not use that path yet.)
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-BLEGAL-235 (docs, Low) — docs/usermanuals/en/operations/troubleshooting.md: stale — [IS-T03-06] 'bitsandbytes doesn't support Metal/MPS. macOS hosts can't run 4-bit quantised training (QLoRA).'
  severity Low
  Where. docs/usermanuals/en/operations/troubleshooting.md:18 (Turkish mirror: docs/usermanuals/tr/operations/troubleshooting.md:18) · type: stale
  The document says. [IS-T03-06] 'bitsandbytes doesn't support Metal/MPS. macOS hosts can't run 4-bit quantised training (QLoRA).'
  What the code does. bitsandbytes 0.50.x supports 4-bit (QLoRA) and LLM.int8 on macOS 14+ CPU and MPS (torch >= 2.9). The macOS limit comes from ForgeLM's Linux-only extra marker and CUDA-only 4-bit gate.
  Proposed correction. Cause: ForgeLM installs bitsandbytes only on Linux and its 4-bit loader requires CUDA, so ForgeLM cannot run QLoRA on macOS hosts (upstream bitsandbytes ≥ 0.50 does support 4-bit on MPS).
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D2B05-030 (docs, Low) — docs/guides/getting-started.md: wrong detail — On macOS 'bitsandbytes does not currently support Metal/MPS, so the import fails silently and doctor reports the extra as missing'
  severity Low
  Where. docs/guides/getting-started.md:132 · type: wrong detail
  The document says. On macOS 'bitsandbytes does not currently support Metal/MPS, so the import fails silently and doctor reports the extra as missing'
  What the code does. Doctor never imports extras; it uses importlib.util.find_spec. The qlora extra pins bitsandbytes with sys_platform == 'linux', so on macOS it is simply not installed. The fallback to full precision (with a warning) is accurate.
  Code references. forgelm/cli/subcommands/_doctor.py:421-443, pyproject.toml:73-75, forgelm/model.py:451-460
  Proposed correction. "The [qlora] extra installs bitsandbytes only on Linux (platform marker), so on macOS the module is absent and doctor reports the extra as missing."
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}