PEFT, quantization and memory methods: bitsandbytes platform scope: sys_platform=='linux' marker, CUDA-only 4-bit gate, 'bitsandbytes does not support Metal/MPS' docs (incorrect-claim) (TECH-T03-007) #488

Wave 2 · theme D2-7: Next-release technology updates · highest severity Low · 3 finding(s) · effort not estimated

Summary

The documented reason has been false since 2026-07-25. Behaviour still matches the docs, because ForgeLM's own marker and CUDA gate block macOS/Windows/XPU/CPU 4-bit; ROCm passes the gate. This is a ForgeLM scope decision presented as an upstream limitation.

Primary finding

TECH-T03-007 — PEFT, quantization and memory methods: bitsandbytes platform scope: sys_platform=='linux' marker, CUDA-only 4-bit gate, 'bitsandbytes does not support Metal/MPS' docs (incorrect-claim)

severity Next release

Item. bitsandbytes platform scope: sys_platform=='linux' marker, CUDA-only 4-bit gate, 'bitsandbytes does not support Metal/MPS' docs · verdict: incorrect-claim · urgency: Next release

State in ForgeLM. The [qlora] extra is Linux-only (pyproject.toml:74). The code applies 4-bit only when torch.cuda.is_available() (forgelm/model.py:440, :602); otherwise it warns and loads in full precision (model.py:451-461). Docs give an upstream reason: installation.md:141, getting-started.md:132 and usermanual operations/troubleshooting.md:18 (plus the three TR mirrors) say bitsandbytes does not support Metal/MPS; guides/troubleshooting.md:11 says bitsandbytes (QLoRA) only works on Linux.

State of the ecosystem on 2026-10-01. bitsandbytes 0.50.2 ships wheels for macosx_14_0_arm64, manylinux_2_24 x86_64/aarch64, win_amd64 and win_arm64. The README matrix marks QLoRA 4-bit as supported on Linux CPU/CUDA/ROCm/XPU, Windows and macOS 14+ CPU and Metal (mps); only 8-bit optimizers on MPS are pending. The 0.50.0 notes (2026-07-25) say 'all 4-bit and LLM.int8() configurations now work on MPS ... The MPS path requires torch >= 2.9', and that ROCm 'is now considered stable'.

Impact. The documented reason has been false since 2026-07-25. Behaviour still matches the docs, because ForgeLM's own marker and CUDA gate block macOS/Windows/XPU/CPU 4-bit; ROCm passes the gate. This is a ForgeLM scope decision presented as an upstream limitation.

Recommendation. Either widen the marker and gate (bnb>=0.50 on win32/darwin; device check instead of cuda-only, MPS needs torch>=2.9), or keep the Linux+CUDA/ROCm scope and reword the EN and TR docs as a ForgeLM support decision.

Sources.

• bitsandbytes 0.50.2 README (accelerator support matrix) (2026-08-27)
• bitsandbytes GitHub releases (0.50.0 notes: MPS all 4-bit/int8, torch>=2.9; ROCm stable; removed non-blockwise optimize… (2026-08-27)
• PyPI JSON: bitsandbytes (0.50.2; wheels macOS arm64, manylinux, win_amd64, win_arm64; torch>=2.4,<3) (2026-08-27)
• bitsandbytes 0.50.2 installation doc (2026-08-27)

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

DOC-BLEGAL-195 — docs/guides/troubleshooting.md: stale — [IS-T03-07] 'bitsandbytes (QLoRA) only works on Linux'

severity Low

Where. docs/guides/troubleshooting.md:11 · type: stale

The document says. [IS-T03-07] 'bitsandbytes (QLoRA) only works on Linux'

What the code does. bitsandbytes 0.50.2 ships win_amd64, win_arm64 and macOS arm64 wheels with 4-bit support. Linux-only is ForgeLM's extra-marker choice (the [qlora] extra resolves only on Linux).

User impact. States an upstream limitation that no longer exists. The practical advice (disable 4-bit outside Linux) is still right for ForgeLM.

Proposed correction. ForgeLM's [qlora] extra installs bitsandbytes only on Linux (and its 4-bit path requires CUDA):

Turkish mirror. the Turkish page has the same error; fix both in the same PR

DOC-BLEGAL-234 — docs/usermanuals/en/operations/troubleshooting.md: stale — Cause: bitsandbytes doesn't support Metal/MPS. macOS hosts can't run 4-bit quantised training (QLoRA).

severity Low

Where. docs/usermanuals/en/operations/troubleshooting.md:18 (Turkish mirror: docs/usermanuals/tr/operations/troubleshooting.md:18) · type: stale

The document says. Cause: bitsandbytes doesn't support Metal/MPS. macOS hosts can't run 4-bit quantised training (QLoRA).

What the code does. bitsandbytes 0.50.x supports 4-bit and LLM.int8 on MPS (torch >= 2.9). The cause on macOS is ForgeLM's CUDA-only 4-bit gate and its Linux-only install marker.

User impact. The stated root cause is wrong.

Proposed correction. 'Cause: ForgeLM enables 4-bit (QLoRA) only on CUDA hosts and installs bitsandbytes only on Linux; bitsandbytes itself supports MPS since 0.50.' TR: 'Sebep: ForgeLM 4-bit'i (QLoRA) yalnızca CUDA'da etkinleştirir; bitsandbytes 0.50'den beri MPS'i destekler.'

Turkish mirror. the Turkish page has the same error; fix both in the same PR

Acceptance criteria

• Each statement is re-checked against the code at fix time.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}