lora.method: pissa with load_in_4bit: true (the default, and the wizard's PiSSA choice) crashes on every CUDA host (CR-A03-003) #402

Wave 1 · theme D1-5: Training, merging and export correctness · highest severity High · 3 finding(s) · effort small

Summary

On a CUDA box, a user picks PiSSA in the wizard and runs the generated config. get_peft_model raises TypeError and training exits 2.

Primary finding

CR-A03-003 — lora.method: pissa with load_in_4bit: true (the default, and the wizard's PiSSA choice) crashes on every CUDA host

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace)

What happens. _build_lora_config sets init_lora_weights='pissa' (model.py:539-541). Under CUDA + load_in_4bit, get_peft_model (609) wraps bitsandbytes Linear4bit layers. PEFT's update_layer then calls pissa_init, which raises TypeError for any base weight dtype other than fp32/fp16/bf16 (peft tuners/lora/layer.py:222-224, 363-367). Params4bit storage is uint8, as model.py:630-633 itself notes. load_in_4bit defaults to true (config.py:256), the wizard's PiSSA strategy hard-codes load_in_4bit=True (wizard/_collectors.py:1076-1080), and no config validator rejects the combination. --dry-run passes and the run dies at adapter injection. On non-CUDA hosts it 'works' only because 4-bit silently falls back to full precision.

Where.

• forgelm/model.py:539-541
• forgelm/model.py:602-609
• forgelm/wizard/_collectors.py:1076-1081
• forgelm/config.py:259 (load_in_4bit default; cited as 256)
• forgelm/model.py:440-449 (BitsAndBytesConfig load); forgelm/model.py:602 (prepare_model_for_kbit_training)
• forgelm/config.py:370-372
• forgelm/cli/_training.py:141
• forgelm/cli/_training.py:173-180
• forgelm/wizard/_orchestrator.py:425
• docs/usermanuals/en/reference/yaml-templates.md:206-208

Evidence.

• Static: installed peft 0.19.1 source, tuners/lora/layer.py:222-224 (update_layer calls pissa_init) and 363-367 (raise TypeError 'Please initialize PiSSA under float32, float16, or bfloat16'); bnb Linear4bit uses LoraLayer.update_layer (tuners/lora/bnb.py:311-330). No CUDA/bitsandbytes on this host, so not executed.

Recommended fix. Reject pissa together with load_in_4bit at config validation (or load in full precision for PiSSA init and requantize), and fix the wizard choice.

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

CR-A14-016 — Wizard offers PiSSA on the 4-bit (QLoRA) branch, which PEFT's PiSSA init rejects; PiSSA plus unsloth also passes schema validation

severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced (plumbing only)

What happens. _STRATEGY_CHOICES defines PiSSA with load_in_4bit=True, so the wizard writes lora.method: pissa together with model.load_in_4bit: true. ForgeConfig accepts the pair and the wizard reports success. At training time on CUDA the model loads with bitsandbytes, whose Linear4bit weights are packed uint8. PEFT's LoRA layer init for init_lora_weights='pissa' calls pissa_init, which raises TypeError('Please initialize PiSSA under float32, float16, or bfloat16...') for any other dtype. model.py builds LoraConfig(init_lora_weights='pissa') with no dequantise or guard. Separately, the welcome step recommends model.backend: unsloth on Linux with a GPU, and model.py:343 raises ConfigError for pissa under unsloth. The schema validator and wizard do not catch that pair either. Confidence is medium for the 4-bit path because there is no CUDA or bitsandbytes in this review environment.

Failure scenario. On a CUDA box the operator chooses 'PiSSA' in the wizard, gets a valid-looking config, and training dies at get_peft_model with TypeError. On Linux with a GPU the wizard also pre-selects the unsloth backend, which rejects pissa with ConfigError at load.

Where.

• forgelm/wizard/_collectors.py:1076-1081
• forgelm/wizard/_orchestrator.py:223-235
• forgelm/model.py:343-351
• forgelm/model.py:539-541
• forgelm/model.py:441-450
• forgelm/model.py:609
• forgelm/wizard/_orchestrator.py:406
• forgelm/config.py:259
• forgelm/config.py:370
• docs/usermanuals/en/reference/yaml-templates.md:206-208

Evidence.

• Wizard run with the PiSSA choice writes method: pissa and load_in_4bit: true and prints 'Schema validation passed'. ForgeConfig.model_validate accepts pissa+4bit and pissa+unsloth. Source read in peft 0.19.0, 0.19.1 and 0.21.2: lora/layer.py:360-367 pissa_init raises TypeError on non-float dtypes, and lora/bnb.py Linear4bit.init calls update_layer, which reaches pissa_init for 'pissa'. A plumbing-only simulation with a uint8 weight parameter reproduces the exact TypeError.

Recommended fix. Make the PiSSA menu entry load_in_4bit=False, or add a cross-field validator rejecting pissa with load_in_4bit=True and pissa with backend=unsloth. Have the wizard warn or re-prompt.

TECH-T03-004 — PEFT, quantization and memory methods: PiSSA combined with 4-bit QLoRA (lora.method=pissa with the default model.load_in_4bit=true, offered by the wizard) (at-risk)

severity Next release

Item. PiSSA combined with 4-bit QLoRA (lora.method=pissa with the default model.load_in_4bit=true, offered by the wizard) · verdict: at-risk · urgency: Next release

State in ForgeLM. load_in_4bit defaults to True (forgelm/config.py:259); no validator rejects pissa+4-bit (the only pissa reference in config.py is :370-372); the wizard's PiSSA card sets load_in_4bit=True (forgelm/wizard/_collectors.py:1076-1081). On CUDA the base is loaded with BitsAndBytesConfig(load_in_4bit=True) (forgelm/model.py:440-450), then get_peft_model runs (model.py:609) with init_lora_weights='pissa' (model.py:539-540). ForgeLM never sets bnb_4bit_quant_storage (no match in forgelm/).

State of the ecosystem on 2026-10-01. peft LoraLayer.pissa_init raises TypeError('Please initialize PiSSA under float32, float16, or bfloat16. Subsequently, re-quantize the residual model...') when the base weight dtype is not a float: v0.21.2 layer.py:429-436 and v0.19.0 layer.py:360-367. bnb 4-bit weights use packed uint8 storage by default, so the guard fires. The upstream PiSSA README says 4-bit/8-bit fine-tuning must 'first decompose the original model at full precision and then reload the residual model'. peft issue #​3742 (opened 2026-09-14, open) confirms the guard is dtype-only: it raises on default storage and is bypassed, corrupting packed data, when a float quant_storage is set.

Impact. On a CUDA host, every config with method=pissa that keeps the default load_in_4bit=true, including every wizard PiSSA pick, fails at get_peft_model (exit 2) after the model download. README.md:154 lists '4-bit QLoRA ... PiSSA' together. The failure is loud, not silent. It was established from peft source and the upstream issue; nobody ran it on real bitsandbytes/CUDA (not available here).

Recommendation. Add a config validator that rejects method=pissa with load_in_4bit=true on the transformers backend (or implement decompose-then-quantise-residual). Set load_in_4bit=False on the wizard's PiSSA card. Never combine PiSSA with a float bnb_4bit_quant_storage.

Sources.

• peft v0.21.2 lora/layer.py (pissa_init dtype guard :429-436) (2026-10-01)
• peft v0.19.0 lora/layer.py (pissa_init dtype guard :360-367) (2026-04-14)
• peft PiSSA fine-tuning README (4-bit: decompose at full precision, reload residual) (2026-10-01)
• peft issue #3742: PiSSA/MiCA/CorDA init dtype-only guard on bnb 4-bit layers (open) (2026-09-16)

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}