You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
MiniMax H3: token refiner gets fp32 q/k beside a bf16 v (sage refuses it), and preprocess_text_embeds drops transformer_options #17713
MiniMax H3: the token refiner hands attention fp32 q/k beside a bf16 v, and preprocess_text_embeds drops transformer_options
ComfyUI 0.45.0 (a592c0a65), torch 2.16.0a0+rocm10.3.0a20261110 (gfx1101), sageattention 2.2.0, launched with --use-sage-attention. Both points below are still true on master (c5751c8).
Symptom
Sampling a MiniMax H3 image model logs this once to four times per image, and those calls run on pytorch attention instead of sage:
Error running sage attention: All tensors must have the same dtype., using pytorch attention instead.
On some shapes ComfyUI's own fallback raises as well instead of degrading — attention_pytorch calls F.scaled_dot_product_attention, which needs one dtype too:
Expected query, key, and value to have the same dtype, but got query.dtype: float ... value.dtype: c10::BFloat16
Two separate causes
transformer_options never reach the refiner.comfy/model_base.py calls diffusion_model.preprocess_text_embeds(...) without them (the call is where kwargs is in scope), and preprocess_text_embeds (comfy/ldm/minimax/model.py:552) calls self.token_refiner(...) with no options either, so Attention gets the default {}. No attention override, block patch or hook applies inside the refiner — including the per-run attention overrides trainers install.
The no-rope branch leaves q/k in the norms' dtype.Attention.forward's else branch (:201-202) does q = self.q_norm(...), k = self.k_norm(...) while v (:184) comes straight off the qkv split. When the norm weight is fp32 (the refiner is built in fp32 by sd's model detection in some paths), F.rms_norm promotes and q/k are fp32 beside a bf16 v. The rope branch keeps q's dtype because it is in place, which is why only the refiner is affected.
Suggested fixes
Thread transformer_options through preprocess_text_embeds and its comfy/model_base.py callers (it is available at each call site). That alone lets existing overrides fix it, and is right regardless.
Independently, keep q/k in the activation dtype in the no-rope branch (cast the norm output, or build the norms in the activation dtype), so the model does not depend on an override to be sampleable.
How it was found
A training run surfaced it in its sample renders; it reproduces in plain sampling of the same official checkpoint with --use-sage-attention on ROCm. A downstream workaround (cast q/k to v's dtype at the model's optimized_attention entry point) is in Nynxz/ZenTrainer#73 commit 3 — a workaround for this, not the right home for it.
Activity