Repository navigation
feat: expose V100 LoRA FP32 protection in WebUI - #10
Conversation
| else: | ||
| current[key] = normalized == "true" | ||
| continue | ||
| if key == "lora_fp32_compute": |
There was a problem hiding this comment.
🟠 High: Legacy network_args still overrides this controlnnAn existing network_args = ["lora_fp32_compute=false"] remains in the overlay while this writes the new top-level value. The trainer intentionally gives network_args precedence, so selecting true still runs false. Remove/promote the legacy entry when loading and saving this field.
| if normalized not in _LORA_FP32_COMPUTE_CHOICES: | ||
| raise ValueError("lora_fp32_compute must be auto, true, or false") | ||
| if normalized == "auto": | ||
| out.pop("lora_fp32_compute", None) |
There was a problem hiding this comment.
🟡 Medium: auto cannot clear a custom preset overridennDropping the method key works only when lower layers also omit it. If a custom preset explicitly sets true/false, choosing auto at the variant layer leaves that preset value effective. Persist a method-layer reset sentinel and consume it after the training merge.
| # explicit true/false force the user's choice. Surface the omitted state as | ||
| # "auto" in the WebUI without writing that sentinel into the training TOML. | ||
| if ( | ||
| merged.get("network_module") == "networks.lora_anima" |
There was a problem hiding this comment.
🟡 Medium: Non-LoRA variants can still display this fieldnnThis condition only controls injection when the key is absent. A custom preset carrying lora_fp32_compute leaves the key in merged even after a non-LoRA method replaces network_module, so EasyControl and other families show an irrelevant control. Remove it when the effective module is not networks.lora_anima.
| "torch_compile": "Enable torch.compile for the forward pass. Faster training after initial compilation. Compiles each DiT block at native shapes — one graph per token-count bucket family, no static padding.", | ||
| "compile_dynamic_seq": "Mark only the native token-sequence axis dynamic inside each compiled DiT block. Required with free-fit shapes so a tier reuses one graph instead of compiling one graph per token count.", | ||
| "v100_flash_stability": "V100 FlashAttention diagnostic mode. off uses full Flash; hybrid routes cross-attention to Torch SDPA; safe keeps Flash and enables attention-boundary finite checks. A generated production preset uses off only after every local validation gate passes; safe is diagnostic only.", | ||
| "lora_fp32_compute": "FP32 protection for the LoRA family. auto enables it only for V100/sm_70 FP16 training; true forces FP32 adapter compute; false explicitly disables it. When active, adapter parameters, optimizer inputs, and saved checkpoint tensors stay FP32 while frozen DiT weights keep the model dtype. Usually leave this on auto.", |
There was a problem hiding this comment.
🟡 Medium: false is not unconditional in reliable block-swap modennThe runtime explicitly overrides lora_fp32_compute=false for V100 reliable block-swap mode to preserve its safety contract. Saying false always disables protection is misleading; document that exception here and in the other locales.
Summary
Validation