[torch] Add gemma4 LLM template and GPTQ algo config for MoE quantization

[jimmy-adams]

Closes #39

Summary

Add built-in gemma4 template and GPTQ algo config so Gemma4 MoE models (e.g. google/gemma-4-26B-A4B) can be quantized via the standard quantize_quark.py CLI without hand-rolled configs.

Diff: +41 lines in 2 files (template.py, algo_configs.py).

Changes

template.py — gemma4

• KV-cache groups: *language_model.*{k,v}_proj
• Q-layer: *language_model.*q_proj
• Exclude: vision/audio/projector, lm_head, router.proj
• F2F weight converters: SplitFusedExperts for fused gate_up_proj / down_proj

algo_configs.py — gemma4 GPTQ

• Quantize: self_attn.*, dense mlp.*, and unfused experts.*.{gate,up,down}_proj
• Decoder layers path: model.language_model.layers

Validation (mxfp4_weight_only RTN)

Verified on full google/gemma-4-26B-A4B checkpoint:

python examples/torch/language_modeling/llm_ptq/quantize_quark.py \
  --model_dir /shareddata/google/gemma-4-26B-A4B \
  --output_dir /tmp/gemma4-26b-mxfp4-rtn \
  --multi_gpu \
  --quant_scheme mxfp4_weight_only \
  --dataset pileval --num_calib_data 128 --seq_len 512 \
  --data_type bfloat16 --model_export hf_format \
  --skip_evaluation --device cuda

Check	Result
Template match	vision/audio/projector/lm_head/router.proj excluded
MoE preprocess	30× QuarkExperts unfused, 11,725 Linear quantized
Export	HF safetensors (~15G) exported successfully
Runtime	~107s on 7×GPU, exit code 0

Known limitation (out of scope)

Unfused per-expert GPTQ on 128-expert Gemma4 can fail with Hessian not positive-definite on cold experts (insufficient calibration tokens). RTN / weight-only paths work; GPTQ improvements (fused-expert path or cold-expert fallback) tracked separately.

Test plan

• Manual: mxfp4_weight_only RTN on Gemma4-26B-A4B — pass
• CI: existing LLM PTQ regression (no Gemma4 in CI yet)

Made with Cursor