Skip to content

Add Gemma 4 E2B MLP quantization and multi-pass sweep configuration. - #375

Open
copybara-service[bot] wants to merge 1 commit into
mainfrom
test_980111536
Open

copybara-service[bot] wants to merge 1 commit into
mainfrom
test_980111536

Conversation

@copybara-service

Copy link
Copy Markdown

Add Gemma 4 E2B MLP quantization and multi-pass sweep configuration.

  • Support YAML sweep specifications in xm_launch.py with macro expansion, dry-run support, and per-trial output directories.
  • Support hierarchical scaling and multi-pass quantization configurations for MLP layers in MaxText (base.yml, types.py, quantizations.py).
  • Add gemma4_e2b_mlp_quant_sweep.yaml defining 13 parallel trials on Gemma 4 E2B (Ghostfish 2x2x1):
    • Trial 1: Baseline (unquantized BF16)
    • Trial 2: 1-pass mxfp8_16 on all MLP tensors (flat scales)
    • Trial 3: 1-pass mxfp8_16 on all MLP tensors + hierarchical scaling on weight gradients (drhs_mlp_grad)
    • Trial 4: 1-pass mxint8 on all MLP tensors (flat scales)
    • Trial 5: 1-pass mxint8 on all MLP tensors + hierarchical scaling on weight gradients
    • Trial 6: 3-pass triangular mxfp8_16 (all MLP, flat scales)
    • Trial 7: 3-pass triangular mxfp8_16 + hierarchical scaling on weight gradients
    • Trial 8: 4-pass full-cross mxfp8_16 (all MLP, flat scales)
    • Trial 9: 4-pass full-cross mxfp8_16 + hierarchical scaling on weight gradients
    • Trial 10: 3-pass triangular mxint8 (all MLP, flat scales)
    • Trial 11: 3-pass triangular mxint8 + hierarchical scaling on weight gradients
    • Trial 12: 4-pass full-cross mxint8 (all MLP, flat scales)
    • Trial 13: 4-pass full-cross mxint8 + hierarchical scaling on weight gradients
  • Add launch_gemma4_e2b_mlp_quant_sweep.sh using pretraining reference settings from cl/980100775 (Grain dataset loading, Ghostfish 2x2x1, 10k steps, BS=4, GA=16, 1.05M tokens/step, /cns/yo-d/ storage).

- Support YAML sweep specifications in xm_launch.py with macro expansion, dry-run support, and per-trial output directories.
- Support hierarchical scaling and multi-pass quantization configurations for MLP layers in MaxText (base.yml, types.py, quantizations.py).
- Add gemma4_e2b_mlp_quant_sweep.yaml defining 13 parallel trials on Gemma 4 E2B (Ghostfish 2x2x1):
  * Trial 1: Baseline (unquantized BF16)
  * Trial 2: 1-pass mxfp8_16 on all MLP tensors (flat scales)
  * Trial 3: 1-pass mxfp8_16 on all MLP tensors + hierarchical scaling on weight gradients (drhs_mlp_grad)
  * Trial 4: 1-pass mxint8 on all MLP tensors (flat scales)
  * Trial 5: 1-pass mxint8 on all MLP tensors + hierarchical scaling on weight gradients
  * Trial 6: 3-pass triangular mxfp8_16 (all MLP, flat scales)
  * Trial 7: 3-pass triangular mxfp8_16 + hierarchical scaling on weight gradients
  * Trial 8: 4-pass full-cross mxfp8_16 (all MLP, flat scales)
  * Trial 9: 4-pass full-cross mxfp8_16 + hierarchical scaling on weight gradients
  * Trial 10: 3-pass triangular mxint8 (all MLP, flat scales)
  * Trial 11: 3-pass triangular mxint8 + hierarchical scaling on weight gradients
  * Trial 12: 4-pass full-cross mxint8 (all MLP, flat scales)
  * Trial 13: 4-pass full-cross mxint8 + hierarchical scaling on weight gradients
- Add launch_gemma4_e2b_mlp_quant_sweep.sh using pretraining reference settings from cl/980100775 (Grain dataset loading, Ghostfish 2x2x1, 10k steps, BS=4, GA=16, 1.05M tokens/step, /cns/yo-d/ storage).

PiperOrigin-RevId: 980111536
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants