[ToluClassics]

Summary

Make quantized text inference a first-class, release-gated feature for v0.3.0.

• add validated QuantizationConfig and inspectable QuantizationInfo
• auto-detect and load MLX pre-quantized safetensors checkpoints
• support explicit in-memory quantization of regular checkpoints while preserving legacy loader flags
• add model-family-aware load_causal_model / generate_text APIs
• add the mlx-transformers-generate CLI with finite generation, offline loading, safe dtype defaults, and no remote-code execution
• document peak-memory behavior, supported MLX modes, lifecycle priority, and offline usage

Why

The loader already had low-level quantization hooks, but users had no cohesive public API or generic command for running quantized checkpoints. Runtime settings were only loosely validated, metadata was inconsistent, and the existing quantized example was Llama-specific. This PR turns those mechanics into an explicit v0.3.0 support contract.

Validation

Run locally on Apple silicon on 2026-07-26:

• Ruff lint and formatting: passed
• MLX 0.32.0 / Transformers 5.14.1: 114 discovered, 92 passed, 22 Hub-gated skips
• MLX 0.31.0 / Transformers 4.57.6: 114 discovered, 92 passed, 22 Hub-gated skips
• synthetic regular Llama checkpoint: load, affine 4-bit runtime quantization, finite forward, bounded cached generation
• synthetic pre-quantized Llama checkpoint: save, strict reload, finite forward, bounded cached generation
• cached mlx-community/Phi-3-mini-4k-instruct-4bit: packaged CLI auto-detected checkpoint quantization as affine 4-bit/group-64 and generated Yes. fully offline
• wheel and sdist build: passed
• Twine metadata checks: passed
• final wheel installed into a clean temporary Python 3.13 environment with no package dependencies; metadata import reported version 0.3.0
• final wheel installed into a clean functional environment; console entry point executed successfully

Scope and limitations

Real-checkpoint evidence currently covers affine 4-bit quantization. MXFP4, MXFP8, and NVFP4 settings are validated against MLX constraints but are not advertised as real-checkpoint-verified. Runtime quantization temporarily materializes the regular checkpoint, so large models should prefer reviewed pre-quantized MLX checkpoints. OpenELM and Persimmon remain maintenance-only.