🚀 HaloBlock: Faster and Better Decoding for dLLMs

Official implementation of HaloBlock, a training-free difficulty-aware adaptive halo mechanism for block-wise decoding in diffusion large language models (dLLMs). It mitigates the left-dominance effect and boundary contextual discontinuity in block-wise decoding, achieving better generation quality and faster inference speed.

Installation

conda create -n haloblock python==3.10
conda activate haloblock
pip install -r requirements.txt

Evaluation

For LLaDA evaluation,

cd llada

# GSM8K (5-shot)
sh eval_gsm8k.sh

# MATH (4-shot)
sh eval_math.sh

# HumanEval (0-shot)
sh eval_humaneval.sh

# MBPP (3-shot)
sh eval_mbpp.sh

For Dream evaluation,

cd dream

# GSM8K (5-shot)
sh eval_gsm8k.sh

# MATH (4-shot)
sh eval_math.sh

# HumanEval (0-shot)
sh eval_humaneval.sh

# MBPP (3-shot)
sh eval_mbpp.sh

Citation

If you find this work useful, please cite our paper:

@Article{zheng2026haloblock,
  title={HaloBlock: Faster and Better Decoding for Diffusion Large Language Models},
  author={Zheng, Zhaohui and He, Chenhang and Zhang, Jinrui and Cheng, Ming-Ming and Zhang, Lei},
  journal={arXiv preprint arXiv:2610.17948},
  year={2026}
}

Acknowledgements

This codebase is inherited from Fast dLLM and dParallel. We would like to thank the authors of LLaDA and Dream for open-sourcing the models, and the lm-evaluation-harness team for open-sourcing the evaluation framework.