nielsr's picture
nielsr HF Staff
Add model card metadata, links to paper, code, and project page
7c13e28 verified
|
Raw
History Blame
1.64 kB
---
license: mit
library_name: transformers
pipeline_tag: text-generation
---
# Multi-Block Diffusion Language Models (MBD-LMs)
This repository contains the model weights for **Multi-Block Diffusion Language Models (MBD-LMs)**, presented in the paper [Multi-Block Diffusion Language Models](https://arxiv.org/abs/2606.29215).
- **Project Page:** [sjtu-deng-lab.github.io/mbd-lms](https://sjtu-deng-lab.github.io/mbd-lms)
- **GitHub Repository:** [SJTU-DENG-Lab/mbd-lms](https://github.com/SJTU-DENG-Lab/mbd-lms)
## Introduction
Block Diffusion Language Models (BD-LMs) improve diffusion-based text generation with KV caching and flexible-length generation. MBD-LMs extend them from Single-Block Diffusion (SingleBD) to Multi-Block Diffusion (MultiBD), where a running-set of consecutive blocks is decoded concurrently for inter-block parallelism.
This model is obtained by post-training BD-LMs with Multi-block Teacher Forcing (MultiTF), which integrates teacher forcing and diffusion forcing by training on bounded noise-groups conditioned on clean prefixes.
For setup guidelines, training configurations, and optimized inference engine setups, please refer to the [official repository](https://github.com/SJTU-DENG-Lab/mbd-lms) and the [Diffulex engine](https://github.com/SJTU-DENG-Lab/Diffulex).
## Citation
```bibtex
@article{jin2026multiblock,
title={Multi-Block Diffusion Language Models},
author={Yijie Jin and Jiajun Xu and Yuxuan Liu and Chenkai Xu and Yi Tu and Jiajun Li and Dandan Tu and Xiaohui Yan and Kai Yu and Pengfei Liu and Zhijie Deng},
journal={arXiv preprint arXiv:2606.29215},
year={2026}
}
```