README.md

---
title: Multi-Block Diffusion Language Models
emoji: 🧩
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 5.50.0
app_file: app.py
short_description: Chat with MBD-Math-SDAR-8B block diffusion language model
python_version: "3.12"
startup_duration_timeout: 1h
---

# Multi-Block Diffusion Language Models

This Space demonstrates **MBD-Math-SDAR-8B-Chat-b32**, a Multi-Block Diffusion Language Model from [SJTU-DENG-Lab](https://github.com/SJTU-DENG-Lab/mbd-lms).

Unlike standard autoregressive LLMs that generate tokens one at a time, this model uses **block diffusion**: it generates blocks of 32 tokens simultaneously through iterative denoising, enabling parallel generation within each block.

## Model Details

- **Architecture**: SDAR (modified Qwen3 with non-causal attention)
- **Parameters**: 8.2B
- **Precision**: bfloat16
- **Block size**: 32 tokens
- **Training**: Multi-block Teacher Forcing (MultiTF) on math reasoning data

## How it works

1. The prompt is tokenized and padded to a block boundary
2. A new block of 32 MASK tokens is appended
3. The model iteratively denoises the MASK tokens (diffusion steps)
4. The process repeats for each new block until EOS or max blocks reached

## References

- 📄 Paper: [Multi-Block Diffusion Language Models](https://huggingface.co/papers/2606.29215)
- 🐙 GitHub: [SJTU-DENG-Lab/mbd-lms](https://github.com/SJTU-DENG-Lab/mbd-lms)
- 🤗 Model: [SJTU-DENG-Lab/MBD-Math-SDAR-8B-Chat-b32](https://huggingface.co/SJTU-DENG-Lab/MBD-Math-SDAR-8B-Chat-b32)