| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <meta name="description" content="Project page for Multi-Block Diffusion Language Models."> |
| <title>Multi-Block Diffusion Language Models</title> |
| <link rel="preconnect" href="https://fonts.googleapis.com"> |
| <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> |
| <link href="https://fonts.googleapis.com/css2?family=Noto+Sans:ital,wght@0,400;0,500;0,600;0,700;1,400&display=swap" rel="stylesheet"> |
| <link rel="stylesheet" href="style.css"> |
| </head> |
| <body> |
| <nav class="nav" aria-label="Site navigation"> |
| <div class="container"> |
| <a class="nav-logo" href="https://sjtu-deng-lab.github.io/" aria-label="DENG Lab website"> |
| <img src="assets/sjtu-deng-lab-logo.png" alt="DENG Lab"> |
| </a> |
| <a href="videos.html">Videos</a> |
| <a href="mbd-lms.html">MBD-LMs</a> |
| <a href="diffulex.html">Diffulex Engine</a> |
| </div> |
| </nav> |
|
|
| <header class="hero"> |
| <div class="container"> |
| <h1>Multi-Block Diffusion Language Models</h1> |
| <p class="subtitle">A training recipe, paradigm definition, and runnable inference path for practical Multi-Block Diffusion Language Models.</p> |
| <p class="authors"><a class="author-link" href="https://github.com/drewjin">Yijie Jin</a><sup>1</sup>, Jiajun Xu<sup>2</sup>, Yuxuan Liu<sup>1</sup>, Chenkai Xu<sup>1</sup>, Yi Tu<sup>3</sup>, Jiajun Li<sup>3</sup>, Dandan Tu<sup>3</sup>, Xiaohui Yan<sup>3</sup>, Kai Yu<sup>1</sup>, Pengfei Liu<sup>1</sup>, <a class="author-link" href="https://thudzj.github.io/">Zhijie Deng</a><sup>1,†</sup></p> |
| <p class="affiliations"><sup>1</sup>Shanghai Jiao Tong University | <sup>2</sup>Xi'an Jiao Tong University | <sup>3</sup>Huawei</p> |
| <p class="author-note"><sup>†</sup>Corresponding Author</p> |
| <p class="hero-links"> |
| <a href="https://arxiv.org/abs/2606.29215v1">Paper</a> |
| <span aria-hidden="true">·</span> |
| <a href="https://github.com/SJTU-DENG-Lab/mbd-lms">MBD-LMs</a> |
| <span aria-hidden="true">·</span> |
| <a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/mbd-lms">Reproduce</a> |
| <span aria-hidden="true">·</span> |
| <a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/main">Diffulex</a> |
| <span aria-hidden="true">·</span> |
| <a href="https://huggingface.co/collections/SJTU-DENG-Lab/mbd-lms">HuggingFace Weights & Data</a> |
| </p> |
| </div> |
| </header> |
|
|
| <main class="container"> |
| <section id="featured-video" class="video-section"> |
| <div class="section-kicker video-kicker">Featured Demo</div> |
| <h2>Watch Diffulex Decode in Motion</h2> |
| <p class="section-lede">From vanilla LLaDA2-Mini to MBD-LLaDA2-Mini to MBD-LLaDA2-Mini-DMax — the <strong>same prompts, same model backbone</strong>, massive decoding speedup. All on a single A100. See the full progression across four model variants on the videos page.</p> |
|
|
| <div class="video-feature"> |
| <div class="video-player"> |
| <video class="video-preview" muted loop controls preload="metadata" playsinline> |
| <source src="assets/videos/chosen.mp4" type="video/mp4"> |
| </video> |
| </div> |
| <div class="video-feature-copy"> |
| <span>Featured Diffulex trace</span> |
| <h3>MBD-LLaDA2-Mini-DMax Demo</h3> |
| <p>This selected trace uses <strong>MBD-LLaDA2-Mini-DMax</strong>, the fastest model we trained, running on a <strong>single NVIDIA A100-SXM4-80GB</strong> GPU through the Diffulex engine.</p> |
| <p class="video-note"><strong>Playback note.</strong> The demo videos pass through a Streamlit frontend, which can consume much of the engine-side throughput advantage. Use the aggregate TPS numbers on the <a href="diffulex.html">Diffulex page</a> to judge the actual engine path.</p> |
| <a class="button secondary" href="videos.html">See All 16 Demos</a> |
| </div> |
| </div> |
| </section> |
|
|
| <section id="explore"> |
| <h2>Explore the Project</h2> |
| <p class="section-lede">MBD-LMs spans three parts: demonstrated decoding results, the model-side paradigm, and the inference engine that makes it runnable.</p> |
| <div class="landing-nav"> |
| <a class="landing-nav-card videos" href="videos.html"> |
| <span class="card-kicker">Demo Videos</span> |
| <h2>Watch Diffulex in Motion</h2> |
| <p>16 traces across four model variants — LLaDA2-Mini, MBD-LLaDA2, MBD-LLaDA2-DMax, and DiffusionGemma — all generated by the Diffulex inference engine.</p> |
| </a> |
| <a class="landing-nav-card mbd" href="mbd-lms.html"> |
| <span class="card-kicker">Method</span> |
| <h2>MBD-LMs Paradigm</h2> |
| <p>MultiBD formulation, MultiTF post-training, Block Buffer runtime, interactive decode traces, full evaluation results, and throughput analysis.</p> |
| </a> |
| <a class="landing-nav-card diffulex" href="diffulex.html"> |
| <span class="card-kicker">Inference Engine</span> |
| <h2>Diffulex Engine</h2> |
| <p>The runnable inference path for block-style diffusion LMs. GSM8K benchmarks at mainstream-engine throughput, with MultiBD, prefix caching, and CUDA Graph support.</p> |
| </a> |
| </div> |
| </section> |
|
|
| </main> |
|
|
| <footer> |
| <div class="container"> |
| Multi-Block Diffusion Language Models. Train with <a href="https://github.com/SJTU-DENG-Lab/mbd-lms">mbd-lms</a>; reproduce with <a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/mbd-lms">Diffulex mbd-lms</a>; develop with <a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/main">Diffulex main</a>. |
| </div> |
| </footer> |
|
|
| <script> |
| (() => { |
| document.querySelectorAll(".video-preview").forEach((video) => { |
| video.play().catch(() => {}); |
| }); |
| })(); |
| </script> |
| </body> |
| </html> |
|
|
