File size: 5,938 Bytes
3cd1076 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 | <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Project page for Multi-Block Diffusion Language Models.">
<title>Multi-Block Diffusion Language Models</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Noto+Sans:ital,wght@0,400;0,500;0,600;0,700;1,400&display=swap" rel="stylesheet">
<link rel="stylesheet" href="style.css">
</head>
<body>
<nav class="nav" aria-label="Site navigation">
<div class="container">
<a class="nav-logo" href="https://sjtu-deng-lab.github.io/" aria-label="DENG Lab website">
<img src="assets/sjtu-deng-lab-logo.png" alt="DENG Lab">
</a>
<a href="videos.html">Videos</a>
<a href="mbd-lms.html">MBD-LMs</a>
<a href="diffulex.html">Diffulex Engine</a>
</div>
</nav>
<header class="hero">
<div class="container">
<h1>Multi-Block Diffusion Language Models</h1>
<p class="subtitle">A training recipe, paradigm definition, and runnable inference path for practical Multi-Block Diffusion Language Models.</p>
<p class="authors"><a class="author-link" href="https://github.com/drewjin">Yijie Jin</a><sup>1</sup>, Jiajun Xu<sup>2</sup>, Yuxuan Liu<sup>1</sup>, Chenkai Xu<sup>1</sup>, Yi Tu<sup>3</sup>, Jiajun Li<sup>3</sup>, Dandan Tu<sup>3</sup>, Xiaohui Yan<sup>3</sup>, Kai Yu<sup>1</sup>, Pengfei Liu<sup>1</sup>, <a class="author-link" href="https://thudzj.github.io/">Zhijie Deng</a><sup>1,†</sup></p>
<p class="affiliations"><sup>1</sup>Shanghai Jiao Tong University | <sup>2</sup>Xi'an Jiao Tong University | <sup>3</sup>Huawei</p>
<p class="author-note"><sup>†</sup>Corresponding Author</p>
<p class="hero-links">
<a href="https://arxiv.org/abs/2606.29215v1">Paper</a>
<span aria-hidden="true">·</span>
<a href="https://github.com/SJTU-DENG-Lab/mbd-lms">MBD-LMs</a>
<span aria-hidden="true">·</span>
<a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/mbd-lms">Reproduce</a>
<span aria-hidden="true">·</span>
<a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/main">Diffulex</a>
<span aria-hidden="true">·</span>
<a href="https://huggingface.co/collections/SJTU-DENG-Lab/mbd-lms">HuggingFace Weights & Data</a>
</p>
</div>
</header>
<main class="container">
<section id="featured-video" class="video-section">
<div class="section-kicker video-kicker">Featured Demo</div>
<h2>Watch Diffulex Decode in Motion</h2>
<p class="section-lede">From vanilla LLaDA2-Mini to MBD-LLaDA2-Mini to MBD-LLaDA2-Mini-DMax — the <strong>same prompts, same model backbone</strong>, massive decoding speedup. All on a single A100. See the full progression across four model variants on the videos page.</p>
<div class="video-feature">
<div class="video-player">
<video class="video-preview" muted loop controls preload="metadata" playsinline>
<source src="assets/videos/chosen.mp4" type="video/mp4">
</video>
</div>
<div class="video-feature-copy">
<span>Featured Diffulex trace</span>
<h3>MBD-LLaDA2-Mini-DMax Demo</h3>
<p>This selected trace uses <strong>MBD-LLaDA2-Mini-DMax</strong>, the fastest model we trained, running on a <strong>single NVIDIA A100-SXM4-80GB</strong> GPU through the Diffulex engine.</p>
<p class="video-note"><strong>Playback note.</strong> The demo videos pass through a Streamlit frontend, which can consume much of the engine-side throughput advantage. Use the aggregate TPS numbers on the <a href="diffulex.html">Diffulex page</a> to judge the actual engine path.</p>
<a class="button secondary" href="videos.html">See All 16 Demos</a>
</div>
</div>
</section>
<section id="explore">
<h2>Explore the Project</h2>
<p class="section-lede">MBD-LMs spans three parts: demonstrated decoding results, the model-side paradigm, and the inference engine that makes it runnable.</p>
<div class="landing-nav">
<a class="landing-nav-card videos" href="videos.html">
<span class="card-kicker">Demo Videos</span>
<h2>Watch Diffulex in Motion</h2>
<p>16 traces across four model variants — LLaDA2-Mini, MBD-LLaDA2, MBD-LLaDA2-DMax, and DiffusionGemma — all generated by the Diffulex inference engine.</p>
</a>
<a class="landing-nav-card mbd" href="mbd-lms.html">
<span class="card-kicker">Method</span>
<h2>MBD-LMs Paradigm</h2>
<p>MultiBD formulation, MultiTF post-training, Block Buffer runtime, interactive decode traces, full evaluation results, and throughput analysis.</p>
</a>
<a class="landing-nav-card diffulex" href="diffulex.html">
<span class="card-kicker">Inference Engine</span>
<h2>Diffulex Engine</h2>
<p>The runnable inference path for block-style diffusion LMs. GSM8K benchmarks at mainstream-engine throughput, with MultiBD, prefix caching, and CUDA Graph support.</p>
</a>
</div>
</section>
</main>
<footer>
<div class="container">
Multi-Block Diffusion Language Models. Train with <a href="https://github.com/SJTU-DENG-Lab/mbd-lms">mbd-lms</a>; reproduce with <a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/mbd-lms">Diffulex mbd-lms</a>; develop with <a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/main">Diffulex main</a>.
</div>
</footer>
<script>
(() => {
document.querySelectorAll(".video-preview").forEach((video) => {
video.play().catch(() => {});
});
})();
</script>
</body>
</html>
|