| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <meta name="description" content="MBD-LMs: A training recipe, paradigm definition, and runnable inference path for practical Multi-Block Diffusion Language Models."> |
| <title>MBD-LMs Paradigm — Multi-Block Diffusion Language Models</title> |
| <link rel="preconnect" href="https://fonts.googleapis.com"> |
| <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> |
| <link href="https://fonts.googleapis.com/css2?family=Noto+Sans:ital,wght@0,400;0,500;0,600;0,700;1,400&display=swap" rel="stylesheet"> |
| <link rel="stylesheet" href="style.css"> |
| </head> |
| <body> |
| <nav class="side-toc" aria-label="Table of contents"> |
| <ol> |
| <li><a href="#tldr"><span class="toc-num">1</span> TL;DR</a></li> |
| <li><a href="#contributions"><span class="toc-num">2</span> Contributions</a></li> |
| <li><a href="#decode-demo"><span class="toc-num">3</span> Decode Trace</a></li> |
| <li><a href="#motivation"><span class="toc-num">4</span> Motivation</a></li> |
| <li><a href="#formulation"><span class="toc-num">5</span> Formulation</a></li> |
| <li><a href="#multitf"><span class="toc-num">6</span> MultiTF</a></li> |
| <li><a href="#block-buffer"><span class="toc-num">7</span> Block Buffer</a></li> |
| <li><a href="#training-code"><span class="toc-num">8</span> Training Code</a></li> |
| <li><a href="#results"><span class="toc-num">9</span> Results</a></li> |
| <li><a href="#throughput"><span class="toc-num">10</span> Throughput</a></li> |
| <li><a href="#citation"><span class="toc-num">11</span> Citation</a></li> |
| </ol> |
| </nav> |
| <nav class="nav" aria-label="Site navigation"> |
| <div class="container"> |
| <a class="nav-logo" href="https://sjtu-deng-lab.github.io/" aria-label="DENG Lab website"> |
| <img src="assets/sjtu-deng-lab-logo.png" alt="DENG Lab"> |
| </a> |
| <a href="index.html">Home</a> |
| <a href="videos.html">Videos</a> |
| <a href="mbd-lms.html" class="is-current">MBD-LMs</a> |
| <a href="diffulex.html">Diffulex Engine</a> |
| </div> |
| </nav> |
|
|
| <main class="container"> |
| <section id="tldr"> |
| <h2>1. TL;DR</h2> |
| <p>Block Diffusion Language Models (BD-LMs) make diffusion-based text generation more practical by supporting KV caching and flexible-length generation. However, native BD-LMs usually perform <strong>Single-Block Diffusion (SingleBD)</strong>: each forward pass refines one noisy block conditioned on a clean cached prefix. This preserves the serving benefits of BD-LMs, but blocks are still processed sequentially.</p> |
| <p>We propose <strong>Multi-Block Diffusion Language Models (MBD-LMs)</strong>, a formulation and post-training recipe for reliable <strong>Multi-Block Diffusion (MultiBD)</strong>. On the model side, MBD-LMs are BD-LMs post-trained with <strong>Multi-block Teacher Forcing (MultiTF)</strong> so they can handle practical MultiBD running-set states. On the inference side, MBD-LMs decode a bounded running-set of consecutive blocks through an optimized <strong>Block Buffer</strong> runtime.</p> |
| <div class="definition-summary" role="group" aria-label="MBD-LMs definition"> |
| <div class="definition-summary-head"> |
| <span>MBD-LMs Definition</span> |
| <strong>BD-LMs trained for MultiBD states, then executed by a MultiBD runtime.</strong> |
| </div> |
| <div class="definition-grid"> |
| <div class="definition-card model-definition"> |
| <div class="definition-card-label">Model focus</div> |
| <p>MultiTF post-training teaches the model bounded noisy block groups, heterogeneous slot-wise mask ratios, and block-causal visibility patterns.</p> |
| </div> |
| <div class="definition-card inference-definition"> |
| <div class="definition-card-label">Inference focus</div> |
| <p>MultiBD keeps a bounded running-set, uses Block Buffer execution, preserves prefix KV caching, commits completed blocks, and runs through <a href="diffulex.html">Diffulex</a>.</p> |
| </div> |
| </div> |
| </div> |
| <p>The training and method code lives in <strong>SJTU-DENG-Lab/mbd-lms</strong>. The executable inference runtime is <strong>Diffulex</strong>: use the Diffulex <a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/mbd-lms"><code>mbd-lms</code></a> branch for experiment reproduction, and Diffulex <a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/main"><code>main</code></a> for engine development and new decoding algorithms.</p> |
| <p>Empirically, <strong>MBD-LLaDA2-Mini</strong> increases average Tokens Per Forward pass (TPF) from <strong>3.47 to 6.19</strong> and improves average accuracy from <strong>79.95% to 81.03%</strong>. When combined with DMax, <strong>MBD-LLaDA2-Mini-DMax</strong> reaches an average TPF of <strong>9.34</strong> with only a <strong>1.02 percentage-point</strong> average accuracy drop on math and code benchmarks.</p> |
| <figure class="figure noborder"> |
| <img src="assets/fig1_singlebd_vs_multibd.png" alt="SingleBD versus MultiBD"> |
| <figcaption><strong>Figure 1.</strong> SingleBD decodes blocks sequentially and creates KV-cache storing bubbles. MultiBD overlaps future-block refinement with KV-cache storing of completed blocks, enabling inter-block parallelism.</figcaption> |
| </figure> |
| </section> |
|
|
| <section id="contributions" class="contribution-section"> |
| <div class="section-kicker">Key Contributions</div> |
| <h2>2. Contributions</h2> |
| <p class="section-lede">The useful takeaway from MBD-LMs is not just "decode more blocks." The project ties together a model-side training distribution, an inference-time running-set abstraction, and a runtime path that keeps the system executable.</p> |
| <div class="contribution-grid"> |
| <article class="contribution-card"> |
| <div class="contribution-index">01</div> |
| <h3>MBD-LMs</h3> |
| <p>Reframes BD-LM generation as a bounded running-set of consecutive blocks, making inter-block parallelism explicit while preserving clean prefix KV semantics.</p> |
| </article> |
| <article class="contribution-card"> |
| <div class="contribution-index">02</div> |
| <h3>MultiTF post-training</h3> |
| <p>Constructs inference-like noisy block groups with systematic/random layouts, heterogeneous slot-wise mask ratios, and group-aware dual-stream masking.</p> |
| </article> |
| <article class="contribution-card"> |
| <div class="contribution-index">03</div> |
| <h3>Block Buffer runtime</h3> |
| <p>Executes MultiBD with fixed physical block slots, dummy-slot activation, prefix-cache reuse, decode-store overlap, and CUDA Graph-friendly shapes.</p> |
| </article> |
| <article class="contribution-card"> |
| <div class="contribution-index">04</div> |
| <h3>Train-to-engine split</h3> |
| <p>Keeps method training in <code>mbd-lms</code> and runnable inference in <a href="diffulex.html">Diffulex</a>, so reproduction and new dLLM serving work have clear entry points.</p> |
| </article> |
| </div> |
| </section> |
|
|
| <section id="decode-demo" class="demo-section"> |
| <div class="section-kicker demo-kicker">Interactive Decode Trace</div> |
| <h2>3. Why MultiBD Removes the Store Bubble</h2> |
| <p class="section-lede">Click through the same request under SingleBD and MultiBD. SingleBD can only refine one noisy block at a time; after the block is complete, it still spends a KV-store forward pass that produces no new output. MultiBD keeps a bounded running-set, admits the next block before the front block fully leaves the buffer, and overlaps KV storing with later-block decoding.</p> |
|
|
| <div class="decode-demo" role="group" aria-label="Interactive comparison between SingleBD and MultiBD"> |
| <div class="demo-topbar"> |
| <div> |
| <div class="demo-step-label" id="demo-step-label">Step 1 of 6</div> |
| <h3 id="demo-step-title">Start from a clean cached prefix</h3> |
| <p id="demo-step-detail">Both methods refine block 1 conditioned on the prefix KV cache.</p> |
| </div> |
| <div class="demo-controls" role="group" aria-label="Decode demo controls"> |
| <button type="button" id="demo-prev">Prev</button> |
| <button type="button" id="demo-next">Next step</button> |
| <button type="button" id="demo-reset">Reset</button> |
| </div> |
| </div> |
|
|
| <div class="demo-stage"> |
| <article class="decode-panel single-panel" aria-label="SingleBD trace"> |
| <div class="decode-panel-head"> |
| <span class="panel-tag single-tag">BD-LM / SingleBD</span> |
| <strong id="single-phase">Decode B1 only</strong> |
| </div> |
| <div class="panel-summary"> |
| <div><span>Forward</span><strong id="single-forward">B1 denoising</strong></div> |
| <div><span>Output</span><strong id="single-output">B1 token updates</strong></div> |
| <div><span>KV action</span><strong id="single-kv">Read prefix KV</strong></div> |
| </div> |
| <div class="block-track" role="group" aria-label="SingleBD block states"> |
| <div class="decode-block prefix-block state-prefix"> |
| <span class="block-name">Prefix</span> |
| <span class="block-status">KV cached</span> |
| </div> |
| <div class="decode-block" id="single-b1"> |
| <span class="block-name">Block 1</span> |
| <span class="block-status"></span> |
| <span class="block-fill"></span> |
| </div> |
| <div class="decode-block" id="single-b2"> |
| <span class="block-name">Block 2</span> |
| <span class="block-status"></span> |
| <span class="block-fill"></span> |
| </div> |
| <div class="decode-block" id="single-b3"> |
| <span class="block-name">Block 3</span> |
| <span class="block-status"></span> |
| <span class="block-fill"></span> |
| </div> |
| <div class="decode-block" id="single-b4"> |
| <span class="block-name">Block 4</span> |
| <span class="block-status"></span> |
| <span class="block-fill"></span> |
| </div> |
| </div> |
| <div class="timeline" role="group" data-timeline="single" aria-label="SingleBD timeline"> |
| <span class="timeline-step">Decode B1</span> |
| <span class="timeline-step">Decode B1</span> |
| <span class="timeline-step">Finalize B1</span> |
| <span class="timeline-step bubble-step">Store B1 only</span> |
| <span class="timeline-step">Start B2</span> |
| <span class="timeline-step">Decode B2</span> |
| <span class="timeline-step">Finalize B2</span> |
| <span class="timeline-step bubble-step">Store B2 only</span> |
| <span class="timeline-step">Start B3</span> |
| <span class="timeline-step">Decode B3</span> |
| </div> |
| <p class="panel-note" id="single-note">Later blocks wait until block 1 is stored into KV.</p> |
| </article> |
|
|
| <article class="decode-panel multi-panel" aria-label="MultiBD trace"> |
| <div class="decode-panel-head"> |
| <span class="panel-tag multi-tag">MBD-LM / MultiBD</span> |
| <strong id="multi-phase">Decode B1 in a buffer</strong> |
| </div> |
| <div class="panel-summary"> |
| <div><span>Forward</span><strong id="multi-forward">B1 denoising</strong></div> |
| <div><span>Output</span><strong id="multi-output">B1 token updates</strong></div> |
| <div><span>KV action</span><strong id="multi-kv">Read prefix KV</strong></div> |
| </div> |
| <div class="block-track" role="group" aria-label="MultiBD block states"> |
| <div class="decode-block prefix-block state-prefix"> |
| <span class="block-name">Prefix</span> |
| <span class="block-status">KV cached</span> |
| </div> |
| <div class="decode-block" id="multi-b1"> |
| <span class="block-name">Block 1</span> |
| <span class="block-status"></span> |
| <span class="block-fill"></span> |
| </div> |
| <div class="decode-block" id="multi-b2"> |
| <span class="block-name">Block 2</span> |
| <span class="block-status"></span> |
| <span class="block-fill"></span> |
| </div> |
| <div class="decode-block" id="multi-b3"> |
| <span class="block-name">Block 3</span> |
| <span class="block-status"></span> |
| <span class="block-fill"></span> |
| </div> |
| <div class="decode-block" id="multi-b4"> |
| <span class="block-name">Block 4</span> |
| <span class="block-status"></span> |
| <span class="block-fill"></span> |
| </div> |
| </div> |
| <div class="timeline" role="group" data-timeline="multi" aria-label="MultiBD timeline"> |
| <span class="timeline-step">Decode B1</span> |
| <span class="timeline-step">Stabilize B1</span> |
| <span class="timeline-step overlap-step">Admit B2</span> |
| <span class="timeline-step overlap-step">Decode B1+B2</span> |
| <span class="timeline-step overlap-step">Store B1 + Decode B2</span> |
| <span class="timeline-step overlap-step">Slide buffer</span> |
| <span class="timeline-step overlap-step">Decode B2+B3</span> |
| <span class="timeline-step overlap-step">Admit B4</span> |
| <span class="timeline-step overlap-step">Store B2 + Decode tail</span> |
| <span class="timeline-step overlap-step">Steady pipeline</span> |
| </div> |
| <p class="panel-note" id="multi-note">The next block is already inside the running-set before block 1 leaves the buffer.</p> |
| </article> |
| </div> |
|
|
| <div class="demo-takeaway" id="demo-takeaway"> |
| <strong>Takeaway:</strong> MultiBD turns the store-only bubble into useful later-block decoding work. |
| </div> |
| </div> |
| </section> |
|
|
| <section id="motivation"> |
| <h2>4. From SingleBD to MultiBD</h2> |
| <p>Diffusion Language Models (DLMs) generate text through iterative denoising and naturally support parallel token refinement. Fully bidirectional DLMs, however, are difficult to serve efficiently because they do not naturally support KV caching or dynamic-length generation. BD-LMs address this issue by generating text in block-causal form: completed blocks become a clean cached prefix, and the current block is denoised under block-causal attention.</p> |
| <p>This design gives native BD-LMs efficient <strong>intra-block</strong> parallelism, but not <strong>inter-block</strong> parallelism. In SingleBD, a later block cannot begin refinement until the current block has finished decoding and has been committed to the KV cache. The result is a storing bubble: during cache storing, no new token is generated and no decode-store overlap is exploited.</p> |
| <p>MultiBD removes this bottleneck by maintaining a small running-set of consecutive blocks. Earlier blocks in the running-set may be completed and waiting to enter the cache, while later blocks can already be active noisy blocks. This enables the model to refine future blocks while completed blocks are being committed to the KV cache.</p> |
|
|
| <h3>Why training-free MultiBD is not enough</h3> |
| <p>A natural question is whether existing BD-LMs can simply run MultiBD at inference time. The paper shows that this is only partially effective. Direct MultiBD inference increases TPF, confirming that multi-block decoding relaxes the single-block bottleneck, but it can degrade accuracy because the model was not trained on practical MultiBD states.</p> |
| <p>The mismatch has two components. First, practical MultiBD does not decode an unbounded noisy suffix. It uses a <strong>bounded running-set</strong>, often with an active part around two blocks and occasional expansion to three or four active blocks. Second, active slots can have <strong>heterogeneous mask-ratio patterns</strong>: adjacent slots may differ substantially in noise level. Reliable MultiBD therefore requires training states that match both the bounded running-set structure and the slot-wise noise patterns observed during inference.</p> |
| <figure class="figure"> |
| <img src="assets/fig2_alignment_stats.png" alt="Train-inference statistics for MultiBD"> |
| <figcaption><strong>Figure 2.</strong> Train-inference statistics for MultiBD. D2F-style schedules, chain-uniform MultiTF schedules, inference-time mask ratios, and active-block trajectories reveal the bounded and heterogeneous nature of practical MultiBD inference.</figcaption> |
| </figure> |
| </section> |
|
|
| <section id="formulation"> |
| <h2>5. MBD-LMs: A Running-Set View of BD-LMs</h2> |
| <p>MBD-LMs formulate BD-LM generation around a <strong>running-set</strong> of consecutive blocks. At decoding step <code>s</code>, the running-set contains the blocks that have not yet entered the prefix KV cache. It includes active noisy blocks and completed preceding blocks waiting to be cached. Blocks before the running-set form the clean cached prefix.</p> |
| <p>This view unifies several regimes. Teacher-Forcing-trained BD-LMs correspond to the SingleBD extreme, where the model observes one noisy block conditioned on a clean cached prefix. D2F introduces visibility among multiple noisy blocks, but its training states still differ from practical MultiBD in running-set size and slot-wise noise patterns. Practical MultiBD is the bounded intermediate regime: the running-set should be larger than one to expose inter-block parallelism, but small enough to keep each forward pass efficient.</p> |
| <figure class="figure"> |
| <img class="figure-narrow" src="assets/fig3_train_inference_paradigms.png" alt="Train-inference alignment across paradigms"> |
| <figcaption><strong>Figure 3.</strong> TF and D2F provide existing BD-LM training states, but neither directly matches practical MultiBD. MultiTF builds inference-like noise-groups with heterogeneous slot-wise noise patterns.</figcaption> |
| </figure> |
| </section> |
|
|
| <section id="multitf"> |
| <h2>6. MultiTF: Post-Training BD-LMs for MultiBD</h2> |
| <p><strong>Multi-block Teacher Forcing (MultiTF)</strong> turns BD-LMs into MBD-LMs by constructing training states that resemble practical MultiBD inference. Instead of corrupting only one block as in standard teacher forcing, MultiTF corrupts a bounded group of consecutive blocks, called a <strong>noise-group</strong>, while conditioning later groups on clean earlier groups.</p> |
| <div class="grid"> |
| <div class="card"><h3>Noise-group layouts</h3><p>Systematic layouts enumerate group sizes and shifts so that blocks appear at different group-relative positions. Random layouts add non-regular group-size combinations and boundary patterns.</p></div> |
| <div class="card"><h3>Chain-uniform scheduling</h3><p>Within each noise-group, mask ratios are sampled monotonically but randomly, producing larger and more diverse slot-wise noise gaps than a fixed D2F-style monotonic schedule.</p></div> |
| <div class="card"><h3>Dual-stream masking</h3><p>Noisy blocks inside the same noise-group can attend to each other under block-causal visibility, each noise-group can condition on its clean prefix, and clean tokens are prevented from attending to noisy tokens.</p></div> |
| </div> |
| <p>The resulting inputs are used for masked-token cross-entropy, and model-specific objectives such as DMax OPUT can be applied on top of the same MultiTF input construction.</p> |
| <figure class="figure"> |
| <img src="assets/fig4_multitf_overview.png" alt="Overview of MultiTF"> |
| <figcaption><strong>Figure 4.</strong> MultiTF constructs systematic and random noise-group layouts, applies a Group-Aware Dual-Stream Mask, and post-trains BD-LMs into MBD-LMs.</figcaption> |
| </figure> |
| </section> |
|
|
| <section id="block-buffer"> |
| <h2>7. Optimized MultiBD with Block Buffer</h2> |
| <p>MultiBD is useful only if the additional parallelism can be translated into wall-clock speedup. A naive implementation directly materializes the current running-set as the physical input to each forward pass. This exposes inter-block parallelism, but the number of processed tokens changes over time and across requests, making CUDA Graph capture and replay difficult.</p> |
| <p>To make MultiBD practically executable, the paper introduces the <strong>Block Buffer</strong> mechanism. A Block Buffer contains a fixed number of physical block slots. Real resident blocks inside the buffer form the logical running-set, while trailing dummy slots reserve capacity for future blocks. A future block enters decoding by activating an existing dummy slot instead of extending the physical input sequence. When the front block is completed, it is committed to the KV cache and the buffer slides forward by appending a new dummy slot at the tail.</p> |
| <p>Each slot follows the state transition <strong>dummy → active → to-cache → in-cache</strong>. This design preserves prefix-cache reuse, keeps input shapes static, overlaps decoding with KV-cache storing, and supports CUDA Graph replay.</p> |
| <figure class="figure"> |
| <img src="assets/fig5_block_buffer.png" alt="Block Buffer inference pipeline"> |
| <figcaption><strong>Figure 5.</strong> MultiBD inference with Block Buffer. A fixed block-buffer hierarchy enables parallel block refinement while preserving prefix-cache semantics and static-shape execution.</figcaption> |
| </figure> |
| </section> |
|
|
| <section id="training-code" class="training-section"> |
| <div class="section-kicker">Training Repository</div> |
| <h2>8. mbd-lms Defines and Trains MBD-LMs</h2> |
| <p class="section-lede">The <a href="https://github.com/SJTU-DENG-Lab/mbd-lms"><code>SJTU-DENG-Lab/mbd-lms</code></a> repository is the home for the method-side work. It is where Multi-block Teacher Forcing is implemented, where training configs live, and where checkpoints are prepared before they are evaluated through the <a href="diffulex.html">Diffulex</a> runtime.</p> |
| <div class="grid"> |
| <div class="card"> |
| <h3>MultiTF training</h3> |
| <p>The repository contains the post-training path that constructs bounded noisy block groups, heterogeneous slot-wise mask ratios, and group-aware attention masks for practical MultiBD states.</p> |
| </div> |
| <div class="card"> |
| <h3>Training assets</h3> |
| <p>Use this repo for environment setup, dataset preparation, model-specific training configs, multi-node launch scripts, and checkpoint conversion utilities.</p> |
| </div> |
| <div class="card"> |
| <h3>Method documentation</h3> |
| <p>The project page, guidelines, and figures define the SingleBD-to-MultiBD transition, MultiTF, Block Buffer inference, and the reported training/evaluation setup.</p> |
| </div> |
| </div> |
| <div class="repo-flow"> |
| <div> |
| <h3>Train and Prepare</h3> |
| <p>Start here when working on MBD-LM training, reproducing MultiTF data construction, or converting trained checkpoints into usable model artifacts.</p> |
| <a class="button" href="https://github.com/SJTU-DENG-Lab/mbd-lms">Open mbd-lms Training Repo</a> |
| </div> |
| <div class="engine-target"> |
| <h3>Run and Serve</h3> |
| <p>Move to Diffulex when you need benchmark execution, HTTP serving, optimized kernels, prefix caching, and system-level MultiBD runtime behavior.</p> |
| <a class="button engine secondary" href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/mbd-lms">Open Diffulex Reproduction Branch</a> |
| </div> |
| </div> |
| </section> |
|
|
| <section id="results"> |
| <h2>9. Main Results</h2> |
| <p>The experiments evaluate mathematical reasoning on GSM8K and MATH500, and code generation on MBPP+ and HumanEval+. The paper reports Accuracy, Tokens Per Forward pass (TPF), and Accuracy Under Parallelism (AUP), where TPF measures decoding parallelism and AUP summarizes the accuracy-parallelism trade-off.</p> |
| <p>The main trend is consistent across models: MBD-LMs substantially improve TPF over native SingleBD, and MultiTF often recovers or improves the quality lost by training-free MultiBD. On LLaDA2-Mini, MultiTF raises average accuracy from <strong>78.59%</strong> under training-free MultiBD to <strong>81.03%</strong>, while further increasing average TPF from <strong>4.41</strong> to <strong>6.19</strong>. On SDAR-8B-Chat-b32, MBD-SDAR-8B-Chat-b32 increases average TPF from <strong>2.54</strong> to <strong>4.46</strong> and improves average accuracy from <strong>69.00%</strong> to <strong>69.74%</strong>.</p> |
| <div class="result-highlights" role="group" aria-label="Selected result highlights"> |
| <div class="result-highlight"> |
| <span>Training-free MultiBD</span> |
| <strong>3.47 -> 4.41 TPF</strong> |
| <p>LLaDA2-Mini gains parallelism immediately, but accuracy drops before alignment training.</p> |
| </div> |
| <div class="result-highlight is-primary"> |
| <span>MultiTF aligned</span> |
| <strong>4.41 -> 6.19 TPF</strong> |
| <p>MBD-LLaDA2-Mini recovers quality and raises average accuracy to 81.03%.</p> |
| </div> |
| <div class="result-highlight"> |
| <span>DMax compatible</span> |
| <strong>9.34 TPF</strong> |
| <p>MBD-LLaDA2-Mini-DMax reaches the highest reported average parallelism.</p> |
| </div> |
| </div> |
| <div class="table-wrap result-table-wrap"> |
| <table class="result-table"> |
| <caption>Selected aggregate results from the reported math and code evaluations.</caption> |
| <thead> |
| <tr> |
| <th scope="col">Model / Setting</th> |
| <th scope="col">Avg. Accuracy</th> |
| <th scope="col">Avg. TPF</th> |
| <th scope="col">Interpretation</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td>LLaDA2-Mini SingleBD</td> |
| <td>79.95%</td> |
| <td>3.47</td> |
| <td>Native one-block baseline.</td> |
| </tr> |
| <tr> |
| <td>LLaDA2-Mini training-free MultiBD</td> |
| <td>78.59%</td> |
| <td>4.41</td> |
| <td>Parallelism improves, but train-inference mismatch hurts quality.</td> |
| </tr> |
| <tr class="is-best"> |
| <td>MBD-LLaDA2-Mini</td> |
| <td>81.03%</td> |
| <td>6.19</td> |
| <td>MultiTF aligns the model with practical MultiBD states.</td> |
| </tr> |
| <tr> |
| <td>SDAR-8B-Chat-b32 SingleBD</td> |
| <td>69.00%</td> |
| <td>2.54</td> |
| <td>Second-model baseline for transfer.</td> |
| </tr> |
| <tr class="is-best"> |
| <td>MBD-SDAR-8B-Chat-b32</td> |
| <td>69.74%</td> |
| <td>4.46</td> |
| <td>Shows the same parallelism-quality trend beyond LLaDA2.</td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| <div class="table-wrap paper-table-wrap"> |
| <table class="paper-table main-results-table"> |
| <caption><strong>Table 1.</strong> Evaluation results across math and code benchmarks. MBD-LMs consistently improve TPF over SingleBD and improve the accuracy-parallelism trade-off in most settings.</caption> |
| <thead> |
| <tr> |
| <th scope="col" rowspan="2">Model</th> |
| <th scope="col" colspan="2">GSM8K</th> |
| <th scope="col" colspan="2">MATH500</th> |
| <th scope="col" colspan="2">MBPP+</th> |
| <th scope="col" colspan="2">HumanEval+</th> |
| <th scope="col" colspan="3">Average</th> |
| </tr> |
| <tr> |
| <th scope="col">Acc</th> |
| <th scope="col">TPF</th> |
| <th scope="col">Acc</th> |
| <th scope="col">TPF</th> |
| <th scope="col">Acc</th> |
| <th scope="col">TPF</th> |
| <th scope="col">Acc</th> |
| <th scope="col">TPF</th> |
| <th scope="col">Acc</th> |
| <th scope="col">TPF</th> |
| <th scope="col">AUP</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr class="paper-group-row"><th scope="rowgroup" colspan="12">LLaDA2-Mini-DMax <span>(bufsz=2, blksz=32)</span></th></tr> |
| <tr> |
| <td>SingleBD (Native)</td><td>91.89</td><td>5.70</td><td>76.80</td><td>6.13</td><td>72.22</td><td>6.14</td><td>77.44</td><td>7.44</td><td>79.59</td><td>6.35</td><td>459.54</td> |
| </tr> |
| <tr class="training-free-row"> |
| <td>MultiBD (training-free)</td><td>89.84</td><td>8.76</td><td>73.80</td><td>9.08</td><td>72.22</td><td>8.44</td><td>76.83</td><td>10.96</td><td>78.17</td><td>9.31</td><td>651.98</td> |
| </tr> |
| <tr class="mbd-row"> |
| <td>MBD-LLaDA2-Mini-DMax</td><td>91.74</td><td>8.95</td><td>75.00</td><td>9.31</td><td>70.11</td><td>8.34</td><td>77.44</td><td>10.78</td><td>78.57</td><td>9.34</td><td>661.28</td> |
| </tr> |
| <tr class="paper-group-row"><th scope="rowgroup" colspan="12">LLaDA2-Mini <span>(bufsz=2, blksz=32)</span></th></tr> |
| <tr> |
| <td>SingleBD (Native)</td><td>91.89</td><td>2.27</td><td>74.20</td><td>2.83</td><td>75.66</td><td>3.25</td><td>78.05</td><td>5.53</td><td>79.95</td><td>3.47</td><td>247.41</td> |
| </tr> |
| <tr class="training-free-row"> |
| <td>MultiBD (training-free)</td><td>92.65</td><td>2.76</td><td>73.60</td><td>3.53</td><td>72.49</td><td>3.97</td><td>75.61</td><td>7.37</td><td>78.59</td><td>4.41</td><td>301.81</td> |
| </tr> |
| <tr class="mbd-row"> |
| <td>MBD-LLaDA2-Mini</td><td>91.96</td><td>5.55</td><td>79.20</td><td>6.02</td><td>72.49</td><td>5.35</td><td>80.49</td><td>7.85</td><td>81.03</td><td>6.19</td><td>449.18</td> |
| </tr> |
| <tr class="paper-group-row"><th scope="rowgroup" colspan="12">SDAR-8B-Chat-b32 <span>(bufsz=4, blksz=32)</span></th></tr> |
| <tr> |
| <td>SingleBD (Native)</td><td>90.07</td><td>2.52</td><td>65.60</td><td>3.81</td><td>52.65</td><td>1.83</td><td>67.68</td><td>2.00</td><td>69.00</td><td>2.54</td><td>141.64</td> |
| </tr> |
| <tr class="training-free-row"> |
| <td>MultiBD (training-free)</td><td>89.01</td><td>2.78</td><td>60.60</td><td>5.06</td><td>52.12</td><td>1.97</td><td>65.85</td><td>2.24</td><td>66.89</td><td>3.01</td><td>156.35</td> |
| </tr> |
| <tr class="mbd-row"> |
| <td>MBD-SDAR-8B-Chat-b32</td><td>89.16</td><td>3.08</td><td>68.00</td><td>5.08</td><td>58.99</td><td>4.87</td><td>62.80</td><td>4.82</td><td>69.74</td><td>4.46</td><td>210.42</td> |
| </tr> |
| <tr class="paper-group-row"><th scope="rowgroup" colspan="12">SDAR-8B-Chat-b4 <span>(bufsz=4, blksz=4)</span></th></tr> |
| <tr> |
| <td>SingleBD (Native)</td><td>91.05</td><td>1.33</td><td>72.80</td><td>1.46</td><td>64.80</td><td>1.13</td><td>73.70</td><td>1.07</td><td>75.59</td><td>1.25</td><td>85.46</td> |
| </tr> |
| <tr class="training-free-row"> |
| <td>MultiBD (training-free)</td><td>90.45</td><td>2.39</td><td>70.60</td><td>2.68</td><td>65.80</td><td>1.55</td><td>74.39</td><td>1.47</td><td>75.31</td><td>2.00</td><td>129.59</td> |
| </tr> |
| <tr class="mbd-row"> |
| <td>MBD-SDAR-8B-Chat-b4</td><td>91.81</td><td>2.28</td><td>72.40</td><td>2.52</td><td>64.29</td><td>2.62</td><td>72.56</td><td>2.24</td><td>75.27</td><td>2.42</td><td>148.65</td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| <p>Ablations further support the training-state alignment story. Combining systematic and random layouts gives the best AUP among the layout variants. Replacing the chain-uniform scheduler with other schedulers reduces the accuracy-parallelism trade-off; in particular, the D2F-style monotonic scheduler causes a large accuracy drop in the reported ablation, indicating that noisy-block visibility alone is not sufficient when slot-wise noise patterns are mismatched.</p> |
| <div class="table-stack"> |
| <div class="table-wrap paper-table-wrap"> |
| <table class="paper-table transfer-table"> |
| <caption><strong>Table 2a.</strong> Training-free MultiBD transfers to additional model variants.</caption> |
| <thead> |
| <tr> |
| <th scope="col" rowspan="2">Model</th> |
| <th scope="col" colspan="2">GSM8K</th> |
| <th scope="col" colspan="2">MATH500</th> |
| <th scope="col" colspan="3">Average</th> |
| </tr> |
| <tr> |
| <th scope="col">Acc</th> |
| <th scope="col">TPF</th> |
| <th scope="col">Acc</th> |
| <th scope="col">TPF</th> |
| <th scope="col">Acc</th> |
| <th scope="col">TPF</th> |
| <th scope="col">AUP</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr class="paper-group-row"><th scope="rowgroup" colspan="8">LLaDA2-Mini-CAP <span>(bufsz=2, blksz=32)</span></th></tr> |
| <tr> |
| <td>SingleBD (Native)</td><td>91.74</td><td>3.08</td><td>77.80</td><td>3.71</td><td>84.77</td><td>3.40</td><td>247.30</td> |
| </tr> |
| <tr class="training-free-row"> |
| <td>MultiBD (training-free)</td><td>91.21</td><td>4.00</td><td>77.20</td><td>4.94</td><td>84.21</td><td>4.47</td><td>319.17</td> |
| </tr> |
| <tr class="paper-group-row"><th scope="rowgroup" colspan="8">LLaDA2.1-Mini <span>(bufsz=2, blksz=32)</span></th></tr> |
| <tr> |
| <td>SingleBD (Native)</td><td>93.03</td><td>4.12</td><td>81.40</td><td>4.87</td><td>87.22</td><td>4.50</td><td>390.64</td> |
| </tr> |
| <tr class="training-free-row"> |
| <td>MultiBD (training-free)</td><td>92.27</td><td>5.80</td><td>81.00</td><td>7.20</td><td>86.63</td><td>6.50</td><td>558.52</td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| <div class="ablation-split"> |
| <div class="table-wrap paper-table-wrap"> |
| <table class="paper-table ablation-table"> |
| <caption><strong>Table 2b-i.</strong> Noise-group layout construction ablation.</caption> |
| <thead> |
| <tr> |
| <th scope="col">Configuration</th> |
| <th scope="col">Acc</th> |
| <th scope="col">TPF</th> |
| <th scope="col">AUP</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td>SingleBD (Native)</td><td>84.67</td><td>6.57</td><td>536.89</td> |
| </tr> |
| <tr> |
| <td>+ systematic layouts</td><td>83.22</td><td>9.71</td><td>774.03</td> |
| </tr> |
| <tr> |
| <td>+ random layouts</td><td>82.72</td><td>9.42</td><td>747.46</td> |
| </tr> |
| <tr class="mbd-row"> |
| <td>systematic + random layouts (ours)</td><td>84.59</td><td>9.87</td><td>805.34</td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| <div class="table-wrap paper-table-wrap"> |
| <table class="paper-table ablation-table"> |
| <caption><strong>Table 2b-ii.</strong> Block-level noise-scheduler ablation.</caption> |
| <thead> |
| <tr> |
| <th scope="col">Configuration</th> |
| <th scope="col">Acc</th> |
| <th scope="col">TPF</th> |
| <th scope="col">AUP</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td>SingleBD (Native)</td><td>84.67</td><td>6.57</td><td>536.89</td> |
| </tr> |
| <tr> |
| <td>D2F-style monotonic scheduler</td><td>—</td><td>—</td><td>—</td> |
| </tr> |
| <tr> |
| <td>random scheduler</td><td>83.14</td><td>9.70</td><td>771.74</td> |
| </tr> |
| <tr> |
| <td>sorted-uniform scheduler</td><td>81.28</td><td>9.73</td><td>748.73</td> |
| </tr> |
| <tr class="mbd-row"> |
| <td>chain-uniform scheduler (ours)</td><td>84.59</td><td>9.87</td><td>805.34</td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| </div> |
| </div> |
| </section> |
|
|
| <section id="throughput"> |
| <h2>10. Throughput: From TPF to TPS</h2> |
| <p>Higher TPF does not automatically imply proportional wall-clock speedup because MultiBD processes a larger static Block Buffer at each forward pass. The paper therefore separates useful committed tokens from the per-step computational workload. Increasing the buffer size can improve throughput when the useful-token gain outweighs the extra per-step cost introduced by resident blocks and dummy slots.</p> |
| <p class="note"><strong>Engine version note.</strong> Table 3 was measured with the older Diffulex release used by the public <a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/mbd-lms"><code>mbd-lms</code></a> reproduction branch. Current Diffulex is faster, but we have not refreshed this exact H100 TP=2 result because we do not currently have access to an H100 machine with CUDA 13 under the original setting.</p> |
| <p>On the legacy H100 TP=2 setup reported in Table 3, MBD-LLaDA2-Mini increases average TPF from <strong>3.47</strong> to <strong>6.19</strong> while step latency rises from <strong>7.07 ms</strong> to <strong>8.78 ms</strong>. The measured average TPS increases from <strong>517.16</strong> to <strong>745.92</strong>. With DMax, MBD-LLaDA2-Mini-DMax increases average TPF from <strong>6.35</strong> to <strong>9.34</strong>, and average TPS rises from <strong>779.49</strong> to <strong>926.67</strong> in the same table.</p> |
| <div class="throughput-strip" role="group" aria-label="Throughput highlights"> |
| <div> |
| <span>MBD-LLaDA2-Mini</span> |
| <strong>517.16 -> 745.92 TPS</strong> |
| </div> |
| <div> |
| <span>MBD-LLaDA2-Mini-DMax</span> |
| <strong>779.49 -> 926.67 TPS</strong> |
| </div> |
| <div> |
| <span>Hardware</span> |
| <strong>2x H100, TP=2</strong> |
| </div> |
| </div> |
| <div class="table-wrap paper-table-wrap"> |
| <table class="paper-table throughput-table"> |
| <caption><strong>Table 3.</strong> Throughput and single-step latency comparison on two H100 GPUs with TP=2, measured with the older Diffulex <code>mbd-lms</code> branch used for reproduction. MultiBD improves realized TPS despite increasing per-step latency.</caption> |
| <thead> |
| <tr> |
| <th scope="col" rowspan="2">Model</th> |
| <th scope="col" colspan="4">Forward-step statistics</th> |
| <th scope="col" colspan="6">Realized throughput</th> |
| </tr> |
| <tr> |
| <th scope="col">Avg. TPF</th> |
| <th scope="col">TPF Gain</th> |
| <th scope="col">Step Lat. (ms)</th> |
| <th scope="col">Lat. Cost</th> |
| <th scope="col">GSM8K TPS</th> |
| <th scope="col">MATH500 TPS</th> |
| <th scope="col">MBPP+ TPS</th> |
| <th scope="col">HumanEval+ TPS</th> |
| <th scope="col">Avg. TPS</th> |
| <th scope="col">TPS Gain</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td>LLaDA2-Mini</td><td>3.47</td><td>—</td><td>7.07</td><td>1.00x</td><td>344.05</td><td>403.45</td><td>496.19</td><td>824.94</td><td>517.16</td><td>—</td> |
| </tr> |
| <tr class="mbd-row"> |
| <td>MBD-LLaDA2-Mini</td><td>6.19</td><td>+78.39%</td><td>8.78</td><td>1.24x</td><td>687.87</td><td>707.89</td><td>646.73</td><td>941.18</td><td>745.92</td><td>+44.24%</td> |
| </tr> |
| <tr> |
| <td>LLaDA2-Mini-DMax</td><td>6.35</td><td>+83.00%</td><td>9.02</td><td>1.28x</td><td>700.82</td><td>730.60</td><td>754.97</td><td>931.55</td><td>779.49</td><td>+50.73%</td> |
| </tr> |
| <tr class="mbd-row"> |
| <td>MBD-LLaDA2-Mini-DMax</td><td>9.34</td><td>+169.16%</td><td>11.20</td><td>1.58x</td><td>834.52</td><td>851.07</td><td>896.65</td><td>1124.43</td><td>926.67</td><td>+79.19%</td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| </section> |
|
|
| <section id="citation" class="compact"> |
| <h2>11. Citation</h2> |
| <p>A formal arXiv record is on the way. Until then, please cite MBD-LMs with the temporary BibTeX entry below.</p> |
| <div class="citation-box"> |
| <div class="citation-box-head"> |
| <span>Temporary BibTeX</span> |
| <button type="button" id="copy-bibtex" aria-label="Copy temporary BibTeX">Copy</button> |
| </div> |
| <pre><code id="bibtex-code">@misc{jin2026mbdlms, |
| title = {Multi-Block Diffusion Language Models}, |
| author = {Yijie Jin and Jiajun Xu and Yuxuan Liu and Chenkai Xu and Yi Tu and Jiajun Li and Dandan Tu and Xiaohui Ye and Kai Yu and Pengfei Liu and Zhijie Deng}, |
| year = {2026}, |
| note = {arXiv on the way} |
| }</code></pre> |
| </div> |
| <h3>References</h3> |
| <ol> |
| <li>Marianne Arriola et al. <em>Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models</em>. ICLR, 2025.</li> |
| <li>Tiwei Bie et al. <em>LLaDA2.0: Scaling Up Diffusion Language Models to 100B</em>. arXiv preprint, 2025.</li> |
| <li>Xu Wang et al. <em>Diffusion LLMs Can Do Faster-than-AR Inference via Discrete Diffusion Forcing</em>. arXiv preprint, 2025.</li> |
| <li>Zigeng Chen et al. <em>DMax: Aggressive Parallel Decoding for dLLMs</em>. arXiv preprint, 2026.</li> |
| <li>Shuang Cheng et al. <em>SDAR: A Synergistic Diffusion-Autoregression Paradigm for Scalable Sequence Generation</em>. arXiv preprint, 2025.</li> |
| </ol> |
| </section> |
| </main> |
|
|
| <footer> |
| <div class="container"> |
| Multi-Block Diffusion Language Models. Train with <a href="https://github.com/SJTU-DENG-Lab/mbd-lms">mbd-lms</a>; reproduce with <a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/mbd-lms">Diffulex mbd-lms</a>; develop with <a href="https://github.com/SJTU-DENG-Lab/Diffulex/tree/main">Diffulex main</a>. |
| </div> |
| </footer> |
| <script> |
| (() => { |
| const steps = [ |
| { |
| title: "Start from a clean cached prefix", |
| detail: "Both methods refine block 1 conditioned on the prefix KV cache.", |
| takeaway: "Both systems begin the same way: one clean prefix, one noisy block.", |
| single: { |
| phase: "Decode B1 only", |
| forward: "B1 denoising", |
| output: "B1 token updates", |
| kv: "Read prefix KV", |
| note: "Later blocks wait until block 1 is stored into KV.", |
| timeline: 0, |
| blocks: { |
| b1: ["active", "Denoising", 24], |
| b2: ["waiting", "Not admitted", 0], |
| b3: ["future", "Future", 0], |
| b4: ["future", "Future", 0] |
| } |
| }, |
| multi: { |
| phase: "Decode B1 in a buffer", |
| forward: "B1 denoising", |
| output: "B1 token updates", |
| kv: "Read prefix KV", |
| note: "The buffer already reserves physical slots for later blocks.", |
| timeline: 0, |
| blocks: { |
| b1: ["active", "Denoising", 24], |
| b2: ["dummy", "Dummy slot", 0], |
| b3: ["dummy", "Dummy slot", 0], |
| b4: ["future", "Future", 0] |
| } |
| } |
| }, |
| { |
| title: "Refine block 1 inside the current window", |
| detail: "SingleBD spends another forward on the same block. MultiBD does the same useful work, but its fixed buffer already has room for the next active block.", |
| takeaway: "At this point, the speed difference is not visible yet; MultiBD is preparing the overlap.", |
| single: { |
| phase: "Continue B1", |
| forward: "B1 denoising", |
| output: "B1 token updates", |
| kv: "Read prefix KV", |
| note: "The decode window still contains only block 1.", |
| timeline: 1, |
| blocks: { |
| b1: ["active", "More stable", 48], |
| b2: ["waiting", "Not admitted", 0], |
| b3: ["future", "Future", 0], |
| b4: ["future", "Future", 0] |
| } |
| }, |
| multi: { |
| phase: "Stabilize B1", |
| forward: "B1 denoising", |
| output: "B1 token updates", |
| kv: "Read prefix KV", |
| note: "Trailing buffer slots stay dummy, so the physical shape can remain stable.", |
| timeline: 1, |
| blocks: { |
| b1: ["active", "More stable", 48], |
| b2: ["dummy", "Ready slot", 0], |
| b3: ["dummy", "Dummy slot", 0], |
| b4: ["future", "Future", 0] |
| } |
| } |
| }, |
| { |
| title: "Block 1 becomes near-committable", |
| detail: "The front block is mostly solved. SingleBD still cannot start block 2; MultiBD can activate the next slot before block 1 leaves the buffer.", |
| takeaway: "MultiBD starts exposing inter-block parallelism before the KV store happens.", |
| single: { |
| phase: "Finalize B1", |
| forward: "B1 final denoising", |
| output: "B1 final tokens", |
| kv: "Read prefix KV", |
| note: "Block 2 remains untouched until block 1 is completely committed.", |
| timeline: 2, |
| blocks: { |
| b1: ["ready", "Near complete", 88], |
| b2: ["waiting", "Not admitted", 0], |
| b3: ["future", "Future", 0], |
| b4: ["future", "Future", 0] |
| } |
| }, |
| multi: { |
| phase: "Admit B2", |
| forward: "B1 + B2 denoising", |
| output: "B1 final tokens + B2 updates", |
| kv: "Read prefix KV", |
| note: "Block 2 enters the running-set while block 1 is still physically present.", |
| timeline: 2, |
| blocks: { |
| b1: ["ready", "Near complete", 88], |
| b2: ["active", "Denoising", 18], |
| b3: ["dummy", "Ready slot", 0], |
| b4: ["future", "Future", 0] |
| } |
| } |
| }, |
| { |
| title: "Joint decode happens before the store-only point", |
| detail: "SingleBD finishes block 1 alone. MultiBD uses the same step budget to continue block 2 as well.", |
| takeaway: "The later block has already accumulated progress before the front block enters KV.", |
| single: { |
| phase: "Finish B1", |
| forward: "B1 final denoising", |
| output: "B1 final tokens", |
| kv: "Read prefix KV", |
| note: "The next forward will be a pure KV-store forward.", |
| timeline: 2, |
| blocks: { |
| b1: ["ready", "Complete", 100], |
| b2: ["waiting", "Not admitted", 0], |
| b3: ["future", "Future", 0], |
| b4: ["future", "Future", 0] |
| } |
| }, |
| multi: { |
| phase: "Decode B1 + B2", |
| forward: "Joint denoising", |
| output: "B1 + B2 updates", |
| kv: "Read prefix KV", |
| note: "The front block is complete, and the tail block keeps improving.", |
| timeline: 3, |
| blocks: { |
| b1: ["ready", "Complete", 100], |
| b2: ["active", "Denoising", 42], |
| b3: ["dummy", "Ready slot", 0], |
| b4: ["future", "Future", 0] |
| } |
| } |
| }, |
| { |
| title: "SingleBD hits a KV-store-only forward", |
| detail: "Native BD-LM serving still needs a forward to materialize block 1 into KV. That forward produces no new output. MultiBD overlaps the same KV-store work with block 2 decoding.", |
| takeaway: "The store bubble becomes useful work: store B1 KV while decoding B2.", |
| single: { |
| phase: "Store B1 only", |
| forward: "KV-store forward", |
| output: "No output", |
| kv: "Store B1 KV", |
| note: "This is the store bubble: the GPU runs a forward, but visible generation does not advance.", |
| timeline: 3, |
| blocks: { |
| b1: ["bubble", "Store KV", 100], |
| b2: ["waiting", "Still waiting", 0], |
| b3: ["future", "Future", 0], |
| b4: ["future", "Future", 0] |
| } |
| }, |
| multi: { |
| phase: "Store B1 + Decode B2", |
| forward: "Overlap forward", |
| output: "B2 token updates", |
| kv: "Store B1 KV", |
| note: "Block 1 is a pure KV lane, while block 2 keeps decoding in the same step.", |
| timeline: 4, |
| blocks: { |
| b1: ["store", "Store KV", 100], |
| b2: ["active", "Denoising", 62], |
| b3: ["dummy", "Ready slot", 0], |
| b4: ["future", "Future", 0] |
| } |
| } |
| }, |
| { |
| title: "The buffer slides after block 1 commits", |
| detail: "SingleBD only now starts block 2. MultiBD has already spent useful forwards on block 2 and can slide the buffer without changing the physical execution pattern.", |
| takeaway: "After one block, MultiBD is already ahead on the next block.", |
| single: { |
| phase: "Start B2", |
| forward: "B2 denoising", |
| output: "B2 token updates", |
| kv: "Read prefix+B1 KV", |
| note: "Block 2 starts after block 1 has fully left the decode path.", |
| timeline: 4, |
| blocks: { |
| b1: ["prefix", "In KV", 100], |
| b2: ["active", "Denoising", 18], |
| b3: ["waiting", "Not admitted", 0], |
| b4: ["future", "Future", 0] |
| } |
| }, |
| multi: { |
| phase: "Slide buffer", |
| forward: "B2 denoising", |
| output: "B2 token updates", |
| kv: "Read prefix+B1 KV", |
| note: "B1 becomes prefix KV, B2 moves to the front, and a later dummy slot stays available.", |
| timeline: 5, |
| blocks: { |
| b1: ["prefix", "In KV", 100], |
| b2: ["active", "Mostly stable", 72], |
| b3: ["dummy", "Ready slot", 0], |
| b4: ["future", "Future", 0] |
| } |
| } |
| }, |
| { |
| title: "MultiBD admits block 3 while SingleBD is still on block 2", |
| detail: "SingleBD continues B2 alone. MultiBD uses the bounded running-set to decode B2 and B3 together.", |
| takeaway: "The pipeline effect repeats across blocks, not just once.", |
| single: { |
| phase: "Decode B2", |
| forward: "B2 denoising", |
| output: "B2 token updates", |
| kv: "Read prefix+B1 KV", |
| note: "Block 3 cannot start yet.", |
| timeline: 5, |
| blocks: { |
| b1: ["prefix", "In KV", 100], |
| b2: ["active", "More stable", 46], |
| b3: ["waiting", "Not admitted", 0], |
| b4: ["future", "Future", 0] |
| } |
| }, |
| multi: { |
| phase: "Decode B2 + B3", |
| forward: "Joint denoising", |
| output: "B2 + B3 updates", |
| kv: "Read prefix+B1 KV", |
| note: "B3 starts before B2 is committed, preserving a bounded active tail.", |
| timeline: 6, |
| blocks: { |
| b1: ["prefix", "In KV", 100], |
| b2: ["active", "Near complete", 88], |
| b3: ["active", "Denoising", 26], |
| b4: ["dummy", "Ready slot", 0] |
| } |
| } |
| }, |
| { |
| title: "The running-set expands at the tail", |
| detail: "SingleBD is still finalizing B2. MultiBD can keep B2 near the front, continue B3, and prepare B4 as the next tail slot.", |
| takeaway: "The bounded buffer gives MultiBD parallelism without decoding an unbounded suffix.", |
| single: { |
| phase: "Finalize B2", |
| forward: "B2 final denoising", |
| output: "B2 final tokens", |
| kv: "Read prefix+B1 KV", |
| note: "The same store-only bubble is about to appear again.", |
| timeline: 6, |
| blocks: { |
| b1: ["prefix", "In KV", 100], |
| b2: ["ready", "Complete", 100], |
| b3: ["waiting", "Not admitted", 0], |
| b4: ["future", "Future", 0] |
| } |
| }, |
| multi: { |
| phase: "Admit B4", |
| forward: "B2 + B3 + B4 denoising", |
| output: "Tail-block updates", |
| kv: "Read prefix+B1 KV", |
| note: "B4 enters as the new tail while earlier active blocks remain bounded.", |
| timeline: 7, |
| blocks: { |
| b1: ["prefix", "In KV", 100], |
| b2: ["ready", "Complete", 100], |
| b3: ["active", "Denoising", 48], |
| b4: ["active", "Denoising", 16] |
| } |
| } |
| }, |
| { |
| title: "The second store bubble is also overlapped", |
| detail: "SingleBD must store block 2 with no output. MultiBD stores block 2 while block 3 and block 4 continue making progress.", |
| takeaway: "Store-only forwards become decode-store overlap in the steady state.", |
| single: { |
| phase: "Store B2 only", |
| forward: "KV-store forward", |
| output: "No output", |
| kv: "Store B2 KV", |
| note: "Each completed block creates another no-output forward.", |
| timeline: 7, |
| blocks: { |
| b1: ["prefix", "In KV", 100], |
| b2: ["bubble", "Store KV", 100], |
| b3: ["waiting", "Still waiting", 0], |
| b4: ["future", "Future", 0] |
| } |
| }, |
| multi: { |
| phase: "Store B2 + Decode tail", |
| forward: "Overlap forward", |
| output: "B3 + B4 updates", |
| kv: "Store B2 KV", |
| note: "The front completed block commits while the tail remains productive.", |
| timeline: 8, |
| blocks: { |
| b1: ["prefix", "In KV", 100], |
| b2: ["store", "Store KV", 100], |
| b3: ["active", "Mostly stable", 68], |
| b4: ["active", "Denoising", 34] |
| } |
| } |
| }, |
| { |
| title: "Steady-state block pipeline", |
| detail: "SingleBD starts block 3 only after storing block 2. MultiBD already has later blocks in flight, so the request keeps producing useful decode work across store events.", |
| takeaway: "MBD-LMs convert sequential block processing into bounded inter-block parallelism.", |
| single: { |
| phase: "Start B3", |
| forward: "B3 denoising", |
| output: "B3 token updates", |
| kv: "Read prefix+B1+B2 KV", |
| note: "The decode-store alternation repeats block by block.", |
| timeline: 8, |
| blocks: { |
| b1: ["prefix", "In KV", 100], |
| b2: ["prefix", "In KV", 100], |
| b3: ["active", "Denoising", 18], |
| b4: ["waiting", "Not admitted", 0] |
| } |
| }, |
| multi: { |
| phase: "Steady pipeline", |
| forward: "Tail denoising", |
| output: "B3 + B4 updates", |
| kv: "Read committed prefix", |
| note: "The running-set keeps decode work available while completed blocks enter KV.", |
| timeline: 9, |
| blocks: { |
| b1: ["prefix", "In KV", 100], |
| b2: ["prefix", "In KV", 100], |
| b3: ["active", "Near complete", 86], |
| b4: ["active", "Denoising", 58] |
| } |
| } |
| } |
| ]; |
| |
| const stateClass = (state) => `decode-block state-${state}`; |
| let current = 0; |
| |
| const setText = (id, text) => { |
| const node = document.getElementById(id); |
| if (node) node.textContent = text; |
| }; |
| |
| const setBlock = (side, name, block) => { |
| const node = document.getElementById(`${side}-${name}`); |
| if (!node) return; |
| const [state, status, fill] = block; |
| node.className = stateClass(state); |
| node.style.setProperty("--fill", `${fill}%`); |
| const statusNode = node.querySelector(".block-status"); |
| if (statusNode) statusNode.textContent = status; |
| }; |
| |
| const setTimeline = (side, activeIndex) => { |
| document.querySelectorAll(`[data-timeline="${side}"] .timeline-step`).forEach((node, index) => { |
| node.classList.toggle("is-active", index === activeIndex); |
| node.classList.toggle("is-done", index < activeIndex); |
| }); |
| }; |
| |
| const setPanel = (side, data) => { |
| setText(`${side}-phase`, data.phase); |
| setText(`${side}-forward`, data.forward); |
| setText(`${side}-output`, data.output); |
| setText(`${side}-kv`, data.kv); |
| setText(`${side}-note`, data.note); |
| Object.entries(data.blocks).forEach(([name, block]) => setBlock(side, name, block)); |
| setTimeline(side, data.timeline); |
| }; |
| |
| const render = () => { |
| const step = steps[current]; |
| setText("demo-step-label", `Step ${current + 1} of ${steps.length}`); |
| setText("demo-step-title", step.title); |
| setText("demo-step-detail", step.detail); |
| setText("demo-takeaway", `Takeaway: ${step.takeaway}`); |
| setPanel("single", step.single); |
| setPanel("multi", step.multi); |
| |
| const prev = document.getElementById("demo-prev"); |
| const next = document.getElementById("demo-next"); |
| if (prev) prev.disabled = current === 0; |
| if (next) next.textContent = current === steps.length - 1 ? "Replay" : "Next step"; |
| }; |
| |
| document.getElementById("demo-prev")?.addEventListener("click", () => { |
| current = Math.max(0, current - 1); |
| render(); |
| }); |
| |
| document.getElementById("demo-next")?.addEventListener("click", () => { |
| current = current === steps.length - 1 ? 0 : current + 1; |
| render(); |
| }); |
| |
| document.getElementById("demo-reset")?.addEventListener("click", () => { |
| current = 0; |
| render(); |
| }); |
| |
| document.getElementById("copy-bibtex")?.addEventListener("click", async (event) => { |
| const button = event.currentTarget; |
| const text = document.getElementById("bibtex-code")?.textContent ?? ""; |
| try { |
| await navigator.clipboard.writeText(text); |
| button.textContent = "Copied"; |
| } catch { |
| button.textContent = "Unavailable"; |
| } |
| window.setTimeout(() => { |
| button.textContent = "Copy"; |
| }, 1600); |
| }); |
| |
| render(); |
| |
| |
| const tocLinks = document.querySelectorAll(".side-toc a"); |
| const sections = [...tocLinks].map((a) => document.querySelector(a.getAttribute("href"))); |
| let activeId = null; |
| |
| const observer = new IntersectionObserver( |
| (entries) => { |
| let topmost = null; |
| let topmostY = Infinity; |
| entries.forEach((entry) => { |
| if (entry.isIntersecting && entry.boundingClientRect.top < topmostY) { |
| topmostY = entry.boundingClientRect.top; |
| topmost = entry.target; |
| } |
| }); |
| if (!topmost) { |
| |
| let best = null, bestY = -Infinity; |
| entries.forEach((entry) => { |
| if (entry.boundingClientRect.bottom <= entry.rootBounds.top && entry.boundingClientRect.bottom > bestY) { |
| bestY = entry.boundingClientRect.bottom; |
| best = entry.target; |
| } |
| }); |
| topmost = best; |
| } |
| if (topmost) { |
| const id = topmost.getAttribute("id"); |
| if (id !== activeId) { |
| activeId = id; |
| tocLinks.forEach((a) => a.classList.remove("is-visible")); |
| const link = document.querySelector(`.side-toc a[href="#${id}"]`); |
| if (link) { |
| link.classList.add("is-visible"); |
| link.scrollIntoView({ block: "nearest", behavior: "smooth" }); |
| } |
| } |
| } |
| }, |
| { rootMargin: "-10% 0px -60% 0px", threshold: 0 } |
| ); |
| |
| sections.forEach((s) => s && observer.observe(s)); |
| })(); |
| </script> |
| </body> |
| </html> |
|
|
