Update README.md

README.md

@@ -1,3 +1,47 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+---
+
+<div align=center>
+  
+# [miXed Diffusion Language Modeling](https://arxiv.org/pdf/2602.01362)
+</div>
+
+
+This repository contains the checkpoints for [CODE](https://github.com/MzeroMiko/XDLM).
+
+
+<div align=center>
+<img src="https://cdn-uploads.huggingface.co/production/uploads/65aa76b1cb5b4fb08ecb087c/cLJKItZwLzXPac2Yo8Xw0.png" width="80%">
+</div>
+<div align=center>
+</div>
+
+## Introduction
+
+In discrete generative modeling, two dominant paradigms demonstrate divergent capabilities: Masked Diffusion Language Models (MDLM) excel at semantic understanding and zero-shot generalization, whereas Uniform-noise Diffusion Language Models (UDLM) achieve strong few-step generation quality, yet neither attains balanced performance across both dimensions. To address this, we propose XDLM, which bridges the two paradigms via a stationary noise kernel. XDLM offers two key contributions: (1) it provides a principled theoretical unification of MDLM and UDLM, recovering each paradigm as a special case; and (2) an alleviated memory bottleneck enabled by an algebraic simplification of the posterior probabilities. Experiments demonstrate that XDLM advances the Pareto frontier between understanding capability and generation quality. Quantitatively, XDLM surpasses UDLM by 5.4 points on zero-shot text benchmarks and outperforms MDLM in few-step image generation (FID 54.1 vs. 80.8). When scaled to tune an 8B-parameter large language model, XDLM achieves 15.0 MBPP in just 32 steps, effectively doubling the baseline performance. Finally, analysis of training dynamics reveals XDLM’s superior potential for long-term scaling.
+
+
+## Highlights
+
+### LM1B Case
+
+***Step-wise evolution of a generated sequence (T = 32).***
+XDLM shows three different transition dynamics inherent to the hybrid noise process: `Green` represents new tokens generated from masks; `Blue` represents lexical refinement; and `Red` highlights the re-masking
+operation where previously generated tokens are rejected and reverted to `[MASK]`.
+
+<div align=center>
+<img src="https://cdn-uploads.huggingface.co/production/uploads/65aa76b1cb5b4fb08ecb087c/jb0CFdezPWfIsk7X7lcIJ.png" width="80%">
+</div>
+
+### LLaDA Continue Pretraining
+
+[Huggingface Link](https://huggingface.co/Mzero17/LLaDA-XDLM)
+
+***LLaDA-XDLM with sampling budget of 32.***
+Evaluation of adapting LLaDA-8B to our XDLM formulation (LLaDA-XDLM): (a) LLaDA-XDLM consistently out-performs baselines across diverse benchmarks with 32 sampling steps; (b) Improvements are particularly pronounced in code generation (MBPP), where the
+model substantially reduces generation failures.
+
+<div align=center>
+<img src="https://cdn-uploads.huggingface.co/production/uploads/65aa76b1cb5b4fb08ecb087c/oPbIv32EgvA1BbCqd2r6E.png" width="80%">
+</div>