| --- |
| language: |
| - en |
| tags: |
| - image-generation |
| - class-conditional |
| - diffusion |
| - mixture-of-experts |
| - pytorch |
| --- |
| |
| # PixelMoE: Scaling Pixel-Space Diffusion with LLM-Style Mixture-of-Experts |
|
|
| <div align="center"> |
|
|
| [](https://arxiv.org/abs/2610.18857) |
| [](https://github.com/KlingAIResearch/PixelMoE) |
| [](https://huggingface.co/KlingTeam/PixelMoE) |
|
|
| </div> |
|
|
|  |
|
|
| ## 📖 Introduction |
|
|
| **PixelMoE** brings standard LLM-style Mixture-of-Experts (MoE) to pixel-space diffusion for class-conditional ImageNet generation at **256×256** and **512×512** resolution. Each image token activates **4 of 32 routed experts**, alongside a shared expert, with bias-based load balancing. Pixel-Space Hierarchical Flow Matching generates images in two stages, first establishing the overall structure at low resolution, then refining details at the target resolution. |
|
|
| PixelMoE achieves FID **1.47** at 256×256 and **1.53** at 512×512 on class-conditional ImageNet-1K, without representation alignment or perceptual losses. |
|
|
| ## 📦 Checkpoints |
|
|
| | Model | 256×256 | FID ↓ | 512×512 | FID ↓ | |
| |---|---|---|---|---| |
| | PixelMoE-B-E32A4 | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_b_e32a4_256.pth) | 1.88 | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_b_e32a4_512.pth) | 2.05 | |
| | PixelMoE-L-E32A4 | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_l_e32a4_256.pth) | 1.54 | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_l_e32a4_512.pth) | 1.58 | |
| | PixelMoE-H-E32A4 | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_h_e32a4_256.pth) | **1.47** | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_h_e32a4_512.pth) | **1.53** | |
|
|
| FID scores are the ImageNet-1K results reported in the paper. B, L, and H denote Base, Large, and Huge; E32A4 denotes 32 routed experts with 4 activated per token. |
|
|
| ## 🚀 Download Models |
|
|
| Download a checkpoint with the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli): |
|
|
| ```bash |
| pip install -U huggingface_hub |
| hf download KlingTeam/PixelMoE pixelmoe_h_e32a4_256.pth --local-dir checkpoints |
| ``` |
|
|
| To download the entire model repository instead: |
|
|
| ```bash |
| hf download KlingTeam/PixelMoE --local-dir checkpoints |
| ``` |
|
|
| ## 📌 Note |
|
|
| This repository hosts the pretrained checkpoints of **PixelMoE**. For the official implementation and instructions on installation, training, and evaluation, please refer to our [GitHub repository](https://github.com/KlingAIResearch/PixelMoE). |
|
|
| ## 📖 Citation |
|
|
| If you find PixelMoE useful in your research, please cite our paper: |
|
|
| ```bibtex |
| @misc{dong2026pixelmoescalingpixelspacediffusion, |
| title={PixelMoE: Scaling Pixel-Space Diffusion with LLM-Style Mixture-of-Experts}, |
| author={Ruixiao Dong and Jianmin Bao and Bozhi Luan and Zijie Meng and Xin Tao and Pengfei Wan and Kun Gai and Li Li and Houqiang Li}, |
| year={2026}, |
| eprint={2610.18857}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CV}, |
| url={https://arxiv.org/abs/2610.18857}, |
| } |
| ``` |
|
|
| ## 📬 Contact |
|
|
| For questions, please contact [dongruixiao@kuaishou.com](mailto:dongruixiao@kuaishou.com). |
|
|
