---
language:
  - en
tags:
  - image-generation
  - class-conditional
  - diffusion
  - mixture-of-experts
  - pytorch
---

# PixelMoE: Scaling Pixel-Space Diffusion with LLM-Style Mixture-of-Experts

<div align="center">

[![arXiv](https://img.shields.io/badge/arXiv-2610.18857-b31b1b.svg)](https://arxiv.org/abs/2610.18857)&nbsp;
[![Code](https://img.shields.io/badge/Code-PixelMoE-green)](https://github.com/KlingAIResearch/PixelMoE)&nbsp;
[![Hugging Face](https://img.shields.io/badge/HuggingFace-PixelMoE-yellow)](https://huggingface.co/KlingTeam/PixelMoE)&nbsp;

</div>

![Images generated by PixelMoE](assets/visual.png)

## 📖 Introduction

**PixelMoE** brings standard LLM-style Mixture-of-Experts (MoE) to pixel-space diffusion for class-conditional ImageNet generation at **256×256** and **512×512** resolution. Each image token activates **4 of 32 routed experts**, alongside a shared expert, with bias-based load balancing. Pixel-Space Hierarchical Flow Matching generates images in two stages, first establishing the overall structure at low resolution, then refining details at the target resolution.

PixelMoE achieves FID **1.47** at 256×256 and **1.53** at 512×512 on class-conditional ImageNet-1K, without representation alignment or perceptual losses.

## 📦 Checkpoints

| Model | 256×256 | FID ↓ | 512×512 | FID ↓ |
|---|---|---|---|---|
| PixelMoE-B-E32A4 | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_b_e32a4_256.pth) | 1.88 | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_b_e32a4_512.pth) | 2.05 |
| PixelMoE-L-E32A4 | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_l_e32a4_256.pth) | 1.54 | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_l_e32a4_512.pth) | 1.58 |
| PixelMoE-H-E32A4 | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_h_e32a4_256.pth) | **1.47** | [Download](https://huggingface.co/KlingTeam/PixelMoE/resolve/main/pixelmoe_h_e32a4_512.pth) | **1.53** |

FID scores are the ImageNet-1K results reported in the paper. B, L, and H denote Base, Large, and Huge; E32A4 denotes 32 routed experts with 4 activated per token.

## 🚀 Download Models

Download a checkpoint with the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli):

```bash
pip install -U huggingface_hub
hf download KlingTeam/PixelMoE pixelmoe_h_e32a4_256.pth --local-dir checkpoints
```

To download the entire model repository instead:

```bash
hf download KlingTeam/PixelMoE --local-dir checkpoints
```

## 📌 Note

This repository hosts the pretrained checkpoints of **PixelMoE**. For the official implementation and instructions on installation, training, and evaluation, please refer to our [GitHub repository](https://github.com/KlingAIResearch/PixelMoE).

## 📖 Citation

If you find PixelMoE useful in your research, please cite our paper:

```bibtex
@misc{dong2026pixelmoescalingpixelspacediffusion,
  title={PixelMoE: Scaling Pixel-Space Diffusion with LLM-Style Mixture-of-Experts}, 
  author={Ruixiao Dong and Jianmin Bao and Bozhi Luan and Zijie Meng and Xin Tao and Pengfei Wan and Kun Gai and Li Li and Houqiang Li},
  year={2026},
  eprint={2610.18857},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2610.18857}, 
}
```

## 📬 Contact

For questions, please contact [dongruixiao@kuaishou.com](mailto:dongruixiao@kuaishou.com).