Diffusers

Add model card, pipeline tag, and links to paper/code

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +87 -0
README.md CHANGED
@@ -1,3 +1,90 @@
1
  ---
2
  license: mit
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ pipeline_tag: image-to-image
4
  ---
5
+
6
+ # Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance
7
+
8
+ [![Project Page](https://img.shields.io/badge/Project_Page-https://hustvl.github.io/Moebius-purple)](https://hustvl.github.io/Moebius)
9
+ [![arxiv](https://img.shields.io/badge/ECCV'26-paper-orange)](https://arxiv.org/abs/2606.19195)
10
+ [![GitHub](https://img.shields.io/badge/GitHub-Repository-blue)](https://github.com/hustvl/Moebius)
11
+
12
+ **Moebius** is a highly efficient, lightweight image inpainting framework that operates with a mere **0.22B parameters** (less than 2% of the size of the 11.9B FLUX.1-Fill-Dev) while delivering comparable or superior generation quality and a $>15\times$ acceleration in total inference time.
13
+
14
+ <div align="center">
15
+ <img src="https://github.com/hustvl/Moebius/raw/main/assets/pipeline.png" width="100%" alt="Moebius Pipeline">
16
+ </div>
17
+
18
+ ## Key Features
19
+ - **Extreme Parametric Efficiency (< 2%)**: Moebius operates with a mere **0.22B (226M) parameters**, bypassing heavy-compute bottlenecks.
20
+ - **15ร— Inference Speedup**: Blistering inference latency of only **26.01 ms per step** on a single GPU.
21
+ - **10B-Level Inpainting Quality**: Performs on par with, and in certain scenarios surpasses, SOTA generalist models (*FLUX.1-Fill-Dev*, *SD3.5 Large-Inpainting*) across both natural and portrait scenes.
22
+
23
+ ---
24
+
25
+ ## ๐Ÿ“ฆ Environment Setup
26
+
27
+ To get started, clone the official GitHub repository, set up the environment, and install dependencies:
28
+
29
+ ```bash
30
+ git clone https://github.com/hustvl/Moebius.git
31
+ cd Moebius
32
+ conda create -n moebius python=3.14.4
33
+ conda activate moebius
34
+ pip install -r requirements.txt
35
+ ```
36
+
37
+ ## ๐Ÿ—ƒ๏ธ Model Checkpoints Setup
38
+
39
+ Organize your model weights in the `./weight` folder as follows:
40
+
41
+ 1. Download the VAE checkpoint from [hustvl/PixelHacker](https://huggingface.co/hustvl/PixelHacker/tree/main/vae) and put it into `./weight/vae`.
42
+ 2. Download the Moebius checkpoints (e.g., pretrained, ft_places2, ft_celebahq, ft_ffhq) and place them under `./weight/Moebius`.
43
+
44
+ ```bash
45
+ โ”œโ”€โ”€ weight
46
+ | โ”œโ”€โ”€ Moebius
47
+ | โ”œโ”€โ”€ pretrained
48
+ | โ”œโ”€โ”€ diffusion_pytorch_model.bin
49
+ | โ”œโ”€โ”€ ft_places2
50
+ | โ”œโ”€โ”€ diffusion_pytorch_model.bin
51
+ | โ”œโ”€โ”€ ft_celebahq
52
+ | โ”œโ”€โ”€ diffusion_pytorch_model.bin
53
+ | โ”œโ”€โ”€ ft_ffhq
54
+ | โ”œโ”€โ”€ diffusion_pytorch_model.bin
55
+ | โ”œโ”€โ”€ vae
56
+ | โ”œโ”€โ”€ config.json
57
+ | โ”œโ”€โ”€ diffusion_pytorch_model.bin
58
+ ```
59
+
60
+ ## ๐Ÿ”ฎ Inference
61
+
62
+ Run the following command to perform inpainting on custom images and masks. Place your input images and masks with matching filenames under `data/images` and `data/masks` respectively:
63
+
64
+ ```bash
65
+ python -m infer.infer_moebius \
66
+ --model-config config/model_cfg/moebius.yaml \
67
+ --model-weight weight/Moebius/ft_celebahq/diffusion_pytorch_model.bin \
68
+ --real-dir data/images \
69
+ --mask-dir data/masks \
70
+ --save-dir ./outputs \
71
+ --cfg 2.0 \
72
+ --batch-size 8 \
73
+ --num-workers 8
74
+ ```
75
+
76
+ ## ๐ŸŽ“ Citation
77
+
78
+ If you find Moebius useful in your research, please consider citing:
79
+
80
+ ```bibtex
81
+ @misc{DuanAndXu2026Moebius,
82
+ title={Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance},
83
+ author={Kangsheng Duan and Ziyang Xu and Wenyu Liu and Xiaohu Ruan and Xiaoxin Chen and Xinggang Wang},
84
+ year={2026},
85
+ eprint={2606.19195},
86
+ archivePrefix={arXiv},
87
+ primaryClass={cs.CV},
88
+ url={https://arxiv.org/abs/2606.19195},
89
+ }
90
+ ```