moonbox-dev/Moebius-ONNX

Moebius ONNX

ONNX builds of Moebius, made by Moonbox for on-device use with ONNX Runtime (CUDA, DirectML, CPU). Each precision is a folder with one graph per part of the pipeline; large weights sit next to each graph as ONNX external data (*.onnx.data), so keep a folder's files together.

Files

The last column is how far each graph's output is from the original PyTorch model run in fp32 on the same inputs (relative RMS error). fp32 is the yardstick because the original's own bf16 run is not exact either. Image-quality scores are added as each precision passes the image eval.

Precision	File	Size	sha256	Checked against fp32
fp16	fp16/unet/model.onnx	0.91 GB	3ab9796281da	fp32 graph (fp16 compute: 10 % off)
fp16	fp16/vae_decoder/model.onnx	0.10 GB	2818a7cbd433	1.9e-04
fp16	fp16/vae_encoder/model.onnx	0.07 GB	8e37212ea885	6.3e-05
int8	int8/unet/model.onnx	0.27 GB	6bed99be95b9	2.3 % (int8 weights)
int8	int8/vae_decoder/model.onnx	0.10 GB	2818a7cbd433	1.9e-04 (fp16 graph)
int8	int8/vae_encoder/model.onnx	0.07 GB	8e37212ea885	6.3e-05 (fp16 graph)

How to run

Moebius (0.22 B) is an image-inpainting / object-removal model: give it a picture and a mask (white = remove / repaint) and it fills the masked region from the surroundings. No text prompt. These folders hold its three graphs at 512 px (the short side the original pipeline uses): vae_encoder, unet (with classifier-free guidance and the learned prompt embeddings inside) and vae_decoder. The VAE is PixelHacker's (the SDXL VAE, scaling 0.13025); it stays fp32 where fp16 overflows (see the table).

Run it (ONNX Runtime + diffusers' DDIM scheduler; the original's defaults: 20 steps at strength 0.99, CFG 2.5, noise offset 0.0357, result pasted back under a blurred mask):

import numpy as np, onnxruntime as ort, torch
from diffusers import DDIMScheduler
from PIL import Image, ImageFilter

d, prov = "int8", ["CUDAExecutionProvider", "CPUExecutionProvider"]
enc, unet, dec = (ort.InferenceSession(f"{d}/{p}/model.onnx", providers=prov) for p in ("vae_encoder", "unet", "vae_decoder"))
img = Image.open("photo.jpg").convert("RGB").resize((512, 512), Image.LANCZOS)    # graphs are 512 x 512
mask = Image.open("mask.png").convert("L").resize((512, 512), Image.NEAREST)
x = np.asarray(img, np.float32).transpose(2, 0, 1)[None] / 127.5 - 1
m = (np.asarray(mask, np.float32)[None, None] / 255 >= 0.5).astype(np.float32)
lat, mlat = enc.run(None, {"image": x})[0], enc.run(None, {"image": x * (1 - m)})[0]
m64 = m[:, :, ::8, ::8].copy()
s = DDIMScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear", clip_sample=False)
s.set_timesteps(20); ts = s.timesteps[1:]                                          # strength 0.99: 19 steps
g = torch.Generator().manual_seed(0)
noise = torch.randn(lat.shape, generator=g) + 0.0357 * torch.randn((1, 4, 1, 1), generator=g)
z = s.add_noise(torch.from_numpy(lat), noise, ts[:1])
for t in ts:
    eps = unet.run(None, {"latent": z.numpy(), "mask": m64, "masked_latent": mlat,
                          "timestep": np.array([float(t)], np.float32), "guidance": np.array([2.5], np.float32)})[0]
    z = s.step(torch.from_numpy(eps), t, z).prev_sample
y = dec.run(None, {"latent": (z.numpy() / 0.13025).astype(np.float32)})[0][0]
out = Image.fromarray((np.clip((y.transpose(1, 2, 0) + 1) / 2, 0, 1) * 255).astype(np.uint8))
blur = np.asarray(mask.convert("RGB").filter(ImageFilter.GaussianBlur(3)), np.float32) / 255
Image.fromarray((np.asarray(out) * blur + np.asarray(img) * (1 - blur)).astype(np.uint8)).save("filled.png")

Precisions. fp16: every graph in fp16 where it stays within 5 % of fp32 on a test input, otherwise fp32 (the table says which). int8: the UNet's weights int8 per output channel (weight-only), VAE graphs as in fp16. int4: the UNet's MatMul weights 4-bit (MatMulNBits, block 32) and its other weights int8, VAE graphs as in fp16. No bf16 (no bf16 convolution in ONNX Runtime's CUDA provider).

Benchmark. BSD100's 100 photos with seeded free-form masks (10-26 % of the picture), the pipeline above on an RTX 3080 Laptop GPU with ONNX Runtime 1.30: PSNR / SSIM / LPIPS against the original photo and PSNR against the fp32 pipeline's result (same seed). Inpainting has no single right answer, so the ground-truth numbers are for comparing precisions, not a quality score.

Pipeline	Size	vs fp32 (PSNR, same seed)	PSNR / SSIM / LPIPS vs original	Time / picture	Peak VRAM
fp32 (reference, not published)	1.24 GB	-	26.35 / 0.906 / 0.067	3.8 s	7.8 GB
fp16 (UNet fp32)	1.07 GB	46.5 dB (worst 35.6)	26.35 / 0.906 / 0.067	3.6 s	5.9 GB
int8	0.44 GB	40.3 dB (worst 32.5)	26.38 / 0.906 / 0.066	3.9 s	5.9 GB

How these were made

From Moebius at revision cd01f47fb6: each part exported with torch.onnx, then converted or quantized with ONNX Runtime's tools (fp16 / bf16 casts, int8 and int4 weight quantization). Every graph was checked against the original before upload; the sha256 column above is what the files must hash to.

License

MIT (weights; the GitHub code is Apache-2.0). These files are a derivative of the original model and carry its license: it is copied in this repository, and it governs your use of these files. The original model card follows unchanged.

---

Original model card

《Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance》
Project Page: https://hustvl.github.io/Moebius
Paper: https://arxiv.org/abs/2606.19195
Code: https://github.com/hustvl/Moebius
weight: https://huggingface.co/hustvl/Moebius/tree/main