Hugging Face's logo

mlx-community
/
Moebius-Places2-fp16

Image-to-Image
MLX
inpainting
image-inpainting
object-removal
diffusion
Model card Files
xet
 Community
Moebius-Places2-fp16 / README.md
xocialize's picture
xocialize
Moebius Places2 fp16 (UNet + SDXL KL-f8 VAE) β€” MLX conversion, layout-preserving; fp16 chosen by measured parity
8cef318 verified
preview code
|
Raw
History Blame Contribute Delete
3.6 kB
metadata 
library_name: mlx
license: mit
pipeline_tag: image-to-image
base_model: hustvl/Moebius
tags:
  - mlx
  - inpainting
  - image-inpainting
  - object-removal
  - diffusion

mlx-community/Moebius-Places2-fp16

MLX conversion of hustvl/Moebius (the Places2 fine-tune) β€” a 0.22B-parameter latent-diffusion image inpainting / object removal model ("Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance"), distilled from PixelHacker. Reference code: hustvl/Moebius (Apache-2.0).

The repo is consolidated: unet.safetensors (the LambdaNetworks-based denoiser, fp16, 453 MB) plus vae.safetensors (the SDXL-family KL-f8 AutoencoderKL from hustvl/PixelHacker, scaling_factor 0.13025, fp16, 167 MB). The VAE conversion is lossless β€” upstream ships it fp16 (round-trip worst |Ξ”| = 0).

Conversion notes

  • Tensors are stored in PyTorch's own layouts (OIHW conv weights, original key names, no remapping); consumers perform their own layout transposes at load.
  • The 124 num_batches_tracked BatchNorm counters are dropped (int64, never read at inference).
  • BatchNorm running_mean / running_var stay fp32 inside the fp16 file β€” a small variance that rounds toward zero at fp16 would make rsqrt explode.
  • fp16 was chosen by measurement, not default: per-forward error vs the fp32 reference is rel 9.3e-04 (at the GPU fp32 noise floor), end-to-end trajectory cosine 0.99994, output visually identical. bf16 was rejected (~9Γ— worse per forward β€” the 8-bit mantissa punishes these well-scaled conv weights).

Usage notes (things that will silently produce wrong output)

  • 512Γ—512 only. rel_pos_emb is spatially baked at construction (64Γ—64 latent) and the attention wrapper assumes a square grid β€” other resolutions are structurally invalid, not merely off-distribution. Resize in, paste the fill back at the original resolution through a blurred mask.
  • Mask convention: white = remove (luma β‰₯ 0.5), binarized.
  • Denoiser input is 9 channels: noisy(4) + mask(1) + masked-image(4) β€” the mask is the middle channel group.
  • Conditioning is a learned 20Γ—3072 category table (no text encoder): rows 0–9 = conditional, rows 10–19 = unconditional, CFG batch-doubled [uncond, cond] through one forward (reference CFG scale 2.5).
  • Sampler: DDIM, beta_start 0.00085 / beta_end 0.012 / scaled_linear / 1000 train steps, clip_sample false, eta 0. Reference runs num_steps 20 at strength 0.99 β‡’ 19 real steps starting at t=900, initialized from the clean image latents noised at timesteps[0] (not pure noise), with noise_offset 0.0357 per-channel offset noise.

Swift / MLX runtime

This checkpoint is consumed by xocialize/mlx-moebius-swift (MLX-Swift port, parity-locked component-by-component against the PyTorch reference: full-UNet forward rel 1.1e-06 fp32 CPU vs oracle; full 19-step pipeline matches the reference's decoded image at the VAE's measured fp32 floor; ~4 s per 512Β² inpaint on an M-series GPU in release).

Licenses

  • Weights: MIT (per the upstream hustvl/Moebius model card). VAE: MIT (hustvl/PixelHacker repo; SDXL KL-f8 family).
  • Reference code: Apache-2.0 (github.com/hustvl/Moebius).
  • Fine-tune training data is Places2; see upstream for dataset terms.