README.md

---
library_name: mlx
license: mit
pipeline_tag: image-to-image
base_model: hustvl/Moebius
tags:
  - mlx
  - inpainting
  - image-inpainting
  - object-removal
  - diffusion
---

# mlx-community/Moebius-Places2-fp16

MLX conversion of [hustvl/Moebius](https://huggingface.co/hustvl/Moebius) (the **Places2**
fine-tune) — a 0.22B-parameter latent-diffusion **image inpainting / object removal** model
(["Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance"](https://arxiv.org/abs/2606.19195)),
distilled from PixelHacker. Reference code: [hustvl/Moebius](https://github.com/hustvl/Moebius)
(Apache-2.0).

The repo is consolidated: `unet.safetensors` (the LambdaNetworks-based denoiser, fp16, 453 MB)
plus `vae.safetensors` (the SDXL-family KL-f8 `AutoencoderKL` from
[hustvl/PixelHacker](https://huggingface.co/hustvl/PixelHacker), `scaling_factor 0.13025`, fp16,
167 MB). The VAE conversion is **lossless** — upstream ships it fp16 (round-trip worst |Δ| = 0).

## Conversion notes

- Tensors are stored in **PyTorch's own layouts** (OIHW conv weights, original key names, no
  remapping); consumers perform their own layout transposes at load.
- The 124 `num_batches_tracked` BatchNorm counters are dropped (int64, never read at inference).
- BatchNorm `running_mean` / `running_var` stay **fp32** inside the fp16 file — a small variance
  that rounds toward zero at fp16 would make `rsqrt` explode.
- **fp16 was chosen by measurement, not default**: per-forward error vs the fp32 reference is
  rel 9.3e-04 (at the GPU fp32 noise floor), end-to-end trajectory cosine 0.99994, output visually
  identical. **bf16 was rejected** (~9× worse per forward — the 8-bit mantissa punishes these
  well-scaled conv weights).

## Usage notes (things that will silently produce wrong output)

- **512×512 only.** `rel_pos_emb` is spatially baked at construction (64×64 latent) and the
  attention wrapper assumes a square grid — other resolutions are structurally invalid, not merely
  off-distribution. Resize in, paste the fill back at the original resolution through a blurred
  mask.
- **Mask convention: white = remove** (luma ≥ 0.5), binarized.
- Denoiser input is **9 channels: noisy(4) + mask(1) + masked-image(4)** — the mask is the
  *middle* channel group.
- Conditioning is a learned 20×3072 category table (no text encoder): rows 0–9 = conditional,
  rows 10–19 = unconditional, CFG batch-doubled `[uncond, cond]` through one forward
  (reference CFG scale 2.5).
- Sampler: **DDIM**, `beta_start 0.00085 / beta_end 0.012 / scaled_linear / 1000` train steps,
  `clip_sample false`, `eta 0`. Reference runs `num_steps 20` at `strength 0.99` ⇒ **19 real
  steps starting at t=900**, initialized from the *clean* image latents noised at `timesteps[0]`
  (not pure noise), with `noise_offset 0.0357` per-channel offset noise.

## Swift / MLX runtime

This checkpoint is consumed by [`xocialize/mlx-moebius-swift`](https://github.com/xocialize/mlx-moebius-swift)
(MLX-Swift port, parity-locked component-by-component against the PyTorch reference: full-UNet
forward rel 1.1e-06 fp32 CPU vs oracle; full 19-step pipeline matches the reference's decoded
image at the VAE's measured fp32 floor; ~4 s per 512² inpaint on an M-series GPU in release).

## Licenses

- Weights: **MIT** (per the upstream [hustvl/Moebius](https://huggingface.co/hustvl/Moebius) model
  card). VAE: MIT (hustvl/PixelHacker repo; SDXL KL-f8 family).
- Reference code: Apache-2.0 (github.com/hustvl/Moebius).
- Fine-tune training data is Places2; see upstream for dataset terms.