| --- |
| license: apache-2.0 |
| tags: |
| - inpainting |
| - onnx |
| --- |
| |
| # LaMa – ONNX export for browser inference |
|
|
| This is an ONNX export of [LaMa (Large Mask inpainting)](https://github.com/advimman/lama), the resolution-robust inpainting model by Samsung Research. |
|
|
| Used by **[wipe.photos](https://wipe.photos)** — a fully private, in-browser object remover powered by [onnxruntime-web](https://github.com/microsoft/onnxruntime) with WebGPU acceleration. |
|
|
| ## Example |
|
|
| | Original | Mask | fp32 | fp16 | int8 | |
| |----------|------|------|------|------| |
| |  |  |  |  |  | |
|
|
| ## Files |
|
|
| | File | Size | Description | |
| |------|------|-------------| |
| | `lama.onnx` | 209 MB | Original fp32 export | |
| | `lama_fp16.onnx` | 110 MB | Weight-only fp16 | |
| | `lama_int8.onnx` | 61.5 MB | Weight-only int8 | |
| | `lama_512_fp16.onnx` | 107 MB | Full fp16 compute, FFT as matmuls, fixed 512×512 input — ~5× faster on WebGPU (recommended) | |
| | `lama_512_int8.onnx` | 62 MB | Weight-only int8, FFT as matmuls, fixed 512×512 input — ~1.8× faster on CPU/WASM | |
|
|
| **`lama_512_fp16.onnx` is recommended when you can fix the input to 512×512** (as wipe.photos does): the whole graph computes in fp16, and the FFTs are replaced by fixed-size matmuls so every op runs on the GPU with no CPU fallbacks — 0.43s vs 2.18s per 512×512 inference on WebGPU (M-series Mac), at the same quality as the weight-only variants (zero pixels differing by more than 50/255 from fp32). `lama_512_int8.onnx` is the same graph with the weight-only int8 quantization of `lama_int8.onnx` — identical output, ~1.8× faster (1.51s vs 2.69s on CPU), the right choice for memory-constrained WASM targets like mobile browsers. For arbitrary input sizes, use `lama_fp16.onnx` — half the size of fp32 with a max pixel difference of just 3–11/255. |
|
|
| ## Input |
|
|
| | Name | Shape | Description | |
| |-------|-----------------|-------------| |
| | input | `[1, 4, H, W]` | Channels 0–2: masked image (RGB, 0–1, masked region zeroed out). Channel 3: binary mask (0 or 1). H and W must be multiples of 32 — except the `lama_512_*` variants, which require exactly H = W = 512. | |
|
|
| ## Output |
|
|
| | Name | Shape | Description | |
| |--------|-----------------|-------------| |
| | output | `[1, 3, H, W]` | Inpainted image (RGB, 0–1) | |
|
|
| ## Quantization approach |
|
|
| A naïve full-graph fp16 or INT8 conversion corrupts the model. LaMa's local spatial branch accumulates very large activations over 18 FFCResnetBlocks — BatchNorm `running_var` values reach ~480 000, well above fp16 max (65504) and far outside INT8 range. Truncating those values breaks normalisation. |
|
|
| The fix is **weight-only compression**: only the 222 Conv weight tensors are compressed (they are small, well-behaved values). A dequantization node (`Cast` for fp16, `DequantizeLinear` for int8) is inserted before each Conv so all computation remains in fp32. BN parameters stay untouched as fp32. |
|
|
| For int8 specifically, **per-channel asymmetric UINT8** is used — each output channel gets its own scale derived from its actual `[min, max]` range. Symmetric INT8 was tried first and produced a faint artifact (max diff 185, 70 bad pixels); switching to asymmetric eliminated it entirely (max diff 48, zero pixels above 50). |
|
|
| `lama_512_fp16.onnx` goes further and computes in fp16 end to end. Two changes make this possible: every BatchNorm is re-parameterized to plain scale/shift form (eliminating the huge `running_var` values), and the FFTs are replaced by matmuls with precomputed 64×64 DFT matrices — valid because a fixed 512×512 input fixes the FFT size. The second change is also what makes it fast: ONNX `DFT` has no WebGPU implementation, so the other variants stall on CPU fallbacks and GPU↔CPU copies; matmuls run everywhere. `lama_512_int8.onnx` applies the weight-only int8 scheme to the same matmul-FFT graph — the FFT change alone makes it ~1.8× faster than `lama_int8.onnx` on CPU/WASM, with identical output. |
|
|
