File size: 3,044 Bytes
d553b22 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | ---
license: mit
base_model: ZhengPeng7/BiRefNet_lite
pipeline_tag: image-segmentation
tags:
- background-removal
- matting
- dichotomous-image-segmentation
- onnx
- webgpu
- onnxruntime-web
---
# BiRefNet_lite — WebGPU-ready ONNX (fp16)
[BiRefNet](https://github.com/ZhengPeng7/BiRefNet) (lite / Swin-Tiny variant), rebuilt so it
actually runs on **onnxruntime-web's WebGPU EP** in the browser. The stock ONNX exports
(e.g. `onnx-community/BiRefNet_lite-ONNX`) fail there three ways
(see [microsoft/onnxruntime#21968](https://github.com/microsoft/onnxruntime/issues/21968)):
1. The decoder contains `Concat` nodes with up to **1024 inputs** and `Split` nodes with
**32 outputs**, exceeding WebGPU's storage-buffers-per-shader limit.
2. The variadic `Sum` op has no WebGPU kernel; its CPU fallback sits exactly where the
unrolled deformable convolutions materialize ~784 MB tensors, which walks the 4 GB
wasm heap into `std::bad_alloc`.
3. The fp16 export defeats onnxruntime's in-browser constant folding (no CPU fp16
kernels), leaving hundreds of CPU-only `ConstantOfShape` islands.
This artifact is the **same weights** after offline graph surgery:
- ONNX Runtime BASIC-level offline optimization (constant fold: ~16k → ~4k nodes,
eliminates every CPU-only shape op);
- wide `Concat`/`Split` fanouts rewritten into cascades capped at 6, so any shader needs
at most 7 storage buffers (safe even at the WebGPU minimum limit of 8);
- variadic `Sum` rewritten into `Add` chains;
- float16 conversion with `keep_io_types` — **graph I/O stays float32**, so you feed and
read plain `Float32Array`s.
Verified bit-exact against the source graph at fp32; after fp16, max sigmoid-space
deviation is ~2e-3. Measured ~700 ms/frame warm at 1024×1024 on an RTX 4090
(Chrome, onnxruntime-web 1.27, `ort.webgpu.min.mjs` — use the JSEP build, not the
`.bundle` native-EP build).
## I/O
| | name | shape | dtype |
|---|---|---|---|
| input | `input_image` | `[1, 3, 1024, 1024]` | float32, RGB, ImageNet mean/std, squish-resized |
| output | `output_image` | `[1, 1, 1024, 1024]` | float32 **logits** — apply sigmoid |
## Usage (onnxruntime-web)
```js
const ort = (await import('https://cdn.jsdelivr.net/npm/onnxruntime-web@1.27.0/dist/ort.webgpu.min.mjs')).default;
ort.env.wasm.wasmPaths = 'https://cdn.jsdelivr.net/npm/onnxruntime-web@1.27.0/dist/';
const sess = await ort.InferenceSession.create(modelArrayBuffer, { executionProviders: ['webgpu'] });
const out = await sess.run({ input_image: new ort.Tensor('float32', chw, [1, 3, 1024, 1024]) });
// sigmoid(out.output_image.data[i]) = foreground alpha
```
Rebuilt by [`studio/tools/export-birefnet.py`](https://github.com/stevecastle) from the
Lowkey Studio project; the same script converts the other BiRefNet variants
(general / HR / matting / dynamic).
Credits: [ZhengPeng7/BiRefNet](https://github.com/ZhengPeng7/BiRefNet) (MIT) for the model;
[onnx-community](https://huggingface.co/onnx-community/BiRefNet_lite-ONNX) for the source ONNX export.
|