| --- |
| license: mit |
| base_model: ZhengPeng7/BiRefNet_lite |
| pipeline_tag: image-segmentation |
| tags: |
| - background-removal |
| - matting |
| - dichotomous-image-segmentation |
| - onnx |
| - webgpu |
| - onnxruntime-web |
| --- |
| |
| # BiRefNet_lite — WebGPU-ready ONNX (fp16) |
| |
| [BiRefNet](https://github.com/ZhengPeng7/BiRefNet) (lite / Swin-Tiny variant), rebuilt so it |
| actually runs on **onnxruntime-web's WebGPU EP** in the browser. The stock ONNX exports |
| (e.g. `onnx-community/BiRefNet_lite-ONNX`) fail there three ways |
| (see [microsoft/onnxruntime#21968](https://github.com/microsoft/onnxruntime/issues/21968)): |
|
|
| 1. The decoder contains `Concat` nodes with up to **1024 inputs** and `Split` nodes with |
| **32 outputs**, exceeding WebGPU's storage-buffers-per-shader limit. |
| 2. The variadic `Sum` op has no WebGPU kernel; its CPU fallback sits exactly where the |
| unrolled deformable convolutions materialize ~784 MB tensors, which walks the 4 GB |
| wasm heap into `std::bad_alloc`. |
| 3. The fp16 export defeats onnxruntime's in-browser constant folding (no CPU fp16 |
| kernels), leaving hundreds of CPU-only `ConstantOfShape` islands. |
|
|
| This artifact is the **same weights** after offline graph surgery: |
|
|
| - ONNX Runtime BASIC-level offline optimization (constant fold: ~16k → ~4k nodes, |
| eliminates every CPU-only shape op); |
| - wide `Concat`/`Split` fanouts rewritten into cascades capped at 6, so any shader needs |
| at most 7 storage buffers (safe even at the WebGPU minimum limit of 8); |
| - variadic `Sum` rewritten into `Add` chains; |
| - float16 conversion with `keep_io_types` — **graph I/O stays float32**, so you feed and |
| read plain `Float32Array`s. |
|
|
| Verified bit-exact against the source graph at fp32; after fp16, max sigmoid-space |
| deviation is ~2e-3. Measured ~700 ms/frame warm at 1024×1024 on an RTX 4090 |
| (Chrome, onnxruntime-web 1.27, `ort.webgpu.min.mjs` — use the JSEP build, not the |
| `.bundle` native-EP build). |
|
|
| ## I/O |
|
|
| | | name | shape | dtype | |
| |---|---|---|---| |
| | input | `input_image` | `[1, 3, 1024, 1024]` | float32, RGB, ImageNet mean/std, squish-resized | |
| | output | `output_image` | `[1, 1, 1024, 1024]` | float32 **logits** — apply sigmoid | |
|
|
| ## Usage (onnxruntime-web) |
|
|
| ```js |
| const ort = (await import('https://cdn.jsdelivr.net/npm/onnxruntime-web@1.27.0/dist/ort.webgpu.min.mjs')).default; |
| ort.env.wasm.wasmPaths = 'https://cdn.jsdelivr.net/npm/onnxruntime-web@1.27.0/dist/'; |
| const sess = await ort.InferenceSession.create(modelArrayBuffer, { executionProviders: ['webgpu'] }); |
| const out = await sess.run({ input_image: new ort.Tensor('float32', chw, [1, 3, 1024, 1024]) }); |
| // sigmoid(out.output_image.data[i]) = foreground alpha |
| ``` |
|
|
| Rebuilt by [`studio/tools/export-birefnet.py`](https://github.com/stevecastle) from the |
| Lowkey Studio project; the same script converts the other BiRefNet variants |
| (general / HR / matting / dynamic). |
|
|
| Credits: [ZhengPeng7/BiRefNet](https://github.com/ZhengPeng7/BiRefNet) (MIT) for the model; |
| [onnx-community](https://huggingface.co/onnx-community/BiRefNet_lite-ONNX) for the source ONNX export. |
|
|
