| --- |
| license: mit |
| library_name: transformers.js |
| base_model: onnx-community/BiRefNet_lite-ONNX |
| pipeline_tag: image-segmentation |
| tags: |
| - background-removal |
| - webgpu |
| --- |
| |
| # BiRefNet_lite (fp16), patched to run fully on WebGPU |
| |
| This is [onnx-community/BiRefNet_lite-ONNX](https://huggingface.co/onnx-community/BiRefNet_lite-ONNX) (`onnx/model_fp16.onnx`), an ONNX export of [ZhengPeng7/BiRefNet_lite](https://github.com/ZhengPeng7/BiRefNet). The graph is rewritten so it runs entirely on WebGPU in the browser. The weights are unchanged. |
|
|
| ## Why |
|
|
| On some GPUs the original fails in ONNX Runtime Web's WebGPU backend. It was measured on Intel Iris Xe in Chrome, where three problems showed up: |
|
|
| 1. Concat/Split nodes with up to 1024 inputs or 32 outputs exceed the per-shader storage-buffer limit: "Too many storage buffers in shader. Current: 17, Max is 16". |
| 2. The deformable-convolution index math is int64. WebGPU has no int64, so those nodes run on the CPU. |
| 3. There is no WebGPU `Sum` kernel. Large fp16 tensors were therefore summed on the CPU, which went past the browser's 4 GB memory limit (`std::bad_alloc`). |
|
|
| ## What changed |
|
|
| - Each oversized Concat or Split is now a tree of the same op, each using at most 8 buffers. 8 is WebGPU's guaranteed minimum. |
| - The int64 index chains are int32, with a cast back to int64 right before each `GatherND`. |
| - Each `Sum` is now a chain of `Add`s. |
|
|
| The output is bit-identical to the original: all 1,048,576 output values matched on a test image (onnxruntime-node, CPU). |
|
|
| ## Usage (Transformers.js) |
|
|
| ```js |
| import { AutoModel, AutoProcessor, RawImage } from "@huggingface/transformers"; |
| const model = await AutoModel.from_pretrained("<this repo>", { dtype: "fp16", device: "webgpu" }); |
| const processor = await AutoProcessor.from_pretrained("<this repo>"); |
| const { pixel_values } = await processor(await RawImage.fromURL(url)); |
| const { output_image } = await model({ input_image: pixel_values }); |
| ``` |
|
|
| ## Licence |
|
|
| MIT, the same as the original. See `LICENSE`. The original copyright notice is kept as the licence requires. |
|
|
