| --- |
| license: apache-2.0 |
| library_name: executorch |
| pipeline_tag: image-segmentation |
| inference: false |
| tags: |
| - executorch |
| - webgpu |
| - on-device |
| - browser |
| --- |
| # MODNet portrait matting for ExecuTorch WebGPU |
|
|
| This repository contains an [ExecuTorch](https://github.com/pytorch/executorch) conversion of |
| https://github.com/ZHKKKe/MODNet for the ExecuTorch **WebGPU** backend, so the model runs entirely in a web browser on |
| the visitor's GPU. It powers the corresponding demo in the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)). |
|
|
| - **Base model:** https://github.com/ZHKKKe/MODNet |
| - **Format:** ExecuTorch `.pte` programs delegated to the WebGPU backend |
| - **Quantization:** fp32 (as exported) |
| - **Total size:** 24.9 MB |
|
|
| ## About the original model |
|
|
| A real-time portrait matting network that separates people from the background without a trimap. |
|
|
| - **Developed by:** Zhanghan Ke et al. |
| - **Paper:** [MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition](https://arxiv.org/abs/2011.11961) |
| - 8.7M parameters in the released checkpoint |
| - Photographic portrait matting checkpoint |
|
|
| For training data, evaluation results and full details, see the original model card: https://github.com/ZHKKKe/MODNet. |
|
|
| ## Intended use and limitations |
|
|
| Cutting people out of photos or camera frames, for example to blur or replace the background. |
|
|
| Trained for portraits: results degrade on non-human subjects, several overlapping people, or unusual poses and lighting. |
|
|
| ## What changed in this conversion |
|
|
| - Fixed 256x256 input; the matte is upsampled to the frame size outside the program |
|
|
| ## Inputs and outputs |
|
|
| - Input `[1, 3, 256, 256]` float32 RGB normalized to [-1, 1]: `(x / 255 - 0.5) / 0.5` |
| - Output `[1, 1, 256, 256]` float32 alpha matte in [0, 1] |
|
|
| ## Files |
|
|
| | File | Size | SHA-256 | |
| |---|---|---| |
| | `modnet_webgpu.pte` | 24.9 MB | `0c2ff92e26087e98…` | |
|
|
| Full checksums are in `manifest.json`. The playground verifies them after download. |
|
|
| ## Running it |
|
|
| These programs were exported with the ExecuTorch WebGPU backend |
| ([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu)) and validated with the |
| WebAssembly runtime build used by the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)). That runtime build is not published yet; until it is, |
| the playground is the supported way to run them. It downloads these files once, caches them in the browser, |
| and runs everything locally. |
|
|
| Requirements: a browser with WebGPU enabled. |
|
|
| ## Provenance |
|
|
| - **Weights:** `boopathiraj/MODNet` @ `a967175f8fd63de67a1c161d8daac966d2b57e3a`, file `modnet/pretrained/modnet_photographic_portrait_matting.ckpt`, sha256 `7c22235f0925deba15d4d63e53afcb654c47055bbcd98f56e393ab2584007ed8` (from a community mirror of the official release) |
| - **Checked against the official checkpoint:** `modnet_photographic_portrait_matting.ckpt` is byte-identical to the official checkpoint linked from github.com/ZHKKKe/MODNet (sha256 `7c22235f0925deba15d4d63e53afcb654c47055bbcd98f56e393ab2584007ed8`), and the model code used for the export is identical to the official `src/models/` (one import path differs), checked by the project owner on 2026-10-16 |
| - **Exported with:** executorch 1.5.1+cpu, torch 2.14.1+cpu, torchao 0.18.0+cpu, transformers 5.19.0 (executorch git `3b6068392324`) |
| - **Reproduce:** `python recipes/modnet.py --out out/modnet` (recipe in `export/`) |
| - Re-running the recipe produces byte-identical files. |
|
|
| ## Citation |
|
|
| If you use this model, please cite the original work: |
|
|
| ```bibtex |
| @inproceedings{ke2022modnet, |
| title={MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition}, |
| author={Ke, Zhanghan and Sun, Jiayu and Li, Kaican and Yan, Qiong and Lau, Rynson W.H.}, |
| booktitle={AAAI}, |
| year={2022} |
| } |
| ``` |
|
|
| ## License |
|
|
| This is a derivative of the upstream model and is distributed under the same terms |
| (`apache-2.0`); see `LICENSE`. |
| Modifications (conversion to the ExecuTorch format) by Meta Platforms, Inc. |
|
|
