psiddh commited on
Commit
fd4bfe7
·
verified ·
1 Parent(s): 9a2525b

Card: weights and model code verified against the official MODNet release

Browse files
Files changed (2) hide show
  1. NOTICE +1 -1
  2. README.md +50 -7
NOTICE CHANGED
@@ -1,6 +1,6 @@
1
  This repository contains a modified version of https://github.com/ZHKKKe/MODNet.
2
 
3
  Modifications by Meta Platforms, Inc.: conversion to the ExecuTorch program format (.pte) for the
4
- ExecuTorch WebGPU backend, and quantization as described in README.md.
5
 
6
  The original work and these modifications are distributed under the license in LICENSE.
 
1
  This repository contains a modified version of https://github.com/ZHKKKe/MODNet.
2
 
3
  Modifications by Meta Platforms, Inc.: conversion to the ExecuTorch program format (.pte) for the
4
+ ExecuTorch WebGPU backend.
5
 
6
  The original work and these modifications are distributed under the license in LICENSE.
README.md CHANGED
@@ -2,6 +2,7 @@
2
  license: apache-2.0
3
  library_name: executorch
4
  pipeline_tag: image-segmentation
 
5
  tags:
6
  - executorch
7
  - webgpu
@@ -12,14 +13,39 @@ tags:
12
 
13
  This repository contains an [ExecuTorch](https://github.com/pytorch/executorch) conversion of
14
  https://github.com/ZHKKKe/MODNet for the ExecuTorch **WebGPU** backend, so the model runs entirely in a web browser on
15
- the visitor's GPU. It powers the corresponding demo in the
16
- ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)).
17
 
18
  - **Base model:** https://github.com/ZHKKKe/MODNet
19
  - **Format:** ExecuTorch `.pte` programs delegated to the WebGPU backend
20
  - **Quantization:** fp32 (as exported)
21
  - **Total size:** 24.9 MB
22
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
  ## Files
24
 
25
  | File | Size | SHA-256 |
@@ -30,20 +56,37 @@ Full checksums are in `manifest.json`. The playground verifies them after downlo
30
 
31
  ## Running it
32
 
33
- These programs need the ExecuTorch WebGPU runtime compiled to WebAssembly
34
- ([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu)). The easiest way to
35
- try the model is the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)),
36
- which downloads these files once, caches them in the browser, and runs everything locally.
 
 
 
37
 
38
  ## Provenance
39
 
40
  - **Weights:** `boopathiraj/MODNet` @ `a967175f8fd63de67a1c161d8daac966d2b57e3a`, file `modnet/pretrained/modnet_photographic_portrait_matting.ckpt`, sha256 `7c22235f0925deba15d4d63e53afcb654c47055bbcd98f56e393ab2584007ed8` (from a community mirror of the official release)
 
41
  - **Exported with:** executorch 1.5.1+cpu, torch 2.14.1+cpu, torchao 0.18.0+cpu, transformers 5.19.0 (executorch git `3b6068392324`)
42
  - **Reproduce:** `python recipes/modnet.py --out out/modnet` (recipe in `export/`)
43
  - Re-running the recipe produces byte-identical files.
44
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
  ## License
46
 
47
  This is a derivative of the upstream model and is distributed under the same terms
48
  (`apache-2.0`); see `LICENSE`.
49
- Modifications (conversion to the ExecuTorch format and quantization) by Meta Platforms, Inc.
 
2
  license: apache-2.0
3
  library_name: executorch
4
  pipeline_tag: image-segmentation
5
+ inference: false
6
  tags:
7
  - executorch
8
  - webgpu
 
13
 
14
  This repository contains an [ExecuTorch](https://github.com/pytorch/executorch) conversion of
15
  https://github.com/ZHKKKe/MODNet for the ExecuTorch **WebGPU** backend, so the model runs entirely in a web browser on
16
+ the visitor's GPU. It powers the corresponding demo in the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)).
 
17
 
18
  - **Base model:** https://github.com/ZHKKKe/MODNet
19
  - **Format:** ExecuTorch `.pte` programs delegated to the WebGPU backend
20
  - **Quantization:** fp32 (as exported)
21
  - **Total size:** 24.9 MB
22
 
23
+ ## About the original model
24
+
25
+ A real-time portrait matting network that separates people from the background without a trimap.
26
+
27
+ - **Developed by:** Zhanghan Ke et al.
28
+ - **Paper:** [MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition](https://arxiv.org/abs/2011.11961)
29
+ - 8.7M parameters in the released checkpoint
30
+ - Photographic portrait matting checkpoint
31
+
32
+ For training data, evaluation results and full details, see the original model card: https://github.com/ZHKKKe/MODNet.
33
+
34
+ ## Intended use and limitations
35
+
36
+ Cutting people out of photos or camera frames, for example to blur or replace the background.
37
+
38
+ Trained for portraits: results degrade on non-human subjects, several overlapping people, or unusual poses and lighting.
39
+
40
+ ## What changed in this conversion
41
+
42
+ - Fixed 256x256 input; the matte is upsampled to the frame size outside the program
43
+
44
+ ## Inputs and outputs
45
+
46
+ - Input `[1, 3, 256, 256]` float32 RGB normalized to [-1, 1]: `(x / 255 - 0.5) / 0.5`
47
+ - Output `[1, 1, 256, 256]` float32 alpha matte in [0, 1]
48
+
49
  ## Files
50
 
51
  | File | Size | SHA-256 |
 
56
 
57
  ## Running it
58
 
59
+ These programs were exported with the ExecuTorch WebGPU backend
60
+ ([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu)) and validated with the
61
+ WebAssembly runtime build used by the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)). That runtime build is not published yet; until it is,
62
+ the playground is the supported way to run them. It downloads these files once, caches them in the browser,
63
+ and runs everything locally.
64
+
65
+ Requirements: a browser with WebGPU enabled.
66
 
67
  ## Provenance
68
 
69
  - **Weights:** `boopathiraj/MODNet` @ `a967175f8fd63de67a1c161d8daac966d2b57e3a`, file `modnet/pretrained/modnet_photographic_portrait_matting.ckpt`, sha256 `7c22235f0925deba15d4d63e53afcb654c47055bbcd98f56e393ab2584007ed8` (from a community mirror of the official release)
70
+ - **Checked against the official checkpoint:** `modnet_photographic_portrait_matting.ckpt` is byte-identical to the official checkpoint linked from github.com/ZHKKKe/MODNet (sha256 `7c22235f0925deba15d4d63e53afcb654c47055bbcd98f56e393ab2584007ed8`), and the model code used for the export is identical to the official `src/models/` (one import path differs), checked by the project owner on 2026-10-16
71
  - **Exported with:** executorch 1.5.1+cpu, torch 2.14.1+cpu, torchao 0.18.0+cpu, transformers 5.19.0 (executorch git `3b6068392324`)
72
  - **Reproduce:** `python recipes/modnet.py --out out/modnet` (recipe in `export/`)
73
  - Re-running the recipe produces byte-identical files.
74
 
75
+ ## Citation
76
+
77
+ If you use this model, please cite the original work:
78
+
79
+ ```bibtex
80
+ @inproceedings{ke2022modnet,
81
+ title={MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition},
82
+ author={Ke, Zhanghan and Sun, Jiayu and Li, Kaican and Yan, Qiong and Lau, Rynson W.H.},
83
+ booktitle={AAAI},
84
+ year={2022}
85
+ }
86
+ ```
87
+
88
  ## License
89
 
90
  This is a derivative of the upstream model and is distributed under the same terms
91
  (`apache-2.0`); see `LICENSE`.
92
+ Modifications (conversion to the ExecuTorch format) by Meta Platforms, Inc.