bhanc commited on
Commit
63ac809
·
verified ·
1 Parent(s): 6432eb1

Align model card with the established ExecuTorch model-card format

Browse files
Files changed (1) hide show
  1. README.md +45 -85
README.md CHANGED
@@ -1,102 +1,62 @@
1
  ---
2
  license: apache-2.0
3
  pipeline_tag: image-segmentation
4
- tags:
5
- - executorch
6
- - react-native
7
- - background-removal
8
- - salient-object-detection
9
  ---
10
 
11
- # U²-Netp for React Native ExecuTorch
12
 
13
- The small U²-Net salient object model, adapted for the existing
14
- `createSemanticSegmenter` / `useSemanticSegmenter` pipeline in
15
- [React Native ExecuTorch](https://github.com/software-mansion/react-native-executorch).
16
- The source model and pretrained checkpoint are from
17
- [xuebinqin/U-2-Net](https://github.com/xuebinqin/U-2-Net), under Apache-2.0.
18
 
19
- This model predicts salient foreground objects, including products and animals,
20
- as well as people. It produces one soft foreground mask; it does not classify
21
- object categories or separate individual instances. Fine structures and
22
- low-contrast boundaries can be missed.
23
 
24
- ## Files
 
25
 
26
- | Backend | File | Size | Compute precision |
27
- | --- | --- | --- | --- |
28
- | XNNPACK | `xnnpack/u2netp_xnnpack_fp32.pte` | 4.6 MB | FP32 |
29
- | CoreML | `coreml/u2netp_coreml_fp16.pte` | 3.3 MB | FP16 |
30
- | CoreML | `coreml/u2netp_coreml_fp32.pte` | 5.5 MB | FP32 |
31
 
32
- Root `config.json` declares `modelName: u2netp`. Each backend has its own
33
- `config.json` with the exported tensor signatures. `sha256.json` records the
34
- artifact checksums. Models were exported with torch 2.14.0, ExecuTorch 1.4.1 and
35
- coremltools 9.0. CoreML targets iOS 18 / macOS 15 or newer.
 
36
 
37
- ## Tensor contract and processing
 
 
 
 
 
38
 
39
- `forward(float32[1,3,320,320]) -> float32[1,1,320,320]`
40
 
41
- Input is RGB, stretched to 320×320, in NCHW layout with values in `[0,1]`.
42
- The exported graph divides by the image maximum (clamped to `1e-6` for an
43
- all-black input), then applies ImageNet mean/std normalization. Do not apply
44
- ImageNet normalization a second time on the client.
45
-
46
- Output is the fused foreground **logit** map. The seven upstream probability
47
- outputs are replaced by the fused logit before sigmoid, retaining the pretrained
48
- parameters. The client applies sigmoid once and resizes the mask to the source
49
- image dimensions. The upstream demo's per-image output min/max contrast
50
- stretch is omitted, so intensities remain foreground probabilities.
51
-
52
- In RNE, `models.semanticSegmentation.U2NETP` configures RGB scaling by `1/255`,
53
- linear input/output interpolation, and labels `background` / `foreground`.
54
- The semantic segmenter returns an RGBA grayscale mask: RGB contains foreground
55
- intensity from 0 to 255, alpha is opaque, and `colormap` is undefined. Use an RGB
56
- channel as the alpha mask when compositing the original image; the returned
57
- buffer itself is not an image cutout.
58
-
59
- ## Validation and performance
60
-
61
- Host checks used an Apple M1 Pro running macOS 26.5.2. These are warm host
62
- measurements, not phone benchmarks, and exclude image preprocessing and mask
63
- rendering in React Native.
64
-
65
- XNNPACK runs through the ExecuTorch runtime. Its probability error against
66
- upstream eager PyTorch is below 0.00015 on the tested bicycle, boat and animal
67
- images. The graph contains one XNNPACK delegate plus a portable `amax` input
68
- reduction. Per-op timing was unavailable because the installed runtime wheel
69
- does not enable event tracing.
70
-
71
- The actual CoreML partitions were extracted from the saved `.pte` files and
72
- executed directly through CoreML, since the installed ExecuTorch wheel cannot
73
- execute its multifunction CoreML packages. Both CoreML exports contain one
74
- delegate with no portable operations. The profiler's compute-plan tool places
75
- 369 of 370 dispatched FP16 operations on the Neural Engine; one cast runs on
76
- CPU.
77
-
78
- On the three test images, FP16 foreground-probability MAE is 0.00089–0.00453
79
- and thresholded-mask IoU against FP32 is 0.9873–0.9991. Individual edge pixels
80
- can differ by up to 0.312, so FP16 is a speed/quality tradeoff. CoreML FP32
81
- matches the eager thresholded masks on all three images, with maximum
82
- probability error below 0.00017. Warm direct-CoreML medians are approximately
83
- 7.2 ms (FP16) and 16.7 ms (FP32).
84
-
85
- Black/white inputs were also checked for finite results. This is a conversion
86
- smoke test, not a dataset accuracy evaluation. Full CoreML execution through
87
- RNE and both backends' performance require validation on the target devices.
88
 
89
- ## Reproduce the exports
90
 
91
- From the export-scripts repository:
 
 
 
92
 
93
- ```sh
94
- uv run --project exporter_models/u2netp exporter export u2netp --backend xnnpack --output-dir exports/u2netp/xnnpack
95
- uv run --project exporter_models/u2netp exporter export u2netp --backend coreml --variant fp16 --output-dir exports/u2netp/coreml
96
- uv run --project exporter_models/u2netp exporter export u2netp --backend coreml --variant fp32 --output-dir exports/u2netp/coreml
97
- ```
98
 
99
- The model code is vendored from upstream commit
100
- `ac7e1c817ecab7c7dff5ce6b1abba61cd213ff29`. The checkpoint is downloaded from the
101
- author's linked Google Drive artifact and checked against SHA-256
102
- `e7567cde013fb64813973ce6e1ecc25a80c05c3ca7adbc5a54f3c3d90991b854`.
 
1
  ---
2
  license: apache-2.0
3
  pipeline_tag: image-segmentation
4
+ library_name: executorch
 
 
 
 
5
  ---
6
 
7
+ # u2netp
8
 
9
+ This repository hosts the **U²-Netp** models exported for the
10
+ [React Native ExecuTorch](https://www.npmjs.com/package/react-native-executorch)
11
+ library as ExecuTorch `.pte` programs, ready to run on device.
 
 
12
 
13
+ U²-Netp predicts a soft mask for salient foreground objects, including animals
14
+ and products as well as people. It does not classify object categories or
15
+ separate individual instances.
 
16
 
17
+ Upstream model: [U²-Net](https://github.com/xuebinqin/U-2-Net),
18
+ revision `ac7e1c817ecab7c7dff5ce6b1abba61cd213ff29`.
19
 
20
+ ## Variants
 
 
 
 
21
 
22
+ | Path | Backend | Precision |
23
+ | --------------------------------- | ------- | --------- |
24
+ | `coreml/u2netp_coreml_fp16.pte` | coreml | fp16 |
25
+ | `coreml/u2netp_coreml_fp32.pte` | coreml | fp32 |
26
+ | `xnnpack/u2netp_xnnpack_fp32.pte` | xnnpack | fp32 |
27
 
28
+ The exports take RGB float32 `[1,3,320,320]` input in `[0,1]` and return one
29
+ float32 `[1,1,320,320]` foreground logit map. Image-maximum scaling and ImageNet
30
+ normalization are included in the graph. RNE applies sigmoid and returns an
31
+ opaque RGBA grayscale mask; use its foreground intensity as an alpha mask when
32
+ compositing the original image. Fine structures can be missed. FP16 favors
33
+ speed and may shift mask boundaries; FP32 preserves the reference more closely.
34
 
35
+ ## Repository structure
36
 
37
+ ```
38
+ config.json 28 B
39
+ coreml/config.json 1.5 kB
40
+ coreml/u2netp_coreml_fp16.pte 3.3 MB
41
+ coreml/u2netp_coreml_fp32.pte 5.5 MB
42
+ xnnpack/config.json 888 B
43
+ xnnpack/u2netp_xnnpack_fp32.pte 4.6 MB
44
+ LICENSE 11.4 kB
45
+ sha256.json 314 B
46
+ ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
 
48
+ ## Compatibility
49
 
50
+ These files are published for the **ExecuTorch v1.4.1** runtime. ExecuTorch
51
+ gives no forward compatibility guarantee, so an older runtime may fail to load
52
+ them. CoreML requires iOS 18 / macOS 15 or newer; full RNE execution and
53
+ performance still require validation on the target device.
54
 
55
+ To use them in React Native ExecuTorch, pass
56
+ `models.semanticSegmentation.U2NETP.DEFAULT` to `useSemanticSegmenter` or
57
+ `createSemanticSegmenter`. See the
58
+ [documentation](https://docs.swmansion.com/react-native-executorch/docs/fundamentals/downloading-models).
 
59
 
60
+ To load these files in your own ExecuTorch runtime, read the
61
+ [compatibility note](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md)
62
+ first.