File size: 2,429 Bytes
2d6b460 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | ---
license: other
tags:
- onnx
- sapiens2
- cpu
- segmentation
- pose-estimation
- normal-estimation
- depth-estimation
---
# Sapiens2 ONNX
CPU-friendly ONNX exports of Meta's `facebook/sapiens2-*`. 15 task heads across 4 tasks and 4 sizes.
## Folder layout
Each task has its own folder. Each model is split into a small `.onnx` graph file plus a `.onnx.data` external sidecar (both must live in the same directory at download time).
| Task | 0.4b | 0.8b | 1b | 5b |
|---|---|---|---|---|
| seg | `seg/seg_0.4b_fp16.onnx` (777 MB, fp16) | `seg/seg_0.8b_fp32.onnx` (3.3 GB) | `seg/seg_1b_fp32.onnx` (5.9 GB) | `seg/seg_5b_int8.onnx` (5.2 GB) |
| normal | `normal/normal_0.4b_fp32.onnx` (1.7 GB) | `normal/normal_0.8b_fp32.onnx` (3.5 GB) | `normal/normal_1b_fp32.onnx` (6.2 GB) | `normal/normal_5b_int8.onnx` (6.1 GB) |
| pointmap | `pointmap/pointmap_0.4b_fp32.onnx` (2.0 GB) | `pointmap/pointmap_0.8b_fp32.onnx` (3.9 GB) | `pointmap/pointmap_1b_fp32.onnx` (6.5 GB) | `pointmap/pointmap_5b_int8.onnx` (6.2 GB) |
| pose | `pose/pose_0.4b_fp32.onnx` (1.6 GB) | `pose/pose_0.8b_fp32.onnx` (3.4 GB) | `pose/pose_1b_fp32.onnx` (6.1 GB) | not shipped |
Cosine similarity vs the PyTorch reference is 0.999 or better on every shipped file.
## Precision notes
* seg-0.4b is fp16 (50 percent smaller than fp32 and verified cos 0.99999)
* 0.4b/0.8b/1b for normal, pointmap, pose are fp32. Naive fp16 cast produces NaN (normal L2-normalize divides near zero) or cos around 0.7 (pointmap metric scale, pose sigmoid heatmaps saturate)
* 5B variants are INT8 (per-channel symmetric, MatMulIntegerToFloat)
* pose-5b is not shipped (the int8 quantize attempt did not complete on the available hardware)
## Inference
```python
import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
# Download both the .onnx graph and the .onnx.data sidecar side by side
for fn in ("seg/seg_0.4b_fp16.onnx", "seg/seg_0.4b_fp16.onnx.data"):
hf_hub_download(repo_id="WeReCooking/sapiens2-onnx", filename=fn, local_dir=".")
sess = ort.InferenceSession("seg/seg_0.4b_fp16.onnx", providers=["CPUExecutionProvider"])
# Input expects (N, 3, 1024, 768) fp32 BGR mean-subtracted preprocessed tensor
out = sess.run(None, {"input": preprocessed})
```
For a standalone CLI without sapiens2 or PyTorch, see `app.py onnx ...` in the source Space `WeReCooking/sapiens2-cpu`.
## License
Same as upstream `facebook/sapiens2-*`.
|