File size: 3,371 Bytes
d798759 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 | ---
license: apache-2.0
base_model: facebook/sam2.1-hiera-tiny
tags:
- segment-anything
- onnx
- mobile
- image-segmentation
library_name: onnx
---
# SAM 2.1 hiera-tiny — split ONNX (w8a8), multimask decoder
Repackaged for **on-device, CPU-only** tap-to-segment in
[DreamUI](https://github.com/AbrahamPaulJ/dreamui). Derived from Qualcomm AI Hub's
[Segment-Anything-Model-2](https://huggingface.co/qualcomm/Segment-Anything-Model-2)
w8a8 ONNX release (v0.59.0), which is itself an export of Meta's
[SAM 2.1 hiera-tiny](https://github.com/facebookresearch/sam2).
`sam2-split-w8a8.zip` (86,835,041 bytes) contains six flat files — each `.onnx`
references its `.data` by bare filename, so they must extract to one directory:
| File | Inputs → outputs | Runs |
|---|---|---|
| `trunk.onnx` / `.data` | `image` → embeddings, high-res features, pix_feat | once per image |
| `prompt.onnx` / `.data` | `unnorm_coords`, `labels` → `sparse_embedding` | once per tap (3 KB) |
| `decoder.onnx` / `.data` | embeddings + `sparse_embedding` → `masks`, `scores` | once per tap |
## Two changes from the AI Hub release
**1. The encoder is split into trunk + prompt.** AI Hub fuses the prompt encoder into the
image encoder, so a naive pipeline re-runs a 33.5M-parameter trunk on every click. The
trunk's outputs are bit-identical across clicks — only `sparse_embedding` varies — so
cutting at that seam turns ~817 ms per tap into ~817 ms per *image* plus ~28 ms per tap
(measured, Snapdragon 8 Elite, CPU EP, 4 threads).
**2. The decoder emits all four mask tokens.** The upstream export computes
`masks [1,4,256,256]` and slices `[0:1]`. Token 0 is the *single-mask* head, which blends
the competing interpretations of an ambiguous point prompt and visibly bleeds past object
boundaries; tokens 1–3 are the real granularity candidates. The `Slice` node's `ends` is
widened `1 → 4`. The trailing `QuantizeLinear` is untouched, so masks remain `uint8` on
the same scale/zero-point (0.3612250089645386 / 165) — only the channel count changes.
Measured on a 512×512 fixture, five clicks: token 3 was tighter than token 0 at **every**
click (25–55% smaller area) while the model's own IoU head rated the two within 0.04.
## Verified, not assumed
- Decoder **channel 0 is bit-identical** to the stock AI Hub decoder at all five clicks.
- `trunk + prompt + decoder` is **bit-identical** to `encoder + decoder` at all five clicks.
## Usage notes
- ⚠ **`unnorm_coords` wants NORMALISED coordinates**, in `[0, 1]`, despite the name.
Pixel coordinates return a confidently misplaced mask.
- ⚠ **Label *values* are ignored** by this export — only whether the second label is `-1`
(marking the second point slot unused) matters. There are no negative/background points.
- ⚠ **Blur the logits before thresholding.** Raw thresholding produces heavy
salt-and-pepper stipple along soft boundaries — checkerboard artifacting from the mask
decoder's transposed convolutions, present in the float export too. A 3×3 box blur on
the 256×256 logit field cuts it ~10×.
Quantization parameters for every tensor are in the AI Hub bundle's `metadata.json`.
## Licence
Apache 2.0, inherited from `facebookresearch/sam2`. Redistributed with attribution to
Meta AI (original model) and Qualcomm AI Hub (the ONNX export these files derive from).
|