sam3edge-micro

SAM 3's text-prompted detector cut down for 4 GB boards. It runs with no PyTorch: numpy, Pillow and ONNX Runtime, three small Python files and a bundle of two graphs (three with masks). Give it a picture and a concept from its list ("bus", "person", "forklift") and it returns a box and a score for every instance, and a mask if you ask.

Try it in the browser: the demo Space.

Runs on a Rubik Pi 3 (Qualcomm QCS6490) and other ARM boards, x86 PCs, NVIDIA GPUs, Qualcomm Hexagon NPUs
Needs 0.43 GB of memory and 220 MB of files for the smallest bundle
Speed 0.5 s (NPU) to 0.95 s (CPU) for a first concept on a Rubik Pi 3 with the smallest bundle; 0.27 s on a laptop CPU
Accuracy F1 0.29 to 0.42 on a 60-picture COCO sample, against 0.73 for a full-size 8-bit SAM 3
Size 50M parameters (fast bundles) or 482M (quality bundle); SAM 3 has 860M

It is restricted on purpose:

  • A fixed concept list. The text encoder is left off the board. These bundles hold 104 concepts: the 80 COCO categories and some street and site terms (micro.json lists them).
  • One picture at a time, no video tracking.

Boxes are the default. Masks are there when you ask (--masks, masks=True): the mask head is a third graph that is loaded only then, so a boxes-only board pays nothing for it.

For any phrase, and masks, use sam3edge-mini. Nothing here was trained by this project: the weights are SAM 3's and its distilled encoders from EfficientSAM3, converted and packaged with a runtime.

Model For Prompts Returns
sam3edge-mini 8 GB boards, PCs, GPUs any phrase boxes and masks
sam3edge-micro (this one) 4 GB boards a fixed list of 104 concepts boxes, or masks
sam3edge-nano microcontroller-class boards planned, not released

Quick start

pip install onnxruntime numpy pillow huggingface_hub
hf download eoinedge/sam3edge-micro --include "sam3edge_micro/*" --include "bundles/fast-672/*" --local-dir sam3edge-micro
cd sam3edge-micro
python -m sam3edge_micro detect bundles/fast-672 frame.jpg --concept bus --concept person            # boxes
python -m sam3edge_micro detect bundles/fast-672 frame.jpg --concept bus --concept person --masks    # and masks
from PIL import Image
from sam3edge_micro import Micro

micro = Micro("bundles/fast-672")
for det in micro.detect(Image.open("frame.jpg"), ["bus", "person"]):                 # masks=True adds det["mask"]
    print(det["concept"], det["score"], det["box_xyxy"])

python -m sam3edge_micro info bundles/fast-672 prints the concept list and the memory the bundle needs, and bench times it on your machine. The runtime refuses to load when the board has less free memory than the bundle needs (--force overrides): on a board with no swap, running short stalls everything on it. On a board with a Hexagon NPU, pip install onnxruntime-qnn and add --provider npu; on an NVIDIA GPU, install onnxruntime-gpu[cuda,cudnn] and add --provider cuda.

Which bundle

Bundle Image encoder Files Memory Best for
bundles/fast-672 distilled EfficientViT-B1, 672 px, 24 x 24 tokens 220 MB 0.43 GB the smallest boards; large, clear objects
bundles/fast-1008 the same encoder, 1008 px, 36 x 36 tokens 221 MB 0.59 GB more detail, about twice the time
bundles/quality-336 SAM 3's own image encoder, 8-bit weights, 336 px 600 MB 0.83 GB low-resolution feeds and judgement over speed

The two fast bundles also carry a fixed-point image encoder for boards with a Hexagon NPU (image-npu.onnx). In fast-672 that graph feeds the detector only, so a masks call there runs the image encoder on the CPU.

Samples

Each picture is the bundle's own output on a laptop CPU. Nothing is hand-edited.

Fast bundle on a clear photo. People and shoes, boxes.

sam3edge-micro fast bundle at 672 px finding people and shoes

Quality bundle on a COCO street picture, with masks. The bus at 0.95 and three cars at 0.86 to 0.92, nothing else.

sam3edge-micro quality bundle at 336 px on a COCO street picture, with masks: one bus, three cars

Fast bundle on the same picture. This is its weak spot. It finds the bus and two of the cars at low scores, and it also answers "person" four times, once on the bus.

sam3edge-micro fast bundle at 672 px on the same picture: the bus, two cars and four wrong "person" answers

Speed

sam3edge-micro: seconds to answer the first concept, by bundle and processor

Rubik Pi 3 (Qualcomm QCS6490), the board's vision services paused, measured with this repository's runtime. "Four small cores" is the board's four Cortex-A55 cores under a 900 MB memory cap.

Bundle Where Image pass Detector pass First concept
fast-672 four Cortex-A78 cores 0.50 s 0.45 s 0.95 s
fast-672 image encoder on the Hexagon NPU 0.085 s 0.45 s 0.53 s
fast-672 four small cores 1.26 s 1.23 s 2.49 s
fast-1008 four Cortex-A78 cores 1.22 s 1.10 s 2.32 s
fast-1008 image encoder on the Hexagon NPU 0.22 s 1.14 s 1.36 s
fast-1008 four small cores 2.76 s 2.58 s 5.34 s
quality-336 four Cortex-A78 cores 4.16 s 0.51 s 4.67 s
quality-336 four small cores 9.72 s 1.21 s 10.9 s

Each further concept on the same picture costs the detector pass only. Masks add about 0.7 s a concept on the big cores (0.70 s with fast-672, 0.76 s with quality-336) and 40 MB of memory. The small-core rows and the fast-672 NPU row were measured earlier the same day, before the mask head was added; the boxes path has not changed since.

The small-core rows are a stand-in for an Arduino UNO Q, not a measurement of one. The UNO Q's Cortex-A53 cores are slower than Cortex-A55 and lack the dot-product instructions the quality bundle's 8-bit encoder uses, so expect it to be slower there than these rows, the quality bundle most of all. It has not been run on a UNO Q.

Accuracy

A seeded sample of 60 COCO val2017 pictures: every category COCO labels in a picture is asked for, plus two it does not hold; a detection counts when it overlaps an unused ground-truth box at IoU 0.5. A relative check between settings, not a benchmark.

sam3edge-micro: F1 on a 60-image COCO sample by bundle

Bundle F1 at threshold 0.5 F1 at threshold 0.3
fast-1008 0.39 0.42
quality-336 0.41 0.30
fast-672 0.24 0.29

For scale, a full-size 8-bit SAM 3 scores 0.73 on the same sample and sam3edge-mini at 672 px scores 0.70. With the image encoder on the NPU the fast-672 figure is unchanged (0.30 against 0.29). python validate/validate.py --bundles none --micro fast-672,fast-1008,quality-336 in sam3edge-mini re-runs this table on your machine.

Against the other quantised SAM 3 models

Hugging Face lists 34 other models quantised from facebook/sam3. Every one is the full 860M-parameter model, built for GPUs, phones or Apple Silicon. One of them (tepirale/sam3-ONNX, in the variant its card shows closest to the original) was run beside sam3edge-micro with the same script, on the same laptop CPU and the same pictures.

Seconds to answer the first phrase on the same laptop CPU

Peak memory on the same laptop

Accuracy on the same 60-image COCO sample

Files First concept Memory F1
Full SAM 3, 8-bit weights, 1008 px 1.06 GB 17.3 s 5.9 GB 0.73
sam3edge-micro quality-336 0.60 GB 1.54 s 0.89 GB 0.41
sam3edge-micro fast-1008 0.22 GB 0.66 s 0.63 GB 0.42 (threshold 0.3)
sam3edge-micro fast-672 0.22 GB 0.27 s 0.47 GB 0.29 (threshold 0.3)

A fast bundle is the smallest complete model on that list (220 MB; the next is 547 MB and runs on Apple Silicon only), the lightest in memory and the fastest on a CPU. It is also the least accurate: the full-size model finds far more. No other card on the list reports a time on an ARM board's CPU or a run on a Qualcomm NPU. The full comparison is in sam3edge-mini's COMPARISON.md, and its validate/validate.py re-runs these numbers on your machine.

Hexagon NPU

The fast bundles' image encoder runs on a QCS6490's NPU about five times faster than on its CPU. The NPU takes fixed-point graphs only; the layout that keeps the features is 16-bit activations with 8-bit per-channel weights.

Image encoder time on four Cortex-A78 cores against the Hexagon NPU

Similarity of the quantised encoder's features to the float ones for four fixed-point layouts

The first load compiles the graph for the NPU (about 25 s) and caches it beside the bundle. On the NPU the answers shift a little: on one test picture a car scored 0.79 against 0.83 on the CPU, and a truck at 0.45 fell below the threshold.

What is in this repository

Path What
sam3edge_micro/ the runtime: three small Python files
bundles/<name>/ image.onnx, detect.onnx, mask.onnx, their weights, vocab.npz (the concept list's text tokens) and micro.json
model.safetensors the fast bundle's weights in one file, for inspection and for the size shown beside the model's name
assets/ the pictures and charts on this page

Limits

  • The fast bundles are weak on small objects and on low-resolution or dark scenes, and they answer concepts that are not in the picture more often than SAM 3 does ("person" on animals and vehicles above all). Use a threshold near 0.3 and treat results as hints.
  • The quality bundle at 336 px loses small objects and takes four times as long.
  • A fixed concept list and no tracking. Masks come from a 4x-upsampled grid (96 px across for fast-672 and quality-336), so thin structures are coarse.
  • Not measured on an Arduino UNO Q or any Cortex-A53 board.
  • Detections are advisory. Do not use them alone for safety, identification or enforcement decisions.

Licence and provenance

Everything here derives from SAM 3 and is distributed under the SAM License (LICENSE), which requires this licence to travel with any redistribution and forbids several uses, including military and weapons applications. The distilled encoders come from EfficientSAM3 (Apache-2.0 on its model card). The concept list's text tokens were made with a text encoder that follows Apple's MobileCLIP-S0, which Apple publishes under a research-only licence: weigh that before commercial use. NOTICE.md lists each part. If you publish work that uses these files, the SAM License asks you to acknowledge the use of SAM Materials.

Sample pictures: a SAM 3 test image; COCO train2017 image 531286 (Creative Commons Attribution 2.0).

Citation

sam3edge-micro is a conversion. Please cite the work it converts: SAM 3: Segment Anything with Concepts (Carion et al., Meta, 2025, arXiv:2511.16719) and EfficientSAM3, whose distilled image encoder the fast bundles use.

Downloads last month

-

Downloads are not tracked for this model. How to track
Safetensors
Model size
50.2M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for eoinedge/sam3edge-micro

Base model

facebook/sam3
Quantized
(36)
this model

Space using eoinedge/sam3edge-micro 1

Paper for eoinedge/sam3edge-micro

Evaluation results