sam3edge-mini

SAM 3's text-prompted detector with masks, cut down to run on boards with no GPU and with no PyTorch: numpy, Pillow and ONNX Runtime. Give it a picture and a short noun phrase ("red bus", "person wearing a helmet") and it returns a box, a score and, if you ask, a mask for every instance. Any phrase works: the text encoder is in the bundle.

Try it in the browser: the demo Space.

Runs on a Rubik Pi 3 (Qualcomm QCS6490) and other ARM boards, x86 PCs, NVIDIA GPUs, Qualcomm Hexagon NPUs
Needs 0.7 GB to 1.9 GB of memory and 0.4 GB to 0.8 GB of files, depending on the bundle
Speed 1.4 s to 4.3 s for a first phrase on a Rubik Pi 3; 0.7 s to 6.6 s on a laptop CPU
Accuracy F1 0.70 on a 60-picture COCO sample at 672 px, against 0.73 for a full-size 8-bit SAM 3
Size 524M parameters (quality bundles, 445M of them in 8 bits) or 93M (fast bundle); SAM 3 has 860M

Nothing here was trained by this project. The weights are SAM 3's, and its distilled encoders from EfficientSAM3, converted, quantised and packaged with a runtime.

It is one of a family:

Model For Prompts Returns
sam3edge-mini (this one) 8 GB boards, PCs, GPUs any phrase boxes, or masks
sam3edge-micro 4 GB boards a fixed list of 104 concepts boxes, or masks
sam3edge-nano microcontroller-class boards planned, not released

Quick start

pip install onnxruntime numpy pillow huggingface_hub
hf download eoinedge/sam3edge-mini --include "sam3edge_mini/*" --include "bundles/quality-336/*" --local-dir sam3edge-mini
cd sam3edge-mini
python -m sam3edge_mini detect bundles/quality-336 frame.jpg --concept "red bus" --concept person --masks --out out.jpg
from PIL import Image
from sam3edge_mini import Mini

mini = Mini("bundles/quality-336")
image = Image.open("frame.jpg")
for det in mini.detect(image, ["red bus", "person"], masks=True):
    print(det["concept"], det["score"], det["box_xyxy"], det["mask"].shape)   # mask: a boolean array, the picture's size

The image pass is the expensive part and is paid once per picture: features = mini.encode_image(image), then mini.detect(image, [...], features=features) for as many phrases as you like. python -m sam3edge_mini bench times a bundle on your machine and info prints what it needs. The runtime refuses to load when a board has less free memory than the bundle needs (--force overrides), because a board with no swap stalls when it runs out.

On an NVIDIA GPU install onnxruntime-gpu[cuda,cudnn] instead and pass --provider cuda (Mini(..., provider="cuda")). On a board with a Hexagon NPU install onnxruntime-qnn and pass --provider npu with the fast bundle.

Which bundle

Bundle Image encoder Input Files Memory Best for
bundles/quality-672 SAM 3's own, 8-bit weights 672 px 0.77 GB 1.9 GB accuracy; cameras of 640 px and up
bundles/quality-336 SAM 3's own, 8-bit weights 336 px 0.77 GB 1.0 GB 8 GB boards; low-resolution feeds
bundles/fast-1008 distilled EfficientViT-B1 1008 px, 36 x 36 tokens 0.39 GB 0.76 GB speed; large, clear objects

quality-672 also carries its image encoder in float (image-fp32.onnx, 1.8 GB) for GPUs, which cannot run the 8-bit form; on a CPU, skip it with --exclude "*image-fp32*". fast-1008 also carries a fixed-point image encoder for a Hexagon NPU (image-npu.onnx).

Samples

Every picture below is the model's own output on a laptop CPU. Nothing is hand-edited.

All three bundles on the SAM 3 test images. The fast bundle misses the truck and is unsure about the bags; the quality bundles find the truck and all four bags at either size.

Fast bundle, quality bundle at 336 px and at 672 px on a pickup truck (prompts: truck, wheel) and a car boot of groceries (prompt: paper bag)

All three on COCO pictures. The fast bundle answers "person" on the bus and on the cars. The quality bundles return one bus and three cars and no person, at 0.87 and above. On the horse team they return four horses and six people at 336 px, and eight horses and eleven people at 672 px, where the small figures on the wagon become visible.

Fast bundle, quality bundle at 336 px and at 672 px on a bus in traffic (prompts: bus, car, person) and a team of horses in a street (prompts: horse, person)

A clip, one frame at a time. The quality bundle at 336 px answering "person" on every second frame of SAM 3's sample video (1280 x 720). There is no tracker: each frame is answered on its own, so the count changes from frame to frame and nobody keeps an identity.

Per-frame "person" detection with masks on SAM 3's sample video of four dancers

The same clip as video: assets/sample-clip.mp4.

Speed

On a Rubik Pi 3 (Qualcomm QCS6490, four Cortex-A78 cores, the board's vision services paused), measured with this repository's runtime:

Seconds to answer the first phrase on a Rubik Pi 3, split into image pass and detector pass

Bundle Image encoder on Image pass First phrase Each further phrase Masks, extra per phrase Memory
quality-336 the CPU 3.84 s 4.34 s 0.49 s 0.71 s 0.94 GB
fast-1008 the CPU 1.19 s 2.29 s 1.08 s 0.83 s 0.70 GB
fast-1008 the Hexagon NPU 0.22 s 1.37 s 1.13 s 0.88 s 0.55 GB

"First phrase" is the image pass, the text encoder and the detector. EfficientSAM3's EV-M, whose weights the fast bundle uses, takes 18.4 s on the same board run as published (PyTorch, SAM 3's 72 x 72 token grid, the mask head on every call). quality-672 was not timed on the Pi: the board had 2.6 GB free beside its other services, too little margin for a bundle that needs 1.9 GB.

On a laptop CPU (Ryzen 7 4700U, 8 cores), the same measurement through validate/validate.py:

Bundle Image pass First phrase Each further phrase Masks, extra per phrase Memory
quality-672 6.03 s 6.63 s 0.59 s 0.57 s 1.9 GB
quality-336 1.36 s 1.48 s 0.12 s 0.12 s 1.0 GB
fast-1008 0.39 s 0.68 s 0.30 s 0.32 s 0.76 GB

On an NVIDIA GPU every graph runs through onnxruntime-gpu (--provider cuda). It has not been timed yet; validate/gpu_validate.sh does that on your own card.

Accuracy

A seeded sample of 60 COCO val2017 pictures: every category COCO labels in a picture is asked for, plus two it does not hold; a detection counts when it overlaps an unused ground-truth box at IoU 0.5. It is a relative check between settings, not a benchmark, and you can re-run it (see below).

F1 on a 60-image COCO sample for each bundle and a full-size 8-bit SAM 3

F1 at threshold 0.5 F1 at threshold 0.3 Precision / recall at 0.5
Full SAM 3, 8-bit weights, 1008 px (another publisher's export) 0.73 0.64 0.67 / 0.80
quality-672 0.70 0.61 0.68 / 0.72
quality-672, float image encoder (the GPU form, run on a CPU) 0.70 0.60 0.68 / 0.72
quality-336 0.41 0.30 0.46 / 0.37
fast-1008 0.39 0.42 0.74 / 0.26

Against the other quantised SAM 3 models

Hugging Face lists 34 other models quantised from facebook/sam3. Every one is the full 860M-parameter model, nearly all at 1008 px, built for GPUs, phones or Apple Silicon. One of them (tepirale/sam3-ONNX, in the variant its card shows closest to the original) was run here beside sam3edge-mini with the same script, on the same laptop CPU and the same pictures.

Seconds to answer the first phrase on the same laptop CPU

Peak memory on the same laptop

Files First phrase Each further phrase Memory F1
Full SAM 3, 8-bit weights, 1008 px 1.06 GB 17.3 s 2.53 s 5.9 GB 0.73
quality-672 0.77 GB 6.63 s 0.59 s 1.9 GB 0.70
quality-336 0.77 GB 1.48 s 0.12 s 1.0 GB 0.41
fast-1008 0.39 GB 0.68 s 0.30 s 0.76 GB 0.39

The quality bundle at 672 px answers 2.6 times sooner in a third of the memory, for 0.03 of F1: the same precision and less recall, because it sees the picture smaller. The full-size model is the more accurate. On a CPU, other cards on the list report 46 s to 68 s for a picture; none reports a time on an ARM board's CPU or a run on a Qualcomm NPU. Several are far faster than this on a GPU. COMPARISON.md lists all 34 with what their cards report.

Check it yourself

# on a CPU: speed, memory and the 60 COCO pictures
hf download eoinedge/sam3edge-mini --include "sam3edge_mini/*" --include "validate/*" --local-dir sam3edge-mini
python sam3edge-mini/validate/validate.py

# on an NVIDIA GPU (Ubuntu): the same, in its own Python environment, each answer compared with the CPU's
curl -fsSLO https://huggingface.co/eoinedge/sam3edge-mini/resolve/main/validate/gpu_validate.sh
bash gpu_validate.sh cuda

The script prints your numbers beside the ones on this page. --other tepirale/sam3-ONNX runs the comparison model through the same scoring. Details are in validate/README.md.

Hexagon NPU

The fast bundle's image encoder runs on a QCS6490's Hexagon NPU through ONNX Runtime's QNN provider. The NPU takes fixed-point graphs only, and which layout keeps the features was measured:

Image encoder time on four Cortex-A78 cores against the Hexagon NPU at 672 px and 1008 px

Similarity of the quantised encoder's features to the float ones for four fixed-point layouts

Activations Weights Similarity to float
8-bit 8-bit 0.25 to 0.36 detections vanish
16-bit 8-bit per tensor 0.67 to 0.80
16-bit 8-bit per channel 0.997 to 0.999 what image-npu.onnx is
16-bit 16-bit 0.9994 to 0.9996 refused by this chip's NPU

The first load compiles the graph for the NPU and caches it beside the bundle. The quality bundles' encoder is not on the NPU.

What is in this repository

Path What
sam3edge_mini/ the runtime: four small Python files and the tokenizer's vocabulary
bundles/<name>/ image.onnx, text.onnx, detect.onnx, mask.onnx, their weights (.data) and mini.json
model.safetensors the quality bundle's weights in one file, for inspection and for the size shown beside the model's name
validate/ the script and the 60-picture sample behind the numbers on this page
COMPARISON.md the other quantised SAM 3 models and how this one stands against them
assets/ the pictures and charts on this page

Limits

  • Less accurate than full-size SAM 3. The quality bundle at 672 px is 0.03 of F1 behind it on the sample above; at 336 px it loses small objects; the fast bundle is weaker again and answers "person" on animals and vehicles.
  • Match the input to the camera. 336 px suits low-resolution feeds. On 352 x 288 street cameras the fast bundle misses buses, and at night even the quality bundle misses vehicles.
  • Single pictures. Video is answered frame by frame, with no tracker and no identities. SAM 3's video mode and its point and box prompts are not here.
  • English noun phrases. Long sentences are cut at 16 tokens (32 in the fast bundle).
  • Not measured on a QCS8300 board.
  • Detections are advisory. Do not use them alone for safety, identification or enforcement decisions.

How it was made

SAM 3's image model was reduced to what a text prompt needs (one feature level for the detector, the mask head only on request) and its input size made a choice. Each part was exported to an ONNX graph. For the quality bundles the image encoder's weights are stored in 8 bits; for the fast bundle a second, fixed-point copy of the image encoder was calibrated for the Hexagon NPU. The runtime is plain numpy around four ONNX Runtime sessions, and it reproduces the PyTorch model it came from: on four test pictures, 29 detections matched with scores within 0.0001 and identical masks.

Licence and provenance

Everything here derives from SAM 3 and is distributed under the SAM License (LICENSE), which requires this licence to travel with any redistribution and forbids several uses, including military and weapons applications. The distilled encoders come from EfficientSAM3 (Apache-2.0 on its model card). The text encoder follows Apple's MobileCLIP-S0, which Apple publishes under a research-only licence: weigh that before commercial use. NOTICE.md lists each part. If you publish work that uses these files, the SAM License asks you to acknowledge the use of SAM Materials.

Sample pictures: SAM 3's test images and sample video; COCO train2017 images 531286 and 206430 (Creative Commons Attribution 2.0).

Citation

sam3edge-mini is a conversion. Please cite the work it converts: SAM 3: Segment Anything with Concepts (Carion et al., Meta, 2025, arXiv:2511.16719) and EfficientSAM3, whose distilled encoders the fast bundle and the text encoder use.

Downloads last month

-

Downloads are not tracked for this model. How to track
Safetensors
Model size
0.5B params
Tensor type
F32
ยท
I8
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for eoinedge/sam3edge-mini

Base model

facebook/sam3
Quantized
(36)
this model

Space using eoinedge/sam3edge-mini 1

Paper for eoinedge/sam3edge-mini

Evaluation results