File size: 3,316 Bytes
fa57c10 5b2b9fe fa57c10 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | ---
license: other
license_name: sam-license
license_link: LICENSE
base_model: facebook/sam3
pipeline_tag: mask-generation
tags:
- sam3
- segmentation
- open-vocabulary
- core-ai
- apple-silicon
---
# SAM 3 — Core AI (float16)
Masks by name, on Apple silicon: Meta's **SAM 3** converted to a Core AI segmenter bundle
(`sam3-float16/`, 1.6 GB). Give it an image and a word or short phrase — `parrot`, `eye`,
`the red car` — and it returns every instance as a mask, a box and a score.
## Credit and licence
- **Model:** [SAM 3](https://huggingface.co/facebook/sam3) (Segment Anything with Concepts),
Meta. The weights are under the **SAM License** — a copy is in [`LICENSE`](LICENSE) and
applies to this conversion. Its trade-control terms apply to anyone using it.
- **Conversion:** Apple's official recipe, [apple/coreai-models](https://github.com/apple/coreai-models)
(BSD-3) at commit `52dbd93`, unmodified: `uv run models/sam3/export.py --full --dtype float16`.
This repository only converts and hosts the weights; all credit for the model belongs to Meta.
## Bundle
```
sam3-float16/
metadata.json kind: segmenter, assets.main → sam3_float16.aimodel
sam3_float16.aimodel/ main.mlirb (float16 weights + graph) + main.hash + metadata.json
tokenizer/ CLIP tokenizer
```
`main.mlirb` sha256 `856b7520fcf8d86c4e928bb2374a3ffd2c82370bad9f16d4b47a72832ad81f61`.
An `.aimodel` is a build artefact — the same recipe on another OS build gives different bytes —
so this one is pinned by hash; its behaviour is what is measured below.
## Measured (macOS 27, M3 Max, 36 GB, GPU)
| Measure | Result |
|---|---|
| first load (compiles for this machine; cached by the OS after) | 11.6 s |
| one prompt, warm, 1008 px internally | 0.84 s |
| peak memory footprint | 1.1 GB |
| IoU on ten known-mask composites (five animals × plain / busy background) | **0.986–0.993** |
| same composites, SAM 2.1 Tiny clicked at the object's centre | 0.58–0.99 |
On the five stock animal photos macOS ships, every true match scored **≥ 0.84** and the worst
false one 0.571 (an eagle's nostril for "eye"); a concept that is absent ("sky" in a close-up)
scored ≤ 0.07. The tools built on this use **0.6** as the floor. Two of a kind come back as two
instances (an owl's eyes: 0.955 and 0.951).
Apple's iOS "lite" export (336 px, palettized) was measured as well and is not hosted: 0.2 s and
383 MB, but its scores spread from 0.07 to 0.93 on the same composites (one animal missed) and
IoU fell to 0.75–0.97.
## Use
With the `cutout` and `img` command-line tools, which fetch this on first use:
```sh
cutout photo.jpg --what "the dog" # every dog, onto transparency
img grade photo.jpg --mask sky --exposure -0.5 # darken only the sky
```
From Swift, with [swift-concept-segmenter](https://github.com/arraypress/swift-concept-segmenter):
```swift
import ConceptSegmenter
let segmenter = try await ConceptSegmenter(bundle: folder) // this repo's sam3-float16/
let eyes = try await segmenter.find("eye", in: image) // [ConceptMatch], best first
let sky = try await segmenter.mask(of: "sky", in: image) // every match in one mask
```
Or directly with Apple's runtime: `ImageSegmenter(resourcesAt:)` from `CoreAISegmentation`.
|