| --- |
| license: other |
| license_name: sam-license |
| license_link: LICENSE |
| base_model: facebook/sam3 |
| pipeline_tag: mask-generation |
| tags: |
| - sam3 |
| - segmentation |
| - open-vocabulary |
| - core-ai |
| - apple-silicon |
| --- |
| |
| # SAM 3 — Core AI (float16) |
|
|
| Masks by name, on Apple silicon: Meta's **SAM 3** converted to a Core AI segmenter bundle |
| (`sam3-float16/`, 1.6 GB). Give it an image and a word or short phrase — `parrot`, `eye`, |
| `the red car` — and it returns every instance as a mask, a box and a score. |
|
|
| ## Credit and licence |
|
|
| - **Model:** [SAM 3](https://huggingface.co/facebook/sam3) (Segment Anything with Concepts), |
| Meta. The weights are under the **SAM License** — a copy is in [`LICENSE`](LICENSE) and |
| applies to this conversion. Its trade-control terms apply to anyone using it. |
| - **Conversion:** Apple's official recipe, [apple/coreai-models](https://github.com/apple/coreai-models) |
| (BSD-3) at commit `52dbd93`, unmodified: `uv run models/sam3/export.py --full --dtype float16`. |
|
|
| This repository only converts and hosts the weights; all credit for the model belongs to Meta. |
|
|
| ## Bundle |
|
|
| ``` |
| sam3-float16/ |
| metadata.json kind: segmenter, assets.main → sam3_float16.aimodel |
| sam3_float16.aimodel/ main.mlirb (float16 weights + graph) + main.hash + metadata.json |
| tokenizer/ CLIP tokenizer |
| ``` |
|
|
| `main.mlirb` sha256 `856b7520fcf8d86c4e928bb2374a3ffd2c82370bad9f16d4b47a72832ad81f61`. |
| An `.aimodel` is a build artefact — the same recipe on another OS build gives different bytes — |
| so this one is pinned by hash; its behaviour is what is measured below. |
|
|
| ## Measured (macOS 27, M3 Max, 36 GB, GPU) |
|
|
| | Measure | Result | |
| |---|---| |
| | first load (compiles for this machine; cached by the OS after) | 11.6 s | |
| | one prompt, warm, 1008 px internally | 0.84 s | |
| | peak memory footprint | 1.1 GB | |
| | IoU on ten known-mask composites (five animals × plain / busy background) | **0.986–0.993** | |
| | same composites, SAM 2.1 Tiny clicked at the object's centre | 0.58–0.99 | |
|
|
| On the five stock animal photos macOS ships, every true match scored **≥ 0.84** and the worst |
| false one 0.571 (an eagle's nostril for "eye"); a concept that is absent ("sky" in a close-up) |
| scored ≤ 0.07. The tools built on this use **0.6** as the floor. Two of a kind come back as two |
| instances (an owl's eyes: 0.955 and 0.951). |
|
|
| Apple's iOS "lite" export (336 px, palettized) was measured as well and is not hosted: 0.2 s and |
| 383 MB, but its scores spread from 0.07 to 0.93 on the same composites (one animal missed) and |
| IoU fell to 0.75–0.97. |
|
|
| ## Use |
|
|
| With the `cutout` and `img` command-line tools, which fetch this on first use: |
|
|
| ```sh |
| cutout photo.jpg --what "the dog" # every dog, onto transparency |
| img grade photo.jpg --mask sky --exposure -0.5 # darken only the sky |
| ``` |
|
|
| From Swift, with [swift-concept-segmenter](https://github.com/arraypress/swift-concept-segmenter): |
|
|
| ```swift |
| import ConceptSegmenter |
| |
| let segmenter = try await ConceptSegmenter(bundle: folder) // this repo's sam3-float16/ |
| let eyes = try await segmenter.find("eye", in: image) // [ConceptMatch], best first |
| let sky = try await segmenter.mask(of: "sky", in: image) // every match in one mask |
| ``` |
|
|
| Or directly with Apple's runtime: `ImageSegmenter(resourcesAt:)` from `CoreAISegmentation`. |
|
|
