| --- |
| license: other |
| license_name: sam-license |
| license_link: https://huggingface.co/facebook/sam3/blob/main/LICENSE |
| base_model: facebook/sam3 |
| pipeline_tag: image-segmentation |
| tags: |
| - coreai |
| - apple |
| - aimodel |
| - apple-silicon |
| - segmentation |
| - sam3 |
| --- |
| |
| # SAM 3 — official Apple Core AI export |
|
|
| A pre-converted **`.aimodel` segmenter bundle from Apple's official |
| [coreai-models](https://github.com/apple/coreai-models) export recipe — unmodified**, with |
| the exact environment, hashes, and measured performance published. |
|
|
| [SAM 3](https://huggingface.co/facebook/sam3) (Segment Anything Model 3, Meta) is a unified, |
| **promptable** segmentation model. This bundle is the **image + text-prompt** variant: give |
| it an image and a phrase (`cat`, `the red car`) and it returns instance masks, boxes, and |
| per-instance scores — open-vocabulary, no fixed class list. It runs on the **stock** Core AI |
| runtime via Apple's `CoreAIImageSegmenter` (no engine patch). |
|
|
| ```bash |
| # from an apple/coreai-models checkout (gated model — accept the SAM license + hf auth login first) |
| uv run models/sam3/export.py --dtype float16 |
| ``` |
|
|
| ## Why a pre-converted bundle? |
|
|
| An `.aimodel` is a build artifact, not a pure function of the recipe — the toolchain and OS |
| lowering matter, and the source checkpoint is **gated**. This is the exact, hash-stamped |
| bundle behind the published numbers, so you can drop it in and reproduce them without a |
| conversion environment or accepting the upstream license just to get the converted graph. |
|
|
| ## Bundle layout & integrity |
|
|
| A segmenter bundle directory (`metadata.json` schema 0.2 + the `.aimodel` + a CLIP `tokenizer/`): |
|
|
| ``` |
| sam3_float16.aimodel/ main.mlirb + main.hash + metadata.json |
| tokenizer/ tokenizer.json + tokenizer_config.json |
| metadata.json kind: segmenter, assets.main → sam3_float16.aimodel |
| ``` |
|
|
| | File | Contents | SHA-256 | |
| |---|---|---| |
| | `sam3_float16.aimodel/main.mlirb` | float16 weights + graph (~1.5 GB) | `be9484b4c5fa75f9011668c481e27de253e1db55fcde39a1ba0cab8698706620` | |
| | `sam3_float16.aimodel/main.hash` | graph hash | `9f631da3be044ad5de56a1d29af88394e7e725e3cf52f50ac556dcce9cdd568c` | |
|
|
| ## Measured |
|
|
| Apple's official `image-segmenter` CLI, **M4 Max**, the COCO two-cats image (640×480, resized |
| to 1008×1008 internally), text prompt `cat`: |
|
|
| | Metric | Value | |
| |---|---| |
| | Inference (warm, incl. pre/post-processing) | **0.55 s** | |
| | Output | 2 high-confidence instance masks for "cat" (scores **0.97** and **0.96**); spurious queries fall below the 0.5 threshold | |
|
|
| **float16 vs float32 fidelity** (same recipe, both exported and run through the engine): top-2 |
| scores differ by **≤ 1e-4** (0.9746 vs 0.9747, 0.9646 vs 0.9645), bounding boxes within **1 px**, |
| and the top-3 masks have **identical** foreground pixel counts. float16 is shipped (half the |
| size, iOS-friendly); float32 reproduces with `--dtype float32`. |
|
|
| ## Export environment |
|
|
| - macOS 27.0 beta · Xcode 27.0 beta |
| - `coreai-core 1.0.0b1` · `coreai-torch 0.4.0` · `transformers 5.x` |
| - apple/coreai-models recipe `models/sam3/export.py` (unmodified upstream) |
|
|
| ## Run it |
|
|
| ```bash |
| # CLI (from an apple/coreai-models checkout) |
| swift run -c release image-segmenter \ |
| --model <downloaded-bundle-dir> --prompt "cat" --image cats.jpg |
| ``` |
|
|
| Or in the [CoreAISegment sample app](https://github.com/john-rocky/coreai-model-zoo/tree/main/apps/CoreAISegment) |
| (macOS + iOS): pick an image, type a prompt, tap Segment. |
|
|
| ```swift |
| import CoreAIImageSegmenter |
| let segmenter = try await ImageSegmenter(resourcesAt: "<bundle-dir>") |
| let result = try await segmenter.segment(image: cgImage, prompt: "cat") |
| // result.segments: [Segment] — each has .mask, .box, .score |
| ``` |
|
|
| iOS bundles should be AOT-compiled before device use: |
| `xcrun coreai-build compile <ir>.aimodel --platform iOS --architecture h18p` |
| (h18p = iPhone 17 Pro), then point `metadata.json` `assets.main` at the `.aimodelc`. |
|
|
| ## License |
|
|
| SAM 3 is distributed under Meta's **SAM License** (see `LICENSE` and the |
| [upstream model card](https://huggingface.co/facebook/sam3)). This bundle is a format |
| conversion of those weights and inherits that license; review it before use. |
|
|
| --- |
| Maintained alongside [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo) |
| (see [`official/`](https://github.com/john-rocky/coreai-model-zoo/tree/main/official) for the |
| other unmodified official-recipe conversions). |
|
|
