arraypress/segment-sam3

SAM 3 — Core AI (float16)

Masks by name, on Apple silicon: Meta's SAM 3 converted to a Core AI segmenter bundle (sam3-float16/, 1.6 GB). Give it an image and a word or short phrase — parrot, eye, the red car — and it returns every instance as a mask, a box and a score.

Credit and licence

• Model: SAM 3 (Segment Anything with Concepts), Meta. The weights are under the SAM License — a copy is in LICENSE and applies to this conversion. Its trade-control terms apply to anyone using it.
• Conversion: Apple's official recipe, apple/coreai-models (BSD-3) at commit 52dbd93, unmodified: uv run models/sam3/export.py --full --dtype float16.

This repository only converts and hosts the weights; all credit for the model belongs to Meta.

Bundle

sam3-float16/
  metadata.json            kind: segmenter, assets.main → sam3_float16.aimodel
  sam3_float16.aimodel/    main.mlirb (float16 weights + graph) + main.hash + metadata.json
  tokenizer/               CLIP tokenizer

main.mlirb sha256 856b7520fcf8d86c4e928bb2374a3ffd2c82370bad9f16d4b47a72832ad81f61. An .aimodel is a build artefact — the same recipe on another OS build gives different bytes — so this one is pinned by hash; its behaviour is what is measured below.

Measured (macOS 27, M3 Max, 36 GB, GPU)

Measure	Result
first load (compiles for this machine; cached by the OS after)	11.6 s
one prompt, warm, 1008 px internally	0.84 s
peak memory footprint	1.1 GB
IoU on ten known-mask composites (five animals × plain / busy background)	0.986–0.993
same composites, SAM 2.1 Tiny clicked at the object's centre	0.58–0.99

On the five stock animal photos macOS ships, every true match scored ≥ 0.84 and the worst false one 0.571 (an eagle's nostril for "eye"); a concept that is absent ("sky" in a close-up) scored ≤ 0.07. The tools built on this use 0.6 as the floor. Two of a kind come back as two instances (an owl's eyes: 0.955 and 0.951).

Apple's iOS "lite" export (336 px, palettized) was measured as well and is not hosted: 0.2 s and 383 MB, but its scores spread from 0.07 to 0.93 on the same composites (one animal missed) and IoU fell to 0.75–0.97.

Use

With the cutout and img command-line tools, which fetch this on first use:

cutout photo.jpg --what "the dog"          # every dog, onto transparency
img grade photo.jpg --mask sky --exposure -0.5    # darken only the sky

From Swift, with swift-concept-segmenter:

import ConceptSegmenter

let segmenter = try await ConceptSegmenter(bundle: folder)   // this repo's sam3-float16/
let eyes = try await segmenter.find("eye", in: image)          // [ConceptMatch], best first
let sky = try await segmenter.mask(of: "sky", in: image)       // every match in one mask

Or directly with Apple's runtime: ImageSegmenter(resourcesAt:) from CoreAISegmentation.