---
library_name: mlx
license: apache-2.0
license_link: https://github.com/facebookresearch/EdgeTAM/blob/main/LICENSE
base_model: facebookresearch/EdgeTAM
pipeline_tag: image-segmentation
tags:
  - mlx
  - segmentation
  - promptable-segmentation
  - sam2
  - edgetam
---

# mlx-community/EdgeTAM-fp16

[EdgeTAM](https://github.com/facebookresearch/EdgeTAM) — on-device SAM 2 for promptable segmentation +
video tracking — converted to **Apple MLX** (`-fp16`) for the
[`mlx-edgetam-swift`](https://github.com/xocialize/mlx-edgetam-swift) Swift package (MLXEngine
`promptSegment` + `trackObject` ModelPackage). 22× faster than SAM 2, 16 FPS on iPhone 15 Pro Max.

From-scratch MLX-Swift architecture port. **Image-mode** (point/box → mask): RepViT-M1 encoder + FPN +
SAM prompt encoder + two-way mask decoder — parity-locked vs the PyTorch oracle on the CPU stream
(image_embed 9.7e-6, mask logits 8.9e-5; end-to-end mask **IoU 0.99** vs PyTorch). **Video-mode**
(`trackObject`, click on one frame → per-frame masklet): adds the video memory stack — PerceiverResampler
+ MemoryEncoder + MemoryAttention (RoPE-2D) + the SAM2 memory-bank state machine — every op parity-locked
vs the oracle; full masklet propagation min-IoU 0.92. This single `-fp16` file carries both (874 tensors).

## Use

```swift
// Package.swift → .package(url: "https://github.com/xocialize/mlx-edgetam-swift", from: "0.1.0")
import EdgeTAM
// Image: click → object mask
let p = try EdgeTAMPredictor.fromPretrained(weightsPath, dtype: .float16)
p.setImage(sourceCGImage)
let (mask, score, _, _) = p.predict(point: (500, 375))
// Video: click on a frame → per-frame masklet
let vp = try EdgeTAMVideoPredictor.fromPretrained(weightsPath, dtype: .float16)
let track = vp.track(frames: cgImages, clickFrame: 0, points: [[210, 350]], labels: [1])
```

Or as an MLXEngine ModelPackage (`MLXEdgeTAM.EdgeTAMPackage`) — `promptSegment` (image) + `trackObject`
(video) surfaces — resolving this repo via the Hub.

Weights: Apache-2.0 (facebookresearch/EdgeTAM). Port code: MIT.