MediaPipe MagicTouch (Interactive Segmenter)

Stateful point/stroke-prompted interactive segmentation bundle (interactive_segmentation.task, 30.5 MB, INT8 v2 encoder/decoder at 768 x 768) that encodes an image once and iteratively segments user-indicated objects via positive/negative strokes in the MediaPipe Interactive Segmenter (InteractiveSegmenter) task.

Documentation: MediaPipe Interactive Segmenter Docs

Samples: Android · iOS · Python Colab · Web · Flutter

Try live: Web Demo

interactive_segmentation.task (interactive_segmenter_v2) separates image feature extraction from interactive decoding for low-latency refinement:

  1. interactive_segmentation_encoder.int8.tflite (28.3 MB): Runs once in set_image(), encoding [-1, 768, 768, 3] RGB images into 5 multi-scale feature maps (384×384×16 .. 24×24×192).
  2. interactive_segmentation_decoder.int8.tflite (2.3 MB): Runs per user stroke call in segment(strokes), combining the prompt canvas ([1, 768, 768, 3]) and cached encoder features into a [1, 768, 768, 1] mask.

Bundle version note: The bundled Model Card MagicTouch.pdf (March 13, 2023) documents the earlier v1 single-graph 512 x 512 x 4 (≈6 MB) FP model (≈30 FPS on Pixel 7 GPU), whereas this .task bundle packages the upgraded 768×768 INT8 two-stage v2 encoder/decoder.

Files

File Precision Size SHA-256
interactive_segmentation.task int8 30.53 MB (30,525,312 B) 38431bc66b883404e8397f74c3579404315b9b52b04a46c6346fe906a7309b03

Companion Files in Repo Root

File Role Size SHA-256
Model Card MagicTouch.pdf Upstream Google model card (PDF) 301.8 KB (301,841 B) b4accf5f7be991903646e6dd9765faa6974cdf69c5a9e84841684dffa27df0f1

Download

pip install -U huggingface_hub
hf download litert-community/MediaPipe-InteractiveSegmenter-MagicTouch interactive_segmentation.task --local-dir .

Inputs, Outputs, and Model Specifications

Bundle Contents (interactive_segmentation.task)

Bundled Member Uncompressed Size SHA-256 (prefix)
interactive_segmentation_decoder.int8.tflite 2.27 MB (2,268,000 B) 140d36971403c38b…
interactive_segmentation_encoder.int8.tflite 28.26 MB (28,256,696 B) 0edbceec02cb1570…
manifest.pb 0.1 KB (50 B) c18392128d95dc4d…
metadata.pb 0.0 KB (12 B) d4945e22f0e49ea2…

Sub-model interactive_segmentation_decoder.int8.tflite

Direction Tensor Shape Type Notes
Input input_21 [1, 768, 768, 3] float32 —
Input input_22 [1, 384, 384, 16] float32 —
Input input_23 [1, 192, 192, 24] float32 —
Input input_24 [1, 96, 96, 40] float32 —
Input input_25 [1, 48, 48, 80] float32 —
Input input_26 [1, 24, 24, 192] float32 —
Output Identity [1, 768, 768, 1] float32 —

Sub-model interactive_segmentation_encoder.int8.tflite

Direction Tensor Shape Type Notes
Input input_1 [-1, 768, 768, 3] float32 —
Output Identity [-1, 384, 384, 16] float32 —
Output Identity_1 [-1, 192, 192, 24] float32 —
Output Identity_2 [-1, 96, 96, 40] float32 —
Output Identity_3 [-1, 48, 48, 80] float32 —
Output Identity_4 [-1, 24, 24, 192] float32 —

Use with MediaPipe Tasks

Python Quickstart

pip install -U mediapipe huggingface_hub
import mediapipe as mp
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(repo_id="litert-community/MediaPipe-InteractiveSegmenter-MagicTouch", filename="interactive_segmentation.task")

BaseOptions = mp.tasks.BaseOptions
InteractiveSegmenter = mp.tasks.vision.InteractiveSegmenter
InteractiveSegmenterOptions = mp.tasks.vision.InteractiveSegmenterOptions
BrushMode = mp.tasks.vision.InteractiveSegmenterBrushMode
Stroke = mp.tasks.vision.InteractiveSegmenterStroke
StrokePoint = mp.tasks.vision.InteractiveSegmenterStrokePoint

options = InteractiveSegmenterOptions(
    base_options=BaseOptions(model_asset_path=model_path)
)

with InteractiveSegmenter.create_from_options(options) as segmenter:
    segmenter.set_image(mp.Image.create_from_file("input.jpg"))
    strokes = [
        Stroke(brush_mode=BrushMode.POSITIVE, points=[StrokePoint(x=0.5, y=0.5)], is_completed=True)
    ]
    mask = segmenter.segment(strokes)
    print("Segmented mask size:", mask.width, "x", mask.height)

Cross-Platform Guides & Samples

Latency Benchmarks

End-to-end MediaPipe Task pipeline average latency reported by the Interactive Segmenter documentation:

Model Variant Device CPU Latency GPU Latency
MagicTouch (v2 int8) Pixel 10 208.2 ms 580.6 ms

Intended Use, Limitations, and Fairness

Summarized from the bundled Model Card MagicTouch.pdf (Valentin Bazarevsky & Ben Hahn, Google, March 13, 2023):

  • Intended applications: Interactive photo editing and semi-automatic object annotation (objects, animals, humans) from user clicks or brush strokes.
  • Out-of-scope uses:
    • Any form of surveillance or identity recognition is explicitly out of scope and not enabled by this technology.
    • Human segmentation not consented to by the subject being segmented.
    • Human life-critical decisions, or small objects / people > 4 meters (14 ft) from the camera.
  • Fairness evaluation (v1 MagicTouch PDF summary, 1,870 smartphone AR images vs. 98.74% human inter-annotator IoU):
    • Monk Skin Tones 1–10: average IoU 87.7% (range 85.3% Tones 8–10 to 89.4% Tone 5; max discrepancy 4.1%).
    • Perceived gender: average IoU 87.7% (Male 87.3%, Female 88.0%; max discrepancy 0.7%).

Provenance

Every model binary in this repository is byte-identical (unmodified weights, metadata, and .task bundle archive bytes) to the official artifact served by Google from storage.googleapis.com/mediapipe-models.

Repo File Upstream URL GCS Generation Upstream Last-Modified Pinned Bucket Version
interactive_segmentation.task interactive_segmentation.task 1781216228896365 Thu, 11 Jun 2026 22:17:08 GMT /1/

License and Citation

  • License: apache-2.0 — Stated as Apache License, Version 2.0 in the bundled upstream Google model-card PDF.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including litert-community/MediaPipe-InteractiveSegmenter-MagicTouch