--- license: apache-2.0 library_name: mediapipe pipeline_tag: mask-generation tags: - mediapipe - litert - tflite - mask-generation - on-device - android - ios - web - python - flutter - interactive-segmentation - promptable-segmentation - magictouch - int8 --- # MediaPipe MagicTouch (Interactive Segmenter) Stateful point/stroke-prompted interactive segmentation bundle (`interactive_segmentation.task`, 30.5 MB, INT8 v2 encoder/decoder at `768 x 768`) that encodes an image once and iteratively segments user-indicated objects via positive/negative strokes in the [MediaPipe Interactive Segmenter (`InteractiveSegmenter`)](https://developers.google.com/edge/mediapipe/solutions/vision/interactive_segmenter) task.

Documentation: MediaPipe Interactive Segmenter Docs

Samples: Android · iOS · Python Colab · Web · Flutter

Try live: Web Demo

`interactive_segmentation.task` (`interactive_segmenter_v2`) separates image feature extraction from interactive decoding for low-latency refinement: 1. **`interactive_segmentation_encoder.int8.tflite` (28.3 MB):** Runs once in `set_image()`, encoding `[-1, 768, 768, 3]` RGB images into 5 multi-scale feature maps (`384×384×16` .. `24×24×192`). 2. **`interactive_segmentation_decoder.int8.tflite` (2.3 MB):** Runs per user stroke call in `segment(strokes)`, combining the prompt canvas (`[1, 768, 768, 3]`) and cached encoder features into a `[1, 768, 768, 1]` mask. **Bundle version note:** The bundled [`Model Card MagicTouch.pdf`](./Model%20Card%20MagicTouch.pdf) (March 13, 2023) documents the earlier v1 single-graph `512 x 512 x 4` (≈6 MB) FP model (≈30 FPS on Pixel 7 GPU), whereas this `.task` bundle packages the upgraded **768×768 INT8 two-stage v2** encoder/decoder. ## Files | File | Precision | Size | SHA-256 | |---|---|---:|---| | [`interactive_segmentation.task`](https://huggingface.co/litert-community/MediaPipe-InteractiveSegmenter-MagicTouch/blob/main/interactive_segmentation.task) | `int8` | 30.53 MB (30,525,312 B) | `38431bc66b883404e8397f74c3579404315b9b52b04a46c6346fe906a7309b03` | ### Companion Files in Repo Root | File | Role | Size | SHA-256 | |---|---|---:|---| | [`Model Card MagicTouch.pdf`](./Model%20Card%20MagicTouch.pdf) | Upstream Google model card (PDF) | 301.8 KB (301,841 B) | `b4accf5f7be991903646e6dd9765faa6974cdf69c5a9e84841684dffa27df0f1` | ### Download ```bash pip install -U huggingface_hub hf download litert-community/MediaPipe-InteractiveSegmenter-MagicTouch interactive_segmentation.task --local-dir . ``` ## Inputs, Outputs, and Model Specifications ### Bundle Contents (`interactive_segmentation.task`) | Bundled Member | Uncompressed Size | SHA-256 (prefix) | |---|---:|---| | `interactive_segmentation_decoder.int8.tflite` | 2.27 MB (2,268,000 B) | `140d36971403c38b…` | | `interactive_segmentation_encoder.int8.tflite` | 28.26 MB (28,256,696 B) | `0edbceec02cb1570…` | | `manifest.pb` | 0.1 KB (50 B) | `c18392128d95dc4d…` | | `metadata.pb` | 0.0 KB (12 B) | `d4945e22f0e49ea2…` | ### Sub-model `interactive_segmentation_decoder.int8.tflite` | Direction | Tensor | Shape | Type | Notes | |---|---|---|---|---| | Input | `input_21` | `[1, 768, 768, 3]` | `float32` | — | | Input | `input_22` | `[1, 384, 384, 16]` | `float32` | — | | Input | `input_23` | `[1, 192, 192, 24]` | `float32` | — | | Input | `input_24` | `[1, 96, 96, 40]` | `float32` | — | | Input | `input_25` | `[1, 48, 48, 80]` | `float32` | — | | Input | `input_26` | `[1, 24, 24, 192]` | `float32` | — | | Output | `Identity` | `[1, 768, 768, 1]` | `float32` | — | ### Sub-model `interactive_segmentation_encoder.int8.tflite` | Direction | Tensor | Shape | Type | Notes | |---|---|---|---|---| | Input | `input_1` | `[-1, 768, 768, 3]` | `float32` | — | | Output | `Identity` | `[-1, 384, 384, 16]` | `float32` | — | | Output | `Identity_1` | `[-1, 192, 192, 24]` | `float32` | — | | Output | `Identity_2` | `[-1, 96, 96, 40]` | `float32` | — | | Output | `Identity_3` | `[-1, 48, 48, 80]` | `float32` | — | | Output | `Identity_4` | `[-1, 24, 24, 192]` | `float32` | — | ## Use with MediaPipe Tasks ### Python Quickstart ```bash pip install -U mediapipe huggingface_hub ``` ```python import mediapipe as mp from huggingface_hub import hf_hub_download model_path = hf_hub_download(repo_id="litert-community/MediaPipe-InteractiveSegmenter-MagicTouch", filename="interactive_segmentation.task") BaseOptions = mp.tasks.BaseOptions InteractiveSegmenter = mp.tasks.vision.InteractiveSegmenter InteractiveSegmenterOptions = mp.tasks.vision.InteractiveSegmenterOptions BrushMode = mp.tasks.vision.InteractiveSegmenterBrushMode Stroke = mp.tasks.vision.InteractiveSegmenterStroke StrokePoint = mp.tasks.vision.InteractiveSegmenterStrokePoint options = InteractiveSegmenterOptions( base_options=BaseOptions(model_asset_path=model_path) ) with InteractiveSegmenter.create_from_options(options) as segmenter: segmenter.set_image(mp.Image.create_from_file("input.jpg")) strokes = [ Stroke(brush_mode=BrushMode.POSITIVE, points=[StrokePoint(x=0.5, y=0.5)], is_completed=True) ] mask = segmenter.segment(strokes) print("Segmented mask size:", mask.width, "x", mask.height) ``` ### Cross-Platform Guides & Samples - **Task overview & options:** [Interactive Segmenter documentation](https://developers.google.com/edge/mediapipe/solutions/vision/interactive_segmenter) - **Platform setup guides:** [Python](https://developers.google.com/edge/mediapipe/solutions/setup_python) · [Web / JavaScript](https://developers.google.com/edge/mediapipe/solutions/setup_web) · [Android (Kotlin / Java)](https://developers.google.com/edge/mediapipe/solutions/setup_android) · [iOS (Swift)](https://developers.google.com/edge/mediapipe/solutions/setup_ios) · [Flutter (Dart)](https://developers.google.com/edge/mediapipe/solutions/setup_flutter) - **Runnable sample apps:** [Web Demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/vision/interactive_segmenter) · [Web Source](https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/interactive-segmenter.ts) · [Android](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/interactive_segmentation/android) · [iOS](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/interactive_segmentation/ios) · [Python Notebook](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/interactive_segmentation/python/interactive_segmenter.ipynb) · [Flutter](https://github.com/google/flutter-mediapipe/tree/main/samples/vision/interactive_segmenter) ## Latency Benchmarks End-to-end MediaPipe Task pipeline average latency reported by the [Interactive Segmenter documentation](https://developers.google.com/edge/mediapipe/solutions/vision/interactive_segmenter#benchmarks): | Model Variant | Device | CPU Latency | GPU Latency | |---|---|---:|---:| | MagicTouch (v2 int8) | Pixel 10 | 208.2 ms | 580.6 ms | ## Intended Use, Limitations, and Fairness Summarized from the bundled [`Model Card MagicTouch.pdf`](./Model%20Card%20MagicTouch.pdf) (Valentin Bazarevsky & Ben Hahn, Google, March 13, 2023): - **Intended applications:** Interactive photo editing and semi-automatic object annotation (objects, animals, humans) from user clicks or brush strokes. - **Out-of-scope uses:** - **Any form of surveillance or identity recognition is explicitly out of scope and not enabled by this technology.** - Human segmentation not consented to by the subject being segmented. - Human life-critical decisions, or small objects / people > 4 meters (14 ft) from the camera. - **Fairness evaluation (v1 MagicTouch PDF summary, 1,870 smartphone AR images vs. 98.74% human inter-annotator IoU):** - **Monk Skin Tones 1–10:** average IoU **87.7%** (range **85.3%** Tones 8–10 to **89.4%** Tone 5; max discrepancy 4.1%). - **Perceived gender:** average IoU **87.7%** (Male **87.3%**, Female **88.0%**; max discrepancy 0.7%). ## Provenance Every model binary in this repository is byte-identical (unmodified weights, metadata, and `.task` bundle archive bytes) to the official artifact served by Google from `storage.googleapis.com/mediapipe-models`. | Repo File | Upstream URL | GCS Generation | Upstream `Last-Modified` | Pinned Bucket Version | |---|---|---|---|---| | `interactive_segmentation.task` | [`interactive_segmentation.task`](https://storage.googleapis.com/mediapipe-models/interactive_segmenter_v2/magic_touch/int8/latest/interactive_segmentation.task) | `1781216228896365` | Thu, 11 Jun 2026 22:17:08 GMT | `/1/` | ## License and Citation - License: `apache-2.0` — Stated as Apache License, Version 2.0 in the bundled upstream Google model-card PDF.