MediaPipe Selfie Multiclass Segmenter (256×256)

Vision Transformer human multiclass segmentation model (selfie_multiclass_256x256.tflite, 16.4 MB, FP32) classifying every pixel into 6 semantic categories (background, hair, body-skin, face-skin, clothes, others) for single- or multi-person AR and video effects with the MediaPipe Image Segmenter (ImageSegmenter) task.

Documentation: MediaPipe Image Segmenter Docs

Samples: Android · iOS · Python Colab · Web · Flutter

Try live: Web Demo

Accepts [1, 256, 256, 3] normalized RGB frames (portrait, landscape, or any aspect ratio) and outputs a [1, 256, 256, 6] per-class probability tensor for:

0: background · 1: hair · 2: body-skin · 3: face-skin · 4: clothes · 5: others (accessories)

Files

File Precision Size SHA-256
selfie_multiclass_256x256.tflite float32 16.37 MB (16,371,837 B) c6748b1253a99067ef71f7e26ca71096cd449baefa8f101900ea23016507e0e0

Companion Files in Repo Root

File Role Size SHA-256
Model Card Multiclass Segmentation.pdf Upstream Google model card (PDF) 1.38 MB (1,378,158 B) 59372fce0666d4c75b3877540fa77f80ac4b2a0f75e914711417f85fc490a9e6

Download

pip install -U huggingface_hub
hf download litert-community/MediaPipe-ImageSegmenter-Selfie-Multiclass selfie_multiclass_256x256.tflite --local-dir .

Inputs, Outputs, and Model Specifications

selfie_multiclass_256x256.tflite

  • Embedded TFLITE_METADATA: name ImageSegmenter · version 1 · author MediaPipe · packed associated files: labels.txt
Direction Tensor Shape Type Notes
Input input_29 [1, 256, 256, 3] float32 image — Input image to be processed. — Normalization: mean=[127.5], std=[127.5]
Output Identity [1, 256, 256, 6] float32 segmentation_masks — Masks over the target objects with high accuracy. — Labels: labels.txt

Use with MediaPipe Tasks

Python Quickstart

pip install -U mediapipe huggingface_hub
import mediapipe as mp
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="litert-community/MediaPipe-ImageSegmenter-Selfie-Multiclass",
    filename="selfie_multiclass_256x256.tflite",
)

BaseOptions = mp.tasks.BaseOptions
ImageSegmenter = mp.tasks.vision.ImageSegmenter
ImageSegmenterOptions = mp.tasks.vision.ImageSegmenterOptions
VisionRunningMode = mp.tasks.vision.RunningMode

options = ImageSegmenterOptions(
    base_options=BaseOptions(model_asset_path=model_path),
    running_mode=VisionRunningMode.IMAGE,
    output_category_mask=True,
    output_confidence_masks=True,
)

with ImageSegmenter.create_from_options(options) as runner:
    mp_image = mp.Image.create_from_file("input.jpg")
    result = runner.segment(mp_image)
    print("Masks:", len(result.confidence_masks), "shape:", result.category_mask.numpy_view().shape)

Cross-Platform Guides & Samples

Latency Benchmarks

End-to-end MediaPipe Task pipeline average latency reported by the Image Segmenter documentation:

Model Variant Device CPU Latency GPU Latency
SelfieMulticlass (256 x 256) Pixel 6 217.76 ms 71.24 ms

Intended Use, Limitations, and Fairness

Summarized from the bundled Model Card Multiclass Segmentation.pdf (Adel Ahmadyan, Google, May 10, 2023):

  • Intended applications: Real-time multiclass segmentation of human subjects (selfies, full body, single or multiple people) in interactive entertainment and video conferencing.
  • Out-of-scope uses:
    • Any form of surveillance or identity recognition is explicitly out of scope and not enabled by this technology.
    • Human life-critical decisions and applications requiring pixel-perfect masks.
  • Fairness evaluation (256×256 model, 918 images / 1,902 human subjects):
    • Overall mean IoU: 77.23% (SD 11.14).
    • Perceived gender: Male 77.10% (39.9%), Female 77.32% (60.1%).
    • Monk Skin Tone scale (1–10): Range from 68.25% (Tones 9+10, 1.6% of eval set) to 79.18% (Tone 6), within the 1-SD fairness threshold of the 512×512/256×256 model evaluation.

Provenance

Every model binary in this repository is byte-identical (unmodified FlatBuffer weights and embedded TFLITE_METADATA bytes) to the official artifact served by Google from storage.googleapis.com/mediapipe-models.

Repo File Upstream URL GCS Generation Upstream Last-Modified Pinned Bucket Version
selfie_multiclass_256x256.tflite selfie_multiclass_256x256.tflite 1682480017063560 Wed, 26 Apr 2023 03:33:37 GMT /1/

License and Citation

  • License: apache-2.0 — Stated as Apache License, Version 2.0 in the bundled upstream Google model-card PDF.
Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including litert-community/MediaPipe-ImageSegmenter-Selfie-Multiclass