MediaPipe Gesture Recognizer

Nested two-bundle MediaPipe pipeline (gesture_recognizer.task, 8.4 MB, FP16 weights) combining hand_landmarker.task (palm detector + 21-keypoint 3D hand landmarker) and hand_gesture_recognizer.task (landmark embedder + 8-class canned gesture classifier) for the MediaPipe Gesture Recognizer (GestureRecognizer) task.

Documentation: MediaPipe Gesture Recognizer Docs

Samples: Android · iOS · Python Colab · Web · Flutter

Try live: Web Demo

gesture_recognizer.task packages two nested .task archives: hand_landmarker.task (palm detection + 21-point 3D landmarks) and hand_gesture_recognizer.task, which classifies hand landmark geometry (never raw pixels) into 8 built-in gesture classes:

Index Category Label Description
0 Unknown (None) Unrecognized hand pose
1 Closed_Fist Closed fist
2 Open_Palm Open palm
3 Pointing_Up Pointing up
4 Thumb_Down Thumbs down
5 Thumb_Up Thumbs up
6 Victory Victory / peace sign
7 ILoveYou I Love You gesture

Files

File Precision Size SHA-256
gesture_recognizer.task float16 8.37 MB (8,373,440 B) 97952348cf6a6a4915c2ea1496b4b37ebabc50cbbf80571435643c455f2b0482

Companion Files in Repo Root

File Role Size SHA-256
model_card_hand_gesture_classification_with_faireness_2022.pdf Upstream Google model card (PDF) 424.0 KB (423,983 B) 87ca48bdad620e55ed7808522f8dd7d1a0fe41d47b65ff7aae7e4ee5b7c29656

Download

pip install -U huggingface_hub
hf download litert-community/MediaPipe-GestureRecognizer gesture_recognizer.task --local-dir .

Inputs, Outputs, and Model Specifications

Bundle Contents (gesture_recognizer.task)

Bundled Member Uncompressed Size SHA-256 (prefix)
hand_landmarker.task 7.82 MB (7,819,105 B) 1c454b1f487e6a9f…
hand_landmarker.task -> hand_detector.tflite 2.34 MB (2,339,878 B) 60d1bf8d70a80aba…
hand_landmarker.task -> hand_landmarks_detector.tflite 5.48 MB (5,478,949 B) 6acda74af3fbf40e…
hand_gesture_recognizer.task 554.1 KB (554,062 B) 1b47ac4a3d9b0e46…
hand_gesture_recognizer.task -> gesture_embedder.tflite 546.0 KB (546,000 B) 54abe78de1d1cd5e…
hand_gesture_recognizer.task -> canned_gesture_classifier.tflite 7.8 KB (7,773 B) 62a87ded76da0515…

Sub-model hand_landmarker.task -> hand_detector.tflite

  • Embedded TFLITE_METADATA: name Multi Hand Detection · version 1 · author MediaPipe · packed associated files: labels.txt
Direction Tensor Shape Type Notes
Input input_1 [1, 192, 192, 3] float32 image — Input image to be detected — Normalization: mean=[0.0], std=[255.0]
Output Identity [1, 2016, 18] float32 palm detection — regressed detected palm locations
Output Identity_1 [1, 2016, 1] float32 scores — palm detection scores — Labels: labels.txt

Sub-model hand_landmarker.task -> hand_landmarks_detector.tflite

  • Embedded TFLITE_METADATA: name HandLandmarkDetector · version 1 · author MediaPipe · packed associated files: handedness.txt
Direction Tensor Shape Type Notes
Input input_1 [1, 224, 224, 3] float32 image — Input image to be detected. — Normalization: mean=[0.0, 0.0, 0.0], std=[255.0, 255.0, 255.0]
Output Identity [1, 63] float32 landmarks — The hand landmarks in normalized coordinates.
Output Identity_1 [1, 1] float32 presense score — The score of hand presence in the image.
Output Identity_2 [1, 1] float32 handedness — The score of left or right handedness. — Labels: handedness.txt
Output Identity_3 [1, 63] float32 world landmarks — The hand landmarks in world coordinates.

Sub-model hand_gesture_recognizer.task -> gesture_embedder.tflite

Direction Tensor Shape Type Notes
Input hand [1, 21, 3] float32 —
Input handedness [1, 1] float32 —
Input world_hand [1, 21, 3] float32 —
Output Identity [1, 128] float32 —

Sub-model hand_gesture_recognizer.task -> canned_gesture_classifier.tflite

  • Embedded TFLITE_METADATA: name Gesture Classifier · version 1 · author MediaPipe · packed associated files: labels.txt
Direction Tensor Shape Type Notes
Input hand_embedding [-1, 128] float32 gesture embedding — embedding feature vector from gesture embedder.
Output Identity [-1, 8] float32 hand gestures — hand gesture category scores — Labels: labels.txt

Use with MediaPipe Tasks

Python Quickstart

pip install -U mediapipe huggingface_hub
import mediapipe as mp
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="litert-community/MediaPipe-GestureRecognizer",
    filename="gesture_recognizer.task",
)

BaseOptions = mp.tasks.BaseOptions
GestureRecognizer = mp.tasks.vision.GestureRecognizer
GestureRecognizerOptions = mp.tasks.vision.GestureRecognizerOptions
VisionRunningMode = mp.tasks.vision.RunningMode

options = GestureRecognizerOptions(
    base_options=BaseOptions(model_asset_path=model_path),
    running_mode=VisionRunningMode.IMAGE,
    num_hands=2,
)

with GestureRecognizer.create_from_options(options) as runner:
    mp_image = mp.Image.create_from_file("input.jpg")
    result = runner.recognize(mp_image)
    for gesture in result.gestures:
        top = gesture[0]
        print(f"{top.category_name}: {top.score:.3f}")

Cross-Platform Guides & Samples

Latency Benchmarks

End-to-end MediaPipe Task pipeline average latency reported by the Gesture Recognizer documentation:

Model Variant Device CPU Latency GPU Latency
GestureRecognizer Pixel 6 16.76 ms 20.87 ms

Intended Use, Limitations, and Fairness

Summarized from the bundled model_card_hand_gesture_classification_with_faireness_2022.pdf (2022):

  • Intended applications: Real-time static single-hand gesture classification from 3D MediaPipe hand landmarks (no raw RGB pixels enter the classifier head) for mobile AR, entertainment, and touchless hand controls.
  • Out-of-scope uses:
    • Any form of surveillance or identity recognition is explicitly out of scope and not enabled by this technology.
    • Human life-critical decisions.
    • Multi-hand gestures (e.g., two-handed heart), dynamic/motion gestures (e.g., waving), or translating sign language.
  • Fairness evaluation (summary): Measured with Specificity-Sensitivity F1 Score (SS F1):
    • Perceived Monk Skin Tone (Tones 1–10, ≈2,000 images): Weighted average SS F1 95.5% (range 94.3%–97.8%).
    • Perceived Gender Expression (≈8,000 images): Weighted average SS F1 93.9% (Male 94.4%, Female 93.3%, Unknown 93.1%).

Provenance

Every model binary in this repository is byte-identical (unmodified weights, metadata, and .task bundle archive bytes) to the official artifact served by Google from storage.googleapis.com/mediapipe-models.

Repo File Upstream URL GCS Generation Upstream Last-Modified Pinned Bucket Version
gesture_recognizer.task gesture_recognizer.task 1682480004119009 Wed, 26 Apr 2023 03:33:24 GMT /1/

License and Citation

  • License: apache-2.0 — Distributed under Apache License, Version 2.0 alongside the MediaPipe release artifacts.

Citation

@article{zhang2020mediapipehands,
  title   = {MediaPipe Hands: On-device Real-time Hand Tracking},
  author  = {Zhang, Fan and Bazarevsky, Valentin and Vakunov, Andrey and Tkachenka, Andrei and Sung, George and Chang, Chuo-Ling and Grundmann, Matthias},
  journal = {arXiv preprint arXiv:2006.10214},
  year    = {2020},
  url     = {https://arxiv.org/abs/2006.10214}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including litert-community/MediaPipe-GestureRecognizer

Paper for litert-community/MediaPipe-GestureRecognizer