MediaPipe Gesture Recognizer
Nested two-bundle MediaPipe pipeline (gesture_recognizer.task, 8.4 MB, FP16 weights) combining hand_landmarker.task (palm detector + 21-keypoint 3D hand landmarker) and hand_gesture_recognizer.task (landmark embedder + 8-class canned gesture classifier) for the MediaPipe Gesture Recognizer (GestureRecognizer) task.
gesture_recognizer.task packages two nested .task archives: hand_landmarker.task (palm detection + 21-point 3D landmarks) and hand_gesture_recognizer.task, which classifies hand landmark geometry (never raw pixels) into 8 built-in gesture classes:
| Index |
Category Label |
Description |
0 |
Unknown (None) |
Unrecognized hand pose |
1 |
Closed_Fist |
Closed fist |
2 |
Open_Palm |
Open palm |
3 |
Pointing_Up |
Pointing up |
4 |
Thumb_Down |
Thumbs down |
5 |
Thumb_Up |
Thumbs up |
6 |
Victory |
Victory / peace sign |
7 |
ILoveYou |
I Love You gesture |
Files
| File |
Precision |
Size |
SHA-256 |
gesture_recognizer.task |
float16 |
8.37 MB (8,373,440 B) |
97952348cf6a6a4915c2ea1496b4b37ebabc50cbbf80571435643c455f2b0482 |
Companion Files in Repo Root
Download
pip install -U huggingface_hub
hf download litert-community/MediaPipe-GestureRecognizer gesture_recognizer.task --local-dir .
Inputs, Outputs, and Model Specifications
Bundle Contents (gesture_recognizer.task)
| Bundled Member |
Uncompressed Size |
SHA-256 (prefix) |
hand_landmarker.task |
7.82 MB (7,819,105 B) |
1c454b1f487e6a9f… |
hand_landmarker.task -> hand_detector.tflite |
2.34 MB (2,339,878 B) |
60d1bf8d70a80aba… |
hand_landmarker.task -> hand_landmarks_detector.tflite |
5.48 MB (5,478,949 B) |
6acda74af3fbf40e… |
hand_gesture_recognizer.task |
554.1 KB (554,062 B) |
1b47ac4a3d9b0e46… |
hand_gesture_recognizer.task -> gesture_embedder.tflite |
546.0 KB (546,000 B) |
54abe78de1d1cd5e… |
hand_gesture_recognizer.task -> canned_gesture_classifier.tflite |
7.8 KB (7,773 B) |
62a87ded76da0515… |
Sub-model hand_landmarker.task -> hand_detector.tflite
- Embedded
TFLITE_METADATA: name Multi Hand Detection · version 1 · author MediaPipe · packed associated files: labels.txt
| Direction |
Tensor |
Shape |
Type |
Notes |
| Input |
input_1 |
[1, 192, 192, 3] |
float32 |
image — Input image to be detected — Normalization: mean=[0.0], std=[255.0] |
| Output |
Identity |
[1, 2016, 18] |
float32 |
palm detection — regressed detected palm locations |
| Output |
Identity_1 |
[1, 2016, 1] |
float32 |
scores — palm detection scores — Labels: labels.txt |
Sub-model hand_landmarker.task -> hand_landmarks_detector.tflite
- Embedded
TFLITE_METADATA: name HandLandmarkDetector · version 1 · author MediaPipe · packed associated files: handedness.txt
| Direction |
Tensor |
Shape |
Type |
Notes |
| Input |
input_1 |
[1, 224, 224, 3] |
float32 |
image — Input image to be detected. — Normalization: mean=[0.0, 0.0, 0.0], std=[255.0, 255.0, 255.0] |
| Output |
Identity |
[1, 63] |
float32 |
landmarks — The hand landmarks in normalized coordinates. |
| Output |
Identity_1 |
[1, 1] |
float32 |
presense score — The score of hand presence in the image. |
| Output |
Identity_2 |
[1, 1] |
float32 |
handedness — The score of left or right handedness. — Labels: handedness.txt |
| Output |
Identity_3 |
[1, 63] |
float32 |
world landmarks — The hand landmarks in world coordinates. |
Sub-model hand_gesture_recognizer.task -> gesture_embedder.tflite
| Direction |
Tensor |
Shape |
Type |
Notes |
| Input |
hand |
[1, 21, 3] |
float32 |
— |
| Input |
handedness |
[1, 1] |
float32 |
— |
| Input |
world_hand |
[1, 21, 3] |
float32 |
— |
| Output |
Identity |
[1, 128] |
float32 |
— |
Sub-model hand_gesture_recognizer.task -> canned_gesture_classifier.tflite
- Embedded
TFLITE_METADATA: name Gesture Classifier · version 1 · author MediaPipe · packed associated files: labels.txt
| Direction |
Tensor |
Shape |
Type |
Notes |
| Input |
hand_embedding |
[-1, 128] |
float32 |
gesture embedding — embedding feature vector from gesture embedder. |
| Output |
Identity |
[-1, 8] |
float32 |
hand gestures — hand gesture category scores — Labels: labels.txt |
Use with MediaPipe Tasks
Python Quickstart
pip install -U mediapipe huggingface_hub
import mediapipe as mp
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(
repo_id="litert-community/MediaPipe-GestureRecognizer",
filename="gesture_recognizer.task",
)
BaseOptions = mp.tasks.BaseOptions
GestureRecognizer = mp.tasks.vision.GestureRecognizer
GestureRecognizerOptions = mp.tasks.vision.GestureRecognizerOptions
VisionRunningMode = mp.tasks.vision.RunningMode
options = GestureRecognizerOptions(
base_options=BaseOptions(model_asset_path=model_path),
running_mode=VisionRunningMode.IMAGE,
num_hands=2,
)
with GestureRecognizer.create_from_options(options) as runner:
mp_image = mp.Image.create_from_file("input.jpg")
result = runner.recognize(mp_image)
for gesture in result.gestures:
top = gesture[0]
print(f"{top.category_name}: {top.score:.3f}")
Cross-Platform Guides & Samples
Latency Benchmarks
End-to-end MediaPipe Task pipeline average latency reported by the Gesture Recognizer documentation:
| Model Variant |
Device |
CPU Latency |
GPU Latency |
| GestureRecognizer |
Pixel 6 |
16.76 ms |
20.87 ms |
Intended Use, Limitations, and Fairness
Summarized from the bundled model_card_hand_gesture_classification_with_faireness_2022.pdf (2022):
- Intended applications: Real-time static single-hand gesture classification from 3D MediaPipe hand landmarks (no raw RGB pixels enter the classifier head) for mobile AR, entertainment, and touchless hand controls.
- Out-of-scope uses:
- Any form of surveillance or identity recognition is explicitly out of scope and not enabled by this technology.
- Human life-critical decisions.
- Multi-hand gestures (e.g., two-handed heart), dynamic/motion gestures (e.g., waving), or translating sign language.
- Fairness evaluation (summary): Measured with Specificity-Sensitivity F1 Score (SS F1):
- Perceived Monk Skin Tone (Tones 1–10, ≈2,000 images): Weighted average SS F1 95.5% (range 94.3%–97.8%).
- Perceived Gender Expression (≈8,000 images): Weighted average SS F1 93.9% (Male 94.4%, Female 93.3%, Unknown 93.1%).
Provenance
Every model binary in this repository is byte-identical (unmodified weights, metadata, and .task bundle archive bytes) to the official artifact served by Google from storage.googleapis.com/mediapipe-models.
| Repo File |
Upstream URL |
GCS Generation |
Upstream Last-Modified |
Pinned Bucket Version |
gesture_recognizer.task |
gesture_recognizer.task |
1682480004119009 |
Wed, 26 Apr 2023 03:33:24 GMT |
/1/ |
License and Citation
- License:
apache-2.0 — Distributed under Apache License, Version 2.0 alongside the MediaPipe release artifacts.
Citation
@article{zhang2020mediapipehands,
title = {MediaPipe Hands: On-device Real-time Hand Tracking},
author = {Zhang, Fan and Bazarevsky, Valentin and Vakunov, Andrey and Tkachenka, Andrei and Sung, George and Chang, Chuo-Ling and Grundmann, Matthias},
journal = {arXiv preprint arXiv:2006.10214},
year = {2020},
url = {https://arxiv.org/abs/2006.10214}
}