---
license: other
license_name: mediapipe-pytorch-license
license_link: https://github.com/zmurez/MediaPipePyTorch/blob/master/LICENSE
library_name: coremltools
pipeline_tag: object-detection
base_model: qualcomm/MediaPipe-Hand-Detection
tags:
  - coreml
  - apple-silicon
  - mediapipe
  - hand-detection
  - hand-landmarks
  - gesture-control
  - macos
---

# MediaPipe Hand Detection & Landmarks — Core ML

A ready-to-use two-model Core ML pipeline for real-time hand tracking on Apple Silicon, converted from Qualcomm AI Hub's MediaPipe Hand ONNX release.

The bundle contains both models required by the pipeline:

1. **Palm detector** — finds hands and seven detector keypoints.
2. **Hand landmark detector** — predicts presence, handedness, and 21 three-dimensional landmarks from each detected hand crop.

Together they enable gesture-controlled apps, games, plugins, and agents. Gesture recognition itself is implemented by interpreting the 21 landmarks; these models do not contain a separate gesture classifier. See [Hugging Mac](https://github.com/devilyouwei/hugging-mac) for the complete local macOS pipeline.

## Model details

| Component | Input | Outputs | Parameters |
|---|---|---|---:|
| `hand_detector.mlpackage` | `image`: `1 × 3 × 256 × 256` FP32 RGB | `box_coords`: `1 × 2944 × 18`; `box_scores`: `1 × 2944 × 1` | 1.76M |
| `hand_landmark_detector.mlpackage` | `image`: `1 × 3 × 256 × 256` FP32 RGB crop | `scores`: `1`; `lr`: `1`; `landmarks`: `1 × 21 × 3` | 2.01M |

- Precision: FP32
- Core ML deployment target: macOS 13 or later
- Combined package size: 15.3 MB
- The landmark model can be loaded lazily only when landmarks are requested.

## Core ML example

```python
from pathlib import Path

import coremltools as ct
import numpy as np
from huggingface_hub import snapshot_download
from PIL import Image, ImageOps

root = Path(snapshot_download(
    repo_id="hugging-mac/mediapipe-hand-coreml",
    allow_patterns=["coreml/**"],
)) / "coreml"

detector = ct.models.MLModel(
    root / "hand_detector.mlpackage",
    compute_units=ct.ComputeUnit.ALL,
)
image = ImageOps.pad(Image.open("hand.jpg").convert("RGB"), (256, 256))
value = np.asarray(image, dtype=np.float32) / 255.0
tensor = np.ascontiguousarray(value.transpose(2, 0, 1))[None]
outputs = detector.predict({"image": tensor})

print(outputs["box_coords"].shape)  # (1, 2944, 18)
print(outputs["box_scores"].shape)  # (1, 2944, 1)
```

Raw detector outputs require MediaPipe anchor decoding and weighted NMS. The resulting rotated hand crop is passed to the landmark model. Use the [Hugging Mac MediaPipe Hand SDK](https://github.com/devilyouwei/hugging-mac/tree/main/packages/hugging_mac_sdk/src/hugging_mac_sdk/models/mediapipe_hand_detection) for the complete pipeline, coordinate projection, and handedness handling.

## Conversion verification

The Core ML models were compared with ONNX Runtime on identical random inputs. Maximum absolute differences:

- Detector boxes: `6.866e-5`
- Detector scores: `1.526e-4`
- Hand presence: `3.92e-8`
- Handedness: `4.59e-6`
- Landmarks: `4.47e-7`

## Provenance and integrity

- Upstream: [Qualcomm MediaPipe Hand Detection](https://huggingface.co/qualcomm/MediaPipe-Hand-Detection)
- Upstream revision: `013e27b599e37c3b4c69439de15de53cc5b5708e`
- Qualcomm AI Hub release: `v0.60.0`, ONNX float
- `hand_detector.mlpackage` SHA-256: `a81143ef0bfa896d4206caf226697141ab8c052d57ba5e95fc7c7ac2660dc83d`
- `hand_landmark_detector.mlpackage` SHA-256: `d4049538d8fe4b4ffa77f71e1efa43705fc010e721c0456a95d9c97453c13716`
- Complete bundle SHA-256: `5432c988be7ed2fd6847097f823e0d813f97be7dc7a7d6587129386dbcd75929`

## License

The upstream repository declares a custom license and links to the MediaPipePyTorch license, which states that the implementation follows the Apache License 2.0 terms used by MediaPipe. Review the upstream Qualcomm distribution terms and linked source license before redistribution or commercial use. Hugging Mac is not affiliated with or endorsed by Qualcomm, Google, or MediaPipe.