Add MediaPipe Hand detector and landmark Core ML packages

README.md

@@ -0,0 +1,92 @@
+---
+license: other
+license_name: mediapipe-pytorch-license
+license_link: https://github.com/zmurez/MediaPipePyTorch/blob/master/LICENSE
+library_name: coremltools
+pipeline_tag: object-detection
+base_model: qualcomm/MediaPipe-Hand-Detection
+tags:
+  - coreml
+  - apple-silicon
+  - mediapipe
+  - hand-detection
+  - hand-landmarks
+  - gesture-control
+  - macos
+---
+
+# MediaPipe Hand Detection & Landmarks — Core ML
+
+A ready-to-use two-model Core ML pipeline for real-time hand tracking on Apple Silicon, converted from Qualcomm AI Hub's MediaPipe Hand ONNX release.
+
+The bundle contains both models required by the pipeline:
+
+1. **Palm detector** — finds hands and seven detector keypoints.
+2. **Hand landmark detector** — predicts presence, handedness, and 21 three-dimensional landmarks from each detected hand crop.
+
+Together they enable gesture-controlled apps, games, plugins, and agents. Gesture recognition itself is implemented by interpreting the 21 landmarks; these models do not contain a separate gesture classifier. See [Hugging Mac](https://github.com/devilyouwei/hugging-mac) for the complete local macOS pipeline.
+
+## Model details
+
+| Component | Input | Outputs | Parameters |
+|---|---|---|---:|
+| `hand_detector.mlpackage` | `image`: `1 × 3 × 256 × 256` FP32 RGB | `box_coords`: `1 × 2944 × 18`; `box_scores`: `1 × 2944 × 1` | 1.76M |
+| `hand_landmark_detector.mlpackage` | `image`: `1 × 3 × 256 × 256` FP32 RGB crop | `scores`: `1`; `lr`: `1`; `landmarks`: `1 × 21 × 3` | 2.01M |
+
+- Precision: FP32
+- Core ML deployment target: macOS 13 or later
+- Combined package size: 15.3 MB
+- The landmark model can be loaded lazily only when landmarks are requested.
+
+## Core ML example
+
+```python
+from pathlib import Path
+
+import coremltools as ct
+import numpy as np
+from huggingface_hub import snapshot_download
+from PIL import Image, ImageOps
+
+root = Path(snapshot_download(
+    repo_id="hugging-mac/mediapipe-hand-coreml",
+    allow_patterns=["coreml/**"],
+)) / "coreml"
+
+detector = ct.models.MLModel(
+    root / "hand_detector.mlpackage",
+    compute_units=ct.ComputeUnit.ALL,
+)
+image = ImageOps.pad(Image.open("hand.jpg").convert("RGB"), (256, 256))
+value = np.asarray(image, dtype=np.float32) / 255.0
+tensor = np.ascontiguousarray(value.transpose(2, 0, 1))[None]
+outputs = detector.predict({"image": tensor})
+
+print(outputs["box_coords"].shape)  # (1, 2944, 18)
+print(outputs["box_scores"].shape)  # (1, 2944, 1)
+```
+
+Raw detector outputs require MediaPipe anchor decoding and weighted NMS. The resulting rotated hand crop is passed to the landmark model. Use the [Hugging Mac MediaPipe Hand SDK](https://github.com/devilyouwei/hugging-mac/tree/main/packages/hugging_mac_sdk/src/hugging_mac_sdk/models/mediapipe_hand_detection) for the complete pipeline, coordinate projection, and handedness handling.
+
+## Conversion verification
+
+The Core ML models were compared with ONNX Runtime on identical random inputs. Maximum absolute differences:
+
+- Detector boxes: `6.866e-5`
+- Detector scores: `1.526e-4`
+- Hand presence: `3.92e-8`
+- Handedness: `4.59e-6`
+- Landmarks: `4.47e-7`
+
+## Provenance and integrity
+
+- Upstream: [Qualcomm MediaPipe Hand Detection](https://huggingface.co/qualcomm/MediaPipe-Hand-Detection)
+- Upstream revision: `013e27b599e37c3b4c69439de15de53cc5b5708e`
+- Qualcomm AI Hub release: `v0.60.0`, ONNX float
+- `hand_detector.mlpackage` SHA-256: `a81143ef0bfa896d4206caf226697141ab8c052d57ba5e95fc7c7ac2660dc83d`
+- `hand_landmark_detector.mlpackage` SHA-256: `d4049538d8fe4b4ffa77f71e1efa43705fc010e721c0456a95d9c97453c13716`
+- Complete bundle SHA-256: `5432c988be7ed2fd6847097f823e0d813f97be7dc7a7d6587129386dbcd75929`
+
+## License
+
+The upstream repository declares a custom license and links to the MediaPipePyTorch license, which states that the implementation follows the Apache License 2.0 terms used by MediaPipe. Review the upstream Qualcomm distribution terms and linked source license before redistribution or commercial use. Hugging Mac is not affiliated with or endorsed by Qualcomm, Google, or MediaPipe.

coreml/hand_detector.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de163c1bee3e5845eca334483b77d63adda4539fc35190c5b9146a6f3bb73cff
+size 130040

coreml/hand_detector.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94ae575261d4d6d47829b38d5167e3ca29e15fd85e14434adbf629e49066809b
+size 7038336

coreml/hand_detector.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "11300D9C-7B69-44DA-BE56-E880BAA4E2AF": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "D5A6C0DB-5D82-4498-BBC8-F42D3AD0A070": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "D5A6C0DB-5D82-4498-BBC8-F42D3AD0A070"
+}

coreml/hand_landmark_detector.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0897d8c4085735b89a3aabada2e4ae73c9a04968f6d1fc30b8789f389af5f6ca
+size 135488

coreml/hand_landmark_detector.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3b0a59b2cd25b3d8431a49282159f831acb5b749de0eec7542d50bcf85c909a
+size 8035776

coreml/hand_landmark_detector.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "92AA890C-BF1E-406F-99AD-CEBD001476F3": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "CAE6CE42-BE83-49AC-AD79-CA03D33F641B": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "CAE6CE42-BE83-49AC-AD79-CA03D33F641B"
+}

mediapipe-hand-coreml.tar

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b80701c800ee166ef609e38d10761e3ac958a9637e41f8a6c961ebab098882e7
+size 15350784