watzon/wormhole-mediapipe-coreml

Wormhole Converted Core ML Models

This repository contains Core ML artifacts generated for wormhole from publicly available MediaPipe TensorFlow Lite models.

These artifacts are intended for:

• local development of the wormhole perception runtime,
• benchmarking model-backed tracking against the legacy heuristic path,
• staging Core ML packages and compiled model bundles for Apple-platform testing.

Provenance

The source models come from MediaPipe's public model distribution and are fetched from the mediapipe-assets bucket referenced by MediaPipe's legacy solution docs.

Examples:

• hand_landmark_full.tflite
• hand_landmark_lite.tflite
• palm_detection_full.tflite
• palm_detection_lite.tflite
• face_landmark.tflite
• face_detection_short_range.tflite
• face_detection_full_range.tflite
• iris_landmark.tflite

The conversion pipeline currently uses:

1. tflite2tensorflow
2. coremltools
3. xcrun coremlcompiler

with compatibility patches required for the current local toolchain.

Artifact Layout

• coreml/: .mlpackage bundles suitable for Core ML inspection and distribution
• compiled/: .mlmodelc bundles compiled for Apple runtime loading

wormhole prefers the compiled .mlmodelc form at runtime.

Included Models

The current staged set is:

• hand_landmark_full
• hand_landmark_lite
• palm_detection_full
• palm_detection_lite
• face_landmark
• face_detection_short_range
• face_detection_full_range
• iris_landmark

Intended Runtime Use

These models are being evaluated as replacements for the current heuristic tracking path in wormhole.

Current mapping:

• hand tracking: palm detector + hand landmark model
• face tracking: face detector + face landmark model
• iris tracking: iris landmark model

Known Gaps

The following public MediaPipe models are not currently included because the present conversion path does not handle them successfully:

• face_landmark_with_attention reason: unresolved custom op Landmarks2TransformMatrix
• face_detection_full_range_sparse reason: current tflite2tensorflow path fails during sparse/densify handling

Notes

• These artifacts are generated outputs, not hand-authored model implementations.
• The binaries are meant to be reproducible from the original MediaPipe assets using the scripts in the main wormhole repository.
• Licensing and redistribution should be reviewed against the upstream MediaPipe assets and their associated model cards before wider distribution.