File size: 3,650 Bytes
7ec033d bfda283 e1c6e35 63b184e d6d7a0f e1c6e35 7ec033d e1c6e35 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | ---
license: apache-2.0
base_model: robbyant/lingbot-map
tags:
- coreml
- 3d-reconstruction
- camera-pose-estimation
- depth-estimation
- apple-silicon
- metal
- fp32
- fp16
pipeline_tag: depth-estimation
library_name: coremltools
---
# Apple CoreML LingBot-MAP Model Packages
[](https://opensource.org/licenses/Apache-2.0)
[](https://github.com/uqer1244/coreML_lingbot-map)
[](https://huggingface.co/uqer1244/coreML_lingbot-map)
This repository hosts converted Apple CoreML `.mlpackage` model bundles for **LingBot-MAP / Geometric Context Transformer (GCT)**, optimized for real-time 3D spatial mapping, camera pose estimation, and metric depth prediction on macOS and iOS (Apple Silicon M1/M2/M3/M4 & Apple Neural Engine).
---
## 🔗 Links & Resources
- **GitHub Repository**: [uqer1244/coreML_lingbot-map](https://github.com/uqer1244/coreML_lingbot-map)
- **Hugging Face Repository**: [uqer1244/coreML_lingbot-map](https://huggingface.co/uqer1244/coreML_lingbot-map)
- **MLX Repository**: [uqer1244/mlx_lingbot-map](https://github.com/uqer1244/mlx_lingbot-map)
- **Original Model Weights**: [robbyant/lingbot-map](https://huggingface.co/robbyant/lingbot-map)
---
## 📦 Hosted CoreML Model Packages
| Package Name | Input Spec | Output Spec | Description |
| :--- | :--- | :--- | :--- |
| **`Aggregator.mlpackage`** | `images`: `[1, 2, 3, 518, 378]` | `t0..t3`: `[1, 2, 1005, 2048]` | Feature trunk backbone for spatio-temporal embeddings |
| **`StatefulCameraHead.mlpackage`** | `t0..t3`, `k_cache_in`, `v_cache_in`, `frame_idx_in` | `pred_pose_enc`, `k_cache_out`, `v_cache_out`, `frame_idx_out` | Stateful KV Cache camera pose head (Rank-3 state tensors) |
| **`DepthHead.mlpackage`** | `t0..t3`, `images` | `depth`: `[1, 2, 518, 378, 1]`, `confidence` | Metric depth map ($518 \times 378$) & confidence estimator |
| **`Encoder.mlpackage`** | `images`: `[1, 2, 3, 518, 378]` | `tokens`: `[1, 2, 1005, 1024]` | ViT patch embedder (Integrated inside Aggregator) |
---
## 🛠️ Technical Specifications
- **Preprocessing**: Aspect-preserving portrait center-crop ($518 \times 378$) preserving $1:1$ physical pixel geometry.
- **State Management**: Rank-3 KV Cache state tensors ($256 \times 64 \times 128$) with dynamic frame index tracking.
- **Pose Transformation**: Closed-form SE3 matrix inversion ($W2C \rightarrow C2W$) and cumulative camera transformation ($C2W_k = C2W_{k-1} \times \Delta C2W_k$).
---
## 💻 Download & Usage Guide
### 1. Download via Hugging Face Hub (Python)
```bash
pip install huggingface_hub
```
```python
from huggingface_hub import snapshot_download
# Download all CoreML model packages to local models/ directory
snapshot_download(repo_id="uqer1244/coreML_lingbot-map", local_dir="./models")
```
### 2. Standalone Python MVP Runner
Clone the GitHub repository and run inference:
```bash
git clone https://github.com/uqer1244/coreML_lingbot-map.git
cd coreML_lingbot-map/LingBot-CoreML-MVP
pip install coremltools numpy opencv-python matplotlib viser
python3 run_coreml_mvp.py
```
### 3. Swift Native Xcode Integration
Add `LingBot-CoreML-Runtime` as a Swift Package Manager dependency in Xcode:
```swift
import LingBotCoreMLRuntime
let runtime = StreamingRuntime(config: .default)
try runtime.loadModels(from: modelDirectoryURL)
```
---
## 📄 License
This repository is licensed under the Apache 2.0 License.
|