Instructions to use mlboydaisuke/xfeat-litert with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use mlboydaisuke/xfeat-litert with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: litert
|
| 4 |
+
pipeline_tag: keypoint-detection
|
| 5 |
+
tags: [keypoint-detection, image-matching, local-features, xfeat, litert, tflite, on-device, gpu]
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# XFeat (Accelerated Features) — LiteRT (CompiledModel GPU)
|
| 9 |
+
|
| 10 |
+
**XFeat** (Apache-2.0, ~1.5M, a lightweight pure-CNN local feature extractor for image matching —
|
| 11 |
+
SLAM / AR / image registration) re-authored to a **GPU-native** LiteRT `.tflite` via **litert_torch**.
|
| 12 |
+
FP16, **1.4 MB**, input **[1, 480, 640, 1]** NHWC normalized grayscale.
|
| 13 |
+
|
| 14 |
+
Verified on a Pixel 8a: full **LITERT_CL residency (72/72 nodes, 1 partition), ~0.4 ms**, GPU output
|
| 15 |
+
matches CPU/PyTorch (corr 0.9999).
|
| 16 |
+
|
| 17 |
+
## I/O
|
| 18 |
+
- **Input** `[1, 480, 640, 1]` NHWC, grayscale, per-image **InstanceNorm** applied host-side
|
| 19 |
+
(`(g - mean)/sqrt(var+1e-5)` over the image).
|
| 20 |
+
- **Outputs** (all at H/8 × W/8 = 60×80): `feats` `[1,64,60,80]` dense descriptors; `keypoints`
|
| 21 |
+
`[1,65,60,80]` keypoint logits; `heatmap` `[1,1,60,80]` reliability. Keypoint NMS, descriptor
|
| 22 |
+
bilinear-sampling, and mutual-nearest-neighbor matching run host-side.
|
| 23 |
+
|
| 24 |
+
## GPU-clean re-authoring
|
| 25 |
+
- Input gray + InstanceNorm moved **host-side** (its spatial reduction over H·W would overflow fp16
|
| 26 |
+
on the delegate).
|
| 27 |
+
- `_unfold2d(x, 8)` (space-to-depth via unfold → >4-D / GATHER_ND) → a one-hot `Conv2d(1,64,k=8,s=8)`
|
| 28 |
+
(exact, single CONV_2D). Result: zero GATHER/SELECT/TopK/Cast, no >4-D — full GPU residency.
|
| 29 |
+
|
| 30 |
+
## Training data & PII
|
| 31 |
+
XFeat is trained on public correspondence data (MegaDepth + synthetic homographies). It outputs
|
| 32 |
+
geometric keypoints/descriptors only — no faces, identities, or personal attributes. Official
|
| 33 |
+
weights; only the op graph was re-authored for GPU.
|
| 34 |
+
|
| 35 |
+
## Sample app + conversion script
|
| 36 |
+
https://github.com/google-ai-edge/litert-samples (compiled_model_api, two-image matching).
|