mlboydaisuke commited on
Commit
0cef45b
·
verified ·
1 Parent(s): 903e08e

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +36 -0
README.md ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: litert
4
+ pipeline_tag: keypoint-detection
5
+ tags: [keypoint-detection, image-matching, local-features, xfeat, litert, tflite, on-device, gpu]
6
+ ---
7
+
8
+ # XFeat (Accelerated Features) — LiteRT (CompiledModel GPU)
9
+
10
+ **XFeat** (Apache-2.0, ~1.5M, a lightweight pure-CNN local feature extractor for image matching —
11
+ SLAM / AR / image registration) re-authored to a **GPU-native** LiteRT `.tflite` via **litert_torch**.
12
+ FP16, **1.4 MB**, input **[1, 480, 640, 1]** NHWC normalized grayscale.
13
+
14
+ Verified on a Pixel 8a: full **LITERT_CL residency (72/72 nodes, 1 partition), ~0.4 ms**, GPU output
15
+ matches CPU/PyTorch (corr 0.9999).
16
+
17
+ ## I/O
18
+ - **Input** `[1, 480, 640, 1]` NHWC, grayscale, per-image **InstanceNorm** applied host-side
19
+ (`(g - mean)/sqrt(var+1e-5)` over the image).
20
+ - **Outputs** (all at H/8 × W/8 = 60×80): `feats` `[1,64,60,80]` dense descriptors; `keypoints`
21
+ `[1,65,60,80]` keypoint logits; `heatmap` `[1,1,60,80]` reliability. Keypoint NMS, descriptor
22
+ bilinear-sampling, and mutual-nearest-neighbor matching run host-side.
23
+
24
+ ## GPU-clean re-authoring
25
+ - Input gray + InstanceNorm moved **host-side** (its spatial reduction over H·W would overflow fp16
26
+ on the delegate).
27
+ - `_unfold2d(x, 8)` (space-to-depth via unfold → >4-D / GATHER_ND) → a one-hot `Conv2d(1,64,k=8,s=8)`
28
+ (exact, single CONV_2D). Result: zero GATHER/SELECT/TopK/Cast, no >4-D — full GPU residency.
29
+
30
+ ## Training data & PII
31
+ XFeat is trained on public correspondence data (MegaDepth + synthetic homographies). It outputs
32
+ geometric keypoints/descriptors only — no faces, identities, or personal attributes. Official
33
+ weights; only the op graph was re-authored for GPU.
34
+
35
+ ## Sample app + conversion script
36
+ https://github.com/google-ai-edge/litert-samples (compiled_model_api, two-image matching).