edgarriba commited on
Commit
bd880c6
·
verified ·
1 Parent(s): 3b0233c

docs: add the RaCo-ALIKED ONNX/TensorRT matcher exports

Browse files
Files changed (1) hide show
  1. README.md +42 -0
README.md CHANGED
@@ -41,3 +41,45 @@ fraction of the latency. ICCV 2023.
41
  year = {2023}
42
  }
43
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
  year = {2023}
42
  }
43
  ```
44
+
45
+ ## ONNX / TensorRT exports for vision-rt
46
+
47
+ Alongside the PyTorch checkpoints above, this repo holds the **LightGlue+ matcher for
48
+ RaCo-ALIKED features** as a standalone ONNX graph plus prebuilt TensorRT engines, used by
49
+ [`vision-rt`](https://github.com/kornia/vision-rt)'s `vrt-lightglue` crate. These are the
50
+ ONNX/TensorRT form of `raco_aliked_lightglue.pth` — not a new set of weights.
51
+
52
+ ```
53
+ normalized_keypoints (2P,1,K,2) f32 long-edge normalised
54
+ descriptors (2P,1,K,128) f32 L2-normalised
55
+ -> matches0 (P,K) i32 index into image 1, or -1 if unmatched
56
+ -> mscores0 (P,K) f32 match confidence in [0,1]
57
+ ```
58
+
59
+ The matching extractor half lives in
60
+ [`kornia/raco-aliked`](https://huggingface.co/kornia/raco-aliked); both halves must come
61
+ from the same `kN` export, since `K` is baked into each.
62
+
63
+ **Why a split graph.** Upstream
64
+ ([fabio-sim/LightGlue-ONNX](https://github.com/fabio-sim/LightGlue-ONNX)) publishes only a
65
+ fused extractor+matcher graph, which can match only the two images handed to it in a
66
+ single forward pass. Splitting the matcher out means descriptors extracted at *any* time —
67
+ from a map, a relocalization database, a keyframe store — can be matched against a live
68
+ frame. Cutting upstream of LightGlue's `NonZero` compaction also removes the graph's only
69
+ data-dependent shapes and its int64 output.
70
+
71
+ **Matching cost is O(K²)**, so unlike the extractor it gets rapidly worse with K. On a
72
+ Jetson Orin Nano (MAXN_SUPER, TRT 10.3.0.30, fp16, one pair): **7.9 ms** at k512,
73
+ **21.6 ms** at k1024, **126.5 ms** at k3072. Extraction moves the other way (the RaCo
74
+ ranker is bypassed at K≥3072), so pick k512 if you match every frame and k3072 if
75
+ extraction dominates.
76
+
77
+ **Engines are machine-locked** to the exact TensorRT version and GPU architecture they were
78
+ built for (`trt10.3.0.30`, `sm87` — JetPack 6 on Orin), and to the shape profile they were
79
+ built at. Elsewhere, build from the ONNX; `vrt-hub` does it automatically.
80
+
81
+ **Licences.** The ONNX/engine files are derived works combining LightGlue (Apache-2.0),
82
+ RaCo (Apache-2.0) and ALIKED (**BSD-3-Clause**) weights. BSD-3-Clause requires its
83
+ attribution to be reproduced in redistributions in binary form, which these files are — see
84
+ [`LICENSE-NOTICE.md`](LICENSE-NOTICE.md). That notice is absent from the upstream export
85
+ repo and is reproduced deliberately.