RaCo-ALIKED — extractor half (ONNX + TensorRT engines)

Keypoint detection (RaCo) + 128-D descriptors (ALIKED), as a standalone ONNX graph, for vision-rt's vrt-raco-aliked crate.

images (B,3,H,W) f32   H,W multiples of 32, RGB in [0,1]
  -> keypoints            (B,K,2)   f32   model-resolution pixels (x,y)
  -> normalized_keypoints (B,K,2)   f32   long-edge normalised, matcher input
  -> descriptors          (B,K,128) f32   already L2-normalised

The matcher half lives in kornia/lightglue-onnx.

Why these files exist

Upstream (fabio-sim/LightGlue-ONNX) publishes only a fused extractor+matcher graph. That graph emits int64, has data-dependent output shapes, and never exposes descriptors — so it cannot feed a descriptor bank or a relocalization database, and a fused forward can only ever match the two images handed to it.

All three problems share one root: LightGlue computes per-query matches and only then compacts them with a single NonZero. That NonZero is the only TensorRT-hostile op in the whole graph. Cutting upstream of it yields fully static shapes; cutting again at the extractor/matcher seam yields two independently reusable halves. These files are that cut, produced by split_raco_pipeline.py.

The split is verified against the fused model under onnxruntime — keypoints, match set and scores all identical.

K is baked in, and changes the graph

K is fixed at export. It is not just a keypoint budget: at K ≥ 3072 RaCo's learned ranker is omitted entirely, which roughly halves extraction cost while returning 3× the keypoints. Measured on a Jetson Orin Nano (MAXN_SUPER, TRT 10.3.0.30, fp16, 640²):

K	ranker	extract / image
512	dense	49.1 ms
1024	boundary	55.2 ms
3072	bypass	28.5 ms

Extraction is non-monotonic in K. The trade is accuracy under rotation: inliers against a ground-truth affine drop from ~98% (k1024) to ~93% (k3072) at 90°, but k3072 returns ~2.6× more matches, so it yields more correct correspondences in absolute terms. Note the matcher scales the other way (O(K²)) — see the matcher repo.

TensorRT engines

Engines are machine-locked: they only load on the exact TensorRT version and GPU architecture they were built for (here trt10.3.0.30, sm87 — JetPack 6 on Orin). On any other machine, build from the ONNX instead; vrt-hub does this automatically.

Each engine here is built at the shape profile RaCoAliked::engine_profile() declares (min 1x3x256x256, opt 2x3x512x512, max 2x3x640x640). That matters: vrt-hub selects a prebuilt on TensorRT version, SM and precision alone and does not check the shape profile, so an engine built at a different profile would be served and then reject frames it cannot handle.

The released graph targets TRT 10.16 / CUDA 13, whose ONNX parser constant-folds aggressively. TRT 10.3 does not, and rejects two torch-dynamo constructs (Conv with a computed zero bias, Reduce* with computed axes). The split script bakes both into initializers — without which the unmodified fused model also fails to parse on TRT 10.3.

Licences and model credit

These artifacts are derived from three separately licensed upstreams and carry all three obligations. See LICENSE-NOTICE.md.

Component	Source	Licence
RaCo detector + ranker	cvg/RaCo	Apache-2.0
ALIKED descriptors	Shiaoming/ALIKED	BSD-3-Clause
ONNX export tooling	fabio-sim/LightGlue-ONNX	Apache-2.0

All model credit belongs to the original authors:

• RaCo — Shenoi, Lindenberger, Sarlin, Pollefeys, "RaCo: Ranking and Covariance for Practical Learned Keypoints", 3DV 2026, arXiv:2602.15755.
• ALIKED — Zhao et al., "ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation", IEEE TIM 2023.

The TensorRT graph optimizations baked into these files (hierarchical chunked TopK, BatchNorm folding, native SELU + logit-space NMS, ranker boundary-reranking, the DeformConv→GridSample rewrite, and the integer-floor-div fp16 fix) are fabio-sim's work and are preserved verbatim by the split.