Add model card and robotics pipeline tag
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,3 +1,54 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: mit
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
pipeline_tag: robotics
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# LoTIS: Learning to Localize Reference Trajectories in Image-Space for Visual Navigation
|
| 7 |
+
|
| 8 |
+
[🤖 Demo](https://huggingface.co/spaces/fnnBsch/lotis-demo) | [📝 Paper](https://huggingface.co/papers/2602.18803) | [🏠 Project Page](https://finnbusch.com/lotis) | [💻 GitHub](https://github.com/KTH-RPL/lotis)
|
| 9 |
+
|
| 10 |
+
LoTIS is a model for visual navigation that provides robot-agnostic image-space guidance by localizing a reference RGB trajectory in the robot's current view. Given a reference trajectory (a sequence of RGB images) and a query image from the robot's current viewpoint, LoTIS predicts the 2D image-space coordinates, visibility, and relative distance of each trajectory pose as it would appear in the query view.
|
| 11 |
+
|
| 12 |
+
## Python API Usage
|
| 13 |
+
|
| 14 |
+
To use the model programmatically, first clone the [GitHub repository](https://github.com/KTH-RPL/lotis) and follow the installation instructions to setup the environment and download the required DINOv3 backbone weights.
|
| 15 |
+
|
| 16 |
+
```python
|
| 17 |
+
from lotis import TrajectoryLocalizer
|
| 18 |
+
|
| 19 |
+
localizer = TrajectoryLocalizer.from_checkpoint(
|
| 20 |
+
checkpoint_path="final_model.pth",
|
| 21 |
+
config_path="final_config.yaml",
|
| 22 |
+
dinov3_weights="/path/to/dinov3_vitb16_pretrain.pth",
|
| 23 |
+
)
|
| 24 |
+
|
| 25 |
+
# Encode a trajectory — do this once and reuse
|
| 26 |
+
encoding = localizer.encode_trajectory("path/to/trajectory.mp4")
|
| 27 |
+
|
| 28 |
+
# Localize a query image
|
| 29 |
+
result = localizer.localize("query.jpg", encoding)
|
| 30 |
+
|
| 31 |
+
print(f"Closest trajectory frame: {result.closest_frame()}")
|
| 32 |
+
print(f"Visible frames: {result.visible_indices()}")
|
| 33 |
+
|
| 34 |
+
# Save and reload the trajectory encoding
|
| 35 |
+
import torch
|
| 36 |
+
torch.save(encoding.to_dict(), "encoding.pt")
|
| 37 |
+
encoding = TrajectoryEncoding.from_dict(torch.load("encoding.pt"))
|
| 38 |
+
```
|
| 39 |
+
|
| 40 |
+
## Citation
|
| 41 |
+
|
| 42 |
+
If you find LoTIS useful in your research, please cite our work:
|
| 43 |
+
|
| 44 |
+
```bibtex
|
| 45 |
+
@misc{busch2026learninglocalizereferencetrajectories,
|
| 46 |
+
title={Learning to Localize Reference Trajectories in Image-Space for Visual Navigation},
|
| 47 |
+
author={Finn Lukas Busch and Matti Vahs and Quantao Yang and Jesús Gerardo Ortega Peimbert and Yixi Cai and Jana Tumova and Olov Andersson},
|
| 48 |
+
year={2026},
|
| 49 |
+
eprint={2602.18803},
|
| 50 |
+
archivePrefix={arXiv},
|
| 51 |
+
primaryClass={cs.RO},
|
| 52 |
+
url={https://arxiv.org/abs/2602.18803},
|
| 53 |
+
}
|
| 54 |
+
```
|