Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,59 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: cc-by-4.0
|
| 3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-4.0
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
# FAINT
|
| 6 |
+
|
| 7 |
+
Fast, Appearance-Invariant Navigation Transformer (FAINT) is a learned policy for vision-based topological navigation.
|
| 8 |
+
|
| 9 |
+
Please see the [Project Page]() for more information.
|
| 10 |
+
|
| 11 |
+
## Model Details
|
| 12 |
+
|
| 13 |
+
The `FAINT-Sim` model uses [`Theia-Tiny-CDDSV`](https://theia.theaiinstitute.com/) as backbone, and was trained for 10 rounds of DAgger with ~12M samples from the Habitat simulator.
|
| 14 |
+
It is capable of zero-shot transfer for navigation with real robots.
|
| 15 |
+
|
| 16 |
+
This repo contains two versions of the trained model weights.
|
| 17 |
+
- `model_pytorch.pt`: Weights-only state dict of the Pytorch model.
|
| 18 |
+
- `model_torchscript.pt`: A 'standalone' Torchscript model for deployment.
|
| 19 |
+
|
| 20 |
+
## Usage
|
| 21 |
+
|
| 22 |
+
See the main Github [repo]() for details, input preprocessing etc.
|
| 23 |
+
|
| 24 |
+
### Torchscript
|
| 25 |
+
|
| 26 |
+
Only dependency is Pytorch.
|
| 27 |
+
|
| 28 |
+
```bash
|
| 29 |
+
import torch
|
| 30 |
+
ckpt_path = 'FAINT-Sim/model_torchscript.pt'
|
| 31 |
+
model = torch.jit.load(ckpt_path)
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
+
### Pytorch
|
| 35 |
+
|
| 36 |
+
Need to have the Faint library installed.
|
| 37 |
+
|
| 38 |
+
```bash
|
| 39 |
+
import torch
|
| 40 |
+
from faint.common.models.faint import FAINT
|
| 41 |
+
|
| 42 |
+
ckpt_path = 'FAINT-Sim/model_pytorch.pt'
|
| 43 |
+
state_dict = torch.load(ckpt_path)
|
| 44 |
+
|
| 45 |
+
model = FAINT() # The weights in this repo correspond to FAINT initialized with the default arguments
|
| 46 |
+
model.load_state_dict(state_dict)
|
| 47 |
+
```
|
| 48 |
+
|
| 49 |
+
## Citation
|
| 50 |
+
|
| 51 |
+
If you use FAINT in your research, please use the following BibTeX entry:
|
| 52 |
+
```bibtex
|
| 53 |
+
@article{Suomela_2025_Outperforming,
|
| 54 |
+
title={Outperforming Real-World Data for Training Vision-Based Navigation Policies},
|
| 55 |
+
author={Suomela, Lauri and Kuruppu Arachchige, Sasanka and Torres, German F. and Edelman, Harry and Kämäräinen, Joni-Kristian}
|
| 56 |
+
journal={arXiv:},
|
| 57 |
+
year={2025}
|
| 58 |
+
}
|
| 59 |
+
```
|