File size: 2,051 Bytes
4f95937 c8e38a1 4f95937 63234c4 c8e38a1 63234c4 c8e38a1 63234c4 c8e38a1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
---
language:
- en
license: apache-2.0
library_name: transformers
pipeline_tag: robotics
---
# FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigation
**FantasyVLN** is a unified multimodal Chain-of-Thought (CoT) reasoning framework that enables efficient and precise navigation based on natural language instructions and visual observations.
- **Paper:** [FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigation](https://huggingface.co/papers/2601.13976)
- **Project Page:** [https://fantasy-amap.github.io/fantasy-vln/](https://fantasy-amap.github.io/fantasy-vln/)
- **Code:** [https://github.com/Fantasy-AMAP/fantasy-vln](https://github.com/Fantasy-AMAP/fantasy-vln)
## Introduction
Achieving human-level performance in Vision-and-Language Navigation (VLN) requires an embodied agent to jointly understand multimodal instructions and visual-spatial context while reasoning over long action sequences.
FantasyVLN combines the benefits of textual, visual, and multimodal CoT reasoning by constructing a unified representation space across these reasoning modes. To enable efficient reasoning, we align these CoT reasoning modes with non-CoT reasoning during training, while using only non-CoT reasoning at test time. Notably, we perform visual CoT in the latent space of a [VAR](https://github.com/FoundationVision/VAR) model, where only low-scale latent representations are predicted. Compared to traditional pixel-level visual CoT methods, our approach significantly improves both training and inference efficiency, reducing inference latency by an order of magnitude compared to explicit CoT methods.
## Citation
If you find this work helpful, please consider citing:
```bibtex
@article{zuo2025fantasyvln,
title={FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigation},
author={Zuo, Jing and Mu, Lingzhou and Jiang, Fan and Ma, Chengcheng and Xu, Mu and Qi, Yonggang},
journal={arXiv preprint arXiv:2601.13976},
year={2025}
}
``` |