| | --- |
| | language: |
| | - en |
| | license: apache-2.0 |
| | library_name: transformers |
| | pipeline_tag: robotics |
| | --- |
| | |
| | # FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigation |
| |
|
| | **FantasyVLN** is a unified multimodal Chain-of-Thought (CoT) reasoning framework that enables efficient and precise navigation based on natural language instructions and visual observations. |
| |
|
| | - **Paper:** [FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigation](https://huggingface.co/papers/2601.13976) |
| | - **Project Page:** [https://fantasy-amap.github.io/fantasy-vln/](https://fantasy-amap.github.io/fantasy-vln/) |
| | - **Code:** [https://github.com/Fantasy-AMAP/fantasy-vln](https://github.com/Fantasy-AMAP/fantasy-vln) |
| |
|
| | ## Introduction |
| |
|
| | Achieving human-level performance in Vision-and-Language Navigation (VLN) requires an embodied agent to jointly understand multimodal instructions and visual-spatial context while reasoning over long action sequences. |
| |
|
| | FantasyVLN combines the benefits of textual, visual, and multimodal CoT reasoning by constructing a unified representation space across these reasoning modes. To enable efficient reasoning, we align these CoT reasoning modes with non-CoT reasoning during training, while using only non-CoT reasoning at test time. Notably, we perform visual CoT in the latent space of a [VAR](https://github.com/FoundationVision/VAR) model, where only low-scale latent representations are predicted. Compared to traditional pixel-level visual CoT methods, our approach significantly improves both training and inference efficiency, reducing inference latency by an order of magnitude compared to explicit CoT methods. |
| |
|
| | ## Citation |
| |
|
| | If you find this work helpful, please consider citing: |
| |
|
| | ```bibtex |
| | @article{zuo2025fantasyvln, |
| | title={FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigation}, |
| | author={Zuo, Jing and Mu, Lingzhou and Jiang, Fan and Ma, Chengcheng and Xu, Mu and Qi, Yonggang}, |
| | journal={arXiv preprint arXiv:2601.13976}, |
| | year={2025} |
| | } |
| | ``` |
