microsoft/MuseVLA

MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation

MuseVLA is an adaptive multimodal sensing Vision-Language-Action (VLA) model for robotic manipulation. Built on top of VITRA, MuseVLA treats novel sensors as on-demand tools: it first selects the modality needed for a task, grounds the selected sensor observation in the RGB image, and then generates robot actions. The model supports thermal, acoustic, and mmWave radar sensing in addition to RGB observations. MuseVLA achieves an average success rate of 80.6% across thermal-, audio-, and radar-guided manipulation tasks, as well as 66.7% average success on unseen sensor-guided tasks.

All our code and pre-trained model weights are licensed under the MIT license.

Please refer to our paper, code repository, and dataset for more details.

Model Summary

• Model type: Vision-Language-Action Model
• Sensor modalities: RGB, thermal, acoustic, and mmWave radar
• Language(s) (NLP): en
• License: MIT
• Training Dataset: MuseVLA-dataset
• Repository: https://github.com/microsoft/MuseVLA
• Paper: MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation

Citation

@misc{liu2026musevlaadaptivemultimodalsensing,
  title={MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation},
  author={Xingyuming Liu and Ruichun Ma and Heyu Guo and Qixiu Li and Qingwen Yang and Lin Luo and Shiqi Jiang and Chenren Xu and Jiaolong Yang and Baining Guo},
  year={2026},
  eprint={2606.17598},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2606.17598},
}