---
license: mit
pipeline_tag: robotics
library_name: transformers
---

# Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Process

This repository contains the UD-VLA checkpoint for the CALVIN ABCD->D benchmark.

Vision-language-action (VLA) models aim to understand natural language instructions and visual observations and to execute corresponding actions as an embodied agent. UD-VLA addresses limitations in prior VLA models by optimizing generation and action jointly through a synchronous denoising process, where iterative refinement enables actions to evolve from initialization under constant visual guidance. This approach, grounded in its proposed Joint Discrete Denoising Diffusion Process (JD3P), integrates multiple modalities into a single denoising trajectory, enabling understanding, generation, and acting to be intrinsically synergistic.

-   **Paper**: [Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Process](https://arxiv.org/abs/2511.01718)
-   **Project page**: [https://irpn-eai.github.io/UD-VLA.github.io](https://irpn-eai.github.io/UD-VLA.github.io)
-   **Code**: [https://github.com/OpenHelix-Team/UD-VLA](https://github.com/OpenHelix-Team/UD-VLA)

## Installation and Evaluation Example

To get started with UD-VLA and run an evaluation, follow these steps directly from the GitHub repository:

### 1. Install Base environment.

```bash
# Create and activate conda environment
conda create -n udvla-calvin python=3.10 -y
conda activate udvla-calvin
# Clone and install the openvla repo
git clone https://github.com/OpenHelix-Team/UD-VLA.git
cd UD-VLA
pip install -r requirements.txt
```

### 2. Install Calvin environment (for evaluation).

This setup is only for evaluation. The following steps are required to set up the environment:

```shell
# Install dependencies
cd reference/RoboVLMs

# This will install the required environment and download the calvin dataset.
bash scripts/setup_calvin.sh

# Only for rendering environment.
bash scripts/setup_calvin_vla.sh

# Check if the environment is set up correctly
python eval/calvin/env_test.py
```

### 3. Run Model Evaluation

You can also download the pre-trained checkpoint fituned on CALVIN ABCD→D at [UD-VLA_CALVIN](https://huggingface.co/chenpyyy/UD-VLA_CALVIN-ABCD).

```shell
cd reference/RoboVLMs

# 4 GPUs inference, we set diffusion step to 72.
bash scripts/run_eval_calvin_univla_i2ia_dis.sh

# The above command will generate 4 results in the `results` folder, calculate the final average score
python tools/evaluation/calvin_score.py
```

## Experiment Result
### Performance on CALVIN ABCD→D Benchmark.
*UniVLA** denotes the variant without historical frames for fair comparison. We evaluate 500 rollouts for our model, where each rollout involves a sequence of 5 consecutive sub-tasks.
<div align="center">

<table>
  <thead>
    <tr>
      <th>Method</th><th>Task</th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>Avg. Len ↑</th>
    </tr>
  </thead>
  <tbody>
    <tr><td>MCIL</td><td>ABCD→D</td><td>0.373</td><td>0.027</td><td>0.002</td><td>0.000</td><td>0.000</td><td>0.40</td></tr>
    <tr><td>RT-1</td><td>ABCD→D</td><td>0.844</td><td>0.617</td><td>0.438</td><td>0.323</td><td>0.227</td><td>2.45</td></tr>
    <tr><td>Robo-Flamingo</td><td>ABCD→D</td><td>0.964</td><td>0.896</td><td>0.824</td><td>0.740</td><td>0.660</td><td>4.09</td></tr>
    <tr><td>GR-1</td><td>ABCD→D</td><td>0.949</td><td>0.896</td><td>0.844</td><td>0.789</td><td>0.731</td><td>4.21</td></tr>
    <tr><td>ReconVLA</td><td>ABCD→D</td><td>0.980</td><td>0.900</td><td>0.845</td><td>0.785</td><td>0.705</td><td>4.23</td></tr>
    <tr><td>UniVLA*</td><td>ABCD→D</td><td>0.948</td><td>0.906</td><td>0.862</td><td>0.834</td><td>0.690</td><td>4.24</td></tr>
    <tr><td>UP-VLA</td><td>ABCD→D</td><td>0.962</td><td>0.921</td><td>0.879</td><td>0.842</td><td>0.812</td><td>4.42</td></tr>
    <tr><td><b>UD-VLA (ours)</b></td><td>ABCD→D</td><td><b>0.992</b></td><td><b>0.968</b></td><td><b>0.936</b></td><td><b>0.904</b></td><td><b>0.840</b></td><td><b>4.64</b></td></tr>
  </tbody>
</table>
</div>

### Performance on Real-world.
Our real-world setup consists of a 6-DoF UR5e robotic arm equipped with a 6-DoF Inspire RH56E2 robotic hand for dexterous manipulation. 
![](real-world.png)

## Other Simulation Benchmark Setup
- [LIBERO](docs/libero.md)
- [SimplerEnv](docs/simpler.md)

## ❤️ Acknowledgment

We thank [UniVLA](https://github.com/baaivision/UniVLA), [Emu3](https://github.com/baaivision/Emu3), [RobotVLM](https://github.com/Robot-VLAs/RoboVLMs) and [Show-o](https://github.com/showlab/Show-o) for their open-sourced work!

We thank [Yuqi Wang](https://github.com/Robertwyq) and [Zhide zhong](https://scholar.google.com/citations?user=msy4tL4AAAAJ&hl=zh-CN) for their guidance about experiment!

## 📖 Citation
If you find UD-VLA useful, please consider citing our work🤗:

```bibtex
@article{udvla2025,
title={Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Process},
author={Jiayi Chen, Wenxuan Song, Pengxiang Ding, Ziyang Zhou, Han Zhao, Feilong Tang, Donglin Wang, Haoang Li},
year={2025},
journal={arXiv preprint arXiv:2511.01718}
}
```