---
license: mit
language:
- en
pipeline_tag: robotics
library_name: transformers
tags:
- multimodal
- robotics
- vision-language-action
- univla
- action-decoder
- continuous-control
datasets:
- VLA-Arena/VLA_Arena_L0_L_rlds
---

# UniVLA - Action Decoder (Deployment Head)

## About VLA-Arena

**VLA-Arena** is a comprehensive benchmark designed to quantitatively understand the limits and failure modes of Vision-Language-Action (VLA) models. While VLAs are advancing towards generalist robot policies, measuring their true capability frontiers remains challenging. VLA-Arena addresses this by proposing a novel structured task design framework that quantifies difficulty across three orthogonal axes:

1.  **Task Structure**: 170+ tasks grouped into four key dimensions:
    * **Safety**: Operating reliably under strict constraints.
    * **Distractor**: Handling environmental unpredictability.
    * **Extrapolation**: Generalizing to unseen scenarios.
    * **Long Horizon**: Executing complex, multi-step tasks.
2.  **Language Command**: Variations in instruction complexity.
3.  **Visual Observation**: Perturbations in visual input.

Tasks are designed with hierarchical difficulty levels (L0-L2). In this benchmark setting, fine-tuning is typically performed on **L0** tasks to assess the model's ability to generalize to higher difficulty levels and strictly follow safety constraints.

## Model Overview

This model is the **Action Decoder Head** for **UniVLA**. Unlike the Latent Action Model (LAM) which is used for tokenizing video data, this decoder is a lightweight transformer module attached to the UniVLA backbone during deployment.

Its specific role is **Detokenization**: it takes the sequence of **Latent Action Tokens** (predicted by the VLM backbone) and **Visual Embeddings**, and decodes them into precise, continuous **Action Chunks** (7-DoF trajectories) executable by the robot.

---

## Model Architecture

The Action Decoder is designed to bridge the gap between the discrete latent space of the VLM and the continuous action space of the robot. It utilizes **Multi-Head Attention Pooling** to extract context-specific features from both latent actions and visual observations.

| Component | Description |
| :--- | :--- |
| **Input** | Latent Action Embeddings + Visual Embeddings (VLM Last Layer) |
| **Context Mechanism** | **Attention Pooling** (Visual tokens query Action tokens) |
| **Output** | **Action Chunks** (Sequence of continuous poses) |
| **Parameter Count** | **12.6M** (Lightweight Adapter) |

### Architecture Configuration
The decoder consists of attention pooling layers followed by projection MLPs. For real-world deployment, it also includes a proprioceptive projection layer.

| Parameter | Value |
| :--- | :--- |
| **Attention Heads** | 8 |
| **Head Dimension** | 64 |
| **Hidden Size** | 512 |
| **MLP Ratio** | 4 |
| **Proprioception Projection** | 2 Layers (Hidden Size 512) |

### Key Feature: Action Chunking
Unlike OpenVLA which predicts actions step-by-step, this decoder outputs **Action Chunks** (default size $N=12$ for real-world tasks). This allows for significantly smoother control and higher inference frequency ($\sim$10Hz).

---

## Training Details

### Dataset
This model was fine-tuned on the **[VLA-Arena/VLA_Arena_L0_L_rlds](https://huggingface.co/datasets/VLA-Arena/VLA_Arena_L0_L_rlds)** dataset.

### Training Strategy
This decoder is trained **end-to-end** with the UniVLA backbone (via LoRA). While the backbone learns to predict the correct *discrete* latent token, this decoder simultaneously learns to map that token to the correct *continuous* physical action.

| Parameter | Value |
| :--- | :--- |
| **Loss Function** | **L1 Loss** (Ground Truth vs. Predicted Action) |
| **Optimization** | Joint optimization with VLM Next-Token Prediction |
| **Visual Conditioning** | **Enabled** (Visual embeddings used as queries) |

---

## Evaluation & Usage

This model must be used in conjunction with the **UniVLA** backbone.
1.  **Backbone Phase**: The VLM predicts a sequence of discrete latent tokens (e.g., `<ACT_1>`, `<ACT_2>`).
2.  **Decoder Phase (This Model)**: These tokens, along with the visual context, are passed to this Action Decoder to generate the final $7\times N$ action vector (End-effector pose + Gripper).

Ablation studies show that this specific **Visual-Attention Decoder** outperforms standard auto-regressive decoding by **42.1%** on long-horizon tasks (LIBERO-Long), proving its efficacy in reducing ambiguity and improving precision.

For detailed evaluation instructions, metrics, and scripts, please refer to the [VLA-Arena repository](https://github.com/PKU-Alignment/VLA-Arena).