---
license: apache-2.0
pipeline_tag: robotics
---

# APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies

This repository contains the checkpoints for APT, a two-stage training method for Vision-Language-Action (VLA) models that emphasizes Action Expert Pretraining to improve generalization to out-of-distribution (OOD) instructions.

* **Paper**: [APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies](https://huggingface.co/papers/2606.12366)
* **Project Page**: [https://xukechun.github.io/papers/APT/](https://xukechun.github.io/papers/APT/)
* **Code/Github**: [https://github.com/xukechun/APT](https://github.com/xukechun/APT)

## Method Overview

APT factorizes the VLA policy into a Vision-Action (VA) prior and a language-conditioned VLA likelihood. In Stage 1, the action expert is pretrained as a VA prior on vision-action pairs from a frozen VLM to bypass language imbalance. In Stage 2, language tokens are injected through a gated fusion mechanism that integrates VLM features while preserving the learned visuomotor prior.

## Sample Usage

For local inference, you can instantiate the planner directly:

```python
from apt.infer.planner import TrajPlanner

planner = TrajPlanner(
    ckpt_path="checkpoints/APT/ft_vla/ckpt_latest.pt",
    device="cuda:0",
    ensemble=4,
    use_ema=False,
)
planner.set_prompt("Pick up the grape and place it on the pink box.")
planner.add_obs_frame(obs_frame)
actions = planner.get_action()
```

## Citation

```bibtex
@article{xu2026apt,
      title={APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies},
      author={Xu, Kechun and Zhu, Zhenjie and Chen, Anzhe and Xiong, Rong and Wang, Yue},
      journal={arXiv preprint arXiv:2606.12366},
      year={2026}
    }
```