Add model card for AnyPos
#1
by nielsr HF Staff - opened
README.md
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: robotics
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
# AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation
|
| 6 |
+
|
| 7 |
+
AnyPos (also referred to as **Vidar** in the codebase) is a unified pipeline that integrates large-scale automated task-agnostic exploration with robust embodiment modeling through inverse dynamics learning. By focusing on exploring all feasible actions of the embodiment, AnyPos learns embodiment dynamics directly from task-agnostic action data and decouples them from high-level policy learning.
|
| 8 |
+
|
| 9 |
+
- **Paper:** [AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation](https://huggingface.co/papers/2507.12768)
|
| 10 |
+
- **Project Page:** [https://embodiedfoundation.github.io/vidar_anypos](https://embodiedfoundation.github.io/vidar_anypos)
|
| 11 |
+
- **Repository:** [https://github.com/yaofeng1998/Vidar](https://github.com/yaofeng1998/Vidar)
|
| 12 |
+
|
| 13 |
+
## Introduction
|
| 14 |
+
|
| 15 |
+
AnyPos generates diverse yet safe trajectories at scale, then learns embodiment representations by decoupling arm and end-effector motions. It employs a direction-aware decoder to stabilize predictions under distribution shift, allowing it to be coupled with diverse high-level policy models. The system has demonstrated significant success rate improvements across tasks such as operating a microwave, folding clothes, and watering plants.
|
| 16 |
+
|
| 17 |
+
## Environment Setup
|
| 18 |
+
|
| 19 |
+
The code has been tested with CUDA 12.4.
|
| 20 |
+
|
| 21 |
+
### 1. Create and Activate Conda Environment
|
| 22 |
+
```bash
|
| 23 |
+
conda create -n vidar python==3.11.9
|
| 24 |
+
conda activate vidar
|
| 25 |
+
```
|
| 26 |
+
|
| 27 |
+
### 2. Install Dependencies
|
| 28 |
+
```bash
|
| 29 |
+
conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=12.4 -c pytorch -c nvidia
|
| 30 |
+
python -m pip install -r requirements.txt
|
| 31 |
+
python -m pip install ninja
|
| 32 |
+
python -m pip install git+https://github.com/Dao-AILab/flash-attention.git@v2.6.3
|
| 33 |
+
python -m pip install xfuser==0.4.0
|
| 34 |
+
```
|
| 35 |
+
|
| 36 |
+
## Usage
|
| 37 |
+
|
| 38 |
+
The AnyPos pipeline consists of a Video Diffusion Model and a Masked Inverse Dynamic Model.
|
| 39 |
+
|
| 40 |
+
### Video Diffusion Model (Vidar) Inference
|
| 41 |
+
To generate a video based on an initial frame and a text instruction:
|
| 42 |
+
```bash
|
| 43 |
+
scripts/vm/sample.sh
|
| 44 |
+
```
|
| 45 |
+
|
| 46 |
+
### Masked Inverse Dynamic Model Inference
|
| 47 |
+
To evaluate the inverse dynamic model:
|
| 48 |
+
```bash
|
| 49 |
+
scripts/idm/eval.sh
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
## Citation
|
| 53 |
+
|
| 54 |
+
```bibtex
|
| 55 |
+
@article{anypos2025,
|
| 56 |
+
title={AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation},
|
| 57 |
+
author={Yao, Feng and others},
|
| 58 |
+
journal={arXiv preprint arXiv:2507.12768},
|
| 59 |
+
year={2025}
|
| 60 |
+
}
|
| 61 |
+
```
|