Add model card for AnyPos

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +61 -0
README.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: robotics
3
+ ---
4
+
5
+ # AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation
6
+
7
+ AnyPos (also referred to as **Vidar** in the codebase) is a unified pipeline that integrates large-scale automated task-agnostic exploration with robust embodiment modeling through inverse dynamics learning. By focusing on exploring all feasible actions of the embodiment, AnyPos learns embodiment dynamics directly from task-agnostic action data and decouples them from high-level policy learning.
8
+
9
+ - **Paper:** [AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation](https://huggingface.co/papers/2507.12768)
10
+ - **Project Page:** [https://embodiedfoundation.github.io/vidar_anypos](https://embodiedfoundation.github.io/vidar_anypos)
11
+ - **Repository:** [https://github.com/yaofeng1998/Vidar](https://github.com/yaofeng1998/Vidar)
12
+
13
+ ## Introduction
14
+
15
+ AnyPos generates diverse yet safe trajectories at scale, then learns embodiment representations by decoupling arm and end-effector motions. It employs a direction-aware decoder to stabilize predictions under distribution shift, allowing it to be coupled with diverse high-level policy models. The system has demonstrated significant success rate improvements across tasks such as operating a microwave, folding clothes, and watering plants.
16
+
17
+ ## Environment Setup
18
+
19
+ The code has been tested with CUDA 12.4.
20
+
21
+ ### 1. Create and Activate Conda Environment
22
+ ```bash
23
+ conda create -n vidar python==3.11.9
24
+ conda activate vidar
25
+ ```
26
+
27
+ ### 2. Install Dependencies
28
+ ```bash
29
+ conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=12.4 -c pytorch -c nvidia
30
+ python -m pip install -r requirements.txt
31
+ python -m pip install ninja
32
+ python -m pip install git+https://github.com/Dao-AILab/flash-attention.git@v2.6.3
33
+ python -m pip install xfuser==0.4.0
34
+ ```
35
+
36
+ ## Usage
37
+
38
+ The AnyPos pipeline consists of a Video Diffusion Model and a Masked Inverse Dynamic Model.
39
+
40
+ ### Video Diffusion Model (Vidar) Inference
41
+ To generate a video based on an initial frame and a text instruction:
42
+ ```bash
43
+ scripts/vm/sample.sh
44
+ ```
45
+
46
+ ### Masked Inverse Dynamic Model Inference
47
+ To evaluate the inverse dynamic model:
48
+ ```bash
49
+ scripts/idm/eval.sh
50
+ ```
51
+
52
+ ## Citation
53
+
54
+ ```bibtex
55
+ @article{anypos2025,
56
+ title={AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation},
57
+ author={Yao, Feng and others},
58
+ journal={arXiv preprint arXiv:2507.12768},
59
+ year={2025}
60
+ }
61
+ ```