Improve model card and add robotics pipeline tag
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,6 +1,32 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
| 3 |
---
|
| 4 |
-
This is the model of the paper [MVP-LAM: Learning Action-Centric Latent Action via Cross-Viewpoint Reconstruction](https://jmsnu.github.io/).
|
| 5 |
|
| 6 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
pipeline_tag: robotics
|
| 4 |
---
|
|
|
|
| 5 |
|
| 6 |
+
# MVP-LAM: Learning Action-Centric Latent Action via Cross-Viewpoint Reconstruction
|
| 7 |
+
|
| 8 |
+
[**Paper**](https://arxiv.org/abs/2602.03668) | [**Project Page**](https://jmsnu.github.io/) | [**Code**](https://github.com/jmSNU/mvp_lam)
|
| 9 |
+
|
| 10 |
+
MVP-LAM (Multi-ViewPoint Latent Action Model) is a framework designed to learn latent actions that are highly informative about ground-truth actions from multi-view videos. It uses a cross-viewpoint reconstruction objective, ensuring that a latent action from one view can explain the future in another view, which reduces reliance on viewpoint-specific cues and results in more action-centric representations.
|
| 11 |
+
|
| 12 |
+
## Highlights
|
| 13 |
+
|
| 14 |
+
- **Cross-viewpoint reconstruction**: Latent actions inferred from one view must explain the future in another view, promoting viewpoint-invariant features.
|
| 15 |
+
- **Action-centric representations**: Achieves higher mutual information with ground-truth actions and improved action prediction, even in out-of-distribution scenarios.
|
| 16 |
+
- **Improved VLA pretraining**: Using MVP-LAM latent actions as pseudo-labels enhances the downstream manipulation performance of Vision-Language-Action (VLA) models on benchmarks like SIMPLER and LIBERO-Long.
|
| 17 |
+
|
| 18 |
+
## Citation
|
| 19 |
+
|
| 20 |
+
If you find this work useful, please cite:
|
| 21 |
+
|
| 22 |
+
```bibtex
|
| 23 |
+
@misc{lee2026mvplam,
|
| 24 |
+
title = {MVP-LAM: Learning Action-Centric Latent Action via Cross-Viewpoint Reconstruction},
|
| 25 |
+
author = {Jung Min Lee and Dohyeok Lee and Seokhun Ju and Taehyun Cho and Jin Woo Koo and Li Zhao and Sangwoo Hong and Jungwoo Lee},
|
| 26 |
+
year = {2026},
|
| 27 |
+
eprint = {2602.03668},
|
| 28 |
+
archivePrefix = {arXiv},
|
| 29 |
+
primaryClass = {cs.RO},
|
| 30 |
+
url = {https://arxiv.org/abs/2602.03668}
|
| 31 |
+
}
|
| 32 |
+
```
|