Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,90 @@
+---
+license: apache-2.0
+language:
+- en
+tags:
+- robotics
+- 3d-detection
+- vision-language-action
+- pose-estimation
+- grounding
+library_name: transformers
+pipeline_tag: robotics
+---
+
+<div align="center">
+
+# PoseVLA Stage-1: Universal Pose Pretraining for Generalizable Vision-Language-Action Policies
+
+[![arXiv](https://img.shields.io/badge/arXiv-2602.19710-b31b1b.svg)](https://arxiv.org/abs/2602.19710)
+[![Project Page](https://img.shields.io/badge/Project_Page-PoseVLA-2ea44f.svg)](https://hetolin.github.io/PoseVLA/)
+[![GitHub](https://img.shields.io/badge/GitHub-PoseVLA-181717.svg)](https://github.com/hetolin/PoseVLA)
+
+</div>
+
+## Model Description
+
+**PoseVLA** is a Vision-Language-Action (VLA) model that leverages universal 3D pose pretraining for generalizable robotic manipulation. This checkpoint is the **Stage-1 pretrained model**, jointly trained on large-scale 3D detection and robot action data.
+
+- **Architecture**: PaliGemma-3B + Action Expert (π0-based, trained from scratch) with Flow Matching
+- **Pretraining Data**: Omni3D, Omni6D, BOP, GraspClutter6D (3D tasks) + Agibot, InternData-A1 (robot actions)
+- **Input**: Multi-view RGB images + Depth priors + Camera intrinsics
+- **Output**: 3D object detection (Next-Token Prediction) / Robot actions (Flow Matching)
+
+## Usage
+
+### 3D Grounding Inference
+
+```python
+from posevla.modeling_posevla import PoseVLAPolicy, PoseVLAConfig, bin_tokenizer
+from utils.mapping_token import decode_text_to_scene_with_tokenizer
+
+# Load model
+policy = PoseVLAPolicy.from_pretrained("hetolin/PoseVLA-stage1", local_files_only=False, config=posevla_config)
+policy = policy.eval().to(torch.bfloat16).cuda()
+
+# Inference
+output_res = policy.forward_evaluate_ntp(batch)
+pred_text = output_res["pred"][0]
+pred_res = decode_text_to_scene_with_tokenizer(pred_text, bin_tokenizer)
+```
+
+### Full Inference Script
+
+See [`infer_grounding3d.py`](https://github.com/hetolin/PoseVLA/blob/main/infer_grounding3d.py) for complete real-world RGB-D inference pipeline.
+
+## Training Details
+
+| Hyperparameter | Value |
+|:---|:---|
+| Base model | PaliGemma-3B-pt-224 |
+| Action Expert | π0 (Flow Matching, from scratch) |
+| Image resolution | 224 × 224 |
+| Optimizer | AdamW |
+| Learning rate | 5e-5 |
+| Weight decay | 1e-10 |
+| Precision | bf16 |
+| GPUs | 16 × H20 |
+| Batch size | 7 per GPU |
+| Training steps | 100K |
+
+## Intended Use
+
+- **3D Object Grounding**: Open-vocabulary 3D detection from RGB-D images
+- **Robot Manipulation**: Pretrained backbone for downstream robotic fine-tuning (e.g., RoboTwin)
+- **Research**: Studying the synergy between 3D spatial understanding and robot action learning
+
+## Citation
+
+```bibtex
+@article{lin2026posevla,
+  title={PoseVLA: Universal Pose Pretraining for Generalizable Vision-Language-Action Policies},
+  author={Lin, Haitao and Yu, Hanyang and Huang, Jingshun and Zhang, He and Ling, Yonggen and Tan, Ping and Xue, Xiangyang and Fu, Yanwei},
+  journal={arXiv preprint arXiv:2602.19710},
+  year={2026}
+}
+```
+
+## License
+
+This model is released under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).