Create README.md

README.md

@@ -0,0 +1,122 @@
+---
+tags:
+- Embodied-AI
+- Robotics
+- Offline-to-Online-RL
+- Value-Function
+- VLA-Model
+- Robot-Foundation-Model
+- Vision-Language-Action
+- Robot-Learning
+- Imitation-Learning
+---
+
+# Robo-ValueRL
+
+[[Project Page](https://gewu-lab.github.io/robo-valuerl/)] [[GitHub](https://github.com/Open-X-Humanoid/Robo-ValueRL)] [[Paper](#)]
+
+This repository contains the **models and data** for **Robo-ValueRL: Reliable Value Estimation for Offline-to-Online Reinforcement Learning**.
+
+Robo-ValueRL studies how reliable value estimation can guide robotic policy learning from heterogeneous offline demonstrations and online rollout trajectories. The framework learns a history-conditioned value estimator, converts value differences into action-quality conditions for offline policy pretraining, and uses value-guided rollout filtering for stable online residual adaptation.
+
+## 🤖 Model Description
+
+**Robo-ValueRL** is an offline-to-online robotic reinforcement learning framework centered on reliable value estimation. Instead of only reporting final task success, Robo-ValueRL explicitly diagnoses whether learned values capture global task progress and local action-level preference, then propagates these value signals into downstream policy learning.
+
+The released model suite includes:
+
+1. **History-Conditioned Value Estimator**  
+   Predicts normalized task progress from multi-view robot observations, language instructions, and visual history. The temporal context helps reduce ambiguity caused by occlusions, repeated motions, and visually similar task stages.
+
+2. **Quality-Conditioned VLA Policy**  
+   Uses value differences to derive action-quality conditions. These conditions guide a Vision-Language-Action policy during offline pretraining, allowing the policy to prioritize useful behaviors from mixed-quality demonstrations.
+
+3. **Online Residual Adaptation Module**  
+   Learns lightweight corrections from value-filtered online rollouts while keeping the pretrained base policy frozen. This enables targeted failure recovery and self-correction without overwriting the offline prior.
+
+## 📦 Data Description
+
+The Robo-ValueRL dataset contains heterogeneous real-robot experience collected on two long-horizon manipulation tasks:
+
+- **Chip Insertion**: millimeter-level precision manipulation requiring PCB grasping, pose adjustment, chip grasping, and insertion.
+- **Block Disassembly**: generalizable object disassembly requiring robust grasping, separation, and classification behaviors.
+
+The dataset includes:
+
+- **240 hours** of offline demonstrations
+- **3,000+** online rollout trajectories
+- Multi-view robot observations
+- Language task instructions
+- Robot states and action chunks
+- Mixed-quality trajectories, including successful demonstrations, corrections, suboptimal behaviors, and failure cases
+- Value-derived action-quality labels / conditions for policy learning
+- Online rollout segments for residual adaptation
+
+## 🧱 Model Hierarchy
+
+1. **Value Estimation**
+   - Learns history-conditioned value functions from heterogeneous robot data.
+   - Evaluates value reliability with global-progress and local-preference metrics.
+
+2. **Offline Policy Pretraining**
+   - Converts value differences into action-quality conditions.
+   - Trains a quality-conditioned VLA policy on mixed-quality demonstrations.
+
+3. **Online Policy Improvement**
+   - Uses reliable value estimates to filter online rollout data.
+   - Trains a lightweight residual adapter for targeted real-world improvement.
+
+## ✨ Key Features
+
+* **Reliable Value Estimation**: Uses visual history to produce smoother progress estimates and sharper error responses.
+* **Value-Guided Data Utilization**: Prioritizes useful demonstrations and rollout segments from heterogeneous robot experience.
+* **Quality-Conditioned Policy Learning**: Conditions the VLA policy on value-derived action quality.
+* **Stable Offline-to-Online Improvement**: Improves real-world performance through residual adaptation while preserving the pretrained base policy.
+* **Real-Robot Evaluation**: Evaluated on precision chip insertion and generalizable block disassembly.
+
+## 📊 Highlights
+
+- 86% final success on chip insertion
+- 84% final success on block disassembly
+- +26% offline gain on chip insertion
+- +34% offline gain on block disassembly
+- 240h offline demonstrations
+- 3,000+ online rollout trajectories
+
+## 🛠 Usage
+
+The released assets are organized for reproducing Robo-ValueRL's model and data pipeline.
+
+Typical usage includes:
+
+1. Load the offline demonstration dataset.
+2. Train or evaluate the history-conditioned value estimator.
+3. Derive action-quality conditions from value differences.
+4. Train the quality-conditioned VLA policy.
+5. Use online rollout data to train the residual adaptation module.
+
+Please refer to the [GitHub repository](https://github.com/Open-X-Humanoid/Robo-ValueRL) for setup instructions, inference scripts, training code, and data-processing utilities.
+
+## 📁 Repository Contents
+
+This release may include:
+
+- Value estimator checkpoints
+- Quality-conditioned VLA policy checkpoints
+- Online residual adapter checkpoints
+- Offline demonstration data
+- Online rollout trajectories
+- Value labels and action-quality conditions
+- Evaluation scripts
+- Visualization and analysis tools
+
+## 📝 Citation
+
+Citation will be updated after the arXiv release.
+
+```bibtex
+# Coming soon
+📜 License
+Please refer to the license file in the GitHub repository.
+📬 Contact
+For questions, please open an issue on our GitHub repository.