Update README.md

README.md

@@ -1,3 +1,358 @@
 ---
 license: apache-2.0
 
+tags:
+  - humanoid-robotics
+  - loco-manipulation
+  - world-action-model
+  - embodied-ai
+  - robot-learning
+  - vision-language-action
+  - diffusion
+  - latent-world-model
+  - unitree-g1
+---
+
+# ω-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation
+
+**ω-0 (OMEGA-0)** is a latent predictive world action model for real-world humanoid **concurrent loco-manipulation**.
+
+Given a language instruction, visual observations, and robot proprioceptive state, ω-0 predicts **controller-compatible whole-body action latents** for humanoid robot execution. Instead of reconstructing future RGB videos in pixel space, ω-0 predicts compact future visual representations and couples them with diffusion-based whole-body action generation.
+
+The released checkpoint is intended to be used with the official OMEGA-0 codebase for inference and real-robot deployment.
+
+> **Paper:** *ω-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation*  
+> **Code:** https://github.com/gentlefress/OMEGA-0  
+> **Project Page:** https://gentlefress.github.io/OMEGA-0_page/  
+> **Dataset:** https://huggingface.co/datasets/keycharon/omega-HOME
+
+---
+
+## 🌟 Highlights
+
+- **Whole-body loco-manipulation:** jointly coordinates locomotion, posture, balance, and manipulation.
+- **Latent predictive modeling:** predicts future visual representations rather than reconstructing future frames in pixel space.
+- **Action generation:** generates whole-body action latents compatible with the SONIC controller.
+- **Multimodal conditioning:** supports language, visual observations, and robot state.
+- **Real-world humanoid deployment:** demonstrated on the Unitree G1.
+- **Multiple visual viewpoints:** supports egocentric RGB, exocentric RGB, and exocentric depth.
+- **Single unified policy:** one model is used across a diverse set of household loco-manipulation tasks.
+- **Open-source workflow:** the official repository provides training, inference, teleoperation, recording, and deployment code.
+
+---
+
+## 🧠 Model Overview
+
+ω-0 follows a simple principle:
+
+> **Predict how the scene will evolve, while directly generating how the whole body should act.**
+
+The model combines three complementary components:
+
+```text
+Language Instruction
+        │
+        ├──────────────┐
+        │              │
+Visual Observation   Robot State
+        │              │
+        └──────┬───────┘
+               ↓
+        Multimodal Encoder
+               │
+               ↓
+     ┌─────────────────────┐
+     │   ω-0 WAM Predictor │
+     └─────────────────────┘
+          │           │
+          │           └────────→ Future Visual Latent
+          │
+          └────────────────────→ Whole-body Action Latent
+                                      │
+                                      ↓
+                                   SONIC
+                                      │
+                                      ↓
+                                Unitree G1
+```
+
+The predictive branch provides a compact representation of future scene evolution, while the action branch generates controller-compatible whole-body actions.
+
+---
+
+## 📥 Inputs
+
+Depending on the configured model and deployment setup, ω-0 can use:
+
+### Language
+
+A natural-language task instruction, for example:
+
+```text
+"Pick up the clothes from the washing machine."
+```
+
+### Visual Observation
+
+Supported observation modes include:
+
+- Egocentric RGB
+- Exocentric RGB
+- Exocentric depth
+
+### Robot Proprioception
+
+Robot state information can be used as specified by the corresponding configuration.
+
+---
+
+## 📤 Outputs
+
+The model predicts **whole-body action latents**, which are subsequently decoded/interpreted by the OMEGA-0 action pipeline and executed through the **SONIC** whole-body controller.
+
+The checkpoint therefore should **not** be interpreted as a standalone joint-position predictor.
+
+The complete execution pipeline is:
+
+```text
+Instruction + Observation + Robot State
+                    ↓
+                  ω-0
+                    ↓
+          Whole-body Action Latent
+                    ↓
+              Action Decoder
+                    ↓
+                  SONIC
+                    ↓
+               Unitree G1
+```
+
+---
+
+## 🤖 Real-World Platform
+
+The released system targets the following experimental setup:
+
+- **Robot:** Unitree G1
+- **Hands:** Inspire dexterous hands
+- **Controller:** SONIC
+- **Egocentric camera:** ZED Mini
+- **Teleoperation:** PICO body tracking through XRoboToolkit
+
+Hardware and software configurations may need to be adapted to your own setup.
+
+---
+
+## 🚀 Quick Start
+
+### 1. Clone the official repository
+
+```bash
+git clone https://github.com/gentlefress/OMEGA-0.git
+cd OMEGA-0
+```
+
+### 2. Install the environment
+
+The official repository uses `uv`:
+
+```bash
+uv sync --locked --extra train --extra deploy --python 3.10 --inexact
+source .venv/bin/activate
+
+uv pip install flash-attn --no-build-isolation
+```
+
+Verify the installation:
+
+```bash
+python -c "import omega, torch; print('PyTorch:', torch.__version__); print('CUDA:', torch.cuda.is_available())"
+```
+
+### 3. Download the checkpoint
+
+Download the checkpoint from this Hugging Face repository and place it in your local model directory.
+
+For example:
+
+```text
+/path/to/omega0_checkpoint/
+```
+
+### 4. Configure the inference server
+
+The official OMEGA-0 repository provides an inference configuration under:
+
+```text
+src/configs/serve.yaml
+```
+
+For an exported checkpoint, configure the model artifact:
+
+```yaml
+model:
+  artifact: /path/to/omega0_checkpoint
+  device: cuda
+```
+
+Then validate the configuration:
+
+```bash
+python -m omega.inference.run --config src/configs/serve.yaml --check
+```
+
+Launch the inference server:
+
+```bash
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=4 \
+HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
+python -m omega.inference.run --config src/configs/serve.yaml
+```
+
+The default inference server listens on:
+
+```text
+127.0.0.1:8014
+```
+
+For multi-machine deployment, configure the server and client addresses according to your network setup.
+
+---
+
+## 🦾 Robot Deployment
+
+The official repository provides a robot client for Unitree G1 deployment.
+
+Set the inference endpoint and task instruction:
+
+```bash
+export INFERENCE_ENDPOINT=http://127.0.0.1:8014
+export INSTRUCTION="Pick up the clothes from the washing machine."
+```
+
+Then run:
+
+```bash
+python -m omega_real.run --config real/configs/deploy.yaml --check
+python -m omega_real.run --config real/configs/deploy.yaml --arm
+```
+
+Please follow the official deployment instructions and safety procedures before enabling robot commands.
+
+---
+
+## 🏋️ Training
+
+The released codebase contains two main training stages:
+
+### Stage 1 — Action-Token Pretraining
+
+The model first learns to associate language and visual observations with whole-body motion tokens.
+
+```text
+Language + Visual Observation
+              ↓
+           Qwen3-VL
+              ↓
+       Whole-body motion tokens
+```
+
+### Stage 2 — WAM Fine-Tuning
+
+The model is further optimized to jointly predict future visual representations and whole-body action latents.
+
+```text
+Language + Observation + State
+              ↓
+           ω-0 WAM
+        ↙             ↘
+Future Visual      Action Latent
+Representation
+                      ↓
+                    SONIC
+```
+
+Training and dataset preparation instructions are available in the official repository.
+
+---
+
+## 📊 ω-HOME Dataset
+
+ω-0 is accompanied by **ω-HOME**, a 40+ hour real-world household humanoid dataset containing synchronized multimodal observations, whole-body motion, robot states, and action latents.
+
+The dataset is available on Hugging Face:
+
+https://huggingface.co/datasets/keycharon/omega-HOME
+
+---
+
+## 🎯 Intended Use
+
+This checkpoint is intended for:
+
+- Research on humanoid robot loco-manipulation
+- World action models
+- Whole-body robot learning
+- Vision-language-action modeling
+- Latent predictive modeling
+- Real-world humanoid manipulation
+- Reproduction and extension of the ω-0 experiments
+
+The checkpoint is primarily a **research model** and is not intended for unsupervised deployment in safety-critical environments.
+
+---
+
+## ⚠️ Limitations and Safety
+
+ω-0 is a research system and should be deployed with appropriate hardware-level and software-level safety mechanisms.
+
+In particular:
+
+- Real-robot behavior depends on the underlying robot, controller, camera, calibration, and environment.
+- The released model is developed and evaluated primarily on the Unitree G1 platform.
+- Performance may degrade under substantial distribution shifts in scenes, objects, embodiments, or tasks.
+- Generated actions should be checked by the downstream controller and deployment stack.
+- Do not operate the robot near people without appropriate safety precautions.
+- The model should not be treated as a guaranteed safe or autonomous control policy.
+
+---
+
+## 📚 Citation
+
+If you use ω-0 or this checkpoint in your research, please cite:
+
+```bibtex
+@article{li2026omega0,
+  title   = {{$\omega$-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation}},
+  author  = {Zhe Li and Zhenzhe Zhang and Yangyang Wei and Wenjie Zhang and
+             Xichen Yuan and Peiyuan Zhi and Gen Li and Xinying Guo and
+             Fengjie Gao and Jianfei Yang and Shanghang Zhang},
+  journal = {arXiv preprint arXiv:2608.06375},
+  year    = {2026},
+  doi     = {10.48550/arXiv.2608.06375},
+  url     = {https://arxiv.org/abs/2608.06375}
+}
+```
+
+---
+
+## 🙏 Acknowledgements
+
+We acknowledge the open-source projects that contributed to the development of OMEGA-0, including:
+
+- Ψ₀
+- GR00T-WholeBodyControl
+- Qwen3-VL
+- V-JEPA
+- Wan
+
+Please refer to the official repository and the corresponding third-party licenses for the exact dependencies and attribution requirements.
+
+---
+
+## 📄 License
+
+OMEGA-0 code is released under the **Apache 2.0 License**. Third-party code, models, and assets remain subject to their respective licenses.
+
+Please check the license terms of all external components before redistribution or commercial use.