Add comprehensive model card for Real-to-Sim Robot Policy Evaluation

This PR enhances the model card by:
- Adding the `robotics` pipeline tag to help users discover this work on the Hugging Face Hub.
- Including direct links to the paper, project website, and the GitHub repository for easy access to source materials and code.
- Incorporating the paper abstract to give users a quick overview of the research.
- Copying key sections from the GitHub README, such as Overview, Installation, Assets and Checkpoints, and Run the Code, to provide comprehensive setup and usage instructions.
- Adding a visual teaser image from the project's GitHub repository.
- Including the BibTeX citation for proper attribution.

These changes significantly improve the model card's informativeness and usability for the community.

README.md

@@ -1,3 +1,106 @@
----
-license: mit
----
+---
+license: mit
+pipeline_tag: robotics
+---
+
+# Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions
+
+This repository contains the official implementation of the **Real-to-Sim Robot Policy Evaluation** framework, as presented in the paper:
+
+[**Paper: Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions**](https://huggingface.co/papers/2511.04665)
+[**Project Website**](https://real2sim-eval.github.io/)
+[**GitHub Repository**](https://github.com/kywind/real2sim-eval)
+
+![Teaser Image](https://github.com/kywind/real2sim-eval/raw/main/assets/media/teaser.jpg)
+
+## Abstract
+Robotic manipulation policies are advancing rapidly, but their direct evaluation in the real world remains costly, time-consuming, and difficult to reproduce, particularly for tasks involving deformable objects. Simulation provides a scalable and systematic alternative, yet existing simulators often fail to capture the coupled visual and physical complexity of soft-body interactions. We present a real-to-sim policy evaluation framework that constructs soft-body digital twins from real-world videos and renders robots, objects, and environments with photorealistic fidelity using 3D Gaussian Splatting. We validate our approach on representative deformable manipulation tasks, including plush toy packing, rope routing, and T-block pushing, demonstrating that simulated rollouts correlate strongly with real-world execution performance and reveal key behavioral patterns of learned policies. Our results suggest that combining physics-informed reconstruction with high-quality rendering enables reproducible, scalable, and accurate evaluation of robotic manipulation policies.
+
+## Overview
+
+This repository contains the official implementation of the **Real-to-Sim-Policy-Eval** framework. The main components are:
+
+-   Construct simulation assets: visualize object Gaussians, articulate robot Gaussians, define and render different object layouts.
+-   Deploy trained [PhysTwin](https://github.com/Jianghanxiao/PhysTwin) to simulate deformable object Gaussians.
+-   Train policies using the [policy_training](https://github.com/shuosha/policy_training) submodule located in `policy/`.
+-   Evaluate policy in the constructed simulation environment.
+-   Interactive control of the Gaussian-based simulation.
+
+## Installation
+
+### Prerequisites
+
+-   This project uses [uv](https://docs.astral.sh/uv/), a fast Python package and project manager that serves as a lightweight replacement for Conda. A detailed uv tutorial is available [here](https://docs.astral.sh/uv/guides/install-python/#getting-started). You can also install all dependencies with Conda following similar steps.
+-   The installation has been tested with CUDA 12.8. The CUDA 12.8 installation guide is available [here](https://developer.nvidia.com/cuda-12-8-0-download-archive). If you use a different CUDA version, make sure to install PyTorch with the matching CUDA toolkit; otherwise, the build of `diff-gaussian-rasterizer-w-depth` may fail.
+-   GLM is required to compile `diff-gaussian-rasterizer-w-depth`. Install it with: `sudo apt-get update && sudo apt-get install -y libglm-dev`
+-   FFmpeg is required for policy rollout video generation. FFmpeg cannot be installed directly via uv, but you can install it either system-wide or through Conda. In the latter case, activate your Conda environment before activating the uv virtual environment.
+
+### Installation Steps
+```
+# create and activate a python venv
+uv venv --python=3.11
+source .venv/bin/activate
+
+# install main dependencies
+uv pip install torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/cu128
+uv pip install -r pyproject.toml
+
+# pull the policy submodule and install dependencies
+git submodule update --init --recursive
+cd policy
+uv pip install -r pyproject.toml
+
+# install Gaussian Splatting rasterizer
+cd ../third-party/diff-gaussian-rasterization-w-depth
+uv pip install --no-build-isolation -e .
+
+# install a modified version of urdfpy to avoid version conflicts
+cd ../urdfpy-0.0.22
+uv pip install -e .
+
+# return to the project home
+cd ../..
+```
+
+## Assets and Checkpoints
+
+We provide the Gaussian Splatting assets, PhysTwin checkpoints, and the policy checkpoints we used in the paper on Huggingface.
+
+| Task                | Hugging Face Dataset Collection                                                                        |
+| ------------------- | ------------------------------------------------------------------------------------------------------ |
+| **Gaussian Splatting assets**    | [gs-scans](https://huggingface.co/shashuo0104/gs-scans)       |
+| **PhysTwin checkpoints**    | [phystwin-toy](https://huggingface.co/shashuo0104/phystwin-toy), [phystwin-rope](https://huggingface.co/shashuo0104/phystwin-rope), [phystwin-T-block](https://huggingface.co/shashuo0104/phystwin-T-block)       |
+| **Checkpoints**     | [see policy submodule README](https://github.com/shuosha/policy_training/blob/ceb025db01db539f3df954d6836cda2bd39d0ef3/README.md)         |
+
+After downloading, extract the files to the following folders:
+-   Gaussian Splatting scans: `log/gs/scans/`. E.g., there should be `log/gs/scans/rope/`, `log/gs/scans/sloth/`, etc.
+-   PhysTwin assets: `log/phystwin/<object_name>/`, and rename `<object_name>` to one of `rope`, `sloth`, `T`. E.g., there should be `log/phystwin/rope/data/`, `log/phystwin/sloth/data/`, `log/phystwin/T/data/`, etc.
+-   Policy checkpoints: `log/policy_checkpoints/`. E.g., there should be `log/policy_checkpoints/act-rope-routing/checkpoints/001000/` (lerobot-style) and `log/policy_checkpoints/pi0-rope-routing/10000/` (openpi-style).
+
+## Run the code
+
+For detailed instructions on configuration, policy evaluation, keyboard interactive control, motion replay, and policy training, please refer to the [GitHub repository's README](https://github.com/kywind/real2sim-eval).
+
+Here's an example command for policy evaluation:
+```bash
+CUDA_VISIBLE_DEVICES=0 python experiments/eval_policy.py \
+gs=rope \
+env=xarm_gripper \
+physics.ckpt_path=log/phystwin/rope \
+physics.case_name=rope_0001 \
+policy.inference_cfg_path=policy/configs/inference/insert_rope.json \
+policy.checkpoint_path=log/policy_checkpoints/act-rope-routing/checkpoints/007000
+```
+Execution-ready bash files for various scenarios are located in `experiments/scripts/eval_policy`.
+
+## Citation
+
+If you find this repo useful, please consider citing the paper:
+```bibtex
+@article{zhang2025real,
+    title={Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions},
+    author={Zhang, Kaifeng and Sha, Shuo and Jiang, Hanxiao and Loper, Matthew and Song, Hyunjong and Cai, Guangyan and Xu, Zhuo and Hu, Xiaochen and Zheng, Changxi and Li, Yunzhu},
+    journal={arXiv preprint arXiv:2511.04665},
+    year={2025}
+}
+```