nielsr HF Staff commited on
Commit
171ee01
·
verified ·
1 Parent(s): 83ebfe7

Improve model card: Add pipeline tag, lerobot library, and comprehensive details

Browse files

This PR significantly enhances the model card for the "Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions" model.

It adds:
- The `robotics` pipeline tag for better discoverability.
- The `lerobot` library tag, as evidenced by training configurations, enabling automated code snippets.
- A link to the paper ([Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions](https://huggingface.co/papers/2511.04665)).
- A link to the project page ([https://real2sim-eval.github.io/](https://real2sim-eval.github.io/)).
- A link to the GitHub repository ([https://github.com/kywind/real2sim-eval](https://github.com/kywind/real2sim-eval)).
- Comprehensive content, including the paper abstract, an overview of the framework, detailed installation instructions, asset/checkpoint information, and usage guidelines, all adapted from the original GitHub README to provide a rich user experience on the Hub.

Please review and merge this PR to make this valuable robotics artifact more accessible and understandable on the Hugging Face Hub!

Files changed (1) hide show
  1. README.md +58 -3
README.md CHANGED
@@ -1,3 +1,58 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ pipeline_tag: robotics
4
+ library_name: lerobot
5
+ ---
6
+
7
+ # Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions
8
+
9
+ This repository contains the official implementation of the **Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions** framework.
10
+
11
+ **Paper:** [https://huggingface.co/papers/2511.04665](https://huggingface.co/papers/2511.04665)
12
+ **Project Page:** [https://real2sim-eval.github.io/](https://real2sim-eval.github.io/)
13
+ **Code:** [https://github.com/kywind/real2sim-eval](https://github.com/kywind/real2sim-eval)
14
+
15
+ ## Abstract
16
+ Robotic manipulation policies are advancing rapidly, but their direct evaluation in the real world remains costly, time-consuming, and difficult to reproduce, particularly for tasks involving deformable objects. Simulation provides a scalable and systematic alternative, yet existing simulators often fail to capture the coupled visual and physical complexity of soft-body interactions. We present a real-to-sim policy evaluation framework that constructs soft-body digital twins from real-world videos and renders robots, objects, and environments with photorealistic fidelity using 3D Gaussian Splatting. We validate our approach on representative deformable manipulation tasks, including plush toy packing, rope routing, and T-block pushing, demonstrating that simulated rollouts correlate strongly with real-world execution performance and reveal key behavioral patterns of learned policies. Our results suggest that combining physics-informed reconstruction with high-quality rendering enables reproducible, scalable, and accurate evaluation of robotic manipulation policies.
17
+
18
+ ## Overview
19
+ This framework allows for:
20
+ - Constructing simulation assets: visualizing object Gaussians, articulating robot Gaussians, defining and rendering different object layouts.
21
+ - Deploying trained PhysTwin to simulate deformable object Gaussians.
22
+ - Training policies using the provided `policy_training` submodule.
23
+ - Evaluating policies in the constructed simulation environment.
24
+ - Interactive control of the Gaussian-based simulation.
25
+
26
+ ## Installation
27
+ For detailed installation instructions, including prerequisites and steps for setting up the Python environment and external dependencies, please refer to the [Installation section of the GitHub repository](https://github.com/kywind/real2sim-eval#2-installation).
28
+
29
+ ## Assets and Checkpoints
30
+ The Gaussian Splatting assets, PhysTwin checkpoints, and policy checkpoints used in the paper are available on Hugging Face:
31
+
32
+ | Task | Hugging Face Dataset Collection |
33
+ | :----------------------- | :------------------------------------------------------------------------------------------------------- |
34
+ | **Gaussian Splatting assets** | [gs-scans](https://huggingface.co/shashuo0104/gs-scans) |
35
+ | **PhysTwin checkpoints** | [phystwin-toy](https://huggingface.co/shashuo0104/phystwin-toy), [phystwin-rope](https://huggingface.co/shashuo0104/phystwin-rope), [phystwin-T-block](https://huggingface.co/shashuo0104/phystwin-T-block) |
36
+ | **Policy Checkpoints** | [See policy submodule README](https://github.com/shuosha/policy_training/blob/ceb025db01db539f3df954d6836cda2bd39d0ef3/README.md) |
37
+
38
+ Please refer to the [Assets and Checkpoints section on GitHub](https://github.com/kywind/real2sim-eval#3-assets-and-checkpoints) for detailed instructions on downloading and organizing these files.
39
+
40
+ ## Usage
41
+ The repository provides scripts for:
42
+ - **Policy evaluation**: `experiments/eval_policy.py` (and parallel version `eval_policy_parallel.py`).
43
+ - **Keyboard interactive control**: `experiments/keyboard_teleop.py`.
44
+ - **Motion replay**: `experiments/replay.py`.
45
+ - **Policy training**: Refer to the [policy submodule README](https://github.com/shuosha/policy_training/blob/ceb025db01db539f3df954d6836cda2bd39d0ef3/README.md).
46
+
47
+ For detailed commands and configuration specifics, please visit the [Run the code section of the GitHub repository](https://github.com/kywind/real2sim-eval#4-run-the-code).
48
+
49
+ ## Citation
50
+ If you find this work useful, please consider citing the paper:
51
+ ```bibtex
52
+ @article{zhang2025real,
53
+ title={Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions},
54
+ author={Zhang, Kaifeng and Sha, Shuo and Jiang, Hanxiao and Loper, Matthew and Song, Hyunjong and Cai, Guangyan and Xu, Zhuo and Hu, Xiaochen and Zheng, Changxi and Li, Yunzhu},
55
+ journal={arXiv preprint arXiv:2511.04665},
56
+ year={2025}
57
+ }
58
+ ```