| --- |
| license: apache-2.0 |
| language: |
| - en |
| library_name: pytorch |
| datasets: |
| - MINT-SJTU/Evo-DexBench-dataset |
| tags: |
| - robotics |
| - dexterous-manipulation |
| - diffusion-policy |
| - tactile |
| - simulation |
| --- |
| |
| # Evo-DexBench Model Access Examples |
|
|
| [](https://arxiv.org/abs/2610.21180) |
| [](https://github.com/MINT-SJTU/Evo-DexBench) |
| [](https://mint-sjtu.github.io/Evo-DexBench.io) |
| [](https://huggingface.co/datasets/MINT-SJTU/Evo-DexBench-dataset) |
| [](https://huggingface.co/datasets/MINT-SJTU/Evo-DexBench_Assets) |
|  |
|
|
| Six inference checkpoint examples for **Evo-DexBench**, developed by |
| **MINT-SJTU**. Use them to connect a policy server to the benchmark's simulator |
| and reproduce the model integration workflow. |
|
|
| ## ✨ Overview |
|
|
| - **Three policies:** DP-T, DP-C, and KineDex. |
| - **Two checkpoints per policy:** Single2 Core and Double3 Core. |
| - **Four tasks per checkpoint:** separate single-arm and bimanual policies. |
| - **Independent model and simulation environments**, connected through the |
| benchmark's WebSocket + MessagePack Policy API. |
|
|
| These are task-family specialists, not universal policies for all 30 benchmark |
| tasks. This repository contains model checkpoints, not simulation assets, |
| demonstrations, or training code. |
|
|
| ## 📦 Contents |
|
|
| The six checkpoints total approximately **5.80 GB** (**5.40 GiB**). |
|
|
| ```text |
| Evo-DexBench_model_access_example/ |
| ├── DP-T/ |
| │ ├── single2-core.ckpt |
| │ └── double3-core.ckpt |
| ├── DP-C/ |
| │ ├── single2-core.ckpt |
| │ └── double3-core.ckpt |
| ├── KineDex/ |
| │ ├── single2-core.ckpt |
| │ └── double3-core.ckpt |
| ├── manifest.json |
| └── README.md |
| ``` |
|
|
| | Policy | Architecture | Inputs | Single2 / Double3 size | |
| | --- | --- | --- | --- | |
| | DP-T | Official Diffusion Policy Transformer | RGB + robot state | 0.171 / 0.171 GB | |
| | DP-C | Official Diffusion Policy convolutional U-Net | RGB + robot state | 1.177 / 1.185 GB | |
| | KineDex | Official KineDex visual/tactile diffusion policy with the Evo-DexBench adapter | RGB + robot state + tactile | 1.544 / 1.552 GB | |
|
|
| Each checkpoint was trained jointly on its four Core tasks. The exported |
| payload retains the original inference EMA weights, normalizers, model |
| configuration, and policy metadata **without changing tensor precision or |
| values**. EMA weights are stored under the inference model entry; duplicate |
| weights, optimizer state, and training-progress state have been removed. |
| These files are **inference-only**, not full training-resume checkpoints. |
| They use the native policy-workspace `.ckpt` format, not a Transformers or |
| Diffusers `from_pretrained` model layout. |
|
|
| The [manifest](manifest.json) records file sizes, SHA256 checksums, and |
| observation contracts. |
|
|
| ### Supported tasks |
|
|
| | Checkpoint group | Task ID | Task name | |
| | --- | --- | --- | |
| | `single2-core` | `m06_v1` | FaucetHandle | |
| | | `m07_v1` | MouseWheel | |
| | | `m08_v1` | CoffeeKnob | |
| | | `m09_v1` | KettleLidTab | |
| | `double3-core` | `b11_v1` | LaptopClosing | |
| | | `b12_v1` | StaplerPaperMark | |
| | | `b13_v1` | ScissorsOpening | |
| | | `b14_v1` | BoxLidOpening | |
|
|
| ### Observation and action contract |
|
|
| All checkpoints use **30 Hz control** and **two observation frames**. Benchmark |
| simulation runs at **120 Hz**. Default receding-horizon execution applies eight |
| actions before requesting the next chunk. |
|
|
| | Group | RGB views, in order | State | DP motion action | KineDex action | |
| | --- | --- | ---: | ---: | ---: | |
| | Single2 | context, interaction, wrist_right | 31 | 30 | 30 motion + 5 force | |
| | Double3 | **interaction**, wrist_left, wrist_right | 62 | 60 | 60 motion + 10 force | |
| |
| RGB views are supplied as `224×224` model inputs. KineDex uses full tactile |
| observations with five finger arrays per hand, padded to `29×1` each. Its |
| client converts predicted finger forces into bounded joint-control residuals. |
| DP-T and DP-C do not request tactile observations. These examples do not use |
| depth inputs. |
| |
| **Double3 uses the interaction camera, not the default dual-arm context view.** |
| Select the camera options shown below when evaluating its checkpoints. DP |
| retains the original 100-step DDPM inference; KineDex retains its checkpoint's |
| DDIM settings. Do not change action timing or sensor contracts to compensate |
| for a slower GPU. |
| |
| ## 🚀 Quick Start |
| |
| Run setup and evaluation from an |
| [Evo-DexBench code checkout](https://github.com/MINT-SJTU/Evo-DexBench). |
| DP checkpoints require a code revision containing |
| `examples/policies/diffusion_policy/` and the `policy-dp` setup profile. If these |
| are absent, obtain a DP-enabled release before following the DP commands. |
|
|
| ### 1. Prepare the simulator and assets |
|
|
| ```bash |
| tools/setup_environment.sh --profile sim --name evodex-sim \ |
| --with-dp-client --with-kinedex-client |
| conda activate evodex-sim |
| hf auth login |
| dexbench assets download --asset-dir "$PWD/assets" |
| source assets/activate_evodex_assets.sh |
| ``` |
|
|
| Accept the terms and obtain access to the official |
| [PartNet-Mobility repository](https://huggingface.co/datasets/sapien-sim/PartNetMobility) |
| before configuring assets. See the |
| [asset repository](https://huggingface.co/datasets/MINT-SJTU/Evo-DexBench_Assets) |
| for asset distribution and licensing details. |
|
|
| ### 2. Download checkpoints |
|
|
| In the simulator environment: |
|
|
| ```bash |
| hf download MINT-SJTU/Evo-DexBench_model_access_example \ |
| --repo-type model --local-dir checkpoints/model-access-example |
| ``` |
|
|
| To download only one policy, add, for example, `--include 'DP-C/*.ckpt'`. |
| Only load checkpoints from trusted sources: these `.ckpt` files use Python |
| pickle and must be loaded through the provided model server. |
|
|
| ### 3. Run DP-C or DP-T |
|
|
| Create the independent model environment: |
|
|
| ```bash |
| tools/setup_environment.sh --profile policy-dp --name evodex-dp |
| ``` |
|
|
| Terminal 1 — start a DP-C server: |
|
|
| ```bash |
| conda activate evodex-dp |
| python -m dp_policy.server \ |
| --checkpoint checkpoints/model-access-example/DP-C/single2-core.ckpt \ |
| --upstream-root "$PWD/.third_party/DiffusionPolicy-official" \ |
| --host 127.0.0.1 --port 8100 --device cuda:0 |
| ``` |
|
|
| Terminal 2 — run ten diagnostic episodes: |
|
|
| ```bash |
| conda activate evodex-sim |
| source assets/activate_evodex_assets.sh |
| dexbench evaluate --policy dp_c --task-id m09_v1 \ |
| --episodes 10 --num-envs 1 --obs-mode rgb \ |
| --policy-kwargs '{"host":"127.0.0.1","port":8100,"embodiment":"single","execute_steps":8}' \ |
| --log results/dp_c_m09.jsonl |
| ``` |
|
|
| For **DP-T**, serve `DP-T/single2-core.ckpt` and select `--policy dp_t` in the |
| client. Either single-arm checkpoint supports all four Single2 Core tasks. |
|
|
| For **Double3**, restart the server with the matching |
| `double3-core.ckpt`, then use a supported bimanual task and the interaction view: |
|
|
| ```bash |
| dexbench evaluate --policy dp_c --task-id b11_v1 \ |
| --episodes 10 --num-envs 1 --obs-mode rgb \ |
| --policy-kwargs '{"host":"127.0.0.1","port":8100,"embodiment":"dual","execute_steps":8,"camera_roles":["interaction","wrist_left","wrist_right"]}' \ |
| --log results/dp_c_b11.jsonl |
| ``` |
|
|
| For a DP-T Double3 checkpoint, use the same camera contract and `--policy dp_t`. |
|
|
| ### 4. Run KineDex |
|
|
| Create its independent model environment: |
|
|
| ```bash |
| tools/setup_environment.sh --profile policy-kinedex --name evodex-kinedex |
| ``` |
|
|
| Terminal 1 — start the model server: |
|
|
| ```bash |
| conda activate evodex-kinedex |
| python -m kinedex_policy.server \ |
| --checkpoint checkpoints/model-access-example/KineDex/single2-core.ckpt \ |
| --upstream-root "$PWD/.third_party/KineDex-official" \ |
| --host 127.0.0.1 --port 8400 --device cuda:0 |
| ``` |
|
|
| Terminal 2 — evaluate with full tactile inputs: |
|
|
| ```bash |
| conda activate evodex-sim |
| source assets/activate_evodex_assets.sh |
| dexbench evaluate --policy kinedex --task-id m07_v1 \ |
| --episodes 10 --num-envs 1 --obs-mode rgb \ |
| --policy-kwargs '{"host":"127.0.0.1","port":8400,"embodiment":"single","execute_steps":8,"tactile_mode":"full"}' \ |
| --log results/kinedex_m07.jsonl |
| ``` |
|
|
| For **Double3**, restart the server with `KineDex/double3-core.ckpt`, then run: |
|
|
| ```bash |
| dexbench evaluate --policy kinedex --task-id b11_v1 \ |
| --episodes 10 --num-envs 1 --obs-mode rgb \ |
| --policy-kwargs '{"host":"127.0.0.1","port":8400,"embodiment":"dual","execute_steps":8,"tactile_mode":"full","camera_profile":"interaction"}' \ |
| --log results/kinedex_b11.jsonl |
| ``` |
|
|
| The ten-episode commands are integration checks, not standard benchmark |
| results. To run the standard 100-episode, four-worker protocol, omit |
| `--episodes` and `--num-envs`. Use a new log filename for every invocation. |
| GPU memory use includes both model inference and simulation rendering; a |
| single-worker check is the starting point for memory-constrained machines. |
|
|
| Training and additional policy options are documented in the code repository's |
| DP and KineDex examples. This model repository does not duplicate their code. |
|
|
| ## ⚖️ License |
|
|
| The released checkpoints are provided under **Apache License 2.0** by |
| **MINT-SJTU**. Upstream policy implementations, simulation software, datasets, |
| and assets retain their respective licenses. This repository does not |
| redistribute PartNet-Mobility geometry. |
|
|
| ## 🙏 Acknowledgement |
|
|
| We thank the authors of |
| [Diffusion Policy](https://github.com/real-stanford/diffusion_policy), |
| [KineDex](https://github.com/DinoMini00/KineDex_code), |
| [LeRobot](https://github.com/huggingface/lerobot), |
| [ManiSkill](https://github.com/haosulab/ManiSkill), and |
| [SAPIEN](https://github.com/haosulab/SAPIEN) for their open-source tools. |
| The adapters pin Diffusion Policy to |
| `5ba07ac6661db573af695b419a7947ecb704690f` and KineDex to |
| `9246f77d7b969b181b0f1b600601b9a9c8083011`. |
|
|
| ## 📄 Citation |
|
|
| If you find Evo-DexBench useful in your research, please cite: |
|
|
| ```bibtex |
| @misc{cheng2026evodexbenchbenchmarkinggeneralizationrelated, |
| title={Evo-DexBench: Benchmarking Generalization across Related Tasks for Dexterous Manipulation}, |
| author={Bing Cheng and Yiran Mao and Zewei Ye and Yifan Jin and Xinyi Chen and Ruifeng Tu and Shuhuai Wang and Tao Lin and Zhimu Zhou and Yuxin Du and Fangwei Zhong and Junchi Yan and Bo Zhao}, |
| year={2026}, |
| eprint={2610.21180}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.RO}, |
| url={https://arxiv.org/abs/2610.21180}, |
| } |
| ``` |
|
|
