| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - robotics |
| - bimanual-manipulation |
| - dexterous-manipulation |
| - imitation-learning |
| - diffusion-transformer |
| - tactile-sensing |
| --- |
| |
| # DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter |
| <p> |
| <a href="https://arxiv.org/pdf/2602.05513"> |
| <img src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv"> |
| </a> |
| <a href="https://baai-humanoid.github.io/DECO-webpage/"> |
| <img src="https://img.shields.io/badge/Project-Website-blue"> |
| </a> |
| <a href="https://huggingface.co/datasets/BAAI-Humanoid/DECO-50"> |
| <img src="https://img.shields.io/badge/HuggingFace-Data-yellow?logo=huggingface"> |
| </a> |
| <a href="https://huggingface.co/BAAI-Humanoid/DECO_sim"> |
| <img src="https://img.shields.io/badge/HuggingFace-Model-red?logo=huggingface"> |
| </a> |
| <a href="LICENSE"> |
| <img src="https://img.shields.io/badge/License-Apache--2.0-green"> |
| </a> |
| </p> |
| |
| All checkpoints use a compact model of about **80M parameters** and achieve near-SOTA success rates on their respective benchmarks. |
|
|
| ## Checkpoints |
|
|
| | File | Benchmark | Description | |
| |------|-----------|-------------| |
| | `libero.pth` | [LIBERO](https://github.com/Lifelong-Robot-Learning/LIBERO) | Vision-only policy| |
| | `dexjoco.pth` | [DexJoCo](https://github.com/brave-eai/dexjoco) | Vision-only policy| |
| | `univtac.pth` | [UniVTAC](https://github.com/univtac/UniVTAC) | Vision-only policy | |
| | `univtac-p.pth` | [UniVTAC](https://github.com/univtac/UniVTAC) | Vision policy + plugin tactile adapter | |
| | `robotwin.pth` | [RoboTwin](https://github.com/RoboTwin-Platform/RoboTwin) | Vision policy with a T5 text encoder| |
|
|
| ## Results |
|
|
| ### LIBERO |
| Success rates (50 trials per task): |
|
|
| | Model | Spatial | Object | Goal | Long | Average | |
| |:---:|:---:|:---:|:---:|:---:|:---:| |
| | DECO | 94.2% | 99.4% | 96.8% | 86.4% | **94.2%** | |
|
|
| ### DexJoCo |
| Success rates (50 trials per task); `/B` denotes a bimanual task: |
|
|
| | Model | Hammer Nail | Click Mouse | Pick Bucket | Pinch Tongs | Fold Glasses | Water Plant | Unlock iPad /B | Hanoi /B | Assembly /B | Microwave /B | Photograph /B | Average | |
| |:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| |
| | DECO (rand-obj) | 68% | 90% | 80% | 46% | 50% | 88% | 40% | 18% | 0% | 64% | 76% | **56.4%** | |
| | DECO (rand-full) | 66% | 82% | 88% | 52% | 52% | 80% | 40% | 24% | 16% | 72% | 66% | **58%** | |
|
|
| ### UniVTAC |
| Success rates (100 trials per task): |
|
|
| | Model | grasp_classify | lift_can | lift_bottle | pull_out_key | put_bottle_in_shelf | insert_hole | insert_tube | insert_HDMI | Average | |
| |:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| |
| | DECO (vision) | 96.0% | 99.0% | 100.0% | 96.0% | 90.0% | 63.0% | 88.0% | 25.0% | **82.1%** | |
| | DECO.p (+tactile) | 99.0% | 99.0% | 100.0% | 96.0% | 93.0% | 59.0% | 88.0% | 23.0% | **82.1%** | |
| |
| ### RoboTwin |
| Success rates (%) on the 50 RoboTwin dual-arm tasks under the Seen and Unseen settings (100 trials per task): |
| |
| | Task | Seen Clean | Seen Randomized | Seen Avg. | Unseen Clean | Unseen Randomized | Unseen Avg. | |
| |------|-----------:|----------------:|----------:|-------------:|------------------:|------------:| |
| | adjust_bottle | 100.00% | 100.00% | 100.00% | 99.00% | 99.00% | 99.00% | |
| | beat_block_hammer | 96.00% | 86.00% | 91.00% | 98.00% | 90.00% | 94.00% | |
| | blocks_ranking_rgb | 72.00% | 65.00% | 68.50% | 75.00% | 65.00% | 70.00% | |
| | blocks_ranking_size | 44.00% | 38.00% | 41.00% | 42.00% | 40.00% | 41.00% | |
| | click_alarmclock | 95.00% | 88.00% | 91.50% | 93.00% | 92.00% | 92.50% | |
| | click_bell | 95.00% | 95.00% | 95.00% | 94.00% | 90.00% | 92.00% | |
| | dump_bin_bigbin | 96.00% | 95.00% | 95.50% | 98.00% | 97.00% | 97.50% | |
| | grab_roller | 98.00% | 99.00% | 98.50% | 100.00% | 99.00% | 99.50% | |
| | handover_block | 97.00% | 86.00% | 91.50% | 97.00% | 87.00% | 92.00% | |
| | handover_mic | 100.00% | 96.00% | 98.00% | 99.00% | 97.00% | 98.00% | |
| | hanging_mug | 43.00% | 26.00% | 34.50% | 30.00% | 33.00% | 31.50% | |
| | lift_pot | 100.00% | 100.00% | 100.00% | 100.00% | 99.00% | 99.50% | |
| | move_can_pot | 85.00% | 83.00% | 84.00% | 77.00% | 79.00% | 78.00% | |
| | move_pillbottle_pad | 93.00% | 92.00% | 92.50% | 91.00% | 88.00% | 89.50% | |
| | move_playingcard_away | 100.00% | 96.00% | 98.00% | 98.00% | 99.00% | 98.50% | |
| | move_stapler_pad | 85.00% | 87.00% | 86.00% | 87.00% | 85.00% | 86.00% | |
| | open_laptop | 96.00% | 100.00% | 98.00% | 96.00% | 100.00% | 98.00% | |
| | open_microwave | 88.00% | 84.00% | 86.00% | 85.00% | 86.00% | 85.50% | |
| | pick_diverse_bottles | 84.00% | 77.00% | 80.50% | 85.00% | 74.00% | 79.50% | |
| | pick_dual_bottles | 90.00% | 92.00% | 91.00% | 93.00% | 93.00% | 93.00% | |
| | place_a2b_left | 63.00% | 59.00% | 61.00% | 62.00% | 58.00% | 60.00% | |
| | place_a2b_right | 53.00% | 47.00% | 50.00% | 49.00% | 50.00% | 49.50% | |
| | place_bread_basket | 92.00% | 81.00% | 86.50% | 92.00% | 86.00% | 89.00% | |
| | place_bread_skillet | 91.00% | 81.00% | 86.00% | 91.00% | 83.00% | 87.00% | |
| | place_burger_fries | 97.00% | 100.00% | 98.50% | 99.00% | 98.00% | 98.50% | |
| | place_can_basket | 79.00% | 71.00% | 75.00% | 78.00% | 74.00% | 76.00% | |
| | place_cans_plasticbox | 100.00% | 99.00% | 99.50% | 100.00% | 99.00% | 99.50% | |
| | place_container_plate | 99.00% | 97.00% | 98.00% | 95.00% | 97.00% | 96.00% | |
| | place_dual_shoes | 85.00% | 81.00% | 83.00% | 79.00% | 76.00% | 77.50% | |
| | place_empty_cup | 99.00% | 100.00% | 99.50% | 97.00% | 98.00% | 97.50% | |
| | place_fan | 92.00% | 93.00% | 92.50% | 93.00% | 99.00% | 96.00% | |
| | place_mouse_pad | 83.00% | 75.00% | 79.00% | 85.00% | 74.00% | 79.50% | |
| | place_object_basket | 89.00% | 89.00% | 89.00% | 86.00% | 83.00% | 84.50% | |
| | place_object_scale | 77.00% | 68.00% | 72.50% | 77.00% | 67.00% | 72.00% | |
| | place_object_stand | 90.00% | 87.00% | 88.50% | 96.00% | 85.00% | 90.50% | |
| | place_phone_stand | 83.00% | 85.00% | 84.00% | 77.00% | 83.00% | 80.00% | |
| | place_shoe | 98.00% | 98.00% | 98.00% | 98.00% | 100.00% | 99.00% | |
| | press_stapler | 82.00% | 84.00% | 83.00% | 81.00% | 87.00% | 84.00% | |
| | put_bottles_dustbin | 87.00% | 88.00% | 87.50% | 83.00% | 86.00% | 84.50% | |
| | put_object_cabinet | 75.00% | 84.00% | 79.50% | 84.00% | 79.00% | 81.50% | |
| | rotate_qrcode | 85.00% | 93.00% | 89.00% | 85.00% | 87.00% | 86.00% | |
| | scan_object | 88.00% | 89.00% | 88.50% | 92.00% | 90.00% | 91.00% | |
| | shake_bottle | 100.00% | 98.00% | 99.00% | 100.00% | 98.00% | 99.00% | |
| | shake_bottle_horizontally | 100.00% | 99.00% | 99.50% | 100.00% | 99.00% | 99.50% | |
| | stack_blocks_three | 85.00% | 83.00% | 84.00% | 85.00% | 81.00% | 83.00% | |
| | stack_blocks_two | 96.00% | 99.00% | 97.50% | 97.00% | 98.00% | 97.50% | |
| | stack_bowls_three | 83.00% | 83.00% | 83.00% | 81.00% | 78.00% | 79.50% | |
| | stack_bowls_two | 96.00% | 95.00% | 95.50% | 99.00% | 98.00% | 98.50% | |
| | stamp_seal | 61.00% | 75.00% | 68.00% | 69.00% | 71.00% | 70.00% | |
| | turn_switch | 63.00% | 72.00% | 67.50% | 69.00% | 64.00% | 66.50% | |
| | **Overall** | **86.56%** | **84.76%** | **85.66%** | **86.32%** | **84.36%** | **85.34%** | |
| |
| ## Usage |
| |
| Download all checkpoints: |
| |
| ```bash |
| hf download BAAI-Humanoid/DECO_sim --repo-type model --local-dir ./sim_ckpt/ |
| ``` |
| |
| Load a checkpoint into the DECO policy and evaluate it with the corresponding benchmark configuration. The training and evaluation code lives in the [DECO repository](https://github.com/BAAI-Humanoid/DECO): |
| |
| - LIBERO, DexJoCo, and UniVTAC are on the `sim` branch. |
| - RoboTwin is on the `robotwin` branch. |
| |
| ## Citation |
| |
| ```bibtex |
| @article{li2026deco, |
| title={DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter}, |
| author={Xukun Li and Yu Sun and Lei Zhang and Bosheng Huang and Yibo Peng and Yuan Meng and Haojun Jiang and Shaoxuan Xie and Guacai Yao and Alois Knoll and Zhenshan Bing and Xinlong Wang and Zhenguo Sun}, |
| journal={arXiv preprint arXiv:2602.05513}, |
| year={2026} |
| } |
| ``` |
| |
| ## License |
| |
| [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
| |
