| | --- |
| | license: mit |
| | pipeline_tag: robotics |
| | --- |
| | |
| | # TokenHSI Checkpoint Instruction |
| |
|
| | [2025-04-07] All models have been released! |
| |
|
| | Please download the checkpoints and organize them as the following file structure: |
| |
|
| | ``` |
| | |-- tokenhsi |
| | |-- lpanlib |
| | |-- body_models |
| | |-- smpl |
| | |-- SMPL_FEMALE.npz |
| | |-- SMPL_MALE.npz |
| | |-- SMPL_NEUTRAL.pkl |
| | |-- ... |
| | |-- output |
| | |-- single_task |
| | |-- ckpt_traj.pth |
| | |-- ckpt_sit.pth |
| | |-- ckpt_climb.pth |
| | |-- ckpt_carry.pth |
| | |-- tokenhsi |
| | |-- ckpt_stage1.pth |
| | |-- ckpt_stage2_comp_traj_carry |
| | |-- ckpt_stage2_comp_sit_carry |
| | |-- ckpt_stage2_comp_climb_carry |
| | |-- ckpt_stage2_objShape_chair.pth |
| | |-- ckpt_stage2_objShape_table.pth |
| | |-- ckpt_stage2_terrainShape_traj.pth |
| | |-- ckpt_stage2_terrainShape_carry.pth |
| | |-- ckpt_stage2_longterm.pth |
| | ``` |
| |
|
| | * We first provide four single-task environments, including path-following, sitting, climbing, and carrying. For each task, we train a specialized policy using AMP. The trained models are: |
| | * single_task/ckpt_traj.pth |
| | * single_task/ckpt_sit.pth |
| | * single_task/ckpt_climb.pth |
| | * single_task/ckpt_carry.pth |
| |
|
| | * We then provide our multi-task environment for training the versatile, unified transformer policy. The trained model is: |
| | * tokenhsi/ckpt_stage1.pth |
| | |
| | * Finally, we perform policy adaptation to generalize the learned skills to novel tasks. We list the trained models across various downstream tasks. |
| | * Skill Composition |
| | * Follow + Carry: tokenhsi/ckpt_stage2_comp_traj_carry.pth |
| | * Sit + Carry: tokenhsi/ckpt_stage2_comp_sit_carry.pth |
| | * Climb + Carry: tokenhsi/ckpt_stage2_comp_climb_carry.pth |
| | * Object Shape Variation |
| | * Chair: tokenhsi/ckpt_stage2_objShape_chair.pth |
| | * Table: tokenhsi/ckpt_stage2_objShape_table.pth |
| | * Terrain Shape Variation |
| | * Follow: tokenhsi/ckpt_stage2_terrainShape_traj.pth |
| | * Carry: tokenhsi/ckpt_stage2_terrainShape_carry.pth |
| | * Long-horizon Task Completion |
| | * tokenhsi/ckpt_stage2_longterm.pth |
| | |
| | GitHub repo: https://github.com/liangpan99/TokenHSI |
| | arXiv paper: https://huggingface.co/papers/2503.19901 |
| | Project page: https://liangpan99.github.io/TokenHSI/ |
