metadata
license: mit
pipeline_tag: robotics
TokenHSI Checkpoint Instruction
Please download the checkpoints and organize them as the following file structure:
|-- tokenhsi
|-- lpanlib
|-- body_models
|-- smpl
|-- SMPL_FEMALE.npz
|-- SMPL_MALE.npz
|-- SMPL_NEUTRAL.pkl
|-- ...
|-- output
|-- single_task
|-- ckpt_traj.pth
|-- ckpt_sit.pth
|-- ckpt_climb.pth
|-- ckpt_carry.pth
|-- tokenhsi
|-- ckpt_stage1.pth
|-- ...
|-- (more models are coming soon!)
We first provide four single-task environments, including path-following, sitting, climbing, and carrying. For each task, we train a specialized policy using AMP. The trained models are:
- single_task/ckpt_traj.pth
- single_task/ckpt_sit.pth
- single_task/ckpt_climb.pth
- single_task/ckpt_carry.pth
We then provide our multi-task environment for training the versatile, unified transformer policy. The trained model is:
- tokenhsi/ckpt_stage1.pth
Finally, we perform policy adaptation to generalize the learned skills to novel tasks. We list the trained models across various downstream tasks.
- Skill Composition
- Follow + Carry: tokenhsi/ckpt_stage2_comp_traj_carry.pth
- Sit + Carry: tokenhsi/ckpt_stage2_comp_sit_carry.pth
- Climb + Carry: tokenhsi/ckpt_stage2_comp_climb_carry.pth
- Object Shape Variation
- Chair: tokenhsi/ckpt_stage2_objShape_chair.pth
- Table: tokenhsi/ckpt_stage2_objShape_table.pth
- Terrain Shape Variation
- Follow: tokenhsi/ckpt_stage2_terrainShape_traj.pth
- Carry: tokenhsi/ckpt_stage2_terrainShape_carry.pth
- Long-horizon Task Completion
- tokenhsi/ckpt_stage2_long_task.pth
- Skill Composition
github repo: https://github.com/liangpan99/TokenHSI
arXiv paper: https://arxiv.org/abs/2503.19901