Instructions to use RobotisSW/Task905-943-GR00T-VisionKL-Tactile with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use RobotisSW/Task905-943-GR00T-VisionKL-Tactile with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("RobotisSW/Task905-943-GR00T-VisionKL-Tactile", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Task 905 + 943: vision KL tuning followed by tactile action training
This public repository archives complete resumable checkpoints from the SH5 two-stage experiment. The training configuration is in setup/archive_policy.json.
- Vision: whole visual tower KL tuning, with the LLM and action module frozen; checkpoints at optimizer steps 500 and 1000.
- Action: use the final vision checkpoint at step 1000, freeze its complete VLM, and train the action head plus the existing finger_mlp tactile encoder; checkpoints at steps 20000, 40000, 60000, 80000 and 100000.
- Inputs: tasks 905 and 943 jointly, 174 episodes / 77860 frames; pad plus acting-hand SAM union masks; no letterbox padding.
Each checkpoint is stored under its experiment name and includes model weights, processors/statistics, configuration, optimizer/scheduler/RNG/Trainer state and a checksum manifest. Upload verification precedes local cleanup; the newest checkpoint is retained while its stage runs. After verified VLM export the completed vision checkpoint and duplicate root weights are removed locally; after action completion its final checkpoint and the VLM export remain, together with logs and receipts. Source demonstrations and videos are not uploaded here.
The two-stage run was launched on 2026-11-15 at 18:29 KST. Checkpoints will appear as training reaches each save step. Live training metrics: vision and action, starts after vision. No held-out model selection or robot deployment validation is claimed. Step 1000 is the configured vision completion point, not a measured best checkpoint.
Model tree for RobotisSW/Task905-943-GR00T-VisionKL-Tactile
Base model
nvidia/GR00T-N1.7-3B