| --- |
| license: other |
| license_name: mixed |
| tags: |
| - robotics |
| - vision-language-action |
| - hand-pose |
| --- |
| |
| # VLA-HAND baselines |
|
|
| Checkpoints of the two external baselines trained against V4 in the VLA-HAND project. |
| Both were trained and tested on the same three domains (GigaHands, OakInk2, TACO), splits, |
| normalisation statistics and test protocol as V4. Each is the checkpoint with the lowest |
| validation loss, the one all reported test numbers come from. |
|
|
| | folder | model | init | action space | step | license | |
| |---|---|---|---|---|---| |
| | `vitra_original/` | PaliGemma2-3B + DiT/DDPM action head (VITRA) | `VITRA-VLA/VITRA-VLA-3B` | MANO joint angles | 10,000 | Gemma Terms of Use, see `vitra_original/NOTICE.md` | |
| | `smolvla_official/` | SmolVLA (LeRobot `SmolVLAPolicy`) | SmolVLM2-500M | 21-keypoint hand pose | 18,000 | Apache-2.0 | |
|
|
| ## Test results |
|
|
| Test split, the first 8 control steps of each test window, three domains pooled. |
| Wrist translation in cm, wrist rotation in degrees (geodesic), fingers in cm; lower is better. |
|
|
| | model | wrist translation | wrist rotation | fingers | |
| |---|---|---|---| |
| | SmolVLA | 0.7518 | 3.6235 | 1.0094 | |
| | VITRA | 1.0298 | 4.7789 | 1.7933 | |
|
|
| VITRA predicts joint angles, so its finger error is measured after MANO forward kinematics |
| and includes an FK floor that SmolVLA does not have. The finger columns are not directly |
| comparable. |
|
|
| ## Files |
|
|
| Each folder holds `best.pt` and `training_config.json`, the run config stored inside the |
| checkpoint (its paths refer to the training machine). `best.pt` is a `torch.save` dict with: |
| `step`, `model` (state dict), `config`, `val` (validation metrics at that step), and |
| `action_mean` / `action_std`, the normalisation statistics the model was trained with. |
| There is no optimiser state. |
|
|
| ## Code |
|
|
| Training and test code: https://github.com/kaichen-z/VLA-HAND/tree/Kai_final/yuzhench_distill_editor/baselines |
| (commit `baee56f`). Load with the matching package's `Model` and `load_pretrained(path)`. |
|
|
| ## Full T-ReX human-action checkpoint |
|
|
| [model/trex_full_egotouch](model/trex_full_egotouch) contains all inference weights, the processor, and standalone loading code. |
| No separate upstream checkpoint download is required. This is the no-touch EgoTouch adaptation, not native tactile T-ReX. |
| Its checkpoint uses `human_trex.full.v1`, not the legacy baseline format described above. |
|
|
| ## Tachin adaptation checkpoints |
|
|
| [model/tachin_adaptation_20260922](model/tachin_adaptation_20260922) contains validation-selected SmolVLA, VITRA, and HumanTReX checkpoints after Tachin adaptation. |
| These are trained-parameter checkpoints, not standalone full models. The release documents their frozen initialization dependencies. |
| No tactile editor or full test results are included. The existing three-domain and EgoTouch releases remain unchanged. |
|
|
| ## EgoTouch T-ReX editor replica |
|
|
| [model/trex_editor_egotouch_replica_20260930](model/trex_editor_egotouch_replica_20260930) contains the validation-selected tactile editor from the September 30 video-generation replica. This is not the original paper-run checkpoint. It is separate from the full HumanTReX backbone. The package includes a strict loader and a verification record. |
|
|
