README.md

---
library_name: pytorch
license: other
license_name: upstream-and-data-licenses-apply
tags:
  - motion-generation
  - text-to-motion
  - motion-to-motion
  - smpl-x
  - vimogen
---

# ViMoGen-Hand-456 checkpoints

This model repository stores the selected large checkpoints for the
ViMoGen-Hand-456 experiments. The project extends the original ViMoGen 276D
body representation with 180D articulated hand rotations:

```text
motion456 = body276 || left_hand90 || right_hand90
```

The complete source, experiment configs, training records, reports and
evaluation artifacts are distributed separately in the `ViMoGen_Hand` source
repository. These weights require the original ViMoGen implementation and its
licensed dependencies; they are not standalone Transformers checkpoints.

## Representation

| Channels | Meaning |
|---|---|
| `0:126` | 21 non-root body-joint rotations in Rot6D |
| `126:192` | 22 global body-joint XYZ positions |
| `192:258` | 22 body-joint XYZ velocities |
| `258:264` | root/global orientation in Rot6D |
| `264:270` | root orientation velocity in Rot6D |
| `270:273` | root translation XYZ |
| `273:276` | root translation velocity XYZ |
| `276:366` | 15 left-hand joint rotations in Rot6D |
| `366:456` | 15 right-hand joint rotations in Rot6D |

The hand block contains local rotations only. It does not add hand joint
positions, hand velocities, face parameters or body shape parameters.

TM2M body conditioning uses a 138D reference consisting of body channels
`0:126` and `258:270`. Hand conditioning uses the full 180D hand pose. Unified
models concatenate both conditions into a 318D reference.

## Files

| Remote path | Purpose | Size | SHA256 |
|---|---|---:|---|
| `hand180/step_00014000/model.pt` | Independent Hand180 model used by Concat, as the hand teacher, and by Two-Expert | 9,093,836,032 B | `daa66c4d29d2b94b33da392e3d4920f067fcd37965daadcadd47cf0966066f7b` |
| `unified_distill/stage2/step_00015000/model.pt` | Selected single-network Stage2 unified 456D model | 9,098,076,480 B | `443843959903f4a20639ce9d50e3cd023de694f51628c88f0998da191a381492` |
| `direct456/step_00020000/model.pt` | Selected Direct456 all-parameter ablation | 9,098,076,480 B | `e63221693337d4b2b07e8dd124ea992d5a0b798884bb6ec51e383f1e3a8a69ec` |

All three are RAW FP32 model states used by the archived primary comparisons.
EMA weights and optimizer states are intentionally not included.

The original ViMoGen Body276 checkpoint must be downloaded from the official
ViMoGen release. The compact Two-Expert fusion state is stored in the source
repository because it is only 7.2 MB. The 18.2 GB full Two-Expert checkpoint is
not uploaded because it duplicates the frozen Body276 and Hand180 experts.

## Architectures represented here

### Hand180

The original ViMoGen TM2M topology is retained but its motion/reference/output
domains are changed to `180/180/180`. It contains 30 main Transformer blocks,
15 reference self-attention blocks, hidden width 1536, 12 heads and FFN width
8960. All Hand180 parameters were trained with hand-only flow matching.

### Stage2 unified

A single original-topology ViMoGen Student uses `456/318/456` I/O. It was first
distilled from frozen Body276 and Hand180 teachers, then trained with equal
body/hand GT flow losses plus a 0.1 teacher anchor. The selected raw checkpoint
is step 15000 of Stage2.

### Direct456

A single original-topology `456/318/456` ViMoGen model was initialized from
all shape-compatible Body276 weights and then trained end-to-end on equal
body/hand GT flow losses without teachers. This checkpoint is retained as the
direct all-parameter ablation.

## Training data and common settings

- 25,861 processed Motion-X/Motion-X++ motions;
- 3,194,032 frames at 20 FPS;
- source SMPL-X 322D converted to `[T,456]`;
- T2M/TM2M sampling ratio 20/80;
- AdamW, betas `(0.9, 0.95)`, weight decay `1e-4`;
- BF16 compute with FP32 master parameters, optimizer and EMA;
- 7-GPU FSDP, global batch 112;
- 50 flow sampling steps and denoising strength 0.7 for primary evaluation.

The dataset and SMPL-X body model are not redistributed here.

## Download

```bash
hf download Agentc1/ViMoGen_Hand_456 \
  hand180/step_00014000/model.pt \
  unified_distill/stage2/step_00015000/model.pt \
  direct456/step_00020000/model.pt \
  checkpoint_manifest.json \
  --local-dir checkpoints_external
```

Verify SHA256 values against `checkpoint_manifest.json` before evaluation.

## Main benchmark result

Mean raw-motion MSE on the archived 16-case strict GRAB OOD TM2M suite:

| Method | Full | Body | Hand |
|---|---:|---:|---:|
| Frozen Two-Expert | 0.017182 | 0.026319 | 0.003171 |
| Body276 + Hand180 Concat | 0.017617 | 0.026920 | 0.003353 |
| Stage2 unified | 0.021346 | 0.032927 | 0.003589 |
| Direct456 | 0.021307 | 0.031046 | 0.006373 |

The historical method runs did not use a perfectly identical seed map, so the
small Two-Expert-versus-Concat difference should be treated as a trend rather
than a statistical significance claim.

## Licenses and attribution

Use of these checkpoints is subject to the licenses and terms of the original
ViMoGen/Wan components, Motion-X/Motion-X++ data, SMPL-X models and any source
datasets. This repository does not grant redistribution rights for those
dependencies. Please cite the original ViMoGen, Motion-X and applicable source
dataset papers when using the models.