T2M-GPT on Full_TriVis (VSL) — stage-2 run v1
Gloss/text → pose for Vietnamese Sign Language, trained on Full_TriVis
(split_lab_front.csv, front view, lab). Architecture is T2M-GPT (CVPR 2023) adapted to
128-keypoint DWPose skeletons, with vinai/phobert-base-v2 (frozen, mean-pooled) replacing
CLIP as the text encoder.
⚠️ This is the v1 run, kept for the record — read before using
This checkpoint was abandoned during development and is superseded by a v2 run. It is published because it was asked for explicitly; it is not the model to build on.
| this v1 | the v2 run | |
|---|---|---|
| stage-2 iterations | stopped at 8,000 | 30,000 (best @ 24,000) |
| hands MPJPE (80-clip val) | 0.34756 | 0.22408 |
| len_ratio | 2.77 | 0.959 |
The decisive problem is len_ratio 2.77: generated sequences run ~2.8× longer than the
reference. v1 was selected on greedy decoding, which almost never emits the end token, so
generation runs to the block limit. Switching selection to categorial sampling fixed the
length behaviour and is what produced v2. If you need a working model, use the v2 run.
Files
| file | what |
|---|---|
net_best.pth |
stage-2 GPT, best-by-greedy-metric checkpoint (222 MB) |
net_last.pth |
stage-2 GPT, final checkpoint |
vq_net_best.pth |
stage-1 VQ-VAE (vq_vsl_front_lab), required to decode tokens |
run.log |
full training log incl. the per-eval len_ratio trace |
Stage 1 and stage 2 must be paired: the 512 codes index this specific codebook, so a different tokenizer decodes to noise without erroring.
Stage-1 config: w_body=1.0 w_face=0.5 w_hand=3.0, l1_smooth recon, masked by keypoint
validity, EMA+code-reset quantizer, nb_code=512, down_t=2. Stage-1 val hands MPJPE 0.02716.
Units
Coordinates are frame-normalized DWPose (x/W, y/H) with no body normalization, so 1.0 = one frame width. This is anisotropic (W≠H) and does not normalize signer size — for cross-dataset comparison convert to per-clip shoulder widths.
Licence
Full_TriVis derives from restricted material; this checkpoint is released CC BY-NC 4.0 for research use only.