File size: 2,658 Bytes
b232be0 4dc0c31 b232be0 4004fec | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | ---
library_name: motius
pipeline_tag: other
tags:
- motion-to-text
- motion-captioning
- humanml3d
- vermo
---
<h1 align="center">VerMo Model Card</h1>
<p align="center">
<strong>A Motius-native autoregressive motion-language baseline aligned on HumanML3D captions.</strong>
</p>
<p align="center">
<a href="https://github.com/ZeyuLing/Motius/tree/main/motius/models/vermo">Motius Implementation</a> |
<a href="https://huggingface.co/ZeyuLing/Motius-VerMo-HumanML3D">Motius Checkpoint</a>
</p>
VerMo is a Motius-native research baseline rather than a reproduction of an
external paper. The released M2T checkpoint uses a Llama-3.2-1B-Instruct
language backbone, a 16K motion tokenizer, and an explicit SMPL-22 motion
processor. No external paper or original repository is claimed for this row.
## Release Snapshot
| Item | Value |
| ---- | ----- |
| Released task | M2T |
| Evaluation input | HumanML3D-263, converted through the Motius SMPL-22 bridge |
| Native motion representation | VerMo-138, 20 fps |
| Motion tokenizer | 16K VQ motion tokenizer |
| Language backbone | Llama-3.2-1B-Instruct |
| Checkpoint | [`ZeyuLing/Motius-VerMo-HumanML3D`](https://huggingface.co/ZeyuLing/Motius-VerMo-HumanML3D) |
| Pipeline | `motius.pipelines.vermo.VermoPipeline` |
## Usage
```python
import numpy as np
from motius.pipelines.vermo import VermoPipeline
pipe = VermoPipeline.from_pretrained(
"ZeyuLing/Motius-VerMo-HumanML3D",
bundle_kwargs={"device": "cuda"},
smpl_model_dir="checkpoints/body_models/smpl",
)
motion = np.load("sample.npy") # denormalized HumanML3D-263
caption = pipe.infer_m2t([motion], lengths=[len(motion)])[0]
```
## M2T Evaluation
| Protocol | Samples | BLEU-4 | ROUGE-L | CIDEr | BERT F1 | R@1 | R@2 | R@3 | Matching |
| -------- | ------: | -----: | ------: | ----: | -------: | --: | --: | --: | -------: |
| [HumanML3D M2T](../tasks/m2t.md) | 4,400 | - | - | - | - | - | - | - | - |
## Motion Representation
VerMo-138 stores absolute root translation (3), frame-to-frame root
translation (3), and 22 local joint rotations in column-major 6D form (132).
HumanML3D inputs are recovered to SMPL-22 joints, solved to `motion135` with
position IK, then repacked explicitly from row-major to column-major 6D.
## Motius Components
| Component | Path |
| --------- | ---- |
| Pipeline | `motius/pipelines/vermo/pipeline.py` |
| Bundle | `motius/models/vermo/bundle.py` |
| Processor | `motius/models/vermo/processor.py` |
| Motion tasks | `motius/models/vermo/task_utils/` |
## Direct Loading
```python
from motius import Pipeline
pipeline = Pipeline.from_pretrained("ZeyuLing/Motius-VerMo-HumanML3D")
```
|