SeMoCo: Semantic Motion Tokens with Body-Structured Flow Matching

Code arXiv Project

Introduction

SeMoCo couples a semantically supervised motion codec with body-structured flow matching for text-to-motion generation. An autoregressive model predicts a semantic code sequence from text, and a conditional flow generates continuous kinematic features for the root, torso, and limbs. The semantic codes and kinematic features are decoded together into motion.

This repository contains the pretrained HumanML3D and Ω-MotionVerse models for the main results. Each directory has the tokenizer, semantic model, six-part flow, and the normalization statistics used with those weights.

Model Files

File Contents
humanml3d/tokenizer.pt HumanML3D codec
humanml3d/semantic_ar.pt HumanML3D semantic model
humanml3d/flow.pt HumanML3D six-part flow
humanml3d/kin_stats.npz, humanml3d/feature_stats.npz Latent and Guo263 normalization
omega/tokenizer.pt Ω-MotionVerse codec
omega/semantic_ar.pt Ω-MotionVerse semantic model
omega/flow.pt Ω-MotionVerse six-part flow
omega/kin_stats.npz, omega/feature_stats.npz Latent and UMR499 normalization

Evaluation

Dataset FID R@1 R@2 R@3 MM-Dist
HumanML3D 0.0327 0.5561 0.7473 0.8365 2.750
Ω-MotionVerse 0.01238 0.7325 0.8143 0.8446 0.963

HumanML3D reports the mean of 20 repeats with the checkpoint held fixed. Ω-MotionVerse is one evaluation of the full test set.

Usage

hf download poisonousID/SeMoCo --local-dir checkpoints

Then generate with the configs from the code repository:

semoco-generate --config configs/humanml3d_inference.yaml \
  --inputs data/prompts.npz --output runs/motion.npz

Use configs/omega_inference.yaml for Ω-MotionVerse.

License

Apache-2.0.

Citation

@article{semoco,
  title   = {SeMoCo: Semantic Motion Tokens with Body-Structured Flow Matching},
  author  = {Huang, Tianlv and Guo, Hetian and Cai, Ziyi and Liu, Yu and Peng, Yingzhi and Wang, Song and Fan, Zipei and Song, Xuan and Wu, Guangming and Zheng, Xin},
  journal = {arXiv preprint arXiv:2608.24334},
  year    = {2026}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for poisonousID/SeMoCo