---
license: other
library_name: pytorch
tags:
  - co-speech-gesture-generation
  - co-speech-motion-generation
  - speech-driven-motion
  - human-motion-generation
  - beat2
  - smpl-x
  - semtalk
datasets:
  - H-Liu1997/BEAT2
---

# SemTalk official checkpoints

Official checkpoints for **SemTalk: Holistic Co-speech Motion Generation with Frame-level Semantic Emphasis** (ICCV 2025), a semantic-aware approach to holistic **co-speech gesture generation**.

- [Paper](https://arxiv.org/abs/2412.16563) · [Hugging Face Paper page](https://huggingface.co/papers/2412.16563)
- [Project page](https://xiangyuezhang.com/SemTalk/) · [Code](https://github.com/Xiangyue-Zhang/SemTalk)
- [Published version](https://doi.org/10.1109/ICCV51701.2025.01277)
- [Generated inference data](https://huggingface.co/datasets/X-Zhang/SemTalk-Inference-Data)

## Files

| Archive | Protocol | Contents |
| --- | --- | --- |
| `weights_semtalk_no_smplx.zip` | Speaker 2, paper protocol | Original pretrained representation models and SemTalk checkpoints; SMPL-X files excluded |
| `SemTalk_all_speakers_weights_25spk.zip` | 25 English BEAT2 speakers | Five representation models and selected Base/Sparse checkpoints |

The two archives implement different training protocols and should not be treated as a controlled single-speaker versus multi-speaker ablation. See the code repository for installation paths and configuration details.

Verify both downloads with `sha256sum -c SHA256SUMS`.

## Reported results

| Protocol | Speakers | FGD ↓ | BC ↑ | DIV ↑ | MSE ↓ | LVD ↓ |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Speaker 2 (paper) | 1 | 0.4278 | 0.7770 | 12.910 | 6.153e-8 | 6.938e-5 |
| Released all-speaker checkpoint | 25 | 0.3556 | 0.5097 | 8.409 | 4.439e-8 | 1.435e-5 |

The first row is from the paper. The second row is from the released all-speaker checkpoint. Their protocols differ.

## Terms

This repository mirrors research artifacts released by the authors. No new license is granted by this model card. The Hugging Face archives intentionally exclude `SMPLX_NEUTRAL_2020.npz`; obtain SMPL-X directly from the [official source](https://smpl-x.is.tue.mpg.de/) under its terms. Code, BEAT2, pretrained encoders, and other third-party assets remain subject to their respective terms.

## Citation

```bibtex
@inproceedings{zhang2025semtalk,
  title={SemTalk: Holistic Co-speech Motion Generation with Frame-level Semantic Emphasis},
  author={Zhang, Xiangyue and Li, Jianfang and Zhang, Jiaxu and Dang, Ziqiang and Ren, Jianqiang and Bo, Liefeng and Tu, Zhigang},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={13761--13771},
  year={2025},
  doi={10.1109/ICCV51701.2025.01277}
}
```