File size: 2,792 Bytes
03c6686 595f6ba 03c6686 595f6ba 03c6686 595f6ba 03c6686 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | ---
license: other
library_name: pytorch
tags:
- co-speech-gesture-generation
- co-speech-motion-generation
- speech-driven-motion
- human-motion-generation
- masked-modeling
- beat2
- smpl-x
- echomask
datasets:
- H-Liu1997/BEAT2
---
# EchoMask official checkpoints
Official checkpoints for **EchoMask: Speech-Queried Attention-based Mask Modeling for Holistic Co-Speech Motion Generation** (ACM Multimedia 2025), a speech-conditioned approach to holistic **co-speech gesture generation**.
- [Paper](https://arxiv.org/abs/2504.09209) · [Hugging Face Paper page](https://huggingface.co/papers/2504.09209)
- [Project page](https://xiangyuezhang.com/EchoMask/) · [Code](https://github.com/Xiangyue-Zhang/EchoMask)
- [Published version](https://doi.org/10.1145/3746027.3754847)
- [Generated inference data](https://huggingface.co/datasets/X-Zhang/EchoMask-Inference-Data)
## Files
| Archive | Protocol | Contents |
| --- | --- | --- |
| `weights_echomask_no_smplx.zip` | Speaker 2, paper protocol | Original pretrained representation models and EchoMask checkpoint; SMPL-X files excluded |
| `EchoMask_all_speakers_weights_25spk.zip` | 25 English BEAT2 speakers | Reusable representation models and selected EchoMask checkpoint |
The two archives implement different training protocols and should not be treated as a controlled single-speaker versus multi-speaker ablation. See the code repository for installation paths and configuration details.
Verify both downloads with `sha256sum -c SHA256SUMS`.
## Reported results
| Protocol | Speakers | FGD ↓ | BC ↑ | DIV ↑ | MSE ↓ | LVD ↓ |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Speaker 2 (paper) | 1 | 0.4623 | 0.7738 | 13.370 | 6.761e-8 | 7.290e-5 |
| Released all-speaker checkpoint | 25 | 0.5656 | 0.4951 | 9.299 | 4.700e-8 | 6.090e-5 |
The first row is from the paper. The second row is from the released all-speaker checkpoint. Their protocols differ.
## Terms
This repository mirrors research artifacts released by the authors. No new license is granted by this model card. The Hugging Face archives intentionally exclude `SMPLX_NEUTRAL_2020.npz`; obtain SMPL-X directly from the [official source](https://smpl-x.is.tue.mpg.de/) under its terms. Code, BEAT2, pretrained encoders, and other third-party assets remain subject to their respective terms.
## Citation
```bibtex
@inproceedings{zhang2025echomask,
title={EchoMask: Speech-Queried Attention-based Mask Modeling for Holistic Co-Speech Motion Generation},
author={Zhang, Xiangyue and Li, Jianfang and Zhang, Jiaxu and Ren, Jianqiang and Bo, Liefeng and Tu, Zhigang},
booktitle={Proceedings of the 33rd ACM International Conference on Multimedia},
pages={10827--10836},
year={2025},
doi={10.1145/3746027.3754847}
}
```
|