Hugging Face's logo

CODINGHYE
/
AvatarMoE

PyTorch
3d-gaussian-splatting
animatable-avatar
mixture-of-experts
3d-human-reconstruction
non-rigid-deformation
Model card Files
xet
 Community
AvatarMoE
File size: 4,373 Bytes
190480e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
---
license: mit
library_name: pytorch
tags:
  - 3d-gaussian-splatting
  - animatable-avatar
  - mixture-of-experts
  - 3d-human-reconstruction
  - non-rigid-deformation
---

# AvatarMoE

**Decomposing Non-Rigid Deformation with Part-Aware Experts for 3DGS Avatars**

Hyeri Yang, Junyoung Hong, Shinwoong Kim, Kyungjae Lee
*Computers & Graphics, 2026*

[Paper (DOI)](https://doi.org/10.1016/j.cag.2026.104597) Β· [Project Page](https://codinghye.github.io/AvatarMoE/) Β· [Code](https://github.com/milab-yongin/AvatarMoE)

---

## Overview

AvatarMoE is a part-aware Mixture-of-Experts (MoE) framework for animatable 3D
Gaussian Splatting avatars. Instead of modeling non-rigid deformation with a
single global network β€” which produces tearing and stretching artifacts in
complex poses β€” AvatarMoE decomposes the body into 24 joint-centric regions,
each handled by a lightweight specialized expert:

- **Dynamic GMM-based gating** allocates expert influence according to the input pose.
- **Hybrid expert architecture** shares a HashGrid encoder across 24 lightweight MLP decoders.
- **Intra-Expert Coherence Loss** regularizes each expert's deformation field for robust OOD poses.

This repository hosts the **trained checkpoints**. Code, installation, and
training/evaluation instructions live in the
[GitHub repository](https://github.com/milab-yongin/AvatarMoE); the core model
is in `models/non_rigid.py`.

## Checkpoints

Each subject is a folder containing the checkpoint (`ckpt<iters>.pth`) and the
resolved training config (`.hydra/`):

```
checkpoints/
β”œβ”€β”€ zjumocap_377_mono-best/
β”‚   β”œβ”€β”€ ckpt8000.pth
β”‚   └── .hydra/config.yaml
β”œβ”€β”€ zjumocap_386_mono-best/
β”œβ”€β”€ ...
β”œβ”€β”€ zjumocap_377_refine-best/
β”‚   β”œβ”€β”€ ckpt8000.pth
β”‚   └── .hydra/config.yaml
β”œβ”€β”€ zjumocap_386_refine-best/
β”œβ”€β”€ ...
β”œβ”€β”€ ps_female_3-best/
β”‚   β”œβ”€β”€ ckpt15000.pth
β”‚   └── .hydra/config.yaml
└── ...
```

- **ZJU-MoCap:** 377, 386, 387, 392, 393, 394 (8,000 iters)
- **People-Snapshot:** female-3, female-4, male-3, male-4 (15,000 iters)

## Usage

Checkpoints require the AvatarMoE code and its CUDA extensions β€” clone the repo first:

```bash
git clone --recursive https://github.com/milab-yongin/AvatarMoE.git
cd AvatarMoE
conda env create -f environment.yml
conda activate avatarmoe
pip install submodules/diff-gaussian-rasterization
pip install submodules/simple-knn
```

Download a checkpoint from this repo:

```python
from huggingface_hub import snapshot_download

local_dir = snapshot_download(
    repo_id="<your-username>/AvatarMoE",
    allow_patterns="checkpoints/ps_female_3-best/*",
)
print(local_dir)  # contains ckpt<iters>.pth and .hydra/config.yaml
```

Then evaluate / render following the repo instructions, e.g.:

```bash
python render.py mode=test dataset.test_mode=view dataset=zjumocap_377_mono
```

## Environment

Ubuntu 22.04, Python 3.10, PyTorch 2.1.2 + CUDA 11.8, single NVIDIA RTX 4090
(24 GB). Trained with Adam and an exponential LR schedule (Ξ³ = 0.1); 8,000
iterations for ZJU-MoCap and 15,000 for People-Snapshot.

## Results

On People-Snapshot, AvatarMoE reaches 32.45 PSNR β€” the best among the compared
baselines β€” while remaining competitive on SSIM and LPIPS, and substantially
reduces geometric tearing and flickering under OOD poses relative to the
3DGS-Avatar baseline. See the paper for full quantitative tables.

## Datasets

Prepare datasets following the official protocols of
[3DGS-Avatar](https://github.com/mikeqzy/3dgs-avatar-release) and
[ARAH](https://github.com/taconite/arah-release). SMPL models must be obtained
separately from the [SMPL](https://smpl.is.tue.mpg.de/) and
[SMPLify](https://smplify.is.tue.mpg.de/) project pages under their own licenses.

## License

Released under the MIT License. Some third-party dependencies and submodules
(SMPL, 3DGS rasterization, etc.) are subject to their own licenses.

## Citation

```bibtex
@article{yang2026avatarmoe,
  title={AvatarMoE: Decomposing non-rigid deformation with part-aware experts for 3DGS avatars},
  author={Yang, Hyeri and Hong, Junyoung and Kim, Shinwoong and Lee, Kyungjae},
  journal={Computers \& Graphics},
  pages={104597},
  year={2026},
  publisher={Elsevier}
}
```

## Acknowledgement

Built upon 3D Gaussian Splatting, 3DGS-Avatar, ARAH, smplx, Anim-NeRF, and GART.