README.md

---
license: mit
library_name: pytorch
tags:
  - 3d-gaussian-splatting
  - animatable-avatar
  - mixture-of-experts
  - 3d-human-reconstruction
  - non-rigid-deformation
---

# AvatarMoE

**Decomposing Non-Rigid Deformation with Part-Aware Experts for 3DGS Avatars**

Hyeri Yang, Junyoung Hong, Shinwoong Kim, Kyungjae Lee
*Computers & Graphics, 2026*

[Paper (DOI)](https://doi.org/10.1016/j.cag.2026.104597) · [Project Page](https://codinghye.github.io/AvatarMoE/) · [Code](https://github.com/milab-yongin/AvatarMoE)

---

## Overview

AvatarMoE is a part-aware Mixture-of-Experts (MoE) framework for animatable 3D
Gaussian Splatting avatars. Instead of modeling non-rigid deformation with a
single global network — which produces tearing and stretching artifacts in
complex poses — AvatarMoE decomposes the body into 24 joint-centric regions,
each handled by a lightweight specialized expert:

- **Dynamic GMM-based gating** allocates expert influence according to the input pose.
- **Hybrid expert architecture** shares a HashGrid encoder across 24 lightweight MLP decoders.
- **Intra-Expert Coherence Loss** regularizes each expert's deformation field for robust OOD poses.

This repository hosts the **trained checkpoints**. Code, installation, and
training/evaluation instructions live in the
[GitHub repository](https://github.com/milab-yongin/AvatarMoE); the core model
is in `models/non_rigid.py`.

## Checkpoints

Each subject is a folder containing the checkpoint (`ckpt<iters>.pth`) and the
resolved training config (`.hydra/`):

```
checkpoints/
├── zjumocap_377_mono-best/
│   ├── ckpt8000.pth
│   └── .hydra/config.yaml
├── zjumocap_386_mono-best/
├── ...
├── zjumocap_377_refine-best/
│   ├── ckpt8000.pth
│   └── .hydra/config.yaml
├── zjumocap_386_refine-best/
├── ...
├── ps_female_3-best/
│   ├── ckpt15000.pth
│   └── .hydra/config.yaml
└── ...
```

- **ZJU-MoCap:** 377, 386, 387, 392, 393, 394 (8,000 iters)
- **People-Snapshot:** female-3, female-4, male-3, male-4 (15,000 iters)

## Usage

Checkpoints require the AvatarMoE code and its CUDA extensions — clone the repo first:

```bash
git clone --recursive https://github.com/milab-yongin/AvatarMoE.git
cd AvatarMoE
conda env create -f environment.yml
conda activate avatarmoe
pip install submodules/diff-gaussian-rasterization
pip install submodules/simple-knn
```

Download a checkpoint from this repo:

```python
from huggingface_hub import snapshot_download

local_dir = snapshot_download(
    repo_id="<your-username>/AvatarMoE",
    allow_patterns="checkpoints/ps_female_3-best/*",
)
print(local_dir)  # contains ckpt<iters>.pth and .hydra/config.yaml
```

Then evaluate / render following the repo instructions, e.g.:

```bash
python render.py mode=test dataset.test_mode=view dataset=zjumocap_377_mono
```

## Environment

Ubuntu 22.04, Python 3.10, PyTorch 2.1.2 + CUDA 11.8, single NVIDIA RTX 4090
(24 GB). Trained with Adam and an exponential LR schedule (γ = 0.1); 8,000
iterations for ZJU-MoCap and 15,000 for People-Snapshot.

## Results

On People-Snapshot, AvatarMoE reaches 32.45 PSNR — the best among the compared
baselines — while remaining competitive on SSIM and LPIPS, and substantially
reduces geometric tearing and flickering under OOD poses relative to the
3DGS-Avatar baseline. See the paper for full quantitative tables.

## Datasets

Prepare datasets following the official protocols of
[3DGS-Avatar](https://github.com/mikeqzy/3dgs-avatar-release) and
[ARAH](https://github.com/taconite/arah-release). SMPL models must be obtained
separately from the [SMPL](https://smpl.is.tue.mpg.de/) and
[SMPLify](https://smplify.is.tue.mpg.de/) project pages under their own licenses.

## License

Released under the MIT License. Some third-party dependencies and submodules
(SMPL, 3DGS rasterization, etc.) are subject to their own licenses.

## Citation

```bibtex
@article{yang2026avatarmoe,
  title={AvatarMoE: Decomposing non-rigid deformation with part-aware experts for 3DGS avatars},
  author={Yang, Hyeri and Hong, Junyoung and Kim, Shinwoong and Lee, Kyungjae},
  journal={Computers \& Graphics},
  pages={104597},
  year={2026},
  publisher={Elsevier}
}
```

## Acknowledgement

Built upon 3D Gaussian Splatting, 3DGS-Avatar, ARAH, smplx, Anim-NeRF, and GART.