Add pipeline tag, library name and link to paper (#1)
Browse files- Add pipeline tag, library name and link to paper (1aeb349d08b5035921f25bf6d2a604997fdb64ec)
Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -1,18 +1,30 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
|
| 5 |
<h1 align="center">UNICA: A Unified Neural Framework for Controllable 3D Avatars</h1>
|
| 6 |
|
| 7 |
<p align="center">
|
| 8 |
<a href="https://github.com/zjh21/UNICA"><img src="https://img.shields.io/badge/GitHub-Code-blue?logo=github&logoColor=white" alt="GitHub"></a>
|
| 9 |
-
<
|
| 10 |
</p>
|
| 11 |
|
| 12 |
<p align="center">
|
| 13 |
-
<img src="assets/teaser.png" alt="Teaser" width="100%">
|
| 14 |
</p>
|
| 15 |
|
| 16 |
## Abstract
|
| 17 |
|
| 18 |
-
Controllable 3D human avatars have found widespread applications in 3D games, the metaverse, and AR/VR scenarios. The conventional approach to creating such a 3D avatar requires a lengthy, intricate pipeline encompassing appearance modeling, motion planning, rigging, and physical simulation. In this paper, we introduce UNICA (
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
pipeline_tag: image-to-3d
|
| 4 |
+
library_name: diffusers
|
| 5 |
---
|
| 6 |
|
| 7 |
<h1 align="center">UNICA: A Unified Neural Framework for Controllable 3D Avatars</h1>
|
| 8 |
|
| 9 |
<p align="center">
|
| 10 |
<a href="https://github.com/zjh21/UNICA"><img src="https://img.shields.io/badge/GitHub-Code-blue?logo=github&logoColor=white" alt="GitHub"></a>
|
| 11 |
+
<a href="https://huggingface.co/papers/2604.02799"><img src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv&logoColor=white" alt="arXiv"></a>
|
| 12 |
</p>
|
| 13 |
|
| 14 |
<p align="center">
|
| 15 |
+
<img src="https://huggingface.co/zjh21/UNICA/resolve/main/assets/teaser.png" alt="Teaser" width="100%">
|
| 16 |
</p>
|
| 17 |
|
| 18 |
## Abstract
|
| 19 |
|
| 20 |
+
Controllable 3D human avatars have found widespread applications in 3D games, the metaverse, and AR/VR scenarios. The conventional approach to creating such a 3D avatar requires a lengthy, intricate pipeline encompassing appearance modeling, motion planning, rigging, and physical simulation. In this paper, we introduce **UNICA** (**UNI**fied neural **C**ontrollable **A**vatar), a skeleton-free generative model that unifies all avatar control components into a single neural framework. Given keyboard inputs akin to video game controls, UNICA generates the next frame of a 3D avatar's geometry through an action-conditioned diffusion model operating on 2D position maps. A point transformer then maps the resulting geometry to 3D Gaussian Splatting for high-fidelity free-view rendering. Our approach naturally captures hair and loose clothing dynamics without manually designed physical simulation, and supports extra-long autoregressive generation.
|
| 21 |
+
|
| 22 |
+
## Resources
|
| 23 |
+
- **Paper:** [UNICA: A Unified Neural Framework for Controllable 3D Avatars](https://huggingface.co/papers/2604.02799)
|
| 24 |
+
- **GitHub Repository:** [https://github.com/zjh21/UNICA](https://github.com/zjh21/UNICA)
|
| 25 |
+
|
| 26 |
+
## Installation and Usage
|
| 27 |
+
|
| 28 |
+
Please refer to the official [GitHub repository](https://github.com/zjh21/UNICA) for detailed installation instructions and inference scripts. The pipeline generally involves two stages:
|
| 29 |
+
1. **Geometry Generation:** Using the action-conditioned diffusion model to generate position maps.
|
| 30 |
+
2. **Appearance Mapping:** Mapping geometry to 3D Gaussian Splatting via a point transformer for rendering.
|