zjh21 nielsr HF Staff commited on
Commit
a27f3f5
·
1 Parent(s): 4d09b0d

Add pipeline tag, library name and link to paper (#1)

Browse files

- Add pipeline tag, library name and link to paper (1aeb349d08b5035921f25bf6d2a604997fdb64ec)


Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +15 -3
README.md CHANGED
@@ -1,18 +1,30 @@
1
  ---
2
  license: mit
 
 
3
  ---
4
 
5
  <h1 align="center">UNICA: A Unified Neural Framework for Controllable 3D Avatars</h1>
6
 
7
  <p align="center">
8
  <a href="https://github.com/zjh21/UNICA"><img src="https://img.shields.io/badge/GitHub-Code-blue?logo=github&logoColor=white" alt="GitHub"></a>
9
- <!-- <a href="https://arxiv.org/abs/XXXX.XXXXX"><img src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv&logoColor=white" alt="arXiv"></a> -->
10
  </p>
11
 
12
  <p align="center">
13
- <img src="assets/teaser.png" alt="Teaser" width="100%">
14
  </p>
15
 
16
  ## Abstract
17
 
18
- Controllable 3D human avatars have found widespread applications in 3D games, the metaverse, and AR/VR scenarios. The conventional approach to creating such a 3D avatar requires a lengthy, intricate pipeline encompassing appearance modeling, motion planning, rigging, and physical simulation. In this paper, we introduce UNICA (UNIfied neural Controllable Avatar), a skeleton-free generative model that unifies all avatar control components into a single neural framework. Given keyboard inputs akin to video game controls, UNICA generates the next frame of a 3D avatar's geometry through an action-conditioned diffusion model operating on 2D position maps. A point transformer then maps the resulting geometry to 3D Gaussian Splatting for high-fidelity free-view rendering. Our approach naturally captures hair and loose clothing dynamics without manually designed physical simulation, and supports extra-long autoregressive generation. To the best of our knowledge, UNICA is the first model to unify the workflow of "motion planning, rigging, physical simulation, and rendering".
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ pipeline_tag: image-to-3d
4
+ library_name: diffusers
5
  ---
6
 
7
  <h1 align="center">UNICA: A Unified Neural Framework for Controllable 3D Avatars</h1>
8
 
9
  <p align="center">
10
  <a href="https://github.com/zjh21/UNICA"><img src="https://img.shields.io/badge/GitHub-Code-blue?logo=github&logoColor=white" alt="GitHub"></a>
11
+ <a href="https://huggingface.co/papers/2604.02799"><img src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv&logoColor=white" alt="arXiv"></a>
12
  </p>
13
 
14
  <p align="center">
15
+ <img src="https://huggingface.co/zjh21/UNICA/resolve/main/assets/teaser.png" alt="Teaser" width="100%">
16
  </p>
17
 
18
  ## Abstract
19
 
20
+ Controllable 3D human avatars have found widespread applications in 3D games, the metaverse, and AR/VR scenarios. The conventional approach to creating such a 3D avatar requires a lengthy, intricate pipeline encompassing appearance modeling, motion planning, rigging, and physical simulation. In this paper, we introduce **UNICA** (**UNI**fied neural **C**ontrollable **A**vatar), a skeleton-free generative model that unifies all avatar control components into a single neural framework. Given keyboard inputs akin to video game controls, UNICA generates the next frame of a 3D avatar's geometry through an action-conditioned diffusion model operating on 2D position maps. A point transformer then maps the resulting geometry to 3D Gaussian Splatting for high-fidelity free-view rendering. Our approach naturally captures hair and loose clothing dynamics without manually designed physical simulation, and supports extra-long autoregressive generation.
21
+
22
+ ## Resources
23
+ - **Paper:** [UNICA: A Unified Neural Framework for Controllable 3D Avatars](https://huggingface.co/papers/2604.02799)
24
+ - **GitHub Repository:** [https://github.com/zjh21/UNICA](https://github.com/zjh21/UNICA)
25
+
26
+ ## Installation and Usage
27
+
28
+ Please refer to the official [GitHub repository](https://github.com/zjh21/UNICA) for detailed installation instructions and inference scripts. The pipeline generally involves two stages:
29
+ 1. **Geometry Generation:** Using the action-conditioned diffusion model to generate position maps.
30
+ 2. **Appearance Mapping:** Mapping geometry to 3D Gaussian Splatting via a point transformer for rendering.