gnm / README.md
linoyts's picture
linoyts HF Staff
Add direct-manipulation UI: drag on the 3D head to drive the controls (#1)
b1166a0
|
Raw
History Blame Contribute Delete
3.99 kB
---
title: GNM Head Interactive 3D Face Model
emoji: 🧬
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 6.20.0
app_file: app.py
short_description: Reactive 3D demo of Google's GNM parametric head model
python_version: "3.12"
startup_duration_timeout: 30m
license: apache-2.0
---
# GNM Head — Interactive 3D Face Model
A reactive Gradio demo of **[GNM](https://github.com/google/GNM)** (Generative
aNthropometric Model), Google's state-of-the-art parametric 3D statistical model
of the human head. This Space replicates the official interactive slider demo
from [`gnm/shape/demos/gnm_head_demo.ipynb`](https://github.com/google/GNM/blob/main/gnm/shape/README.md#demo):
move any slider and the 3D head mesh regenerates live.
## Controls
Control labels are named to match exactly what each one changes in the model:
- **Identity** — first 10 components of the linear identity basis, i.e. overall
**head shape**.
- **Expression** — selected blendshape components per region, named as in the
upstream `gnm_head_demo.ipynb`: **left eye**, **right eye**, **mouth** (the
`lower_face_region_*` components the notebook exposes as its `mouth` group),
**tongue**, and **pupil dilation**. The Expression tab also has a
**keyword-expression** control (mirroring the reference
[semantic sampling demo](https://github.com/google/GNM/blob/main/gnm/shape/README.md#demo-1)):
pick a named expression (happy, surprise, wink, smile, ...) — or several to
blend — set the **strength**, and hit **Apply** to drive the expression
sliders toward that expression via GNM's semantic CVAE sampler. It only
*sets* the slider values, so you can keep tweaking each slider by hand
afterwards.
- **Pose** — neck and head **rotations** (pitch / yaw / roll, in degrees), plus
**gaze** (eyeball pitch / yaw with a cross-eye vergence control).
- **Translation** — global position of the whole head (X left/right,
Y up/down, Z forward/back).
The 3D view opens **front-facing** and keeps your current camera position and
orientation as you move the sliders. The viewer is a small custom Three.js
component with a persistent scene: each slider change only streams the new
vertex positions, which are written straight into the existing geometry buffer,
so the mesh **morphs in place** — it is never reloaded/remounted, so there is
no camera reset and no flicker.
The model is pure NumPy (a few matrix multiplies, ~40 ms per evaluation), so the
Space runs on `cpu-basic` with no GPU required.
## Direct manipulation
You can also **grab the 3D head directly** instead of only moving sliders — hover
any region to see what it does (the head is colour-coded by control region):
- **Eye** — drag ↕ to open / close that eye, ↔ to glance (aim the eyes).
- **Mouth** — drag the centre / chin **down** to open the jaw; drag a **lip
corner outward** to smile / stretch.
- **Tongue** — drag down to stick it out.
- **Skull / forehead** — drag to turn & nod the head; **gaze** aims when you grab
an eyeball.
- **Ear** — drag in / out (drives *Head shape 8*); **Cheek** — drag to stretch
the face (drives *Head shape 7*).
- **Alt + drag** free-sculpts the surface under the cursor with a soft falloff
brush (nose, brow, jawline). These are geometric offsets on top of the model —
a **↺ Reset shaping** button clears them.
- **Shift + drag** repositions the head, **right-drag** (or drag empty space)
orbits the camera, **scroll** zooms.
Each region drives the very same slider(s) — hovering one glows the matching
control and its tab — so the sliders and the head stay in sync. Every gesture
runs through the same reactive pipeline as the sliders; slider-driving drags are
throttled and run off the queue so the mesh tracks the cursor in real time.
## Credits
Model and reference demo: [google/GNM](https://github.com/google/GNM),
Apache 2.0. This Space vendors the upstream `gnm.shape` NumPy backend and its
model data (`gnm_head.npz`).