README.md

---
title: FaceAnything
emoji: 📈
colorFrom: pink
colorTo: gray
sdk: gradio
sdk_version: 6.19.0
python_version: '3.12'
app_file: app.py
pinned: false
license: cc-by-nc-4.0
short_description: 4D face reconstruction & tracking from an image sequence
models:
  - depth-anything/DA3-GIANT-1.1
tags:
  - face
  - 4d-reconstruction
  - depth
  - normals
  - point-tracking
---

# Face Anything — Gradio demo

4D face reconstruction and tracking from **any image sequence** — upload up to 40
images **or a video** (its first 40 frames are used) — in a single feed-forward
pass. The model jointly predicts depth and **canonical facial coordinates**, from
which the demo derives:

- **Canonical 2D video** — per-frame canonical facial-coordinate map
- **Depth 2D video** — per-frame depth map (JET)
- **Normals 2D video** — per-frame surface normals (from depth)
- **3D point cloud with colorful tracks** — viewable/orbitable in the 3D viewer
  (rendered as a `.glb` on a white background), with a frame slider to scrub the
  sequence and a downloadable `.zip` containing both the track-colored point
  clouds (`tracks/`) and the plain colored point clouds (`points/`)
- *(bonus)* a 2D point-track overlay video

Reconstruction always uses the model's **predicted camera poses** (a multi-view
consistent world frame).

**Two inference modes** (the repo's `--process-mode`):

- **Joint (all-at-once)** — all frames processed together → more 3D-consistent.
- **One-by-one** — each frame processed independently → more surface detail and
  lower memory (pairs well with a higher processing resolution).

Exposed hyperparameters (defaults match the published `run_inference.py`):
processing resolution, background removal (Robust Video Matting),
depth-confidence cut, number/blur/threshold of point tracks, output FPS, and a
frame cap.

## Deploying this Space

The Gradio code (`app.py`) and the model source (`src/faceanything`,
`src/depth_anything_3`) are included here. To make the Space runnable you still
need the **checkpoint** (~15 GB), which is not committed.

### Checkpoint storage

Don't commit 15 GB into the Space repo. Put the checkpoint in a Hugging Face
**model** repo and expose it to the Space with a mounted volume (HF's current
mechanism for persisting data — the old fixed persistent-storage disk has been
superseded by volumes / storage buckets).

**Recommended — mount the model repo as a read-only volume.** The checkpoint then
appears as a plain local file; no download code, nothing to re-fetch on cold
start, and zero ZeroGPU seconds spent moving it.

```bash
# one-time: upload the checkpoint into your model repo
hf upload UmutKocasari/FaceAnything /path/to/checkpoint.pt checkpoint.pt --repo-type=model

# mount that model repo into the Space at /models (read-only) — restarts the Space
hf spaces volumes set UmutKocasari/FaceAnything \
    -v hf://models/UmutKocasari/FaceAnything:/models
```

Then set one Space variable (*Settings → Variables and secrets*):

```
FACEANYTHING_CHECKPOINT = /models/checkpoint.pt
```

Add the `HF_TOKEN` secret too if the model repo is private. Verify the mount with
`hf spaces volumes ls UmutKocasari/FaceAnything`. (Models/datasets are always
read-only mounts; only storage buckets can be mounted read-write.)

**Alternative — download at startup.** Skip the volume and set
`FACEANYTHING_CHECKPOINT_REPO = UmutKocasari/FaceAnything`; the app calls
`hf_hub_download` on the CPU node at startup. It re-downloads on each cold start
unless you back the HF cache with a read-write **Storage Bucket** volume and point
`HF_HOME` at it:

```bash
hf buckets create UmutKocasari/faceanything-cache
hf spaces volumes set UmutKocasari/FaceAnything \
    -v hf://buckets/UmutKocasari/faceanything-cache:/data
# then set Space variable:  HF_HOME = /data/.huggingface
```

*Last resort:* commit the weights via Git LFS at `checkpoints/checkpoint.pt`
(bloats the Space repo and slows every clone).

### Hardware

This needs a CUDA GPU. On **ZeroGPU**, `@spaces.GPU` is used automatically; raise
`FACEANYTHING_GPU_DURATION` (seconds, default 180) if long clips time out. On a
dedicated GPU Space, `spaces` degrades to a no-op.

The DA3 backbone config/architecture is pulled from the public model
`depth-anything/DA3-GIANT-1.1` on first run (its weights are then overwritten by
the checkpoint); it lands in the same `HF_HOME` cache.

### Environment variables

| Variable | Default | Purpose |
|---|---|---|
| `FACEANYTHING_CHECKPOINT_REPO` | — | HF repo id to download the checkpoint from |
| `FACEANYTHING_CHECKPOINT_FILE` | `checkpoint.pt` | filename within that repo |
| `FACEANYTHING_CHECKPOINT_REPO_TYPE` | `model` | `model` / `dataset` / `space` |
| `FACEANYTHING_CHECKPOINT_REVISION` | — | branch/tag/commit to pin |
| `HF_TOKEN` | — | token for a private checkpoint repo |
| `FACEANYTHING_CHECKPOINT` | `checkpoints/checkpoint.pt` | explicit local path (overrides the repo download if it exists) |
| `FACEANYTHING_ROOT` | the app dir | root holding `src/` and `checkpoints/` |
| `FACEANYTHING_BASE_MODEL` | `depth-anything/DA3-GIANT-1.1` | DA3 backbone id |
| `FACEANYTHING_GPU_DURATION` | `180` | ZeroGPU seconds per request |
| `FACEANYTHING_MAX_IMAGES` | `40` | hard cap on uploaded frames |

## Running locally

```bash
export FACEANYTHING_ROOT=/path/to/FaceAnything        # source checkout (has src/, checkpoints/)
pip install -r requirements.txt
python app.py
```

Project page: <https://kocasariumut.github.io/FaceAnything/> ·
Code: <https://github.com/kocasariumut/FaceAnything>