apapaxionga/3d-face-reconstructor

3D Face Reconstructor

A lightweight CNN model that reconstructs a 3D face mesh from a single 2D image.

Model Overview

Property	Value
Architecture	ResNet-like CNN encoder + FC regressor
Input	RGB face image (128×128)
Output	512 3D vertex coordinates (x, y, z)
Parameters	~11.3M
Training data	Synthetic 3DMM-like parametric face meshes

Approach

This model was trained on synthetically generated face meshes using a parametric 3D face model with:

• 40 identity basis vectors — capturing face shape variations across individuals
• 30 expression basis vectors — capturing facial expression variations
• Random rotations (±0.35 radians) for pose robustness
• Orthographic projection to 2D depth images
• Gaussian-noise-augmented RGB rendering

The CNN learns to invert the rendering process, recovering the full 3D mesh structure from a single 2D image.

Training Details

Hyperparameter	Value
Optimizer	Adam
Learning Rate	1e-3 with cosine warm restarts
Batch Size	32
Loss	MSE (vertex-wise)
Epochs	20
Best Val Loss	0.0218

Usage

import torch
from PIL import Image
import numpy as np

# Load model
ckpt = torch.load("model.pt", map_location="cpu")
n_vertices = ckpt["mean_face"].shape[0]

from model import Face3DNet
model = Face3DNet(num_vertices=n_vertices)
model.load_state_dict(ckpt["model_state_dict"])
model.eval()

# Load and preprocess image
img = Image.open("face.jpg").convert("RGB").resize((128, 128))
arr = np.array(img).astype(np.float32) / 255.0
tensor = torch.from_numpy(arr).permute(2, 0, 1).unsqueeze(0)

# Predict 3D mesh
with torch.no_grad():
    vertices = model(tensor)[0]  # [512, 3]

Live Demo

Try the interactive demo: https://huggingface.co/spaces/apapaxionga/3d-face-reconstructor-demo

Limitations

• Trained on synthetic data — real-world generalization is not guaranteed
• Coarse parametric mesh, not a high-resolution detailed scan
• Best for roughly frontal or mildly rotated faces
• No texture/color reconstruction

Citation

@software{3d_face_reconstructor_2025,
  title = {3D Face Reconstruction from Single 2D Image},
  year = {2025}
}

Generated by ML Intern

This model repository was generated by ML Intern, an agent for machine learning research and development on the Hugging Face Hub.

• Try ML Intern: https://smolagents-ml-intern.hf.space
• Source code: https://github.com/huggingface/ml-intern