Access CLUE

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

CLUE's weights build on Meta's SAM 3D Body and are distributed under the SAM License. They work together with Meta's SAM 3D Body release, which you need to request separately on facebook/sam-3d-body-dinov3.

CLUE

CLUE's weights for single-image 3D human mesh recovery with closed-form per-vertex uncertainty. For every person crop the network predicts the MHR body (pose, shape, scale, hands and face) and a variance for those parameters; CLUE propagates the variance through the rig in closed form, which gives the uncertainty of each of the mesh's 18,439 vertices without sampling.

Paper: Modeling Uncertainty Propagation in 3D Morphable Models. Demo: meghshukla/clue-demo.

What this repository contains

file	content
`clue.safetensors`	the tensors CLUE training produced: SAM 3D Body's decoder and heads (317 tensors) and the variance head (6 tensors); 51.8 M parameters
`config.json`	the two settings of the variance head, and checksums of the Meta files it builds on
`sam3dbody_clue.safetensors`	SAM 3D Body (CLUE): a variance head (6 tensors) trained on top of the released SAM 3D Body, which stays unchanged; it adds per-vertex uncertainty to an existing SAM 3D Body setup (used by the GitHub code, not by the demo)

It does not contain the DINOv3 backbone or the MHR head constants. CLUE training kept both frozen at the values of Meta's SAM 3D Body release, so they are loaded from facebook/sam-3d-body-dinov3 (gated; SAM License) and these weights are put on top. The loader is clue_space/model.py in the demo Space.

Model

Architecture: SAM 3D Body (DINOv3-H+ backbone, promptable decoder, MHR head) plus a variance head on the decoder's pose token.
Training: the decoder, heads and variance head were trained from scratch on SAM 3D Body's frozen DINOv3 backbone, on SA-1B and on the SAM 3D Body annotations of COCO, MPII and the 3DPW training split.
Inference input: a person box in an image and the camera intrinsics (the demo uses focal length = image diagonal).

Results

model	3DPW MPJPE	PA-MPJPE	PVE	PA-PVE	EMDB MPJPE	PA-MPJPE	PVE	PA-PVE
SAM 3D Body	56.0	35.1	67.1	47.2	63.2	38.2	74.7	47.1
CLUE	55.8	34.5	66.4	46.7	64.9	39.7	77.0	49.0

Errors in mm on the 3DPW test split (35,515 people) and EMDB 1 (24,117 frames); PVE is pelvis-aligned. Both rows come from our evaluation pipeline: the same people, boxes and camera intrinsics, and the same fit of the MHR mesh to SMPL. The SAM 3D Body row is Meta's released checkpoint run through this pipeline (its body branch, without the second-stage hand decoder), so it differs from the numbers reported by Meta.

Licence and attribution

These weights build on Meta's SAM 3D Body and are released under the SAM License (LICENSE). If you use them, please credit "CLUE (Shukla et al., EPFL)", link to this page and cite the paper (below; NOTICE). The code of the demo is released under the BSD-3-Clause-Clear License, which grants no patent rights. The MHR rig is Apache 2.0 and the DINOv3 backbone is under the DINOv3 License. Training data: SA-1B, and the SAM 3D Body annotations of COCO, MPII and 3DPW (3DPW is for non-commercial research).

Citation

@misc{shukla2026modelinguncertaintypropagation3d,
      title={Modeling Uncertainty Propagation in 3D Morphable Models}, 
      author={Megh Shukla and Edo Collins and Prashanth Chandran and Karel Petránek and Sergio Orts-Escolano and Mathieu Salzmann and Alexandre Alahi},
      year={2026},
      eprint={2610.24646},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.24646}, 
}

Downloads last month: -

Inference Providers NEW

Image-to-3D

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Space using meghshukla/clue 1

Paper for meghshukla/clue

Modeling Uncertainty Propagation in 3D Morphable Models

Paper • 2610.24646 • Published 4 days ago