GRAFT: Geometric Refinement and Fitting Transformer for Human Scene Reconstruction

Pradyumna YM · Yuxuan Xue · Yue Chen · Nikita Kister · István Sárándi · Gerard Pons-Moll

Project page · arXiv · Code

TL;DR — A learned HSI prior that predicts iterative geometry-grounded refinements to reconstruct humans in scenes from a single image. Matches optimization-based contact quality at ~50× lower runtime; as a plug-and-play prior it boosts existing methods by up to 44% contact F1 — no retraining needed.

This repository holds the released weights and the benchmark assets we produced.

What is here

file	what it is
graft.pt	the main model (16.2 M parameters)
graft_no_visual.pt	the "without visual features" ablation — same architecture with drop_vis_feats=True, so the refinement sees scene geometry only
eval/scene_image.png	the PROX empty-scene view the model consumes as its second input
eval/rich_scene_imgs/	the same for RICH, per (sequence, camera) — 33 files
eval/contact_vertice_weights.pkl	per-vertex weights for the vertex-to-scene metric
eval/prox_quantitative_depth.pkl	PROX scene depth, carried by the evaluation context

These are our artifacts. The datasets themselves (PROX, RICH), the SMPL-X body models, and the NLF detector are not redistributed here — the repository's scripts/download_assets.py tells you where to obtain each.

Licence

These weights are released for non-commercial research use only (CC BY-NC 4.0). The code in the GitHub repository is MIT; this split follows NLF's.

The pipeline also loads MapAnything's weights (CC BY-NC 4.0) and NLF's (non-commercial research), and SMPL-X is under its own non-commercial research licence — so use of GRAFT as released is non-commercial research use.

Usage

git clone https://github.com/pradyumnaym/graft && cd graft
bash install.sh
python scripts/download_assets.py     # pulls this repository

from graft import GRAFTModel

model = GRAFTModel.from_pretrained("graft").cuda().eval()
out = model.predict("photo.jpg", scene_image="empty_room.jpg")
out.vertices        # (n_people, 10475, 3) in the scene's camera frame, metres

The two checkpoints load into the same class and differ only in configuration. Note they use different SMPL-X hand-mean conventions, which is why from_pretrained will not guess a config for a bare checkpoint path — getting that flag wrong changes the hands silently rather than failing.

Results

model	benchmark	PA-MPJPE	V2S L2	V2S angle	Prec	Rec	F1
graft	PROX	49.73	184.36	50.92	0.556	0.638	0.594
graft	RICH	46.32	224.66	34.60	0.473	0.743	0.578
graft-no-visual	PROX	52.56	188.47	51.92	0.534	0.510	0.522
graft-no-visual	RICH	45.52	223.10	34.06	0.434	0.717	0.540

Inference is 174 ms per image on an H100 (one person, three refinement iterations). See the repository for the per-stage breakdown and for what "reproduced exactly" is scoped to.

Citation

@misc{ym2026graft,
  title        = {GRAFT: Geometric Refinement and Fitting Transformer for Human Scene Reconstruction},
  author       = {Pradyumna YM and Yuxuan Xue and Yue Chen and Nikita Kister and Istv{\'a}n S{\'a}r{\'a}ndi and Gerard Pons-Moll},
  year         = {2026},
  eprint       = {2604.19624},
  archivePrefix = {arXiv},
  url          = {https://arxiv.org/abs/2604.19624},
}