license: mit
pipeline_tag: depth-estimation
FE2E: From Editor to Dense Geometry Estimator
FE2E is a Diffusion Transformer (DiT)-based foundation model for monocular dense geometry prediction. It adapts an advanced image editing model to dense geometry tasks, achieving strong zero-shot performance on both monocular depth and normal estimation.
[Project Page] [Paper] [GitHub]
Introduction
FE2E (From Editor to Dense Geometry Estimator) adapts an advanced image editing model based on Diffusion Transformer (DiT) architecture for dense geometry prediction. Key features include:
- Consistent Velocity Objective: Reformulates the editor's original flow matching loss for deterministic tasks.
- Logarithmic Quantization: Resolves precision conflicts between the editor's native BFloat16 format and the high precision demands of geometry tasks.
- Joint Estimation: Leverages DiT's global attention for joint estimation of depth and normals in a single forward pass.
FE2E achieves impressive performance improvements in zero-shot monocular depth and normal estimation, notably achieving over 35% gains on the ETH3D dataset and outperforming models trained on significantly more data.
🕹️ Inference
1. Setup
pip install -r requirements.txt
2. Prepare Model Weights
- Download the base weights from the official Step1X-Edit release.
- Download the FE2E LoRA checkpoint from this repository.
3. Run Evaluation
To run evaluation for depth estimation:
python -u evaluation.py \
--model_path ./pretrain \
--eval_data_root ./infer \
--output_dir ./infer/eval_results \
--num_gpus 8 \
--lora ./lora/LDRN.safetensors \
--single_denoise \
--prompt_type empty \
--norm_type ln \
--task_name depth \
--depth_eval_datasets [dataset]
To run evaluation for normal estimation:
python -u evaluation.py \
--model_path ./pretrain \
--eval_data_root ./infer \
--output_dir ./infer/eval_results \
--num_gpus 8 \
--lora ./lora/LDRN.safetensors \
--single_denoise \
--prompt_type empty \
--norm_type ln \
--task_name normal \
--normal_eval_datasets [dataset]
Citation
@article{wang2025editor,
title={From Editor to Dense Geometry Estimator},
author={Wang, JiYuan and Lin, Chunyu and Sun, Lei and Liu, Rongying and Nie, Lang and Li, Mingxing and Liao, Kang and Chu, Xiangxiang and Zhao, Yao},
journal={arXiv preprint arXiv:2509.04338},
year={2025}
}