File size: 2,782 Bytes
ca1ddb3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | ---
license: mit
pipeline_tag: depth-estimation
---
# FE2E: From Editor to Dense Geometry Estimator
FE2E is a Diffusion Transformer (DiT)-based foundation model for monocular dense geometry prediction. It adapts an advanced image editing model to dense geometry tasks, achieving strong zero-shot performance on both monocular depth and normal estimation.
[[Project Page](https://amap-ml.github.io/FE2E/)] [[Paper](https://huggingface.co/papers/2509.04338)] [[GitHub](https://github.com/AMAP-ML/FE2E)]
## Introduction
FE2E (From Editor to Dense Geometry Estimator) adapts an advanced image editing model based on Diffusion Transformer (DiT) architecture for dense geometry prediction. Key features include:
- **Consistent Velocity Objective**: Reformulates the editor's original flow matching loss for deterministic tasks.
- **Logarithmic Quantization**: Resolves precision conflicts between the editor's native BFloat16 format and the high precision demands of geometry tasks.
- **Joint Estimation**: Leverages DiT's global attention for joint estimation of depth and normals in a single forward pass.
FE2E achieves impressive performance improvements in zero-shot monocular depth and normal estimation, notably achieving over 35% gains on the ETH3D dataset and outperforming models trained on significantly more data.
## 🕹️ Inference
### 1. Setup
```bash
pip install -r requirements.txt
```
### 2. Prepare Model Weights
1. Download the base weights from the official [Step1X-Edit](https://github.com/stepfun-ai/Step1X-Edit) release.
2. Download the FE2E LoRA [checkpoint](https://huggingface.co/exander/FE2E/blob/main/LDRN.safetensors) from this repository.
### 3. Run Evaluation
To run evaluation for **depth estimation**:
```bash
python -u evaluation.py \
--model_path ./pretrain \
--eval_data_root ./infer \
--output_dir ./infer/eval_results \
--num_gpus 8 \
--lora ./lora/LDRN.safetensors \
--single_denoise \
--prompt_type empty \
--norm_type ln \
--task_name depth \
--depth_eval_datasets [dataset]
```
To run evaluation for **normal estimation**:
```bash
python -u evaluation.py \
--model_path ./pretrain \
--eval_data_root ./infer \
--output_dir ./infer/eval_results \
--num_gpus 8 \
--lora ./lora/LDRN.safetensors \
--single_denoise \
--prompt_type empty \
--norm_type ln \
--task_name normal \
--normal_eval_datasets [dataset]
```
## Citation
```bibtex
@article{wang2025editor,
title={From Editor to Dense Geometry Estimator},
author={Wang, JiYuan and Lin, Chunyu and Sun, Lei and Liu, Rongying and Nie, Lang and Li, Mingxing and Liao, Kang and Chu, Xiangxiang and Zhao, Yao},
journal={arXiv preprint arXiv:2509.04338},
year={2025}
}
``` |