README.md

---
license: mit
pipeline_tag: depth-estimation
---

# FE2E: From Editor to Dense Geometry Estimator

FE2E is a Diffusion Transformer (DiT)-based foundation model for monocular dense geometry prediction. It adapts an advanced image editing model to dense geometry tasks, achieving strong zero-shot performance on both monocular depth and normal estimation.

[[Project Page](https://amap-ml.github.io/FE2E/)] [[Paper](https://huggingface.co/papers/2509.04338)] [[GitHub](https://github.com/AMAP-ML/FE2E)]

![teaser](https://github.com/AMAP-ML/FE2E/raw/main/assets/demo.png)

## Introduction
FE2E (From Editor to Dense Geometry Estimator) adapts an advanced image editing model based on Diffusion Transformer (DiT) architecture for dense geometry prediction. Key features include:
- **Consistent Velocity Objective**: Reformulates the editor's original flow matching loss for deterministic tasks.
- **Logarithmic Quantization**: Resolves precision conflicts between the editor's native BFloat16 format and the high precision demands of geometry tasks.
- **Joint Estimation**: Leverages DiT's global attention for joint estimation of depth and normals in a single forward pass.

FE2E achieves impressive performance improvements in zero-shot monocular depth and normal estimation, notably achieving over 35% gains on the ETH3D dataset and outperforming models trained on significantly more data.

## 🕹️ Inference

### 1. Setup
```bash
pip install -r requirements.txt
```

### 2. Prepare Model Weights
1. Download the base weights from the official [Step1X-Edit](https://github.com/stepfun-ai/Step1X-Edit) release.
2. Download the FE2E LoRA [checkpoint](https://huggingface.co/exander/FE2E/blob/main/LDRN.safetensors) from this repository.

### 3. Run Evaluation

To run evaluation for **depth estimation**:
```bash
python -u evaluation.py \
  --model_path ./pretrain \
  --eval_data_root ./infer \
  --output_dir ./infer/eval_results \
  --num_gpus 8 \
  --lora ./lora/LDRN.safetensors \
  --single_denoise \
  --prompt_type empty \
  --norm_type ln \
  --task_name depth \
  --depth_eval_datasets [dataset]
```

To run evaluation for **normal estimation**:
```bash
python -u evaluation.py \
  --model_path ./pretrain \
  --eval_data_root ./infer \
  --output_dir ./infer/eval_results \
  --num_gpus 8 \
  --lora ./lora/LDRN.safetensors \
  --single_denoise \
  --prompt_type empty \
  --norm_type ln \
  --task_name normal \
  --normal_eval_datasets [dataset]
```

## Citation
```bibtex
@article{wang2025editor,
  title={From Editor to Dense Geometry Estimator},
  author={Wang, JiYuan and Lin, Chunyu and Sun, Lei and Liu, Rongying and Nie, Lang and Li, Mingxing and Liao, Kang and Chu, Xiangxiang and Zhao, Yao},
  journal={arXiv preprint arXiv:2509.04338},
  year={2025}
}
```