code/HF_README.md

metadata

license: mit
tags:
  - depth-estimation
  - diffusion
  - monocular-depth
  - lidar-prompted
  - pixel-perfect-depth
  - pytorch

LiDAR-Perfect Depth (LPD)

Implementation of LiDAR-Perfect Depth: Score-Decomposed Diffusion with Kalman-in-the-Loop Denoising for Sparse-Prompted Depth on top of the public Pixel-Perfect Depth (PPD) codebase.

What's in this repo

• code/ — full repo (LPD additions live under ppd/lpd/, plus updated configs and adapter data loaders).
• checkpoints/
  • e000-s001000.ckpt … e004-s005000.ckpt — per-epoch checkpoints from the 5-dataset 1024×768 fine-tune
  • last.ckpt — rolling latest (= e004 here)
  • All 2.0 GB each, weights-only.
• inference_vis/ — 8-sample qualitative comparisons (RGB | GT | PPD | LPD | LPD-variance) generated with experiments/eval_lpd_vs_ppd.py.

Training run summary

Backbone:      PPD (DA-V2 semantics) — 820 M params, frozen
Trainable:     16 M (sparse_prompt_encoder + prompt_gate)
Resolution:    1024 × 768
Batch size:    18 (~119 GB GPU peak on H200)
Steps:         5,000 (5 epochs × 1000 batches)
Mix ratios:    Hypersim 0.5 / UrbanSyn 0.15 / UnrealStereo4K 0.15 / VKITTI2 0.1 / TartanAir 0.1
Init:          official PPD checkpoint (gangweix/Pixel-Perfect-Depth)

Epoch loss:    e0=0.0186 → e4=0.0177  (-4.8% over 5 epochs)

The official paper trains for many more steps; this checkpoint is a partial-train demo to show the pipeline works end-to-end.

Verification suite

cd code/
python -m ppd.lpd.tests.verify_paper
# 30 paper claims tested — all should pass.

Maps every section of paper.tex to a code line. See PAPER_CHECKLIST.md for the per-claim table.

Reproduction

pip install -r code/requirements.txt
# Symlink official weights
ln -sf <PPD ckpt>             code/checkpoints/ppd.pth
ln -sf <DA-V2 ViT-L ckpt>     code/checkpoints/depth_anything_v2_vitl.pth

# Stage 1 — Hypersim 512² pretrain
bash code/train_lpd.sh

# Stage 2 — 5-dataset 1024×768 fine-tune (this run's recipe)
python code/main.py --cfg_file code/ppd/configs/lpd_run5d_10k.yaml

# Inference comparison
python code/experiments/eval_lpd_vs_ppd.py

Datasets

Used the LFS dataset companion: chenming-wu/LiDAR-Perfect-Depth-Datasets.

Citations

• Pixel-Perfect Depth — Xu et al., NeurIPS 2025
• LiDAR-Perfect Depth — paper.tex (anonymous submission)