README.md

metadata

license: apache-2.0
language:
  - en
tags:
  - depth-estimation
  - colonoscopy
  - medical-imaging
  - video
  - lora
  - diffusion
library_name: transformers
base_model:
  - tencent/DepthCrafter
  - stabilityai/stable-video-diffusion-img2vid-xt
pipeline_tag: depth-estimation

ColonCrafter: A Depth Estimation Model for Colonoscopy Videos Using Diffusion Priors

ColonCrafter builds upon DepthCrafter and Stable Video Diffusion to provide temporally consistent depth predictions for colonoscopy video.

Model Details

• Model Type: Video Depth Estimation (Diffusion-based)
• Base Architecture: DepthCrafter UNet with LoRA adaptation
• LoRA Configuration:
  • Rank: 16
  • Target modules: to_q, to_k, to_v, to_out.0
  • Dropout: 0.1
• Precision: FP16

Installation

Please refer to the installation instructions in our repository.

Usage

import torch
from src.depth.models.model import ColonCrafterInference

# Load the model
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = ColonCrafterInference.from_pretrained(
    "romainhardy/coloncrafter",
    device=device
)

# Prepare video tensor: (N, C, H, W) in [0, 1] range
# video = ...

# Run inference
pred_depth, pred_disparity = model.predict_depth(
    video,
    num_inference_steps=1,
    window_size=16,
    overlap=8,
    guidance_scale=1.0,
    seed=42
)

Citation

If you use this model in your research, please cite:

@article{hardy2025coloncrafter,
  title={ColonCrafter: A Depth Estimation Model for Colonoscopy Videos Using Diffusion Priors},
  author={Hardy, Romain and Berzin, Tyler and Rajpurkar, Pranav},
  journal={arXiv preprint arXiv:2509.13525},
  year={2025}
}

Acknowledgments

This model builds upon DepthCrafter and Stable Video Diffusion.