nielsr HF Staff

Update paper link and add usage instructions

231959b verified 23 days ago

2.98 kB

base_model:
  - stabilityai/stable-diffusion-2
library_name: diffusers
pipeline_tag: image-to-image
tags:
  - reflection-removal
  - reflection-separation
  - image-restoration
  - diffusion
  - stable-diffusion
  - cvpr2026

Reflection Separation from a Single Image via Joint Latent Diffusion

This repository contains pre-trained checkpoints for the CVPR 2026 paper Reflection Separation from a Single Image via Joint Latent Diffusion.

Given a single photo taken through glass, the model jointly generates the transmission (reflection-free) and reflection layers using a fine-tuned Stable Diffusion 2 architecture.

Authors: Zheng-Hui Huang, Zhixiang Wang, Yu-Lun Liu, and Yung-Yu Chuang
🌐 Project page: https://brian90709.github.io/diff-reflection-separation/
💻 Code: https://github.com/Brian90709/diff-reflection-separation-code
📄 Paper: arXiv:2606.04107

Method Overview

Single-image reflection separation is challenging under conditions like glare or weak reflections. This method leverages generative diffusion priors to simultaneously generate transmission and reflection layers through a unified diffusion model, incorporating a novel cross-layer self-attention mechanism for better feature disentanglement and a disjoint sampling strategy to reduce interference between layers.

Files

File	Size	Description
`iter_016000/unet/diffusion_pytorch_model.bin`	~3.5 GB	Trained layer-separation UNet.
`fuse_blocks.bin`	~264 MB	CFW refiner for the VAE decoder.
`lrm/iter_008000/aux_net.bin`	~1.3 MB	Latent composition module (LRM), used by `--optimization`.

Usage

Installation

Clone the official repository and set up the environment:

conda create -y -n diffrs python=3.10 && conda activate diffrs
pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt

Inference

Download the weights into ./checkpoints:

huggingface-cli download Brian9999/diff-reflection-separation --repo-type model --local-dir ./checkpoints

Run the inference script on a directory of images:

python infer_layersep.py --input_dir ./samples --save_to_dir ./outputs

Each input yields three files: *_transmission.png (reflection-free result), *_reflection.png, and *_ori_transmission.png (transmission before CFW refinement).

Citation

@inproceedings{huang2026reflection,
  title     = {Reflection Separation from a Single Image via Joint Latent Diffusion},
  author    = {Huang, Zheng-Hui and Wang, Zhixiang and Liu, Yu-Lun and Chuang, Yung-Yu},
  booktitle = {CVPR},
  year      = {2026}
}