File size: 2,978 Bytes
e733c0b 231959b e733c0b 231959b e733c0b 231959b e733c0b 231959b e733c0b 231959b e733c0b 231959b e733c0b 231959b e733c0b 231959b e733c0b 231959b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | ---
base_model:
- stabilityai/stable-diffusion-2
library_name: diffusers
pipeline_tag: image-to-image
tags:
- reflection-removal
- reflection-separation
- image-restoration
- diffusion
- stable-diffusion
- cvpr2026
---
# Reflection Separation from a Single Image via Joint Latent Diffusion
This repository contains pre-trained checkpoints for the CVPR 2026 paper [Reflection Separation from a Single Image via Joint Latent Diffusion](https://huggingface.co/papers/2606.04107).
Given a single photo taken through glass, the model jointly generates the **transmission** (reflection-free) and **reflection** layers using a fine-tuned Stable Diffusion 2 architecture.
- **Authors:** Zheng-Hui Huang, Zhixiang Wang, Yu-Lun Liu, and Yung-Yu Chuang
- 🌐 **Project page:** [https://brian90709.github.io/diff-reflection-separation/](https://brian90709.github.io/diff-reflection-separation/)
- 💻 **Code:** [https://github.com/Brian90709/diff-reflection-separation-code](https://github.com/Brian90709/diff-reflection-separation-code)
- 📄 **Paper:** [arXiv:2606.04107](https://arxiv.org/abs/2606.04107)
## Method Overview
Single-image reflection separation is challenging under conditions like glare or weak reflections. This method leverages generative diffusion priors to simultaneously generate transmission and reflection layers through a unified diffusion model, incorporating a novel cross-layer self-attention mechanism for better feature disentanglement and a disjoint sampling strategy to reduce interference between layers.
## Files
| File | Size | Description |
| --- | --- | --- |
| `iter_016000/unet/diffusion_pytorch_model.bin` | ~3.5 GB | Trained layer-separation UNet. |
| `fuse_blocks.bin` | ~264 MB | CFW refiner for the VAE decoder. |
| `lrm/iter_008000/aux_net.bin` | ~1.3 MB | Latent composition module (LRM), used by `--optimization`. |
## Usage
### Installation
Clone the [official repository](https://github.com/Brian90709/diff-reflection-separation-code) and set up the environment:
```bash
conda create -y -n diffrs python=3.10 && conda activate diffrs
pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt
```
### Inference
Download the weights into `./checkpoints`:
```bash
huggingface-cli download Brian9999/diff-reflection-separation --repo-type model --local-dir ./checkpoints
```
Run the inference script on a directory of images:
```bash
python infer_layersep.py --input_dir ./samples --save_to_dir ./outputs
```
Each input yields three files: `*_transmission.png` (reflection-free result), `*_reflection.png`, and `*_ori_transmission.png` (transmission before CFW refinement).
## Citation
```bibtex
@inproceedings{huang2026reflection,
title = {Reflection Separation from a Single Image via Joint Latent Diffusion},
author = {Huang, Zheng-Hui and Wang, Zhixiang and Liu, Yu-Lun and Chuang, Yung-Yu},
booktitle = {CVPR},
year = {2026}
}
``` |