File size: 2,751 Bytes
5355791 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 |
---
library_name: diffusers
pipeline_tag: image-to-image
---
# Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion
Stream-DiffVSR is a causally conditioned diffusion framework designed for efficient online Video Super-Resolution (VSR). It operates strictly on past frames to maintain low latency, making it suitable for real-time deployment.
[[Paper](https://huggingface.co/papers/2512.23709)] [[Project Page](https://jamichss.github.io/stream-diffvsr-project-page/)] [[GitHub](https://github.com/jamichss/Stream-DiffVSR)]
## Description
Diffusion-based VSR methods often struggle with latency due to multi-step denoising and reliance on future frames. Stream-DiffVSR addresses this with:
- **Causal Conditioning:** Operates only on past frames for online processing.
- **Four-step Distilled Denoiser:** Enables fast inference without sacrificing quality.
- **Auto-regressive Temporal Guidance (ARTG):** Injects motion-aligned cues during denoising.
- **Lightweight Temporal Decoder:** Enhances temporal coherence and fine details.
Stream-DiffVSR can process 720p frames in 0.328 seconds on an RTX 4090, achieving significant latency reductions compared to prior diffusion-based VSR methods.
## Usage
### Installation
```bash
git clone https://github.com/jamichss/Stream-DiffVSR.git
cd Stream-DiffVSR
conda env create -f requirements.yml
conda activate stream-diffvsr
```
### Inference
You can run inference using the following command. The script will automatically fetch the necessary weights from this repository.
```bash
python inference.py \
--model_id 'Jamichsu/Stream-DiffVSR' \
--out_path 'YOUR_OUTPUT_PATH' \
--in_path 'YOUR_INPUT_PATH' \
--num_inference_steps 4
```
The expected file structure for the inference input data is as follows:
```
YOUR_INPUT_PATH/
├── seq1/
│ ├── frame_0001.png
│ ├── frame_0002.png
│ └── ...
├── seq2/
│ ├── frame_0001.png
│ ├── frame_0002.png
│ └── ...
```
For NVIDIA TensorRT acceleration:
```bash
python inference.py \
--model_id 'Jamichsu/Stream-DiffVSR' \
--out_path 'YOUR_OUTPUT_PATH' \
--in_path 'YOUR_INPUT_PATH' \
--num_inference_steps 4 \
--enable_tensorrt \
--image_height <YOUR_TARGET_HEIGHT> \
--image_width <YOUR_TARGET_WIDTH>
```
## Citation
If you find this work useful, please cite:
```bibtex
@article{shiu2025streamdiffvsr,
title={Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion},
author={Shiu, Hau-Shiang and Lin, Chin-Yang and Wang, Zhixiang and Hsiao, Chi-Wei and Yu, Po-Fan and Chen, Yu-Chih and Liu, Yu-Lun},
journal={arXiv preprint arXiv:2512.23709},
year={2025}
}
``` |