license: apache-2.0
pipeline_tag: image-to-image
library_name: diffusers
SwiftVR: Real-Time One-Step Generative Video Restoration
SwiftVR is the first generative video restoration model to reach real-time 1080p streaming on a consumer-grade GPU (≈26 FPS on a single RTX 5090), sustains 31 FPS at QHD (2560×1440) and 14 FPS at 4K (3840×2160) on a single H100, and streams at resolutions where every compared diffusion-based VR baseline runs out of memory.
SwiftVR is a streaming one-step generative video restoration (VR) framework presented in SwiftVR: Real-Time One-Step Generative Video Restoration.
Updates
- [2026/06] Release the inference code and pretrained weights 🎉
✨ Highlights
- Mask-free shifted-window self-attention (MFSWA). Each spatial window is pre-gathered into a dense tensor, so every attention call reduces to a single standard scaled-dot-product (SDPA) call — no attention mask, cyclic shift, or padding ever enters the graph. This gives a 1.62× throughput gain over its full-attention teacher at essentially identical quality, with no dedicated sparse kernel.
- Restoration-aware Autoencoder (ReAE). A lightweight encoder–decoder jointly fine-tuned with the DiT in pixel space removes the heavy-3D-VAE / tiled-decoding bottleneck.
- Causal chunk-wise streaming. A minimal causal protocol (no rolling KV cache, no overlapped DiT inference) bounds the temporal axis, confining the residual (\mathcal{O}(N^2)) cost to the spatial axes.
📊 Results
Efficiency at 2560×1440 (single H100, causal streaming, 24 frames)
| Metric | DOVE (tile) | SeedVR2-3B (tile) | FlashVSR-Tiny | SwiftVR (Ours) |
|---|---|---|---|---|
| Avg. Time (s) ↓ | 27.615 | 17.320 | 2.493 | 0.766 |
| FPS ↑ | 0.85 | 1.39 | 9.61 | 31.32 |
| Peak Mem. (GB) ↓ | 59.24 | 35.35 | 34.35 | 38.01 |
At 3840×2160, every compared diffusion-based VR baseline OOMs on a single H100; SwiftVR sustains 14 FPS.
🛠 Installation
git clone https://github.com/H-oliday/SwiftVR.git
cd SwiftVR
conda create -n swiftvr python=3.10 -y
conda activate swiftvr
# Install PyTorch matching your CUDA toolkit first, e.g. CUDA 12.4:
pip install torch==2.10.0 torchvision==0.25.0 --index-url https://download.pytorch.org/whl/cu124
# Install SwiftVR (editable) and its dependencies:
pip install -e .
🚀 Quick Start
Python API
from swiftvr import SwiftVRPipeline
pipe = SwiftVRPipeline.from_pretrained("H-oliday/SwiftVR").to("cuda", dtype="bfloat16")
pipe.restore_video("low_quality.mp4", "restored.mp4", upscale=4)
Streaming (causal, chunk by chunk)
session = pipe.stream(clip_len=24, resolution=(1920, 1080))
for lq_chunk in read_chunks("low_quality.mp4", n=24): # lq_chunk: [T, H, W, 3] uint8
hq = session.step(lq_chunk) # [1, T', 3, H', W'] in [0, 1], or None if buffered
if hq is not None:
write(hq)
tail = session.flush() # flush the final buffered frames
Command line
python scripts/inference.py \
--input low_quality.mp4 \
--output restored.mp4 \
--checkpoint H-oliday/SwiftVR \
--upscale 4 \
--clip-len 24 \
--dtype bfloat16
🎬 Visual Results
🙏 Acknowledgements
SwiftVR builds on Wan2.2-TI2V-5B, the lightweight autoencoder TAEHV, and the RealBasicVSR degradation pipeline. We thank the authors of DOVE, SeedVR2, and FlashVSR for releasing strong baselines.
📜 Citation
@article{yan2026swiftvr,
title={SwiftVR: Real-Time One-Step Generative Video Restoration},
author={Yan, Jiaqi and Chen, Xiangyu and Zhong, Xinlin and Huang, Haibin and Zhang, Chi and Liu, Jie and Zhou, Jiantao and Li, Xuelong},
journal={arXiv preprint arXiv:2606.09516},
year={2026}
}