File size: 3,658 Bytes
cdc2652 f253468 cdc2652 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 | ---
license: other
license_name: edenlive-mixed
license_link: https://huggingface.co/VAST-AI/EdenLive-5B-Causal-Diffusers#license
library_name: diffusers
pipeline_tag: image-to-video
base_model: VAST-AI/EdenLive
tags:
- video-generation
- diffusers
- EdenLive
---
# EdenLive 5B Causal — Diffusers
[EdenLive](https://github.com/VAST-AI-Research/EdenLive) generates video
from an RGB proxy video, a reference image, and a text prompt.
The reference image provides appearance, while the proxy video provides
motion and scene structure.
This repository contains the 5B causal checkpoint in Diffusers format.
The model generates video in causal chunks, with four denoising steps per chunk by default.
A pipeline call returns the complete video.
[Project](https://vast-eden.ai/?p=edenlive) ·
[Code](https://github.com/VAST-AI-Research/EdenLive) ·
[Original checkpoints](https://huggingface.co/VAST-AI/EdenLive)
## Installation
Install a CUDA-compatible version of PyTorch and torchvision, then install
Diffusers from the official repository and the required dependencies:
```bash
pip install --upgrade "git+https://github.com/huggingface/diffusers.git"
pip install "transformers>=4.56.0" accelerate sentencepiece ftfy \
av imageio imageio-ffmpeg
```
## Quick start
Prepare a reference image and an RGB proxy video sampled at 24 fps.
The example uses the first 49 frames of the proxy video.
Replace the input paths and prompt with your own. Resample your video
before loading it if its frame rate differs from 24 fps.
```python
import torch
from diffusers import EdenLiveCausalPipeline
from diffusers.utils import export_to_video, load_image, load_video
pipe = EdenLiveCausalPipeline.from_pretrained(
"VAST-AI/EdenLive-5B-Causal-Diffusers",
torch_dtype={
"default": torch.bfloat16,
"proxy_encoder": torch.float32,
},
)
pipe.enable_model_cpu_offload()
frames = pipe(
image=load_image("reference.png"),
video=load_video("proxy-24fps.mp4"),
prompt="A character explores a snowy landscape.",
height=704,
width=1280,
num_frames=49,
num_inference_steps=4,
guidance_scale=1.0,
generator=torch.Generator(device="cpu").manual_seed(42),
).frames[0]
export_to_video(frames, "edenlive-5b-causal.mp4", fps=24)
```
## Usage notes
- The pipeline requires a reference image and a proxy video in addition
to the text prompt.
- The output frame count must satisfy $N = 4k + 1$ and must not exceed
the number of input video frames. Here, $k$ is a nonnegative integer.
- Custom output height and width must be multiples of 32.
- Keep the proxy encoder in FP32 when loading the other components in BF16.
The transformer preserves its internal proxy projection in FP32.
- The example uses CPU offload to reduce GPU memory usage. If the model
fits in GPU memory, replace `pipe.enable_model_cpu_offload()` with
`pipe.to("cuda")`.
- The standard Wan VAE is used by default.
- To change the prompt between chunks, pass `chunk_prompts` with one prompt
per chunk instead of `prompt`. With the default chunk size, a 49-frame
video uses five prompts. Each pipeline call starts a new video.
## License
EdenLive components use [Apache 2.0](LICENSE).
DINOv3 components are covered by the
[DINOv3 License](proxy_encoder/LICENSE.md).
## Citation
```bibtex
@misc{edenlive2026,
title = {EdenLive: Real-Time Generative Rendering from Any Visual Reference},
author = {Zehuan Huang and Tianxing Xu and Chunli Peng and Cheng Zhang and Zeyi Zheng and Yifei Song and Zekai Gu and Zhipeng Yu and Peng Wang and Yanpei Cao},
year = {2026},
url = {https://vast-eden.ai/?p=edenlive}
}
```
|