Buckets:

hf-doc-build
/

doc-dev

Files
xet
hf-doc-build/doc-dev / diffusers /pr_15066 /en /api /pipelines /edenlive.md
HuggingFaceDocBuilder
6 days ago
preview code
download
raw
27.7 kB
# EdenLive

[EdenLive](https://github.com/VAST-AI-Research/EdenLive) generates video from a proxy video, a reference
image, and a text prompt. [EdenLivePipeline](/docs/diffusers/pr_15066/en/api/pipelines/edenlive#diffusers.EdenLivePipeline) supports bidirectional generation; [EdenLiveCausalPipeline](/docs/diffusers/pr_15066/en/api/pipelines/edenlive#diffusers.EdenLiveCausalPipeline)
generates a complete video in causal chunks and supports changing prompts between chunks.

## Basic usage

Prepare an RGB reference image and a proxy video with at least 49 frames sampled at 24 fps. The image provides
appearance, while the proxy provides motion and scene structure.

```python
import torch
from diffusers import EdenLivePipeline
from diffusers.utils import export_to_video, load_image, load_video
pipe = EdenLivePipeline.from_pretrained(
    "VAST-AI/EdenLive-5B-Diffusers",
    torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32},
).to("cuda")
image = load_image("reference.png")
video = load_video("proxy-24fps.mp4")
frames = pipe(
    image=image,
    video=video,
    prompt="A character explores a snowy landscape.",
    num_frames=49,
).frames[0]
export_to_video(frames, "edenlive.mp4", fps=24)
```

Keep the proxy encoder in FP32 when loading the other models in BF16, as shown above. The transformer keeps its
internal proxy projection in FP32 when loaded with `from_pretrained`.

## Causal generation

Load the causal pipeline from its dedicated repository. Both model sizes default to four denoising steps per chunk.

```python
from diffusers import EdenLiveCausalPipeline
causal_pipe = EdenLiveCausalPipeline.from_pretrained(
    "VAST-AI/EdenLive-5B-Causal-Diffusers",
    torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32},
).to("cuda")
```

Use `prompt` for fixed text, or `chunk_prompts` to change it during a single video. At the default chunk size,
49 frames require five prompts. Reusing the image and video above:

```python
frames = causal_pipe(
    image=image,
    video=video,
    chunk_prompts=[
        "A character walks through snow.",
        "A character walks through snow.",
        "The character approaches a campfire.",
        "The character warms their hands beside the campfire.",
        "The character rests beside the campfire.",
    ],
    num_frames=49,
).frames[0]
export_to_video(frames, "chunk-prompts.mp4", fps=24)
```

`chunk_prompts` overrides a single `prompt` and cannot be combined with `prompt_embeds`. The negative prompt remains
fixed. A list passed as `prompt` represents a batch of independent videos. Each pipeline call starts a new video.

## LightTAE decoder

The full Wan VAE is the default. For faster decoding, pass the optional [EdenLiveLightTAEDecoder](/docs/diffusers/pr_15066/en/api/pipelines/edenlive#diffusers.EdenLiveLightTAEDecoder) to either pipeline:

```python
from diffusers import EdenLiveLightTAEDecoder
pipe = EdenLivePipeline.from_pretrained(
    "VAST-AI/EdenLive-5B-Diffusers",
    decoder=EdenLiveLightTAEDecoder.from_pretrained(
        "VAST-AI/EdenLive-5B-Diffusers", subfolder="decoder", torch_dtype=torch.bfloat16
    ),
    torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32},
).to("cuda")
```

## Notes

| Checkpoint | Pipeline | Default height × width | Example fps | Steps |
|---|---|---|---|---|
| [5B bidirectional](https://huggingface.co/VAST-AI/EdenLive-5B-Diffusers) | `EdenLivePipeline` | 704 × 1280 | 24 | 48 |
| [14B bidirectional](https://huggingface.co/VAST-AI/EdenLive-14B-Diffusers) | `EdenLivePipeline` | 480 × 832 | 16 | 30 |
| [5B causal](https://huggingface.co/VAST-AI/EdenLive-5B-Causal-Diffusers) | `EdenLiveCausalPipeline` | 704 × 1280 | 24 | 4 per chunk |
| [14B causal](https://huggingface.co/VAST-AI/EdenLive-14B-Causal-Diffusers) | `EdenLiveCausalPipeline` | 480 × 832 | 16 | 4 per chunk |

- Sample proxy frames at the desired output frame rate. `num_frames` must be $4k+1$ and fit within the input video;
  omitting it uses the largest valid prefix. Custom dimensions must be multiples of 32 for 5B or 16 for 14B.
- Guidance defaults to 3 for bidirectional generation and 1 for causal generation. Set `guidance_scale` above one
  to use a `negative_prompt`.
- For CPU offload, load without `.to("cuda")` and call `enable_model_cpu_offload()`.
- See the [example README](https://github.com/huggingface/diffusers/tree/main/examples/edenlive) for installation,
  video resampling, CPU offload, and checkpoint conversion.

## EdenLivePipeline[[diffusers.EdenLivePipeline]]

#### diffusers.EdenLivePipeline[[diffusers.EdenLivePipeline]]

```python
diffusers.EdenLivePipeline(vae: AutoencoderKLWan, text_encoder: UMT5EncoderModel, tokenizer: AutoTokenizer, transformer: EdenLiveTransformer3DModel, scheduler: diffusers.schedulers.scheduling_flow_match_euler_discrete.FlowMatchEulerDiscreteScheduler | diffusers.schedulers.scheduling_unipc_multistep.UniPCMultistepScheduler, proxy_encoder: DINOv3ViTModel, image_encoder: transformers.models.clip.modeling_clip.CLIPVisionModel | None = None, image_processor: transformers.models.clip.image_processing_pil_clip.CLIPImageProcessorPil | None = None, decoder: diffusers.models.autoencoders.autoencoder_edenlive.EdenLiveLightTAEDecoder | None = None, proxy_image_height: int = 480, proxy_image_width: int = 832)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive.py#L59)

**Parameters:**

vae ([AutoencoderKLWan](/docs/diffusers/pr_15066/en/api/models/autoencoder_kl_wan#diffusers.AutoencoderKLWan)) : Wan VAE for reference encoding and full-quality video decoding.

text_encoder (`UMT5EncoderModel`) : Text encoder.
tokenizer (`AutoTokenizer`) : Tokenizer corresponding to the text encoder.
transformer ([EdenLiveTransformer3DModel](/docs/diffusers/pr_15066/en/api/models/edenlive_transformer_3d#diffusers.EdenLiveTransformer3DModel)) : EdenLive bidirectional video denoiser.
scheduler ([FlowMatchEulerDiscreteScheduler](/docs/diffusers/pr_15066/en/api/schedulers/flow_match_euler_discrete#diffusers.FlowMatchEulerDiscreteScheduler) or [UniPCMultistepScheduler](/docs/diffusers/pr_15066/en/api/schedulers/unipc#diffusers.UniPCMultistepScheduler)) : Scheduler corresponding to the transformer checkpoint.
proxy_encoder (`DINOv3ViTModel`) : Trained RGB proxy encoder.

image_encoder (`CLIPVisionModel`, *optional*) : Reference image encoder required by the 14B model.
image_processor (`CLIPImageProcessor`, *optional*) : Processor corresponding to the image encoder.

decoder ([EdenLiveLightTAEDecoder](/docs/diffusers/pr_15066/en/api/pipelines/edenlive#diffusers.EdenLiveLightTAEDecoder), *optional*) : Optional faster decoder. When omitted, decode with the full Wan VAE.

proxy_image_height (`int`, defaults to 480) : Height of the proxy images passed to DINO.

proxy_image_width (`int`, defaults to 832) : Width of the proxy images passed to DINO.

Bidirectional video generation conditioned on a reference image, text, and an RGB proxy video.

#### __call__[[diffusers.EdenLivePipeline.__call__]]

```python
__call__(prompt: str | list[str] | None = None, image = None, video = None, height: int | None = None, width: int | None = None, num_frames: int | None = None, num_inference_steps: int | None = None, guidance_scale: float | None = None, negative_prompt: str | list[str] | None = None, generator: typing.Union[torch.Generator, list[torch.Generator], NoneType] = None, latents: typing.Optional[torch.Tensor] = None, prompt_embeds: typing.Optional[torch.Tensor] = None, negative_prompt_embeds: typing.Optional[torch.Tensor] = None, max_sequence_length: int = 512, output_type: str = 'np', return_dict: bool = True, attention_kwargs: dict | None = None, callback_on_step_end: typing.Optional[typing.Callable] = None, callback_on_step_end_tensor_inputs: list[str] | None = None)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive.py#L327)

**Parameters:**

prompt (`str` or `list[str]`, *optional*) : Text describing the output video. Supply this or `prompt_embeds`.

image (`PIL.Image.Image`, `np.ndarray`, or `torch.Tensor`) : Photoreal first-frame reference. Tensor and NumPy RGB values are in [0, 1]. The image is resized with tensor bilinear interpolation and center-cropped to the output dimensions.

video (`list[PIL.Image.Image]`, `np.ndarray`, or `torch.Tensor`) : Full-rate decoded RGB proxy video, already sampled at the desired output frame rate. Tensor inputs have shape `(T, C, H, W)` or `(B, T, C, H, W)`; NumPy inputs use channels last. Floating-point values are in [0, 1]. Proxy frames are sampled at the VAE temporal stride exactly once.

height (`int`, *optional*) : Output height. Defaults to 704 for 5B or 480 for 14B.

width (`int`, *optional*) : Output width. Defaults to 1280 for 5B or 832 for 14B.

num_frames (`int`, *optional*) : Number of decoded output frames, of the form `4k + 1`. When omitted, use the largest valid input prefix. The requested length cannot exceed the input video; missing frames are never repeated.
num_inference_steps (`int`, *optional*) : Denoising steps; defaults to 48 for 5B or 30 for 14B.
guidance_scale (`float`, *optional*) : Classifier-free guidance strength; defaults to 3. Values greater than one enable the negative prompt branch.

negative_prompt (`str` or `list[str]`, *optional*) : Text describing content to discourage when classifier-free guidance is enabled.
generator (`torch.Generator` or `list[torch.Generator]`, *optional*) : Random generator, or one per batch item. CPU generators are supported.
latents (`torch.Tensor`, *optional*) : Initial noise with shape `(B, C, T_latent, H_latent, W_latent)`. When omitted, noise for the whole video is sampled before image conditioning. The fixed first frame still consumes its noise draw.

prompt_embeds (`torch.Tensor`, *optional*) : Precomputed positive text embeddings in place of `prompt`.
negative_prompt_embeds (`torch.Tensor`, *optional*) : Precomputed negative text embeddings in place of `negative_prompt`.

max_sequence_length (`int`, defaults to 512) : Token count for each text prompt, including zero padding.

output_type (`str`, defaults to `"np"`) : Return NumPy videos (`"np"`, shape `(B, T, H, W, 3)`), PyTorch videos (`"pt"`, shape `(B, T, 3, H, W)`), nested PIL frame lists (`"pil"`), or normalized diffusion latents (`"latent"`, shape `(B, C, T_latent, H_latent, W_latent)`). Decoded tensor and NumPy values are in [0, 1].

return_dict (`bool`, defaults to `True`) : Return `EdenLivePipelineOutput`; otherwise return a one-element tuple containing the frames.
attention_kwargs (`dict`, *optional*) : Transformer options. The `scale` entry sets the LoRA scale when adapters are loaded.

callback_on_step_end (`Callable`, *optional*) : Function called as `callback(self, step_index, timestep, tensors)` after each denoising step. Return a dictionary containing any replacement tensors.

callback_on_step_end_tensor_inputs (`list[str]`, defaults to `["latents"]`) : Tensors supplied to the callback, selected from `latents`, `prompt_embeds`, and `negative_prompt_embeds`.

**Returns:** [EdenLivePipelineOutput](/docs/diffusers/pr_15066/en/api/pipelines/edenlive#diffusers.EdenLivePipelineOutput) or `tuple`

Generated frames, or latent videos when `output_type="latent"`.

Generate a complete bidirectional video.

Examples:
```python
>>> import torch
>>> from diffusers import EdenLivePipeline
>>> from diffusers.utils import export_to_video, load_image, load_video
>>> pipe = EdenLivePipeline.from_pretrained(
...     "VAST-AI/EdenLive-5B-Diffusers",
...     torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32},
... ).to("cuda")
>>> frames = pipe(
...     image=load_image("reference.png"), video=load_video("proxy.mp4"),
...     prompt="A character walks through snow.", num_frames=49,
... ).frames[0]
>>> export_to_video(frames, "edenlive.mp4", fps=24)
```

#### check_inputs[[diffusers.EdenLivePipeline.check_inputs]]

```python
check_inputs(prompt, prompt_embeds, negative_prompt, negative_prompt_embeds, image, video, height, width, output_type, callback_inputs)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive.py#L183)

Validate conditioning inputs, output dimensions, and callback tensor names.

#### decode_latents[[diffusers.EdenLivePipeline.decode_latents]]

```python
decode_latents(latents, output_type)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive.py#L300)

Decode normalized diffusion latents with the VAE or optional LightTAE.

#### encode_prompt[[diffusers.EdenLivePipeline.encode_prompt]]

```python
encode_prompt(prompt = None, negative_prompt = None, do_classifier_free_guidance = True, prompt_embeds = None, negative_prompt_embeds = None, max_sequence_length = 512, device = None, dtype = None)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive.py#L144)

Encode positive and optional negative text with UMT5, zeroing padded token embeddings.

#### prepare_conditions[[diffusers.EdenLivePipeline.prepare_conditions]]

```python
prepare_conditions(image, proxy_video, height, width, num_frames, batch_size, device, dtype)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive.py#L233)

Encode the reference image and sampled RGB proxy frames for the video transformer.

#### prepare_latents[[diffusers.EdenLivePipeline.prepare_latents]]

```python
prepare_latents(batch_size, height, width, num_frames, dtype, device, generator = None, latents = None)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive.py#L218)

Prepare full-video initial noise, including the fixed first frame's noise draw.

## EdenLiveCausalPipeline[[diffusers.EdenLiveCausalPipeline]]

#### diffusers.EdenLiveCausalPipeline[[diffusers.EdenLiveCausalPipeline]]

```python
diffusers.EdenLiveCausalPipeline(vae: AutoencoderKLWan, text_encoder: UMT5EncoderModel, tokenizer: AutoTokenizer, transformer: EdenLiveTransformer3DModel, scheduler: FlowMatchEulerDiscreteScheduler, proxy_encoder: DINOv3ViTModel, image_encoder: transformers.models.clip.modeling_clip.CLIPVisionModel | None = None, image_processor: transformers.models.clip.image_processing_pil_clip.CLIPImageProcessorPil | None = None, decoder: diffusers.models.autoencoders.autoencoder_edenlive.EdenLiveLightTAEDecoder | None = None, proxy_image_height: int = 480, proxy_image_width: int = 832, chunk_size: int = 3, max_context_frames: int = 18, sink_frames: int = 6)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive_causal.py#L59)

**Parameters:**

vae ([AutoencoderKLWan](/docs/diffusers/pr_15066/en/api/models/autoencoder_kl_wan#diffusers.AutoencoderKLWan)) : Wan VAE for reference encoding and full-quality video decoding.

text_encoder (`UMT5EncoderModel`) : Text encoder.
tokenizer (`AutoTokenizer`) : Tokenizer corresponding to the text encoder.
transformer ([EdenLiveTransformer3DModel](/docs/diffusers/pr_15066/en/api/models/edenlive_transformer_3d#diffusers.EdenLiveTransformer3DModel)) : EdenLive causal video denoiser.
scheduler ([FlowMatchEulerDiscreteScheduler](/docs/diffusers/pr_15066/en/api/schedulers/flow_match_euler_discrete#diffusers.FlowMatchEulerDiscreteScheduler)) : Scheduler corresponding to the transformer checkpoint.
proxy_encoder (`DINOv3ViTModel`) : Trained RGB proxy encoder.

image_encoder (`CLIPVisionModel`, *optional*) : Reference image encoder required by the 14B model.
image_processor (`CLIPImageProcessor`, *optional*) : Processor corresponding to the image encoder.

decoder ([EdenLiveLightTAEDecoder](/docs/diffusers/pr_15066/en/api/pipelines/edenlive#diffusers.EdenLiveLightTAEDecoder), *optional*) : Optional faster decoder. When omitted, decode with the full Wan VAE.

proxy_image_height (`int`, defaults to 480) : Height of the proxy images passed to DINO.

proxy_image_width (`int`, defaults to 832) : Width of the proxy images passed to DINO.

chunk_size (`int`, defaults to 3) : Latent frames generated in each chunk.
max_context_frames (`int`, defaults to 18) : Maximum number of latent frames in the attention window, including the current chunk.
sink_frames (`int`, defaults to 6) : Initial latent frames retained when the attention window fills.

Causal video generation conditioned on a reference image, text, and an RGB proxy video.

#### __call__[[diffusers.EdenLiveCausalPipeline.__call__]]

```python
__call__(prompt: str | list[str] | None = None, image = None, video = None, height: int | None = None, width: int | None = None, num_frames: int | None = None, num_inference_steps: int | None = None, guidance_scale: float | None = None, negative_prompt: str | list[str] | None = None, generator: typing.Union[torch.Generator, list[torch.Generator], NoneType] = None, latents: typing.Optional[torch.Tensor] = None, prompt_embeds: typing.Optional[torch.Tensor] = None, negative_prompt_embeds: typing.Optional[torch.Tensor] = None, chunk_prompts: list[str] | None = None, max_sequence_length: int = 512, output_type: str = 'np', return_dict: bool = True, attention_kwargs: dict | None = None, callback_on_step_end: typing.Optional[typing.Callable] = None, callback_on_step_end_tensor_inputs: list[str] | None = None)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive_causal.py#L345)

**Parameters:**

prompt (`str` or `list[str]`, *optional*) : Text describing the output video. Supply this or `prompt_embeds` unless using `chunk_prompts`.

image (`PIL.Image.Image`, `np.ndarray`, or `torch.Tensor`) : Photoreal first-frame reference. Tensor and NumPy RGB values are in [0, 1]. The image is resized with tensor bilinear interpolation and center-cropped to the output dimensions.

video (`list[PIL.Image.Image]`, `np.ndarray`, or `torch.Tensor`) : Full-rate decoded RGB proxy video, already sampled at the desired output frame rate. Tensor inputs have shape `(T, C, H, W)` or `(B, T, C, H, W)`; NumPy inputs use channels last. Floating-point values are in [0, 1]. Proxy frames are sampled at the VAE temporal stride exactly once.

height (`int`, *optional*) : Output height. Defaults to 704 for 5B or 480 for 14B.

width (`int`, *optional*) : Output width. Defaults to 1280 for 5B or 832 for 14B.

num_frames (`int`, *optional*) : Number of decoded output frames, of the form `4k + 1`. When omitted, use the largest valid input prefix. The requested length cannot exceed the input video; missing frames are never repeated.
num_inference_steps (`int`, *optional*) : Denoising steps per chunk. Defaults to four for the released causal checkpoints.
guidance_scale (`float`, *optional*) : Classifier-free guidance strength; defaults to 1. Values greater than one enable the negative prompt branch.

negative_prompt (`str` or `list[str]`, *optional*) : Text describing content to discourage when classifier-free guidance is enabled.
generator (`torch.Generator` or `list[torch.Generator]`, *optional*) : Random generator, or one per batch item. CPU generators are supported.
latents (`torch.Tensor`, *optional*) : Initial noise with shape `(B, C, T_latent, H_latent, W_latent)`. When omitted, noise for the whole video is sampled before image conditioning. The fixed first frame still consumes its noise draw.

prompt_embeds (`torch.Tensor`, *optional*) : Precomputed positive text embeddings in place of `prompt`.
negative_prompt_embeds (`torch.Tensor`, *optional*) : Precomputed negative text embeddings in place of `negative_prompt`.

chunk_prompts (`list[str]`, *optional*) : One positive prompt per latent chunk, including a short final chunk. Overrides a single `prompt`. Requires one video and cannot be combined with `prompt_embeds`. The negative prompt stays fixed. Text changes take effect at chunk boundaries while retaining previously generated video history.

max_sequence_length (`int`, defaults to 512) : Token count for each text prompt, including zero padding.

output_type (`str`, defaults to `"np"`) : Return NumPy videos (`"np"`, shape `(B, T, H, W, 3)`), PyTorch videos (`"pt"`, shape `(B, T, 3, H, W)`), nested PIL frame lists (`"pil"`), or normalized diffusion latents (`"latent"`, shape `(B, C, T_latent, H_latent, W_latent)`). Decoded tensor and NumPy values are in [0, 1].

return_dict (`bool`, defaults to `True`) : Return `EdenLivePipelineOutput`; otherwise return a one-element tuple containing the frames.
attention_kwargs (`dict`, *optional*) : Transformer options. The `scale` entry sets the LoRA scale when adapters are loaded.

callback_on_step_end (`Callable`, *optional*) : Function called as `callback(self, step_index, timestep, tensors)` after each denoising step. Return a dictionary containing any replacement tensors. In causal mode the tensors contain the current chunk and step indices count across all chunks.

callback_on_step_end_tensor_inputs (`list[str]`, defaults to `["latents"]`) : Tensors supplied to the callback, selected from `latents`, `prompt_embeds`, and `negative_prompt_embeds`.

**Returns:** [EdenLivePipelineOutput](/docs/diffusers/pr_15066/en/api/pipelines/edenlive#diffusers.EdenLivePipelineOutput) or `tuple`

Generated frames, or latent videos when `output_type="latent"`.

Generate a complete causal video.

Examples:
```python
>>> import torch
>>> from diffusers import EdenLiveCausalPipeline
>>> from diffusers.utils import export_to_video, load_image, load_video
>>> pipe = EdenLiveCausalPipeline.from_pretrained(
...     "VAST-AI/EdenLive-5B-Causal-Diffusers",
...     torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32},
... ).to("cuda")
>>> frames = pipe(
...     image=load_image("reference.png"), video=load_video("proxy.mp4"),
...     prompt="A character walks through snow.", num_frames=49,
... ).frames[0]
>>> export_to_video(frames, "edenlive.mp4", fps=24)
```

#### check_inputs[[diffusers.EdenLiveCausalPipeline.check_inputs]]

```python
check_inputs(prompt, prompt_embeds, negative_prompt, negative_prompt_embeds, image, video, height, width, output_type, callback_inputs)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive_causal.py#L198)

Validate conditioning inputs, output dimensions, and callback tensor names.

#### decode_latents[[diffusers.EdenLiveCausalPipeline.decode_latents]]

```python
decode_latents(latents, output_type)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive_causal.py#L318)

Decode normalized diffusion latents with the VAE or optional LightTAE.

#### encode_prompt[[diffusers.EdenLiveCausalPipeline.encode_prompt]]

```python
encode_prompt(prompt = None, negative_prompt = None, do_classifier_free_guidance = True, prompt_embeds = None, negative_prompt_embeds = None, max_sequence_length = 512, device = None, dtype = None)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive_causal.py#L158)

Encode positive and optional negative text with UMT5, zeroing padded token embeddings.

#### prepare_conditions[[diffusers.EdenLiveCausalPipeline.prepare_conditions]]

```python
prepare_conditions(image, proxy_video, height, width, num_frames, batch_size, device, dtype)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive_causal.py#L250)

Encode the reference image and sampled RGB proxy frames for the video transformer.

#### prepare_latents[[diffusers.EdenLiveCausalPipeline.prepare_latents]]

```python
prepare_latents(batch_size, height, width, num_frames, dtype, device, generator = None, latents = None)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_edenlive_causal.py#L234)

Prepare full-video initial noise, including the fixed first frame's noise draw.

## EdenLiveLightTAEDecoder[[diffusers.EdenLiveLightTAEDecoder]]

#### diffusers.EdenLiveLightTAEDecoder[[diffusers.EdenLiveLightTAEDecoder]]

```python
diffusers.EdenLiveLightTAEDecoder(latent_channels = 16, patch_size = 1, block_out_channels = (256, 128, 64, 64))
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/models/autoencoders/autoencoder_edenlive.py#L83)

**Parameters:**

latent_channels (`int`, *optional*, defaults to 16) : Latent channels: 16 for Wan 2.1 / EdenLive 14B, or 48 for Wan 2.2 / EdenLive 5B.
patch_size (`int`, *optional*, defaults to 1) : Output pixel shuffle factor: 1 for 14B or 2 for 5B.

block_out_channels (`tuple[int]`, *optional*, defaults to `(256, 128, 64, 64)`) : Decoder widths at its four spatial resolutions.

Decoder-only LightTAE for EdenLive's Wan latents.

Input latents use the VAE's raw (unnormalized) scale and `[batch, channels, frames, height, width]` layout. Decoded
RGB samples use the same layout, range `[-1, 1]`, and `(frames - 1) * 4 + 1` frames. Temporal state belongs to each
`decode` call and is never retained on the model.

#### decode[[diffusers.EdenLiveLightTAEDecoder.decode]]

```python
decode(latents: Tensor, return_dict: bool = True)
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/models/autoencoders/autoencoder_edenlive.py#L141)

**Parameters:**

latents (`torch.Tensor`) : Unnormalized VAE latents with shape `(B, C, T_latent, H_latent, W_latent)` on the decoder device and in its dtype. `C` must equal `config.latent_channels`, and at least one latent frame is required. Pipeline outputs with `output_type="latent"` must first be unnormalized with the VAE mean and standard deviation. The pipeline performs this conversion automatically when decoding frames.

return_dict (`bool`, defaults to `True`) : Return a `DecoderOutput` when true; otherwise return a one-element tuple containing the sample.
**Returns:** `DecoderOutput` or `tuple[torch.Tensor]`
RGB sample in `(B, 3, T, H, W)` layout and [-1, 1] range. The output has `(T_latent - 1) *
scale_factor_temporal + 1` frames and spatial dimensions multiplied by `scale_factor_spatial`. Internal
chunks share temporal state, including a short final chunk.

Decode a complete video from raw VAE latents.

## EdenLivePipelineOutput[[diffusers.EdenLivePipelineOutput]]

#### diffusers.EdenLivePipelineOutput[[diffusers.EdenLivePipelineOutput]]

```python
diffusers.EdenLivePipelineOutput(frames: typing.Union[torch.Tensor, numpy.ndarray, list[list[PIL.Image.Image]]])
```

[Source](https://github.com/huggingface/diffusers/blob/vr_15066/src/diffusers/pipelines/edenlive/pipeline_output.py#L25)

**Parameters:**

frames (`torch.Tensor`, `np.ndarray`, or `list[list[PIL.Image.Image]]`) : Generated videos selected by `output_type`. `"np"` returns shape `(B, T, H, W, 3)` and `"pt"` returns `(B, T, 3, H, W)`, both with floating-point RGB values in [0, 1]. `"pil"` returns one list of RGB images per video. `"latent"` bypasses decoding and returns normalized diffusion latents with shape `(B, C, T_latent, H_latent, W_latent)`; these values are not bounded RGB pixels. `B` is the video batch size and `T` is the decoded frame count. The latent channel count `C` is 48 for 5B and 16 for 14B.

Output of [EdenLivePipeline](/docs/diffusers/pr_15066/en/api/pipelines/edenlive#diffusers.EdenLivePipeline).
Xet Storage Details

Size:: 27.7 kB
Xet hash:: bbc1b86e4354ed54ff14b5eca8fecdf066cd17cb6a4172490090288aa0c78f49
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.