| --- |
| language: |
| - en |
| - de |
| - es |
| - fr |
| - ja |
| - ko |
| - zh |
| - it |
| - pt |
| library_name: diffusers |
| license: other |
| license_name: ltx-2-community-license-agreement |
| license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE.md |
| pipeline_tag: text-to-video |
| arxiv: 2601.03233 |
| tags: |
| - image-to-video |
| - text-to-video |
| - video-to-video |
| - image-text-to-video |
| - audio-to-video |
| - text-to-audio |
| - video-to-audio |
| - audio-to-audio |
| - text-to-audio-video |
| - image-to-audio-video |
| - image-text-to-audio-video |
| - ltx-2 |
| - ltx-video |
| - ltxv |
| - lightricks |
| - ltx-2.5 |
| pinned: true |
| demo: https://app.ltx.studio/ltx-2-playground/i2v |
| extra_gated_description: >- |
| By clicking "Agree and Access" you acknowledge the [Privacy |
| Policy](https://static.lightricks.com/legal/Privacy%20Policy%20-%20LTX%20Platform.pdf) |
| and consent to receive offers and updates including targeted and personalized |
| advertisements. You can unsubscribe at any time. |
| extra_gated_button_content: Agree and Access |
| --- |
| |
| <!-- LTX-2.5 model card — license-first redesign. YAML frontmatter above kept intact. --> |
|
|
| <div class="lg:-mr-20 xl:-mr-24 2xl:-mr-36" style="border-radius:14px;overflow:hidden;"> |
| <div style="position:relative;line-height:0;font-size:0;height:300px;"> |
| <img src="https://huggingface.co/Lightricks/LTX-2.5/resolve/main/hf-hero-web.webp" alt="LTX-2.5 — Video, Audio & World Simulation" style="display:block;width:100%;height:100%;object-fit:cover;margin:0;vertical-align:top;" /> |
| <video autoplay muted loop playsinline style="position:absolute;inset:0;width:100%;height:100%;object-fit:cover;"> |
| <source src="https://videos.ltx.io/LTX-2/ltx-research/hf-hero-web.mp4" type="video/mp4" /> |
| <source src="https://videos.ltx.io/LTX-2/ltx-research/hf-hero-web.webm" type="video/webm" /> |
| </video> |
| </div> |
| <!-- light hero footer --> |
| <div class="dark:hidden" style="background-color:#eef1f5;padding:2rem 1.5rem 2.4rem;text-align:center;isolation:isolate;"> |
| <h1 style="color:#0a0a0a;margin:0 0 0.5rem;font-size:2rem;font-weight:700;letter-spacing:-0.01em;border:none;padding:0;">LTX-2.5 — Video, Audio & World Simulation</h1> |
| <p style="color:#555;margin:0 0 1.25rem;font-size:1.05rem;">Full control and customization — self-host on your infrastructure.</p> |
| <div style="display:flex;flex-wrap:wrap;justify-content:center;gap:0.5rem;"> |
| <a href="https://ltx.io" class="bg-white hover:bg-gray-200 transition-colors duration-150" style="color:#1c1c1c;border:1px solid #d0d3d8;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Homepage</a> |
| <a href="https://docs.ltx.io" class="bg-white hover:bg-gray-200 transition-colors duration-150" style="color:#1c1c1c;border:1px solid #d0d3d8;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Docs</a> |
| <a href="https://github.com/Lightricks/LTX-2" class="bg-white hover:bg-gray-200 transition-colors duration-150" style="color:#1c1c1c;border:1px solid #d0d3d8;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">GitHub</a> |
| <a href="https://huggingface.co/papers/2601.03233" class="bg-white hover:bg-gray-200 transition-colors duration-150" style="color:#1c1c1c;border:1px solid #d0d3d8;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Research</a> |
| <a href="https://console.ltx.io/playground/" class="bg-white hover:bg-gray-200 transition-colors duration-150" style="color:#1c1c1c;border:1px solid #d0d3d8;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">API Playground</a> |
| <a href="https://discord.gg/ltx" class="bg-white hover:bg-gray-200 transition-colors duration-150" style="color:#1c1c1c;border:1px solid #d0d3d8;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Discord</a> |
| </div> |
| <div style="margin-top:1.5rem;"> |
| <a href="https://github.com/Lightricks/LTX-2/blob/main/LICENSE.md" class="bg-green-500 hover:brightness-110 transition-all duration-150" style="display:inline-block;color:#fff;border-radius:8px;padding:0.65rem 1.6rem;font-size:0.9rem;font-weight:700;text-decoration:none;">LTX License</a> |
| </div> |
| </div> |
| <!-- dark hero footer --> |
| <div class="hidden dark:block" style="background-color:#0a0a0a;padding:2rem 1.5rem 2.4rem;text-align:center;isolation:isolate;"> |
| <h1 style="color:#fff;margin:0 0 0.5rem;font-size:2rem;font-weight:700;letter-spacing:-0.01em;border:none;padding:0;">LTX-2.5 — Video, Audio & World Simulation</h1> |
| <p style="color:#c9c9c9;margin:0 0 1.25rem;font-size:1.05rem;">Full control and customization — self-host on your infrastructure.</p> |
| <div style="display:flex;flex-wrap:wrap;justify-content:center;gap:0.5rem;"> |
| <a href="https://ltx.io" class="bg-gray-800 hover:bg-gray-700 transition-colors duration-150" style="color:#fff;border:1px solid #2e2e2e;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Homepage</a> |
| <a href="https://docs.ltx.io" class="bg-gray-800 hover:bg-gray-700 transition-colors duration-150" style="color:#fff;border:1px solid #2e2e2e;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Docs</a> |
| <a href="https://github.com/Lightricks/LTX-2" class="bg-gray-800 hover:bg-gray-700 transition-colors duration-150" style="color:#fff;border:1px solid #2e2e2e;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">GitHub</a> |
| <a href="https://huggingface.co/papers/2601.03233" class="bg-gray-800 hover:bg-gray-700 transition-colors duration-150" style="color:#fff;border:1px solid #2e2e2e;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Research</a> |
| <a href="https://console.ltx.io/playground/" class="bg-gray-800 hover:bg-gray-700 transition-colors duration-150" style="color:#fff;border:1px solid #2e2e2e;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">API Playground</a> |
| <a href="https://discord.gg/ltx" class="bg-gray-800 hover:bg-gray-700 transition-colors duration-150" style="color:#fff;border:1px solid #2e2e2e;border-radius:8px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Discord</a> |
| </div> |
| <div style="margin-top:1.5rem;"> |
| <a href="https://github.com/Lightricks/LTX-2/blob/main/LICENSE.md" class="bg-green-500 hover:brightness-110 transition-all duration-150" style="display:inline-block;color:#fff;border-radius:8px;padding:0.65rem 1.6rem;font-size:0.9rem;font-weight:700;text-decoration:none;">LTX License</a> |
| </div> |
| </div> |
| </div> |
|
|
| <!-- LIGHT tier cards --> |
| <div class="dark:hidden lg:-mr-20 xl:-mr-24 2xl:-mr-36"> |
| <div style="display:grid;grid-template-columns:1fr 1fr;gap:1rem;margin-top:1rem;"> |
| <div style="background:#ecf8f2;border:1px solid #d6e9de;border-radius:6px;padding:1.5rem 1.6rem;"> |
| <div style="width:44px;height:44px;border-radius:6px;background:#dceee4;display:flex;align-items:center;justify-content:center;margin:0 0 0.8rem;color:#1c1c1c;"><svg xmlns="http://www.w3.org/2000/svg" width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><path d="M16 20V4a2 2 0 0 0-2-2h-4a2 2 0 0 0-2 2v16"/><rect width="20" height="14" x="2" y="6" rx="2"/></svg></div> |
| <div style="font-size:1.3rem;font-weight:700;color:#0a0a0a;margin:0 0 0.6rem;line-height:1.35;">Under $10M annual revenue</div> |
| <p style="color:#333;font-size:1.05rem;margin:1.1rem 0;">Commercial and production use at no cost under the LTX-2.x Community License. Transfer of fine-tunes may require a paid license, in accordance with the LTX-2.x Community License.</p> |
| <a href="https://docs.ltx.io/open-source-model/getting-started/overview" class="bg-gray-900 hover:bg-gray-700 transition-colors duration-150" style="display:inline-block;color:#fff;border:1px solid #0a0a0a;border-radius:9999px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Read the Documentation</a> |
| </div> |
| <div style="background:#f3eff9;border:1px solid #e0d8f0;border-radius:6px;padding:1.5rem 1.6rem;"> |
| <div style="width:44px;height:44px;border-radius:6px;background:#e6deef;display:flex;align-items:center;justify-content:center;margin:0 0 0.8rem;color:#1c1c1c;"><svg xmlns="http://www.w3.org/2000/svg" width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><path d="M3 21h18"/><path d="M5 21V5a2 2 0 0 1 2-2h6a2 2 0 0 1 2 2v16"/><path d="M15 21V9a1 1 0 0 1 1-1h3a2 2 0 0 1 2 2v11"/><path d="M9 7h2"/><path d="M9 11h2"/><path d="M9 15h2"/></svg></div> |
| <div style="font-size:1.3rem;font-weight:700;color:#0a0a0a;margin:0 0 0.6rem;line-height:1.35;">Over $10M annual revenue</div> |
| <p style="color:#333;font-size:1.05rem;margin:1.1rem 0;">Paid Commercial Use Agreement for LTX-2.x with full weights, engineering support, LoRAs, and flexible deployment options. To learn about all licensing options, talk to an expert.</p> |
| <a href="https://ltx.io/forms/ltx-contact-sales?kpi=licensing" class="bg-gray-900 hover:bg-gray-700 transition-colors duration-150" style="display:inline-block;color:#fff;border:1px solid #0a0a0a;border-radius:9999px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Talk to a Commercial Licensing Expert</a> |
| </div> |
| </div> |
| </div> |
|
|
| <!-- DARK tier cards --> |
| <div class="hidden dark:block lg:-mr-20 xl:-mr-24 2xl:-mr-36"> |
| <div style="display:grid;grid-template-columns:1fr 1fr;gap:1rem;margin-top:1rem;"> |
| <div style="background:#1a2622;border:1px solid #2a3a32;border-radius:6px;padding:1.5rem 1.6rem;"> |
| <div style="width:44px;height:44px;border-radius:6px;background:#22332c;display:flex;align-items:center;justify-content:center;margin:0 0 0.8rem;color:#fff;"><svg xmlns="http://www.w3.org/2000/svg" width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><path d="M16 20V4a2 2 0 0 0-2-2h-4a2 2 0 0 0-2 2v16"/><rect width="20" height="14" x="2" y="6" rx="2"/></svg></div> |
| <div style="font-size:1.3rem;font-weight:700;color:#fff;margin:0 0 0.6rem;line-height:1.35;">Under $10M annual revenue</div> |
| <p style="color:#c9ced6;font-size:1.05rem;margin:1.1rem 0;">Commercial and production use at no cost under the LTX-2.x Community License. Transfer of fine-tunes may require a paid license, in accordance with the LTX-2.x Community License.</p> |
| <a href="https://docs.ltx.io/open-source-model/getting-started/overview" class="bg-gray-50 hover:bg-gray-300 transition-colors duration-150" style="display:inline-block;color:#1c1c1c;border:1px solid #f8fafc;border-radius:9999px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Read the Documentation</a> |
| </div> |
| <div style="background:#221a2e;border:1px solid #322a42;border-radius:6px;padding:1.5rem 1.6rem;"> |
| <div style="width:44px;height:44px;border-radius:6px;background:#2a2238;display:flex;align-items:center;justify-content:center;margin:0 0 0.8rem;color:#fff;"><svg xmlns="http://www.w3.org/2000/svg" width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><path d="M3 21h18"/><path d="M5 21V5a2 2 0 0 1 2-2h6a2 2 0 0 1 2 2v16"/><path d="M15 21V9a1 1 0 0 1 1-1h3a2 2 0 0 1 2 2v11"/><path d="M9 7h2"/><path d="M9 11h2"/><path d="M9 15h2"/></svg></div> |
| <div style="font-size:1.3rem;font-weight:700;color:#fff;margin:0 0 0.6rem;line-height:1.35;">Over $10M annual revenue</div> |
| <p style="color:#c9ced6;font-size:1.05rem;margin:1.1rem 0;">Paid Commercial Use Agreement for LTX-2.x with full weights, engineering support, LoRAs, and flexible deployment options. To learn about all licensing options, talk to an expert.</p> |
| <a href="https://ltx.io/forms/ltx-contact-sales?kpi=licensing" class="bg-gray-50 hover:bg-gray-300 transition-colors duration-150" style="display:inline-block;color:#1c1c1c;border:1px solid #f8fafc;border-radius:9999px;padding:0.5rem 1.1rem;font-size:0.85rem;font-weight:600;text-decoration:none;">Talk to a Commercial Licensing Expert</a> |
| </div> |
| </div> |
| </div> |
|
|
| --- |
|
|
| # Diffusers weights for [LTX-2.5](https://huggingface.co/Lightricks/LTX-2.5). |
|
|
| --- |
|
|
| **LTX-2.5** is an open world model with open weights, built for local execution and fine-tuning. Its established use is generating synchronized, high-fidelity video and audio from text, image, and video inputs; applicability to emerging domains such as robotics and physical AI is developing. |
|
|
| **Full control and customization** — self-host on your own infrastructure. No per-generation billing, no per-seat lock-in, no forced API dependency. Revenue is measured across the whole entity, including subsidiaries and affiliates under common control. The full, binding terms live in [`LICENSE`](https://github.com/Lightricks/LTX-2/blob/main/LICENSE.md). |
|
|
| --- |
|
|
| ## Layout |
|
|
| | Path | Component | |
| |------|-----------| |
| | `transformer/` | **Distilled** DiT (default in `model_index.json`) | |
| | `transformer_full/` | Full / SFT DiT | |
| | `vae/` | Convolutional video VAE (encode + conv decode) | |
| | `diffusion_decoder/` | Diffusion (DiT) video decoder, decoder-only | |
| | `latent_upsampler/` | Spatial x2 latent upsampler, for the two-stage distilled recipe | |
| | `audio_vae/`, `vocoder/`, `connectors/`, `text_encoder/`, `tokenizer/`, `scheduler/`, `duration_head/` | Shared | |
|
|
| Encoding always uses `vae/`, and `LTX2Pipeline` decodes with `vae/` too. The diffusion decoder is a |
| diffusion model in its own right rather than a pipeline component, so it is driven by |
| `LTX2VideoDiffusionDecodePipeline`: run the pipeline with `output_type="latent"`, then decode. Both |
| decoders consume the same latents. |
|
|
| `scheduler/` is configured for the **distilled** transformer (`use_dynamic_shifting: false`, |
| `shift_terminal: null`) so that distilled sigma schedules are used exactly as given. See |
| [Full / SFT transformer](#full--sft-transformer) for the override the full DiT needs. |
|
|
| ## Install |
|
|
| LTX-2.5 support is not in a `diffusers` release yet, so install from main: |
|
|
| ```bash |
| pip install git+https://github.com/huggingface/diffusers |
| ``` |
|
|
| ## Downloading only what you need |
|
|
| `from_pretrained` fetches just the components listed in `model_index.json`, so the distilled path never |
| pulls `transformer_full/`. If you snapshot the repo instead, exclude it explicitly — 72 GB rather than |
| 110 GB: |
|
|
| ```bash |
| hf download Lightricks/LTX-2.5-Diffusers --exclude "transformer_full/*" |
| ``` |
|
|
| ## Quick start — distilled, convolutional decode |
|
|
| Distilled inference is driven by an explicit sigma schedule, not a step count, and runs unguided — |
| `guidance_scale=1.0` plus STG and modality guidance zeroed, since the pipeline defaults are the SFT |
| values. Passing `num_inference_steps` instead would hand the model a generic linear |
| schedule and quietly cost quality. |
|
|
| ```python |
| import torch |
| from diffusers import LTX2Pipeline |
| from diffusers.pipelines.ltx2.utils import DEFAULT_NEGATIVE_PROMPT, DISTILLED_SIGMA_VALUES |
| from diffusers.utils import encode_video |
| |
| MODEL_ID = "Lightricks/LTX-2.5-Diffusers" |
| |
| pipe = LTX2Pipeline.from_pretrained(MODEL_ID, dtype=torch.bfloat16) |
| pipe.enable_model_cpu_offload() |
| |
| video, audio = pipe( |
| prompt="A cinematic shot of a red fox walking through a snowy forest at dawn, " |
| "the camera tracking alongside, snow crunching underfoot.", |
| negative_prompt=DEFAULT_NEGATIVE_PROMPT, |
| width=960, |
| height=544, |
| num_frames=121, |
| frame_rate=24.0, |
| sigmas=DISTILLED_SIGMA_VALUES, |
| guidance_scale=1.0, |
| audio_guidance_scale=1.0, |
| stg_scale=0.0, |
| audio_stg_scale=0.0, |
| modality_scale=1.0, |
| audio_modality_scale=1.0, |
| generator=torch.Generator("cuda").manual_seed(42), |
| output_type="np", |
| return_dict=False, |
| ) |
| |
| encode_video( |
| video[0], |
| fps=24, |
| output_path="ltx25.mp4", |
| audio=audio[0].float().cpu(), |
| audio_sample_rate=pipe.vocoder.config.output_sampling_rate, |
| ) |
| ``` |
|
|
| Video and audio are generated jointly. Read the audio rate off the vocoder rather than hardcoding it — |
| this checkpoint ships `LTX2VocoderWithBWE`, whose `output_sampling_rate` is 48000. |
|
|
| ## Two-stage distilled generation |
|
|
| Better quality than the single-stage quick start: half resolution, x2 latent upsample, then a 3-sigma tail |
| at full resolution. One generator across both calls, so stage 2 continues the noise stream. |
|
|
| ```python |
| import torch |
| from diffusers import LTX2LatentUpsamplePipeline, LTX2Pipeline |
| from diffusers.pipelines.ltx2.latent_upsampler import LTX2LatentUpsamplerModel |
| from diffusers.pipelines.ltx2.utils import ( |
| DEFAULT_NEGATIVE_PROMPT, |
| DISTILLED_SIGMA_VALUES, |
| STAGE_2_DISTILLED_SIGMA_VALUES, |
| ) |
| from diffusers.utils import encode_video |
| |
| MODEL_ID = "Lightricks/LTX-2.5-Diffusers" |
| # Stage 1 resolution; stage 2 runs at 2x this. |
| HEIGHT, WIDTH, NUM_FRAMES, FRAME_RATE = 544, 960, 121, 24.0 |
| |
| pipe = LTX2Pipeline.from_pretrained(MODEL_ID, dtype=torch.bfloat16) |
| pipe.enable_model_cpu_offload() |
| pipe.vae.enable_tiling() # stage 2 decodes at 2x |
| |
| latent_upsampler = LTX2LatentUpsamplerModel.from_pretrained( |
| MODEL_ID, subfolder="latent_upsampler", dtype=torch.bfloat16 |
| ).to("cuda") |
| upsample_pipe = LTX2LatentUpsamplePipeline(vae=pipe.vae, latent_upsampler=latent_upsampler) |
| |
| generator = torch.Generator("cuda").manual_seed(42) |
| shared = dict( |
| prompt="A cinematic shot of a red fox walking through a snowy forest at dawn, " |
| "the camera tracking alongside, snow crunching underfoot.", |
| negative_prompt=DEFAULT_NEGATIVE_PROMPT, |
| frame_rate=FRAME_RATE, |
| guidance_scale=1.0, |
| audio_guidance_scale=1.0, |
| stg_scale=0.0, |
| audio_stg_scale=0.0, |
| modality_scale=1.0, |
| audio_modality_scale=1.0, |
| generator=generator, |
| return_dict=False, |
| ) |
| |
| stage_1_latents, audio_latents = pipe( |
| height=HEIGHT, width=WIDTH, num_frames=NUM_FRAMES, |
| sigmas=DISTILLED_SIGMA_VALUES, output_type="latent", **shared, |
| ) |
| |
| upsampled_latents = upsample_pipe( |
| latents=stage_1_latents, output_type="latent", return_dict=False |
| )[0] |
| |
| # Stage 2 takes its size from the upsampled latents, so pass no height/width. |
| video, audio = pipe( |
| num_frames=NUM_FRAMES, |
| sigmas=STAGE_2_DISTILLED_SIGMA_VALUES, |
| latents=upsampled_latents, |
| audio_latents=audio_latents, |
| noise_scale=STAGE_2_DISTILLED_SIGMA_VALUES[0], |
| output_type="np", |
| **shared, |
| ) |
| |
| encode_video( |
| video[0], |
| fps=int(FRAME_RATE), |
| output_path="ltx25_two_stage.mp4", |
| audio=audio[0].float().cpu(), |
| audio_sample_rate=pipe.vocoder.config.output_sampling_rate, |
| ) |
| ``` |
|
|
| To finish this with the diffusion decoder instead, ask stage 2 for `output_type="latent"` and follow the |
| next section — including its by-hand audio decode, since latents skip the vocoder. |
|
|
| ## Decode with the diffusion decoder |
|
|
| Ask the pipeline for latents, then decode them. Two things differ from the snippet above: |
| `output_type="latent"` skips the vocoder, so the audio comes back as **latents** and has to be finished by |
| hand; and the decoder needs `pip install kernels`, which fetches the |
| [NATTEN](https://github.com/SHI-Labs/NATTEN) kernel from the Hub (a local `natten` install is not used). |
|
|
| ```python |
| import torch |
| from diffusers import LTX2Pipeline, LTX2VideoDiffusionDecoderModel |
| from diffusers.models.autoencoders.ltx2_diffusion_decoder import LTX2VideoVaeNeighborhoodNattenProcessor |
| from diffusers.pipelines.ltx2.pipeline_ltx2_diffusion_decode import LTX2VideoDiffusionDecodePipeline |
| from diffusers.pipelines.ltx2.utils import DEFAULT_NEGATIVE_PROMPT, DISTILLED_SIGMA_VALUES |
| from diffusers.utils import encode_video |
| |
| MODEL_ID = "Lightricks/LTX-2.5-Diffusers" |
| |
| pipe = LTX2Pipeline.from_pretrained(MODEL_ID, dtype=torch.bfloat16) |
| pipe.enable_model_cpu_offload() |
| generator = torch.Generator("cuda").manual_seed(42) |
| |
| latents, audio_latents = pipe( |
| prompt="A cinematic shot of a red fox walking through a snowy forest at dawn, " |
| "the camera tracking alongside, snow crunching underfoot.", |
| negative_prompt=DEFAULT_NEGATIVE_PROMPT, |
| width=960, height=544, num_frames=121, frame_rate=24.0, |
| sigmas=DISTILLED_SIGMA_VALUES, |
| guidance_scale=1.0, |
| audio_guidance_scale=1.0, |
| stg_scale=0.0, |
| audio_stg_scale=0.0, |
| modality_scale=1.0, |
| audio_modality_scale=1.0, |
| generator=generator, |
| output_type="latent", |
| return_dict=False, |
| ) |
| |
| # `output_type="latent"` skips the vocoder, so finish the audio here. These latents are already |
| # denormalized, which is what `audio_vae.decode` expects. |
| mel = pipe.audio_vae.decode(audio_latents.to(pipe.audio_vae.dtype), return_dict=False)[0] |
| audio = pipe.vocoder(mel) |
| |
| decoder = LTX2VideoDiffusionDecoderModel.from_pretrained( |
| MODEL_ID, subfolder="diffusion_decoder", dtype=torch.bfloat16 |
| ).to("cuda") |
| |
| # The default FlexAttention processor is uncompiled, so it materialises the full score matrix and needs |
| # tens of GB at video resolutions. NATTEN's kernels are what the original implementation uses. |
| # `set_attention_backend` is not an alternative: no backend other than `flex` accepts the BlockMask. |
| # Every attention module in the decoder is the same neighborhood attention, so one call swaps them all. |
| decoder.set_attn_processor(LTX2VideoVaeNeighborhoodNattenProcessor()) |
| # Decode in overlapping tiles so peak memory is bounded by the tile size, not the video size. |
| decoder.enable_tiling() |
| |
| decode_pipe = LTX2VideoDiffusionDecodePipeline(diffusion_decoder=decoder, scheduler=pipe.scheduler) |
| |
| # denormalize=False: `output_type="latent"` already applied the latent statistics, so applying them |
| # again would rescale every channel by its std a second time. The decoder draws its own noise, so pass |
| # a generator for reproducible decoding. |
| video = decode_pipe( |
| latents, generator=generator, output_type="np", denormalize=False, return_dict=False |
| )[0] |
| |
| encode_video( |
| video[0], |
| fps=24, |
| output_path="ltx25_diffusion_decode.mp4", |
| audio=audio[0].float().cpu(), |
| audio_sample_rate=pipe.vocoder.config.output_sampling_rate, |
| ) |
| ``` |
|
|
| `enable_tiling()` bounds peak memory by the tile size rather than by the size of the video. The cheap early |
| upsampling stages still see the full latent; only the last upsampling stage and the diffusion stage, which |
| dominate decode memory, run per tile, so tiling changes the output only near tile borders. Each tile is |
| denoised separately, so a tiled decode does not reproduce an untiled one exactly. The default tile and |
| overlap sizes match the reference implementation's and can be overridden with the `tile_sample_min_*` / |
| `tile_sample_stride_*` arguments. |
|
|
| ## Full / SFT transformer |
|
|
| `transformer_full/` is not in `model_index.json`, so load it explicitly, and re-enable the shifting the |
| distilled `scheduler/` turns off. The pipeline's guidance defaults are already the SFT values, so the only |
| other change is dropping the distilled overrides. As a diff against the quick start: |
|
|
| ```diff |
| import torch |
| -from diffusers import LTX2Pipeline |
| -from diffusers.pipelines.ltx2.utils import DEFAULT_NEGATIVE_PROMPT, DISTILLED_SIGMA_VALUES |
| +from diffusers import FlowMatchEulerDiscreteScheduler, LTX2Pipeline, LTX2VideoTransformer3DModel |
| +from diffusers.pipelines.ltx2.utils import DEFAULT_NEGATIVE_PROMPT |
| from diffusers.utils import encode_video |
| |
| MODEL_ID = "Lightricks/LTX-2.5-Diffusers" |
| |
| -pipe = LTX2Pipeline.from_pretrained(MODEL_ID, dtype=torch.bfloat16) |
| +pipe = LTX2Pipeline.from_pretrained( |
| + MODEL_ID, |
| + transformer=LTX2VideoTransformer3DModel.from_pretrained( |
| + MODEL_ID, subfolder="transformer_full", dtype=torch.bfloat16 |
| + ), |
| + dtype=torch.bfloat16, |
| +) |
| pipe.enable_model_cpu_offload() |
| +pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config( |
| + pipe.scheduler.config, use_dynamic_shifting=True, shift_terminal=0.1 |
| +) |
| |
| video, audio = pipe( |
| prompt="A cinematic shot of a red fox walking through a snowy forest at dawn, " |
| "the camera tracking alongside, snow crunching underfoot.", |
| negative_prompt=DEFAULT_NEGATIVE_PROMPT, |
| width=960, |
| height=544, |
| num_frames=121, |
| frame_rate=24.0, |
| - sigmas=DISTILLED_SIGMA_VALUES, |
| - guidance_scale=1.0, |
| - audio_guidance_scale=1.0, |
| - stg_scale=0.0, |
| - audio_stg_scale=0.0, |
| - modality_scale=1.0, |
| - audio_modality_scale=1.0, |
| generator=torch.Generator("cuda").manual_seed(42), |
| output_type="np", |
| return_dict=False, |
| ) |
| ``` |
|
|
| Passing `transformer=` also keeps `from_pretrained` from fetching the distilled folder. |
|
|
| ## Prompting |
|
|
| The model was trained on long, single-paragraph audio-visual captions and degrades on short prompts. |
| Describe the shot, the motion, the light and the sound in one paragraph, as the examples above do. |
| Prompt enhancement for 2.5 uses a separate Gemma 4 checkpoint (e.g. `google/gemma-4-E2B-it`), not this |
| repo's fine-tuned text encoder. |
|
|
| ## Notes |
|
|
| - Distilled (`transformer/`) is for few-step inference with an explicit `sigmas=` schedule and all |
| guidance disabled; the full DiT (`transformer_full/`) uses step counts and real guidance. |
| - The diffusion decoder denoises rather than decoding deterministically, so decoding is only |
| reproducible with a `generator`. |
| - `processor` and `prompt_enhancer` appear in `model_index.json` but are not shipped here; they load as |
| `None`. |
|
|
