File size: 3,453 Bytes
2621a4e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 | ---
license: other
license_name: ltx-video-2-open-source-license
license_link: https://huggingface.co/Lightricks/LTX-2.3/blob/main/LICENSE
base_model: Lightricks/LTX-2.3
tags:
- ltx2
- video
- audio
- diffusers
pipeline_tag: image-to-video
---
# LTX-2.3 Distilled v1.1 (Diffusers)
Diffusers-format weights for the **v1.1** distilled checkpoint of
[Lightricks/LTX-2.3](https://huggingface.co/Lightricks/LTX-2.3) — a DiT-based foundation
model that jointly generates synchronized video and audio. Runs in **8 steps with CFG = 1**.
This is an unofficial staging conversion. The existing
[`diffusers/LTX-2.3-Distilled-Diffusers`](https://huggingface.co/diffusers/LTX-2.3-Distilled-Diffusers)
repo was converted before the upstream v1.1 distilled checkpoint was published
(v1.0 dates to Mar 5, this repo's source `dg845/LTX-2.3-Distilled-Diffusers` conversion was
done Mar 14, and the v1.1 checkpoint was added upstream on Apr 13) — so that repo still ships
v1.0 weights. This repo swaps in the v1.1 `transformer/` weights on top of the same base
pipeline; every other component (VAE, audio VAE, text encoder, tokenizer, connectors, vocoder,
scheduler config) is unchanged from `diffusers/LTX-2.3-Distilled-Diffusers` and was not
re-converted, since only the DiT was updated upstream.
## Source checkpoint
- File: `ltx-2.3-22b-distilled-1.1.safetensors` from
[`Lightricks/LTX-2.3`](https://huggingface.co/Lightricks/LTX-2.3)
- Converted with `scripts/convert_ltx2_to_diffusers.py --version 2.3 --dit` from a
local `huggingface/diffusers` checkout (`0.40.0.dev0`).
- Architecture (`transformer/config.json`) is byte-for-byte identical to the v1.0 conversion in
`diffusers/LTX-2.3-Distilled-Diffusers` aside from the `_diffusers_version` metadata field —
only the weights changed upstream.
## Usage
Requires a recent build of `diffusers` with LTX-2 support:
```bash
pip install -U git+https://github.com/huggingface/diffusers
```
```python
import torch
from diffusers import LTX2Pipeline
from diffusers.pipelines.ltx2.export_utils import encode_video
from diffusers.pipelines.ltx2.utils import DEFAULT_NEGATIVE_PROMPT, DISTILLED_SIGMA_VALUES
pipe = LTX2Pipeline.from_pretrained(
"rootonchair/LTX-2.3-Distilled-v1.1-Diffusers", torch_dtype=torch.bfloat16
)
pipe.enable_model_cpu_offload()
prompt = "A flowing river in a forest at golden hour, gentle wind in the leaves."
frame_rate = 24.0
video, audio = pipe(
prompt=prompt,
negative_prompt=DEFAULT_NEGATIVE_PROMPT,
width=768,
height=512,
num_frames=121,
frame_rate=frame_rate,
num_inference_steps=8,
sigmas=DISTILLED_SIGMA_VALUES,
guidance_scale=1.0,
output_type="np",
return_dict=False,
)
encode_video(
video[0],
fps=frame_rate,
audio=audio[0].float().cpu(),
audio_sample_rate=pipe.vocoder.config.output_sampling_rate,
output_path="ltx2_distilled_v1_1_t2v.mp4",
)
```
## Notes
- `width`/`height` must be divisible by 32; `num_frames` must equal `8k + 1`.
- Always pass `sigmas=DISTILLED_SIGMA_VALUES`, `num_inference_steps=8`, `guidance_scale=1.0`
for this distilled checkpoint.
- See the [Diffusers LTX-2 docs](https://huggingface.co/docs/diffusers/main/en/api/pipelines/ltx2)
for multimodal guidance, prompt enhancement, and the upscaling/refinement pipeline.
## License
These weights are released under the
[LTX Video 2 Open Source License](https://huggingface.co/Lightricks/LTX-2.3/blob/main/LICENSE).
|