Instructions to use Lightricks/LTX-Video with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use Lightricks/LTX-Video with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Lightricks/LTX-Video", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Inference
- Notebooks
- Google Colab
- Kaggle
Readme: Update inference section
Browse files
README.md
CHANGED
|
@@ -61,17 +61,24 @@ snapshot_download("Lightricks/LTX-Video", local_dir=model_path, local_dir_use_sy
|
|
| 61 |
|
| 62 |
#### Inference Code
|
| 63 |
|
| 64 |
-
To use our model, please follow the inference code in
|
| 65 |
|
| 66 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
```bash
|
| 69 |
-
python inference.py --ckpt_dir 'PATH' --prompt "PROMPT" --height HEIGHT --width WIDTH
|
| 70 |
```
|
| 71 |
|
| 72 |
-
For image-to-video generation:
|
| 73 |
|
| 74 |
-
```
|
| 75 |
-
python inference.py --ckpt_dir 'PATH' --prompt "PROMPT" --input_image_path IMAGE_PATH --height HEIGHT --width WIDTH
|
|
|
|
| 76 |
|
| 77 |
-
|
|
|
|
|
|
| 61 |
|
| 62 |
#### Inference Code
|
| 63 |
|
| 64 |
+
To use our model, please follow the inference code in [inference.py](./inference.py):
|
| 65 |
|
| 66 |
+
#### General tips:
|
| 67 |
+
* The model works on resolutions that are divisible by 32 and number of frames that are divisible by 8 + 1 (e.g. 257). In case the resolution or number of frames are not divisible by 32 or 8 + 1, the input will be padded with -1 and then cropped to the desired resolution and number of frames.
|
| 68 |
+
* The model works best on resolutions under 720 x 1280 and number of frames below 257.
|
| 69 |
+
* Prompts should be in English. The more elaborate the better. Good prompt looks like `The turquoise waves crash against the dark, jagged rocks of the shore, sending white foam spraying into the air. The scene is dominated by the stark contrast between the bright blue water and the dark, almost black rocks. The water is a clear, turquoise color, and the waves are capped with white foam. The rocks are dark and jagged, and they are covered in patches of green moss. The shore is lined with lush green vegetation, including trees and bushes. In the background, there are rolling hills covered in dense forest. The sky is cloudy, and the light is dim.`
|
| 70 |
+
|
| 71 |
+
#### For text-to-video generation:
|
| 72 |
|
| 73 |
```bash
|
| 74 |
+
python inference.py --ckpt_dir 'PATH' --prompt "PROMPT" --height HEIGHT --width WIDTH --num_frames NUM_FRAMES --seed SEED
|
| 75 |
```
|
| 76 |
|
| 77 |
+
#### For image-to-video generation:
|
| 78 |
|
| 79 |
+
```bash
|
| 80 |
+
python inference.py --ckpt_dir 'PATH' --prompt "PROMPT" --input_image_path IMAGE_PATH --height HEIGHT --width WIDTH --num_frames NUM_FRAMES --seed SEED
|
| 81 |
+
```
|
| 82 |
|
| 83 |
+
### ComfyUI Integration
|
| 84 |
+
To use our model with ComfyUI, please follow the instructions at [https://github.com/Lightricks/ComfyUI-LTXVideo/]().
|