Lightricks
/

LTX-Video-0.9.5

@@ -46,6 +46,9 @@ You can use the model for purposes under the license:
 * The model works on resolutions that are divisible by 32 and number of frames that are divisible by 8 + 1 (e.g. 257). In case the resolution or number of frames are not divisible by 32 or 8 + 1, the input will be padded with -1 and then cropped to the desired resolution and number of frames.
 * The model works best on resolutions under 720 x 1280 and number of frames below 257.
 * Prompts should be in English. The more elaborate the better. Good prompt looks like `The turquoise waves crash against the dark, jagged rocks of the shore, sending white foam spraying into the air. The scene is dominated by the stark contrast between the bright blue water and the dark, almost black rocks. The water is a clear, turquoise color, and the waves are capped with white foam. The rocks are dark and jagged, and they are covered in patches of green moss. The shore is lined with lush green vegetation, including trees and bushes. In the background, there are rolling hills covered in dense forest. The sky is cloudy, and the light is dim.`
 ### Online demo
 The model is accessible right away via following links:
@@ -114,7 +117,7 @@ import torch
 from diffusers.pipelines.ltx.pipeline_ltx_condition import LTXConditionPipeline, LTXVideoCondition
 from diffusers.utils import export_to_video, load_video, load_image
-type = torch.bfloat16
 repo = "Lightricks/LTX-Video-0.9.5"
 pipe = LTXConditionPipeline.from_pretrained(repo, torch_dtype=dtype)
 pipe.to("cuda")
@@ -141,7 +144,7 @@ condition2 = LTXVideoCondition(
 prompt = "The video depicts a long, straight highway stretching into the distance, flanked by metal guardrails. The road is divided into multiple lanes, with a few vehicles visible in the far distance. The surrounding landscape features dry, grassy fields on one side and rolling hills on the other. The sky is mostly clear with a few scattered clouds, suggesting a bright, sunny day. And then the camera switch to a inding mountain road covered in snow, with a single vehicle traveling along it. The road is flanked by steep, rocky cliffs and sparse vegetation. The landscape is characterized by rugged terrain and a river visible in the distance. The scene captures the solitude and beauty of a winter drive through a mountainous region."
 negative_prompt='worst quality, inconsistent motion, blurry, jittery, distorted'
 # Generate the video
-generator = torch.Generator(device=device).manual_seed(0)
 video = pipe(
     conditions=[condition1, condition2],
     prompt=prompt,

 * The model works on resolutions that are divisible by 32 and number of frames that are divisible by 8 + 1 (e.g. 257). In case the resolution or number of frames are not divisible by 32 or 8 + 1, the input will be padded with -1 and then cropped to the desired resolution and number of frames.
 * The model works best on resolutions under 720 x 1280 and number of frames below 257.
 * Prompts should be in English. The more elaborate the better. Good prompt looks like `The turquoise waves crash against the dark, jagged rocks of the shore, sending white foam spraying into the air. The scene is dominated by the stark contrast between the bright blue water and the dark, almost black rocks. The water is a clear, turquoise color, and the waves are capped with white foam. The rocks are dark and jagged, and they are covered in patches of green moss. The shore is lined with lush green vegetation, including trees and bushes. In the background, there are rolling hills covered in dense forest. The sky is cloudy, and the light is dim.`
+* Referecne images/videos should align with the prompt for optimal performance.
+* Too different reference videos/imeges can return bad results.
+* Reference images are strongly recommended for tbest quality.
 ### Online demo
 The model is accessible right away via following links:
 from diffusers.pipelines.ltx.pipeline_ltx_condition import LTXConditionPipeline, LTXVideoCondition
 from diffusers.utils import export_to_video, load_video, load_image
+dtype = torch.bfloat16
 repo = "Lightricks/LTX-Video-0.9.5"
 pipe = LTXConditionPipeline.from_pretrained(repo, torch_dtype=dtype)
 pipe.to("cuda")
 prompt = "The video depicts a long, straight highway stretching into the distance, flanked by metal guardrails. The road is divided into multiple lanes, with a few vehicles visible in the far distance. The surrounding landscape features dry, grassy fields on one side and rolling hills on the other. The sky is mostly clear with a few scattered clouds, suggesting a bright, sunny day. And then the camera switch to a inding mountain road covered in snow, with a single vehicle traveling along it. The road is flanked by steep, rocky cliffs and sparse vegetation. The landscape is characterized by rugged terrain and a river visible in the distance. The scene captures the solitude and beauty of a winter drive through a mountainous region."
 negative_prompt='worst quality, inconsistent motion, blurry, jittery, distorted'
 # Generate the video
+generator = torch.Generator(device="cuda").manual_seed(0)
 video = pipe(
     conditions=[condition1, condition2],
     prompt=prompt,