packages/ltx-trainer/docs/quick-start.md

Quick Start Guide

Get up and running with LTX-2 training in just a few steps!

📋 Prerequisites

Before you begin, ensure you have:

1. LTX-2 Model Checkpoint - A local .safetensors file containing the LTX-2 model weights. Download ltx-2.3-22b-dev.safetensors from: HuggingFace Hub The trainer supports LTX-2 and LTX-2.3 checkpoints through the same configuration API; version-specific components are detected from the checkpoint.
2. Gemma Text Encoder - A local directory containing the Gemma model (required for LTX-2). Download from: HuggingFace Hub
3. Linux with CUDA - The trainer requires triton which is Linux-only; CUDA 13+ is recommended
4. GPU with sufficient VRAM - 80GB recommended for the standard config. For GPUs with 32GB VRAM (e.g., RTX 5090), use the low VRAM config which enables INT8 quantization and other memory optimizations

⚡ Installation

First, install uv if you haven't already. Then clone the repository and install the dependencies:

git clone https://github.com/Lightricks/LTX-2

The ltx-trainer package is part of the LTX-2 monorepo. Install the dependencies from the repository root, then navigate to the trainer package:

# From the repository root
uv sync
cd packages/ltx-trainer

The trainer depends on ltx-core and ltx-pipelines packages which are automatically installed from the monorepo.

🏋 Training Workflow

If you are using an agent-enabled environment with repository skills, you can ask for the train-model skill to run this workflow with you. It creates a run workspace, confirms the training mode, prepares data, preprocesses latents, launches training, and monitors the run while stopping for approval before expensive steps.

1. Choose a Training Mode

Start with t2v_lora.yaml for a first run with videos and captions. For modes such as IC-LoRA, inpainting, or outpainting, check Training Modes first because your metadata needs extra columns such as reference_video, video_mask, or audio_mask before preprocessing.

2. Prepare Your Dataset

Organize your videos and captions, then preprocess them:

# Split long videos into scenes (optional)
uv run python scripts/split_scenes.py input.mp4 scenes_output_dir/ --filter-shorter-than 5s

# Generate captions for videos (optional)
uv run python scripts/caption_videos.py scenes_output_dir/ --output dataset.json

# Preprocess the dataset (compute latents and embeddings)
uv run python scripts/process_dataset.py dataset.json \
    --resolution-buckets "960x544x49" \
    --model-path /path/to/ltx-2-model.safetensors \
    --text-encoder-path /path/to/gemma-model

By default, preprocessing writes to .precomputed/. Use that directory as data.preprocessed_data_root in your training config.

See Dataset Preparation for detailed instructions.

3. Configure Training

Create or modify a configuration YAML file. Start with one of the example configs:

• configs/t2v_lora.yaml - Text-to-video LoRA
• configs/t2v_lora_low_vram.yaml - Same as above, tuned for ~32GB VRAM (INT8 quantization and memory optimizations)
• configs/v2v_ic_lora.yaml - IC-LoRA video-to-video

Key settings to update:

model:
  model_path: "/path/to/ltx-2-model.safetensors"
  text_encoder_path: "/path/to/gemma-model"

data:
  preprocessed_data_root: "/path/to/preprocessed/data"

output_dir: "outputs/my_training_run"

See Configuration Reference for all available options.

4. Start Training

uv run python scripts/train.py configs/t2v_lora.yaml

For multi-GPU training:

uv run accelerate launch scripts/train.py configs/t2v_lora.yaml

See Training Guide for distributed training and advanced options.

🎯 Training Modes

First time? Start with t2v_lora.yaml — it's the simplest mode and only requires videos with captions. You can explore other modes once you've confirmed your setup works.

The trainer supports several training modes:

Mode	Description	Example Config
Text-to-Video	Generate video+audio from text prompts	t2v_lora.yaml
Image-to-Video	Animate from a starting image	i2v_lora.yaml
Video Extension	Extend videos temporally (forward/backward)	video_extend_lora.yaml, video_suffix_lora.yaml
IC-LoRA (V2V)	Video-to-video transformations	v2v_ic_lora.yaml
Audio-to-Video	Generate video conditioned on audio	a2v_lora.yaml
Video-to-Audio	Generate audio/foley from video	v2a_lora.yaml
Video Inpainting	Fill in masked regions of video	video_inpainting_lora.yaml
Video Outpainting	Extend video spatially	video_outpainting_lora.yaml
Text-to-Audio	Generate audio from text prompts	t2a_lora.yaml
Audio Extension	Extend audio temporally	audio_extend_lora.yaml, audio_suffix_lora.yaml
Audio Inpainting	Fill in masked regions of audio	audio_inpainting_lora.yaml
IC-LoRA (A2A)	Audio-to-audio transformations	a2a_ic_lora.yaml
AV2AV IC-LoRA	Audio+video IC-LoRA transformations	av2av_ic_lora.yaml
Full Fine-tuning	Full model training (any mode above)	Set model.training_mode: "full"

See Training Modes for detailed explanations of each mode.

Next Steps

Once you've completed your first training run, you can:

• Use your trained LoRA for inference - The ltx-pipelines package provides production-ready inference pipelines for various use cases (T2V, I2V, IC-LoRA, etc.). See the package documentation for details.
• Learn more about Dataset Preparation for advanced preprocessing
• Explore different Training Modes
• Dive deeper into Training Configuration
• Understand the model architecture in LTX-Core Documentation

Need Help?

If you run into issues at any step, see the Troubleshooting Guide for solutions to common problems.

Join our Discord community for real-time help and discussion!