LTX-2.3-Foley-LoRA / README.md
FuzzPuppy's picture
update readme
3b574f5 verified
|
Raw
History Blame Contribute Delete
4.78 kB
---
license: other
license_name: ltx-2-community-license-agreement
license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE
base_model: Lightricks/LTX-2.3
tags:
- ltx
- ltx-2.3
- video-to-audio
- foley
- sound-effects
- lora
- safetensors
library_name: diffusers
---
# LTX-2.3 Foley
LTX-2.3 Foley is a video-to-audio LoRA for LTX-2.3. It adds realistic,
visually synchronized Foley and sound effects to video without adding a music
overlay.
This LoRA is especially useful when LTX-2.3 adds background music, score, or
rhythmic soundtrack material but the desired output is audible real-world sound
effects.
## Tutorial
[![Watch the tutorial: using the LTX-2.3 Foley LoRA](https://img.youtube.com/vi/qnHFDlrySmw/hqdefault.jpg)](https://youtu.be/qnHFDlrySmw)
[Watch the tutorial on YouTube](https://youtu.be/qnHFDlrySmw)
## Demo with and without LoRA
<video controls playsinline preload="metadata" style="width: 100%; max-width: 960px; border-radius: 8px;">
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/sharpei-comparison-hf.mp4" type="video/mp4">
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/sharpei-comparison-hf.mp4">Download video</a>
</video>
## Compatibility
This LoRA is intended for use with LTX-2.3 base or distilled video-to-audio
models.
## License
This model is released under the LTX Community License configured on the
Hugging Face repository.
## Files
- `ltx-2.3-foley-400-steps.safetensors`: 400-step Foley LoRA weights
## Recommended Usage
Use a LoRA multiplier between `1` and `3`.
Start around `1`. If music or score still appears, increase the multiplier
toward `2` or `3`.
Prompt with a short description of the visible action in the video, followed by:
```text
No speech is present. No music is present.
```
Example prompt:
```text
A barista uses an espresso machine to steam milk. No speech is present. No music is present
```
Recommended negative prompt:
```text
music, melody, song, singing, vocals, score, soundtrack, beat, rhythm bed, instrumental backing, tinny, thin, harsh, clipped, distorted, low bitrate
```
## Examples
Example videos generated with the LoRA:
### Door close
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/examples/door-close-hf.mp4" type="video/mp4">
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/examples/door-close-hf.mp4">Download video</a>
</video>
### Pineapple slicing
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/examples/pineapple-hf.mp4" type="video/mp4">
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/examples/pineapple-hf.mp4">Download video</a>
</video>
### Race car
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/examples/racecar-hf.mp4" type="video/mp4">
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/examples/racecar-hf.mp4">Download video</a>
</video>
### Squash
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/examples/squash-hf.mp4" type="video/mp4">
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/examples/squash-hf.mp4">Download video</a>
</video>
Comparisons showing LTX-2.3 outputs with and without the Foley LoRA:
### Barista comparison
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/barista-comparison-hf.mp4" type="video/mp4">
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/barista-comparison-hf.mp4">Download video</a>
</video>
### Tennis comparison
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/tennis-comparison-hf.mp4" type="video/mp4">
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/tennis-comparison-hf.mp4">Download video</a>
</video>
## ComfyUI Workflow
Use the companion ComfyUI workflow here:
[FuzzPuppy/LTX-2.3-Foley-Workflow](https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-Workflow)