Instructions to use BingoG/SpatialAV2AV with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LTX.io
How to use BingoG/SpatialAV2AV with LTX.io:
# Install the LTX-2 pipelines git clone https://github.com/Lightricks/LTX-2.git cd LTX-2 uv sync --frozen
# Download the weights from this repo, plus the Gemma text encoder hf download BingoG/SpatialAV2AV --local-dir models/SpatialAV2AV hf download google/gemma-3-12b-it-qat-q4_0-unquantized --local-dir models/gemma-3-12b
# Fast pipeline (distilled model, no distilled LoRA needed) uv run python -m ltx_pipelines.distilled \ --distilled-checkpoint-path models/SpatialAV2AV/<distilled-checkpoint>.safetensors \ --spatial-upsampler-path models/SpatialAV2AV/<spatial-upsampler>.safetensors \ --gemma-root models/gemma-3-12b \ --prompt "A beautiful sunset over the ocean" \ --output-path output.mp4 # For image-to-video, add: --image path/to/image.jpg 0 0.8# HQ pipeline (two-stage, higher quality) uv run python -m ltx_pipelines.ti2vid_two_stages_hq \ --checkpoint-path models/SpatialAV2AV/<checkpoint>.safetensors \ --distilled-lora models/SpatialAV2AV/<distilled-lora>.safetensors 0.8 \ --spatial-upsampler-path models/SpatialAV2AV/<spatial-upsampler>.safetensors \ --gemma-root models/gemma-3-12b \ --prompt "A beautiful sunset over the ocean" \ --output-path output.mp4 # For image-to-video, add: --image path/to/image.jpg 0 0.8 - Notebooks
- Google Colab
- Kaggle
step_04000 — SpatialAV2AV checkpoint
Latest & best-tested full fine-tune weights for LTX-2.3 (22B) spatial audio-video editing.
Provenance
| Field | Value |
|---|---|
| Source path | /apdcephfs_zwfy11/share_305172035/helensliang/Projects/LTX-2-SpatialAV2AV/outputs_SpatialAV2AV_train/2026.06.19-17.33.11/checkpoints/model_weights_step_04000.safetensors |
| Training run | 2026.06.19-17.33.11 |
| Step | 4000 / 20000 (checkpoints saved every 1000 steps) |
| Checkpoint written | 2026-06-22 14:05 |
| File size | 37,979,154,726 bytes (~36 GiB) |
| Base model | LTX-2.3/ltx-2.3-22b-dev.safetensors (full-parameter fine-tune, FSDP FULL_SHARD, 8×H20) |
| Training time to this step | ~341 h wall (60 s/step) |
Why this checkpoint
Last checkpoint of the run and the one used in the final inference tests
(test8_step4000_* / realtest8_step4000_*, run 2026-07-05). Video-dominant loss stayed low
and stable; stereo L≠R verified on decode.
Loss (video weight 0.85 / audio weight 0.15)
| Step | Loss | v_loss |
|---|---|---|
| 1000 | 0.1089 | 0.0937 |
| 2000 | 0.1528 | 0.1555 |
| 3000 | 0.1099 | 0.1051 |
| 4000 | 0.1191 | 0.1221 |
Key training config
learning_rate: 1.0e-05,steps: 20000,batch_size: 1, gradient checkpointing ontimestep_sampling_mode: shifted_logit_normalloss_video_weight: 0.85,loss_audio_weight: 0.15,with_audio: true,use_ref_audio: falseedit_sample_n_frames: 113,edit_min_size: 288resolution_buckets: 288x384x113; 384x288x113; 320x320x113; 352x352x113training_strategy.name: spatialav2av- data:
SpatialAV2AV_full/all.list(source→edit stereo pairs)
⚠️ Format quirk — read before loading
This .safetensors file is actually a torch.save (zip/pickle) file, not safetensors.
Magic bytes = PK\x03\x04.
- ❌
safetensorsloaders fail withHeaderTooLarge. - ✅
torch.load()+transformer.load_state_dict(..., strict=False)→missing=0 unexpected=0.
Use SpatialAV2AV_inference.py --full_weights <this file> (it handles the injection).