Instructions to use BingoG/SpatialAV2AV with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LTX.io
How to use BingoG/SpatialAV2AV with LTX.io:
# Install the LTX-2 pipelines git clone https://github.com/Lightricks/LTX-2.git cd LTX-2 uv sync --frozen
# Download the weights from this repo, plus the Gemma text encoder hf download BingoG/SpatialAV2AV --local-dir models/SpatialAV2AV hf download google/gemma-3-12b-it-qat-q4_0-unquantized --local-dir models/gemma-3-12b
# Fast pipeline (distilled model, no distilled LoRA needed) uv run python -m ltx_pipelines.distilled \ --distilled-checkpoint-path models/SpatialAV2AV/<distilled-checkpoint>.safetensors \ --spatial-upsampler-path models/SpatialAV2AV/<spatial-upsampler>.safetensors \ --gemma-root models/gemma-3-12b \ --prompt "A beautiful sunset over the ocean" \ --output-path output.mp4 # For image-to-video, add: --image path/to/image.jpg 0 0.8# HQ pipeline (two-stage, higher quality) uv run python -m ltx_pipelines.ti2vid_two_stages_hq \ --checkpoint-path models/SpatialAV2AV/<checkpoint>.safetensors \ --distilled-lora models/SpatialAV2AV/<distilled-lora>.safetensors 0.8 \ --spatial-upsampler-path models/SpatialAV2AV/<spatial-upsampler>.safetensors \ --gemma-root models/gemma-3-12b \ --prompt "A beautiful sunset over the ocean" \ --output-path output.mp4 # For image-to-video, add: --image path/to/image.jpg 0 0.8 - Notebooks
- Google Colab
- Kaggle
Add README for 1000-sample test media
Browse files
test_set_media_1000/README.md
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# test_set_media_1000 — self-contained media for 1000 test samples
|
| 2 |
+
|
| 3 |
+
Full media for the **first 1,000 test samples** (pan/rotate trajectories) of the
|
| 4 |
+
SpatialAV2AV held-out test split, so the test set is directly usable **without** the
|
| 5 |
+
`BingoG/LTX` dataset repo.
|
| 6 |
+
|
| 7 |
+
## Why this exists
|
| 8 |
+
|
| 9 |
+
The test manifests in [`test_set/`](../test_set) list 11,616 samples drawn from the 116k
|
| 10 |
+
`all.list` split. The dataset repo `BingoG/LTX` only carries a 4,400-sample debug subset,
|
| 11 |
+
so it does **not** cover this test set. This folder ships real, self-contained media for a
|
| 12 |
+
usable 1,000-sample slice (pan/rotate — the trajectories the I3 counterfactual evaluation
|
| 13 |
+
needs).
|
| 14 |
+
|
| 15 |
+
## Contents (4,251 files, ~2.24 GB)
|
| 16 |
+
|
| 17 |
+
| Folder | Count | What |
|
| 18 |
+
|--------|------:|------|
|
| 19 |
+
| `final_json/` | 1000 | metadata JSON per sample (relative media paths inside) |
|
| 20 |
+
| `final_source/` | 1000 | source (condition) video, mp4 with embedded stereo |
|
| 21 |
+
| `final_edit/` | 1000 | edit (target) video, mp4 with embedded stereo |
|
| 22 |
+
| `final_audio_edit/` | 1000 | target stereo audio, wav (used for tILD / rank GT) |
|
| 23 |
+
| `final_audio_src/` | 250 | source stereo audio, wav (shared per origin) |
|
| 24 |
+
| `samples_1000.relative.list` | 1 | the 1,000 sample json paths (relative) |
|
| 25 |
+
|
| 26 |
+
Layout mirrors the `speakervid` dataset root, so each json's relative paths
|
| 27 |
+
(`final_source/<clip>.mp4`, etc.) resolve against this folder directly.
|
| 28 |
+
|
| 29 |
+
## Usage
|
| 30 |
+
|
| 31 |
+
```python
|
| 32 |
+
from huggingface_hub import snapshot_download
|
| 33 |
+
root = snapshot_download("BingoG/SpatialAV2AV", allow_patterns="test_set_media_1000/*")
|
| 34 |
+
base = f"{root}/test_set_media_1000"
|
| 35 |
+
# each final_json/<clip>.json has relative paths (final_source/…, final_edit/…,
|
| 36 |
+
# final_audio_edit/…) that resolve under `base` — point the evaluator there.
|
| 37 |
+
```
|
| 38 |
+
|
| 39 |
+
## Scope note
|
| 40 |
+
|
| 41 |
+
- These 1,000 are a **subset** of the full 11,616-sample test set (`test_set/test.list`).
|
| 42 |
+
They are all **pan/rotate** samples. The remaining test samples (other pan/rotate +
|
| 43 |
+
fixed/push_in/pull_out) are not included here yet.
|
| 44 |
+
- Media is real 2-channel/44.1 kHz stereo embedded in the mp4s, plus explicit stereo wavs.
|