Add step_04000 INFO
Browse files- step_04000/INFO.md +50 -0
step_04000/INFO.md
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# step_04000 — SpatialAV2AV checkpoint
|
| 2 |
+
|
| 3 |
+
**Latest & best-tested full fine-tune weights** for LTX-2.3 (22B) spatial audio-video editing.
|
| 4 |
+
|
| 5 |
+
## Provenance
|
| 6 |
+
|
| 7 |
+
| Field | Value |
|
| 8 |
+
|-------|-------|
|
| 9 |
+
| Source path | `/apdcephfs_zwfy11/share_305172035/helensliang/Projects/LTX-2-SpatialAV2AV/outputs_SpatialAV2AV_train/2026.06.19-17.33.11/checkpoints/model_weights_step_04000.safetensors` |
|
| 10 |
+
| Training run | `2026.06.19-17.33.11` |
|
| 11 |
+
| Step | **4000 / 20000** (checkpoints saved every 1000 steps) |
|
| 12 |
+
| Checkpoint written | 2026-06-22 14:05 |
|
| 13 |
+
| File size | 37,979,154,726 bytes (~36 GiB) |
|
| 14 |
+
| Base model | `LTX-2.3/ltx-2.3-22b-dev.safetensors` (full-parameter fine-tune, FSDP FULL_SHARD, 8×H20) |
|
| 15 |
+
| Training time to this step | ~341 h wall (60 s/step) |
|
| 16 |
+
|
| 17 |
+
## Why this checkpoint
|
| 18 |
+
|
| 19 |
+
Last checkpoint of the run and the one used in the final inference tests
|
| 20 |
+
(`test8_step4000_*` / `realtest8_step4000_*`, run 2026-07-05). Video-dominant loss stayed low
|
| 21 |
+
and stable; stereo L≠R verified on decode.
|
| 22 |
+
|
| 23 |
+
## Loss (video weight 0.85 / audio weight 0.15)
|
| 24 |
+
|
| 25 |
+
| Step | Loss | v_loss |
|
| 26 |
+
|------|------|--------|
|
| 27 |
+
| 1000 | 0.1089 | 0.0937 |
|
| 28 |
+
| 2000 | 0.1528 | 0.1555 |
|
| 29 |
+
| 3000 | 0.1099 | 0.1051 |
|
| 30 |
+
| **4000** | **0.1191** | **0.1221** |
|
| 31 |
+
|
| 32 |
+
## Key training config
|
| 33 |
+
|
| 34 |
+
- `learning_rate: 1.0e-05`, `steps: 20000`, `batch_size: 1`, gradient checkpointing on
|
| 35 |
+
- `timestep_sampling_mode: shifted_logit_normal`
|
| 36 |
+
- `loss_video_weight: 0.85`, `loss_audio_weight: 0.15`, `with_audio: true`, `use_ref_audio: false`
|
| 37 |
+
- `edit_sample_n_frames: 113`, `edit_min_size: 288`
|
| 38 |
+
- `resolution_buckets: 288x384x113; 384x288x113; 320x320x113; 352x352x113`
|
| 39 |
+
- `training_strategy.name: spatialav2av`
|
| 40 |
+
- data: `SpatialAV2AV_full/all.list` (source→edit stereo pairs)
|
| 41 |
+
|
| 42 |
+
## ⚠️ Format quirk — read before loading
|
| 43 |
+
|
| 44 |
+
This `.safetensors` file is **actually a `torch.save` (zip/pickle) file**, not safetensors.
|
| 45 |
+
Magic bytes = `PK\x03\x04`.
|
| 46 |
+
|
| 47 |
+
- ❌ `safetensors` loaders fail with `HeaderTooLarge`.
|
| 48 |
+
- ✅ `torch.load()` + `transformer.load_state_dict(..., strict=False)` → `missing=0 unexpected=0`.
|
| 49 |
+
|
| 50 |
+
Use `SpatialAV2AV_inference.py --full_weights <this file>` (it handles the injection).
|