File size: 3,241 Bytes
797378a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 | # SpatialAV2AV — Test Set (held-out evaluation split)
Origin-disjoint **test split** for evaluating SpatialAV2AV spatial audio-video editing.
This folder ships the **manifests + split reports only** (no media). The media (mp4 with
embedded stereo audio) live in the dataset repo [`BingoG/LTX`](https://huggingface.co/datasets/BingoG/LTX).
## Files
| File | What |
|------|------|
| `test.list` | 11,616 metadata-JSON paths (absolute, on the source cluster) — the held-out test samples |
| `test.relative.list` | same 11,616 samples as **relative** paths (`final_json/<clip>.json`) — resolve against `BingoG/LTX` after `snapshot_download` |
| `split_summary.json` | full split statistics (origins, per-split sample & trajectory counts, seed, input SHA) |
| `split_overlap_report.json` | origin-overlap audit — **all zero** (train/val/test are origin-disjoint) |
| `MANIFEST.sha256` | SHA256 of train/val/test.list + input list, for downstream binding |
## How this split was built
- **Source set:** the 116,147-pair training pool (`all.list`, the authoritative
SpatialAV2AV set — the same list the step_04000 weights trained on).
- **Ratios:** 80 / 10 / 10 (train / val / test), **seed 20260718** (frozen).
- **Grouping:** origin-disjoint by `video_id` — one source video (and all 10 of its
camera-trajectory variants) lands entirely in one split. This is what makes the test
score a **generalization** measure, not a memorization one: if `pan_left` were in train
and `pan_right` in test, the model would already have seen the same person/room/voice.
### Numbers
| Split | origins | samples |
|-------|--------:|--------:|
| train | 11,616 | 92,915 |
| val | 1,452 | 11,616 |
| **test** | **1,452** | **11,616** |
Test-set trajectory coverage (each of the 10 canonical trajectories present):
`pan_left/right` and `rotate_left/right` = 1,452 each, `push_in`/`pull_out` = 1,452 each,
`fixed_*` = 692–779 each. 1,452 origins ≫ the ≥100 needed for reliable origin-cluster
bootstrap CIs.
## Validation (all PASS)
1. **Origin overlap** — train∩val, train∩test, val∩test all **0** origins.
2. **Coverage** — 92,915 + 11,616 + 11,616 = 116,147 = input (no sample lost).
3. **Trajectory presence** — test contains all pan/rotate directions (I3 counterfactual
pairs have data).
## Note on how the manifest was generated
The split's deterministic core (`random.Random(20260718)` shuffle + greedy origin
allocation) was reproduced from the filenames because the source media filesystem was
temporarily unreachable for bulk access. `origin_id == video_id == filename-stem-before-'+'`
was verified identical to the JSON contents on a 200-sample check, and `.resolve()` is
idempotent on these paths — so the emitted lists are byte-equivalent to the official
`SpatialAV2AV_build_splits.py` output. Counterfactual pair manifests
(`test_pairs.jsonl`, needed only for I3 eval) are **not** included here; they require
decoding each clip's media and will be added separately.
## Usage
```python
from huggingface_hub import hf_hub_download
p = hf_hub_download("BingoG/SpatialAV2AV", "test_set/test.relative.list")
# point the evaluator's TEST_LIST at the resolved paths under a BingoG/LTX snapshot
```
|