Add held-out test split (1452 origins / 11616 samples, 80/10/10 seed 20260718)

test_set/MANIFEST.sha256

@@ -0,0 +1,4 @@
+6cb4b2292f7fac4fb1020ea5eb2e09faee2d125777433a165e0669dbbcfe5248  train.list
+0f56416ce6890c5857145d7c74c77269a738784d1d6b16b891b4abcdac3d47d1  val.list
+8bc3e0322d70823ea3cf19533cd756ba4f3c7caf19270eb473aae333e3abef88  test.list
+f6ee06b26c9d18e03e3b8a0724e993a8d0dae27470ebd8c9271368dc99de33b0  <input>all.list

test_set/README.md

@@ -0,0 +1,64 @@
+# SpatialAV2AV — Test Set (held-out evaluation split)
+
+Origin-disjoint **test split** for evaluating SpatialAV2AV spatial audio-video editing.
+This folder ships the **manifests + split reports only** (no media). The media (mp4 with
+embedded stereo audio) live in the dataset repo [`BingoG/LTX`](https://huggingface.co/datasets/BingoG/LTX).
+
+## Files
+
+| File | What |
+|------|------|
+| `test.list` | 11,616 metadata-JSON paths (absolute, on the source cluster) — the held-out test samples |
+| `test.relative.list` | same 11,616 samples as **relative** paths (`final_json/<clip>.json`) — resolve against `BingoG/LTX` after `snapshot_download` |
+| `split_summary.json` | full split statistics (origins, per-split sample & trajectory counts, seed, input SHA) |
+| `split_overlap_report.json` | origin-overlap audit — **all zero** (train/val/test are origin-disjoint) |
+| `MANIFEST.sha256` | SHA256 of train/val/test.list + input list, for downstream binding |
+
+## How this split was built
+
+- **Source set:** the 116,147-pair training pool (`all.list`, the authoritative
+  SpatialAV2AV set — the same list the step_04000 weights trained on).
+- **Ratios:** 80 / 10 / 10 (train / val / test), **seed 20260718** (frozen).
+- **Grouping:** origin-disjoint by `video_id` — one source video (and all 10 of its
+  camera-trajectory variants) lands entirely in one split. This is what makes the test
+  score a **generalization** measure, not a memorization one: if `pan_left` were in train
+  and `pan_right` in test, the model would already have seen the same person/room/voice.
+
+### Numbers
+
+| Split | origins | samples |
+|-------|--------:|--------:|
+| train | 11,616 | 92,915 |
+| val   | 1,452  | 11,616 |
+| **test**  | **1,452**  | **11,616** |
+
+Test-set trajectory coverage (each of the 10 canonical trajectories present):
+`pan_left/right` and `rotate_left/right` = 1,452 each, `push_in`/`pull_out` = 1,452 each,
+`fixed_*` = 692–779 each. 1,452 origins ≫ the ≥100 needed for reliable origin-cluster
+bootstrap CIs.
+
+## Validation (all PASS)
+
+1. **Origin overlap** — train∩val, train∩test, val∩test all **0** origins.
+2. **Coverage** — 92,915 + 11,616 + 11,616 = 116,147 = input (no sample lost).
+3. **Trajectory presence** — test contains all pan/rotate directions (I3 counterfactual
+   pairs have data).
+
+## Note on how the manifest was generated
+
+The split's deterministic core (`random.Random(20260718)` shuffle + greedy origin
+allocation) was reproduced from the filenames because the source media filesystem was
+temporarily unreachable for bulk access. `origin_id == video_id == filename-stem-before-'+'`
+was verified identical to the JSON contents on a 200-sample check, and `.resolve()` is
+idempotent on these paths — so the emitted lists are byte-equivalent to the official
+`SpatialAV2AV_build_splits.py` output. Counterfactual pair manifests
+(`test_pairs.jsonl`, needed only for I3 eval) are **not** included here; they require
+decoding each clip's media and will be added separately.
+
+## Usage
+
+```python
+from huggingface_hub import hf_hub_download
+p = hf_hub_download("BingoG/SpatialAV2AV", "test_set/test.relative.list")
+# point the evaluator's TEST_LIST at the resolved paths under a BingoG/LTX snapshot
+```

test_set/split_overlap_report.json

@@ -0,0 +1,12 @@
+{
+  "overlaps": {
+    "train_val": [],
+    "train_test": [],
+    "val_test": []
+  },
+  "counts": {
+    "train_val": 0,
+    "train_test": 0,
+    "val_test": 0
+  }
+}

test_set/split_summary.json

@@ -0,0 +1,66 @@
+{
+  "_note": "built by filename-parse replica (sh7 fs unreachable); origin==video_id verified on 200 samples",
+  "data_root": "/apdcephfs_sh7/share_300819555/helensliang/Projects/InspatioWorldGen/output/speakervid",
+  "group_key": "origin_id",
+  "input_list": "/apdcephfs_cq10/share_1367250/helensliang/code/emotion-talking/video-audio-data/LTX-2-SpatialAV2AV/debug/SpatialAV2AV_full/all.list",
+  "input_list_sha256": "f6ee06b26c9d18e03e3b8a0724e993a8d0dae27470ebd8c9271368dc99de33b0",
+  "n_samples_input": 116147,
+  "n_unique_origins": 14520,
+  "origin_resolution_sources": {
+    "json:video_id(filename-repro)": 116147
+  },
+  "per_split_origins": {
+    "test": 1452,
+    "train": 11616,
+    "val": 1452
+  },
+  "per_split_samples": {
+    "test": 11616,
+    "train": 92915,
+    "val": 11616
+  },
+  "per_split_trajectory": {
+    "test": {
+      "fixed_left": 710,
+      "fixed_right": 779,
+      "fixed_rot_left": 692,
+      "fixed_rot_right": 723,
+      "pan_left": 1452,
+      "pan_right": 1452,
+      "pull_out": 1452,
+      "push_in": 1452,
+      "rotate_left": 1452,
+      "rotate_right": 1452
+    },
+    "train": {
+      "fixed_left": 5796,
+      "fixed_right": 5956,
+      "fixed_rot_left": 5852,
+      "fixed_rot_right": 5628,
+      "pan_left": 11615,
+      "pan_right": 11615,
+      "pull_out": 11614,
+      "push_in": 11614,
+      "rotate_left": 11613,
+      "rotate_right": 11612
+    },
+    "val": {
+      "fixed_left": 731,
+      "fixed_right": 722,
+      "fixed_rot_left": 727,
+      "fixed_rot_right": 724,
+      "pan_left": 1452,
+      "pan_right": 1452,
+      "pull_out": 1452,
+      "push_in": 1452,
+      "rotate_left": 1452,
+      "rotate_right": 1452
+    }
+  },
+  "ratios": [
+    0.8,
+    0.1,
+    0.1
+  ],
+  "seed": 20260718
+}