test_set/README.md

SpatialAV2AV — Test Set (held-out evaluation split)

Origin-disjoint test split for evaluating SpatialAV2AV spatial audio-video editing. This folder ships the manifests + split reports only (no media). The media (mp4 with embedded stereo audio) live in the dataset repo BingoG/LTX.

Files

File	What
test.list	11,616 metadata-JSON paths (absolute, on the source cluster) — the held-out test samples
test.relative.list	same 11,616 samples as relative paths (final_json/<clip>.json) — resolve against BingoG/LTX after snapshot_download
split_summary.json	full split statistics (origins, per-split sample & trajectory counts, seed, input SHA)
split_overlap_report.json	origin-overlap audit — all zero (train/val/test are origin-disjoint)
MANIFEST.sha256	SHA256 of train/val/test.list + input list, for downstream binding

How this split was built

• Source set: the 116,147-pair training pool (all.list, the authoritative SpatialAV2AV set — the same list the step_04000 weights trained on).
• Ratios: 80 / 10 / 10 (train / val / test), seed 20260718 (frozen).
• Grouping: origin-disjoint by video_id — one source video (and all 10 of its camera-trajectory variants) lands entirely in one split. This is what makes the test score a generalization measure, not a memorization one: if pan_left were in train and pan_right in test, the model would already have seen the same person/room/voice.

Numbers

Split	origins	samples
train	11,616	92,915
val	1,452	11,616
test	1,452	11,616

Test-set trajectory coverage (each of the 10 canonical trajectories present): pan_left/right and rotate_left/right = 1,452 each, push_in/pull_out = 1,452 each, fixed_* = 692–779 each. 1,452 origins ≫ the ≥100 needed for reliable origin-cluster bootstrap CIs.

Validation (all PASS)

1. Origin overlap — train∩val, train∩test, val∩test all 0 origins.
2. Coverage — 92,915 + 11,616 + 11,616 = 116,147 = input (no sample lost).
3. Trajectory presence — test contains all pan/rotate directions (I3 counterfactual pairs have data).

Note on how the manifest was generated

The split's deterministic core (random.Random(20260718) shuffle + greedy origin allocation) was reproduced from the filenames because the source media filesystem was temporarily unreachable for bulk access. origin_id == video_id == filename-stem-before-'+' was verified identical to the JSON contents on a 200-sample check, and .resolve() is idempotent on these paths — so the emitted lists are byte-equivalent to the official SpatialAV2AV_build_splits.py output. Counterfactual pair manifests (test_pairs.jsonl, needed only for I3 eval) are not included here; they require decoding each clip's media and will be added separately.

Usage

from huggingface_hub import hf_hub_download
p = hf_hub_download("BingoG/SpatialAV2AV", "test_set/test.relative.list")
# point the evaluator's TEST_LIST at the resolved paths under a BingoG/LTX snapshot