Download code/evaluation/README.md from luyu1021/turbot2va-ltx2: direct link, hf CLI and curl.
- Browser
- Download file 2.31 kB
-
https://huggingface.co/luyu1021/turbot2va-ltx2/resolve/main/code/evaluation/README.md
- Command line
-
hf download hf://luyu1021/turbot2va-ltx2/code/evaluation/README.md
-
curl -L -o README.md https://huggingface.co/luyu1021/turbot2va-ltx2/resolve/main/code/evaluation/README.md
turbot2av-eval
Evaluation utilities for TurboT2AV quality and diversity experiments.
The repository intentionally stores code, configs, and documentation only. Large assets such as videos, audio, checkpoints, ImageBind weights, and JavisBench outputs should stay on local storage or Hugging Face datasets.
What This Runs
- Quality: calls JavisBench from
JavisDiTon one generated sample per prompt. - Diversity generation: calls the TurboT2AV/LTX inference script to produce
num_prompts x num_seedssamples. - Diversity metrics: computes same-prompt pairwise ImageBind distances over video, audio, and joint audio-video embeddings.
Expected Layout
The default config points to the current lab paths:
JavisDiT:/home/jovyan/codes/turbodiff/new_Turbo/JavisDiTLTX-2:/home/jovyan/codes/turbodiff/new_Turbo/TurboT2AV/LTX-2- data root:
/data/datasets/turbodiff_datasets_and_ckpt - diversity outputs:
/data/datasets/turbodiff_datasets_and_ckpt/main_experiment/diversity_200x8
Edit configs/turbot2av_main.yaml when moving to another machine or adding a
new model.
Quick Commands
Install the lightweight launcher environment:
pixi install
Run JavisBench quality on existing diversity seed0000 outputs:
pixi run quality --gpus 0,1,2,3 --only 05_scm1_dmd1_pgmean05_ckpt5000
Generate 200 prompts x 8 seeds, one model per GPU:
pixi run diversity-generate --gpus 0,1,2,3 --only 05_scm1_dmd1_pgmean05_ckpt5000
Compute ImageBind pairwise diversity:
pixi run diversity-metrics --gpu 0 --only 05_scm1_dmd1_pgmean05_ckpt5000
Merge quality and diversity outputs into a compact CSV/Markdown table:
pixi run summary
All scripts remove proxy variables from child processes by default.
Outputs
Quality:
<quality_output_root>/<model>.json
<quality_output_root>/batch_summary.json
Diversity generation:
<diversity_root>/<model>/sample_0000_seed0000.mp4
<diversity_root>/<model>/sample_0000_seed0000.wav
<diversity_root>/generation_summary.json
Diversity metrics:
<diversity_metric_root>/summary.csv
<diversity_metric_root>/summary.json
<diversity_metric_root>/embeddings/<model>.pt
Merged summary:
results/metrics_summary.csv
results/metrics_summary.md