luyu1021's picture
Add documented LTX-2 training source, recipes and checkpoint release plan
76b7b25 verified
|
Raw History Blame Contribute Delete
2.31 kB
# turbot2av-eval
Evaluation utilities for TurboT2AV quality and diversity experiments.
The repository intentionally stores code, configs, and documentation only.
Large assets such as videos, audio, checkpoints, ImageBind weights, and
JavisBench outputs should stay on local storage or Hugging Face datasets.
## What This Runs
- Quality: calls JavisBench from `JavisDiT` on one generated sample per prompt.
- Diversity generation: calls the TurboT2AV/LTX inference script to produce
`num_prompts x num_seeds` samples.
- Diversity metrics: computes same-prompt pairwise ImageBind distances over
video, audio, and joint audio-video embeddings.
## Expected Layout
The default config points to the current lab paths:
- `JavisDiT`: `/home/jovyan/codes/turbodiff/new_Turbo/JavisDiT`
- `LTX-2`: `/home/jovyan/codes/turbodiff/new_Turbo/TurboT2AV/LTX-2`
- data root: `/data/datasets/turbodiff_datasets_and_ckpt`
- diversity outputs: `/data/datasets/turbodiff_datasets_and_ckpt/main_experiment/diversity_200x8`
Edit `configs/turbot2av_main.yaml` when moving to another machine or adding a
new model.
## Quick Commands
Install the lightweight launcher environment:
```bash
pixi install
```
Run JavisBench quality on existing diversity seed0000 outputs:
```bash
pixi run quality --gpus 0,1,2,3 --only 05_scm1_dmd1_pgmean05_ckpt5000
```
Generate 200 prompts x 8 seeds, one model per GPU:
```bash
pixi run diversity-generate --gpus 0,1,2,3 --only 05_scm1_dmd1_pgmean05_ckpt5000
```
Compute ImageBind pairwise diversity:
```bash
pixi run diversity-metrics --gpu 0 --only 05_scm1_dmd1_pgmean05_ckpt5000
```
Merge quality and diversity outputs into a compact CSV/Markdown table:
```bash
pixi run summary
```
All scripts remove proxy variables from child processes by default.
## Outputs
Quality:
```text
<quality_output_root>/<model>.json
<quality_output_root>/batch_summary.json
```
Diversity generation:
```text
<diversity_root>/<model>/sample_0000_seed0000.mp4
<diversity_root>/<model>/sample_0000_seed0000.wav
<diversity_root>/generation_summary.json
```
Diversity metrics:
```text
<diversity_metric_root>/summary.csv
<diversity_metric_root>/summary.json
<diversity_metric_root>/embeddings/<model>.pt
```
Merged summary:
```text
results/metrics_summary.csv
results/metrics_summary.md
```