scripts/README_EXAMPLES.md

# Adding / refreshing curated examples for the Any-to-Any tab

Three phases, run in order, all from inside `hf_space_demo/` on the cluster
(where both the tokenized dataset and the checkpoints actually live).

## Phase 1 -- pick clips + upload tokens

```bash
python scripts/upload_examples_to_hub.py \
    --source_dir /datasets/uzair/weights_from_clariden/test_cvpr_final_set_13_mod \
    --repo_id EPFL-VILAB/Video-4M-examples \
    --num_examples 6 --seed 0
```

This randomly picks `--num_examples` clips that have a file in every modality
subfolder, uploads all of their tokenized data to the `EPFL-VILAB/Video-4M-examples`
dataset repo, and saves the picked clip IDs locally to `examples_manifest.json`
(in whatever directory you ran the command from -- keep this file, phases 2
and 3 both need it).

## Phase 2 -- render human-viewable previews

```bash
cd ..   # repo root, since the script reads hf_space_demo/examples_manifest.json
python visualize_multimodal_pretraining_data_13_modalities.py \
    --config cfgs/default/4m/models/main/4m_L_video_all_modalities_final_config.yaml
```

This reads `hf_space_demo/examples_manifest.json` and renders a
`.mp4` preview per modality for exactly those clips (not a random sample),
saved under `args.output_dir_videos` (default:
`/datasets/uzair/weights_from_clariden/cvpr_generations/GT_visualizations_post_neurips_opticalflow_fixed`),
in `<domain>_detokenized/<clip_basename>.mp4` subfolders.

## Phase 3 -- upload the previews

```bash
cd hf_space_demo
python scripts/upload_examples_to_hub.py \
    --repo_id EPFL-VILAB/Video-4M-examples \
    --detokenized_dir /datasets/uzair/weights_from_clariden/cvpr_generations/GT_visualizations_post_neurips_opticalflow_fixed \
    --previews_only
```

`--previews_only` skips re-uploading the (large, slow) tokens. Without
`--force_repick`, this automatically reuses the exact clips already recorded
in `examples_manifest.json` -- it will not pick a new random set.

Once this is done, the Any-to-Any tab's example/input-modality dropdowns will
show a live preview before you even hit Generate.

## A second curated set (e.g. for the Future Prediction tab)

The Future Prediction tab needs its own hand-picked clips (e.g. ones with
clear, consistent motion) rather than a random sample, and shouldn't
overwrite the any-to-any tab's set. Same repo, same 3 phases, just with
`--stems_file` instead of `--num_examples`/`--seed`, and a distinct
`--manifest_out`/`--manifest_repo_filename` so the two sets don't collide:

```bash
# Phase 1 -- explicit stems (one per line, e.g. "vol_12/clip_000123",
# matching the tok_video_rgb@128 subfolder layout)
python scripts/upload_examples_to_hub.py \
    --source_dir /datasets/uzair/weights_from_clariden/test_cvpr_final_set_13_mod \
    --repo_id EPFL-VILAB/Video-4M-examples \
    --stems_file my_future_pred_stems.txt \
    --manifest_out future_examples_manifest.json \
    --manifest_repo_filename future_examples.json
```

```bash
# Phase 2 -- point the visualization script at this manifest via env var
cd ..
EXAMPLES_MANIFEST_PATH=hf_space_demo/future_examples_manifest.json \
    python visualize_multimodal_pretraining_data_13_modalities.py \
    --config cfgs/default/4m/models/main/4m_L_video_all_modalities_final_config.yaml
```

```bash
# Phase 3 -- upload previews, same distinct manifest names as phase 1
cd hf_space_demo
python scripts/upload_examples_to_hub.py \
    --repo_id EPFL-VILAB/Video-4M-examples \
    --detokenized_dir /datasets/uzair/weights_from_clariden/cvpr_generations/GT_visualizations_post_neurips_opticalflow_fixed \
    --manifest_out future_examples_manifest.json \
    --manifest_repo_filename future_examples.json \
    --previews_only
```

Every stem in `--stems_file` must have a file in every modality subfolder --
the script errors out listing exactly which ones are missing, rather than
silently dropping them (unlike phase 1's random-selection path, which just
skips incomplete candidates).

## Notes

- **Lost your local `examples_manifest.json`?** It was also uploaded to the
  repo as `examples.json` in phase 1 -- pull it back down instead of
  re-picking clips:
  ```bash
  python -c "
  from huggingface_hub import hf_hub_download
  import shutil
  path = hf_hub_download(repo_id='EPFL-VILAB/Video-4M-examples', filename='examples.json', repo_type='dataset')
  shutil.copy(path, 'examples_manifest.json')
  "
  ```
  (For the future-prediction set, substitute `future_examples.json` /
  `future_examples_manifest.json`.)
- **Want a completely different set of clips?** Re-run phase 1 with
  `--force_repick` (optionally a different `--seed`/`--num_examples`, or a
  different `--stems_file`), then repeat phases 2 and 3.
- **Just fixing/re-rendering previews for the same clips?** Skip phase 1,
  start at phase 2 -- the manifest already has the clip IDs.