LTX-2 — how the IC-LoRA training sets were built

Reconstructed 2026-08-21 from the build scripts, the shell history, and the data itself. Everything stated here was checked against the files unless marked inferred.

This directory holds locally built training sets for LTX-2 IC-LoRA (in-context LoRA) video-to-video training. Nothing here is downloaded — each set is derived from a raw dataset that lives in a sibling folder of /mnt/dataset/xinyuy/datasets/.

Plus two re-encodings of existing sets for a different backbone (*-wan-latents), a snapshot of the older /mnt/data LTX directory (old/), and reference LoRAs (cameraman/). Those are at the bottom.

Two further sets condition on a reference image rather than a reference video. They belong to the earlier /mnt/data/xinyuy/datasets/LTX/ generation and are built elsewhere, but they feed the same trainer and are documented here too — see Image-reference sets:

---

The shared contract

Every set has the same flat layout, so any of them drops into the same trainer:

<Dataset>/
├── <stem>.mp4              target — the video the model learns to generate
├── <stem>_<kind>.mp4       reference — the control signal (kind = pose|depth|painting|camera)
├── … × 2,000 pairs
├── dataset.json            [{caption, media_path, reference_path}, …]
└── .precomputed/           written by the LTX-2 trainer, not by the build scripts
    ├── latents/            <stem>.pt   target VAE latents
    ├── reference_latents/  <stem>.pt   reference VAE latents
    ├── conditions/         <stem>.pt   Gemma3 text embeddings
    └── multi_gpu_logs/     shard_N.log

dataset.json entries are exactly three keys, paths relative to the dataset dir:

{
  "caption": "A woman with blue body paint on her face …",
  "media_path": "10026939-hd_1920_1080_30fps.mp4",
  "reference_path": "10026939-hd_1920_1080_30fps_pose.mp4"
}

The trainer's preprocessed_data_root points at the dataset root, not at .precomputed — PRECOMPUTED_DIR_NAME = ".precomputed" is appended automatically (packages/ltx-trainer/src/ltx_trainer/datasets.py:12).

File counts are 4001 per set (2,000 pairs + dataset.json) and ~6,00x files under .precomputed.

The image-reference sets use the same contract, with reference_path naming a .png or .jpg instead of an .mp4; Style-Image-Dataset adds a fourth key, style_prompt.

---

The pipeline, in five stages

Every reference-video set follows the same shape; only stages 1–3 differ per task. (The image-reference sets swap stage 2 for a frame-cropping step — see their own section.)

1. SELECT & TRIM     raw corpus ──► N clips of exactly 81 frames
2. REFERENCE         clips ──────► the control video (depth / pose / mask / camera-twin)
3. CAPTION           clips ──────► one caption per clip
4. BUILD             ────────────► flat {stem}.mp4 + {stem}_<kind>.mp4 + dataset.json
5. PRECOMPUTE        ────────────► .precomputed/{latents,reference_latents,conditions}

Stages 1–4 run in the source dataset folder (<Source>/processing/*.py), under conda env flexcombine-data. Stage 5 runs in ~/flexvideo/LTX-2 under its uv environment. The tooling for stage 2 is shared: ~/dataset_processing/flexcombine-bench/tools/compute_reference.py.

Every build script takes --seed 0 and samples with random.Random(seed).sample(...) over a sorted eligible list — so selections are exactly reproducible. Verified: re-running the selection logic for both Pose-Dataset and Depth-Dataset reproduces the shipped 2,000 stems identically.

---

Pose-Dataset

Source: HumanVID/ — 11,403 horizontal Pexels videos downloaded with aria2c (download_videos.sh, URL list pexels-horizontal-urls-new.txt; failed_videos.txt records what never landed).

1. Select & trim

cd /mnt/dataset/xinyuy/datasets/HumanVID
python ./processing/select_and_trim_videos.py --output-dir selected_videos_dedup --dedup

Keeps the first 81 frames of each selected video. Frame counts come from ffprobe container metadata and the trim is ffmpeg -c copy, so nothing is ever decoded or re-encoded — the stage stays I/O-bound.

--dedup is the important flag and the reason this set exists twice. Pexels clips whose filenames share the first four characters are shoots of the same person; without the flag the same human recurs across the training set. Dedup collapses those groups to one clip each, which is why selected_videos_dedup/ holds 2,415 files against selected_videos/'s 3,003 (the target was 3,000 either way — dedup ran out of distinct groups). This is the note in ../Notes.md.

2. Pose reference — DWPose

cd ~/dataset_processing/flexcombine-bench/tools
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
python compute_reference.py /mnt/dataset/xinyuy/datasets/HumanVID/selected_videos_dedup \
    --type pose --dwpose-repo-path DWPose \
    --output-path /mnt/dataset/xinyuy/datasets/HumanVID/selected_videos_dedup/pose \
    --num-workers 8

Writes <stem>_pose.mp4 skeleton videos — 2,413 of 2,415 clips succeeded.

Adding --no-face renders skeletons without facial keypoints, into a parallel pose_no_face/ folder. That was only ever done for the pre-dedup selected_videos/, and fed Pose-No-Face-Dataset-Legacy. selected_videos_dedup/ has no pose_no_face/.

3. Caption — Gemini

python processing/caption.py selected_videos_dedup --model gemini-2.5-flash --concurrency 8

Writes selected_videos_dedup/captions.json — a flat {stem: caption} dict, 2,402 entries. Needs $GEMINI_API_KEY. The built-in prompt asks for a 35–40 word factual caption covering people count, appearance, scene, broad activity, and mood, and explicitly forbids describing limb positions or pose sequences — the pose skeleton carries that, so putting it in the text too would teach the model to read it from the wrong place.

4. Build

Inferred — this exact invocation is not in shell history (it ran inside a claude session), but the resulting selection reproduces bit-for-bit from these arguments:

python processing/build_pose_dataset.py \
    --videos-dir selected_videos_dedup \
    --pose-dir selected_videos_dedup/pose \
    --pose-no-face-dir selected_videos_dedup/pose \
    --reference-path selected_videos_dedup/pose \
    --output /mnt/dataset/xinyuy/datasets/LTX-2/Pose-Dataset \
    --num-videos 2000 --link-mode copy

--pose-no-face-dir has to be given something that exists — the script validates it even when it isn't the reference — hence pointing it at pose/. Eligible pool: 2,402 stems (present in the videos dir and captions.json and pose/), sample 2,000 at seed 0.

--link-mode copy because the destination is a different concern from the source; the script defaults to hardlink and falls back to a copy across filesystems.

Result: 4.5 GB of media, references are byte-identical copies of selected_videos_dedup/pose/ (verified by md5).

Legacy variants

Pose-Dataset-Legacy and Pose-No-Face-Dataset-Legacy were built 2026-07-07/09 from the pre-dedup selected_videos/, and renamed with mv on 2026-07-09 21:01 when the duplicate-human problem surfaced:

python processing/build_pose_dataset.py --output …/LTX-2/Pose-Dataset \
    --num-videos 2000 --reference-path selected_videos/pose --link-mode copy
python processing/build_pose_dataset.py --output …/LTX-2/Pose-No-Face-Dataset \
    --num-videos 2000 --reference-path selected_videos/pose_no_face --link-mode copy

The two legacy sets share the same 2,000 stems and the same captions — they differ only in whether the skeleton includes the face. The current Pose-Dataset overlaps them by 1,145 clips.

---

Depth-Dataset

Source: VIDGEN-1M/ — Fudan-FUXI/VIDGEN-1M, 2,048 zips extracted to ~1.01 M clips, with VidGen_1M_video_caption.json shipped upstream.

1. Select & trim

cd /mnt/dataset/xinyuy/datasets/VIDGEN-1M
python ./processing/select_and_trim_videos.py --output-dir selected_videos \
    --caption-file VidGen_1M_video_caption.json

This is a different implementation from the HumanVID one, and deliberately so: it decodes with PyAV and re-encodes the 81 kept frames at the source fps, instead of ffmpeg -c copy. Stream-copy trims on this corpus produce container timestamp / edit-list oddities; a single decode pass avoids them. --caption-file restricts the candidate pool to clips that actually have an upstream caption. Result: 3,000 clips.

2. Depth reference — Video-Depth-Anything

cd ~/dataset_processing/flexcombine-bench/tools
python compute_reference.py \
    /mnt/dataset/xinyuy/datasets/VIDGEN-1M/selected_videos \
    --type depth --depth-repo-path Video-Depth-Anything --encoder vitl \
    --output-path /mnt/dataset/xinyuy/datasets/VIDGEN-1M/selected_videos/depth \
    --num-workers 8

3,000 <stem>_depth.mp4 written — full coverage.

3. Caption

None. Captions come straight from upstream VidGen_1M_video_caption.json (a list of {vid, caption}), loaded by the builder.

4. Build

python processing/build_depth_dataset.py \
    --output /mnt/dataset/xinyuy/datasets/LTX-2/Depth-Dataset \
    --num-videos 2000 --reference-path selected_videos/depth --link-mode copy

Eligible pool: all 3,000; sample 2,000 at seed 0. Result: 2.4 GB of media.

Note the flag churn in history — --depth-path and --depth-dir were both tried and rejected before --reference-path stuck. The script's option is --depth-dir; the command that actually ran used --reference-path, which argparse accepts as an unambiguous prefix of nothing else. Use --depth-dir if rebuilding.

---

Inpainting-Dataset

Source: VPData/ (TencentARC VideoPainter) via the VPdata_processing/ pipeline. This one is the most involved, because the control clips cover only a sub-range of their raw video.

Full upstream-processing documentation lives in ../VPdata_processing/README.md — mask extraction, download_pexels.py, overlay_masks.py, outpaint_masks.py. Summary of what feeds this set:

• VPData/selected_videos/ — 3,000 raw Pexels videos
• VPData/inpainting/ — 3,404 control clips, object interior blacked out
• VPData/outpainting/ — 3,000 control clips, everything outside a box blacked out
• VPData/pexels_videovo_train_dataset.csv — 391,416 rows of path, start_frame, end_frame, fps, mask_id, caption

Control clips are named {12-digit index}_{videoId}_{start}-{end}.mp4 and cover only frames [start, end] of the raw video.

Build

cd /mnt/dataset/xinyuy/datasets/VPData
python processing/build_inpainting_outpainting_dataset.py \
    --output /mnt/dataset/xinyuy/datasets/LTX-2/Inpainting-Dataset \
    --csv pexels_videovo_train_dataset.csv --workers 32

Two things make this script different from the pose/depth builders:

1. The target is re-encoded, not linked. Because the control clip covers only [start, end], the {stem}.mp4 target is freshly trimmed out of selected_videos/<videoId>.mp4 to that exact frame range (PyAV, libx264, yuv420p, preset=veryfast) so the pair stays frame-aligned. Only {stem}_painting.mp4 is hardlinked.
2. Distinct-videoId allocation across the two pools. Every outpainting clip's videoId also has an inpainting clip, so a naive 50/50 draw would ship the same footage twice. allocate() fills the smaller pool first and excludes its videoIds from the other pool's candidates.

Verified in the shipped set: exactly 1,000 references hardlinked from VPData/inpainting/ and 1,000 from VPData/outpainting/ (checked by inode), no overlap. Default --inpaint-ratio 0.5, seed 0.

Result: 141 GB — by far the largest set here, because these are full-resolution Pexels clips, not 81-frame trims. The _painting.mp4 half costs no extra disk (143 GB of hardlinks into VPData/); deleting either copy frees nothing.

---

Camera-Dataset / Camera-Dataset-Val

Source: CameraClone-Dataset/ (KwaiVGI CamCloneMaster). Different in kind from the other three: there is no reference extractor. Inside one data/<date>/traj_<group>_<cam>/ directory every video was rendered with the same camera trajectory at a different 3D location — so a pair of videos from one directory already is a (target, same-motion reference) pair, by construction. 1,154,819 CSV rows were checked: the reference is in the same traj_* dir 100 % of the time and never the same scene file.

The build script is CameraClone-Dataset/processing/build_camera_dataset.py, and it has its own detailed README at ../CameraClone-Dataset/processing/README.md — read that one for the motion estimator, the caption-clustering diversity scheme, and the artifact files. Short version:

cd /mnt/dataset/xinyuy/datasets/CameraClone-Dataset/processing
python3 build_camera_dataset.py --stage all                 # train, ~2 min
python3 build_camera_dataset.py --split val --stage all     # val, ~1 min (needs train first)
python3 build_camera_dataset.py --split val --stage verify

Stages index → select → measure → trim → encode → emit → verify, each individually runnable and resumable.

Points worth remembering:

• Video is remuxed, not re-encoded (NATIVE = True): ffmpeg -c:v copy -an, verified bit-identical on 24 sampled clips. HEVC, 15 fps, 77 frames, 1344×768 (batches 0316,
  317. or 1008×576 (the rest). 77 is not 8n+1, so the trainer buckets/truncates — the precompute below used 73 frames. NATIVE = False switches to a normalised 1280×720 / h264 / 73-frame path instead (~10 min, ~9.2 GB).
• Diversity comes from captions, not scene ids — TF-IDF + MiniBatchKMeans(k=60) over 40,767 candidate captions, then round-robin across clusters. Hard constraints: one pair per trajectory group, each target location once, each reference location once.
• Motion was measured, not assumed. The camera index (c07) is not a motion type — trajectories are generated per group. ORB + estimateAffinePartial2D(RANSAC) on every 4th frame at 320 px. Farneback optical flow was tried first and is too contaminated by character motion — do not go back to it.
• Train and val are disjoint in trajectory group, target location, reference location, and file; verify asserts all three both ways.
• The caption describes the target, so the prompt supplies the scene and the reference supplies only the motion.

Result: 2.8 GB train / 65 MB val. Camera-Dataset-Val/ currently has no .precomputed/ — it was built, then deleted on 2026-08-19 19:47 in favour of precomputing the train split.

---

Stage 5 — precompute LTX-2 latents

Run from ~/flexvideo/LTX-2 (its own uv env, source .venv/bin/activate):

export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
uv run python packages/ltx-trainer/scripts/process_dataset_multi_gpu.py --num-gpus 8 \
    /mnt/dataset/xinyuy/datasets/LTX-2/<Dataset>/dataset.json \
    --resolution-buckets "960x544x81" \
    --model-path /home/xinyuy/flexvideo/LTX-2/ckpts/LTX-2.3/ltx-2.3-22b-dev.safetensors \
    --text-encoder-path /mnt/data/xinyuy/models/Gemma3 \
    --reference-column "reference_path"

Writes into <Dataset>/.precomputed/. Resolution bucket per set:

The single-GPU process_dataset.py takes the same arguments minus --num-gpus. --reference-downscale-factor N encodes the reference at 1/N resolution and must match validation.reference_downscale_factor in the training config; all four sets here used the default 1.

To re-run cleanly, delete .precomputed/ first — the multi-GPU script does not invalidate stale shards. Shard logs land in .precomputed/multi_gpu_logs/shard_N.log.

Consumed by

~/flexvideo/LTX-2/packages/ltx-trainer/configs*/ltx2_v2v_ic_lora_*.yaml, e.g.

data:
  preprocessed_data_root: "/mnt/dataset/xinyuy/datasets/LTX-2/Camera-Dataset"
training_strategy:
  name: "video_to_video"
  reference_latents_dir: "reference_latents"

configs/ = single-condition runs, configs_scale_up/ = the 2,000-pair versions, configs_size_abalation/ = the pose 100/1000/2000 × rank 32/64 sweep.

---

The *-wan-latents folders

Depth-Dataset-wan-latents/ and Pose-Dataset-wan-latents/ are not new datasets — they are the same Depth-Dataset / Pose-Dataset manifests re-encoded with the Wan 2.1 T2V-1.3B VAE + umT5, for the Wan port of IC-LoRA at ~/flexvideo/wan_ic_lora/wan-ic-lora (see its IC_LORA.md).

torchrun --nproc_per_node=8 scripts/precompute_ic_lora.py \
  --dataset /mnt/dataset/xinyuy/datasets/LTX-2/Depth-Dataset/dataset.json \
  --ckpt_dir /mnt/dataset/xinyuy/models/wan/Wan2.1-T2V-1.3B \
  --out /mnt/dataset/xinyuy/datasets/LTX-2/Depth-Dataset-wan-latents \
  --size 832*480 --frame_num 81

Output is a separate directory (the source dataset is read-only to it), with latents/, reference_latents/, conditions/ and a meta.json recording exactly what everything was encoded with. Re-running skips finished samples.

Recorded settings for both (from meta.json): task v2v-1.3B, size 832×480, frame_num 81, reference_downscale 1, vae_stride [4,8,8], latent_shape [16, 21, 60, 104]. 2,000 files in each of the three subdirs, 17–18 GB each.

---

Timeline

---

Rebuilding and verifying

Selections are seeded and deterministic. To confirm a set still matches its recipe without touching disk:

# Pose-Dataset — reproduces the shipped 2,000 stems exactly
import json, random
from pathlib import Path
root = Path('/mnt/dataset/xinyuy/datasets/HumanVID/selected_videos_dedup')
caps = json.loads((root / 'captions.json').read_text())
elig = [p.stem for p in sorted(root.glob('*.mp4'))
        if p.stem in caps and (root / 'pose' / f'{p.stem}_pose.mp4').exists()]
assert len(elig) == 2402
sel = sorted(random.Random(0).sample(elig, 2000))

Same shape for Depth-Dataset (eligible pool 3,000, sample 2,000, seed 0). Both were re-run on 2026-08-21 and matched the shipped stems exactly.

For Camera-Dataset, use the script's own stage: python3 build_camera_dataset.py [--split val] --stage verify.

---

Image-reference sets — Style-Image and Subject2Video

The four sets above condition on a reference video. Two earlier sets condition on a reference image instead, and they live in the previous-generation directory /mnt/data/xinyuy/datasets/LTX/, where both entries are symlinks out to the folder that actually built them:

Dereferenced copies of all three are frozen in old/ (see below). Both sets keep the same dataset.json contract — {caption, media_path, reference_path} — the only difference is that reference_path names a .png / .jpg, not an .mp4.

Style-Image-Dataset

Source: OpenVE-3M — /mnt/data/xinyuy/datasets/OpenVE-3M/csv_files/global_style.csv over OpenVE-3M/videos/. Scripts in ~/dataset_processing/OpenVE-3M/.

The task is (style reference image + prompt) → stylized video, and the one design decision that matters is where the reference frame comes from:

The stylized clip is the generation target (media_path). Its conditioning is a single frame sampled at random from a different stylized clip of the same style (reference_path) — drawing the frame from a foreign clip is what keeps the pair honest, since a model cannot then solve the task by copying content out of its own reference.

Verified in manifest.csv: 1,900 / 1,900 rows have reference_source_video != renamed_video. Row 0, for instance, pairs target abstract_000.mp4 with abstract_000_ref.png = frame 32 of abstract_063.mp4.

1. Build

cd ~/dataset_processing/OpenVE-3M
python build_style_transfer_reference.py

Defaults are what shipped: --per-style 100, --sample-seed 42 (clip selection), --seed 20260802 (donor clip + frame choice), --ext png. Style labels are parsed out of the CSV prompt text and filtered to a canonical 19 taxonomy (Abstract Art, Ancient Style, Cartoon, Chinese Ink Wash, Cubist, Cyberpunk, Ghibli, Gongbi, Impressionist, Oil Painting, Pixel Art, Pointillist, Pop Art, Sketch, Steampunk, Surrealist, Thick-Paint Hand-Drawn, Ukiyoe, Watercolor). Confirmed in the manifest: 19 styles × exactly 100 = 1,900, and no original_video is shared across styles.

The _org (unstylized) source clips are deliberately not copied — they have no role once the conditioning is an image — but every manifest row keeps original_org so they can be recovered from the CSV.

--horizontal-only restricts sampling to landscape clips and writes to style-transfer-reference-horizontal/ instead. Because references are cut from another clip of the same style, filtering the clips is enough to make every reference landscape too.

2. Caption

python caption_transfer_reference.py style-transfer-reference \
    --model gemini-2.5-flash --concurrency 8

This one is not a plain captioner. In manifest mode it captions the original, unstylized clip behind each target (the manifest's original_org column) and then rewrites dataset.json so caption describes the video content, moving the original style-transfer instruction into a separate style_prompt field. So entries here have four keys, not three:

{
  "caption": "A young boy in a red shirt rows a red and yellow boat on a tranquil lake…",
  "reference_path": "abstract_000_ref.png",
  "media_path": "abstract_000.mp4",
  "style_prompt": "Apply the Abstract Art style to this video, ensuring seamless temporal…"
}

Useful flags: --limit 5 for a smoke test, --update-only to rewrite dataset.json from an existing captions file without calling Gemini, --no-update-dataset to caption only. Raw captions are kept in source_captions.json, and the pre-rewrite manifest in dataset.json.bak.

3. Precompute

uv run python packages/ltx-trainer/scripts/process_dataset_multi_gpu.py --num-gpus 8 \
    /home/xinyuy/dataset_processing/OpenVE-3M/style-transfer-reference/dataset.json \
    --resolution-buckets "960x544x33" \
    --model-path /home/xinyuy/flexvideo/LTX-2/ckpts/LTX-2.3/ltx-2.3-22b-dev.safetensors \
    --text-encoder-path /mnt/data/xinyuy/models/Gemma3 \
    --reference-column "reference_path"

Source clips are 1280×720, 30 fps, 57 frames. The first run used 960x544x37; it was re-run three minutes later at 960x544x33, which is the value that stuck — 33 satisfies frames % 8 == 1, 37 does not.

Consumed by configs/ltx2_v2v_ic_lora_style_image.yaml (preprocessed_data_root: /mnt/data/xinyuy/datasets/LTX/Style-Image-Dataset, W&B project ltx-2-trainer-style).

Sibling variants

Same builder, same 1,900-row shape, precomputed the same way at 960x544x33:

Not to be confused with Style-Transfer-OpenVE-3M-Dataset/ (5,700 rows), which is the older video→video style-transfer set — reference is a clip, not an image.

Subject2Video-Dataset

Source: BestWishYsh/ConsisID-preview-Data, downloaded to /mnt/data/xinyuy/datasets/ConsisID-preview-Data/. That folder has its own README.md documenting the HF bucket export; the build itself is below. Scripts are kept in both the folder root and processing_codes/.

Upstream ships videos/videos/ (51,242 clips), bboxes/refine_bbox_jsons/ (31,947 per-clip bbox JSONs), masks/track_masks_data/ (31,947 tracking-mask dirs) and video_caption_train.json (28 MB of captions).

1. Crop reference images from bbox annotations

cd /mnt/data/xinyuy/datasets/ConsisID-preview-Data
python generate_reference_images.py --type all

Samples --num 3000 clips at --seed 42 from the video/bbox intersection, picks one random frame in which every requested type has a detection, takes the highest-confidence box per type, and writes <output>/<type>/<stem>.jpg. Result: reference_images/head/ and reference_images/person/, 2,997 each (3 of the 3,000 had no frame valid for all three types).

A second run adds background masking:

python generate_reference_images.py --type all \
    --masks masks/track_masks_data --output ref_imgs_masked

With --masks, valid frames are additionally intersected with the tracking data's valid_frame.json, and every pixel outside the tracked mask is set to white. That constraint is much tighter — ref_imgs_masked/{face,head,person} hold 1,091 each.

2. Materialize Subject2Video-Dataset

The 2,997 selected clips were moved out of videos/videos/ into Subject2Video-Dataset/<stem>.mp4 (original 2024 mtimes preserved; the stems are no longer present in videos/videos/, and all 2,997 still have their bbox JSON), each paired with <stem>_reference.jpg and indexed by a dataset.json whose captions come from video_caption_train.json.

The shipped reference is the face crop, not head or person. Inferred from a build step not in shell history, but pinned down: the reference for --37UfNPPlw.5_segment_0_67-… is 89×134, which matches no head (181×132) or person (347×238) crop, and matches a face box in that clip's bbox JSON exactly. reference_images/face/ is empty today, consistent with those crops having been moved into the dataset.

3. Materialize the masked-reference variant

python create_masked_ref_dataset.py

Hard-coded, no CLI. Builds a deliberate 50/50 mix of two reference kinds at SEED = 42:

• 1,091 masked-person references from ref_imgs_masked/person, videos symlinked from videos/videos/ — the whole pool, since it is the smaller one;
• 1,091 head references from reference_images/head, videos symlinked from Subject2Video-Dataset/, sampled down from 2,997 to match.

2,182 pairs total. Verified on disk: all 2,182 videos are symlinks (into videos/videos/ or Subject2Video-Dataset/), the two reference files spot-checked are byte-identical to their ref_imgs_masked/person and reference_images/head sources, and the two stem pools are disjoint — no clip appears with both a masked-person and a head reference.

Two things the script does not do: it writes to Subject2Video-Dataset-MaskedRef (the directory was renamed to Subject2Video-MaskedRef-Dataset afterwards) and it writes no dataset.json — captions were joined on in a later step, and the shipped manifest has 2,182 entries.

4. Precompute

uv run python packages/ltx-trainer/scripts/process_dataset.py \
    /mnt/data/xinyuy/datasets/LTX/Subject2Video-Dataset/dataset.json \
    --resolution-buckets "960x544x33" \
    --model-path /home/xinyuy/flexvideo/LTX-2/ckpts/LTX-2.3/ltx-2.3-22b-dev.safetensors \
    --text-encoder-path /mnt/data/xinyuy/models/Gemma3 \
    --reference-column "reference_path"

Clips are 1280×720, 24 fps, ~60 frames; bucket 960x544x33. Same command for Subject2Video-MaskedRef-Dataset. Adding --reference-only re-encodes just reference_latents/ — used when the reference images changed but the target clips did not, which is exactly what happens when swapping crop types.

Both sets are fully precomputed: 2,997 × 3 and 2,182 × 3 tensors.

Consumed by configs/ltx2_v2v_ic_lora_subject.yaml (preprocessed_data_root: /mnt/data/xinyuy/datasets/LTX/Subject2Video-MaskedRef-Dataset, W&B project ltx-2-trainer-subject), plus the ltx2_v2v_ic_lora_subject_{tiling,no_tiling}_{offset,no_offset}.yaml ablation grid and configs_scale_up/ltx2_v2v_ic_lora_subject.yaml.

Backup

hf://buckets/fishxinyu/ConsisID-preview-Data — 35,167 files, ~70.4 GiB: both datasets with their .precomputed/, the two crop folders, and the three scripts. Raw intermediates (masks/, videos/, bboxes/, video_caption_train.json — ~137 GB) are local only. The style sets sync to hf://buckets/fishxinyu/OpenVE-3M and …/OpenVE-3M-process.

---

The other folders

old/ — 74 GB

A rsync -aL snapshot of /mnt/data/xinyuy/datasets/LTX, taken 2026-08-17. Eight of its sixteen entries were symlinks into Senorita / ConsisID / OpenVE-3M; they were dereferenced into real files, so this copy is self-contained. Provenance table, per-directory file counts, and the original_symlinks.tsv mapping are in old/README.md. These are the previous generation of training sets (Canny, Depth-Prediction, Object-{Addition, Removal,Swap}, Style-Transfer, Subject2Video, …) — kept for reference, not fed by anything here.

Three of its entries are the image-reference sets documented above: old/Style-Image-Dataset (9,512 files), old/Subject2Video-Dataset (14,986) and old/Subject2Video-MaskedRef-Dataset (10,911). All three were symlinks in the source and are real files here, .precomputed/ included — so this is the one self-contained copy of them, and the only one that survives if the live build folders are cleaned. The training configs still point at the live /mnt/data/…/LTX/ symlinks, not at these.

cameraman/ — 1.8 GB

Reference material gathered while designing Camera-Dataset, not training data: LTX2.3-22B_IC-LoRA-Cameraman_v1/v2 (the released camera IC-LoRAs — v2's hand-curated motion distribution is the 27 % → 7 % baseline the README compares against), LTX-2.3-3DREAL-LoRA, and ComfyUI-CameraReference3D.

Backup

This whole directory syncs to hf://buckets/fishxinyu/LTX-2-training — note the -training suffix; the bucket plainly named LTX-2 is /mnt/data/xinyuy/outputs/LTX-2, a different thing. Last full sync succeeded 2026-08-17, 181,697 files / 758,606,143,719 B, exact match.

The sync is driven by a retry supervisor at ../.hf-sync/ltx2-sync.sh — hf buckets sync dies on 429s and connection resets, and the supervisor restarts it with exponential backoff (30 s → 300 s cap), treating any attempt over 10 min as progress and bailing out after 10 consecutive sub-30 s failures as a sign of a real auth/quota/path error. State in ../.hf-sync/status.json, per-attempt logs in ../.hf-sync/attempts/.

---

Gotchas worth keeping

• --dedup matters for Pexels-sourced data. Same first four filename chars = same shoot = same human. This is the only reason Pose-Dataset-Legacy exists.
• build_pose_dataset.py validates --pose-no-face-dir even when it isn't the reference. With a dedup folder that has no pose_no_face/, point it at pose/.
• The two select_and_trim_videos.py are not the same script. HumanVID's uses ffprobe + stream copy; VIDGEN's uses a single PyAV decode pass, on purpose.
• Inpainting targets are re-encoded, not linked — the control clip covers a sub-range, so the target must be trimmed to match or the pair desyncs.
• Inpainting-Dataset/*_painting.mp4 are hardlinks into VPData/. 143 GB that exists once. Deleting one side frees nothing.
• Camera clips are 77 frames, not 8n+1. Precompute used bucket …x73.
• Delete .precomputed/ before re-precomputing — stale shards are not invalidated.
• The style reference frame must come from a different clip. Same style, foreign clip. Cut it from the target's own frames and the task collapses into copying.
• Subject2Video-Dataset's clips were moved, not copied, out of videos/videos/. Their stems are gone from the raw pool, so anything that re-scans it (including generate_reference_images.py) can no longer see them — which is why the masked-crop pool is disjoint from the main set.
• create_masked_ref_dataset.py writes symlinks and the wrong directory name (Subject2Video-Dataset-MaskedRef), and writes no dataset.json. Rename after, and join the captions separately.
• LTX-2 needs frames % 8 == 1. The first style precompute used bucket …x37 and was re-run at …x33 minutes later.
• PyAV is used for encoding everywhere instead of cv2.VideoWriter, whose bundled FFmpeg can only open the mp4v fourcc that most players refuse.