LLMasterLL/LVDU/LLaVA-Video-83K - Storage Bucket

LLaVA-Video-83K

An 83,000-sample subset of lmms-lab/LLaVA-Video-178K, selected by stratified density-flattened sampling rather than uniformly at random, and shipped with its media so it is usable straight out of the bucket.

Records	83,000 (5.0% of the 1,650,547 upstream records)
QA pairs	279,679
Distinct media	71,224 (59,428 mp4 + 11,796 llava_hound frame scenes)
Media size	428.9 GiB
Tasks	Caption 10.97% · Open-ended VQA 75.84% · Multiple-choice VQA 13.19%

Media for the llava_hound split comes from ShareGPTVideo/train_video_and_instruction (train_300k), which covers all 179,318 referenced scenes.

Layout

LLaVA-Video-83K/
├── annotations/
│   ├── all_sampled.json          # 83,000 records, upstream schema unchanged
│   └── by_source/*.json          # 39 files, split as upstream
├── metadata/
│   ├── manifest.parquet          # record -> media, task, duration bucket, source
│   ├── shard_index.parquet       # member -> (shard, offset, nbytes)   ← random access
│   ├── scene_index.parquet       # llava_hound scene -> frame count
│   └── SAMPLING_REPORT.md        # how the subset was drawn, and its verification
├── embeddings/                   # MiniLM-L6 vector per record, for re-drawing subsets
├── shards/
│   ├── video/video-NNNNN.tar     # uncompressed, ~2 GiB each
│   └── frame/frame-NNNNN.tar     # uncompressed, ~2 GiB each
└── videos_large/                 # the 188 mp4s above 64 MiB, stored loose

A record's video field is unchanged from upstream and is exactly the member name inside the shards, so nothing in the annotations needs rewriting.

Why tar shards

Media is stored as uncompressed tar plus a byte-offset index, not as loose files. A member therefore occupies a contiguous byte range and is one pread away — decord decodes it from BytesIO byte-identically to reading the file from disk, seeks included (verified on files up to 707 MiB).

The reason is I/O, not a Hub limit: 377k loose objects make transfer, listing and Lustre metadata operations painfully slow, while ~220 shards of 2 GiB read at full bandwidth. Videos above 64 MiB are the exception — they stay loose so a dataloader never has to buffer a 700 MiB file in RAM.

Usage

from lvdu_reader import ShardReader

r = ShardReader("/path/to/LLaVA-Video-83K")

vr = r.video_reader("academic_source/youcook2/203/Qh_rtJaCpXs/split_6.mp4")   # decord VideoReader
frames = r.scene_frames("shareVideoGPTV/frames/all_frames/v_XXX-Scene-001")

ShardReader is safe under DataLoader workers: shard handles are opened lazily per process and never shared across a fork.

How the subset was drawn

Uniform random sampling inherits the upstream corpus's pathologies. This subset instead:

• keeps the task / duration / source mix of the parent pool (task shares match to 0.01%);
• gives small sources (activitynetqa, nextqa, perceptiontest — 41k records between them) a floor so they survive the cut;
• drops records whose answers are entirely boilerplate, cutting ultra-short open-ended answers from 5.65% (random) to 2.39%;
• flattens density inside each stratum: cluster the MiniLM vectors, then allocate per cluster ∝ size^0.5, so dense topic clusters stop dominating;
• constrains the multiple-choice length shortcut: a blind "pick the longest option" guesser scores 51.23% on the full set, and 49.79% here.

Measured effect of the flattening, against an equally-sized random draw: cluster hit rate is unchanged (the metric saturates at 5% sampling), but concentration drops — max samples per cluster 287 → 161 globally and 52 → 45 inside the largest stratum; Gini 0.712 → 0.697 and 0.394 → 0.341.

Full numbers, including the verification of every claim above, are in metadata/SAMPLING_REPORT.md.

Provenance and licence

This is a derived subset. The videos and annotations remain under the terms of the upstream releases — see lmms-lab/LLaVA-Video-178K and ShareGPTVideo/train_video_and_instruction. Cite those datasets, not this repackaging, for the underlying data.