# Data

Music3Lab ships **dataset metadata only**. No audio of any kind is included or
redistributed.

## LAION-DISCO-12M (external real-music corpus)

[LAION-DISCO-12M](https://huggingface.co/datasets/laion/LAION-DISCO-12M) is an
**index** dataset: it contains YouTube video IDs and tags, licensed **Apache-2.0**
for the metadata, and explicitly contains **no audio samples**. Audio referenced
by those IDs is *not* part of the dataset and is *not* redistributed here.

### What is shipped in this repo (`data/laion_disco/`)

- `interim_tranche_678/manifest.jsonl` — the 678 accepted tracks as records of:
  YouTube `source_id`/`source_url`, LAION `source_dataset_revision`, canonical
  audio content hashes (`canonical_sha256`, `pcm_sha256`), sample rate / channels
  / frame counts, and split assignment. **No audio.**
- `interim_tranche_678/splits.json` — deterministic, source-level train/val/heldout
  split (algorithm, seed, assignment, counts).
- `interim_tranche_678/summary.json`, `candidates.summary.json`,
  `tranche.stopped.json`, `environment.json` — provenance and stop-reason records.
- `UPSTREAM_README.md` — the corpus builder's own notes.

### What is NOT shipped

- ❌ Any downloaded audio (`files/*.wav`, `canonical/`).
- ❌ The full 20k candidate list is included only as a summary; the raw
  `candidates.jsonl` and the private author-song exclusion hashes
  (`known_user22_*`) are **not** published.

### Reproducing the corpus

`scripts/data/laion_ingest.py` deterministically selects candidates from the
pinned LAION revision (`6e7bf3758a77301e46a715af894fefd79bb1da53`), then uses
`yt-dlp` + `ffmpeg` to fetch and canonicalize audio **on your own machine**.
`scripts/data/laion_freeze_interim.py` freezes an immutable, hash-verified
tranche and split. Both scripts contain instance-specific absolute paths from the
original run and need their `ROOT`/paths adjusted before use.

> ⚠️ **You are responsible** for complying with YouTube's Terms of Service and
> applicable copyright when downloading any audio. Music3Lab neither hosts nor
> distributes that audio. The original run stopped at 678/2,000 tracks when
> YouTube returned bot-verification challenges — expect the same and throttle
> accordingly.

## The author's own songs (private)

The project also evaluated on the author's own tracks (8, then 22 after
deduplication). These are **private by default** and are **not** in this repo,
not in the LAION splits, and were **never** used for training or checkpoint
selection — only as out-of-distribution evaluation. Only their content **hashes**
were used, to guarantee they never leaked into training. If you want a public
demo, publish only short excerpts of audio you own and are licensed to share.

## MiniMax-Music3 teacher data

Some experiments use WAV/latent/token pairs *captured from* MiniMax-Music3
generations. These captures are derivatives of the base model and are **not**
shipped. `scripts/` can regenerate them from the model you download yourself.