| # Data |
|
|
| Music3Lab ships **dataset metadata only**. No audio of any kind is included or |
| redistributed. |
|
|
| ## LAION-DISCO-12M (external real-music corpus) |
|
|
| [LAION-DISCO-12M](https://huggingface.co/datasets/laion/LAION-DISCO-12M) is an |
| **index** dataset: it contains YouTube video IDs and tags, licensed **Apache-2.0** |
| for the metadata, and explicitly contains **no audio samples**. Audio referenced |
| by those IDs is *not* part of the dataset and is *not* redistributed here. |
|
|
| ### What is shipped in this repo (`data/laion_disco/`) |
| |
| - `interim_tranche_678/manifest.jsonl` — the 678 accepted tracks as records of: |
| YouTube `source_id`/`source_url`, LAION `source_dataset_revision`, canonical |
| audio content hashes (`canonical_sha256`, `pcm_sha256`), sample rate / channels |
| / frame counts, and split assignment. **No audio.** |
| - `interim_tranche_678/splits.json` — deterministic, source-level train/val/heldout |
| split (algorithm, seed, assignment, counts). |
| - `interim_tranche_678/summary.json`, `candidates.summary.json`, |
| `tranche.stopped.json`, `environment.json` — provenance and stop-reason records. |
| - `UPSTREAM_README.md` — the corpus builder's own notes. |
|
|
| ### What is NOT shipped |
|
|
| - ❌ Any downloaded audio (`files/*.wav`, `canonical/`). |
| - ❌ The full 20k candidate list is included only as a summary; the raw |
| `candidates.jsonl` and the private author-song exclusion hashes |
| (`known_user22_*`) are **not** published. |
|
|
| ### Reproducing the corpus |
|
|
| `scripts/data/laion_ingest.py` deterministically selects candidates from the |
| pinned LAION revision (`6e7bf3758a77301e46a715af894fefd79bb1da53`), then uses |
| `yt-dlp` + `ffmpeg` to fetch and canonicalize audio **on your own machine**. |
| `scripts/data/laion_freeze_interim.py` freezes an immutable, hash-verified |
| tranche and split. Both scripts contain instance-specific absolute paths from the |
| original run and need their `ROOT`/paths adjusted before use. |
|
|
| > ⚠️ **You are responsible** for complying with YouTube's Terms of Service and |
| > applicable copyright when downloading any audio. Music3Lab neither hosts nor |
| > distributes that audio. The original run stopped at 678/2,000 tracks when |
| > YouTube returned bot-verification challenges — expect the same and throttle |
| > accordingly. |
|
|
| ## The author's own songs (private) |
|
|
| The project also evaluated on the author's own tracks (8, then 22 after |
| deduplication). These are **private by default** and are **not** in this repo, |
| not in the LAION splits, and were **never** used for training or checkpoint |
| selection — only as out-of-distribution evaluation. Only their content **hashes** |
| were used, to guarantee they never leaked into training. If you want a public |
| demo, publish only short excerpts of audio you own and are licensed to share. |
|
|
| ## MiniMax-Music3 teacher data |
|
|
| Some experiments use WAV/latent/token pairs *captured from* MiniMax-Music3 |
| generations. These captures are derivatives of the base model and are **not** |
| shipped. `scripts/` can regenerate them from the model you download yourself. |
|
|
