license: apache-2.0
base_model: MiniMaxAI/MiniMax-Music3
tags:
- music
- audio
- music-generation
- minimax-music3
- research
- toolkit
Music3Lab
Open research toolkit extending MiniMax-Music3 with arbitrary-audio encoding, continuation, inpainting, prepend generation, prompt-free generation, and automated objective evaluation.
Music3Lab is a reproducible, evidence-first toolkit built around the released MiniMax-Music3 weights. It does not modify or redistribute those weights. Every capability below was gated on preregistered objective metrics, and the failures are published alongside the successes — they are the more useful part of the research.
Honesty note. This is a research toolkit, not a finished product. Several headline goals (native WAV→token encoding, true arbitrary-audio history-aware continuation/inpainting/prepend, direct long-form reference conditioning) were attempted and did not pass their gates. Those negative results, their code, configs, and exact metrics are all here on purpose.
What it does
| Capability | Status | Notes / measured result |
|---|---|---|
| Checkpoint audit of the released weights | ✅ available | Every released tensor classified; proves there is no native RVQ waveform tokenizer in dav.pth. See FINDINGS.md. |
| Continuous WAV → Flow-latent encoder | ✅ available | One-pass ≈1.34 ms (base pilot). External real-music fine-tune improved held-out audio ruler 74.9% and SI-SDR 2.03→8.24 dB, but is a rejected specialist (protected teacher latent regressed +13.2%), not a promoted champion. |
| Latent inversion (research/oracle mode) | ✅ available | Iterative; a 1 s external clip reached 22.2 dB SI-SDR / 0.997 correlation. Slow (~208 s per 1 s) — a teacher, not a real-time encoder. |
| Masked-Flow inpainting (captured Music3 conditions) | ✅ available | +30.8% latent NMSE, +20.1% hole audio-ruler vs zero-adapter. Captured-condition only. |
| Captured-state style continuation | ✅ available | 12 s → 16 s, four candidates, objective style/seam ranking. Captured-state only, deterministic-from-frame-0. |
| Full-state resume | ✅ available | Serializes KV cache + CUDA RNG; reproduces frames + Flow chunks exactly across processes (deterministic backend). |
| Reference-guided append (CPU) | ✅ available | Appends a chosen reference-style candidate with bit-exact source preservation outside the crossfade. |
| Prompt-free generation | ✅ available | No user text; internal MIR/planning→text bridge. Best-of-N up to 90 s (max policy: 3/8 eligible full-length). |
| Reference-style generation | 🟡 partial | 8 s, ranked by direct continuous-latent + MIR similarity. Not direct model conditioning or full style transfer. |
| Objective evaluation suite | 🟡 partial | Integrity, reconstruction, SI-SDR/SNR, correlation, loudness/stereo, anti-copy. No learned musicality/aesthetic judges. |
| Native WAV → Music3 RVQ tokens | ❌ blocked | Released dav.pth has no quantizer/codebooks. (62000_generator is a PyTorch ZIP folder name, not a component.) |
| Arbitrary-WAV continuation | ❌ failed | Learned conditioner lost to repeat/roll baselines. |
| Two-sided acoustic FIM (arbitrary WAV) | ❌ failed | +5.9% ruler vs required +10%; boundaries worse than interpolation. |
| Arbitrary-WAV / waveform prepend | ❌ failed | Failed seam / anti-copy gates. |
| Native-state Stage-1 residual prediction | ❌ failed | Near-modal; mean CE 6.834, exact full token rows 0. |
| Direct long-form reference conditioning | ❌ failed | Tempo drift / early-EOS; no eligible 60 s candidate. |
| Enforceable negative prompts (e.g. "no vocals") | ❌ not enforceable | Reported honestly as NOT_ENFORCEABLE. |
music3lab status --json is the authoritative machine-readable capability
matrix. Full write-ups are in reports/ and
reports/FINAL_RESULTS.md.
What it is NOT
- ❌ It does not include MiniMax-Music3 weights. You download those yourself.
- ❌ It does not redistribute any audio — not LAION/YouTube audio, not the author's own songs. Only dataset metadata (IDs, hashes, splits) is included.
- ❌ It is not a native audio tokenizer for Music3. That does not exist in the public release (see FINDINGS.md).
Quickstart
# 1. Environment (Python 3.10; exact pins in requirements.lock)
python -m venv .venv && . .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -e . # core (CPU inspection/eval)
pip install -e ".[capture]" # + diffusers/transformers for generation (GPU)
# 2. Get the base model yourself (NOT bundled). See REPRODUCING.md.
hf download MiniMaxAI/MiniMax-Music3 --local-dir ./models/minimax-music3
# 3. Inspect the released checkpoint (CPU, no GPU, no weights modified)
python inspect_dav.py ./models/minimax-music3/dav.pth --json --sha256
# 4. Machine-readable capability matrix
music3lab status --json
Full setup, model download, one-command demo, and benchmark commands: REPRODUCING.md.
Repository layout
music3lab/
├── src/music3lab/ # the installable package (tested; layout preserved)
│ ├── codec/ # encoders, external fine-tune, native-state experiments
│ ├── editing/ # continuation / inpaint / prepend / append
│ ├── autonomous/ # champion/challenger promotion + rollback
│ ├── inversion*.py # latent inversion (research mode)
│ ├── eval.py # objective evaluation
│ └── ... # baseline capture, checkpoint audit, pipeline, release
├── configs/ # frozen experiment/training configs (34)
├── scripts/ # runnable training / experiment / benchmark scripts (38)
│ └── data/ # LAION downloader (laion_ingest.py, laion_freeze_interim.py)
├── tests/ # focused + adversarial suites (58)
├── reports/ # per-capability write-ups + FINAL_RESULTS.md
├── data/laion_disco/ # dataset METADATA only (IDs, hashes, splits) — no audio
├── checkpoints/ # POINTERS to trained adapters (no weights) — see README there
├── examples/ # how to reproduce demo outputs (no bundled audio)
├── requirements.lock # exact pinned environment
├── LICENSE NOTICE THIRD_PARTY.md MODEL_CARD.md DATA.md TRAINING.md REPRODUCING.md
Training. Every trainable component ships its training script + config + tests. See TRAINING.md for the full table and how to push the open problems (native tokenization, arbitrary-audio editing).
Note on structure. The conceptual grouping (encoder / continuation / inpaint
/ prepend / eval) is preserved thematically via the codec/ and editing/
subpackages and this map, rather than by physically splitting src/ — that keeps
the 48-test suite green and the package importable for a reproducible v0.1.0. A
physical refactor into top-level encoder/continuation/... packages is a
possible later, separately-tested change.
The core finding
The released dav.pth is a continuous DAV analysis encoder + Gaussian
posterior heads + flow model + waveform decoder. It contains no RVQ/VQ
quantizer, no acoustic codebooks, and no generator/62000_generator
tensors. The string 62000_generator is only the root folder name inside the
PyTorch ZIP archive — not a model component. Music3's eight-stream token space
therefore cannot be produced from an arbitrary WAV with the released weights.
Everything Music3Lab does works either in the continuous Flow-latent space or
from captured generation state. Details and reproduction: FINDINGS.md.
Trained checkpoints & data
- Adapters/encoders are released separately (Hugging Face) — see checkpoints/README.md and MODEL_CARD.md. ⚠️ They are derivatives of MiniMax-Music3 and may be governed by its license; confirm before redistributing.
- Dataset: only LAION-DISCO-12M metadata + a downloader are shipped. No audio. See DATA.md.
License
Original Music3Lab code: Apache-2.0 (LICENSE, NOTICE). Third-party components and their separate licenses: THIRD_PARTY.md.