| --- |
| license: cc-by-nc-sa-4.0 |
| tags: |
| - music |
| - symbolic-music |
| - midi |
| - music-generation |
| - piano |
| - romantic |
| library_name: pytorch |
| --- |
| |
| # Maurice 🎹 |
|
|
| > ### ⚠️ This is a **symbolic (MIDI)** model — not an audio model |
| > |
| > Maurice generates **note events** — pitch, onset, duration, velocity — and writes a **`.mid` file**. |
| > It does **not** generate waveforms, and it does not take audio as input. There is no vocoder, no |
| > spectrogram, no `.wav` anywhere in the pipeline. |
| > |
| > To *hear* the output you render the MIDI yourself, with a SoundFont (FluidSynth), a sampled piano |
| > library (SFZ/Kontakt), or any synth or DAW. The quality of what you hear therefore depends heavily |
| > on the piano you render it through — the same file can sound thin through a GM font and gorgeous |
| > through a good sampled grand. |
| > |
| > If you want text-to-audio music generation, this is not that kind of model — look at MusicGen, |
| > Stable Audio or similar. Maurice is closer in kind to a composer writing a score than to a |
| > recording of one. |
|
|
|
|
| **Maurice** is a from-scratch, ~300M-parameter **symbolic-music** decoder that **cold-starts a solo-piano piece from silence** — conditioned on **composer** and **key**. It's the thing a continuation model can't give you: the *beginning*, and everything after it. Point it at "Liszt in E♭ major" or "Debussy in D major" and it improvises real two-hand romantic-piano texture — melody, harmony and bass together — building a coherent ~2–3 minute piece with an arc, not a scale. |
|
|
| - **Architecture:** Qwen3-style decoder — RMSNorm (pre-norm, fp32 upcast), RoPE, SwiGLU, grouped-query attention, QK-Norm, tied embeddings. **16 layers, d=1280, 20 heads / 4 KV heads, 4096 context.** |
| - **Parameters:** ~298M |
| - **Tokenizer:** [Aria](https://github.com/EleutherAI/aria-utils) `AbsTokenizer` (≈3 tokens/note: pitch+velocity, onset, duration), extended with 42 composer and 24 key prefix tokens (see `conditioning.json`). |
| - **Training:** best checkpoint at step 37,000, **val loss 1.085 (perplexity ≈ 2.96)**. Trained on 4×H100 (DDP, 40k steps) with ×12 transposition augmentation and from-piece-boundary windowed sampling. |
| - **What it is:** a **self-starter** — it cold-starts a *whole* piece (up to the 4096-token context, ≈ 2–3 minutes) from silence. Ask for a short `--tokens` opening instead if you want to pair it with a continuation model (e.g. Aria). |
|
|
| ## Hear it |
|
|
| **Elegy in C-sharp minor** — 608 notes composed by the model, then performed. The *same notes* twice: |
|
|
| | | | |
| |---|---| |
| | **[▶ as the model composed it](https://huggingface.co/roygbiv33/Maurice/resolve/main/output_examples/elegy_01_score_as_composed.mp3)** | raw output: no pedal, flat velocities, everything square on the beat | |
| | **[▶ after a performance pass](https://huggingface.co/roygbiv33/Maurice/resolve/main/output_examples/elegy_02_performed.mp3)** | identical pitches; only voicing, rolls, rubato and pedal differ | |
|
|
| Play those back to back. It's the fastest way to hear what this model gives you and what it doesn't, |
| and why we think dry-sounding symbolic output is usually *unperformed* rather than badly composed. |
| Source MIDI is in [`output_examples/`](https://huggingface.co/roygbiv33/Maurice/tree/main/output_examples). |
| (Generated with the [attribute-conditioned fine-tune](https://huggingface.co/roygbiv33/Maurice-Attr), which adds nine musical dials on top of this architecture.) |
|
|
| ## Install |
|
|
| ```bash |
| pip install torch safetensors pretty_midi |
| # tokenizer (AbsTokenizer) — installed from source, no PyPI package: |
| pip install git+https://github.com/EleutherAI/aria-utils.git |
| ``` |
|
|
| ## Generate |
|
|
| ```bash |
| # a full piece (CFG off by default — the most musical setting) |
| python generate.py --composer liszt --key "Eb major" --tokens 2000 --out piece.mid |
| |
| # a short opening to hand to a continuation model |
| python generate.py --composer debussy --key "D major" --tokens 320 --out opening.mid |
| |
| # best-of-8, keep the most musical (rhythm-scored) |
| python generate.py --composer scriabin --best-of 8 --tokens 2000 --out piece.mid |
| ``` |
|
|
| Or from Python: |
|
|
| ```python |
| from modeling_maurice import Maurice |
| from generate import load, cold_start |
| |
| model, tok = load() |
| midi = cold_start(model, tok, composer="scriabin", key="Eb minor", best_of=1, max_tokens=2000) |
| midi.save("piece.mid") |
| ``` |
|
|
| ### Classifier-free guidance (CFG) |
|
|
| CFG is available via `--guidance w`. With a specific composer it decodes conditioned (composer) and unconditioned (`unknown`) streams in lockstep and steers `logits = uncond + w·(cond − uncond)` to amplify the composer's identity. **It defaults to `1.0` (off), which sounds best.** Higher `w` (2–4) does pull harder toward the composer — but on this checkpoint it trades musicality for density, compressing the piece into faster, more frantic figuration. Useful as an experimental knob; not recommended for the most musical output. No effect with `composer="unknown"`. |
|
|
| ## Conditioning |
|
|
| - **`composer`** — one of 42 (see `conditioning.json`). Strongest by ear: **`liszt`, `chopin`, `debussy`, `scriabin`, `ravel`**. Use **`unknown`** for a generic-romantic voice. |
| - **`key`** — friendly strings like `"Db"`, `"C# minor"`, `"Eb major"`; omit for a random key per sample. |
| - Every call is **fresh** (no seeding) — call again for a different piece. Conditioning is real but **soft** at the default (g1): the composer token nudges density/character (Chopin denser & quicker, Debussy more spacious) more than it dictates a signature. It lives in the corpus's shared romantic center of gravity — CFG can exaggerate it, at the cost above. |
|
|
| ## Best-of-N + the musicality scorer |
|
|
| Cold-starts vary in quality, so `generate.py` includes an **advisory** scorer (`rhythm_score`) for best-of-N selection. Its dominant signal is **rhythmic aliveness** (distinct-duration count): rhythmically *dead* openings (droning, blocky) score low however "correct" their harmony; lively, harmonically-moving ones score high. It's a prior to shortlist candidates — **not a judge**; trust your ear. |
|
|
| ## Architecture at a glance |
|
|
| Compared to a LLaMA-style base, Maurice adds Qwen3's **QK-Norm** for training stability and uses GQA + SwiGLU + RoPE with tied embeddings. Inference uses a **KV cache** (`Maurice.infer`) verified to match naive recompute to within 2e-5 logits, so best-of-N and CFG (two-stream) stay cheap. |
|
|
| ## Prior art & lineage |
|
|
| Maurice stands on a lot of other people's work, and it's worth being precise about what came from where. |
|
|
| **Aria** ([loubb/aria-medium-base](https://huggingface.co/loubb/aria-medium-base), [aria-utils](https://github.com/EleutherAI/aria-utils)) is the most direct ancestor. Maurice uses Aria's **`AbsTokenizer`** unchanged (≈3 tokens/note: pitch+velocity, onset, duration) and trains on the **Aria-MIDI** corpus. The two models are complements rather than competitors: Aria is a **continuation** model — give it a prompt and it carries on, beautifully — whereas Maurice **cold-starts from silence**, which is the one thing a continuation model structurally cannot do. If you want a beginning, use Maurice; if you want to extend material you already have, use Aria. They share a tokenizer, so the outputs interoperate directly. |
|
|
| **GiantMIDI-Piano** (Kong, Li, Song, Hantrakul & Wang, ByteDance) supplied a large slice of the classical/romantic repertoire — ~10k works transcribed from audio with high-resolution piano transcription. It is the reason Maurice has heard a broad composer range rather than a narrow canon, and it is where much of the composer conditioning gets its coverage. |
|
|
| **MAESTRO** (Hawthorne et al., Google Magenta) contributed ~200 hours of aligned virtuoso performance from the International Piano-e-Competition. Because MAESTRO is *performed* rather than quantized, it is disproportionately responsible for whatever expressive micro-timing and velocity shaping Maurice has learned. |
|
|
| Also in the broader lineage: **Music Transformer** (Huang et al.) for relative-attention long-form symbolic modeling, the **Anticipatory Music Transformer** (Thickstun et al.), and orchestral/score-level systems such as **SymphonyNet** and **NotaGen**. On the performance side — see the architecture note below — **VirtuosoNet** (Jeong et al.), the Vienna **Basis Mixer** (Cancino-Chacón, Grachten, Widmer) and the older **KTH performance rules** (Friberg, Sundberg, Bresin) are the reference points, with **ASAP** providing score↔performance alignment over MAESTRO. |
|
|
| ## The architecture around the model |
|
|
| Maurice is one stage of a larger system, and the system is arguably the more interesting result. Weights alone will get you notes; they will not get you music, and the gap between those two is where nearly all the work went. |
|
|
| ``` |
| brief |
| │ |
| ▼ PLAN a structural skeleton: germ motif, section arc, per-section |
| │ collection / density / register / dynamic targets, where the climax lands |
| ▼ MAURICE composes EVERY note, plan-driven — states a theme, develops it |
| │ (fragment / invert / transpose / augment), brings it back transformed |
| ▼ GATE verifies ADHERENCE to the plan — arithmetic only, never taste |
| │ |
| ▼ PERFORMER an interpreter plays the score: voicing, rolls, rubato, dynamics, pedal |
| │ — pitches strictly preserved, everything else decided |
| ▼ RENDER MIDI → sampled piano |
| ``` |
|
|
| Three findings from building it, all of which will save you time: |
|
|
| **1. Composition and performance are separate layers, and the performance layer carries an enormous share of the perceived quality.** This is testable in both directions. Take a piece that sounds good, quantize its onsets to a grid and flatten its velocities — same pitches, nothing else changed — and it collapses into mush. Take a raw, "dry", unpromising model output and have it genuinely *performed* — same pitches — and it comes alive. A dry-sounding generation is usually not a composition failure at all; it is an **unperformed** one. Judge a symbolic model's output only after something has played it. |
|
|
| **2. Rolled chords and rubato are not decoration — they are acoustics.** Staggering a chord's attacks by 20–70 ms spreads beating partials out in time and measurably lowers sensory roughness. "Expressive micro-timing" and "sounds less muddy" turn out to be the same phenomenon, which is why quantizing a dense passage makes it *rougher* rather than merely stiffer. |
|
|
| **3. Structure has to be imposed from outside.** Maurice has a 4096-token context (~2–3 minutes) and no notion of a piece's overall arc. Wrapping generation in an explicit plan — capture a theme, develop it under named transformations, return it — is what turns a fluent texture generator into something with a beginning, a middle and an end. For material longer than the context, carry the captured theme forward into the next span so later sections keep developing the same germ instead of starting over. |
|
|
| ## Limitations |
|
|
| - **Soft conditioning at the musical setting** — composer/key steer character but don't lock a signature; CFG sharpens the pull but tips toward frantic/over-dense, so the default leaves it off. |
| - **Uneven composers** — well-represented, stylistically bold composers (Liszt, Chopin, Debussy, Scriabin, Ravel) condition best; thin/subtle ones drift toward the generic-romantic center. |
| - Solo **piano** only; symbolic MIDI output (render with your own SoundFont/synth). |
|
|
| ## Data & license provenance |
|
|
| Maurice was trained on transcribed/curated solo-piano MIDI: **Aria-MIDI**, **GiantMIDI-Piano**, and **MAESTRO**, plus period-romantic pieces of unknown authorship (the `unknown` token), with ×12 transposition augmentation. It uses Aria's `AbsTokenizer`. |
|
|
| **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** This is a **non-commercial, research** release, matching the most restrictive of the training sources (MAESTRO is CC BY-NC-SA 4.0; Aria-MIDI and GiantMIDI carry research/non-commercial terms). You may use, share, and adapt Maurice for non-commercial purposes with attribution, sharing derivatives under the same terms. See `LICENSE`. |
|
|
| ## Files |
|
|
| | file | what | |
| |---|---| |
| | `model.safetensors` | weights (fp32, ~1.2 GB) | |
| | `config.json` | architecture config | |
| | `modeling_maurice.py` | model definition + `from_pretrained` | |
| | `generate.py` | end-to-end cold-start (tokenizer → MIDI) + best-of-N + CFG | |
| | `conditioning.json` | composer + key token maps and special-token ids | |
| | `requirements.txt` | dependencies | |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{maurice2026, |
| title = {Maurice: a polyphonic romantic-piano cold-start model}, |
| year = {2026}, |
| note = {From-scratch ~298M Qwen3-style decoder, composer+key conditioned, 4096 context} |
| } |
| ``` |
|
|
