PearlLeeStudio/TheArtist-MusicTransformer-lora-classical

TheArtist Music Transformer — LoRA Adapter (Classical)

LoRA adapter that conditions the F1 base (PearlLeeStudio/TheArtist-MusicTransformer-ft-pop80) toward classical chord progressions. One of eleven per-genre adapters released alongside the paper Empirical Study of Pop and Jazz Mix Ratios for Genre-Adaptive Chord Generation (Lee, 2026).

Adapter summary

Field	Value
Base model	PearlLeeStudio/TheArtist-MusicTransformer-ft-pop80 (F1, 25.6M params)
Adapter type	LoRA (Q/K/V projections)
LoRA rank	8
LoRA alpha	16
LoRA dropout	0.05
Target modules	w_q, w_k, w_v
Trainable parameters	200K (0.78% of base)
Adapter file size	~800 KB
Base vocabulary	351 tokens (jazz/pop)
Vocabulary extension	+13 genre tokens (embedding_extension.pt)
Training epochs	8

Training data

1,000 chord-progression sequences in the classical subset of the Chordonomicon dataset. Bach chorales come from the public-domain music21 corpus and are license-clean for commercial use; the CC BY-NC restriction below applies only because the LoRA's runtime is loaded on top of an F1 base that itself trained on Chordonomicon.

Genre character

Bach chorales (curated subset from the music21 corpus)

Evaluation

Validation token-level metrics on the genre-specific val split (37 sequences, no key augmentation). The F1 base column uses the same val split, same dataloader, and the same [GENRE:none]-initialized embedding-extension setup as the LoRA run — only the LoRA parameters and the trained embedding rows differ. The LoRA snapshot is the best-validation epoch (7/8).

Metric	F1 base alone	F1 + this LoRA	Δ
Top-1 accuracy (%)	43.54	58.7	+15.16
Top-5 accuracy (%)	72.82	85.6	+12.78
Cross-entropy loss	2.8653	1.3486	-1.5167

Source: ai/results/f1_per_genre_baseline.csv + ai/logs/ft_f1_lora_classical_*.log. Higher top-1/top-5 and lower loss are better. The 11-adapter comparison and the genre-distance-vs-gain pattern are reported in the 2026 workshop paper.

License and use

The adapter weights are released under CC BY-NC 4.0 (matching Chordonomicon, the upstream training corpus). Permitted: research, paper replication, portfolio, demo. Not permitted: commercial deployment without separate licensing of upstream data.

Usage

import torch
from huggingface_hub import hf_hub_download
from peft import PeftModel
from model import MusicTransformer
from tokenizer import ChordTokenizer

# 1. Load the F1 base
base_path = hf_hub_download(
    repo_id="PearlLeeStudio/TheArtist-MusicTransformer-ft-pop80",
    filename="best.pt",
)
base_ckpt = torch.load(base_path, map_location="cpu", weights_only=False)
tokenizer = ChordTokenizer()
model = MusicTransformer(
    vocab_size=tokenizer.vocab_size,
    d_model=512, n_heads=8, d_ff=2048, n_layers=8,
    max_seq_len=256, dropout=0.0, pad_id=tokenizer.pad_id,
)
model.load_state_dict(base_ckpt["model_state_dict"])

# 2. Extend the embedding to fit the LoRA's expanded vocabulary
ext_path = hf_hub_download(repo_id="PearlLeeStudio/TheArtist-MusicTransformer-lora-classical", filename="embedding_extension.pt")
ext = torch.load(ext_path, map_location="cpu", weights_only=False)
# (See model/README.md for the apply-extension recipe.)

# 3. Apply the LoRA adapter
adapter_dir = hf_hub_download(repo_id="PearlLeeStudio/TheArtist-MusicTransformer-lora-classical", filename="adapter_model.safetensors")
model = PeftModel.from_pretrained(model, adapter_dir.rsplit("/", 1)[0])
model.eval()

Citation

Preprint: arXiv:2605.04998.

@misc{lee2026chordmix,
  title         = {Empirical Study of Pop and Jazz Mix Ratios for Genre-Adaptive Chord Generation},
  author        = {Lee, Jinju},
  year          = {2026},
  eprint        = {2605.04998},
  archivePrefix = {arXiv}
}