README.md

metadata

license: cc-by-nc-sa-4.0
pipeline_tag: feature-extraction
library_name: pytorch
tags:
  - music
  - music-information-retrieval
  - contrastive-learning
  - embeddings
  - jazz

Jazz Harmony Embeddings — flat contrastive encoder (3-seed ensemble)

A small transformer that reads the chord progression of a jazz tune and returns a single 128-dimensional vector, trained so that tunes with related harmony — transpositions, alternate charts, contrafacts — land close together in the vector space.

Trained from scratch on ~8,000 chord charts. Code, evaluation harness, and full experiment records: https://github.com/eigenben/jazz-harmony-embeddings. Precomputed embeddings for 6,900 jazz standards: https://huggingface.co/datasets/eigenben/jazz-harmony-embeddings.

Model description

• Input: a chord chart, tokenized as one token per chord event over the "changes skeleton" (adjacent repeats of the same chord merged, no-chord markers dropped; up to 256 events). Each token carries the chord's root pitch class, interval from the previous root, quality, triad and seventh class, bass interval, duration, metrical strength, and a continuation flag. Absolute key is deliberately not an input.
• Architecture: 3-layer transformer encoder (d_model 192, 6 heads, FFN 768), attention pooling, projected to a 128-d L2-normalized embedding. ~1.4M parameters per checkpoint.
• Training: NT-Xent contrastive loss (temperature 0.08). Positive pairs are (a) two independently transposed views of the same tune and (b) with probability 0.7, a real alternate chart of the same tune from another corpus. 20 epochs, batch of 256 tunes, AdamW (lr 3e-4, weight decay 0.01, 2 warmup epochs). Music-theory synthetic augmentations (tritone substitutions, ii–V insertions, etc.) were implemented and ablated — they performed worse than real family positives and are OFF in this model.
• Released model: three checkpoints (seeds 7, 17, 29); the promoted model is the L2-normalized mean of their embeddings ("mean-cosine ensemble").
• Similarity: cosine (all vectors are unit-length).

Training data

8,089 chord charts merged from four corpora (iReal Pro "Jazz 1460" community playlist, the Bunks Jazz-Chord-Progressions-Corpus, ChoCo's real-book partition, and the Jazz Harmony Treebank), deduplicated into 4,790 tune-families with family-leakage-safe train/validation/test splits — a tune and its duplicates or contrafacts never straddle a split boundary. Chord symbols only; no melodies, no audio. See the repository's DATA.md for licensing and why the published artifacts exclude Treebank-derived data.

Evaluation

Protocol v2 (documented in eval/benchmark-policy.md): model selection used validation loss and a development partition of held-out test families only; the confirmation partition below was never consulted during development. The curated contrafact set was consulted during iteration and is reported as a development benchmark.

Held-out unseen families (confirmation partition, 157 queries): median best-positive rank 1, MRR 0.667, Recall@20 0.677, nDCG@20 0.609.

Curated graded contrafacts (37 queries, development benchmark), vs. classical baselines on the same corpus:

method	median rank	MRR	Recall@20	nDCG@20
tf-idf over chord n-grams (B0)	8	0.379	0.515	0.280
sequence alignment + rerank (B2)	2	0.504	0.522	0.374
Bunks membrane-area, ISMIR 2023 (B3)	2	0.492	0.626	0.320
chord2vec + SIF pooling (B4)	153	0.149	0.179	0.133
this model (3-seed ensemble)	4	0.422	0.464	0.393

Honest summary: the ensemble has the best graded ranking quality (nDCG@20) on the contrafact benchmark but is not uniformly better than the strongest pairwise baselines, which retain a better median rank. What the baselines cannot do is produce a vector space — one point per tune that can be indexed, clustered, and mapped. Transposition invariance, the property the training recipe targets, holds at 99.5% (the same tune transposed to a random key retrieves itself at rank 1 in 995/1000 trials); hierarchical bar/phrase-level variants of this model were also trained and all failed that gate, which is why this simpler flat model is the released one.

Limitations

• Chords only: two tunes with identical changes but unrelated melodies are "the same" to this model, and reharmonized versions of one melody are far apart.
• The curated contrafact set was consulted during development; a fresh blind test set was planned but not built, so treat contrafact numbers as development-benchmark results. The confirmation-partition numbers are the clean held-out claim.
• Trained on lead-sheet-style jazz standards; unlikely to transfer to other genres or to beat-level performance transcriptions.
• Non-commercial license (CC BY-NC-SA 4.0) because the training corpus included CC BY-NC-SA data.

Usage

The checkpoints are for re-embedding chord charts with the project's tokenizer and schema; if you just want vectors for known jazz standards, use the precomputed dataset instead.

# pip install "jazz-harmony-embeddings @ git+https://github.com/eigenben/jazz-harmony-embeddings"
from huggingface_hub import hf_hub_download
from jazz_harmony_embeddings.models.inference import load_checkpoint, embed_tunes

paths = [
    hf_hub_download("eigenben/jazz-harmony-embeddings", f"checkpoints/best-s{seed}.pt")
    for seed in (7, 17, 29)
]
models = [load_checkpoint(path)[0] for path in paths]
# tunes: list[jazz_harmony_embeddings.data.schema.Tune]
# ensemble = L2-normalized mean of embed_tunes(model, tunes) across the three models