README.md

---
license: cc-by-nc-sa-4.0
pipeline_tag: feature-extraction
library_name: pytorch
tags:
- music
- music-information-retrieval
- contrastive-learning
- embeddings
- jazz
---

# Jazz Harmony Embeddings — flat contrastive encoder (3-seed ensemble)

A small transformer that reads the chord progression of a jazz tune and
returns a single 128-dimensional vector, trained so that tunes with related
harmony — transpositions, alternate charts, contrafacts — land close together
in the vector space.

Trained from scratch on ~8,000 chord charts. Code, evaluation harness, and
full experiment records: <https://github.com/eigenben/jazz-harmony-embeddings>.
Precomputed embeddings for 6,900 jazz standards:
<https://huggingface.co/datasets/eigenben/jazz-harmony-embeddings>.

## Model description

- **Input**: a chord chart, tokenized as one token per chord event over the
  "changes skeleton" (adjacent repeats of the same chord merged, no-chord
  markers dropped; up to 256 events). Each token carries the chord's root
  pitch class, interval from the previous root, quality, triad and seventh
  class, bass interval, duration, metrical strength, and a continuation flag.
  Absolute key is deliberately *not* an input.
- **Architecture**: 3-layer transformer encoder (d_model 192, 6 heads,
  FFN 768), attention pooling, projected to a 128-d L2-normalized embedding.
  ~1.4M parameters per checkpoint.
- **Training**: NT-Xent contrastive loss (temperature 0.08). Positive pairs
  are (a) two independently transposed views of the same tune and (b) with
  probability 0.7, a real alternate chart of the same tune from another
  corpus. 20 epochs, batch of 256 tunes, AdamW (lr 3e-4, weight decay 0.01,
  2 warmup epochs). Music-theory synthetic augmentations (tritone
  substitutions, ii–V insertions, etc.) were implemented and ablated — they
  performed worse than real family positives and are OFF in this model.
- **Released model**: three checkpoints (seeds 7, 17, 29); the promoted model
  is the L2-normalized mean of their embeddings ("mean-cosine ensemble").
- **Similarity**: cosine (all vectors are unit-length).

## Training data

8,089 chord charts merged from four corpora (iReal Pro "Jazz 1460" community
playlist, the Bunks Jazz-Chord-Progressions-Corpus, ChoCo's real-book
partition, and the Jazz Harmony Treebank), deduplicated into 4,790
tune-families with family-leakage-safe train/validation/test splits — a tune
and its duplicates or contrafacts never straddle a split boundary. Chord
symbols only; no melodies, no audio. See the repository's
[DATA.md](https://github.com/eigenben/jazz-harmony-embeddings/blob/main/DATA.md)
for licensing and why the published artifacts exclude Treebank-derived data.

## Evaluation

Protocol v2 (documented in
[`eval/benchmark-policy.md`](https://github.com/eigenben/jazz-harmony-embeddings/blob/main/eval/benchmark-policy.md)):
model selection used validation loss and a *development* partition of held-out
test families only; the *confirmation* partition below was never consulted
during development. The curated contrafact set was consulted during iteration
and is reported as a development benchmark.

**Held-out unseen families (confirmation partition, 157 queries):**
median best-positive rank 1, MRR 0.667, Recall@20 0.677, nDCG@20 0.609.

**Curated graded contrafacts (37 queries, development benchmark), vs. classical
baselines on the same corpus:**

| method | median rank | MRR | Recall@20 | nDCG@20 |
|---|---:|---:|---:|---:|
| tf-idf over chord n-grams (B0) | 8 | 0.379 | 0.515 | 0.280 |
| sequence alignment + rerank (B2) | 2 | 0.504 | 0.522 | 0.374 |
| Bunks membrane-area, ISMIR 2023 (B3) | 2 | 0.492 | 0.626 | 0.320 |
| chord2vec + SIF pooling (B4) | 153 | 0.149 | 0.179 | 0.133 |
| **this model (3-seed ensemble)** | 4 | 0.422 | 0.464 | **0.393** |

Honest summary: the ensemble has the best graded ranking quality (nDCG@20) on
the contrafact benchmark but is not uniformly better than the strongest
pairwise baselines, which retain a better median rank. What the baselines
cannot do is produce a vector space — one point per tune that can be indexed,
clustered, and mapped. **Transposition invariance**, the property the training
recipe targets, holds at 99.5% (the same tune transposed to a random key
retrieves itself at rank 1 in 995/1000 trials); hierarchical bar/phrase-level
variants of this model were also trained and all failed that gate, which is
why this simpler flat model is the released one.

## Limitations

- Chords only: two tunes with identical changes but unrelated melodies are
  "the same" to this model, and reharmonized versions of one melody are far
  apart.
- The curated contrafact set was consulted during development; a fresh blind
  test set was planned but not built, so treat contrafact numbers as
  development-benchmark results. The confirmation-partition numbers are the
  clean held-out claim.
- Trained on lead-sheet-style jazz standards; unlikely to transfer to other
  genres or to beat-level performance transcriptions.
- Non-commercial license (CC BY-NC-SA 4.0) because the training corpus
  included CC BY-NC-SA data.

## Usage

The checkpoints are for re-embedding chord charts with the project's
tokenizer and schema; if you just want vectors for known jazz standards, use
the [precomputed dataset](https://huggingface.co/datasets/eigenben/jazz-harmony-embeddings)
instead.

```python
# pip install "jazz-harmony-embeddings @ git+https://github.com/eigenben/jazz-harmony-embeddings"
from huggingface_hub import hf_hub_download
from jazz_harmony_embeddings.models.inference import load_checkpoint, embed_tunes

paths = [
    hf_hub_download("eigenben/jazz-harmony-embeddings", f"checkpoints/best-s{seed}.pt")
    for seed in (7, 17, 29)
]
models = [load_checkpoint(path)[0] for path in paths]
# tunes: list[jazz_harmony_embeddings.data.schema.Tune]
# ensemble = L2-normalized mean of embed_tunes(model, tunes) across the three models
```