JIT-MPE and CRNN comparison checkpoints

Trained weights for the paper JIT-MPE: Just-in-Time Multi-Pitch Estimation under Output Deadlines (Yoshiyuki Kobayashi, Frieve, 2026). Source code · Paper page · License · Data and software credits

JIT-MPE is a small causal model (about 86k parameters) that estimates which of the 88 piano pitches are sounding in mixed-ensemble audio. It processes the audio in 20 ms frames and can revise a frame until its output deadline (for example, 50 ms). On one CPU thread it takes about 2.4 ms per frame.

This repository contains the same weights as the GitHub release, unchanged (no conversion or retraining). It also includes the comparison CRNN models that the author trained for the paper. The external systems compared in the paper are not included; get them from their own authors.

Models

System ID	Model	Parameters	Type
S	JIT-MPE	86,425	Causal, five delay heads
A0	Harmonic CRNN	85,813	Causal, one head
A1d	Dilated CRNN	95,767	Causal, one head
B1	Plain CRNN	95,767	Causal, one head
A0-bi	Harmonic BiCRNN	83,729	Bidirectional, full-length input only
A1d-bi	Dilated BiCRNN	93,936	Bidirectional, full-length input only
B1-bi	Plain BiCRNN	93,936	Bidirectional, full-length input only

Each system was trained with seeds 1, 3 and 5. These are three separate runs, not an ensemble; the paper reports their mean. Parameter counts exclude the auxiliary onset head, which is used only in training (it is still stored in the checkpoints).

Files

weights/<SYSTEM>-s<SEED>.safetensors       # model weights (21 files)
weights/<SYSTEM>-s<SEED>.calibration.json  # decision thresholds for each model (21 files)
weights/head-selection-s<SEED>.json        # which delay head to use for each deadline (one per seed)
configs/<SYSTEM>.json                      # training settings (not Transformers configs)
requirements-paper.txt                     # Python environment used for the paper
release-manifest.json                      # size and SHA-256 of every file, source revision, paper metadata
LICENSE
THIRD_PARTY.md
CITATION.cff

To run one model you need three files: its .safetensors, its .calibration.json and the head-selection file for the same seed. Keep them in the same folder. The model code is in the GitHub package, so AutoModel and the Transformers pipeline cannot load these weights.

Usage

Requirements: Python 3.10 or later and PyTorch for your platform. Install the source code at the matching revision and the Hugging Face Hub client:

git clone https://github.com/Frieve-A/jit-mpe.git
cd jit-mpe
git checkout 08bbd5acbb223962895f847a86c2d47a36179896
python -m pip install -e .
python -m pip install huggingface_hub==0.34.3

Python example (downloads only the three files for S, seed 1):

from pathlib import Path
import torch
from huggingface_hub import snapshot_download
from jit_mpe import load
from jit_mpe.infer import estimate, read_audio, write

name, seed = "S", 1
snapshot = snapshot_download(
    repo_id="Frieve-A/jit-mpe",
    revision="main",  # replace with a full Hub commit hash to pin a published snapshot
    allow_patterns=[
        f"weights/{name}-s{seed}.safetensors",
        f"weights/{name}-s{seed}.calibration.json",
        f"weights/head-selection-s{seed}.json",
    ],
)
system = load(name, seed, weights_dir=Path(snapshot) / "weights", device="cpu")
torch.set_num_threads(1)
with torch.inference_mode():
    times, frequencies = estimate(
        read_audio("song.wav"), system,
        clock="scheduled_stale_hold", deadline_ms=50,
    )
write("song.csv", times, frequencies)

Command-line example (replace HUB_COMMIT with a commit hash of this repository, or use main):

hf download Frieve-A/jit-mpe weights/S-s1.safetensors weights/S-s1.calibration.json weights/head-selection-s1.json --revision HUB_COMMIT --local-dir hub-model
python -m jit_mpe.infer song.wav --weights hub-model/weights --system S --seed 1 --clock scheduled_stale_hold --deadline 50 --out song.csv

Input and output

• Input: any audio file. read_audio mixes it down to mono and resamples it to 44.1 kHz.
• Output: one row every 10 ms. A CSV row has the time in seconds, followed by the frequencies (Hz) of the pitches that are sounding. An NPZ file has times [Q] and frequencies [Q, 88]; absent pitches are 0.
• Pitch range: MIDI 21–108 (A0–C8), with a fine frequency estimate for each active pitch.

Deadlines

A deadline D means that the answer for time q must be ready by q + D. With scheduled_stale_hold, the output for q is the most recent frame that is ready by then, so there is always an answer, but it may be slightly old.

S has five delay heads that look 0, 20, 40, 80 and 160 ms ahead. For each deadline, the head-selection file says which head to use; for 50 ms it is the 40 ms head. To process a whole file without a deadline, omit clock and deadline_ms (Python) or --clock and --deadline (CLI). The -bi models only work this way, because they use future audio.

estimate reads a complete file and reproduces what a real-time system would have output by each deadline. It is not a live audio callback; for frame-by-frame streaming with model.step, see bench/cpu.py in the source code.

Training

The models were trained on GuitarSet, Slakh2100-redux, MusicNet, Schubert Winterreise, Cantoria, Dagstuhl ChoirSet, vocadito and Groove MIDI. Versions, links and licenses are listed in THIRD_PARTY.md. This repository does not include any training audio, annotations, prepared features or external models.

Training has two stages: 10,400 steps at learning rate 0.002, then 3,000 steps at 0.0005 with an added onset loss (batch size 16, 192-frame clips, 16 warm-up frames). Afterwards, the decision thresholds are fitted on a calibration split and the delay heads are selected on a development set. The GitHub README gives the full commands for data preparation, training and evaluation. requirements-paper.txt records the environment used for the paper (Python 3.12, PyTorch 2.5.1); a CUDA build is not needed for CPU inference.

Results

The numbers below are copied from Table 4 of the paper (arXiv v1); they were not re-measured for this upload. They are F1 scores (%) for JIT-MPE with scheduled_stale_hold at a 50 ms deadline, averaged per recording and over the three seeds, with pitches rounded to semitones.

Held-out set	F1 (%)
AAM	82.6
ChoraleBricks	78.9
BSED-real	64.5
BSED-synthetic	77.0

Notes:

• The paper also reports 50-cent (continuous f0) scores and full-length scores. These use different settings and should not be compared directly with the table above.
• ChoraleBricks is annotated with continuous f0; the semitone score above is given only so that all sets use the same metric.
• The BSED recordings were not used in training, but some of the pieces also appear in the MusicNet training data.

See the paper for development-set results, comparisons with other systems, confidence intervals and scoring details.

Intended use and limitations

Intended for research on frame-level pitch estimation, pitch-display prototypes and experiments with output deadlines. The models do not identify instruments and do not produce note events (onsets/offsets).

• The paper does not evaluate note-level transcription, singing voice or real popular-music recordings.
• Performance on other instruments or recording conditions has not been tested.
• The reported CPU timings depend on the hardware and do not include audio I/O, scheduling or display latency.
• Retraining with the released code will not give bit-identical weights, because these weights were trained before deterministic training was added. The public vocadito split also differs from the one used in the paper (see the determinism section of the GitHub README). To reproduce the paper's inference results, use these checkpoints.

License and citation

The code and these weights are released by the author under the MIT license. This does not cover the training datasets or external models, which keep their own licenses. In particular, the Schubert Winterreise recordings have their own conditions, described in the original dataset release.

@misc{kobayashi2026jitmpe,
  title = {{JIT-MPE}: Just-in-Time Multi-Pitch Estimation under Output Deadlines},
  author = {Kobayashi, Yoshiyuki},
  year = {2026},
  eprint = {2610.33113},
  archivePrefix = {arXiv},
  primaryClass = {cs.SD},
  doi = {10.48550/arXiv.2610.33113},
  url = {https://arxiv.org/abs/2610.33113}
}