dettato transcriber, drums + grouper β€” ONNX

ONNX Runtime exports of the transcriber, the drum model and the instrument grouper from kobimusic/dettato, kobimusic/dettato-drums and kobimusic/dettato-grouper (Apache-2.0), for running on CPU without PyTorch (for example with onnxruntime-node). Weights and network are unchanged; see NOTICE for attribution and the list of changes. Credit for the model belongs to its author (KobiMusic / Nathan Mills). This repo is not affiliated with or endorsed by them.

These are three stages of dettato's pipeline: the transcriber finds pitched notes (88 piano keys, two voices per key), the drum model finds drum-kit and orchestral percussion hits, and the grouper sorts the notes into instrument tracks. The evidence repairer (dettato-evidence) is not included.

Files

File Size What
transcriber.onnx 81.9 MB transcriber, fp32, harmonic log-frequency front end inside the graph
drums.onnx 47.6 MB drum model (drum-kit net, 3 orchestral experts and router), fp32, three log-spectrogram front ends inside the graph
transcriber-taps.onnx 81.9 MB the transcriber with two extra outputs (hidden state, stem) that the grouper reads
grouper.onnx 32.1 MB grouper network (one 24 s window of notes), front end inside the graph
grouper-head.json 0.8 MB the grouper's track head (3 linear layers), decode parameters and window settings
SHA256SUMS checksums
tools/export_dettato_onnx.py the export script

Graph interface

input audio f32 [1, 128000] β€” one 8 s window at 16 kHz mono
output maps f32 [1, 6, 400, 88] β€” sigmoid probabilities for onset, frame, offset, onset2, frame2, offset2; 20 ms frames; lane i = MIDI pitch 21 + i

Run overlapping windows (the reference hops 2 s; hopping 4 s is about twice as fast with 97–100% note agreement on the clips I tried), average the maps per frame with a Hann weight, then pick notes from the peaks (onset β‰₯ 0.25, second voice β‰₯ 0.3, offset β‰₯ 0.3, frame β‰₯ 0.4). The reference decode is in dettato/transcriber.

drums.onnx

input audio44 f32 [1, N] β€” mono audio at 44.1 kHz (raw level; the βˆ’18 dBFS normalisation is inside the graph)
input audio16 f32 [1, M] β€” the same audio resampled to 16 kHz
input frames i64 [] β€” feature frames to keep: min(M // 160, (N - 1) // 441 + 1)
outputs router f32 [], then kit, kit_vel, drum, vel f32 [1, T, 61] (GM keys 27–87) and timpani f32 [1, T, 13] (keys 41–53); all probabilities at 50 fps

Decode as in the reference: if router < 0.5 use kit, otherwise drum and timpani; hits are local maxima β‰₯ 0.5, at least 3 frames apart per key. The model reads the 7.8–20 kHz band, so only use sources recorded at 40 kHz or more. The attention runs over the whole recording, so memory grows with its length (quadratically in frames); the reference has the same behaviour.

transcriber-taps.onnx and grouper.onnx

transcriber-taps.onnx: same input as transcriber.onnx; outputs maps f32 [1, 6, 400, 88], hidden f32 [1, 368, 400, 88] and stem f32 [1, 368, 400, 96].

grouper.onnx: inputs audio f32 [1, 408000] (a 24 s window with 0.5 s before and 1 s after, 16 kHz), notes f32 [1, 768, 12] (per-note scalars, see gather in the reference), mask f32 [1, 768] and feat f32 [1, 768, 3, 368] (the transcriber's states at each note); outputs inst f32 [1, 768, 73] (logits), emb f32 [1, 768, 32] (unit track embeddings) and h f32 [1, 768, 176]. Everything after the network (tuning, note gathering, windowing, clustering, naming) is plain code: see the reference's grouper/ package. The grouper's PCEN smoother is an exact causal FIR here instead of the reference's FFT convolution, which is noisy in quiet bins; the exported network therefore differs from PyTorch by about 0.2 in logits of magnitude ~17 (argmax agreement 99–100% on random inputs), while end-to-end results match (below).

Fidelity

Same 8 s window, ONNX Runtime vs the PyTorch model (fp32): max |Ξ”| 1.5e-6 over all six maps. Full recordings (7 and 5 windows, hop 2 s), notes decoded from the ONNX maps with a TypeScript port of the reference decode vs the reference's own notes: identical (37/37 and 33/33 notes, onset/offset times within 5e-8 s).

Drums: ONNX maps vs the PyTorch pipeline within 7e-4 and identical router values on a drum loop and a piano loop (lengths differ from the 6 s used for the export). Hits decoded in TypeScript (own windowed-sinc resampler instead of soxr) vs the reference: 109/110 and 196/197 hits matched within one 20 ms frame on two drum loops, velocities within 1, no hits on a piano loop (as in the reference).

Grouper, end to end: a 30 s mix of several loops (355 transcribed notes) through a TypeScript port of the reference pipeline (tuning, slots, note gathering, windows, decode) with these ONNX graphs gave the same tracks as the PyTorch reference β€” piano 191 / flute 60 / electric guitar 40 / cello 31 notes, all 322 notes in the same-named track.

Reproducing

DETTATO_WEIGHTS=/path/to/weights python tools/export_dettato_onnx.py transcriber --out out
DETTATO_WEIGHTS=/path/to/weights python tools/export_dettato_onnx.py drums --out out
DETTATO_WEIGHTS=/path/to/weights python tools/export_dettato_onnx.py grouper --out out
DETTATO_WEIGHTS=/path/to/weights python tools/export_grouper_head.py
# /path/to/weights/kobimusic/<repo>/{config.json,model.safetensors} from huggingface.co/kobimusic/dettato and dettato-drums

Citation

@software{dettato2026,
  title  = {Dettato: multi-instrument audio-to-MIDI transcription},
  author = {Mills, Nathan},
  year   = {2026},
  url    = {https://github.com/kobimusic/dettato}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for fpp-ableton/dettato-onnx

Quantized
(2)
this model