S-KEY — musical key detection (ONNX, audio → key)

ONNX export of S-KEY, Deezer's key detector (harmonic VQT + ChromaNet), packaged for the musetric packages/ai runtime (onnxruntime-web on the wasm execution provider).

Unlike this project's chord and separation exports, this graph is self-contained: feature extraction, the network and the reference runner's postprocessing are all baked in, so the host feeds raw audio and reads back 24 numbers. At 0.33 MB it needs no WebGPU host and runs directly in Node.

audio [1, N] @ 22050 Hz mono, peak-normalized
  -> nnAudio harmonic VQT (log-amplitude HCQT)   [1, 1, 99, T]
  -> crop 84 bins                                [1, 1, 84, T]
  -> ChromaNet (ConvNeXt stack -> chroma -> 24)  [1, 24] (softmax'd)
  -> mean over batch -> softmax                  [24]

The whole track is fed at once; ChromaNet's time pooling collapses it to a single key. Two ops that cannot export with a dynamic time axis — an affine-free layer_norm whose normalized_shape would carry T, and AdaptiveAvgPool2d over a dynamic width — are swapped at export time for numerically identical equivalents, and the vendored CropCQT is inlined as a slice (the reference always crops at transpose = 0). The graph therefore carries no approximation: it reproduces the reference implementation's key and confidence up to float noise (~1e-6).

Intended uses & limitations

Intended:

Global key estimation for a music track, as a stage in an audio pipeline.
Client/edge inference through onnxruntime-web; the graph is small enough for the wasm EP.

Out of scope:

Key changes — time pooling returns one key for whatever you feed it. Segment the audio host-side if you need modulation tracking.
Anything outside the 24 major/minor classes (modal, atonal, ambiguous).
Use in other training frameworks — this is an inference-only export.

Limitations:

The graph has no resampler and no normalizer. Input must be 22050 Hz mono and peak-normalized (divide by max|x|), matching the reference load_audio.
confidence is not calibrated. The reference runner softmaxes an already softmaxed vector, which squeezes all 24 classes toward the 1/24 = 0.0417 floor and leaves the winner at only ~0.05–0.10. It is reproduced faithfully for parity, so treat it as a display value: use argmax for the decision, and top1 - top2 if you need a margin (worst case 8.8e-3 over our 20-track set).
Memory scales with track length — T is dynamic and the whole track is one forward pass.
The decoder is part of the contract. Peak-normalization cancels gain differences, but resamplers still differ: ffmpeg versus torchaudio moves confidence by ~1e-4 (argmax unaffected — see Validation).
Validate on the material fed in production. For this pipeline that is the instrumental stem, not the full mix and not a vocal stem: an isolated vocal yields the relative minor as often as the key.
Training-data provenance of the upstream checkpoint is not documented here.

How to use

argmax(probs) indexes config.json's keyMap ("A Major", "C minor", …); the probability at that index is the confidence.

import * as ort from 'onnxruntime-web';

const session = await ort.InferenceSession.create('skey.onnx', {
  executionProviders: ['wasm'],
});

// audio: Float32Array, whole track, 22050 Hz mono, peak-normalized.
const { probs } = await session.run({
  audio: new ort.Tensor('float32', audio, [1, audio.length]),
});
// probs.data: Float32Array(24) -> argmax -> keyMap index

See the musetric packages/ai host code (key/analyzeKey.node.ts) for the decode + peak-normalize + argmax pipeline around the session.

Files

File	Size	SHA256
`skey.onnx`	338,482 B	`5113c1378c1007c8559fcb767593366ba9794397b060535eb80a113db50530fc`
`config.json`	712 B	`20be1e139e1b05dea4bae2e2dde717d593c10c30bb38b300aeedc6693be88a52`

config.json records the I/O contract, the sample rate, the mono/peak-normalize flags and the 24-entry keyMap that gives argmax its meaning.

Signature — float32 weights, opset ai.onnx 17:

Tensor	Type	Shape	Meaning
`audio` (in)	float32	`[1, samples]`	whole track, 22050 Hz mono, peak-normalized; `samples` is dynamic
`probs` (out)	float32	`[24]`	key probabilities; `argmax` indexes `keyMap`

Validation

This export vs the PyTorch run_skey reference pipeline, key argmax:

Metric	Value
20 instrumental stems (the production input)	20/20
20 full mixes	20/20
15 isolated lead-vocal stems	15/15
degenerate outputs	0

Agreement is exact because the graph substitutes nothing: it reuses the same nnAudio VQT the reference uses, and the export-time op swaps are mathematically identical. Parity therefore does not depend on the material — unlike an export that stands a different transform in for the reference's.

End to end, the shipped Node path (ffmpeg decode + peak-normalize + wasm EP) against the Python CLI it replaces, over the same 20 instrumental stems:

Metric	Value
`{root, mode}` — the user-visible key	20/20 identical
`confidence` — max abs difference	9.08e-05
worst `top1 - top2` margin	8.81e-3

The residual ~1e-4 is the decoder, not the graph: torchaudio and ffmpeg resample to 22050 Hz with different filters. It sits ~100× below the model's worst decision margin, so the key is unaffected.

Re-run the parity gate on the exact published bytes before relying on it.

Source & lineage

Code license and weight license are separate; ONNX conversion does not change the weight license. Documented only as far as it is verifiable.

Architecture: harmonic VQT (harmonics=[1], fmin=27.5, n_bins=99, bins_per_octave=12, log-amplitude) → crop to 84 bins → ChromaNet: seven time-downsampling + ConvNeXt blocks, an octave pool to 12 chroma, a global time pool, a 1×1 classifier and a batch norm, softmaxed to 24 classes. Small — the whole graph is 0.33 MB in float32.
Reference implementation and weights: deezer/skey, MIT (per its LICENSE, Copyright (c) 2019-present, Deezer SA).
Checkpoint: skey/models/skey.pt @ commit 918b83d2 — 765,465 B, git blob 7ae0e9f2d226dcd2d34a65707dac5012b603e2ca — fetched at export time from raw.githubusercontent.com. The URL is pinned to that commit, so the fetch does not follow a moving branch.
Vendored code: the inference subset lives under musetric_toolkit/key_audio/skey in musetric-toolkit; see its thirdPartyNotices.md.
Export tooling: scripts/onnx/skey in musetric-toolkit.
Host runtime: packages/ai in musetric.

This export preserves the upstream MIT license; we do not claim authorship of the original weights.

License

MIT, inherited from the upstream weights.

Downloads last month: 31