rousseau/allinone-harmonix

all-in-one (Harmonix, 8 folds) — safetensors for Burn

Weights of all-in-one (Taejun Kim and Juhan Nam), converted unchanged from the original PyTorch checkpoints to safetensors so that Rusty Music can run the model in Rust with Burn. No retraining and no fine-tuning: these are the eight harmonix-fold0 … harmonix-fold7 models of the harmonix-all ensemble.

• Paper: Taejun Kim and Juhan Nam, All-In-One Metrical and Functional Structure Analysis with Neighborhood Attentions on Demixed Audio, IEEE WASPAA 2023.
• Original code: https://github.com/mir-aidj/all-in-one (MIT).
• Original weights: https://huggingface.co/taejunkim/allinone (MIT).

Contents

allinone-harmonix-fold{0..7}.safetensors: each one is the state_dict of one fold (491 tensors, float32, about 1.2 MB). The metadata records the source checkpoint and the licence.

Inputs and outputs are the same as in the original model:

• Input: madmom log spectrograms of the four HTDemucs stems (bass, drums, other, vocals), at 100 frames/s, 30 Hz–17 kHz, 81 bands.
• Output: beat, downbeat, section-boundary and 10 function logits (Harmonix labels). Average them over the folds.

Port and parity

The Burn port, including dilated neighbourhood attention re-implemented without NATTEN, is in crates/transcription/src/sections/ of Rusty Music (GPL-3.0-or-later).

Parity with the original Python, on the same stems:

• logits within 1e-5;
• identical sections on 94 annotated songs.

Run in float32: in float16 the logits drift by several units.

Licence

MIT, as the original weights. Copyright (c) 2023 Taejun Kim.

@inproceedings{kim2023allinone,
  title={All-In-One Metrical and Functional Structure Analysis with Neighborhood Attentions on Demixed Audio},
  author={Kim, Taejun and Nam, Juhan},
  booktitle={IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)},
  year={2023}
}