README.md

---
license: cc-by-4.0
language:
- en
tags:
- audio
- music-information-retrieval
- guitar
- tablature
- guitar-tablature-estimation
- onnx
- tabcnn
- guitarset
- guitarprofx
- cqt
pipeline_tag: audio-classification
library_name: onnx
---

# TabCNN (ONNX) — guitar tablature estimation

ONNX exports of **TabCNN** (Wiggins & Kim, ISMIR 2019) — a small CNN that
estimates guitar **tablature** (which fret is played on each of the 6 strings)
from audio, frame by frame. Both exports end in a per-string **LogSoftmax** head
so a decoder can consume log-probs directly, and share the **frozen output
contract**: `[N, 6, 21]`, **class 0 = string silent/closed**, **class k = fret
k−1** (class 1 = open, class 20 = fret 19).

Built for the pure-Dart [`onnx_runtime_dart`](https://github.com/CrispStrobe/onnx_runtime_dart)
runtime (no native ORT / FFI, web-capable), but it's standard ONNX.

## Two models — pick by input

| File | Variant | Best for | EGSet12 (real electric) | Front-end |
|---|---|---|---|---|
| **`tabcnn-gpfx.onnx`** ⭐ | **GuitarProFX-augmented** (Pedroza et al., DAFx-24) | electric guitar, effects, real tones | **F1 ≈ 0.59** | CQT → **dB → [0,1]** |
| `tabcnn.onnx` | vanilla (trained on GuitarSet) | clean / acoustic-ish | ~0.45 zero-shot | CQT → **raw magnitude** |
| `tabcnn-cqt.bin` | — | the shared 192-bin CQT filterbank both need | — | — |

**Use `tabcnn-gpfx.onnx`** unless you specifically want raw-magnitude features —
it's the robust variant (the vanilla model collapses on distorted/electric tones).

- `tabcnn-gpfx.onnx` — sha256 `8d9ce59157bdab37fb4816d32d7f29f3da0cdbf3c7876707c819af4d1f88e6b7`, 833,982 params
- `tabcnn.onnx` — sha256 `15c58000ed2d1deb3d3fc07581aa1823482dad91d913399dc0209ef240ad8a51`
- `tabcnn-cqt.bin` — sha256 `4e5dfa1f10f76545a30cbfd3224431503dbad943b1def78624632284e6df597a`

## Inputs / outputs (both models)

- **Input** `input : float32[N, 192, 9, 1]` — per frame, a 9-frame context window
  of the 192-bin CQT (`bins × context × channel`). `N` = a batch of windows.
- **Output** `output : float32[N, 6, 21]` — per string, a **LogSoftmax** over 21
  classes; **class 0 = closed**, class k = fret k−1.
- **Frame hop** = 512 / 22050 = **0.023220 s** (≈ 43 fps).

## CQT front-end — the #1 correctness risk (differs by variant!)

Both use the same CQT geometry (sr 22050, hop 512, n_bins 192, bins_per_octave
24, fmin C1 = 32.703 Hz), and `tabcnn-cqt.bin` is that 192-bin filterbank
precomputed (banded, n_fft 32768, boxcar STFT) for a `librosa`-free front-end —
it matches `librosa.cqt` at **cosine 0.999947 / median magnitude ratio 0.9999**.

**The normalization after the CQT differs:**

```python
import librosa, numpy as np
C = np.abs(librosa.cqt(y, sr=22050, hop_length=512, n_bins=192, bins_per_octave=24))  # [192, frames]

# tabcnn-gpfx.onnx  (GuitarProFX — recommended):  per-clip dB, then [0,1]
feats = librosa.amplitude_to_db(C, ref=np.max)          # [-80, 0], per clip
feats = (feats - feats.min()) / (feats.max() - feats.min() + 1e-9)

# tabcnn.onnx  (vanilla):  RAW magnitude, no log/norm
# (peak-normalize the *waveform* first: y = librosa.util.normalize(y))
feats = C

repr_ = np.swapaxes(feats, 0, 1)   # [frames, 192]; window = pad 4 each side, [f:f+9] -> [192,9,1]
```

With `tabcnn-cqt.bin`, magnitude = `|Σ band·boxcarSTFT| / √length` (the blob's
`mean`/`std` header fields are 0/1 and unused). Apply the per-variant
normalization above to that magnitude.

## Performance & verification

- `tabcnn-gpfx.onnx`: **EGSet12 (12-track, frame-level tab F1) = 0.59 micro / 0.55
  macro — MEASURED here** (per-track 0.30–0.80), matching the paper's reported
  0.585 for this model; on clean GuitarSet it reaches ~0.77. A big lift over the
  vanilla model's ~0.45 EGSet12 zero-shot.
- `tabcnn.onnx`: held-out GuitarSet tab F1 0.745 (paper 0.748).
- Both run on `onnx_runtime_dart` faithfully vs the reference (per-string argmax
  agreement; standard ops only — Conv/MaxPool/Relu/MatMul/LogSoftmax + a class
  reorder for gpfx).

## Class-layout note (gpfx)

The GuitarProFX (`amt-tools`) model natively uses **class 20 = silence, class k =
fret k**. The export **remaps** that to the shared contract above (class 0 =
silent, class k = fret k−1) with a `roll`, so both `.onnx` files present the
identical `[6,21]` layout to the decoder.

## Licence & attribution (CC BY 4.0)

- Vanilla: trained here on **GuitarSet** (CC BY 4.0). Attribution to GuitarSet required.
- GuitarProFX: weights from **Zenodo 11406378** (`best_TabCNN_tablature_trancription_model`,
  CC BY 4.0), the DAFx-24 GuitarProFX model, built on Cwitkowitz's
  [`amt-tools`](https://github.com/cwitkowitz/amt-tools) (MIT). Attribution to
  GuitarSet + Pedroza et al. required.

```bibtex
@inproceedings{xi2018guitarset, title={GuitarSet: A Dataset for Guitar Transcription},
  author={Xi, Qingyang and Bittner, Rachel M. and Pauwels, Johan and Ye, Xuzhou and Bello, Juan Pablo}, booktitle={ISMIR}, year={2018}}
@inproceedings{wiggins2019tabcnn, title={Guitar Tablature Estimation with a Convolutional Neural Network},
  author={Wiggins, Andrew and Kim, Youngmoo}, booktitle={ISMIR}, year={2019}}
@inproceedings{pedroza2024guitarprofx, title={Leveraging Real Electric Guitar Tones and Effects to Improve Robustness in Guitar Tablature Transcription Modeling},
  author={Pedroza, Hegel and others}, booktitle={DAFx}, year={2024}}
```

Reproduction (vanilla) + export scripts:
[`onnx_runtime_dart/tool/tabcnn/`](https://github.com/CrispStrobe/onnx_runtime_dart/tree/main/tool/tabcnn).

## Provenance — mixed authorship, and outside the GPAI definition

Added 2026-08-02 during an account-wide provenance review.

**This repository holds weights of two different origins**, which is why the
per-file table above matters more here than in a typical conversion repo:
`tabcnn.onnx` (vanilla) was **trained here** on GuitarSet (CC BY 4.0), while the GuitarProFX weights were **re-exported** from Zenodo record 11406378 (Pedroza et al.). Attribution obligations differ between them and are set out above.

Most `cstr/*` repositories are conversions where the upstream research team
remains the provider of the model. That description is only half true here, and
a blanket "format conversion only" statement — of the kind added across the
conversion repos in this review — would have been inaccurate for this one. It is
deliberately not made.

**EU AI Act Art. 53 does not apply.** Art. 53 binds providers of *general-purpose
AI models*, defined by Art. 3(63) as models displaying "significant generality"
and capable of "competently performing a wide range of distinct tasks". A CNN transcribing guitar tablature from audio
does not meet that definition, so the Art. 53(1)(c) copyright-policy and
53(1)(d) training-content duties are not engaged for either set of weights. The
training-data and licence documentation above is published because attribution
is required and because it is useful — not because Art. 53 compels it.