Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Flova/omr_transformer
|
| 4 |
+
pipeline_tag: image-to-text
|
| 5 |
+
tags:
|
| 6 |
+
- optical-music-recognition
|
| 7 |
+
- omr
|
| 8 |
+
- turkish-makam-music
|
| 9 |
+
- onnx
|
| 10 |
+
library_name: onnx
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# omr-weights — Turkish (makam) optical music recognition
|
| 14 |
+
|
| 15 |
+
ONNX graphs for [KomaVision](https://komavision.netlify.app), an optical music recognition model
|
| 16 |
+
for **Classical Turkish (makam) music**: a photo or screenshot of sheet music in, notes out —
|
| 17 |
+
including the microtonal accidentals (koma, küçük mücennep, bakiye, büyük mücennep) that Western
|
| 18 |
+
OMR models have no vocabulary for.
|
| 19 |
+
|
| 20 |
+
Source code: <https://github.com/EmreDikimen/Turkish_note_to_solfeggio_converter>
|
| 21 |
+
|
| 22 |
+
## What is here
|
| 23 |
+
|
| 24 |
+
Int8-quantized ONNX exports of a Donut-style vision-encoder-decoder (~143M parameters):
|
| 25 |
+
|
| 26 |
+
| File | Role |
|
| 27 |
+
|---|---|
|
| 28 |
+
| `encoder_model.onnx` | image encoder |
|
| 29 |
+
| `decoder_model.onnx` | first decode step |
|
| 30 |
+
| `decoder_with_past_model.onnx` | subsequent steps, with KV cache |
|
| 31 |
+
|
| 32 |
+
Input is a **409×583** grayscale strip — one staff line's worth of music, not a whole page. The
|
| 33 |
+
application slices a page into strips before decoding. Output is a LilyPond-flavoured token stream
|
| 34 |
+
with AEU accidental tokens.
|
| 35 |
+
|
| 36 |
+
The same graphs run in two places: in the browser via `onnxruntime-web` (the offline fallback) and
|
| 37 |
+
on a CPU server via `onnxruntime-node`. There is one decode implementation, shared.
|
| 38 |
+
|
| 39 |
+
## Licence and attribution
|
| 40 |
+
|
| 41 |
+
**Apache-2.0**, inherited from the base model.
|
| 42 |
+
|
| 43 |
+
Fine-tuned from **[`Flova/omr_transformer`](https://huggingface.co/Flova/omr_transformer)**
|
| 44 |
+
(Apache-2.0) — a pretrained OMR transformer. That model is the reason this project did not need to
|
| 45 |
+
train an OMR system from scratch, and its licence and attribution travel with these weights as
|
| 46 |
+
Apache-2.0 §4 requires.
|
| 47 |
+
|
| 48 |
+
## Training data
|
| 49 |
+
|
| 50 |
+
- **Self-rendered synthetic strips.** Turkish scores engraved by the project's own VexFlow
|
| 51 |
+
renderer, with augmentation aimed at what users actually upload (screenshots more than photos).
|
| 52 |
+
The pixels and the labels come from one code path, so a label can never disagree with its image.
|
| 53 |
+
- **Real printed pages**, hand-labelled, from freely-published Turkish score archives. These are
|
| 54 |
+
used as training and evaluation data locally and are **not redistributed** here or anywhere.
|
| 55 |
+
- Score metadata for the synthetic renders derives from
|
| 56 |
+
**[SymbTr](https://github.com/MTG/SymbTr)** (Karaosmanoğlu et al.), which is licensed
|
| 57 |
+
**CC BY-NC-SA 4.0** — attributed here accordingly.
|
| 58 |
+
|
| 59 |
+
No Western rehearsal data was used in fine-tuning; coverage comes from self-rendered Turkish
|
| 60 |
+
strips.
|
| 61 |
+
|
| 62 |
+
## Intended use and limits
|
| 63 |
+
|
| 64 |
+
Intended for reading Classical Turkish music notation. It is **not** a general-purpose OMR model —
|
| 65 |
+
it was fine-tuned on a Turkish token vocabulary and will not do anything sensible with orchestral
|
| 66 |
+
or piano scores.
|
| 67 |
+
|
| 68 |
+
Known limits, stated plainly:
|
| 69 |
+
|
| 70 |
+
- Accuracy on **clean synthetic** strips is effectively solved; accuracy on **real printed pages**
|
| 71 |
+
is the open problem and is substantially lower. Treat every decode as a draft to be corrected —
|
| 72 |
+
the application ships an editor for exactly that reason.
|
| 73 |
+
- The microtonal accidentals are the hard part, and the koma/küçük mücennep distinction is where
|
| 74 |
+
the remaining errors concentrate.
|
| 75 |
+
- Long or dense staff lines can overrun the decoder's token budget.
|
| 76 |
+
- Handwritten manuscript is out of scope.
|
| 77 |
+
|
| 78 |
+
## Citation
|
| 79 |
+
|
| 80 |
+
If the base model is useful to you, cite that first —
|
| 81 |
+
[`Flova/omr_transformer`](https://huggingface.co/Flova/omr_transformer). For SymbTr:
|
| 82 |
+
|
| 83 |
+
> M. K. Karaosmanoğlu, "A Turkish makam music symbolic database for music information retrieval:
|
| 84 |
+
> SymbTr", ISMIR 2012.
|