Spaces:
Running
Running
Add murmel-flash-m-v5 and murmel-flash-l-v5 results
#3
by maartenmaarten - opened
Results for two murmel-flash models, run with the new murmel-flash backend (https://github.com/tvosch/Dutch-ASR-leaderboard/pull/3), normalizer v2, package defaults (beam 4), on an A100.
| Model | FLEURS | VoxPopuli | MLS | Average |
|---|---|---|---|---|
| theaifactorybv/murmel-flash-l-v5 | 6.56 | 9.21 | 6.75 | 7.51 |
| theaifactorybv/murmel-flash-m-v5 | 8.65 | 10.35 | 8.64 | 9.21 |
- murmel-flash-m-v5: 128M parameters, open weights on Hugging Face (gated, automatic approval), licence murmel-flash-licence-1.0.
- murmel-flash-l-v5: 397M parameters, proprietary. The weights are not public.
All 4,576 samples were transcribed, none failed. RTF was measured on an A100, not on the H100 used for the other local models.
Files:
results/theaifactorybv__murmel-flash-m-v5.jsonresults/theaifactorybv__murmel-flash-l-v5.jsonapp.py: murmel-flash-l-v5 links to murmel.nl and counts as proprietary (its weights are private); runtime label for the murmel-flash backend.