karna-large / eval /large_teacher.json
Darayut's picture
Add v1 evaluation results (student, fine-tuned teacher, stock 1B, inference time)
f370090 verified
Raw History Blame Contribute Delete
15.2 kB
{
"ckpt": "runs/ftfirst/stage1_finetune/teacher.pt",
"params_total": 975065300,
"params_encoder": 962496128.0,
"gmacs_per_audio_second": 50.277134336,
"dtype": "bf16",
"results": {
"ddd_dev": {
"cer": 0.033986958765304685,
"n": 400,
"rtf": 0.006292361957186866,
"examples": [
{
"ref": "αž˜αŸ’αž…αžΆαžŸαŸ‹αž’αžΆαž•αžΆαžαž˜αž·αž“αžαŸ‚αž„αžαŸ‚αž‡αŸ’αžšαžΎαžŸαžšαžΎαžŸαž‚αŸ’αžšαžΏαž„αžŸαž„αŸ’αž αžΆαžšαž·αž˜αžŠαŸ‚αž›αž˜αžΆαž“αž‚αž»αžŽαž—αžΆαž–αžαŸ’αž–αžŸαŸ‹ αž“αž·αž„αž˜αžΆαž“αžŸαŸ„αž—αŸαžŽαž—αžΆαž–αžŸαŸ’αžšαžŸαŸ‹αžŸαŸ’αž’αžΆαžαž”αŸ†αž•αž»αž",
"hyp": "αž˜αŸ’αž…αžΆαžŸαŸ‹αž’αžΆαž•αžΆαžαžαž˜αž·αž“αžαŸ‚αž„αžαŸ‚αž‡αŸ’αžšαžΎαžŸαžšαžΎαžŸαž‚αŸ’αžšαžΏαž„αžŸαž„αŸ’αž αžΆαžšαžΉαž˜αžŠαŸ‚αž›αž˜αžΆαž“αž‚αž»αžŽαž—αžΆαž–αžαŸ’αž–αžŸαŸ‹ αž“αž·αž„αž˜αžΆαž“αžŸαŸ„αž—αŸαžŽαž—αžΆαž–αžŸαŸ’αžšαžŸαŸ‹αžŸαŸ’αž’αžΆαžαž”αŸ†αž•αž»αž"
},
{
"ref": "αžŸαŸ’αžαžΆαž”αžαŸ’αž™αž€αž˜αŸ’αž˜αž’αžΆαžŽαžΆαž“αž·αž‚αž˜αž”αžΆαžšαžΆαŸ†αž„αžŠαŸ‚αž›αž™αžΎαž„αžƒαžΎαž‰αž˜αžΆαž“αž“αŸ…αž—αŸ’αž“αŸ†αž–αŸαž‰ αž“αž·αž„αž”αžΆαžαŸ‹αžŠαŸ†αž”αž„ αž˜αžΆαž“αž›αž€αŸ’αžαžŽαŸˆαž–αž·αžŸαŸαžŸαž˜αž½αž™αž…αŸ†αž“αž½αž“αžŠαŸ‚αž›αž’αŸ’αžœαžΎαž±αŸ’αž™αžœαžΆαž›αŸαž…αž’αŸ’αž›αŸ„",
"hyp": "αžŸαŸ’αžαžΆαž”αžαŸ’αž™αž€αž˜αŸ’αž˜αž’αžΆαžŽαžΆαž“αž·αž‚αž˜αž”αžΆαžšαžΆαŸ†αž„αžŠαŸ‚αž›αž™αžΎαž„αžƒαžΎαž‰αž˜αžΆαž“αž“αŸ…αž—αŸ’αž“αŸ†αž–αŸαž‰ αž“αž·αž„αž”αžΆαžαŸ‹αžŠαŸ†αž”αž„ αž˜αžΆαž“αž›αž€αŸ’αžαžŽαŸˆαž–αž·αžŸαŸαžŸαž˜αž½αž™αž…αŸ†αž“αž½αž“αžŠαŸ‚αž›αž’αŸ’αžœαžΎαž±αŸ’αž™αžœαžΆαž›αŸαž…αž’αŸ’αž›αŸ„"
},
{
"ref": "αž–αž½αž€αž‚αŸαž€αŸαž˜αžΆαž“αž‡αŸ†αž“αžΏαž›αžΎαžœαž·αž‰αŸ’αž‰αžΆαžŽαž’αž˜αŸ’αž˜αž‡αžΆαžαž· αž“αž·αž„αž˜αžΆαž“αž–αž·αž’αžΈαž”αžΌαž‡αžΆαžŠαŸ†αžšαžΈ αž“αž·αž„αžšαž”αžšαž…αž·αž‰αŸ’αž…αžΉαž˜αžŠαŸ†αžšαžΈ αž€αžΆαžšαž’αŸ’αžœαžΎαž…αž˜αŸ’αž€αžΆαžšαž€αžΆαž αŸ’αžœαŸ αž“αž·αž„αžŠαŸ†αžŽαžΆαŸ†αž•αŸ’αžŸαŸαž„ αž€αŸαž‡αžΆαž•αŸ’αž“αŸ‚αž€αž˜αž½αž™αž“αŸƒαž‡αžΈαžœαž—αžΆαž–αžšαž”αžŸαŸ‹αž–αž½αž€αž‚αŸαžŠαŸ‚αžš",
"hyp": "αž–αž½αž€αž‚αŸαž€αŸαž˜αžΆαž“αž‡αŸ†αž“αžΏαž›αžΎαžœαž·αž‰αŸ’αž‰αžΆαžŽαž’αž˜αŸ’αž˜αž‡αžΆαžαž· αž“αž·αž„αž˜αžΆαž“αž–αž·αž’αžΈαž”αžΌαž‡αžΆαžαž˜αžšαžΈ αž“αž·αž„αžšαž”αžšαž…αž·αž‰αŸ’αž…αžΉαž˜αžαŸ†αžšαžΈ αž€αžΆαžšαž’αŸ’αžœαžΎαž…αž˜αŸ’αž€αžΆαžšαž€αžΆαž αŸ’αžœαŸ αž“αž·αž„αžŠαŸ†αžŽαžΆαŸ†αž•αŸ’αžŸαŸαž„ αž€αŸαž‡αžΆαž•αŸ’αž“αŸ‚αž€αž˜αž½αž™αž“αŸƒαž‡αžΈαžœαž—αžΆαž–αžšαž”αžŸαŸ‹αž–αž½αž€αž‚αŸαžŠαŸ‚αžš"
}
]
},
"ddd_test": {
"cer": 0.024715486779578407,
"n": 1993,
"rtf": 0.0059981543289738416,
"examples": [
{
"ref": "αž›αŸ’αž–αŸ…αžŸαž˜αŸ’αž”αžΌαžšαž‡αžΆαžαž·αžŸαžšαžŸαŸƒ αžŠαŸ‚αž›αž‡αž½αž™αž‡αŸ†αžšαž»αž‰αž…αž›αž“αžΆαž–αŸ„αŸ‡αžœαŸ€αž“αž²αŸ’αž™αž˜αžΆαž“αž›αž€αŸ’αžαžŽαŸˆαž’αž˜αŸ’αž˜αžαžΆ αž€αžΆαžšαž–αžΆαžšαž€αžΆαžšαž‘αž›αŸ‹αž›αžΆαž˜αž€ αž“αž·αž„αžšαž€αŸ’αžŸαžΆαž”αŸ’αžšαž–αŸαž“αŸ’αž’αžšαŸ†αž›αžΆαž™αž’αžΆαž αžΆαžšαž²αŸ’αž™αž˜αžΆαž“αžŸαž»αžαž—αžΆαž–αž›αŸ’αž’",
"hyp": "αž›αŸ’αž–αŸ…αžŸαž˜αŸ’αž”αžΌαžšαž‡αžΆαžαž·αžŸαžšαžŸαŸƒ αžŠαŸ‚αž›αž‡αž½αž™αž‡αŸ†αžšαž»αž‰αž…αž›αž“αžΆαž–αŸ‡αžœαŸ€αž“αž±αŸ’αž™αž˜αžΆαž“αž›αž€αŸ’αžαžŽαŸˆαž’αž˜αŸ’αž˜αžαžΆ αž€αžΆαžšαž–αžΆαžšαž€αžΆαžšαžαŸ‹αž›αžΆαž˜αž αž“αž·αž„αžšαž€αŸ’αžŸαžΆαž”αŸ’αžšαž–αŸαž“αŸ’αž’αžšαŸ†αž›αžΆαž™αž’αžΆαž αžΆαžšαž±αŸ’αž™αž˜αžΆαž“αžŸαž»αžαž—αžΆαž–αž›αŸ’αž’"
},
{
"ref": "αž€αžΆαžšαž”αž“αŸ’αžαŸ‚αž˜ αžαŸ’αž‘αž·αŸ‡αžŠαžΌαž„αž‡αž½αž™αž²αŸ’αž™αžŸαž„αŸ‹αžαŸ’αž™αžΆαž˜αžΆαž“αžŸαž—αžΆαž–αž‘αž“αŸ‹αž›αŸ’αž˜αžΎαž™ αž αžΎαž™αžŸαŸ’αž›αžΉαž€αžαžΎαž™αž€αŸαž‡αž½αž™αž”αž“αŸ’αžαŸ‚αž˜αž“αžΌαžœαž€αŸ’αž›αž·αž“αž€αŸ’αžšαž’αžΌαž”αž”αŸ’αžšαž αžΎαžšαž•αž„αžŠαŸ‚αžš",
"hyp": "αž€αžΆαžšαž”αž“αŸ’αžαŸ‚αž˜αžαŸ’αž‘αž·αŸ‡αžŠαžΌαž„αž‡αž½αž™αž±αŸ’αž™αžαŸ’αžŸαž„αŸ‹αžαŸ’αž™αžΆαžšαž˜αžΆαž“αž—αžΆαž–αž‘αž“αŸ‹αž›αŸ’αž˜αžΎαž™ αž αžΎαž™αžŸαŸ’αž›αžΉαž€αžαžΎαž™αž€αŸαž‡αž½αž™αž”αž“αŸ’αžαŸ‚αž˜αž“αžΌαžœαž€αŸ’αž›αž·αž“αž€αŸ’αžšαž’αžΌαž”αž”αŸ’αžšαž αžΎαžšαž•αž„αžŠαŸ‚αžš"
},
{
"ref": "αž‘αžΉαž€αž›αŸ’αž–αŸ…αž”αžΆαž“αž€αŸ’αž›αžΆαž™αž‡αžΆαž—αŸαžŸαž‡αŸ’αž‡αŸˆαžŠαŸαž–αŸαž‰αž“αž·αž™αž˜αž˜αž½αž™ αž€αŸ’αž“αž»αž„αž…αŸ†αžŽαŸ„αž˜αž’αŸ’αž“αž€αžŠαŸ‚αž›αž™αž€αž…αž·αžαŸ’αžαž‘αž»αž€αžŠαžΆαž€αŸ‹αž›αžΎαžŸαž»αžαž—αžΆαž– αž“αž·αž„αžŸαŸ’αžœαŸ‚αž„αžšαž€αž—αŸαžŸαž‡αŸ’αž‡αŸˆαžŠαŸ‚αž›αž˜αžΆαž“αž‚αž»αžŽαž”αŸ’αžšαž™αŸ„αž‡αž“αŸαžαŸ’αž–αžŸαŸ‹",
"hyp": "αž‘αžΉαž€αž›αŸ’αž–αŸ…αž”αžΆαž“αž€αŸ’αž›αžΆαž™αž‡αžΆαž—αŸαžŸαž‡αŸ’αž‡αŸˆαžŠαŸαž–αŸαž‰αž“αž·αž™αž˜αž˜αž½αž™αž€αŸ’αž“αž»αž„αž…αŸ†αžŽαŸ„αž˜αž’αŸ’αž“αž€αžŠαŸ‚αž›αž™αž€αž…αž·αžαŸ’αžαž‘αž»αž€αžŠαžΆαž€αŸ‹αž›αžΎαžŸαž»αžαž—αžΆαž– αž“αž·αž„αžŸαŸ’αžœαŸ‚αž„αžšαž€αž—αŸαžŸαž‡αŸ’αž‡αŸˆαžŠαŸ‚αž›αž˜αžΆαž“αž‚αž»αžŽαž”αŸ’αžšαž™αŸ„αž‡αž“αŸαžαŸ’αž–αžŸαŸ‹"
}
]
},
"fleurs_km_dev": {
"cer": 0.08833827748869329,
"n": 326,
"rtf": 0.006745861517385598,
"examples": [
{
"ref": "αž€αžΆαž›αžŠαžΎαž˜αž‘αžΎαž™αžŸαž˜αŸ’αž›αŸ€αž€αž”αŸ†αž–αžΆαž€αŸ‹αž‚αžΊαž’αŸ’αž„αž“αŸ‹αž‘αž‘αž½αž›αž₯αž‘αŸ’αžœαž·αž–αž›αžŠαŸ„αž™αžœαž”αŸ’αž”αž’αž˜αŸŒyzantine αž“αŸ…αž—αžΆαž‚αžαžΆαž„αž€αžΎαž",
"hyp": "αž€αžΆαžšαžŠαžΎαž˜αž‘αžΎαž™αžŸαž˜αŸ’αž›αŸ€αž„αž”αŸ†αž–αžΆαž€αŸ‹αž‚αžΊαž’αŸ’αž„αž“αŸ‹αž‘αž‘αž½αž›αž₯αž‘αŸ’αž’αž·αž–αž›αžŠαŸ„αž™αžœαž”αŸ’αž”αž’αž˜αŸŒαž”αŸƒαžŸαžΆαž“αžΈαž“ αž“αŸ…αž—αžΆαž‚αžαžΆαž„αž€αžΎαž"
},
{
"ref": "αžœαžΆαž˜αžΆαž“αž•αž›αž”αŸ‰αŸ‡αž–αžΆαž›αŸ‹αžŸαž„αŸ’αž‚αž˜αž“αž·αž„αž“αž™αŸ„αž”αžΆαž™αž‡αžΆαž…αŸ’αžšαžΎαž“αžŠαžΌαž…αž‡αžΆαž€αžΆαžšαž”αŸ’αžšαžΎαž”αŸ’αžšαžΆαžŸαŸ‹αž”αŸ’αžšαž–αŸαž“αŸ’αž’αž˜αŸ‰αžΆαž‘αŸ’αžšαžΈαž€ αž€αžΆαžšαž•αŸ’αž›αžΆαžŸαŸ‹αž”αŸ’αžαžΌαžšαž–αžΈαžšαž”αž”αž•αŸ’αžαžΆαž…αŸ‹αž€αžΆαžšαž‘αŸ…αž‡αžΆαžšαž”αž”αžŸαžΆαž’αžΆαžšαžŽαŸˆ αž‡αžΆαžαž·αž“αž·αž™αž˜ αž“αž·αž„αž‡αŸ†αž“αžΏαž‡αžΆαž€αŸ‹αž›αžΆαž€αŸ‹αžαžΆαž”αŸ’αžšαž‘αŸαžŸαž“αŸαŸ‡αž‡αžΆαžšαž”αžŸαŸ‹αž”αŸ’αžšαž‡αžΆαžšαžΆαžŸαŸ’αžšαŸ’αžαž˜αž·αž“αž˜αŸ‚αž“αž‡αžΆαžšαž”αžŸαŸ‹αž’αŸ’αž“αž€αž‚αŸ’αžšαž”αŸ‹αž‚αŸ’αžšαž„αž˜αŸ’αž“αžΆαž€αŸ‹αž“αŸ„αŸ‡αž‘αŸ",
"hyp": "αžœαžΆαž˜αžΆαž“αž•αž›αž”αŸ‰αŸ‡αž–αžΆαž›αŸ‹αžŸαž„αŸ’αž‚αž˜αž“αž·αž„αž“αž™αŸ„αž”αžΆαž™αž‡αžΆαž…αŸ’αžšαžΎαž“ αžŠαžΌαž…αž‡αžΆαž€αžΆαžšαž”αŸ’αžšαžΎαž”αŸ’αžšαžΆαžŸαŸ‹αž”αŸ’αžšαž–αŸαž“αŸ’αž’αž˜αŸ‰αžΆαž‘αŸ’αžšαžΈαžŸ αž€αžΆαžšαž•αŸ’αž›αžΆαžŸαŸ‹αž”αŸ’αžαžΌαžšαž§αž–αžΆαžšαž”αžŸαŸ‹αž”αŸ’αž›αž·αŸ‹αž€αžΆαžšαžαŸ‚αž‡αžΆαžšαž”αžŸαŸ‹αžŸαžΆαž’αžΆαžšαžŽαŸˆαž‡αžΆαžαž·αž“αž·αž™αž˜ αž“αž·αž„αž‡αŸ†αž“αžΏαž‡αžΆαž€αŸ‹αž›αžΆαž€αŸ‹αžαžΆαž”αŸ’αžšαž‘αŸαžŸαž“αŸαŸ‡αž‡αžΆαžšαž”αžŸαŸ‹αž”αŸ’αžšαž‡αžΆαžšαžΆαžŸαŸ’αžαŸ’αžš αž˜αž·αž“αž˜αŸ‚αž“αž‡αžΆαžšαž”αžŸαŸ‹αž’αŸ’αž“αž€αž‚αŸ’αžšαž”αŸ‹αž‚αŸ’αžšαž„αž˜αŸ’αž“αžΆαž€αŸ‹αž“αŸ„αŸ‡αž‘αŸ"
},
{
"ref": "αž—αžΆαž‚αžαžΆαž„αž‡αžΎαž„ αž¬αžαŸ†αž”αž“αŸ‹ entinel ange αž˜αžΆαž“αž—αŸ’αž“αŸ†αžαŸ’αž–αžŸαŸ‹αž‡αžΆαž„αž‚αŸαž“αŸƒαž‘αŸ’αžœαžΈαž”αž’αž„αŸ‹αžαžΆαž€αŸ‹αž‘αž·αž€ αž‚αžΊαž—αŸ’αž“αŸ† inson assif αžŠαŸ‚αž›αž˜αžΆαž“αž€αŸ†αž–αžΌαž›αž€αž˜αŸ’αž–αžŸαŸ‹ 4892 αž˜αŸ‰αŸ‚αžαŸ’αžš αž—αŸ’αž“αŸ† ount inson",
"hyp": "αž—αžΆαž‚αžαžΆαž„αž‡αžΎαž„ αž¬αžαŸ†αž”αž“αŸ‹ entinel ring αž˜αžΆαž“αž—αŸ’αž“αŸ†αžαŸ’αž–αžŸαŸ‹αž‡αžΆαž„αž‚αŸαž“αŸƒαž‘αŸ’αžœαžΈαž”αž’αž„αŸ‹αžαžΆαž€αŸ‹αž‘αž·αž€ αž‚αžΊαž—αŸ’αž“αŸ† iαž“son αŸ‰as៊ីf αžŠαŸ‚αž›αž˜αžΆαž“αž€αŸ†αž–αžΌαž›αž€αž˜αŸ’αž–αžŸαŸ‹ 4892 αž˜αŸ‰αŸ‚αžαŸ’αžš αž—αŸ’αž“αŸ†αŸ‰αžΆn iαž“sαž·αž“"
}
]
},
"fleurs_km_test": {
"cer": 0.09440590405904059,
"n": 771,
"rtf": 0.0066833982391532555,
"examples": [
{
"ref": "αž˜αž“αž»αžŸαŸ’αžŸαž˜αž½αž™αž…αŸ†αž“αž½αž“αž‚αž·αžαžαžΆ αž‚αžΆαžαŸ‹αž“αž·αž™αžΆαž™αžαŸ’αžšαžΌαžœ αž”αŸ‰αž»αž“αŸ’αžαŸ‚αž˜αž“αž»αžŸαŸ’αžŸαž‡αžΆαž…αŸ’αžšαžΎαž“αž‡αžΏαžαžΆαž‚αžΆαžαŸ‹αž“αž·αž™αžΆαž™αž˜αž·αž“αžαŸ’αžšαžΌαžœαž‘αŸ αž‚αžΆαžαŸ‹αžαžΆαž”αŸ’αžšαž–αŸαž“αŸ’αž’αž–αŸ’αžšαŸ‡αž’αžΆαž‘αž·αžαŸ’αž™ αž’αŸ’αžœαžΎαžŠαŸ†αžŽαžΎαžšαžœαž·αž›αž‡αž»αŸ†αžœαž·αž‰αž•αŸ‚αž“αžŠαžΈ αžšαž½αž˜αž‘αžΆαŸ†αž„αž–αŸ’αžšαŸ‡αž’αžΆαž‘αž·αžαŸ’αž™ αž“αž·αž„αž•αŸ’αž€αžΆαž™αžŠαž‘αŸƒαž‘αŸ€αžαž•αž„αžŠαŸ‚αžš",
"hyp": "αž˜αž“αž»αžŸαŸ’αžŸαž˜αž½αž™αž…αŸ†αž“αž½αž“αž‚αž·αžαžαžΆ αž‚αžŒαž·αž™αžΆαž™αžαŸ’αžšαžΌαžœ αž”αŸ‰αž»αž“αŸ’αžαŸ‚αž˜αž“αž»αžŸαŸ’αžŸαž‡αžΆαž…αŸ’αžšαžΎαž“αž‡αžΏαžαžΆαž‚αžŒαž·αž™αžΆαž™αž˜αž·αž“αžαŸ’αžšαžΌαžœαž‘αŸ αž‚αžΆαžαŸ‹αžαžΆαž”αŸ’αžšαž–αŸαž“αŸ’αž’αž–αŸ’αžšαŸ‡αž’αžΆαž‘αž·αžαŸ’αž™αž’αŸ’αžœαžΎαžŠαŸ†αžŽαžΎαžšαžœαž·αž›αž‡αž»αŸ†αžœαž·αž‰αž•αŸ‚αž“αžŠαžΈ αžšαž½αž˜αž‘αžΆαŸ†αž„αž–αŸ’αžšαŸ‡αž’αžΆαž‘αž·αžαŸ’αž™αž“αžΉαž„αž•αŸ’αž€αžΆαž™αžŠαŸαž’αž‘αŸƒαž‘αŸ€αžαž•αž„αžŠαŸ‚αžš"
},
{
"ref": "αžŸαŸ†αžŽαžΎαž˜αž“αŸ…αž›αžΎαžŠαŸƒαžšαž”αžŸαŸ‹αž’αŸ’αž“αž€αž“αžΉαž„αž˜αžΆαž“αž”αŸ’αžšαžαž·αž€αž˜αŸ’αž˜αž‡αžΆαž˜αž½αž™αž“αžΉαž„αžŸαŸ’αžšαž‘αžΆαž”αŸ‹αžαžΆαž„αž€αŸ’αžšαŸ… αžŠαŸ‚αž›αž“αžΉαž„αž’αžΆαž…αžŠαžΉαž„αž‡αžΆαž˜αž»αž“αž”αžΆαž“ αžšαž½αž…αž”αž„αŸ’αž€αžΎαžαž‡αžΆαžŸαŸ†αž”αž€αž˜αž½αž™αž”αŸ’αžšαž—αŸαž‘",
"hyp": "αžŸαŸ†αžŽαžΎαž˜ αž“αŸ…αž›αžΎαžŠαŸƒαžšαž”αžŸαŸ‹αž’αŸ’αž“αž€αž“αžΉαž„αž˜αžΆαž“αž”αŸ’αžšαžΆαžαž€αž˜αŸ’αž˜αž‡αžΆαž˜αž½αž™αž“αžΉαž„αžŸαŸ’αžšαž‘αžΆαž”αŸ‹αžαžΆαž„αž€αŸ’αžšαŸ…αžŠαŸ‚αž›αž“αžΉαž„αž’αžΆαž…αžŠαžΉαž„αž‡αžΆαž˜αž»αž“αž”αžΆαž“ αžšαž½αž…αž”αž„αŸ’αž€αžΎαžαž‡αžΆαžŸαŸ†αž”αž€αž˜αž½αž™αž”αŸ’αžšαž—αŸαž‘"
},
{
"ref": "αž‘αžΈαž€αŸ’αžšαž»αž„αž“αŸαŸ‡αž€αŸαž‡αžΆαž˜αžΌαž›αžŠαŸ’αž‹αžΆαž“αžŠαžΎαž˜αŸ’αž”αžΈαž‘αžΎαž„αž—αŸ’αž“αŸ†αž—αŸ’αž›αžΎαž„ yiragongo αžšαž½αž˜αž‡αžΆαž˜αž½αž™αž“αžΉαž„αž€αžΆαžšαžαžΆαž˜ountain orilla αžŠαŸ‚αž›αž˜αžΆαž“αžαž˜αŸ’αž›αŸƒαžαŸ„αž€αž”αŸ†αž•αž»αžαž“αŸ…αž’αžΆαž αŸ’αžœαŸ’αžšαž·αž€",
"hyp": "αž‘αžΈαž€αŸ’αžšαž»αž„αž“αŸαŸ‡αž€αŸαž‡αžΆαž˜αžΌαž›αžŠαŸ’αž‹αžΆαž“αžŠαžΎαž˜αŸ’αž”αžΈαž‘αžΎαž„αž—αŸ’αž“αŸ†αž—αŸ’αž›αžΎαž„ αž“αžΈαžšαŸ‰αžΆαž€oαž“αž€αžΌ αžšαž½αž˜αž‡αžΆαž˜αž½αž™αž“αžΉαž„αž€αžΆαžšαžαžΆ ountan αž€αžΌrីទអ αžŠαŸ‚αž›αž˜αžΆαž“αžαž˜αŸ’αž›αŸƒαžαž”αž”αŸ†αž•αž»αžαž“αŸ…αž’αžΆαž αŸ’αžœαŸ’αžœαž·αž€"
}
]
},
"fleurs_en_dev": {
"cer": 0.05019447589975875,
"n": 394,
"wer": 0.1367542352074638,
"rtf": 0.00638872020448837,
"examples": [
{
"ref": "when you call someone who is thousands of miles away you are using a satellite",
"hyp": "when you call someone who has thousands of miles away you are using a satellite"
},
{
"ref": "now widely available throughout the archipelago javanese cuisine features an array of simply seasoned dishes the predominant flavorings the javanese favor being peanuts chillies sugar especially javanese coconut sugar and various aromatic spices",
"hyp": "now widely available throughout the archipelago japanese quisines features an array of simply season dishes the predominant flavorings the japanese favor being pinat chiles sugar especially japanese cocoanut sugar and various aromatic spices"
},
{
"ref": "the un also hopes to finalize a fund to help countries affected by global warming to cope with the impacts",
"hyp": "the un also hopes to finalize a fund to help countries affected by global warming to cope with the impacts"
}
]
},
"fleurs_en_test": {
"cer": 0.043481360039917316,
"n": 647,
"wer": 0.11942275515895148,
"rtf": 0.0062595013694743485,
"examples": [
{
"ref": "however due to the slow communication channels styles in the west could lag behind by 25 to 30 year",
"hyp": "however due to the slow communication channels styles in the west could lag behind by 25to 30 years"
},
{
"ref": "all nouns alongside the word sie for you always begin with a capital letter even in the middle of a sentence",
"hyp": "all nowns alongside the world say for you always begin with a capital letter even in the middle of a sentence"
},
{
"ref": "to the north and within easy reach is the romantic and fascinating town of sintra and which was made famous to foreigners after a glowing account of its splendours recorded by lord byron",
"hyp": "to the north and within easy reach is the romantic and fascinating town of cintra and which was made famous to foreigners after a glowing account of its splenduress recorded by lord byron"
}
]
},
"libri_dev_clean": {
"cer": 0.01169131218929113,
"n": 2703,
"wer": 0.03722289621705084,
"rtf": 0.006076880907681458,
"examples": [
{
"ref": "mister quilter is the apostle of the middle classes and we are glad to welcome his gospel",
"hyp": "mister quilter is the apostle of the middle classes and we are glad to welcome his gospel"
},
{
"ref": "nor is mister quilter's manner less interesting than his matter",
"hyp": "nor is mister quilter's manner less interesting than his matter"
},
{
"ref": "he tells us that at this festive season of the year with christmas and roast beef looming before us similes drawn from eating and its results occur most readily to the mind",
"hyp": "he tells us that at this festive season of the year with christmas and rose beef looming before us similees drawn from eating and its results occur most readily to the mind"
}
]
},
"libri_test_clean": {
"cer": 0.011085009543385699,
"n": 2620,
"wer": 0.03610012172854534,
"rtf": 0.006125383591282291,
"examples": [
{
"ref": "he hoped there would be stew for dinner turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce",
"hyp": "he hoped there would be stew for dinner turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flower fattened sauce"
},
{
"ref": "stuff it into you his belly counselled him",
"hyp": "stuff it into you his belly counselled him"
},
{
"ref": "after early nightfall the yellow lamps would light up here and there the squalid quarter of the brothels",
"hyp": "after early nightfall the yellow lamps would light up here and there the squalid quarter of the broffles"
}
]
},
"libri_test_other": {
"cer": 0.030955344430813866,
"n": 2939,
"wer": 0.082093116558088,
"rtf": 0.006014662979128097,
"examples": [
{
"ref": "there's iron they say in all our blood and a grain or two perhaps is good but his he makes me harshly feel has got a little too much of steel anon",
"hyp": "there's iron they say in all our blood and a grain or two perhaps is good but his he makes me harshly feel has got a little too much of steel anon"
},
{
"ref": "margaret said mister hale as he returned from showing his guest downstairs i could not help watching your face with some anxiety when mister thornton made his confession of having been a shop boy",
"hyp": "margaret said mister hell as he returned from showing his guest downstairs i could not help watching your face with some anxiety when mister thornton made his confession of having been a shop boy"
},
{
"ref": "you don't mean that you thought me so silly",
"hyp": "you don't mean that you thought me so silly"
}
]
}
}
}