Upload test_results/test_summary_20260310_202532.txt with huggingface_hub
Browse files
test_results/test_summary_20260310_202532.txt
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
============================================================
|
| 2 |
+
ИТОГОВЫЙ ОТЧЁТ ПО ТЕСТИРОВАНИЮ ТОКЕНИЗАТОРОВ
|
| 3 |
+
Дата: 2026-03-10 20:25:32.743689
|
| 4 |
+
Тестовый корпус: 10000 документов, 19479650 символов
|
| 5 |
+
============================================================
|
| 6 |
+
|
| 7 |
+
ЛУЧШИЕ МОДЕЛИ ПО КОМПОЗИТНОЙ ОЦЕНКЕ:
|
| 8 |
+
4. bbpe_fixed_50k (BPE_fixed)
|
| 9 |
+
unknown_rate=0.0000, compression=5.17, word_coverage=1.0000, speed=315922
|
| 10 |
+
2. wp_50k (WordPiece)
|
| 11 |
+
unknown_rate=0.0000, compression=4.67, word_coverage=1.0000, speed=378751
|
| 12 |
+
3. bpe_50k (BPE)
|
| 13 |
+
unknown_rate=0.0000, compression=4.60, word_coverage=1.0000, speed=247421
|
| 14 |
+
7. bpe_50k_freq5 (BPE)
|
| 15 |
+
unknown_rate=0.0000, compression=4.60, word_coverage=1.0000, speed=226591
|
| 16 |
+
5. bbpe_50k (BBPE)
|
| 17 |
+
unknown_rate=0.0000, compression=4.60, word_coverage=1.0000, speed=227322
|