Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -114,6 +114,10 @@ We optimized the tokenizer specifically for Uzbek, achieving significantly bette
|
|
| 114 |
|
| 115 |
> **Fertility Rate**: Average number of tokens per word. Lower is better for efficiency.
|
| 116 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 117 |
### What This Means
|
| 118 |
|
| 119 |
- **22.5% fewer tokens** needed to represent Uzbek text
|
|
|
|
| 114 |
|
| 115 |
> **Fertility Rate**: Average number of tokens per word. Lower is better for efficiency.
|
| 116 |
|
| 117 |
+
<div align="center">
|
| 118 |
+
<img src="assets/fertility_comparison_chart.png" alt="Tokenizer Fertility Rate Comparison" width="700"/>
|
| 119 |
+
</div>
|
| 120 |
+
|
| 121 |
### What This Means
|
| 122 |
|
| 123 |
- **22.5% fewer tokens** needed to represent Uzbek text
|