Update README.md
Browse files
README.md
CHANGED
|
@@ -64,8 +64,9 @@ The vocabulary size was chosen to balance memory efficiency with language covera
|
|
| 64 |
### Vocabulary Statistics
|
| 65 |
|
| 66 |
- **Total Tokens:** 50,000
|
| 67 |
-
- **Average Token Length:**
|
| 68 |
-
- **Max Token Length:**
|
|
|
|
| 69 |
|
| 70 |
## Training and Configuration Details
|
| 71 |
|
|
|
|
| 64 |
### Vocabulary Statistics
|
| 65 |
|
| 66 |
- **Total Tokens:** 50,000
|
| 67 |
+
- **Average Token Length:** 6.46
|
| 68 |
+
- **Max Token Length:** 16
|
| 69 |
+
- **Min Token Length:** 1
|
| 70 |
|
| 71 |
## Training and Configuration Details
|
| 72 |
|