MillID leaderboard against GlotLID and fastText
#1
by byamasupatrick - opened
- README.md +343 -362
- leaderboard_overall.csv +36 -51
- leaderboard_test-common.csv +214 -214
- leaderboard_test.csv +0 -0
- leaderboard_udhr.csv +359 -359
README.md
CHANGED
|
@@ -1,362 +1,343 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
--
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
[fastText language identifier](https://huggingface.co/facebook/fasttext-language-identification) (NLLB),
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
-
|
| 19 |
-
|
| 20 |
-
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
|
| 31 |
-
|
|
| 32 |
-
| `
|
| 33 |
-
| `
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
|
| 45 |
-
|--
|
| 46 |
-
| [
|
| 47 |
-
| [
|
| 48 |
-
| [
|
| 49 |
-
| [
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
|
| 62 |
-
|--
|
| 63 |
-
| [
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
|
| 70 |
-
|
|
| 71 |
-
| [
|
| 72 |
-
| [
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
|
| 84 |
-
|
|
| 85 |
-
| [MillID
|
| 86 |
-
| [
|
| 87 |
-
| [MillID
|
| 88 |
-
| [
|
| 89 |
-
| [
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
|
| 98 |
-
|
|
| 99 |
-
| [
|
| 100 |
-
| [
|
| 101 |
-
| [MillID
|
| 102 |
-
| [MillID medium](https://huggingface.co/baobabtech/millid-medium) | 115.0M | 0.
|
| 103 |
-
| [MillID
|
| 104 |
-
| [
|
| 105 |
-
| [fastText,
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
|
| 114 |
-
|
|
| 115 |
-
|
|
| 116 |
-
|
|
| 117 |
-
|
|
| 118 |
-
|
|
| 119 |
-
|
|
| 120 |
-
|
|
| 121 |
-
|
|
| 122 |
-
|
|
| 123 |
-
|
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
|
|
| 132 |
-
|
|
| 133 |
-
| `
|
| 134 |
-
| `
|
| 135 |
-
| `
|
| 136 |
-
| `
|
| 137 |
-
| `
|
| 138 |
-
| `
|
| 139 |
-
| `
|
| 140 |
-
| `
|
| 141 |
-
| `
|
| 142 |
-
| `
|
| 143 |
-
| `
|
| 144 |
-
| `
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
|
| 149 |
-
|
|
| 150 |
-
| `
|
| 151 |
-
| `
|
| 152 |
-
| `
|
| 153 |
-
| `
|
| 154 |
-
| `
|
| 155 |
-
| `
|
| 156 |
-
| `
|
| 157 |
-
| `
|
| 158 |
-
| `
|
| 159 |
-
| `
|
| 160 |
-
| `
|
| 161 |
-
| `
|
| 162 |
-
| `
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
|
| 169 |
-
|
|
| 170 |
-
| `
|
| 171 |
-
| `
|
| 172 |
-
| `
|
| 173 |
-
| `
|
| 174 |
-
| `
|
| 175 |
-
| `
|
| 176 |
-
| `
|
| 177 |
-
| `
|
| 178 |
-
| `
|
| 179 |
-
| `
|
| 180 |
-
| `
|
| 181 |
-
| `
|
| 182 |
-
| `
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
|
| 189 |
-
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
|
|
| 193 |
-
|
|
| 194 |
-
| `
|
| 195 |
-
| `
|
| 196 |
-
| `
|
| 197 |
-
| `
|
| 198 |
-
| `
|
| 199 |
-
| `
|
| 200 |
-
| `
|
| 201 |
-
| `
|
| 202 |
-
| `
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
|
| 210 |
-
|
|
| 211 |
-
|
|
| 212 |
-
| `
|
| 213 |
-
| `
|
| 214 |
-
| `
|
| 215 |
-
| `
|
| 216 |
-
| `
|
| 217 |
-
| `
|
| 218 |
-
| `
|
| 219 |
-
| `
|
| 220 |
-
| `
|
| 221 |
-
| `
|
| 222 |
-
| `
|
| 223 |
-
| `
|
| 224 |
-
|
| 225 |
-
|
| 226 |
-
|
| 227 |
-
|
|
| 228 |
-
|
|
| 229 |
-
| `
|
| 230 |
-
| `
|
| 231 |
-
| `
|
| 232 |
-
| `
|
| 233 |
-
| `
|
| 234 |
-
| `
|
| 235 |
-
| `
|
| 236 |
-
| `
|
| 237 |
-
| `
|
| 238 |
-
| `
|
| 239 |
-
| `
|
| 240 |
-
| `
|
| 241 |
-
| `
|
| 242 |
-
|
| 243 |
-
|
| 244 |
-
|
| 245 |
-
|
| 246 |
-
|
| 247 |
-
|
|
| 248 |
-
|
|
| 249 |
-
| `
|
| 250 |
-
| `
|
| 251 |
-
| `
|
| 252 |
-
| `
|
| 253 |
-
| `
|
| 254 |
-
| `
|
| 255 |
-
| `
|
| 256 |
-
| `
|
| 257 |
-
| `
|
| 258 |
-
| `
|
| 259 |
-
| `
|
| 260 |
-
| `
|
| 261 |
-
| `
|
| 262 |
-
|
| 263 |
-
|
| 264 |
-
|
| 265 |
-
|
| 266 |
-
|
| 267 |
-
|
| 268 |
-
|
| 269 |
-
|
| 270 |
-
|
| 271 |
-
|
|
| 272 |
-
|
|
| 273 |
-
| `
|
| 274 |
-
| `
|
| 275 |
-
| `
|
| 276 |
-
| `
|
| 277 |
-
| `
|
| 278 |
-
| `
|
| 279 |
-
| `
|
| 280 |
-
| `
|
| 281 |
-
| `
|
| 282 |
-
|
| 283 |
-
|
| 284 |
-
|
| 285 |
-
|
| 286 |
-
|
| 287 |
-
|
| 288 |
-
|
| 289 |
-
|
|
| 290 |
-
|
|
| 291 |
-
| `
|
| 292 |
-
| `
|
| 293 |
-
| `
|
| 294 |
-
| `
|
| 295 |
-
| `
|
| 296 |
-
| `
|
| 297 |
-
| `
|
| 298 |
-
| `
|
| 299 |
-
| `
|
| 300 |
-
| `
|
| 301 |
-
| `
|
| 302 |
-
| `
|
| 303 |
-
|
| 304 |
-
|
| 305 |
-
|
| 306 |
-
|
|
| 307 |
-
|
|
| 308 |
-
| `
|
| 309 |
-
| `
|
| 310 |
-
| `
|
| 311 |
-
| `
|
| 312 |
-
| `
|
| 313 |
-
| `
|
| 314 |
-
| `
|
| 315 |
-
| `
|
| 316 |
-
| `
|
| 317 |
-
| `
|
| 318 |
-
| `
|
| 319 |
-
| `
|
| 320 |
-
| `
|
| 321 |
-
|
| 322 |
-
|
| 323 |
-
|
| 324 |
-
|
| 325 |
-
|
| 326 |
-
|
|
| 327 |
-
|
|
| 328 |
-
| `
|
| 329 |
-
| `
|
| 330 |
-
| `
|
| 331 |
-
| `
|
| 332 |
-
| `
|
| 333 |
-
| `
|
| 334 |
-
| `
|
| 335 |
-
| `
|
| 336 |
-
| `
|
| 337 |
-
| `
|
| 338 |
-
| `
|
| 339 |
-
| `
|
| 340 |
-
| `
|
| 341 |
-
|
| 342 |
-
|
| 343 |
-
|
| 344 |
-
| Label | Language | Passages | GlotLID | GlotLID restricted | fastText | fastText restricted | MillID tiny | MillID medium | MillID large | MillID tiny v1.5 | MillID medium v1.5 | MillID large v1.5 |
|
| 345 |
-
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
| 346 |
-
| `dyu_Latn` | Dyula | 58 | 0.434 | 0.434 | 0.015 | 0.015 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
| 347 |
-
| `gyr_Latn` | Guarayu | 49 | 0.990 | 0.990 | 0.000 | 0.000 | 0.580 | 0.339 | 0.182 | 0.537 | 0.185 | 0.393 |
|
| 348 |
-
| `ell_Grek` | Greek | 115 | 1.000 | 1.000 | 1.000 | 1.000 | 0.670 | 0.655 | 0.670 | 0.670 | 0.670 | 0.670 |
|
| 349 |
-
| `cpu_Latn` | Pichis Ashéninka | 60 | 0.974 | 0.974 | 0.000 | 0.000 | 0.659 | 0.571 | 0.667 | 0.729 | 0.442 | 0.571 |
|
| 350 |
-
| `hin_Deva` | Hindi | 59 | 0.957 | 0.957 | 0.975 | 0.975 | 0.644 | 0.607 | 0.450 | 0.723 | 0.710 | 0.727 |
|
| 351 |
-
| `cbu_Latn` | Candoshi-Shapra | 57 | 0.882 | 0.874 | 0.000 | 0.000 | 0.667 | 0.371 | 0.185 | 0.500 | 0.411 | 0.557 |
|
| 352 |
-
| `qvn_Latn` | North Junín Quechua | 57 | 0.460 | 0.460 | 0.000 | 0.000 | 0.261 | 0.219 | 0.095 | 0.219 | 0.068 | 0.034 |
|
| 353 |
-
| `cni_Latn` | Asháninka | 52 | 0.963 | 0.963 | 0.000 | 0.000 | 0.750 | 0.708 | 0.748 | 0.773 | 0.684 | 0.717 |
|
| 354 |
-
| `koi_Cyrl` | Komi-Permyak | 57 | 1.000 | 1.000 | 0.000 | 0.000 | 0.719 | 0.837 | 0.849 | 0.837 | 0.849 | 0.849 |
|
| 355 |
-
| `hrv_Latn` | Croatian | 58 | 0.727 | 0.727 | 0.474 | 0.474 | 0.497 | 0.537 | 0.583 | 0.500 | 0.521 | 0.559 |
|
| 356 |
-
| `qvm_Latn` | Margos-Yarowilca-Lauricocha Quechua | 60 | 0.500 | 0.500 | 0.000 | 0.000 | 0.125 | 0.094 | 0.182 | 0.356 | 0.033 | 0.356 |
|
| 357 |
-
| `ind_Latn` | Indonesian | 60 | 0.903 | 0.903 | 0.565 | 0.565 | 0.614 | 0.674 | 0.644 | 0.768 | 0.742 | 0.645 |
|
| 358 |
-
| `rmn_Latn` | Balkan Romani | 116 | 0.857 | 0.857 | 0.000 | 0.000 | 0.721 | 0.667 | 0.735 | 0.678 | 0.682 | 0.704 |
|
| 359 |
-
| `maz_Latn` | Central Mazahua | 59 | 0.939 | 0.939 | 0.000 | 0.000 | 0.725 | 0.742 | 0.823 | 0.789 | 0.766 | 0.814 |
|
| 360 |
-
| `shp_Latn` | Shipibo-Conibo | 62 | 0.912 | 0.912 | 0.000 | 0.000 | 0.558 | 0.410 | 0.808 | 0.469 | 0.452 | 0.607 |
|
| 361 |
-
|
| 362 |
-
The models: [MillID tiny](https://huggingface.co/baobabtech/millid-tiny), [MillID medium](https://huggingface.co/baobabtech/millid-medium), [MillID large](https://huggingface.co/baobabtech/millid-large), [MillID tiny v1.5](https://huggingface.co/baobabtech/millid-tiny-v1.5), [MillID medium v1.5](https://huggingface.co/baobabtech/millid-medium-v1.5), [MillID large v1.5](https://huggingface.co/baobabtech/millid-large-v1.5).
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
pipeline_tag: text-classification
|
| 5 |
+
tags:
|
| 6 |
+
- language-identification
|
| 7 |
+
- leaderboard
|
| 8 |
+
datasets:
|
| 9 |
+
- baobabtech/millid-dataset
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# MillID leaderboard
|
| 13 |
+
|
| 14 |
+
**MillID** (MultiLingual Language IDentifier) against [GlotLID](https://huggingface.co/cis-lmu/glotlid) and Meta's [fastText language identifier](https://huggingface.co/facebook/fasttext-language-identification) (NLLB). F1 is per language, then averaged with every language weighted equally (macro).
|
| 15 |
+
|
| 16 |
+
The models: [MillID tiny v1.5](https://huggingface.co/baobabtech/millid-tiny-v1.5), [MillID medium v1](https://huggingface.co/baobabtech/millid-medium-v1), [MillID large v1.5](https://huggingface.co/baobabtech/millid-large-v1.5). Code, and running them in Python or JavaScript without a server: [baobab-tech/millid](https://github.com/baobab-tech/millid).
|
| 17 |
+
|
| 18 |
+
Every score is on the test split of the build the models were trained on. The public [baobabtech/millid-dataset](https://huggingface.co/datasets/baobabtech/millid-dataset) is that build without Save the Children's and Nal'ibali's text, which is not ours to redistribute: the same splits, and a `test` split 1.3% smaller. `udhr` is the same everywhere.
|
| 19 |
+
|
| 20 |
+
- MillID knows 1,641 labels, GlotLID over 2,000, fastText 218. Each external model is shown *restricted to MillID's labels* (its best answer among them), the like-for-like comparison; its as-is scores are below.
|
| 21 |
+
- On `test` and `udhr`, fastText is mostly scored on languages it cannot answer. `test-common` and `udhr-common` keep the passages whose language all three know (about 210 languages): the head-to-head on equal terms.
|
| 22 |
+
- Read GlotLID's scores with their bias in mind: the dataset's FinePDFs and FineWeb-2 passages were labelled by GlotLID, and GlotLID was trained on UDHR, so both favour it.
|
| 23 |
+
- fastText's labels are mapped onto MillID's spelling: Simplified and Traditional Han as `Hani`, `pes`/`prs` as `fas`, `est` as `ekk`, `tgl` as `fil`.
|
| 24 |
+
|
| 25 |
+
## Macro F1, every set
|
| 26 |
+
|
| 27 |
+
| Set | Languages | MillID tiny v1.5 | MillID medium v1 | MillID large v1.5 | GlotLID | fastText |
|
| 28 |
+
|---|---|---|---|---|---|---|
|
| 29 |
+
| `test`: every language, passages from documents never seen in training | 1621 | 0.9554 | 0.9576 | 0.9646 | **0.9755** | 0.0659 |
|
| 30 |
+
| `test-common`: the languages all three systems know | 213 | 0.9513 | 0.9527 | 0.9594 | **0.9846** | 0.9173 |
|
| 31 |
+
| `udhr`: the Declaration of Human Rights, languages MillID knows | 358 | 0.8764 | 0.8780 | 0.8812 | **0.8873** | 0.3499 |
|
| 32 |
+
| `udhr-common`: the Declaration, languages all three know | 170 | 0.9513 | 0.9480 | 0.9529 | **0.9659** | 0.9329 |
|
| 33 |
+
| `udhr-all`: the Declaration, every language in it | 429 | 0.7149 | 0.7184 | 0.7201 | **0.7245** | 0.2797 |
|
| 34 |
+
|
| 35 |
+
The best score on each set is in bold.
|
| 36 |
+
|
| 37 |
+
## test
|
| 38 |
+
|
| 39 |
+
Every language, passages from documents never seen in training.
|
| 40 |
+
|
| 41 |
+
| Model | Size | Top-1 | Top-3 | Top-5 | Macro P | Macro R | Macro F1 | Median F1 | Languages F1 ≥ 0.9 |
|
| 42 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 43 |
+
| [GlotLID, restricted to MillID's labels](https://huggingface.co/cis-lmu/glotlid) | fastText, 1.7 GB | 0.9783 | 0.9978 | 0.9985 | 0.9778 | 0.9765 | 0.9755 | 1.0000 | 1490 of 1621 |
|
| 44 |
+
| [GlotLID](https://huggingface.co/cis-lmu/glotlid) | fastText, 1.7 GB | 0.9776 | 0.9977 | 0.9984 | 0.9783 | 0.9752 | 0.9748 | 1.0000 | 1488 of 1621 |
|
| 45 |
+
| [MillID large v1.5](https://huggingface.co/baobabtech/millid-large-v1.5) | 192.6M | 0.9674 | 0.9924 | 0.9956 | 0.9655 | 0.9663 | 0.9646 | 0.9894 | 1478 of 1621 |
|
| 46 |
+
| [MillID medium v1](https://huggingface.co/baobabtech/millid-medium-v1) | 115.0M | 0.9616 | 0.9910 | 0.9945 | 0.9564 | 0.9614 | 0.9576 | 0.9841 | 1451 of 1621 |
|
| 47 |
+
| [MillID tiny v1.5](https://huggingface.co/baobabtech/millid-tiny-v1.5) | 2.9M | 0.9587 | 0.9890 | 0.9932 | 0.9550 | 0.9588 | 0.9554 | 0.9825 | 1446 of 1621 |
|
| 48 |
+
| [fastText, restricted to MillID's labels](https://huggingface.co/facebook/fasttext-language-identification) | fastText, 1.2 GB | 0.2515 | 0.2611 | 0.2625 | 0.0521 | 0.1209 | 0.0659 | 0.0000 | 26 of 1621 |
|
| 49 |
+
| [fastText (Meta, NLLB)](https://huggingface.co/facebook/fasttext-language-identification) | fastText, 1.2 GB | 0.2511 | 0.2610 | 0.2625 | 0.0520 | 0.1205 | 0.0657 | 0.0000 | 25 of 1621 |
|
| 50 |
+
|
| 51 |
+
## test-common
|
| 52 |
+
|
| 53 |
+
The languages all three systems know.
|
| 54 |
+
|
| 55 |
+
| Model | Size | Top-1 | Top-3 | Top-5 | Macro P | Macro R | Macro F1 | Median F1 | Languages F1 ≥ 0.9 |
|
| 56 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 57 |
+
| [GlotLID, restricted to MillID's labels](https://huggingface.co/cis-lmu/glotlid) | fastText, 1.7 GB | 0.9749 | 0.9927 | 0.9948 | 0.9923 | 0.9790 | 0.9846 | 0.9983 | 203 of 213 |
|
| 58 |
+
| [GlotLID](https://huggingface.co/cis-lmu/glotlid) | fastText, 1.7 GB | 0.9738 | 0.9925 | 0.9946 | 0.9924 | 0.9771 | 0.9835 | 0.9983 | 201 of 213 |
|
| 59 |
+
| [MillID large v1.5](https://huggingface.co/baobabtech/millid-large-v1.5) | 192.6M | 0.9525 | 0.9898 | 0.9938 | 0.9757 | 0.9460 | 0.9594 | 0.9791 | 193 of 213 |
|
| 60 |
+
| [MillID medium v1](https://huggingface.co/baobabtech/millid-medium-v1) | 115.0M | 0.9425 | 0.9882 | 0.9924 | 0.9732 | 0.9361 | 0.9527 | 0.9738 | 191 of 213 |
|
| 61 |
+
| [MillID tiny v1.5](https://huggingface.co/baobabtech/millid-tiny-v1.5) | 2.9M | 0.9396 | 0.9857 | 0.9909 | 0.9709 | 0.9353 | 0.9513 | 0.9726 | 188 of 213 |
|
| 62 |
+
| [fastText, restricted to MillID's labels](https://huggingface.co/facebook/fasttext-language-identification) | fastText, 1.2 GB | 0.9352 | 0.9710 | 0.9763 | 0.9354 | 0.9201 | 0.9173 | 0.9680 | 166 of 213 |
|
| 63 |
+
| [fastText (Meta, NLLB)](https://huggingface.co/facebook/fasttext-language-identification) | fastText, 1.2 GB | 0.9339 | 0.9707 | 0.9762 | 0.9357 | 0.9169 | 0.9149 | 0.9680 | 165 of 213 |
|
| 64 |
+
|
| 65 |
+
## udhr
|
| 66 |
+
|
| 67 |
+
The Declaration of Human Rights, languages MillID knows.
|
| 68 |
+
|
| 69 |
+
| Model | Size | Top-1 | Top-3 | Top-5 | Macro P | Macro R | Macro F1 | Median F1 | Languages F1 ≥ 0.9 |
|
| 70 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 71 |
+
| [GlotLID, restricted to MillID's labels](https://huggingface.co/cis-lmu/glotlid) | fastText, 1.7 GB | 0.8614 | 0.9294 | 0.9433 | 0.9258 | 0.8844 | 0.8873 | 0.9915 | 291 of 358 |
|
| 72 |
+
| [GlotLID](https://huggingface.co/cis-lmu/glotlid) | fastText, 1.7 GB | 0.8598 | 0.9280 | 0.9421 | 0.9261 | 0.8826 | 0.8865 | 0.9915 | 287 of 358 |
|
| 73 |
+
| [MillID large v1.5](https://huggingface.co/baobabtech/millid-large-v1.5) | 192.6M | 0.8579 | 0.9183 | 0.9309 | 0.9171 | 0.8825 | 0.8812 | 0.9914 | 283 of 358 |
|
| 74 |
+
| [MillID medium v1](https://huggingface.co/baobabtech/millid-medium-v1) | 115.0M | 0.8524 | 0.9194 | 0.9340 | 0.9187 | 0.8769 | 0.8780 | 0.9910 | 282 of 358 |
|
| 75 |
+
| [MillID tiny v1.5](https://huggingface.co/baobabtech/millid-tiny-v1.5) | 2.9M | 0.8492 | 0.9117 | 0.9257 | 0.9148 | 0.8736 | 0.8764 | 0.9907 | 274 of 358 |
|
| 76 |
+
| [fastText, restricted to MillID's labels](https://huggingface.co/facebook/fasttext-language-identification) | fastText, 1.2 GB | 0.4629 | 0.4857 | 0.4904 | 0.3152 | 0.4483 | 0.3499 | 0.0000 | 61 of 358 |
|
| 77 |
+
| [fastText (Meta, NLLB)](https://huggingface.co/facebook/fasttext-language-identification) | fastText, 1.2 GB | 0.4625 | 0.4857 | 0.4903 | 0.3152 | 0.4479 | 0.3497 | 0.0000 | 60 of 358 |
|
| 78 |
+
|
| 79 |
+
## udhr-common
|
| 80 |
+
|
| 81 |
+
The Declaration, languages all three know.
|
| 82 |
+
|
| 83 |
+
| Model | Size | Top-1 | Top-3 | Top-5 | Macro P | Macro R | Macro F1 | Median F1 | Languages F1 ≥ 0.9 |
|
| 84 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 85 |
+
| [GlotLID, restricted to MillID's labels](https://huggingface.co/cis-lmu/glotlid) | fastText, 1.7 GB | 0.9338 | 0.9683 | 0.9736 | 0.9888 | 0.9579 | 0.9659 | 1.0000 | 156 of 170 |
|
| 86 |
+
| [GlotLID](https://huggingface.co/cis-lmu/glotlid) | fastText, 1.7 GB | 0.9336 | 0.9678 | 0.9732 | 0.9888 | 0.9576 | 0.9657 | 1.0000 | 155 of 170 |
|
| 87 |
+
| [MillID large v1.5](https://huggingface.co/baobabtech/millid-large-v1.5) | 192.6M | 0.9168 | 0.9630 | 0.9687 | 0.9800 | 0.9436 | 0.9529 | 1.0000 | 152 of 170 |
|
| 88 |
+
| [MillID tiny v1.5](https://huggingface.co/baobabtech/millid-tiny-v1.5) | 2.9M | 0.9123 | 0.9606 | 0.9655 | 0.9794 | 0.9393 | 0.9513 | 1.0000 | 149 of 170 |
|
| 89 |
+
| [MillID medium v1](https://huggingface.co/baobabtech/millid-medium-v1) | 115.0M | 0.9121 | 0.9633 | 0.9688 | 0.9770 | 0.9382 | 0.9480 | 0.9955 | 153 of 170 |
|
| 90 |
+
| [fastText, restricted to MillID's labels](https://huggingface.co/facebook/fasttext-language-identification) | fastText, 1.2 GB | 0.9195 | 0.9647 | 0.9742 | 0.9474 | 0.9441 | 0.9329 | 1.0000 | 142 of 170 |
|
| 91 |
+
| [fastText (Meta, NLLB)](https://huggingface.co/facebook/fasttext-language-identification) | fastText, 1.2 GB | 0.9187 | 0.9647 | 0.9741 | 0.9474 | 0.9431 | 0.9323 | 1.0000 | 141 of 170 |
|
| 92 |
+
|
| 93 |
+
## udhr-all
|
| 94 |
+
|
| 95 |
+
The Declaration, every language in it.
|
| 96 |
+
|
| 97 |
+
| Model | Size | Top-1 | Top-3 | Top-5 | Macro P | Macro R | Macro F1 | Median F1 | Languages F1 ≥ 0.9 |
|
| 98 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 99 |
+
| [GlotLID](https://huggingface.co/cis-lmu/glotlid) | fastText, 1.7 GB | 0.7502 | 0.8145 | 0.8282 | 0.7695 | 0.7529 | 0.7454 | 0.9796 | 277 of 429 |
|
| 100 |
+
| [GlotLID, restricted to MillID's labels](https://huggingface.co/cis-lmu/glotlid) | fastText, 1.7 GB | 0.7378 | 0.7960 | 0.8079 | 0.7425 | 0.7381 | 0.7245 | 0.9748 | 272 of 429 |
|
| 101 |
+
| [MillID large v1.5](https://huggingface.co/baobabtech/millid-large-v1.5) | 192.6M | 0.7348 | 0.7865 | 0.7973 | 0.7392 | 0.7364 | 0.7201 | 0.9643 | 266 of 429 |
|
| 102 |
+
| [MillID medium v1](https://huggingface.co/baobabtech/millid-medium-v1) | 115.0M | 0.7301 | 0.7875 | 0.8000 | 0.7415 | 0.7318 | 0.7184 | 0.9672 | 266 of 429 |
|
| 103 |
+
| [MillID tiny v1.5](https://huggingface.co/baobabtech/millid-tiny-v1.5) | 2.9M | 0.7274 | 0.7809 | 0.7928 | 0.7340 | 0.7290 | 0.7149 | 0.9649 | 254 of 429 |
|
| 104 |
+
| [fastText, restricted to MillID's labels](https://huggingface.co/facebook/fasttext-language-identification) | fastText, 1.2 GB | 0.3964 | 0.4160 | 0.4200 | 0.2479 | 0.3741 | 0.2797 | 0.0000 | 55 of 429 |
|
| 105 |
+
| [fastText (Meta, NLLB)](https://huggingface.co/facebook/fasttext-language-identification) | fastText, 1.2 GB | 0.3961 | 0.4160 | 0.4200 | 0.2479 | 0.3737 | 0.2795 | 0.0000 | 54 of 429 |
|
| 106 |
+
|
| 107 |
+
## Per language, test
|
| 108 |
+
|
| 109 |
+
Against GlotLID restricted, the best MillID is better on **165** languages and worse on **498**; against fastText restricted, the best MillID is better on **1616** languages and worse on **0** (F1 differing by more than 0.01), of 1621. Every language is in `leaderboard_test.csv`.
|
| 110 |
+
|
| 111 |
+
The 30 languages with the most passages:
|
| 112 |
+
|
| 113 |
+
| Label | Language | Passages | GlotLID | GlotLID restricted | fastText | fastText restricted | MillID tiny v1.5 | MillID medium v1 | MillID large v1.5 |
|
| 114 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 115 |
+
| `fra_Latn` | French | 1652 | 0.928 | 0.928 | 0.808 | 0.808 | 0.929 | 0.952 | 0.960 |
|
| 116 |
+
| `eng_Latn` | English | 1444 | 0.858 | 0.859 | 0.653 | 0.648 | 0.917 | 0.916 | 0.934 |
|
| 117 |
+
| `spa_Latn` | Spanish | 1136 | 0.951 | 0.951 | 0.574 | 0.574 | 0.930 | 0.942 | 0.955 |
|
| 118 |
+
| `tam_Taml` | Tamil | 754 | 0.993 | 0.993 | 0.965 | 0.965 | 0.997 | 0.997 | 0.999 |
|
| 119 |
+
| `arb_Arab` | Arabic (Modern Standard) | 745 | 0.785 | 0.785 | 0.513 | 0.512 | 0.814 | 0.829 | 0.833 |
|
| 120 |
+
| `por_Latn` | Portuguese | 657 | 0.947 | 0.949 | 0.756 | 0.755 | 0.947 | 0.955 | 0.965 |
|
| 121 |
+
| `kan_Knda` | Kannada | 601 | 0.993 | 0.993 | 0.922 | 0.922 | 0.969 | 0.982 | 0.965 |
|
| 122 |
+
| `zul_Latn` | Zulu | 594 | 0.896 | 0.896 | 0.539 | 0.539 | 0.846 | 0.833 | 0.878 |
|
| 123 |
+
| `swh_Latn` | Swahili | 592 | 0.917 | 0.916 | 0.346 | 0.346 | 0.848 | 0.858 | 0.870 |
|
| 124 |
+
| `mar_Deva` | Marathi | 589 | 0.989 | 0.990 | 0.926 | 0.926 | 0.965 | 0.981 | 0.976 |
|
| 125 |
+
| `tso_Latn` | Tsonga | 581 | 0.956 | 0.956 | 0.425 | 0.425 | 0.965 | 0.970 | 0.978 |
|
| 126 |
+
| `hun_Latn` | Hungarian | 576 | 0.999 | 0.999 | 0.232 | 0.232 | 0.987 | 0.990 | 0.991 |
|
| 127 |
+
| `afr_Latn` | Afrikaans | 569 | 0.985 | 0.985 | 0.861 | 0.860 | 0.967 | 0.974 | 0.979 |
|
| 128 |
+
| `tel_Telu` | Telugu | 555 | 0.994 | 0.994 | 0.951 | 0.951 | 0.991 | 0.994 | 0.988 |
|
| 129 |
+
| `rus_Cyrl` | Russian | 528 | 0.949 | 0.949 | 0.462 | 0.462 | 0.932 | 0.925 | 0.949 |
|
| 130 |
+
| `nbl_Latn` | South Ndebele | 528 | 0.948 | 0.949 | 0.000 | 0.000 | 0.903 | 0.895 | 0.927 |
|
| 131 |
+
| `ory_Orya` | Odia (individual language) | 511 | 0.991 | 0.994 | 0.976 | 0.976 | 0.988 | 0.993 | 0.993 |
|
| 132 |
+
| `sot_Latn` | Southern Sotho | 491 | 0.977 | 0.978 | 0.660 | 0.660 | 0.966 | 0.966 | 0.972 |
|
| 133 |
+
| `nso_Latn` | Northern Sotho | 479 | 0.969 | 0.969 | 0.915 | 0.914 | 0.956 | 0.960 | 0.962 |
|
| 134 |
+
| `kua_Latn` | Kuanyama | 476 | 0.986 | 0.986 | 0.000 | 0.000 | 0.962 | 0.979 | 0.975 |
|
| 135 |
+
| `asm_Beng` | Assamese | 467 | 0.999 | 0.999 | 0.990 | 0.990 | 0.997 | 0.994 | 0.996 |
|
| 136 |
+
| `ben_Beng` | Bangla | 466 | 0.997 | 0.997 | 0.718 | 0.718 | 0.977 | 0.980 | 0.979 |
|
| 137 |
+
| `pan_Guru` | Punjabi | 460 | 1.000 | 1.000 | 0.990 | 0.990 | 1.000 | 1.000 | 1.000 |
|
| 138 |
+
| `amh_Ethi` | Amharic | 458 | 1.000 | 1.000 | 0.807 | 0.807 | 0.981 | 0.984 | 0.986 |
|
| 139 |
+
| `tsn_Latn` | Tswana | 456 | 0.961 | 0.961 | 0.652 | 0.652 | 0.945 | 0.939 | 0.947 |
|
| 140 |
+
| `vro_Latn` | Võro | 448 | 0.990 | 0.990 | 0.000 | 0.000 | 0.947 | 0.958 | 0.973 |
|
| 141 |
+
| `gaz_Latn` | Oromo (West Central) | 441 | 0.938 | 0.938 | 0.407 | 0.407 | 0.910 | 0.943 | 0.928 |
|
| 142 |
+
| `som_Latn` | Somali | 439 | 0.994 | 0.994 | 0.487 | 0.487 | 0.982 | 0.988 | 0.990 |
|
| 143 |
+
| `uzn_Latn` | Northern Uzbek | 436 | 0.994 | 0.994 | 0.449 | 0.449 | 0.976 | 0.978 | 0.982 |
|
| 144 |
+
| `tha_Thai` | Thai | 431 | 0.980 | 0.980 | 0.650 | 0.650 | 0.979 | 0.970 | 0.981 |
|
| 145 |
+
|
| 146 |
+
Where MillID gains most over GlotLID:
|
| 147 |
+
|
| 148 |
+
| Label | Language | Passages | GlotLID | GlotLID restricted | fastText | fastText restricted | MillID tiny v1.5 | MillID medium v1 | MillID large v1.5 |
|
| 149 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 150 |
+
| `lbj_Tibt` | Ladakhi | 66 | 0.000 | 0.000 | 0.000 | 0.000 | 0.943 | 0.943 | 0.923 |
|
| 151 |
+
| `ain_Latn` | Ainu | 1 | 0.000 | 0.000 | 0.000 | 0.000 | 0.667 | 0.667 | 0.667 |
|
| 152 |
+
| `ajp_Arab` | South Levantine Arabic | 28 | 0.581 | 0.581 | 0.000 | 0.000 | 0.946 | 0.902 | 0.902 |
|
| 153 |
+
| `und_Laoo` | und | 20 | 0.621 | 0.621 | 0.000 | 0.000 | 0.923 | 0.857 | 0.788 |
|
| 154 |
+
| `sgh_Cyrl` | Shughni | 9 | 0.667 | 0.667 | 0.000 | 0.000 | 0.857 | 0.900 | 0.889 |
|
| 155 |
+
| `und_Thaa` | und | 29 | 0.651 | 0.651 | 0.000 | 0.000 | 0.881 | 0.873 | 0.863 |
|
| 156 |
+
| `icr_Latn` | Islander Creole English | 24 | 0.649 | 0.684 | 0.000 | 0.000 | 0.698 | 0.870 | 0.700 |
|
| 157 |
+
| `und_Geor` | und | 32 | 0.694 | 0.694 | 0.000 | 0.000 | 0.877 | 0.706 | 0.792 |
|
| 158 |
+
| `tdt_Latn` | Tetun Dili | 57 | 0.603 | 0.603 | 0.000 | 0.000 | 0.693 | 0.656 | 0.782 |
|
| 159 |
+
| `cek_Latn` | Eastern Khumi Chin | 67 | 0.680 | 0.684 | 0.000 | 0.000 | 0.785 | 0.853 | 0.825 |
|
| 160 |
+
| `und_Syrc` | und | 43 | 0.822 | 0.822 | 0.000 | 0.000 | 0.966 | 0.977 | 0.989 |
|
| 161 |
+
| `neb_Latn` | Toura (Côte d'Ivoire) | 11 | 0.625 | 0.842 | 0.000 | 0.000 | 0.957 | 0.957 | 1.000 |
|
| 162 |
+
| `dag_Latn` | Dagbani | 149 | 0.662 | 0.660 | 0.000 | 0.000 | 0.767 | 0.817 | 0.814 |
|
| 163 |
+
| `tuk_Arab` | Turkmen | 135 | 0.761 | 0.761 | 0.000 | 0.000 | 0.905 | 0.903 | 0.917 |
|
| 164 |
+
| `orv_Cyrl` | Old Russian | 239 | 0.800 | 0.800 | 0.000 | 0.000 | 0.931 | 0.935 | 0.938 |
|
| 165 |
+
|
| 166 |
+
Where MillID loses most to GlotLID:
|
| 167 |
+
|
| 168 |
+
| Label | Language | Passages | GlotLID | GlotLID restricted | fastText | fastText restricted | MillID tiny v1.5 | MillID medium v1 | MillID large v1.5 |
|
| 169 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 170 |
+
| `mus_Latn` | Muscogee | 2 | 1.000 | 1.000 | 0.000 | 0.000 | 0.333 | 0.667 | 0.667 |
|
| 171 |
+
| `ttj_Latn` | Tooro | 28 | 0.946 | 0.946 | 0.000 | 0.000 | 0.485 | 0.490 | 0.622 |
|
| 172 |
+
| `knc_Arab` | Central Kanuri | 68 | 0.823 | 0.825 | 0.007 | 0.007 | 0.462 | 0.509 | 0.532 |
|
| 173 |
+
| `hrv_Latn` | Croatian | 168 | 0.944 | 0.944 | 0.178 | 0.178 | 0.622 | 0.687 | 0.707 |
|
| 174 |
+
| `esi_Latn` | North Alaskan Inupiatun | 9 | 0.783 | 0.783 | 0.000 | 0.000 | 0.455 | 0.500 | 0.556 |
|
| 175 |
+
| `hak_Hani` | Hakka Chinese | 29 | 0.824 | 0.824 | 0.000 | 0.000 | 0.621 | 0.415 | 0.581 |
|
| 176 |
+
| `bos_Latn` | Bosnian | 198 | 0.902 | 0.902 | 0.181 | 0.181 | 0.649 | 0.673 | 0.700 |
|
| 177 |
+
| `plw_Latn` | Brooke's Point Palawano | 8 | 1.000 | 1.000 | 0.000 | 0.000 | 0.727 | 0.800 | 0.762 |
|
| 178 |
+
| `ccp_Latn` | Chakma | 3 | 1.000 | 1.000 | 0.000 | 0.000 | 0.800 | 0.333 | 0.750 |
|
| 179 |
+
| `mbd_Latn` | Dibabawon Manobo | 14 | 1.000 | 1.000 | 0.000 | 0.000 | 0.596 | 0.757 | 0.800 |
|
| 180 |
+
| `taq_Tfng` | Tamasheq | 104 | 0.718 | 0.718 | 0.000 | 0.000 | 0.529 | 0.535 | 0.527 |
|
| 181 |
+
| `zsm_Latn` | Standard Malay | 244 | 0.957 | 0.957 | 0.321 | 0.319 | 0.770 | 0.746 | 0.781 |
|
| 182 |
+
| `biu_Latn` | Biete | 7 | 1.000 | 1.000 | 0.000 | 0.000 | 0.667 | 0.667 | 0.833 |
|
| 183 |
+
| `ldi_Latn` | Laari | 62 | 0.992 | 0.992 | 0.000 | 0.000 | 0.777 | 0.831 | 0.765 |
|
| 184 |
+
| `tzm_Tfng` | Central Atlas Tamazight | 157 | 0.816 | 0.816 | 0.569 | 0.569 | 0.659 | 0.620 | 0.614 |
|
| 185 |
+
|
| 186 |
+
## Per language, test-common
|
| 187 |
+
|
| 188 |
+
Against GlotLID restricted, the best MillID is better on **13** languages and worse on **114**; against fastText restricted, the best MillID is better on **111** languages and worse on **31** (F1 differing by more than 0.01), of 213. Every language is in `leaderboard_test-common.csv`.
|
| 189 |
+
|
| 190 |
+
The 30 languages with the most passages:
|
| 191 |
+
|
| 192 |
+
| Label | Language | Passages | GlotLID | GlotLID restricted | fastText | fastText restricted | MillID tiny v1.5 | MillID medium v1 | MillID large v1.5 |
|
| 193 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 194 |
+
| `fra_Latn` | French | 1652 | 0.955 | 0.955 | 0.963 | 0.963 | 0.943 | 0.974 | 0.983 |
|
| 195 |
+
| `eng_Latn` | English | 1444 | 0.876 | 0.877 | 0.857 | 0.857 | 0.932 | 0.941 | 0.953 |
|
| 196 |
+
| `spa_Latn` | Spanish | 1136 | 0.958 | 0.958 | 0.941 | 0.941 | 0.937 | 0.953 | 0.972 |
|
| 197 |
+
| `tam_Taml` | Tamil | 754 | 1.000 | 1.000 | 0.997 | 0.997 | 0.997 | 0.998 | 0.999 |
|
| 198 |
+
| `arb_Arab` | Arabic (Modern Standard) | 745 | 0.886 | 0.886 | 0.957 | 0.954 | 0.871 | 0.905 | 0.905 |
|
| 199 |
+
| `por_Latn` | Portuguese | 657 | 0.954 | 0.956 | 0.950 | 0.949 | 0.959 | 0.963 | 0.973 |
|
| 200 |
+
| `kan_Knda` | Kannada | 601 | 0.994 | 0.994 | 0.999 | 0.999 | 0.973 | 0.987 | 0.967 |
|
| 201 |
+
| `zul_Latn` | Zulu | 594 | 0.904 | 0.904 | 0.970 | 0.970 | 0.883 | 0.871 | 0.909 |
|
| 202 |
+
| `swh_Latn` | Swahili | 592 | 0.954 | 0.953 | 0.945 | 0.945 | 0.893 | 0.897 | 0.909 |
|
| 203 |
+
| `mar_Deva` | Marathi | 589 | 0.989 | 0.990 | 0.987 | 0.987 | 0.968 | 0.981 | 0.979 |
|
| 204 |
+
| `tso_Latn` | Tsonga | 581 | 0.956 | 0.956 | 0.986 | 0.986 | 0.979 | 0.984 | 0.985 |
|
| 205 |
+
| `hun_Latn` | Hungarian | 576 | 1.000 | 1.000 | 0.980 | 0.980 | 0.989 | 0.991 | 0.993 |
|
| 206 |
+
| `afr_Latn` | Afrikaans | 569 | 0.985 | 0.985 | 0.986 | 0.986 | 0.972 | 0.976 | 0.983 |
|
| 207 |
+
| `tel_Telu` | Telugu | 555 | 0.998 | 0.998 | 0.999 | 0.999 | 0.997 | 1.000 | 0.999 |
|
| 208 |
+
| `rus_Cyrl` | Russian | 528 | 0.985 | 0.985 | 0.927 | 0.927 | 0.945 | 0.929 | 0.956 |
|
| 209 |
+
| `ory_Orya` | Odia (individual language) | 511 | 0.997 | 1.000 | 0.996 | 0.996 | 0.994 | 0.996 | 0.995 |
|
| 210 |
+
| `sot_Latn` | Southern Sotho | 491 | 0.977 | 0.978 | 0.856 | 0.856 | 0.969 | 0.967 | 0.974 |
|
| 211 |
+
| `nso_Latn` | Northern Sotho | 479 | 0.970 | 0.970 | 0.967 | 0.967 | 0.958 | 0.961 | 0.964 |
|
| 212 |
+
| `asm_Beng` | Assamese | 467 | 0.999 | 0.999 | 0.997 | 0.997 | 0.997 | 0.995 | 0.997 |
|
| 213 |
+
| `ben_Beng` | Bangla | 466 | 0.997 | 0.997 | 0.989 | 0.989 | 0.988 | 0.987 | 0.993 |
|
| 214 |
+
| `pan_Guru` | Punjabi | 460 | 1.000 | 1.000 | 0.990 | 0.990 | 1.000 | 1.000 | 1.000 |
|
| 215 |
+
| `amh_Ethi` | Amharic | 458 | 1.000 | 1.000 | 0.998 | 0.998 | 0.984 | 0.987 | 0.986 |
|
| 216 |
+
| `tsn_Latn` | Tswana | 456 | 0.961 | 0.961 | 0.870 | 0.870 | 0.948 | 0.940 | 0.947 |
|
| 217 |
+
| `gaz_Latn` | Oromo (West Central) | 441 | 0.974 | 0.974 | 0.973 | 0.973 | 0.935 | 0.955 | 0.941 |
|
| 218 |
+
| `som_Latn` | Somali | 439 | 0.995 | 0.995 | 0.991 | 0.991 | 0.984 | 0.992 | 0.991 |
|
| 219 |
+
| `uzn_Latn` | Northern Uzbek | 436 | 0.999 | 0.999 | 0.985 | 0.985 | 0.984 | 0.984 | 0.986 |
|
| 220 |
+
| `tha_Thai` | Thai | 431 | 0.980 | 0.980 | 0.981 | 0.981 | 0.980 | 0.970 | 0.981 |
|
| 221 |
+
| `ydd_Hebr` | Eastern Yiddish | 430 | 1.000 | 1.000 | 0.999 | 0.999 | 0.993 | 0.998 | 0.994 |
|
| 222 |
+
| `tur_Latn` | Turkish | 430 | 0.999 | 0.999 | 0.980 | 0.980 | 0.968 | 0.974 | 0.973 |
|
| 223 |
+
| `nno_Latn` | Norwegian Nynorsk | 427 | 0.985 | 0.985 | 0.954 | 0.954 | 0.913 | 0.919 | 0.927 |
|
| 224 |
+
|
| 225 |
+
Where MillID gains most over GlotLID:
|
| 226 |
+
|
| 227 |
+
| Label | Language | Passages | GlotLID | GlotLID restricted | fastText | fastText restricted | MillID tiny v1.5 | MillID medium v1 | MillID large v1.5 |
|
| 228 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 229 |
+
| `ewo_Latn` | Ewondo | 134 | 0.879 | 0.883 | 0.860 | 0.860 | 0.959 | 0.965 | 0.978 |
|
| 230 |
+
| `eng_Latn` | English | 1444 | 0.876 | 0.877 | 0.857 | 0.857 | 0.932 | 0.941 | 0.953 |
|
| 231 |
+
| `arb_Latn` | Standard Arabic | 163 | 0.906 | 0.913 | 0.852 | 0.852 | 0.942 | 0.944 | 0.949 |
|
| 232 |
+
| `tso_Latn` | Tsonga | 581 | 0.956 | 0.956 | 0.986 | 0.986 | 0.979 | 0.984 | 0.985 |
|
| 233 |
+
| `fra_Latn` | French | 1652 | 0.955 | 0.955 | 0.963 | 0.963 | 0.943 | 0.974 | 0.983 |
|
| 234 |
+
| `arb_Arab` | Arabic (Modern Standard) | 745 | 0.886 | 0.886 | 0.957 | 0.954 | 0.871 | 0.905 | 0.905 |
|
| 235 |
+
| `por_Latn` | Portuguese | 657 | 0.954 | 0.956 | 0.950 | 0.949 | 0.959 | 0.963 | 0.973 |
|
| 236 |
+
| `hin_Deva` | Hindi | 392 | 0.877 | 0.877 | 0.863 | 0.863 | 0.855 | 0.817 | 0.893 |
|
| 237 |
+
| `fuv_Latn` | Nigerian Fulfulde | 164 | 0.877 | 0.892 | 0.859 | 0.859 | 0.907 | 0.797 | 0.844 |
|
| 238 |
+
| `kam_Latn` | Kamba | 272 | 0.972 | 0.974 | 0.863 | 0.863 | 0.980 | 0.980 | 0.989 |
|
| 239 |
+
| `knc_Latn` | Central Kanuri | 293 | 0.952 | 0.952 | 0.981 | 0.981 | 0.967 | 0.959 | 0.967 |
|
| 240 |
+
| `spa_Latn` | Spanish | 1136 | 0.958 | 0.958 | 0.941 | 0.941 | 0.937 | 0.953 | 0.972 |
|
| 241 |
+
| `bug_Latn` | Buginese | 78 | 0.981 | 0.981 | 0.938 | 0.938 | 0.981 | 0.987 | 0.994 |
|
| 242 |
+
| `bjn_Arab` | Banjar | 116 | 0.834 | 0.987 | 0.453 | 0.960 | 0.992 | 0.983 | 0.996 |
|
| 243 |
+
| `san_Deva` | Sanskrit | 239 | 0.990 | 0.990 | 0.971 | 0.971 | 0.990 | 0.996 | 0.998 |
|
| 244 |
+
|
| 245 |
+
Where MillID loses most to GlotLID:
|
| 246 |
+
|
| 247 |
+
| Label | Language | Passages | GlotLID | GlotLID restricted | fastText | fastText restricted | MillID tiny v1.5 | MillID medium v1 | MillID large v1.5 |
|
| 248 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 249 |
+
| `knc_Arab` | Central Kanuri | 68 | 0.857 | 0.860 | 0.029 | 0.029 | 0.522 | 0.562 | 0.598 |
|
| 250 |
+
| `bos_Latn` | Bosnian | 198 | 1.000 | 1.000 | 0.897 | 0.897 | 0.685 | 0.727 | 0.762 |
|
| 251 |
+
| `tzm_Tfng` | Central Atlas Tamazight | 157 | 0.934 | 0.934 | 0.751 | 0.751 | 0.697 | 0.645 | 0.649 |
|
| 252 |
+
| `hrv_Latn` | Croatian | 168 | 0.994 | 0.994 | 0.856 | 0.856 | 0.659 | 0.717 | 0.757 |
|
| 253 |
+
| `zsm_Latn` | Standard Malay | 244 | 0.998 | 0.998 | 0.908 | 0.908 | 0.789 | 0.773 | 0.809 |
|
| 254 |
+
| `taq_Tfng` | Tamasheq | 104 | 0.805 | 0.805 | 0.000 | 0.000 | 0.654 | 0.613 | 0.604 |
|
| 255 |
+
| `ind_Latn` | Indonesian | 335 | 0.991 | 0.991 | 0.887 | 0.887 | 0.842 | 0.807 | 0.848 |
|
| 256 |
+
| `nob_Latn` | Norwegian Bokmål | 265 | 0.961 | 0.959 | 0.894 | 0.894 | 0.821 | 0.820 | 0.832 |
|
| 257 |
+
| `kin_Latn` | Kinyarwanda | 210 | 0.981 | 0.981 | 0.854 | 0.854 | 0.849 | 0.818 | 0.864 |
|
| 258 |
+
| `run_Latn` | Rundi | 291 | 0.985 | 0.985 | 0.855 | 0.855 | 0.883 | 0.843 | 0.879 |
|
| 259 |
+
| `mag_Deva` | Magahi | 97 | 1.000 | 1.000 | 0.772 | 0.772 | 0.890 | 0.905 | 0.897 |
|
| 260 |
+
| `cjk_Latn` | Chokwe | 24 | 1.000 | 1.000 | 0.807 | 0.807 | 0.857 | 0.909 | 0.837 |
|
| 261 |
+
| `taq_Latn` | Tamasheq | 106 | 1.000 | 1.000 | 0.838 | 0.838 | 0.902 | 0.909 | 0.922 |
|
| 262 |
+
| `dzo_Tibt` | Dzongkha | 346 | 0.999 | 0.999 | 0.696 | 0.696 | 0.906 | 0.926 | 0.916 |
|
| 263 |
+
| `lim_Latn` | Limburgish | 196 | 1.000 | 1.000 | 0.956 | 0.956 | 0.896 | 0.915 | 0.930 |
|
| 264 |
+
|
| 265 |
+
## Per language, udhr
|
| 266 |
+
|
| 267 |
+
Against GlotLID restricted, the best MillID is better on **75** languages and worse on **57**; against fastText restricted, the best MillID is better on **313** languages and worse on **8** (F1 differing by more than 0.01), of 358. Every language is in `leaderboard_udhr.csv`.
|
| 268 |
+
|
| 269 |
+
The 30 languages with the most passages:
|
| 270 |
+
|
| 271 |
+
| Label | Language | Passages | GlotLID | GlotLID restricted | fastText | fastText restricted | MillID tiny v1.5 | MillID medium v1 | MillID large v1.5 |
|
| 272 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 273 |
+
| `roh_Latn` | Romansh | 399 | 0.999 | 0.999 | 0.946 | 0.946 | 0.994 | 0.991 | 0.998 |
|
| 274 |
+
| `oci_Latn` | Occitan | 398 | 0.376 | 0.376 | 0.435 | 0.435 | 0.277 | 0.331 | 0.338 |
|
| 275 |
+
| `cmn_Hani` | Chinese (Mandarin) | 319 | 0.868 | 0.868 | 0.468 | 0.468 | 0.892 | 0.886 | 0.875 |
|
| 276 |
+
| `krl_Latn` | Karelian | 183 | 0.011 | 0.011 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
|
| 277 |
+
| `qug_Latn` | Chimborazo Highland Quichua | 182 | 0.000 | 0.000 | 0.000 | 0.000 | 0.011 | 0.000 | 0.000 |
|
| 278 |
+
| `acu_Latn` | Achuar-Shiwiar | 180 | 0.854 | 0.854 | 0.000 | 0.000 | 0.828 | 0.743 | 0.801 |
|
| 279 |
+
| `hus_Latn` | Huastec | 175 | 0.986 | 0.986 | 0.000 | 0.000 | 0.977 | 0.991 | 0.988 |
|
| 280 |
+
| `hau_Latn` | Hausa | 155 | 1.000 | 1.000 | 0.876 | 0.876 | 0.990 | 0.997 | 1.000 |
|
| 281 |
+
| `auc_Latn` | Waorani | 154 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.050 | 0.025 |
|
| 282 |
+
| `kwi_Latn` | Awa-Cuaiquer | 145 | 0.877 | 0.877 | 0.000 | 0.000 | 0.880 | 0.775 | 0.863 |
|
| 283 |
+
| `jpn_Jpan` | Japanese | 140 | 1.000 | 1.000 | 0.944 | 0.944 | 1.000 | 1.000 | 1.000 |
|
| 284 |
+
| `snn_Latn` | Siona | 136 | 0.514 | 0.514 | 0.000 | 0.000 | 0.630 | 0.773 | 0.467 |
|
| 285 |
+
| `ktu_Latn` | Kituba (Democratic Republic of Congo) | 118 | 0.680 | 0.680 | 0.000 | 0.000 | 0.765 | 0.769 | 0.773 |
|
| 286 |
+
| `lat_Latn` | Latin | 118 | 0.992 | 0.992 | 0.000 | 0.000 | 0.996 | 0.996 | 0.996 |
|
| 287 |
+
| `hat_Latn` | Haitian Creole | 117 | 0.979 | 0.979 | 0.699 | 0.699 | 0.992 | 0.991 | 0.987 |
|
| 288 |
+
| `nya_Latn` | Nyanja | 116 | 1.000 | 1.000 | 0.806 | 0.806 | 0.991 | 0.987 | 0.978 |
|
| 289 |
+
| `kmb_Latn` | Kimbundu | 116 | 1.000 | 1.000 | 0.877 | 0.877 | 0.970 | 0.979 | 0.991 |
|
| 290 |
+
| `rmn_Latn` | Balkan Romani | 116 | 0.857 | 0.857 | 0.000 | 0.000 | 0.678 | 0.667 | 0.704 |
|
| 291 |
+
| `twi_Latn` | Twi | 116 | 0.996 | 0.996 | 0.399 | 0.396 | 0.983 | 0.987 | 0.991 |
|
| 292 |
+
| `chr_Cher` | Cherokee | 115 | 0.663 | 0.663 | 0.000 | 0.000 | 0.670 | 0.715 | 0.945 |
|
| 293 |
+
| `por_Latn` | Portuguese | 115 | 0.996 | 0.996 | 0.793 | 0.793 | 0.955 | 0.996 | 1.000 |
|
| 294 |
+
| `lvs_Latn` | Standard Latvian | 115 | 1.000 | 1.000 | 0.991 | 0.991 | 0.987 | 0.996 | 1.000 |
|
| 295 |
+
| `pov_Latn` | Upper Guinea Crioulo | 115 | 0.833 | 0.833 | 0.000 | 0.000 | 0.821 | 0.838 | 0.827 |
|
| 296 |
+
| `kng_Latn` | Koongo | 115 | 0.193 | 0.193 | 0.457 | 0.457 | 0.076 | 0.047 | 0.047 |
|
| 297 |
+
| `ell_Grek` | Greek | 115 | 1.000 | 1.000 | 1.000 | 1.000 | 0.670 | 0.655 | 0.670 |
|
| 298 |
+
| `tha_Thai` | Thai | 114 | 1.000 | 1.000 | 0.996 | 0.996 | 1.000 | 1.000 | 1.000 |
|
| 299 |
+
| `tso_Latn` | Tsonga | 114 | 0.781 | 0.781 | 0.691 | 0.691 | 0.860 | 0.882 | 0.774 |
|
| 300 |
+
| `cjk_Latn` | Chokwe | 114 | 1.000 | 1.000 | 0.533 | 0.533 | 0.871 | 0.949 | 0.959 |
|
| 301 |
+
| `fas_Arab` | Persian | 114 | 1.000 | 1.000 | 0.934 | 0.934 | 0.991 | 0.991 | 0.982 |
|
| 302 |
+
| `lin_Latn` | Lingala | 110 | 1.000 | 1.000 | 0.986 | 0.986 | 0.991 | 1.000 | 0.996 |
|
| 303 |
+
|
| 304 |
+
Where MillID gains most over GlotLID:
|
| 305 |
+
|
| 306 |
+
| Label | Language | Passages | GlotLID | GlotLID restricted | fastText | fastText restricted | MillID tiny v1.5 | MillID medium v1 | MillID large v1.5 |
|
| 307 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 308 |
+
| `mic_Latn` | Mi'kmaw | 59 | 0.427 | 0.427 | 0.000 | 0.000 | 0.992 | 1.000 | 1.000 |
|
| 309 |
+
| `tet_Latn` | Tetum | 58 | 0.167 | 0.167 | 0.000 | 0.000 | 0.638 | 0.699 | 0.708 |
|
| 310 |
+
| `fuf_Latn` | Pular | 58 | 0.483 | 0.483 | 0.000 | 0.000 | 0.906 | 0.816 | 0.936 |
|
| 311 |
+
| `skr_Arab` | Saraiki | 63 | 0.500 | 0.500 | 0.000 | 0.000 | 0.867 | 0.923 | 0.875 |
|
| 312 |
+
| `sag_Latn` | Sango | 59 | 0.578 | 0.578 | 0.189 | 0.189 | 0.992 | 0.992 | 0.992 |
|
| 313 |
+
| `zgh_Tfng` | Standard Moroccan Tamazight | 60 | 0.462 | 0.462 | 0.000 | 0.000 | 0.750 | 0.824 | 0.652 |
|
| 314 |
+
| `qvh_Latn` | Huamalíes-Dos de Mayo Huánuco Quechua | 58 | 0.154 | 0.154 | 0.000 | 0.000 | 0.468 | 0.215 | 0.215 |
|
| 315 |
+
| `chr_Cher` | Cherokee | 115 | 0.663 | 0.663 | 0.000 | 0.000 | 0.670 | 0.715 | 0.945 |
|
| 316 |
+
| `tzm_Tfng` | Central Atlas Tamazight | 1 | 0.065 | 0.065 | 0.033 | 0.033 | 0.167 | 0.333 | 0.100 |
|
| 317 |
+
| `snn_Latn` | Siona | 136 | 0.514 | 0.514 | 0.000 | 0.000 | 0.630 | 0.773 | 0.467 |
|
| 318 |
+
| `fkv_Latn` | Kven Finnish | 58 | 0.667 | 0.667 | 0.000 | 0.000 | 0.635 | 0.697 | 0.906 |
|
| 319 |
+
| `bum_Latn` | Bulu | 58 | 0.098 | 0.098 | 0.000 | 0.000 | 0.215 | 0.319 | 0.188 |
|
| 320 |
+
| `mxv_Latn` | Metlatónoc Mixtec | 58 | 0.000 | 0.000 | 0.000 | 0.000 | 0.129 | 0.215 | 0.034 |
|
| 321 |
+
| `pnb_Arab` | Western Panjabi | 61 | 0.739 | 0.739 | 0.000 | 0.000 | 0.882 | 0.931 | 0.884 |
|
| 322 |
+
| `lld_Latn` | Ladin | 59 | 0.731 | 0.731 | 0.000 | 0.000 | 0.909 | 0.899 | 0.881 |
|
| 323 |
+
|
| 324 |
+
Where MillID loses most to GlotLID:
|
| 325 |
+
|
| 326 |
+
| Label | Language | Passages | GlotLID | GlotLID restricted | fastText | fastText restricted | MillID tiny v1.5 | MillID medium v1 | MillID large v1.5 |
|
| 327 |
+
|---|---|---|---|---|---|---|---|---|---|
|
| 328 |
+
| `gyr_Latn` | Guarayu | 49 | 0.990 | 0.990 | 0.000 | 0.000 | 0.537 | 0.339 | 0.393 |
|
| 329 |
+
| `dyu_Latn` | Dyula | 58 | 0.434 | 0.434 | 0.015 | 0.015 | 0.000 | 0.000 | 0.000 |
|
| 330 |
+
| `ell_Grek` | Greek | 115 | 1.000 | 1.000 | 1.000 | 1.000 | 0.670 | 0.655 | 0.670 |
|
| 331 |
+
| `cbu_Latn` | Candoshi-Shapra | 57 | 0.882 | 0.874 | 0.000 | 0.000 | 0.500 | 0.371 | 0.557 |
|
| 332 |
+
| `shp_Latn` | Shipibo-Conibo | 62 | 0.912 | 0.912 | 0.000 | 0.000 | 0.469 | 0.410 | 0.607 |
|
| 333 |
+
| `cpu_Latn` | Pichis Ashéninka | 60 | 0.974 | 0.974 | 0.000 | 0.000 | 0.729 | 0.571 | 0.571 |
|
| 334 |
+
| `qvn_Latn` | North Junín Quechua | 57 | 0.460 | 0.460 | 0.000 | 0.000 | 0.219 | 0.219 | 0.034 |
|
| 335 |
+
| `hin_Deva` | Hindi | 59 | 0.957 | 0.957 | 0.975 | 0.975 | 0.723 | 0.607 | 0.727 |
|
| 336 |
+
| `cni_Latn` | Asháninka | 52 | 0.963 | 0.963 | 0.000 | 0.000 | 0.773 | 0.708 | 0.717 |
|
| 337 |
+
| `hrv_Latn` | Croatian | 58 | 0.727 | 0.727 | 0.474 | 0.474 | 0.500 | 0.537 | 0.559 |
|
| 338 |
+
| `rmn_Latn` | Balkan Romani | 116 | 0.857 | 0.857 | 0.000 | 0.000 | 0.678 | 0.667 | 0.704 |
|
| 339 |
+
| `koi_Cyrl` | Komi-Permyak | 57 | 1.000 | 1.000 | 0.000 | 0.000 | 0.837 | 0.837 | 0.849 |
|
| 340 |
+
| `qvm_Latn` | Margos-Yarowilca-Lauricocha Quechua | 60 | 0.500 | 0.500 | 0.000 | 0.000 | 0.356 | 0.094 | 0.356 |
|
| 341 |
+
| `kea_Latn` | Kabuverdianu | 58 | 0.935 | 0.935 | 0.125 | 0.124 | 0.792 | 0.794 | 0.773 |
|
| 342 |
+
| `ind_Latn` | Indonesian | 60 | 0.903 | 0.903 | 0.565 | 0.565 | 0.768 | 0.674 | 0.645 |
|
| 343 |
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
leaderboard_overall.csv
CHANGED
|
@@ -1,51 +1,36 @@
|
|
| 1 |
-
set,model,repo,size,top1_accuracy,top3_accuracy,top5_accuracy,macro_precision,macro_recall,macro_f1,median_f1,languages_f1_at_least_0_9,languages
|
| 2 |
-
test,MillID tiny,baobabtech/millid-tiny,2.9M,0.
|
| 3 |
-
test,GlotLID,cis-lmu/glotlid,"fastText, 1.7 GB",0.977614145795264,0.9977189572137978,0.9983766301339072,0.9782517709855841,0.9751782136320077,0.9748179587217549,1.0,1488,1621
|
| 4 |
-
test,"GlotLID, restricted to MillID's labels",cis-lmu/glotlid,"fastText, 1.7 GB",0.978292631149554,0.9977897194900122,0.9984557173837938,0.9778492893480586,0.9764998669631992,0.9754780061637239,1.0,1490,1621
|
| 5 |
-
test,"fastText (Meta, NLLB)",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.25110201838986684,0.2610045745730329,0.262482257399861,0.05202750834127428,0.12048375928156996,0.06567764337804372,0.0,25,1621
|
| 6 |
-
test,"fastText, restricted to MillID's labels",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.2514599922577745,0.2610753368492472,0.2625197197813862,0.05205991842533006,0.12089826722608317,0.06593662879256669,0.0,26,1621
|
| 7 |
-
test-common,MillID tiny,baobabtech/millid-tiny,2.9M,0.
|
| 8 |
-
test-common,GlotLID,cis-lmu/glotlid,"fastText, 1.7 GB",0.9738223137297398,0.9924609501989257,0.9946437141043701,0.9924377815827973,0.9770740040297234,0.9835053354686013,0.9982788296041308,201,213
|
| 9 |
-
test-common,"GlotLID, restricted to MillID's labels",cis-lmu/glotlid,"fastText, 1.7 GB",0.9749369165750732,0.9926931591250368,0.994829481245259,0.9922871273519716,0.9790381188248578,0.9845885184031669,0.9983471074380166,203,213
|
| 10 |
-
test-common,"fastText (Meta, NLLB)",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.9338668978435531,0.9706952335247767,0.9761908447760732,0.9356570616737796,0.9169210037343893,0.9148824189777787,0.9680365296803652,165,213
|
| 11 |
-
test-common,"fastText, restricted to MillID's labels",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.9351982290199236,0.9709584036410359,0.9763301701317398,0.9353710554982398,0.9200755454154029,0.9173068426975766,0.9680365296803652,166,213
|
| 12 |
-
udhr,MillID tiny,baobabtech/millid-tiny,2.9M,0.
|
| 13 |
-
udhr,GlotLID,cis-lmu/glotlid,"fastText, 1.7 GB",0.8598419206221734,0.9279766684982459,0.9420516505346802,0.9261030959501081,0.8825915640670888,0.8864649884094992,0.9915248150542268,287,358
|
| 14 |
-
udhr,"GlotLID, restricted to MillID's labels",cis-lmu/glotlid,"fastText, 1.7 GB",0.8614480747284331,0.9294137537512152,0.943319666934359,0.9257695969546421,0.8844312457708232,0.8873151713293488,0.9915248150542268,291,358
|
| 15 |
-
udhr,"fastText (Meta, NLLB)",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.4624878481761697,0.4856502810769686,0.49034194175578005,0.31521227915727285,0.4478627321056064,0.3496561644788506,0.0,60,358
|
| 16 |
-
udhr,"fastText, restricted to MillID's labels",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.4628682530960734,0.4856502810769686,0.4904264761824253,0.3151627394343268,0.4483116546914005,0.34986047374808604,0.0,61,358
|
| 17 |
-
udhr-common,MillID tiny,baobabtech/millid-tiny,2.9M,0.
|
| 18 |
-
udhr-common,GlotLID,cis-lmu/glotlid,"fastText, 1.7 GB",0.9335852225020991,0.9677581863979848,0.973215785054576,0.9887806114106193,0.9575995164374027,0.9657221446088623,1.0,155,170
|
| 19 |
-
udhr-common,"GlotLID, restricted to MillID's labels",cis-lmu/glotlid,"fastText, 1.7 GB",0.9338371116708648,0.9682619647355164,0.9735516372795969,0.9887806114106193,0.9578604445634422,0.9658801542070193,1.0,156,170
|
| 20 |
-
udhr-common,"fastText (Meta, NLLB)",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.9187237615449202,0.964735516372796,0.9740554156171285,0.9473808585265043,0.9431462240812183,0.9323102708833079,1.0,141,170
|
| 21 |
-
udhr-common,"fastText, restricted to MillID's labels",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.9194794290512175,0.964735516372796,0.974223341729639,0.9473808585265043,0.9440916022324787,0.9328542536140149,1.0,142,170
|
| 22 |
-
udhr-all,MillID tiny,baobabtech/millid-tiny,2.9M,0.
|
| 23 |
-
udhr-all,GlotLID,cis-lmu/glotlid,"fastText, 1.7 GB",0.750244361582739,0.8144662056981501,0.8281866560474966,0.7694575973118034,0.7529055089081532,0.7454327849166208,0.9795918367346939,277,429
|
| 24 |
-
udhr-all,"GlotLID, restricted to MillID's labels",cis-lmu/glotlid,"fastText, 1.7 GB",0.7378271730080006,0.7960395322738297,0.8079498968251095,0.7425329775774382,0.7380568437901042,0.724506747472521,0.9747899159663865,272,429
|
| 25 |
-
udhr-all,"fastText (Meta, NLLB)",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.3961191760489447,0.4159577163957572,0.4199761068674655,0.24791077144530446,0.3737409279575923,0.279501714390051,0.0,54,429
|
| 26 |
-
udhr-all,"fastText, restricted to MillID's labels",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.39644499149259677,0.4159577163957572,0.42004851029938817,0.2478791005299098,0.3741155533322177,0.27967804305179117,0.0,55,429
|
| 27 |
-
test,MillID medium,baobabtech/millid-medium,115.0M,0.9616468462918486,0.9910007034602754,0.9945180048368097,0.9564228576676835,0.9614016943550959,0.9575764452347921,0.9841269841269842,1451,1621
|
| 28 |
-
test-common,MillID medium,baobabtech/millid-medium,115.0M,0.9424741087047386,0.988172825363407,0.9923525860334071,0.9732280825173869,0.9360723187927141,0.9527046866271709,0.9738219895287958,191,213
|
| 29 |
-
udhr,MillID medium,baobabtech/millid-medium,115.0M,0.8524028910773913,0.9194386914070756,0.9340208800033813,0.9186545076419138,0.8769388469921001,0.8779986710808113,0.9909909909909909,282,358
|
| 30 |
-
udhr-common,MillID medium,baobabtech/millid-medium,115.0M,0.9120906801007557,0.9633081444164567,0.9688497061293031,0.9770201360820081,0.9382111759192128,0.947989665742032,0.9955099164401491,153,170
|
| 31 |
-
udhr-all,MillID medium,baobabtech/millid-medium,115.0M,0.7300800057922745,0.7874959273069544,0.7999855193136155,0.7415286668963516,0.731804445741659,0.718364827833673,0.9672131147540983,266,429
|
| 32 |
-
test,MillID large,baobabtech/millid-large,192.6M,0.
|
| 33 |
-
test-common,MillID large,baobabtech/millid-large,192.6M,0.
|
| 34 |
-
udhr,MillID large,baobabtech/millid-large,192.6M,0.
|
| 35 |
-
udhr-common,MillID large,baobabtech/millid-large,192.6M,0.
|
| 36 |
-
udhr-all,MillID large,baobabtech/millid-large,192.6M,0.
|
| 37 |
-
test,MillID tiny v1.5,baobabtech/millid-tiny-v1.5,2.9M,0.9587247805328816,0.9890484971341278,0.9932401213781161,0.9549770065582561,0.9588167712047285,0.9553762683202616,0.9824561403508771,1446,1621
|
| 38 |
-
test-common,MillID tiny v1.5,baobabtech/millid-tiny-v1.5,2.9M,0.9395947180209607,0.9856649689614069,0.9908664489062959,0.970922775379856,0.9353270477650256,0.951259698841108,0.9726495726495726,188,213
|
| 39 |
-
udhr,MillID tiny v1.5,baobabtech/millid-tiny-v1.5,2.9M,0.8492328500781944,0.911703791369035,0.9256519717655015,0.914803800630921,0.8736275490421017,0.8764285216392155,0.9906542056074767,274,358
|
| 40 |
-
udhr-common,MillID tiny v1.5,baobabtech/millid-tiny-v1.5,2.9M,0.9123425692695214,0.9606213266162889,0.9654911838790932,0.9794271783100589,0.9393117513722192,0.9512636942896991,1.0,149,170
|
| 41 |
-
udhr-all,MillID tiny v1.5,baobabtech/millid-tiny-v1.5,2.9M,0.7273648770951743,0.7808710132860297,0.7928175795532708,0.7340309394117082,0.7290411714617072,0.7148579318843858,0.9649122807017544,254,429
|
| 42 |
-
test,MillID medium v1.5,baobabtech/millid-medium-v1.5,115.0M,0.9575592842187637,0.9894064710020355,0.9936563700617297,0.9533057182844771,0.957795228936456,0.9535414152154522,0.9823874755381604,1439,1621
|
| 43 |
-
test-common,MillID medium v1.5,baobabtech/millid-medium-v1.5,115.0M,0.9357400498475161,0.985494682415592,0.9911296190225553,0.9693373446332887,0.9301139180681923,0.9467462692735156,0.9702602230483272,186,213
|
| 44 |
-
udhr,MillID medium v1.5,baobabtech/millid-medium-v1.5,115.0M,0.8453019992391901,0.9223973963396593,0.9389661439621286,0.9146460443129042,0.8687651165993733,0.87036932500799,0.9845569018769303,275,358
|
| 45 |
-
udhr-common,MillID medium v1.5,baobabtech/millid-medium-v1.5,115.0M,0.9002518891687658,0.9629722921914358,0.9701931150293871,0.9738075955416304,0.9283847515638696,0.9389938762973807,0.9915966386554621,146,170
|
| 46 |
-
udhr-all,MillID medium v1.5,baobabtech/millid-medium-v1.5,115.0M,0.72399811751077,0.7900300474242479,0.8042211200810918,0.7358829743260015,0.7249834772554209,0.7121266253368457,0.9649122807017544,259,429
|
| 47 |
-
test,MillID large v1.5,baobabtech/millid-large-v1.5,192.6M,0.9674118905598961,0.9924242739582336,0.9955503015721713,0.9655219790304682,0.9662728030347064,0.9646355846561194,0.9893617021276596,1478,1621
|
| 48 |
-
test-common,MillID large v1.5,baobabtech/millid-large-v1.5,192.6M,0.952505534312739,0.989829249036333,0.993761320185148,0.9756915726411073,0.946030279290983,0.9594091462508073,0.9790794979079497,193,213
|
| 49 |
-
udhr,MillID large v1.5,baobabtech/millid-large-v1.5,192.6M,0.85785536159601,0.9182974766473646,0.9308931062175071,0.9171072333156591,0.8824937737402124,0.8812295192906267,0.9914161508989096,283,358
|
| 50 |
-
udhr-common,MillID large v1.5,baobabtech/millid-large-v1.5,192.6M,0.9167926112510495,0.9629722921914358,0.9686817800167926,0.9800375512974252,0.9436021694458361,0.9528568922894297,1.0,152,170
|
| 51 |
-
udhr-all,MillID large v1.5,baobabtech/millid-large-v1.5,192.6M,0.734750027151287,0.7865184809759983,0.7973065923324766,0.7392238103714293,0.7364400256386854,0.7200574273845932,0.9642857142857142,266,429
|
|
|
|
| 1 |
+
set,model,repo,size,top1_accuracy,top3_accuracy,top5_accuracy,macro_precision,macro_recall,macro_f1,median_f1,languages_f1_at_least_0_9,languages
|
| 2 |
+
test,MillID tiny v1.5,baobabtech/millid-tiny-v1.5,2.9M,0.9587247805328816,0.9890484971341278,0.9932401213781161,0.9549770065582561,0.9588167712047285,0.9553762683202616,0.9824561403508771,1446,1621
|
| 3 |
+
test,GlotLID,cis-lmu/glotlid,"fastText, 1.7 GB",0.977614145795264,0.9977189572137978,0.9983766301339072,0.9782517709855841,0.9751782136320077,0.9748179587217549,1.0,1488,1621
|
| 4 |
+
test,"GlotLID, restricted to MillID's labels",cis-lmu/glotlid,"fastText, 1.7 GB",0.978292631149554,0.9977897194900122,0.9984557173837938,0.9778492893480586,0.9764998669631992,0.9754780061637239,1.0,1490,1621
|
| 5 |
+
test,"fastText (Meta, NLLB)",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.25110201838986684,0.2610045745730329,0.262482257399861,0.05202750834127428,0.12048375928156996,0.06567764337804372,0.0,25,1621
|
| 6 |
+
test,"fastText, restricted to MillID's labels",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.2514599922577745,0.2610753368492472,0.2625197197813862,0.05205991842533006,0.12089826722608317,0.06593662879256669,0.0,26,1621
|
| 7 |
+
test-common,MillID tiny v1.5,baobabtech/millid-tiny-v1.5,2.9M,0.9395947180209607,0.9856649689614069,0.9908664489062959,0.970922775379856,0.9353270477650256,0.951259698841108,0.9726495726495726,188,213
|
| 8 |
+
test-common,GlotLID,cis-lmu/glotlid,"fastText, 1.7 GB",0.9738223137297398,0.9924609501989257,0.9946437141043701,0.9924377815827973,0.9770740040297234,0.9835053354686013,0.9982788296041308,201,213
|
| 9 |
+
test-common,"GlotLID, restricted to MillID's labels",cis-lmu/glotlid,"fastText, 1.7 GB",0.9749369165750732,0.9926931591250368,0.994829481245259,0.9922871273519716,0.9790381188248578,0.9845885184031669,0.9983471074380166,203,213
|
| 10 |
+
test-common,"fastText (Meta, NLLB)",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.9338668978435531,0.9706952335247767,0.9761908447760732,0.9356570616737796,0.9169210037343893,0.9148824189777787,0.9680365296803652,165,213
|
| 11 |
+
test-common,"fastText, restricted to MillID's labels",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.9351982290199236,0.9709584036410359,0.9763301701317398,0.9353710554982398,0.9200755454154029,0.9173068426975766,0.9680365296803652,166,213
|
| 12 |
+
udhr,MillID tiny v1.5,baobabtech/millid-tiny-v1.5,2.9M,0.8492328500781944,0.911703791369035,0.9256519717655015,0.914803800630921,0.8736275490421017,0.8764285216392155,0.9906542056074767,274,358
|
| 13 |
+
udhr,GlotLID,cis-lmu/glotlid,"fastText, 1.7 GB",0.8598419206221734,0.9279766684982459,0.9420516505346802,0.9261030959501081,0.8825915640670888,0.8864649884094992,0.9915248150542268,287,358
|
| 14 |
+
udhr,"GlotLID, restricted to MillID's labels",cis-lmu/glotlid,"fastText, 1.7 GB",0.8614480747284331,0.9294137537512152,0.943319666934359,0.9257695969546421,0.8844312457708232,0.8873151713293488,0.9915248150542268,291,358
|
| 15 |
+
udhr,"fastText (Meta, NLLB)",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.4624878481761697,0.4856502810769686,0.49034194175578005,0.31521227915727285,0.4478627321056064,0.3496561644788506,0.0,60,358
|
| 16 |
+
udhr,"fastText, restricted to MillID's labels",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.4628682530960734,0.4856502810769686,0.4904264761824253,0.3151627394343268,0.4483116546914005,0.34986047374808604,0.0,61,358
|
| 17 |
+
udhr-common,MillID tiny v1.5,baobabtech/millid-tiny-v1.5,2.9M,0.9123425692695214,0.9606213266162889,0.9654911838790932,0.9794271783100589,0.9393117513722192,0.9512636942896991,1.0,149,170
|
| 18 |
+
udhr-common,GlotLID,cis-lmu/glotlid,"fastText, 1.7 GB",0.9335852225020991,0.9677581863979848,0.973215785054576,0.9887806114106193,0.9575995164374027,0.9657221446088623,1.0,155,170
|
| 19 |
+
udhr-common,"GlotLID, restricted to MillID's labels",cis-lmu/glotlid,"fastText, 1.7 GB",0.9338371116708648,0.9682619647355164,0.9735516372795969,0.9887806114106193,0.9578604445634422,0.9658801542070193,1.0,156,170
|
| 20 |
+
udhr-common,"fastText (Meta, NLLB)",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.9187237615449202,0.964735516372796,0.9740554156171285,0.9473808585265043,0.9431462240812183,0.9323102708833079,1.0,141,170
|
| 21 |
+
udhr-common,"fastText, restricted to MillID's labels",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.9194794290512175,0.964735516372796,0.974223341729639,0.9473808585265043,0.9440916022324787,0.9328542536140149,1.0,142,170
|
| 22 |
+
udhr-all,MillID tiny v1.5,baobabtech/millid-tiny-v1.5,2.9M,0.7273648770951743,0.7808710132860297,0.7928175795532708,0.7340309394117082,0.7290411714617072,0.7148579318843858,0.9649122807017544,254,429
|
| 23 |
+
udhr-all,GlotLID,cis-lmu/glotlid,"fastText, 1.7 GB",0.750244361582739,0.8144662056981501,0.8281866560474966,0.7694575973118034,0.7529055089081532,0.7454327849166208,0.9795918367346939,277,429
|
| 24 |
+
udhr-all,"GlotLID, restricted to MillID's labels",cis-lmu/glotlid,"fastText, 1.7 GB",0.7378271730080006,0.7960395322738297,0.8079498968251095,0.7425329775774382,0.7380568437901042,0.724506747472521,0.9747899159663865,272,429
|
| 25 |
+
udhr-all,"fastText (Meta, NLLB)",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.3961191760489447,0.4159577163957572,0.4199761068674655,0.24791077144530446,0.3737409279575923,0.279501714390051,0.0,54,429
|
| 26 |
+
udhr-all,"fastText, restricted to MillID's labels",facebook/fasttext-language-identification,"fastText, 1.2 GB",0.39644499149259677,0.4159577163957572,0.42004851029938817,0.2478791005299098,0.3741155533322177,0.27967804305179117,0.0,55,429
|
| 27 |
+
test,MillID medium v1,baobabtech/millid-medium-v1,115.0M,0.9616468462918486,0.9910007034602754,0.9945180048368097,0.9564228576676835,0.9614016943550959,0.9575764452347921,0.9841269841269842,1451,1621
|
| 28 |
+
test-common,MillID medium v1,baobabtech/millid-medium-v1,115.0M,0.9424741087047386,0.988172825363407,0.9923525860334071,0.9732280825173869,0.9360723187927141,0.9527046866271709,0.9738219895287958,191,213
|
| 29 |
+
udhr,MillID medium v1,baobabtech/millid-medium-v1,115.0M,0.8524028910773913,0.9194386914070756,0.9340208800033813,0.9186545076419138,0.8769388469921001,0.8779986710808113,0.9909909909909909,282,358
|
| 30 |
+
udhr-common,MillID medium v1,baobabtech/millid-medium-v1,115.0M,0.9120906801007557,0.9633081444164567,0.9688497061293031,0.9770201360820081,0.9382111759192128,0.947989665742032,0.9955099164401491,153,170
|
| 31 |
+
udhr-all,MillID medium v1,baobabtech/millid-medium-v1,115.0M,0.7300800057922745,0.7874959273069544,0.7999855193136155,0.7415286668963516,0.731804445741659,0.718364827833673,0.9672131147540983,266,429
|
| 32 |
+
test,MillID large v1.5,baobabtech/millid-large-v1.5,192.6M,0.9674118905598961,0.9924242739582336,0.9955503015721713,0.9655219790304682,0.9662728030347064,0.9646355846561194,0.9893617021276596,1478,1621
|
| 33 |
+
test-common,MillID large v1.5,baobabtech/millid-large-v1.5,192.6M,0.952505534312739,0.989829249036333,0.993761320185148,0.9756915726411073,0.946030279290983,0.9594091462508073,0.9790794979079497,193,213
|
| 34 |
+
udhr,MillID large v1.5,baobabtech/millid-large-v1.5,192.6M,0.85785536159601,0.9182974766473646,0.9308931062175071,0.9171072333156591,0.8824937737402124,0.8812295192906267,0.9914161508989096,283,358
|
| 35 |
+
udhr-common,MillID large v1.5,baobabtech/millid-large-v1.5,192.6M,0.9167926112510495,0.9629722921914358,0.9686817800167926,0.9800375512974252,0.9436021694458361,0.9528568922894297,1.0,152,170
|
| 36 |
+
udhr-all,MillID large v1.5,baobabtech/millid-large-v1.5,192.6M,0.734750027151287,0.7865184809759983,0.7973065923324766,0.7392238103714293,0.7364400256386854,0.7200574273845932,0.9642857142857142,266,429
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
leaderboard_test-common.csv
CHANGED
|
@@ -1,214 +1,214 @@
|
|
| 1 |
-
label,language,support,GlotLID,GlotLID restricted,fastText,fastText restricted,MillID tiny
|
| 2 |
-
fra_Latn,French,1652,0.9548,0.9548,0.9632,0.9632,0.
|
| 3 |
-
eng_Latn,English,1444,0.876,0.8768,0.8574,0.8571,0.
|
| 4 |
-
spa_Latn,Spanish,1136,0.9583,0.9583,0.9407,0.9407,0.
|
| 5 |
-
tam_Taml,Tamil,754,1.0,1.0,0.9973,0.9973,0.
|
| 6 |
-
arb_Arab,Arabic (Modern Standard),745,0.8864,0.8864,0.9573,0.9542,0.
|
| 7 |
-
por_Latn,Portuguese,657,0.9538,0.9562,0.95,0.9493,0.
|
| 8 |
-
kan_Knda,Kannada,601,0.9941,0.9941,0.9992,0.9992,0.
|
| 9 |
-
zul_Latn,Zulu,594,0.9042,0.9042,0.9704,0.9704,0.
|
| 10 |
-
swh_Latn,Swahili,592,0.9536,0.9528,0.9454,0.9454,0.
|
| 11 |
-
mar_Deva,Marathi,589,0.9888,0.9897,0.9871,0.9871,0.
|
| 12 |
-
tso_Latn,Tsonga,581,0.956,0.956,0.9861,0.9861,0.
|
| 13 |
-
hun_Latn,Hungarian,576,1.0,1.0,0.9804,0.9804,0.
|
| 14 |
-
afr_Latn,Afrikaans,569,0.9849,0.9849,0.9858,0.9858,0.
|
| 15 |
-
tel_Telu,Telugu,555,0.9982,0.9982,0.9991,0.9991,0.
|
| 16 |
-
rus_Cyrl,Russian,528,0.9846,0.9846,0.9271,0.9271,0.
|
| 17 |
-
ory_Orya,Odia (individual language),511,0.9971,1.0,0.9961,0.9961,0.
|
| 18 |
-
sot_Latn,Southern Sotho,491,0.9774,0.9784,0.8562,0.8562,0.
|
| 19 |
-
nso_Latn,Northern Sotho,479,0.9695,0.9695,0.9673,0.9673,0.
|
| 20 |
-
asm_Beng,Assamese,467,0.9989,0.9989,0.9968,0.9968,0.
|
| 21 |
-
ben_Beng,Bangla,466,0.9968,0.9968,0.9894,0.9894,0.
|
| 22 |
-
pan_Guru,Punjabi,460,1.0,1.0,0.9901,0.9901,1.0,1.0,1.0,1.0,
|
| 23 |
-
amh_Ethi,Amharic,458,1.0,1.0,0.9978,0.9978,0.
|
| 24 |
-
tsn_Latn,Tswana,456,0.9605,0.9605,0.8698,0.8698,0.
|
| 25 |
-
gaz_Latn,Oromo (West Central),441,0.9744,0.9744,0.9733,0.9733,0.
|
| 26 |
-
som_Latn,Somali,439,0.9954,0.9954,0.9909,0.9909,0.
|
| 27 |
-
uzn_Latn,Northern Uzbek,436,0.9989,0.9989,0.9849,0.9849,0.
|
| 28 |
-
tha_Thai,Thai,431,0.9799,0.9799,0.9811,0.9811,0.
|
| 29 |
-
ydd_Hebr,Eastern Yiddish,430,1.0,1.0,0.9988,0.9988,0.
|
| 30 |
-
tur_Latn,Turkish,430,0.9988,0.9988,0.9803,0.9803,0.
|
| 31 |
-
nno_Latn,Norwegian Nynorsk,427,0.9845,0.9845,0.9542,0.9542,0.
|
| 32 |
-
smo_Latn,Samoan,426,0.9988,0.9988,0.9372,0.9372,0.
|
| 33 |
-
pap_Latn,Papiamento,423,0.9988,0.9988,0.9742,0.9731,0.
|
| 34 |
-
ckb_Arab,Central Kurdish,421,0.9769,0.9769,0.9952,0.9952,0.
|
| 35 |
-
pbt_Arab,Southern Pashto,419,0.9988,0.9988,0.9782,0.977,0.
|
| 36 |
-
ekk_Latn,Estonian,419,0.9929,0.9929,0.9857,0.9857,0.
|
| 37 |
-
sna_Latn,Shona,415,1.0,1.0,0.9831,0.9831,0.
|
| 38 |
-
luo_Latn,Luo (Kenya and Tanzania),415,1.0,1.0,0.9564,0.9564,0.
|
| 39 |
-
tgk_Cyrl,Tajik,413,1.0,1.0,0.983,0.983,0.
|
| 40 |
-
bod_Tibt,Tibetan,406,0.9988,0.9988,0.8191,0.8191,0.
|
| 41 |
-
lug_Latn,Ganda,405,0.9863,0.9876,0.979,0.979,0.
|
| 42 |
-
fas_Arab,Persian,396,0.9859,0.9859,0.9439,0.9439,0.
|
| 43 |
-
ell_Grek,Greek,394,0.9987,0.9987,0.97,0.97,0.
|
| 44 |
-
hin_Deva,Hindi,392,0.8768,0.8768,0.863,0.863,0.
|
| 45 |
-
lus_Latn,Mizo,391,0.9974,0.9974,0.9897,0.9897,0.
|
| 46 |
-
dyu_Latn,Dyula,390,1.0,1.0,0.8223,0.8223,0.
|
| 47 |
-
slk_Latn,Slovak,374,0.9987,0.9987,0.9714,0.9714,0.
|
| 48 |
-
ron_Latn,Romanian,369,0.9959,0.9959,0.9759,0.9759,0.
|
| 49 |
-
kal_Latn,Kalaallisut,369,0.9986,0.9986,0.9932,0.9932,0.
|
| 50 |
-
azb_Arab,South Azerbaijani,369,0.9959,0.9959,0.9711,0.9711,0.
|
| 51 |
-
fin_Latn,Finnish,368,1.0,1.0,0.9946,0.9946,0.
|
| 52 |
-
hau_Latn,Hausa,367,0.9932,0.9932,0.9851,0.9837,0.
|
| 53 |
-
xho_Latn,Xhosa,367,0.9481,0.9481,0.9511,0.9511,0.
|
| 54 |
-
kng_Latn,Koongo,362,1.0,1.0,0.9718,0.9718,0.
|
| 55 |
-
twi_Latn,Twi,359,0.9916,0.9902,0.9684,0.9684,0.
|
| 56 |
-
deu_Latn,German,356,0.9972,0.9972,0.9889,0.9889,0.
|
| 57 |
-
guj_Gujr,Gujarati,354,1.0,1.0,0.9842,0.9842,1.0,1.0,1.0,1.0,
|
| 58 |
-
dik_Latn,Southwestern Dinka,350,1.0,1.0,0.99,0.99,0.
|
| 59 |
-
lua_Latn,Luba-Lulua,346,0.9971,0.9971,0.9633,0.9633,0.
|
| 60 |
-
dzo_Tibt,Dzongkha,346,0.9986,0.9986,0.6958,0.6958,0.
|
| 61 |
-
che_Cyrl,Chechen,345,1.0,1.0,0.984,0.984,0.
|
| 62 |
-
fij_Latn,Fijian,344,1.0,1.0,0.9884,0.9884,0.
|
| 63 |
-
ita_Latn,Italian,342,0.9838,0.9838,0.949,0.949,0.
|
| 64 |
-
mya_Mymr,Burmese,342,1.0,1.0,0.9897,0.9897,0.
|
| 65 |
-
kaz_Cyrl,Kazakh,340,1.0,1.0,0.9926,0.9926,0.
|
| 66 |
-
tir_Ethi,Tigrinya,339,0.9881,0.9985,0.9956,0.9956,0.
|
| 67 |
-
mal_Mlym,Malayalam,338,1.0,1.0,0.994,0.994,1.0,0.9985,
|
| 68 |
-
hat_Latn,Haitian Creole,338,0.997,0.997,0.9774,0.9774,0.
|
| 69 |
-
ceb_Latn,Cebuano,338,1.0,1.0,0.9926,0.9926,0.
|
| 70 |
-
vie_Latn,Vietnamese,338,1.0,1.0,0.9836,0.9836,0.
|
| 71 |
-
slv_Latn,Slovenian,338,0.9927,0.9927,0.9653,0.9653,0.
|
| 72 |
-
lao_Laoo,Lao,335,1.0,1.0,0.991,0.991,0.
|
| 73 |
-
chv_Cyrl,Chuvash,335,1.0,1.0,0.9926,0.9926,0.
|
| 74 |
-
ind_Latn,Indonesian,335,0.9911,0.9911,0.8873,0.8873,0.
|
| 75 |
-
uig_Arab,Uyghur,334,1.0,1.0,0.9911,0.9911,0.
|
| 76 |
-
oci_Latn,Occitan,333,0.9911,0.9911,0.9619,0.9619,0.
|
| 77 |
-
mkd_Cyrl,Macedonian,331,1.0,1.0,0.9924,0.9924,0.
|
| 78 |
-
lit_Latn,Lithuanian,330,0.9955,0.9955,0.974,0.974,0.
|
| 79 |
-
swe_Latn,Swedish,328,0.9717,0.9717,0.9767,0.9767,0.
|
| 80 |
-
snd_Arab,Sindhi,325,1.0,1.0,0.9848,0.9833,0.
|
| 81 |
-
kat_Geor,Georgian,324,1.0,1.0,0.9859,0.9859,0.
|
| 82 |
-
jpn_Jpan,Japanese,323,0.9969,0.9969,0.8684,0.8684,0.
|
| 83 |
-
ssw_Latn,Swati,322,0.9684,0.97,0.9721,0.9721,0.
|
| 84 |
-
ukr_Cyrl,Ukrainian,322,1.0,1.0,0.9783,0.9783,0.
|
| 85 |
-
szl_Latn,Silesian,320,0.9969,0.9969,0.9254,0.9254,0.
|
| 86 |
-
cat_Latn,Catalan,318,0.9845,0.9845,0.9528,0.9528,0.
|
| 87 |
-
ayr_Latn,Central Aymara,309,1.0,1.0,0.9752,0.9752,0.
|
| 88 |
-
yor_Latn,Yoruba,308,0.9984,0.9984,0.9683,0.9683,0.
|
| 89 |
-
ewe_Latn,Ewe,308,1.0,1.0,0.9852,0.9852,0.
|
| 90 |
-
roh_Latn,Romansh,307,1.0,1.0,0.9742,0.9742,0.
|
| 91 |
-
mni_Beng,Manipuri,304,1.0,1.0,0.9499,0.9499,0.
|
| 92 |
-
tpi_Latn,Tok Pisin,302,0.9983,0.9983,0.9867,0.9867,0.
|
| 93 |
-
lvs_Latn,Standard Latvian,298,0.995,0.995,0.9472,0.9472,0.
|
| 94 |
-
ton_Latn,Tongan,298,1.0,1.0,0.0,0.0,0.9966,0.9915,0.9949,0.9966,
|
| 95 |
-
epo_Latn,Esperanto,294,0.9983,0.9983,0.9882,0.9882,0.
|
| 96 |
-
knc_Latn,Central Kanuri,293,0.9517,0.9517,0.9811,0.9811,0.
|
| 97 |
-
run_Latn,Rundi,291,0.9848,0.9848,0.8545,0.8545,0.
|
| 98 |
-
mlt_Latn,Maltese,291,0.9932,0.9932,0.9614,0.9614,0.9948,0.9966,0.
|
| 99 |
-
bho_Deva,Bhojpuri,290,0.9983,0.9983,0.8732,0.8732,0.
|
| 100 |
-
bak_Cyrl,Bashkir,289,1.0,1.0,0.9846,0.9846,0.
|
| 101 |
-
kmr_Latn,Northern Kurdish,289,1.0,1.0,0.9747,0.9747,0.
|
| 102 |
-
bul_Cyrl,Bulgarian,289,1.0,1.0,0.9827,0.9827,0.
|
| 103 |
-
cmn_Hani,Chinese (Mandarin),288,0.9663,0.9663,0.4884,0.4884,0.
|
| 104 |
-
bam_Latn,Bambara,287,0.9983,0.9983,0.7923,0.7923,0.
|
| 105 |
-
abk_Cyrl,Abkhazian,285,1.0,1.0,1.0,1.0,
|
| 106 |
-
bxr_Cyrl,Russia Buriat,277,1.0,1.0,0.9387,0.9387,0.
|
| 107 |
-
isl_Latn,Icelandic,276,1.0,1.0,0.7886,0.7886,0.
|
| 108 |
-
lmo_Latn,Lombard,276,0.9875,0.9875,0.9097,0.9081,0.
|
| 109 |
-
npi_Deva,Nepali,273,0.9908,0.9927,0.9745,0.9745,0.
|
| 110 |
-
kam_Latn,Kamba,272,0.9716,0.9736,0.8625,0.8625,0.
|
| 111 |
-
quy_Latn,Ayacucho Quechua,271,0.9246,0.9331,0.9794,0.9794,0.
|
| 112 |
-
srp_Cyrl,Serbian,269,1.0,1.0,0.9834,0.9834,0.
|
| 113 |
-
nob_Latn,Norwegian Bokmål,265,0.9612,0.9594,0.8942,0.8942,0.
|
| 114 |
-
kor_Hang,Korean,265,1.0,1.0,0.8237,0.8237,1.0,1.0,1.0,1.0,
|
| 115 |
-
tum_Latn,Tumbuka,265,0.9962,0.9962,0.9662,0.9662,0.
|
| 116 |
-
khk_Cyrl,Halh Mongolian,262,1.0,1.0,0.9205,0.9205,0.
|
| 117 |
-
nya_Latn,Nyanja,262,0.9786,0.9786,0.9259,0.9259,0.
|
| 118 |
-
nld_Latn,Dutch,259,0.9719,0.9719,0.9597,0.9597,0.
|
| 119 |
-
ltg_Latn,Latgalian,259,0.9981,0.9981,0.9647,0.9647,0.
|
| 120 |
-
ibo_Latn,Igbo,257,0.9981,0.9981,0.9922,0.9922,0.
|
| 121 |
-
bis_Latn,Bislama,257,1.0,1.0,0.9681,0.9681,0.
|
| 122 |
-
urd_Arab,Urdu,257,1.0,1.0,0.9345,0.9345,0.
|
| 123 |
-
pol_Latn,Polish,254,0.9844,0.9844,0.92,0.92,0.
|
| 124 |
-
eus_Latn,Basque,254,0.9922,0.9922,0.9862,0.9862,0.
|
| 125 |
-
ilo_Latn,Iloko,252,1.0,1.0,0.9735,0.9735,0.
|
| 126 |
-
ltz_Latn,Luxembourgish,251,1.0,1.0,0.9544,0.9544,0.
|
| 127 |
-
pcm_Latn,Nigerian Pidgin,251,0.838,0.8559,0.5015,0.5407,0.8228,0.8344,0.
|
| 128 |
-
arn_Latn,Mapuche,250,1.0,1.0,0.9585,0.9585,0.
|
| 129 |
-
plt_Latn,Malagasy (Plateau),248,0.998,0.998,0.9818,0.9818,0.
|
| 130 |
-
lij_Latn,Ligurian,245,0.998,0.998,0.8571,0.8571,0.
|
| 131 |
-
ast_Latn,Asturian,244,0.9702,0.9702,0.9333,0.9333,0.
|
| 132 |
-
zsm_Latn,Standard Malay,244,0.998,0.998,0.9079,0.9079,0.
|
| 133 |
-
nia_Latn,Nias,242,1.0,1.0,0.9876,0.9876,0.
|
| 134 |
-
dan_Latn,Danish,242,0.9499,0.9499,0.9623,0.9623,0.
|
| 135 |
-
kir_Cyrl,Kyrgyz,242,1.0,1.0,0.9279,0.9279,0.
|
| 136 |
-
tah_Latn,Tahitian,242,1.0,1.0,0.9917,0.9917,0.
|
| 137 |
-
san_Deva,Sanskrit,239,0.9895,0.9895,0.9705,0.9705,0.9895,0.9958,0.
|
| 138 |
-
fao_Latn,Faroese,237,1.0,1.0,0.4808,0.4808,0.
|
| 139 |
-
sag_Latn,Sango,237,1.0,1.0,0.985,0.985,0.
|
| 140 |
-
mai_Deva,Maithili,234,0.9957,0.9957,0.9196,0.9196,0.
|
| 141 |
-
sin_Sinh,Sinhala,232,1.0,1.0,0.9978,0.9978,1.0,1.0,1.0,1.0,
|
| 142 |
-
gug_Latn,Paraguayan Guaraní,228,0.9956,0.9956,0.933,0.933,0.
|
| 143 |
-
tuk_Latn,Turkmen,227,0.9978,0.9978,0.9821,0.9821,0.
|
| 144 |
-
gla_Latn,Scottish Gaelic,223,0.9955,0.9955,0.9818,0.9818,0.9888,0.9911,0.
|
| 145 |
-
bem_Latn,Bemba,222,0.9841,0.9818,0.9643,0.9643,0.
|
| 146 |
-
fur_Latn,Friulian,221,0.9955,0.9955,0.9138,0.9074,0.
|
| 147 |
-
kac_Latn,Kachin,216,1.0,1.0,0.986,0.986,0.9977,1.0,
|
| 148 |
-
glg_Latn,Galician,215,0.9663,0.9663,0.9316,0.9316,0.
|
| 149 |
-
lin_Latn,Lingala,215,0.9953,0.9953,0.9838,0.9838,0.
|
| 150 |
-
wol_Latn,Wolof,214,0.9953,0.9953,0.9555,0.9579,0.
|
| 151 |
-
als_Latn,Albanian (Tosk),213,1.0,1.0,0.9812,0.9812,0.
|
| 152 |
-
kin_Latn,Kinyarwanda,210,0.9807,0.9807,0.8535,0.8535,0.
|
| 153 |
-
alt_Cyrl,Southern Altai,208,1.0,1.0,0.8966,0.8966,0.
|
| 154 |
-
kea_Latn,Kabuverdianu,205,0.9976,0.9976,0.8736,0.8736,0.
|
| 155 |
-
heb_Hebr,Hebrew,203,1.0,1.0,0.9975,0.9975,0.
|
| 156 |
-
gle_Latn,Irish,203,1.0,1.0,0.9826,0.9826,0.
|
| 157 |
-
ban_Latn,Balinese,202,1.0,1.0,0.9223,0.9223,0.
|
| 158 |
-
tat_Cyrl,Tatar,201,1.0,1.0,0.9685,0.9685,0.
|
| 159 |
-
nus_Latn,Nuer,199,1.0,1.0,0.9898,0.9898,0.
|
| 160 |
-
bos_Latn,Bosnian,198,1.0,1.0,0.8967,0.8967,0.
|
| 161 |
-
hne_Deva,Chhattisgarhi,198,0.9851,0.9851,0.9016,0.9016,0.
|
| 162 |
-
lim_Latn,Limburgish,196,1.0,1.0,0.9556,0.9556,0.
|
| 163 |
-
yue_Hani,Cantonese,194,0.8983,0.9014,0.2588,0.2588,0.
|
| 164 |
-
war_Latn,Waray,193,1.0,1.0,0.9511,0.9511,0.
|
| 165 |
-
kik_Latn,Kikuyu,193,1.0,1.0,0.8478,0.8478,0.
|
| 166 |
-
crh_Latn,Crimean Tatar,189,1.0,1.0,0.9786,0.9786,0.
|
| 167 |
-
cym_Latn,Welsh,189,1.0,1.0,0.9867,0.9867,0.
|
| 168 |
-
khm_Khmr,Khmer,187,1.0,1.0,1.0,1.0,
|
| 169 |
-
azj_Latn,North Azerbaijani,187,0.9973,0.9973,0.9787,0.9787,0.
|
| 170 |
-
ces_Latn,Czech,183,1.0,1.0,0.9683,0.9683,0.
|
| 171 |
-
fon_Latn,Fon,180,1.0,1.0,0.9535,0.9535,0.
|
| 172 |
-
hye_Armn,Armenian,178,1.0,1.0,0.977,0.977,1.0,1.0,
|
| 173 |
-
shn_Mymr,Shan,174,1.0,1.0,0.9854,0.9854,
|
| 174 |
-
scn_Latn,Sicilian,172,1.0,1.0,0.9243,0.9243,0.
|
| 175 |
-
gom_Deva,Goan Konkani,170,0.9798,0.9798,0.9524,0.9524,0.
|
| 176 |
-
hrv_Latn,Croatian,168,0.9941,0.9941,0.8563,0.8563,0.
|
| 177 |
-
kab_Latn,Kabyle,165,1.0,1.0,0.9558,0.9558,0.
|
| 178 |
-
fuv_Latn,Nigerian Fulfulde,164,0.8767,0.8919,0.8587,0.8587,0.
|
| 179 |
-
arb_Latn,Standard Arabic,163,0.906,0.9133,0.8522,0.8522,0.
|
| 180 |
-
mri_Latn,Māori,162,1.0,1.0,0.6639,0.6639,0.
|
| 181 |
-
udm_Cyrl,Udmurt,159,1.0,1.0,0.8244,0.8244,0.
|
| 182 |
-
krc_Cyrl,Karachay-Balkar,159,1.0,1.0,0.8778,0.8778,0.
|
| 183 |
-
jav_Latn,Javanese,158,0.9937,0.9937,0.9313,0.9313,0.
|
| 184 |
-
tzm_Tfng,Central Atlas Tamazight,157,0.9345,0.9345,0.7512,0.7512,0.
|
| 185 |
-
bel_Cyrl,Belarusian,157,1.0,1.0,0.9937,0.9937,0.
|
| 186 |
-
bjn_Latn,Banjar,153,0.9967,0.9967,0.8688,0.8688,0.
|
| 187 |
-
oss_Cyrl,Ossetic,152,1.0,1.0,1.0,1.0,0.
|
| 188 |
-
sun_Latn,Sundanese,143,1.0,1.0,0.9467,0.9467,0.
|
| 189 |
-
srd_Latn,Sardinian,139,1.0,1.0,0.9363,0.9363,0.
|
| 190 |
-
fil_Latn,Filipino,137,0.9815,0.9815,0.9926,0.9926,0.
|
| 191 |
-
ady_Cyrl,Adyghe,135,1.0,1.0,0.9814,0.9814,0.
|
| 192 |
-
diq_Latn,Dimli (individual language),134,1.0,1.0,0.9333,0.9333,0.
|
| 193 |
-
ewo_Latn,Ewondo,134,0.8787,0.8833,0.8596,0.8596,0.
|
| 194 |
-
mos_Latn,Mossi,133,1.0,1.0,0.9811,0.9811,0.
|
| 195 |
-
xmf_Geor,Mingrelian,132,1.0,1.0,0.963,0.963,0.
|
| 196 |
-
kbp_Latn,Kabiyè,130,1.0,1.0,0.9766,0.9766,
|
| 197 |
-
pag_Latn,Pangasinan,122,0.9959,0.9959,0.959,0.959,0.
|
| 198 |
-
sat_Olck,Santali,122,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,
|
| 199 |
-
kmb_Latn,Kimbundu,119,0.9917,0.9917,0.9633,0.9633,0.9833,0.9833,
|
| 200 |
-
kas_Arab,Kashmiri,118,1.0,1.0,0.9784,0.9784,
|
| 201 |
-
bjn_Arab,Banjar,116,0.8342,0.9869,0.4533,0.9596,0.
|
| 202 |
-
ace_Latn,Acehnese,113,1.0,1.0,0.8943,0.88,0.
|
| 203 |
-
nav_Latn,Navajo,113,1.0,1.0,0.968,0.968,
|
| 204 |
-
taq_Latn,Tamasheq,106,1.0,1.0,0.8377,0.8377,0.
|
| 205 |
-
taq_Tfng,Tamasheq,104,0.8046,0.8046,0.0,0.0,0.
|
| 206 |
-
umb_Latn,Umbundu,100,0.995,0.995,0.8972,0.8972,0.
|
| 207 |
-
mag_Deva,Magahi,97,1.0,1.0,0.7723,0.7723,0.
|
| 208 |
-
awa_Deva,Awadhi,91,0.963,0.963,0.7571,0.7571,0.
|
| 209 |
-
bug_Latn,Buginese,78,0.9811,0.9811,0.9375,0.9375,0.
|
| 210 |
-
knc_Arab,Central Kanuri,68,0.8571,0.8595,0.029,0.029,0.
|
| 211 |
-
min_Latn,Minangkabau,55,0.991,0.991,0.3333,0.3333,0.
|
| 212 |
-
vec_Latn,Venetian,49,0.8605,0.8605,0.6202,0.6202,0.
|
| 213 |
-
kas_Deva,Kashmiri,47,0.7027,0.7027,0.5867,0.5867,0.
|
| 214 |
-
cjk_Latn,Chokwe,24,1.0,1.0,0.807,0.807,0.
|
|
|
|
| 1 |
+
label,language,support,GlotLID,GlotLID restricted,fastText,fastText restricted,MillID tiny v1.5,MillID medium v1,MillID large v1.5,best MillID,best MillID model,gain over GlotLID restricted,gain over fastText restricted
|
| 2 |
+
fra_Latn,French,1652,0.9548,0.9548,0.9632,0.9632,0.943,0.9737,0.9831,0.9831,MillID large v1.5,0.0283,0.0199
|
| 3 |
+
eng_Latn,English,1444,0.876,0.8768,0.8574,0.8571,0.9324,0.9414,0.9525,0.9525,MillID large v1.5,0.0757,0.0954
|
| 4 |
+
spa_Latn,Spanish,1136,0.9583,0.9583,0.9407,0.9407,0.937,0.953,0.9719,0.9719,MillID large v1.5,0.0136,0.0312
|
| 5 |
+
tam_Taml,Tamil,754,1.0,1.0,0.9973,0.9973,0.9973,0.998,0.9987,0.9987,MillID large v1.5,-0.0013,0.0014
|
| 6 |
+
arb_Arab,Arabic (Modern Standard),745,0.8864,0.8864,0.9573,0.9542,0.8712,0.905,0.9046,0.905,MillID medium v1,0.0186,-0.0492
|
| 7 |
+
por_Latn,Portuguese,657,0.9538,0.9562,0.95,0.9493,0.9589,0.9628,0.9731,0.9731,MillID large v1.5,0.0169,0.0238
|
| 8 |
+
kan_Knda,Kannada,601,0.9941,0.9941,0.9992,0.9992,0.9726,0.9865,0.9665,0.9865,MillID medium v1,-0.0076,-0.0127
|
| 9 |
+
zul_Latn,Zulu,594,0.9042,0.9042,0.9704,0.9704,0.883,0.8713,0.9094,0.9094,MillID large v1.5,0.0052,-0.061
|
| 10 |
+
swh_Latn,Swahili,592,0.9536,0.9528,0.9454,0.9454,0.893,0.8967,0.9094,0.9094,MillID large v1.5,-0.0434,-0.036
|
| 11 |
+
mar_Deva,Marathi,589,0.9888,0.9897,0.9871,0.9871,0.9683,0.9811,0.9785,0.9811,MillID medium v1,-0.0086,-0.006
|
| 12 |
+
tso_Latn,Tsonga,581,0.956,0.956,0.9861,0.9861,0.9789,0.9843,0.9852,0.9852,MillID large v1.5,0.0292,-0.0009
|
| 13 |
+
hun_Latn,Hungarian,576,1.0,1.0,0.9804,0.9804,0.9886,0.9913,0.993,0.993,MillID large v1.5,-0.007,0.0126
|
| 14 |
+
afr_Latn,Afrikaans,569,0.9849,0.9849,0.9858,0.9858,0.9716,0.976,0.9831,0.9831,MillID large v1.5,-0.0018,-0.0027
|
| 15 |
+
tel_Telu,Telugu,555,0.9982,0.9982,0.9991,0.9991,0.9973,1.0,0.9991,1.0,MillID medium v1,0.0018,0.0009
|
| 16 |
+
rus_Cyrl,Russian,528,0.9846,0.9846,0.9271,0.9271,0.9453,0.9291,0.9557,0.9557,MillID large v1.5,-0.0289,0.0286
|
| 17 |
+
ory_Orya,Odia (individual language),511,0.9971,1.0,0.9961,0.9961,0.9941,0.9961,0.9951,0.9961,MillID medium v1,-0.0039,0.0
|
| 18 |
+
sot_Latn,Southern Sotho,491,0.9774,0.9784,0.8562,0.8562,0.9694,0.9665,0.9737,0.9737,MillID large v1.5,-0.0047,0.1175
|
| 19 |
+
nso_Latn,Northern Sotho,479,0.9695,0.9695,0.9673,0.9673,0.9579,0.9614,0.9635,0.9635,MillID large v1.5,-0.006,-0.0038
|
| 20 |
+
asm_Beng,Assamese,467,0.9989,0.9989,0.9968,0.9968,0.9968,0.9946,0.9968,0.9968,MillID tiny v1.5,-0.0021,0.0
|
| 21 |
+
ben_Beng,Bangla,466,0.9968,0.9968,0.9894,0.9894,0.9882,0.9871,0.9925,0.9925,MillID large v1.5,-0.0043,0.0031
|
| 22 |
+
pan_Guru,Punjabi,460,1.0,1.0,0.9901,0.9901,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0099
|
| 23 |
+
amh_Ethi,Amharic,458,1.0,1.0,0.9978,0.9978,0.9835,0.9869,0.9857,0.9869,MillID medium v1,-0.0131,-0.0109
|
| 24 |
+
tsn_Latn,Tswana,456,0.9605,0.9605,0.8698,0.8698,0.9475,0.9397,0.9467,0.9475,MillID tiny v1.5,-0.013,0.0777
|
| 25 |
+
gaz_Latn,Oromo (West Central),441,0.9744,0.9744,0.9733,0.9733,0.9354,0.9552,0.9413,0.9552,MillID medium v1,-0.0192,-0.0181
|
| 26 |
+
som_Latn,Somali,439,0.9954,0.9954,0.9909,0.9909,0.9839,0.992,0.9909,0.992,MillID medium v1,-0.0034,0.0011
|
| 27 |
+
uzn_Latn,Northern Uzbek,436,0.9989,0.9989,0.9849,0.9849,0.9839,0.9838,0.9861,0.9861,MillID large v1.5,-0.0128,0.0012
|
| 28 |
+
tha_Thai,Thai,431,0.9799,0.9799,0.9811,0.9811,0.9799,0.9701,0.9811,0.9811,MillID large v1.5,0.0012,0.0
|
| 29 |
+
ydd_Hebr,Eastern Yiddish,430,1.0,1.0,0.9988,0.9988,0.993,0.9977,0.9942,0.9977,MillID medium v1,-0.0023,-0.0011
|
| 30 |
+
tur_Latn,Turkish,430,0.9988,0.9988,0.9803,0.9803,0.968,0.9742,0.9731,0.9742,MillID medium v1,-0.0246,-0.0061
|
| 31 |
+
nno_Latn,Norwegian Nynorsk,427,0.9845,0.9845,0.9542,0.9542,0.9132,0.9189,0.9273,0.9273,MillID large v1.5,-0.0572,-0.0269
|
| 32 |
+
smo_Latn,Samoan,426,0.9988,0.9988,0.9372,0.9372,0.9905,0.9917,0.9977,0.9977,MillID large v1.5,-0.0011,0.0605
|
| 33 |
+
pap_Latn,Papiamento,423,0.9988,0.9988,0.9742,0.9731,0.9784,0.9709,0.9809,0.9809,MillID large v1.5,-0.0179,0.0078
|
| 34 |
+
ckb_Arab,Central Kurdish,421,0.9769,0.9769,0.9952,0.9952,0.9554,0.9619,0.9757,0.9757,MillID large v1.5,-0.0012,-0.0195
|
| 35 |
+
pbt_Arab,Southern Pashto,419,0.9988,0.9988,0.9782,0.977,0.9855,0.994,0.9881,0.994,MillID medium v1,-0.0048,0.017
|
| 36 |
+
ekk_Latn,Estonian,419,0.9929,0.9929,0.9857,0.9857,0.9709,0.9806,0.9782,0.9806,MillID medium v1,-0.0123,-0.0051
|
| 37 |
+
sna_Latn,Shona,415,1.0,1.0,0.9831,0.9831,0.9817,0.9779,0.973,0.9817,MillID tiny v1.5,-0.0183,-0.0014
|
| 38 |
+
luo_Latn,Luo (Kenya and Tanzania),415,1.0,1.0,0.9564,0.9564,0.9879,0.9841,0.989,0.989,MillID large v1.5,-0.011,0.0326
|
| 39 |
+
tgk_Cyrl,Tajik,413,1.0,1.0,0.983,0.983,0.9951,0.9927,0.9915,0.9951,MillID tiny v1.5,-0.0049,0.0121
|
| 40 |
+
bod_Tibt,Tibetan,406,0.9988,0.9988,0.8191,0.8191,0.9354,0.9503,0.9433,0.9503,MillID medium v1,-0.0485,0.1312
|
| 41 |
+
lug_Latn,Ganda,405,0.9863,0.9876,0.979,0.979,0.9682,0.9748,0.9862,0.9862,MillID large v1.5,-0.0014,0.0072
|
| 42 |
+
fas_Arab,Persian,396,0.9859,0.9859,0.9439,0.9439,0.945,0.9342,0.9287,0.945,MillID tiny v1.5,-0.0409,0.0011
|
| 43 |
+
ell_Grek,Greek,394,0.9987,0.9987,0.97,0.97,0.901,0.9086,0.9322,0.9322,MillID large v1.5,-0.0665,-0.0378
|
| 44 |
+
hin_Deva,Hindi,392,0.8768,0.8768,0.863,0.863,0.8552,0.8171,0.8932,0.8932,MillID large v1.5,0.0164,0.0302
|
| 45 |
+
lus_Latn,Mizo,391,0.9974,0.9974,0.9897,0.9897,0.9831,0.9738,0.9778,0.9831,MillID tiny v1.5,-0.0143,-0.0066
|
| 46 |
+
dyu_Latn,Dyula,390,1.0,1.0,0.8223,0.8223,0.9658,0.9729,0.9832,0.9832,MillID large v1.5,-0.0168,0.1609
|
| 47 |
+
slk_Latn,Slovak,374,0.9987,0.9987,0.9714,0.9714,0.9515,0.9486,0.966,0.966,MillID large v1.5,-0.0327,-0.0054
|
| 48 |
+
ron_Latn,Romanian,369,0.9959,0.9959,0.9759,0.9759,0.9672,0.9755,0.9685,0.9755,MillID medium v1,-0.0204,-0.0004
|
| 49 |
+
kal_Latn,Kalaallisut,369,0.9986,0.9986,0.9932,0.9932,0.9986,0.9986,0.9973,0.9986,MillID tiny v1.5,0.0,0.0054
|
| 50 |
+
azb_Arab,South Azerbaijani,369,0.9959,0.9959,0.9711,0.9711,0.9586,0.9622,0.9653,0.9653,MillID large v1.5,-0.0306,-0.0058
|
| 51 |
+
fin_Latn,Finnish,368,1.0,1.0,0.9946,0.9946,0.9291,0.9548,0.9444,0.9548,MillID medium v1,-0.0452,-0.0398
|
| 52 |
+
hau_Latn,Hausa,367,0.9932,0.9932,0.9851,0.9837,0.971,0.9753,0.9835,0.9835,MillID large v1.5,-0.0097,-0.0002
|
| 53 |
+
xho_Latn,Xhosa,367,0.9481,0.9481,0.9511,0.9511,0.8953,0.8617,0.8994,0.8994,MillID large v1.5,-0.0487,-0.0517
|
| 54 |
+
kng_Latn,Koongo,362,1.0,1.0,0.9718,0.9718,0.9446,0.9322,0.9568,0.9568,MillID large v1.5,-0.0432,-0.015
|
| 55 |
+
twi_Latn,Twi,359,0.9916,0.9902,0.9684,0.9684,0.9773,0.9744,0.9745,0.9773,MillID tiny v1.5,-0.0129,0.0089
|
| 56 |
+
deu_Latn,German,356,0.9972,0.9972,0.9889,0.9889,0.9802,0.9597,0.9657,0.9802,MillID tiny v1.5,-0.017,-0.0087
|
| 57 |
+
guj_Gujr,Gujarati,354,1.0,1.0,0.9842,0.9842,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0158
|
| 58 |
+
dik_Latn,Southwestern Dinka,350,1.0,1.0,0.99,0.99,0.9928,0.9957,0.9928,0.9957,MillID medium v1,-0.0043,0.0057
|
| 59 |
+
lua_Latn,Luba-Lulua,346,0.9971,0.9971,0.9633,0.9633,0.9794,0.9809,0.9868,0.9868,MillID large v1.5,-0.0103,0.0235
|
| 60 |
+
dzo_Tibt,Dzongkha,346,0.9986,0.9986,0.6958,0.6958,0.9061,0.9265,0.9163,0.9265,MillID medium v1,-0.0721,0.2307
|
| 61 |
+
che_Cyrl,Chechen,345,1.0,1.0,0.984,0.984,0.9778,0.9898,0.9927,0.9927,MillID large v1.5,-0.0073,0.0087
|
| 62 |
+
fij_Latn,Fijian,344,1.0,1.0,0.9884,0.9884,0.9883,0.9942,0.9927,0.9942,MillID medium v1,-0.0058,0.0058
|
| 63 |
+
ita_Latn,Italian,342,0.9838,0.9838,0.949,0.949,0.9216,0.9259,0.9388,0.9388,MillID large v1.5,-0.045,-0.0102
|
| 64 |
+
mya_Mymr,Burmese,342,1.0,1.0,0.9897,0.9897,0.9745,0.9912,0.9837,0.9912,MillID medium v1,-0.0088,0.0015
|
| 65 |
+
kaz_Cyrl,Kazakh,340,1.0,1.0,0.9926,0.9926,0.9836,0.9852,0.9882,0.9882,MillID large v1.5,-0.0118,-0.0044
|
| 66 |
+
tir_Ethi,Tigrinya,339,0.9881,0.9985,0.9956,0.9956,0.9824,0.9745,0.9825,0.9825,MillID large v1.5,-0.016,-0.0131
|
| 67 |
+
mal_Mlym,Malayalam,338,1.0,1.0,0.994,0.994,1.0,0.9985,0.9985,1.0,MillID tiny v1.5,0.0,0.006
|
| 68 |
+
hat_Latn,Haitian Creole,338,0.997,0.997,0.9774,0.9774,0.9865,0.982,0.9925,0.9925,MillID large v1.5,-0.0045,0.0151
|
| 69 |
+
ceb_Latn,Cebuano,338,1.0,1.0,0.9926,0.9926,0.9807,0.9851,0.985,0.9851,MillID medium v1,-0.0149,-0.0075
|
| 70 |
+
vie_Latn,Vietnamese,338,1.0,1.0,0.9836,0.9836,0.9519,0.9521,0.9742,0.9742,MillID large v1.5,-0.0258,-0.0094
|
| 71 |
+
slv_Latn,Slovenian,338,0.9927,0.9927,0.9653,0.9653,0.96,0.9669,0.9759,0.9759,MillID large v1.5,-0.0168,0.0106
|
| 72 |
+
lao_Laoo,Lao,335,1.0,1.0,0.991,0.991,0.997,0.9985,1.0,1.0,MillID large v1.5,0.0,0.009
|
| 73 |
+
chv_Cyrl,Chuvash,335,1.0,1.0,0.9926,0.9926,0.9985,0.9955,0.9955,0.9985,MillID tiny v1.5,-0.0015,0.0059
|
| 74 |
+
ind_Latn,Indonesian,335,0.9911,0.9911,0.8873,0.8873,0.8419,0.8067,0.8479,0.8479,MillID large v1.5,-0.1432,-0.0394
|
| 75 |
+
uig_Arab,Uyghur,334,1.0,1.0,0.9911,0.9911,0.9955,0.9925,0.9985,0.9985,MillID large v1.5,-0.0015,0.0074
|
| 76 |
+
oci_Latn,Occitan,333,0.9911,0.9911,0.9619,0.9619,0.9574,0.954,0.956,0.9574,MillID tiny v1.5,-0.0337,-0.0045
|
| 77 |
+
mkd_Cyrl,Macedonian,331,1.0,1.0,0.9924,0.9924,0.9727,0.9685,0.9669,0.9727,MillID tiny v1.5,-0.0273,-0.0197
|
| 78 |
+
lit_Latn,Lithuanian,330,0.9955,0.9955,0.974,0.974,0.9676,0.9848,0.9909,0.9909,MillID large v1.5,-0.0046,0.0169
|
| 79 |
+
swe_Latn,Swedish,328,0.9717,0.9717,0.9767,0.9767,0.9588,0.9538,0.9594,0.9594,MillID large v1.5,-0.0123,-0.0173
|
| 80 |
+
snd_Arab,Sindhi,325,1.0,1.0,0.9848,0.9833,0.9891,0.9938,0.9939,0.9939,MillID large v1.5,-0.0061,0.0106
|
| 81 |
+
kat_Geor,Georgian,324,1.0,1.0,0.9859,0.9859,0.9954,0.9892,0.9938,0.9954,MillID tiny v1.5,-0.0046,0.0095
|
| 82 |
+
jpn_Jpan,Japanese,323,0.9969,0.9969,0.8684,0.8684,0.9746,0.9746,0.9762,0.9762,MillID large v1.5,-0.0207,0.1078
|
| 83 |
+
ssw_Latn,Swati,322,0.9684,0.97,0.9721,0.9721,0.9537,0.9525,0.9522,0.9537,MillID tiny v1.5,-0.0163,-0.0184
|
| 84 |
+
ukr_Cyrl,Ukrainian,322,1.0,1.0,0.9783,0.9783,0.9617,0.9762,0.9633,0.9762,MillID medium v1,-0.0238,-0.0021
|
| 85 |
+
szl_Latn,Silesian,320,0.9969,0.9969,0.9254,0.9254,0.9556,0.9688,0.9796,0.9796,MillID large v1.5,-0.0173,0.0542
|
| 86 |
+
cat_Latn,Catalan,318,0.9845,0.9845,0.9528,0.9528,0.9497,0.9662,0.9683,0.9683,MillID large v1.5,-0.0162,0.0155
|
| 87 |
+
ayr_Latn,Central Aymara,309,1.0,1.0,0.9752,0.9752,0.9886,0.9935,0.9951,0.9951,MillID large v1.5,-0.0049,0.0199
|
| 88 |
+
yor_Latn,Yoruba,308,0.9984,0.9984,0.9683,0.9683,0.987,0.9885,0.9852,0.9885,MillID medium v1,-0.0099,0.0202
|
| 89 |
+
ewe_Latn,Ewe,308,1.0,1.0,0.9852,0.9852,0.9902,0.9869,0.9951,0.9951,MillID large v1.5,-0.0049,0.0099
|
| 90 |
+
roh_Latn,Romansh,307,1.0,1.0,0.9742,0.9742,0.9789,0.9854,0.9902,0.9902,MillID large v1.5,-0.0098,0.016
|
| 91 |
+
mni_Beng,Manipuri,304,1.0,1.0,0.9499,0.9499,0.9501,0.9573,0.959,0.959,MillID large v1.5,-0.041,0.0091
|
| 92 |
+
tpi_Latn,Tok Pisin,302,0.9983,0.9983,0.9867,0.9867,0.9934,0.9967,0.9983,0.9983,MillID large v1.5,0.0,0.0116
|
| 93 |
+
lvs_Latn,Standard Latvian,298,0.995,0.995,0.9472,0.9472,0.9603,0.9696,0.9763,0.9763,MillID large v1.5,-0.0187,0.0291
|
| 94 |
+
ton_Latn,Tongan,298,1.0,1.0,0.0,0.0,0.9966,0.9915,0.9949,0.9966,MillID tiny v1.5,-0.0034,0.9966
|
| 95 |
+
epo_Latn,Esperanto,294,0.9983,0.9983,0.9882,0.9882,0.976,0.9724,0.9828,0.9828,MillID large v1.5,-0.0155,-0.0054
|
| 96 |
+
knc_Latn,Central Kanuri,293,0.9517,0.9517,0.9811,0.9811,0.9666,0.9594,0.9666,0.9666,MillID tiny v1.5,0.0149,-0.0145
|
| 97 |
+
run_Latn,Rundi,291,0.9848,0.9848,0.8545,0.8545,0.8832,0.8425,0.8787,0.8832,MillID tiny v1.5,-0.1016,0.0287
|
| 98 |
+
mlt_Latn,Maltese,291,0.9932,0.9932,0.9614,0.9614,0.9948,0.9966,0.9914,0.9966,MillID medium v1,0.0034,0.0352
|
| 99 |
+
bho_Deva,Bhojpuri,290,0.9983,0.9983,0.8732,0.8732,0.9556,0.9453,0.9522,0.9556,MillID tiny v1.5,-0.0427,0.0824
|
| 100 |
+
bak_Cyrl,Bashkir,289,1.0,1.0,0.9846,0.9846,0.9825,0.9842,0.9895,0.9895,MillID large v1.5,-0.0105,0.0049
|
| 101 |
+
kmr_Latn,Northern Kurdish,289,1.0,1.0,0.9747,0.9747,0.9791,0.9718,0.9606,0.9791,MillID tiny v1.5,-0.0209,0.0044
|
| 102 |
+
bul_Cyrl,Bulgarian,289,1.0,1.0,0.9827,0.9827,0.9636,0.9532,0.9704,0.9704,MillID large v1.5,-0.0296,-0.0123
|
| 103 |
+
cmn_Hani,Chinese (Mandarin),288,0.9663,0.9663,0.4884,0.4884,0.9149,0.8913,0.9133,0.9149,MillID tiny v1.5,-0.0514,0.4265
|
| 104 |
+
bam_Latn,Bambara,287,0.9983,0.9983,0.7923,0.7923,0.9521,0.9615,0.9826,0.9826,MillID large v1.5,-0.0157,0.1903
|
| 105 |
+
abk_Cyrl,Abkhazian,285,1.0,1.0,1.0,1.0,0.9965,0.9982,1.0,1.0,MillID large v1.5,0.0,0.0
|
| 106 |
+
bxr_Cyrl,Russia Buriat,277,1.0,1.0,0.9387,0.9387,0.9726,0.9726,0.9835,0.9835,MillID large v1.5,-0.0165,0.0448
|
| 107 |
+
isl_Latn,Icelandic,276,1.0,1.0,0.7886,0.7886,0.949,0.9683,0.963,0.9683,MillID medium v1,-0.0317,0.1797
|
| 108 |
+
lmo_Latn,Lombard,276,0.9875,0.9875,0.9097,0.9081,0.9149,0.9199,0.9316,0.9316,MillID large v1.5,-0.0559,0.0235
|
| 109 |
+
npi_Deva,Nepali,273,0.9908,0.9927,0.9745,0.9745,0.9649,0.9703,0.9743,0.9743,MillID large v1.5,-0.0184,-0.0002
|
| 110 |
+
kam_Latn,Kamba,272,0.9716,0.9736,0.8625,0.8625,0.9795,0.9795,0.9889,0.9889,MillID large v1.5,0.0153,0.1264
|
| 111 |
+
quy_Latn,Ayacucho Quechua,271,0.9246,0.9331,0.9794,0.9794,0.9398,0.9289,0.9095,0.9398,MillID tiny v1.5,0.0067,-0.0396
|
| 112 |
+
srp_Cyrl,Serbian,269,1.0,1.0,0.9834,0.9834,0.974,0.9774,0.9704,0.9774,MillID medium v1,-0.0226,-0.006
|
| 113 |
+
nob_Latn,Norwegian Bokmål,265,0.9612,0.9594,0.8942,0.8942,0.8207,0.8195,0.8321,0.8321,MillID large v1.5,-0.1273,-0.0621
|
| 114 |
+
kor_Hang,Korean,265,1.0,1.0,0.8237,0.8237,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.1763
|
| 115 |
+
tum_Latn,Tumbuka,265,0.9962,0.9962,0.9662,0.9662,0.9773,0.9569,0.975,0.9773,MillID tiny v1.5,-0.0189,0.0111
|
| 116 |
+
khk_Cyrl,Halh Mongolian,262,1.0,1.0,0.9205,0.9205,0.9774,0.9735,0.9905,0.9905,MillID large v1.5,-0.0095,0.07
|
| 117 |
+
nya_Latn,Nyanja,262,0.9786,0.9786,0.9259,0.9259,0.9518,0.9553,0.9509,0.9553,MillID medium v1,-0.0233,0.0294
|
| 118 |
+
nld_Latn,Dutch,259,0.9719,0.9719,0.9597,0.9597,0.9363,0.9435,0.9583,0.9583,MillID large v1.5,-0.0136,-0.0014
|
| 119 |
+
ltg_Latn,Latgalian,259,0.9981,0.9981,0.9647,0.9647,0.9656,0.9674,0.9865,0.9865,MillID large v1.5,-0.0116,0.0218
|
| 120 |
+
ibo_Latn,Igbo,257,0.9981,0.9981,0.9922,0.9922,0.9922,0.9922,0.9922,0.9922,MillID tiny v1.5,-0.0059,0.0
|
| 121 |
+
bis_Latn,Bislama,257,1.0,1.0,0.9681,0.9681,0.9862,0.9882,0.9843,0.9882,MillID medium v1,-0.0118,0.0201
|
| 122 |
+
urd_Arab,Urdu,257,1.0,1.0,0.9345,0.9345,0.9622,0.9704,0.9705,0.9705,MillID large v1.5,-0.0295,0.036
|
| 123 |
+
pol_Latn,Polish,254,0.9844,0.9844,0.92,0.92,0.9565,0.9644,0.9706,0.9706,MillID large v1.5,-0.0138,0.0506
|
| 124 |
+
eus_Latn,Basque,254,0.9922,0.9922,0.9862,0.9862,0.9782,0.9841,0.9881,0.9881,MillID large v1.5,-0.0041,0.0019
|
| 125 |
+
ilo_Latn,Iloko,252,1.0,1.0,0.9735,0.9735,0.9901,0.986,0.992,0.992,MillID large v1.5,-0.008,0.0185
|
| 126 |
+
ltz_Latn,Luxembourgish,251,1.0,1.0,0.9544,0.9544,0.9742,0.9839,0.994,0.994,MillID large v1.5,-0.006,0.0396
|
| 127 |
+
pcm_Latn,Nigerian Pidgin,251,0.838,0.8559,0.5015,0.5407,0.8228,0.8344,0.8119,0.8344,MillID medium v1,-0.0215,0.2937
|
| 128 |
+
arn_Latn,Mapuche,250,1.0,1.0,0.9585,0.9585,0.992,0.994,0.998,0.998,MillID large v1.5,-0.002,0.0395
|
| 129 |
+
plt_Latn,Malagasy (Plateau),248,0.998,0.998,0.9818,0.9818,0.9691,0.9836,0.9878,0.9878,MillID large v1.5,-0.0102,0.006
|
| 130 |
+
lij_Latn,Ligurian,245,0.998,0.998,0.8571,0.8571,0.9379,0.9552,0.9542,0.9552,MillID medium v1,-0.0428,0.0981
|
| 131 |
+
ast_Latn,Asturian,244,0.9702,0.9702,0.9333,0.9333,0.9188,0.9189,0.973,0.973,MillID large v1.5,0.0028,0.0397
|
| 132 |
+
zsm_Latn,Standard Malay,244,0.998,0.998,0.9079,0.9079,0.7891,0.7726,0.8087,0.8087,MillID large v1.5,-0.1893,-0.0992
|
| 133 |
+
nia_Latn,Nias,242,1.0,1.0,0.9876,0.9876,0.9979,0.9938,0.9979,0.9979,MillID tiny v1.5,-0.0021,0.0103
|
| 134 |
+
dan_Latn,Danish,242,0.9499,0.9499,0.9623,0.9623,0.8874,0.8766,0.88,0.8874,MillID tiny v1.5,-0.0625,-0.0749
|
| 135 |
+
kir_Cyrl,Kyrgyz,242,1.0,1.0,0.9279,0.9279,0.9733,0.9835,0.9917,0.9917,MillID large v1.5,-0.0083,0.0638
|
| 136 |
+
tah_Latn,Tahitian,242,1.0,1.0,0.9917,0.9917,0.9833,0.9855,0.9938,0.9938,MillID large v1.5,-0.0062,0.0021
|
| 137 |
+
san_Deva,Sanskrit,239,0.9895,0.9895,0.9705,0.9705,0.9895,0.9958,0.9979,0.9979,MillID large v1.5,0.0084,0.0274
|
| 138 |
+
fao_Latn,Faroese,237,1.0,1.0,0.4808,0.4808,0.975,0.9727,0.9745,0.975,MillID tiny v1.5,-0.025,0.4942
|
| 139 |
+
sag_Latn,Sango,237,1.0,1.0,0.985,0.985,0.9936,0.9958,1.0,1.0,MillID large v1.5,0.0,0.015
|
| 140 |
+
mai_Deva,Maithili,234,0.9957,0.9957,0.9196,0.9196,0.9258,0.9386,0.9448,0.9448,MillID large v1.5,-0.0509,0.0252
|
| 141 |
+
sin_Sinh,Sinhala,232,1.0,1.0,0.9978,0.9978,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0022
|
| 142 |
+
gug_Latn,Paraguayan Guaraní,228,0.9956,0.9956,0.933,0.933,0.9686,0.9648,0.9664,0.9686,MillID tiny v1.5,-0.027,0.0356
|
| 143 |
+
tuk_Latn,Turkmen,227,0.9978,0.9978,0.9821,0.9821,0.9799,0.98,0.989,0.989,MillID large v1.5,-0.0088,0.0069
|
| 144 |
+
gla_Latn,Scottish Gaelic,223,0.9955,0.9955,0.9818,0.9818,0.9888,0.9911,0.9797,0.9911,MillID medium v1,-0.0044,0.0093
|
| 145 |
+
bem_Latn,Bemba,222,0.9841,0.9818,0.9643,0.9643,0.9729,0.9725,0.9773,0.9773,MillID large v1.5,-0.0045,0.013
|
| 146 |
+
fur_Latn,Friulian,221,0.9955,0.9955,0.9138,0.9074,0.9484,0.9528,0.9535,0.9535,MillID large v1.5,-0.042,0.0461
|
| 147 |
+
kac_Latn,Kachin,216,1.0,1.0,0.986,0.986,0.9977,1.0,0.9954,1.0,MillID medium v1,0.0,0.014
|
| 148 |
+
glg_Latn,Galician,215,0.9663,0.9663,0.9316,0.9316,0.9517,0.9406,0.9455,0.9517,MillID tiny v1.5,-0.0146,0.0201
|
| 149 |
+
lin_Latn,Lingala,215,0.9953,0.9953,0.9838,0.9838,0.9789,0.9624,0.9791,0.9791,MillID large v1.5,-0.0162,-0.0047
|
| 150 |
+
wol_Latn,Wolof,214,0.9953,0.9953,0.9555,0.9579,0.969,0.9762,0.9763,0.9763,MillID large v1.5,-0.019,0.0184
|
| 151 |
+
als_Latn,Albanian (Tosk),213,1.0,1.0,0.9812,0.9812,0.976,0.9561,0.9687,0.976,MillID tiny v1.5,-0.024,-0.0052
|
| 152 |
+
kin_Latn,Kinyarwanda,210,0.9807,0.9807,0.8535,0.8535,0.8491,0.8178,0.864,0.864,MillID large v1.5,-0.1167,0.0105
|
| 153 |
+
alt_Cyrl,Southern Altai,208,1.0,1.0,0.8966,0.8966,0.9653,0.9655,0.9903,0.9903,MillID large v1.5,-0.0097,0.0937
|
| 154 |
+
kea_Latn,Kabuverdianu,205,0.9976,0.9976,0.8736,0.8736,0.958,0.9662,0.9655,0.9662,MillID medium v1,-0.0314,0.0926
|
| 155 |
+
heb_Hebr,Hebrew,203,1.0,1.0,0.9975,0.9975,0.9752,0.9901,0.9804,0.9901,MillID medium v1,-0.0099,-0.0074
|
| 156 |
+
gle_Latn,Irish,203,1.0,1.0,0.9826,0.9826,0.9776,0.9926,0.9878,0.9926,MillID medium v1,-0.0074,0.01
|
| 157 |
+
ban_Latn,Balinese,202,1.0,1.0,0.9223,0.9223,0.9695,0.9596,0.9695,0.9695,MillID tiny v1.5,-0.0305,0.0472
|
| 158 |
+
tat_Cyrl,Tatar,201,1.0,1.0,0.9685,0.9685,0.9776,0.9755,0.9826,0.9826,MillID large v1.5,-0.0174,0.0141
|
| 159 |
+
nus_Latn,Nuer,199,1.0,1.0,0.9898,0.9898,0.9925,0.995,0.9949,0.995,MillID medium v1,-0.005,0.0052
|
| 160 |
+
bos_Latn,Bosnian,198,1.0,1.0,0.8967,0.8967,0.6852,0.7268,0.7622,0.7622,MillID large v1.5,-0.2378,-0.1345
|
| 161 |
+
hne_Deva,Chhattisgarhi,198,0.9851,0.9851,0.9016,0.9016,0.9381,0.9618,0.9548,0.9618,MillID medium v1,-0.0233,0.0602
|
| 162 |
+
lim_Latn,Limburgish,196,1.0,1.0,0.9556,0.9556,0.896,0.9155,0.9305,0.9305,MillID large v1.5,-0.0695,-0.0251
|
| 163 |
+
yue_Hani,Cantonese,194,0.8983,0.9014,0.2588,0.2588,0.8937,0.8791,0.8929,0.8937,MillID tiny v1.5,-0.0077,0.6349
|
| 164 |
+
war_Latn,Waray,193,1.0,1.0,0.9511,0.9511,0.9462,0.9488,0.957,0.957,MillID large v1.5,-0.043,0.0059
|
| 165 |
+
kik_Latn,Kikuyu,193,1.0,1.0,0.8478,0.8478,0.9922,0.9738,0.9895,0.9922,MillID tiny v1.5,-0.0078,0.1444
|
| 166 |
+
crh_Latn,Crimean Tatar,189,1.0,1.0,0.9786,0.9786,0.9711,0.9842,0.984,0.9842,MillID medium v1,-0.0158,0.0056
|
| 167 |
+
cym_Latn,Welsh,189,1.0,1.0,0.9867,0.9867,0.9893,0.992,0.9894,0.992,MillID medium v1,-0.008,0.0053
|
| 168 |
+
khm_Khmr,Khmer,187,1.0,1.0,1.0,1.0,1.0,0.974,0.9947,1.0,MillID tiny v1.5,0.0,0.0
|
| 169 |
+
azj_Latn,North Azerbaijani,187,0.9973,0.9973,0.9787,0.9787,0.9731,0.9756,0.9866,0.9866,MillID large v1.5,-0.0107,0.0079
|
| 170 |
+
ces_Latn,Czech,183,1.0,1.0,0.9683,0.9683,0.9389,0.9201,0.9451,0.9451,MillID large v1.5,-0.0549,-0.0232
|
| 171 |
+
fon_Latn,Fon,180,1.0,1.0,0.9535,0.9535,0.9833,0.9888,0.9917,0.9917,MillID large v1.5,-0.0083,0.0382
|
| 172 |
+
hye_Armn,Armenian,178,1.0,1.0,0.977,0.977,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.023
|
| 173 |
+
shn_Mymr,Shan,174,1.0,1.0,0.9854,0.9854,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0146
|
| 174 |
+
scn_Latn,Sicilian,172,1.0,1.0,0.9243,0.9243,0.9767,0.9825,0.9883,0.9883,MillID large v1.5,-0.0117,0.064
|
| 175 |
+
gom_Deva,Goan Konkani,170,0.9798,0.9798,0.9524,0.9524,0.948,0.9741,0.9602,0.9741,MillID medium v1,-0.0057,0.0217
|
| 176 |
+
hrv_Latn,Croatian,168,0.9941,0.9941,0.8563,0.8563,0.6588,0.7172,0.7571,0.7571,MillID large v1.5,-0.237,-0.0992
|
| 177 |
+
kab_Latn,Kabyle,165,1.0,1.0,0.9558,0.9558,0.9354,0.9515,0.9541,0.9541,MillID large v1.5,-0.0459,-0.0017
|
| 178 |
+
fuv_Latn,Nigerian Fulfulde,164,0.8767,0.8919,0.8587,0.8587,0.9073,0.7971,0.8443,0.9073,MillID tiny v1.5,0.0154,0.0486
|
| 179 |
+
arb_Latn,Standard Arabic,163,0.906,0.9133,0.8522,0.8522,0.9419,0.9441,0.9494,0.9494,MillID large v1.5,0.0361,0.0972
|
| 180 |
+
mri_Latn,Māori,162,1.0,1.0,0.6639,0.6639,0.9969,0.9969,0.9907,0.9969,MillID tiny v1.5,-0.0031,0.333
|
| 181 |
+
udm_Cyrl,Udmurt,159,1.0,1.0,0.8244,0.8244,0.9779,0.9843,0.9843,0.9843,MillID medium v1,-0.0157,0.1599
|
| 182 |
+
krc_Cyrl,Karachay-Balkar,159,1.0,1.0,0.8778,0.8778,0.9748,0.9841,0.984,0.9841,MillID medium v1,-0.0159,0.1063
|
| 183 |
+
jav_Latn,Javanese,158,0.9937,0.9937,0.9313,0.9313,0.9551,0.9267,0.9677,0.9677,MillID large v1.5,-0.026,0.0364
|
| 184 |
+
tzm_Tfng,Central Atlas Tamazight,157,0.9345,0.9345,0.7512,0.7512,0.6973,0.6452,0.6491,0.6973,MillID tiny v1.5,-0.2372,-0.0539
|
| 185 |
+
bel_Cyrl,Belarusian,157,1.0,1.0,0.9937,0.9937,0.9841,0.9904,0.984,0.9904,MillID medium v1,-0.0096,-0.0033
|
| 186 |
+
bjn_Latn,Banjar,153,0.9967,0.9967,0.8688,0.8688,0.9369,0.9377,0.9568,0.9568,MillID large v1.5,-0.0399,0.088
|
| 187 |
+
oss_Cyrl,Ossetic,152,1.0,1.0,1.0,1.0,0.9868,0.9867,0.98,0.9868,MillID tiny v1.5,-0.0132,-0.0132
|
| 188 |
+
sun_Latn,Sundanese,143,1.0,1.0,0.9467,0.9467,0.9677,0.979,0.9756,0.979,MillID medium v1,-0.021,0.0323
|
| 189 |
+
srd_Latn,Sardinian,139,1.0,1.0,0.9363,0.9363,0.9751,0.9893,0.9749,0.9893,MillID medium v1,-0.0107,0.053
|
| 190 |
+
fil_Latn,Filipino,137,0.9815,0.9815,0.9926,0.9926,0.9466,0.9354,0.9779,0.9779,MillID large v1.5,-0.0036,-0.0147
|
| 191 |
+
ady_Cyrl,Adyghe,135,1.0,1.0,0.9814,0.9814,0.9963,0.9925,0.9963,0.9963,MillID tiny v1.5,-0.0037,0.0149
|
| 192 |
+
diq_Latn,Dimli (individual language),134,1.0,1.0,0.9333,0.9333,0.9339,0.9502,0.9261,0.9502,MillID medium v1,-0.0498,0.0169
|
| 193 |
+
ewo_Latn,Ewondo,134,0.8787,0.8833,0.8596,0.8596,0.9591,0.9653,0.9776,0.9776,MillID large v1.5,0.0943,0.118
|
| 194 |
+
mos_Latn,Mossi,133,1.0,1.0,0.9811,0.9811,0.9774,0.9771,0.9886,0.9886,MillID large v1.5,-0.0114,0.0075
|
| 195 |
+
xmf_Geor,Mingrelian,132,1.0,1.0,0.963,0.963,0.9848,0.9732,0.981,0.9848,MillID tiny v1.5,-0.0152,0.0218
|
| 196 |
+
kbp_Latn,Kabiyè,130,1.0,1.0,0.9766,0.9766,1.0,0.9923,0.9886,1.0,MillID tiny v1.5,0.0,0.0234
|
| 197 |
+
pag_Latn,Pangasinan,122,0.9959,0.9959,0.959,0.959,0.9573,0.9437,0.9791,0.9791,MillID large v1.5,-0.0168,0.0201
|
| 198 |
+
sat_Olck,Santali,122,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 199 |
+
kmb_Latn,Kimbundu,119,0.9917,0.9917,0.9633,0.9633,0.9833,0.9833,0.9917,0.9917,MillID large v1.5,0.0,0.0284
|
| 200 |
+
kas_Arab,Kashmiri,118,1.0,1.0,0.9784,0.9784,0.9957,0.9915,0.9957,0.9957,MillID tiny v1.5,-0.0043,0.0173
|
| 201 |
+
bjn_Arab,Banjar,116,0.8342,0.9869,0.4533,0.9596,0.9915,0.9831,0.9957,0.9957,MillID large v1.5,0.0088,0.0361
|
| 202 |
+
ace_Latn,Acehnese,113,1.0,1.0,0.8943,0.88,0.9776,0.9867,0.9867,0.9867,MillID medium v1,-0.0133,0.1067
|
| 203 |
+
nav_Latn,Navajo,113,1.0,1.0,0.968,0.968,1.0,0.9867,0.9956,1.0,MillID tiny v1.5,0.0,0.032
|
| 204 |
+
taq_Latn,Tamasheq,106,1.0,1.0,0.8377,0.8377,0.9016,0.9091,0.9223,0.9223,MillID large v1.5,-0.0777,0.0846
|
| 205 |
+
taq_Tfng,Tamasheq,104,0.8046,0.8046,0.0,0.0,0.654,0.6129,0.6042,0.654,MillID tiny v1.5,-0.1506,0.654
|
| 206 |
+
umb_Latn,Umbundu,100,0.995,0.995,0.8972,0.8972,0.98,0.9798,0.9901,0.9901,MillID large v1.5,-0.0049,0.0929
|
| 207 |
+
mag_Deva,Magahi,97,1.0,1.0,0.7723,0.7723,0.89,0.9053,0.8969,0.9053,MillID medium v1,-0.0947,0.133
|
| 208 |
+
awa_Deva,Awadhi,91,0.963,0.963,0.7571,0.7571,0.8977,0.8764,0.9357,0.9357,MillID large v1.5,-0.0273,0.1786
|
| 209 |
+
bug_Latn,Buginese,78,0.9811,0.9811,0.9375,0.9375,0.9806,0.987,0.9935,0.9935,MillID large v1.5,0.0124,0.056
|
| 210 |
+
knc_Arab,Central Kanuri,68,0.8571,0.8595,0.029,0.029,0.5217,0.5625,0.5979,0.5979,MillID large v1.5,-0.2616,0.5689
|
| 211 |
+
min_Latn,Minangkabau,55,0.991,0.991,0.3333,0.3333,0.8947,0.9038,0.9346,0.9346,MillID large v1.5,-0.0564,0.6013
|
| 212 |
+
vec_Latn,Venetian,49,0.8605,0.8605,0.6202,0.6202,0.7664,0.7664,0.8367,0.8367,MillID large v1.5,-0.0238,0.2165
|
| 213 |
+
kas_Deva,Kashmiri,47,0.7027,0.7027,0.5867,0.5867,0.557,0.6905,0.6824,0.6905,MillID medium v1,-0.0122,0.1038
|
| 214 |
+
cjk_Latn,Chokwe,24,1.0,1.0,0.807,0.807,0.8571,0.9091,0.8372,0.9091,MillID medium v1,-0.0909,0.1021
|
leaderboard_test.csv
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
leaderboard_udhr.csv
CHANGED
|
@@ -1,359 +1,359 @@
|
|
| 1 |
-
label,language,support,GlotLID,GlotLID restricted,fastText,fastText restricted,MillID tiny
|
| 2 |
-
roh_Latn,Romansh,399,0.9987,0.9987,0.9455,0.9455,0.
|
| 3 |
-
oci_Latn,Occitan,398,0.3755,0.3755,0.4355,0.4355,0.
|
| 4 |
-
cmn_Hani,Chinese (Mandarin),319,0.8681,0.8681,0.4676,0.4676,0.
|
| 5 |
-
krl_Latn,Karelian,183,0.0109,0.0109,0.0,0.0,0.0,0.0,0.0,0.0,
|
| 6 |
-
qug_Latn,Chimborazo Highland Quichua,182,0.0,0.0,0.0,0.0,0.
|
| 7 |
-
acu_Latn,Achuar-Shiwiar,180,0.8535,0.8535,0.0,0.0,0.
|
| 8 |
-
hus_Latn,Huastec,175,0.9855,0.9855,0.0,0.0,0.
|
| 9 |
-
hau_Latn,Hausa,155,1.0,1.0,0.8757,0.8757,0.
|
| 10 |
-
auc_Latn,Waorani,154,0.0,0.0,0.0,0.0,0.0,0.0503,0.
|
| 11 |
-
kwi_Latn,Awa-Cuaiquer,145,0.8769,0.8769,0.0,0.0,0.
|
| 12 |
-
jpn_Jpan,Japanese,140,1.0,1.0,0.9437,0.9437,1.0,1.0,1.0,1.0,
|
| 13 |
-
snn_Latn,Siona,136,0.5137,0.5137,0.0,0.0,0.
|
| 14 |
-
ktu_Latn,Kituba (Democratic Republic of Congo),118,0.6803,0.6803,0.0,0.0,0.
|
| 15 |
-
lat_Latn,Latin,118,0.9915,0.9915,0.0,0.0,0.9957,0.9958,0.
|
| 16 |
-
hat_Latn,Haitian Creole,117,0.9785,0.9785,0.6988,0.6988,0.
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
chr_Cher,Cherokee,115,0.6628,0.6628,0.0,0.0,0.6705,0.7151,0.
|
| 22 |
-
|
| 23 |
-
lvs_Latn,Standard Latvian,115,1.0,1.0,0.9914,0.9914,0.
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
fas_Arab,Persian,114,1.0,1.0,0.9344,0.9344,0.
|
| 31 |
-
lin_Latn,Lingala,110,1.0,1.0,0.9864,0.9864,0.
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
umb_Latn,Umbundu,107,0.9906,0.9906,0.9083,0.9083,0.
|
| 35 |
-
mal_Mlym,Malayalam,100,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,
|
| 36 |
-
ron_Latn,Romanian,97,1.0,1.0,0.8727,0.8727,
|
| 37 |
-
ven_Latn,Venda,89,1.0,1.0,0.0,0.0,1.0,1.0,
|
| 38 |
-
spa_Latn,Spanish,78,0.9872,0.9872,0.5571,0.5571,0.
|
| 39 |
-
hye_Armn,Armenian,70,1.0,1.0,0.9855,0.9855,
|
| 40 |
-
tam_Taml,Tamil,68,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
knc_Latn,Central Kanuri,64,1.0,1.0,0.4384,0.4384,
|
| 46 |
-
njo_Latn,Ao Naga,64,0.9921,0.9921,0.0,0.0,
|
| 47 |
-
skr_Arab,Saraiki,63,0.5,0.5,0.0,0.0,0.
|
| 48 |
-
smo_Latn,Samoan,63,1.0,1.0,0.7922,0.7922,1.0,1.0,1.0,1.0,
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
cab_Latn,Garifuna,60,0.9831,0.9831,0.0,0.0,
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
|
| 170 |
-
|
| 171 |
-
|
| 172 |
-
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
|
| 178 |
-
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
|
| 189 |
-
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
|
| 210 |
-
|
| 211 |
-
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
|
| 215 |
-
|
| 216 |
-
|
| 217 |
-
|
| 218 |
-
|
| 219 |
-
|
| 220 |
-
|
| 221 |
-
|
| 222 |
-
|
| 223 |
-
|
| 224 |
-
|
| 225 |
-
|
| 226 |
-
|
| 227 |
-
|
| 228 |
-
|
| 229 |
-
|
| 230 |
-
|
| 231 |
-
|
| 232 |
-
|
| 233 |
-
|
| 234 |
-
|
| 235 |
-
|
| 236 |
-
|
| 237 |
-
|
| 238 |
-
|
| 239 |
-
|
| 240 |
-
|
| 241 |
-
|
| 242 |
-
|
| 243 |
-
|
| 244 |
-
|
| 245 |
-
|
| 246 |
-
|
| 247 |
-
|
| 248 |
-
|
| 249 |
-
|
| 250 |
-
|
| 251 |
-
|
| 252 |
-
|
| 253 |
-
|
| 254 |
-
|
| 255 |
-
|
| 256 |
-
|
| 257 |
-
|
| 258 |
-
|
| 259 |
-
|
| 260 |
-
|
| 261 |
-
|
| 262 |
-
|
| 263 |
-
|
| 264 |
-
|
| 265 |
-
zyb_Latn,Yongbei Zhuang,57,0.9828,0.9744,0.0,0.0,
|
| 266 |
-
|
| 267 |
-
|
| 268 |
-
|
| 269 |
-
|
| 270 |
-
|
| 271 |
-
|
| 272 |
-
|
| 273 |
-
cha_Latn,Chamorro,57,0.9245,0.9245,0.0,0.0,
|
| 274 |
-
|
| 275 |
-
|
| 276 |
-
|
| 277 |
-
|
| 278 |
-
|
| 279 |
-
|
| 280 |
-
|
| 281 |
-
|
| 282 |
-
|
| 283 |
-
|
| 284 |
-
|
| 285 |
-
|
| 286 |
-
|
| 287 |
-
|
| 288 |
-
|
| 289 |
-
|
| 290 |
-
|
| 291 |
-
|
| 292 |
-
|
| 293 |
-
|
| 294 |
-
|
| 295 |
-
|
| 296 |
-
|
| 297 |
-
|
| 298 |
-
|
| 299 |
-
|
| 300 |
-
|
| 301 |
-
|
| 302 |
-
|
| 303 |
-
|
| 304 |
-
|
| 305 |
-
|
| 306 |
-
|
| 307 |
-
|
| 308 |
-
|
| 309 |
-
|
| 310 |
-
|
| 311 |
-
|
| 312 |
-
|
| 313 |
-
ady_Cyrl,Adyghe,56,0.9818,0.9818,0.6707,0.6707,0.
|
| 314 |
-
|
| 315 |
-
|
| 316 |
-
|
| 317 |
-
|
| 318 |
-
|
| 319 |
-
|
| 320 |
-
|
| 321 |
-
|
| 322 |
-
|
| 323 |
-
|
| 324 |
-
|
| 325 |
-
|
| 326 |
-
|
| 327 |
-
koo_Latn,Konzo,54,0.9615,0.9615,0.0,0.0,0.
|
| 328 |
-
dje_Latn,Zarma,54,0.9412,0.9515,0.0,0.0,0.
|
| 329 |
-
|
| 330 |
-
|
| 331 |
-
|
| 332 |
-
|
| 333 |
-
|
| 334 |
-
|
| 335 |
-
|
| 336 |
-
|
| 337 |
-
|
| 338 |
-
|
| 339 |
-
zdj_Latn,Ngazidja Comorian,50,0.9796,0.9899,0.0,0.0,
|
| 340 |
-
nyn_Latn,Nyankole,50,0.9796,0.9796,0.0,0.0,1.0,1.0,1.0,1.0,
|
| 341 |
-
|
| 342 |
-
lug_Latn,Ganda,49,1.0,1.0,0.359,0.359,0.
|
| 343 |
-
eve_Cyrl,Even,49,0.8864,0.8864,0.0,0.0,1.0,0.9897,1.0,1.0,
|
| 344 |
-
|
| 345 |
-
|
| 346 |
-
wuu_Hani,Wu Chinese,48,0.5562,0.5529,0.0,0.0,0.
|
| 347 |
-
|
| 348 |
-
|
| 349 |
-
bin_Latn,Bini,47,0.9318,0.9318,0.0,0.0,0.
|
| 350 |
-
lia_Latn,West-Central Limba,45,1.0,1.0,0.0,0.0,
|
| 351 |
-
|
| 352 |
-
|
| 353 |
-
ojb_Cans,Northwestern Ojibwa,43,0.9639,0.9639,0.0,0.0,1.0,1.0,1.0,1.0,
|
| 354 |
-
cbs_Latn,Cashinahua,43,0.9647,0.9647,0.0,0.0,0.9882,0.9885,1.0,
|
| 355 |
-
ese_Latn,Ese Ejja,35,0.9855,0.9855,0.0,0.0,1.0,0.9855,
|
| 356 |
-
tzh_Latn,Tzeltal,25,0.9804,0.9804,0.0,0.0,0.
|
| 357 |
-
chv_Cyrl,Chuvash,25,1.0,1.0,0.9434,0.9434,1.0,1.0,1.0,1.0,
|
| 358 |
-
taj_Deva,Eastern Tamang,1,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,
|
| 359 |
-
tzm_Tfng,Central Atlas Tamazight,1,0.0645,0.0645,0.0328,0.0328,0.
|
|
|
|
| 1 |
+
label,language,support,GlotLID,GlotLID restricted,fastText,fastText restricted,MillID tiny v1.5,MillID medium v1,MillID large v1.5,best MillID,best MillID model,gain over GlotLID restricted,gain over fastText restricted
|
| 2 |
+
roh_Latn,Romansh,399,0.9987,0.9987,0.9455,0.9455,0.9937,0.9912,0.9975,0.9975,MillID large v1.5,-0.0012,0.052
|
| 3 |
+
oci_Latn,Occitan,398,0.3755,0.3755,0.4355,0.4355,0.2771,0.3312,0.3382,0.3382,MillID large v1.5,-0.0373,-0.0973
|
| 4 |
+
cmn_Hani,Chinese (Mandarin),319,0.8681,0.8681,0.4676,0.4676,0.8921,0.886,0.875,0.8921,MillID tiny v1.5,0.024,0.4245
|
| 5 |
+
krl_Latn,Karelian,183,0.0109,0.0109,0.0,0.0,0.0,0.0,0.0,0.0,MillID tiny v1.5,-0.0109,0.0
|
| 6 |
+
qug_Latn,Chimborazo Highland Quichua,182,0.0,0.0,0.0,0.0,0.0109,0.0,0.0,0.0109,MillID tiny v1.5,0.0109,0.0109
|
| 7 |
+
acu_Latn,Achuar-Shiwiar,180,0.8535,0.8535,0.0,0.0,0.828,0.7431,0.8013,0.828,MillID tiny v1.5,-0.0255,0.828
|
| 8 |
+
hus_Latn,Huastec,175,0.9855,0.9855,0.0,0.0,0.9766,0.9914,0.9884,0.9914,MillID medium v1,0.0059,0.9914
|
| 9 |
+
hau_Latn,Hausa,155,1.0,1.0,0.8757,0.8757,0.9904,0.9968,1.0,1.0,MillID large v1.5,0.0,0.1243
|
| 10 |
+
auc_Latn,Waorani,154,0.0,0.0,0.0,0.0,0.0,0.0503,0.0255,0.0503,MillID medium v1,0.0503,0.0503
|
| 11 |
+
kwi_Latn,Awa-Cuaiquer,145,0.8769,0.8769,0.0,0.0,0.8803,0.775,0.8627,0.8803,MillID tiny v1.5,0.0034,0.8803
|
| 12 |
+
jpn_Jpan,Japanese,140,1.0,1.0,0.9437,0.9437,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0563
|
| 13 |
+
snn_Latn,Siona,136,0.5137,0.5137,0.0,0.0,0.6305,0.7731,0.4667,0.7731,MillID medium v1,0.2594,0.7731
|
| 14 |
+
ktu_Latn,Kituba (Democratic Republic of Congo),118,0.6803,0.6803,0.0,0.0,0.7653,0.7687,0.773,0.773,MillID large v1.5,0.0927,0.773
|
| 15 |
+
lat_Latn,Latin,118,0.9915,0.9915,0.0,0.0,0.9957,0.9958,0.9957,0.9958,MillID medium v1,0.0043,0.9958
|
| 16 |
+
hat_Latn,Haitian Creole,117,0.9785,0.9785,0.6988,0.6988,0.9915,0.9914,0.9871,0.9915,MillID tiny v1.5,0.013,0.2927
|
| 17 |
+
nya_Latn,Nyanja,116,1.0,1.0,0.8056,0.8056,0.9913,0.9871,0.9784,0.9913,MillID tiny v1.5,-0.0087,0.1857
|
| 18 |
+
kmb_Latn,Kimbundu,116,1.0,1.0,0.8775,0.8775,0.9702,0.9787,0.9914,0.9914,MillID large v1.5,-0.0086,0.1139
|
| 19 |
+
rmn_Latn,Balkan Romani,116,0.8571,0.8571,0.0,0.0,0.678,0.6667,0.7039,0.7039,MillID large v1.5,-0.1532,0.7039
|
| 20 |
+
twi_Latn,Twi,116,0.9957,0.9957,0.3993,0.3959,0.9831,0.9871,0.9914,0.9914,MillID large v1.5,-0.0043,0.5955
|
| 21 |
+
chr_Cher,Cherokee,115,0.6628,0.6628,0.0,0.0,0.6705,0.7151,0.945,0.945,MillID large v1.5,0.2822,0.945
|
| 22 |
+
por_Latn,Portuguese,115,0.9957,0.9957,0.7931,0.7931,0.9554,0.9957,1.0,1.0,MillID large v1.5,0.0043,0.2069
|
| 23 |
+
lvs_Latn,Standard Latvian,115,1.0,1.0,0.9914,0.9914,0.9868,0.9957,1.0,1.0,MillID large v1.5,0.0,0.0086
|
| 24 |
+
pov_Latn,Upper Guinea Crioulo,115,0.8325,0.8325,0.0,0.0,0.8205,0.8384,0.8265,0.8384,MillID medium v1,0.0059,0.8384
|
| 25 |
+
kng_Latn,Koongo,115,0.1928,0.1928,0.4571,0.4571,0.0758,0.0469,0.0469,0.0758,MillID tiny v1.5,-0.117,-0.3813
|
| 26 |
+
ell_Grek,Greek,115,1.0,1.0,1.0,1.0,0.6705,0.655,0.6705,0.6705,MillID tiny v1.5,-0.3295,-0.3295
|
| 27 |
+
tha_Thai,Thai,114,1.0,1.0,0.9956,0.9956,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0044
|
| 28 |
+
tso_Latn,Tsonga,114,0.7807,0.7807,0.6909,0.6909,0.86,0.8824,0.7742,0.8824,MillID medium v1,0.1017,0.1915
|
| 29 |
+
cjk_Latn,Chokwe,114,1.0,1.0,0.5327,0.5327,0.8713,0.9493,0.9589,0.9589,MillID large v1.5,-0.0411,0.4262
|
| 30 |
+
fas_Arab,Persian,114,1.0,1.0,0.9344,0.9344,0.9912,0.9912,0.9821,0.9912,MillID tiny v1.5,-0.0088,0.0568
|
| 31 |
+
lin_Latn,Lingala,110,1.0,1.0,0.9864,0.9864,0.991,1.0,0.9955,1.0,MillID medium v1,0.0,0.0136
|
| 32 |
+
urd_Arab,Urdu,108,0.986,0.986,0.6353,0.6353,0.9907,0.9953,0.9811,0.9953,MillID medium v1,0.0093,0.36
|
| 33 |
+
fuv_Latn,Nigerian Fulfulde,108,0.6813,0.6885,0.4747,0.4747,0.6588,0.4051,0.5732,0.6588,MillID tiny v1.5,-0.0297,0.1841
|
| 34 |
+
umb_Latn,Umbundu,107,0.9906,0.9906,0.9083,0.9083,0.9662,0.8737,0.9612,0.9662,MillID tiny v1.5,-0.0244,0.0579
|
| 35 |
+
mal_Mlym,Malayalam,100,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 36 |
+
ron_Latn,Romanian,97,1.0,1.0,0.8727,0.8727,0.9949,0.9949,0.9948,0.9949,MillID tiny v1.5,-0.0051,0.1222
|
| 37 |
+
ven_Latn,Venda,89,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 38 |
+
spa_Latn,Spanish,78,0.9872,0.9872,0.5571,0.5571,0.974,0.9872,0.9935,0.9935,MillID large v1.5,0.0063,0.4364
|
| 39 |
+
hye_Armn,Armenian,70,1.0,1.0,0.9855,0.9855,1.0,0.9928,0.9928,1.0,MillID tiny v1.5,0.0,0.0145
|
| 40 |
+
tam_Taml,Tamil,68,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 41 |
+
tiv_Latn,Tiv,67,1.0,1.0,0.0,0.0,1.0,0.9926,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 42 |
+
deu_Latn,German,67,1.0,1.0,0.9571,0.9571,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0429
|
| 43 |
+
sun_Latn,Sundanese,66,0.9924,0.9924,0.7692,0.7692,0.9924,0.9767,1.0,1.0,MillID large v1.5,0.0076,0.2308
|
| 44 |
+
pam_Latn,Pampanga,66,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 45 |
+
knc_Latn,Central Kanuri,64,1.0,1.0,0.4384,0.4384,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.5616
|
| 46 |
+
njo_Latn,Ao Naga,64,0.9921,0.9921,0.0,0.0,1.0,0.9922,0.9846,1.0,MillID tiny v1.5,0.0079,1.0
|
| 47 |
+
skr_Arab,Saraiki,63,0.5,0.5,0.0,0.0,0.8673,0.9231,0.875,0.9231,MillID medium v1,0.4231,0.9231
|
| 48 |
+
smo_Latn,Samoan,63,1.0,1.0,0.7922,0.7922,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.2078
|
| 49 |
+
nob_Latn,Norwegian Bokmål,62,0.9764,0.9764,0.992,0.992,0.9134,0.9516,0.9593,0.9593,MillID large v1.5,-0.0171,-0.0327
|
| 50 |
+
mcf_Latn,Matsés,62,0.992,0.992,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.008,1.0
|
| 51 |
+
shp_Latn,Shipibo-Conibo,62,0.9123,0.9123,0.0,0.0,0.4691,0.4103,0.6067,0.6067,MillID large v1.5,-0.3056,0.6067
|
| 52 |
+
cbi_Latn,Chachi,62,0.9919,0.9919,0.0,0.0,0.9841,1.0,0.9919,1.0,MillID medium v1,0.0081,1.0
|
| 53 |
+
gla_Latn,Scottish Gaelic,62,0.9836,0.9836,0.9587,0.9587,0.9919,0.976,1.0,1.0,MillID large v1.5,0.0164,0.0413
|
| 54 |
+
pnb_Arab,Western Panjabi,61,0.7394,0.7394,0.0,0.0,0.8824,0.9313,0.8841,0.9313,MillID medium v1,0.1919,0.9313
|
| 55 |
+
dan_Latn,Danish,61,0.9833,0.9833,0.9677,0.9677,0.9431,0.9587,0.9516,0.9587,MillID medium v1,-0.0246,-0.009
|
| 56 |
+
fin_Latn,Finnish,61,1.0,1.0,0.2998,0.2998,1.0,0.9836,0.9752,1.0,MillID tiny v1.5,0.0,0.7002
|
| 57 |
+
sey_Latn,Secoya,61,0.5744,0.5888,0.0,0.0,0.5799,0.5839,0.5263,0.5839,MillID medium v1,-0.0049,0.5839
|
| 58 |
+
swe_Latn,Swedish,61,1.0,1.0,1.0,1.0,0.9917,1.0,1.0,1.0,MillID medium v1,0.0,0.0
|
| 59 |
+
boa_Latn,Bora,61,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 60 |
+
amr_Latn,Amarakaeri,61,0.976,0.976,0.0,0.0,0.9919,1.0,0.9833,1.0,MillID medium v1,0.024,1.0
|
| 61 |
+
ace_Latn,Acehnese,61,0.9917,0.9917,0.5357,0.5357,1.0,0.9833,0.9917,1.0,MillID tiny v1.5,0.0083,0.4643
|
| 62 |
+
sna_Latn,Shona,60,0.9917,0.9917,0.5729,0.5729,0.9756,0.9836,0.9836,0.9836,MillID medium v1,-0.0081,0.4107
|
| 63 |
+
bug_Latn,Buginese,60,1.0,1.0,0.5742,0.5742,0.9298,0.9655,0.9916,0.9916,MillID large v1.5,-0.0084,0.4174
|
| 64 |
+
cab_Latn,Garifuna,60,0.9831,0.9831,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0169,1.0
|
| 65 |
+
bod_Tibt,Tibetan,60,0.9009,0.9009,0.8992,0.8992,0.9748,0.9748,0.9483,0.9748,MillID tiny v1.5,0.0739,0.0756
|
| 66 |
+
cot_Latn,Caquinte,60,0.9916,0.9916,0.0,0.0,0.9916,0.9831,0.9916,0.9916,MillID tiny v1.5,0.0,0.9916
|
| 67 |
+
cpu_Latn,Pichis Ashéninka,60,0.9744,0.9744,0.0,0.0,0.7292,0.5714,0.5714,0.7292,MillID tiny v1.5,-0.2452,0.7292
|
| 68 |
+
cbr_Latn,Cashibo-Cacataibo,60,1.0,1.0,0.0,0.0,1.0,0.9917,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 69 |
+
tbz_Latn,Ditammari,60,0.3784,0.3784,0.0,0.0,0.0,0.3333,0.209,0.3333,MillID medium v1,-0.0451,0.3333
|
| 70 |
+
zgh_Tfng,Standard Moroccan Tamazight,60,0.4615,0.4615,0.0,0.0,0.75,0.8235,0.6517,0.8235,MillID medium v1,0.362,0.8235
|
| 71 |
+
ton_Latn,Tongan,60,0.9917,0.9917,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0083,1.0
|
| 72 |
+
tca_Latn,Ticuna,60,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 73 |
+
zam_Latn,Miahuatlán Zapotec,60,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,MillID tiny v1.5,0.0,0.0
|
| 74 |
+
cym_Latn,Welsh,60,1.0,1.0,0.8108,0.8108,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.1892
|
| 75 |
+
qvm_Latn,Margos-Yarowilca-Lauricocha Quechua,60,0.5,0.5,0.0,0.0,0.3562,0.0938,0.3562,0.3562,MillID tiny v1.5,-0.1438,0.3562
|
| 76 |
+
quy_Latn,Ayacucho Quechua,60,0.7763,0.7763,0.1039,0.1039,0.6744,0.6747,0.6835,0.6835,MillID large v1.5,-0.0928,0.5796
|
| 77 |
+
fil_Latn,Filipino,60,1.0,1.0,0.553,0.553,1.0,0.9916,0.9917,1.0,MillID tiny v1.5,0.0,0.447
|
| 78 |
+
mad_Latn,Madurese,60,0.9748,0.9748,0.0,0.0,0.9833,0.9655,0.9833,0.9833,MillID tiny v1.5,0.0085,0.9833
|
| 79 |
+
mcd_Latn,Sharanahua,60,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 80 |
+
ekk_Latn,Estonian,60,1.0,1.0,0.3448,0.3448,0.9917,1.0,1.0,1.0,MillID medium v1,0.0,0.6552
|
| 81 |
+
ctd_Latn,Tedim Chin,60,0.6957,0.7234,0.0,0.0,0.6047,0.7879,0.7755,0.7879,MillID medium v1,0.0645,0.7879
|
| 82 |
+
gaz_Latn,Oromo (West Central),60,0.9752,0.9752,0.5381,0.5381,0.958,0.9916,0.9917,0.9917,MillID large v1.5,0.0165,0.4536
|
| 83 |
+
ind_Latn,Indonesian,60,0.9027,0.9027,0.5652,0.5652,0.7677,0.6739,0.6452,0.7677,MillID tiny v1.5,-0.135,0.2025
|
| 84 |
+
jav_Latn,Javanese,60,0.9916,0.9916,0.513,0.513,0.9833,0.9672,0.9833,0.9833,MillID tiny v1.5,-0.0083,0.4703
|
| 85 |
+
jiv_Latn,Shuar,60,0.8027,0.8027,0.0,0.0,0.8227,0.7066,0.7284,0.8227,MillID tiny v1.5,0.02,0.8227
|
| 86 |
+
vie_Latn,Vietnamese,60,1.0,1.0,0.9524,0.9524,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0476
|
| 87 |
+
wol_Latn,Wolof,60,1.0,1.0,0.2362,0.2362,0.9917,0.9836,0.9833,0.9917,MillID tiny v1.5,-0.0083,0.7555
|
| 88 |
+
gjn_Latn,Gonja,60,0.9831,0.9831,0.0,0.0,0.9916,0.9916,0.9833,0.9916,MillID tiny v1.5,0.0085,0.9916
|
| 89 |
+
war_Latn,Waray,60,0.9917,0.9917,0.944,0.944,0.9836,1.0,1.0,1.0,MillID medium v1,0.0083,0.056
|
| 90 |
+
ban_Latn,Balinese,60,1.0,1.0,0.5607,0.5607,0.9744,0.9831,0.9833,0.9833,MillID large v1.5,-0.0167,0.4226
|
| 91 |
+
udu_Latn,Uduk,60,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 92 |
+
arl_Latn,Arabela,60,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 93 |
+
afr_Latn,Afrikaans,60,1.0,1.0,0.944,0.944,0.9916,1.0,1.0,1.0,MillID medium v1,0.0,0.056
|
| 94 |
+
ada_Latn,Adangme,59,0.9365,0.9365,0.0,0.0,0.9916,0.9752,0.9672,0.9916,MillID tiny v1.5,0.0551,0.9916
|
| 95 |
+
ceb_Latn,Cebuano,59,1.0,1.0,0.6146,0.6146,0.9916,0.9833,1.0,1.0,MillID large v1.5,0.0,0.3854
|
| 96 |
+
als_Latn,Albanian (Tosk),59,1.0,1.0,0.7329,0.7329,0.9915,0.9828,0.9739,0.9915,MillID tiny v1.5,-0.0085,0.2586
|
| 97 |
+
ajg_Latn,Aja (Benin),59,0.9916,0.9916,0.0,0.0,0.9219,0.9593,0.9833,0.9833,MillID large v1.5,-0.0083,0.9833
|
| 98 |
+
bos_Latn,Bosnian,59,0.3121,0.3121,0.1979,0.1979,0.2,0.2281,0.1913,0.2281,MillID medium v1,-0.084,0.0302
|
| 99 |
+
bre_Latn,Breton,59,0.9915,0.9915,0.0,0.0,0.9831,1.0,1.0,1.0,MillID medium v1,0.0085,1.0
|
| 100 |
+
ssw_Latn,Swati,59,1.0,1.0,0.9516,0.9516,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0484
|
| 101 |
+
hsb_Latn,Upper Sorbian,59,1.0,1.0,0.0,0.0,1.0,1.0,0.9915,1.0,MillID tiny v1.5,0.0,1.0
|
| 102 |
+
yua_Latn,Yucateco,59,1.0,1.0,0.0,0.0,0.9833,1.0,1.0,1.0,MillID medium v1,0.0,1.0
|
| 103 |
+
fur_Latn,Friulian,59,1.0,1.0,0.6082,0.6051,0.9667,1.0,1.0,1.0,MillID medium v1,0.0,0.3949
|
| 104 |
+
gaa_Latn,Ga,59,0.9916,0.9833,0.0,0.0,0.9752,0.9916,0.9672,0.9916,MillID medium v1,0.0083,0.9916
|
| 105 |
+
gsw_Latn,Swiss German,59,0.9915,0.9915,0.0,0.0,0.8462,0.8081,0.9273,0.9273,MillID large v1.5,-0.0642,0.9273
|
| 106 |
+
hin_Deva,Hindi,59,0.9573,0.9573,0.9752,0.9752,0.7234,0.6067,0.7273,0.7273,MillID large v1.5,-0.23,-0.2479
|
| 107 |
+
hns_Latn,Caribbean Hindustani,59,0.9369,0.9369,0.0,0.0,0.9273,0.9739,0.9739,0.9739,MillID medium v1,0.037,0.9739
|
| 108 |
+
hun_Latn,Hungarian,59,1.0,1.0,0.3224,0.3224,1.0,0.9916,0.9833,1.0,MillID tiny v1.5,0.0,0.6776
|
| 109 |
+
sin_Sinh,Sinhala,59,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 110 |
+
srp_Latn,Serbian,59,0.45,0.45,0.0,0.0,0.3486,0.4754,0.4074,0.4754,MillID medium v1,0.0254,0.4754
|
| 111 |
+
sag_Latn,Sango,59,0.5783,0.5783,0.1892,0.1892,0.9915,0.9915,0.9915,0.9915,MillID tiny v1.5,0.4132,0.8023
|
| 112 |
+
srp_Cyrl,Serbian,59,0.9915,0.9915,0.9915,0.9915,0.9915,0.9828,0.9915,0.9915,MillID tiny v1.5,0.0,0.0
|
| 113 |
+
slv_Latn,Slovenian,59,0.9752,0.9752,0.8027,0.8027,0.9661,0.9483,0.9831,0.9831,MillID large v1.5,0.0079,0.1804
|
| 114 |
+
kdh_Latn,Tem,59,0.8654,0.9915,0.0,0.0,0.9916,1.0,1.0,1.0,MillID medium v1,0.0085,1.0
|
| 115 |
+
khm_Khmr,Khmer,59,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 116 |
+
kqn_Latn,Kaonde,59,1.0,1.0,0.0,0.0,0.9831,0.9916,0.9831,0.9916,MillID medium v1,-0.0084,0.9916
|
| 117 |
+
kbr_Latn,Kafa,59,0.9915,0.9915,0.0,0.0,1.0,1.0,0.9916,1.0,MillID tiny v1.5,0.0085,1.0
|
| 118 |
+
kin_Latn,Kinyarwanda,59,0.8868,0.8868,0.8571,0.8571,0.8039,0.9412,0.8727,0.9412,MillID medium v1,0.0544,0.0841
|
| 119 |
+
kbp_Latn,Kabiyè,59,1.0,1.0,0.3893,0.3893,0.7815,0.9593,0.9672,0.9672,MillID large v1.5,-0.0328,0.5779
|
| 120 |
+
xsm_Latn,Kasem,59,0.9915,0.9915,0.0,0.0,0.9273,0.9268,0.9492,0.9492,MillID large v1.5,-0.0423,0.9492
|
| 121 |
+
gag_Latn,Gagauz,59,0.9915,0.9915,0.0,0.0,0.9828,0.9828,0.9915,0.9915,MillID large v1.5,0.0,0.9915
|
| 122 |
+
ydd_Hebr,Eastern Yiddish,59,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 123 |
+
tly_Latn,Talysh,59,0.9915,0.9915,0.0,0.0,0.9915,0.9915,0.9915,0.9915,MillID tiny v1.5,0.0,0.9915
|
| 124 |
+
bis_Latn,Bislama,59,1.0,1.0,0.7516,0.7516,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.2484
|
| 125 |
+
pol_Latn,Polish,59,1.0,1.0,0.7296,0.7296,0.9916,0.9655,0.9833,0.9916,MillID tiny v1.5,-0.0084,0.262
|
| 126 |
+
nba_Latn,Nyemba,59,0.0,0.0,0.0,0.0,0.0328,0.0,0.0,0.0328,MillID tiny v1.5,0.0328,0.0328
|
| 127 |
+
nav_Latn,Navajo,59,1.0,1.0,0.7024,0.7024,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.2976
|
| 128 |
+
rar_Latn,Rarotongan,59,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 129 |
+
qwh_Latn,Huaylas Ancash Quechua,59,0.6545,0.6545,0.0,0.0,0.494,0.4955,0.5306,0.5306,MillID large v1.5,-0.1239,0.5306
|
| 130 |
+
dga_Latn,Southern Dagaare,59,0.9915,1.0,0.0,0.0,1.0,0.9915,0.9831,1.0,MillID tiny v1.5,0.0,1.0
|
| 131 |
+
dzo_Tibt,Dzongkha,59,0.9134,0.9134,0.8571,0.8571,0.9748,0.9748,0.9508,0.9748,MillID tiny v1.5,0.0614,0.1177
|
| 132 |
+
crh_Latn,Crimean Tatar,59,1.0,1.0,0.8551,0.8551,0.9916,0.9916,0.9833,0.9916,MillID tiny v1.5,-0.0084,0.1365
|
| 133 |
+
not_Latn,Nomatsiguenga,59,0.9915,0.9915,0.0,0.0,0.9831,0.9915,1.0,1.0,MillID large v1.5,0.0085,1.0
|
| 134 |
+
ewe_Latn,Ewe,59,1.0,1.0,0.6517,0.6517,0.9916,1.0,1.0,1.0,MillID medium v1,0.0,0.3483
|
| 135 |
+
fij_Latn,Fijian,59,1.0,1.0,0.9752,0.9752,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0248
|
| 136 |
+
eus_Latn,Basque,59,1.0,1.0,0.8923,0.8923,0.9219,0.9672,0.9833,0.9833,MillID large v1.5,-0.0167,0.091
|
| 137 |
+
lld_Latn,Ladin,59,0.7312,0.7312,0.0,0.0,0.9091,0.8991,0.8807,0.9091,MillID tiny v1.5,0.1779,0.9091
|
| 138 |
+
uig_Latn,Uyghur,59,0.9833,0.9752,0.0,0.0,0.9833,0.9916,1.0,1.0,MillID large v1.5,0.0248,1.0
|
| 139 |
+
bcl_Latn,Central Bikol,59,0.9916,0.9916,0.0,0.0,0.9915,0.9916,0.9916,0.9916,MillID medium v1,0.0,0.9916
|
| 140 |
+
ayr_Latn,Central Aymara,59,1.0,1.0,0.716,0.716,0.9916,1.0,0.9587,1.0,MillID medium v1,0.0,0.284
|
| 141 |
+
wln_Latn,Walloon,59,0.655,0.655,0.0,0.0,0.6118,0.6667,0.6591,0.6667,MillID medium v1,0.0117,0.6667
|
| 142 |
+
cnr_Latn,Serbian (Montenegro),59,0.0303,0.0303,0.0,0.0,0.0,0.0,0.0,0.0,MillID tiny v1.5,-0.0303,0.0
|
| 143 |
+
cof_Latn,Colorado,59,0.9273,0.9369,0.0,0.0,1.0,1.0,0.9831,1.0,MillID tiny v1.5,0.0631,1.0
|
| 144 |
+
cos_Latn,Corsican,59,0.9915,0.9915,0.0,0.0,1.0,0.9916,0.9831,1.0,MillID tiny v1.5,0.0085,1.0
|
| 145 |
+
cjs_Cyrl,Shor,59,0.9369,0.9369,0.0,0.0,0.9558,0.9649,0.9739,0.9739,MillID large v1.5,0.037,0.9739
|
| 146 |
+
qva_Latn,Ambo-Pasco Quechua,59,0.0328,0.0328,0.0,0.0,0.0,0.0,0.0,0.0,MillID tiny v1.5,-0.0328,0.0
|
| 147 |
+
ote_Latn,Mezquital Otomi,59,0.0,0.0,0.0,0.0,0.1538,0.0656,0.0656,0.1538,MillID tiny v1.5,0.1538,0.1538
|
| 148 |
+
nzi_Latn,Nzima,59,1.0,1.0,0.0,0.0,1.0,1.0,0.9916,1.0,MillID tiny v1.5,0.0,1.0
|
| 149 |
+
tsz_Latn,Purepecha,59,0.9655,0.9744,0.0,0.0,0.9739,0.9915,1.0,1.0,MillID large v1.5,0.0256,1.0
|
| 150 |
+
tob_Latn,Toba,59,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 151 |
+
ame_Latn,Yanesha',59,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 152 |
+
tur_Latn,Turkish,59,0.9916,0.9916,0.59,0.59,0.9833,0.9833,0.9916,0.9916,MillID large v1.5,0.0,0.4016
|
| 153 |
+
tuk_Latn,Turkmen,59,1.0,1.0,0.9672,0.9672,0.9828,0.9916,1.0,1.0,MillID large v1.5,0.0,0.0328
|
| 154 |
+
top_Latn,Papantla Totonac,59,0.9916,0.9916,0.0,0.0,0.958,0.9833,1.0,1.0,MillID large v1.5,0.0084,1.0
|
| 155 |
+
maz_Latn,Central Mazahua,59,0.9391,0.9391,0.0,0.0,0.7891,0.7421,0.8138,0.8138,MillID large v1.5,-0.1253,0.8138
|
| 156 |
+
men_Latn,Mende,59,0.9916,0.9916,0.0,0.0,1.0,0.9916,0.9916,1.0,MillID tiny v1.5,0.0084,1.0
|
| 157 |
+
mic_Latn,Mi'kmaw,59,0.4267,0.4267,0.0,0.0,0.9915,1.0,1.0,1.0,MillID medium v1,0.5733,1.0
|
| 158 |
+
ltz_Latn,Luxembourgish,59,0.9915,0.9915,0.648,0.648,0.9915,0.9916,1.0,1.0,MillID large v1.5,0.0085,0.352
|
| 159 |
+
min_Latn,Minangkabau,59,0.9916,0.9916,0.127,0.127,0.9916,0.9915,1.0,1.0,MillID large v1.5,0.0084,0.873
|
| 160 |
+
mor_Latn,Moro,59,0.9915,0.9915,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0085,1.0
|
| 161 |
+
mlt_Latn,Maltese,59,1.0,1.0,0.8939,0.8939,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.1061
|
| 162 |
+
mar_Deva,Marathi,59,1.0,1.0,1.0,1.0,1.0,0.9916,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 163 |
+
ibo_Latn,Igbo,58,1.0,1.0,0.8227,0.8227,1.0,0.9915,1.0,1.0,MillID tiny v1.5,0.0,0.1773
|
| 164 |
+
kat_Geor,Georgian,58,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 165 |
+
kha_Latn,Khasi,58,1.0,1.0,0.0,0.0,1.0,1.0,0.9915,1.0,MillID tiny v1.5,0.0,1.0
|
| 166 |
+
kab_Latn,Kabyle,58,0.9913,0.9913,0.9831,0.9748,0.9259,0.9915,0.9913,0.9915,MillID medium v1,0.0002,0.0167
|
| 167 |
+
heb_Hebr,Hebrew,58,1.0,1.0,1.0,1.0,0.9825,0.9913,1.0,1.0,MillID large v1.5,0.0,0.0
|
| 168 |
+
sco_Latn,Scots,58,0.9735,0.9735,0.0,0.0,0.9825,0.9825,0.9825,0.9825,MillID tiny v1.5,0.009,0.9825
|
| 169 |
+
srr_Latn,Serer,58,0.9831,0.9831,0.0,0.0,1.0,1.0,0.9831,1.0,MillID tiny v1.5,0.0169,1.0
|
| 170 |
+
kaz_Cyrl,Kazakh,58,1.0,1.0,0.4444,0.4444,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.5556
|
| 171 |
+
ces_Latn,Czech,58,1.0,1.0,0.9508,0.9508,0.9825,0.9915,1.0,1.0,MillID large v1.5,0.0,0.0492
|
| 172 |
+
bul_Cyrl,Bulgarian,58,1.0,1.0,1.0,1.0,0.9828,0.9913,0.9828,0.9913,MillID medium v1,-0.0087,-0.0087
|
| 173 |
+
suk_Latn,Sukuma,58,0.9828,0.9828,0.0,0.0,0.9828,0.9825,1.0,1.0,MillID large v1.5,0.0172,1.0
|
| 174 |
+
zul_Latn,Zulu,58,0.9744,0.9744,0.6073,0.6073,0.9189,0.9107,0.9573,0.9573,MillID large v1.5,-0.0171,0.35
|
| 175 |
+
chk_Latn,Chuukese,58,1.0,1.0,0.0,0.0,0.9915,1.0,0.9915,1.0,MillID medium v1,0.0,1.0
|
| 176 |
+
guj_Gujr,Gujarati,58,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 177 |
+
guc_Latn,Wayuu,58,1.0,1.0,0.0,0.0,0.9831,0.9667,0.9831,0.9831,MillID tiny v1.5,-0.0169,0.9831
|
| 178 |
+
tzo_Latn,Tzotzil,58,0.9913,0.9913,0.0,0.0,0.9643,0.9735,0.9913,0.9913,MillID large v1.5,0.0,0.9913
|
| 179 |
+
vmw_Latn,Makhuwa,58,0.9825,0.9913,0.0,0.0,0.9825,0.9825,0.9825,0.9825,MillID tiny v1.5,-0.0088,0.9825
|
| 180 |
+
vec_Latn,Venetian,58,1.0,1.0,0.8056,0.8056,1.0,0.9828,1.0,1.0,MillID tiny v1.5,0.0,0.1944
|
| 181 |
+
ukr_Cyrl,Ukrainian,58,1.0,1.0,0.9667,0.9667,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0333
|
| 182 |
+
bci_Latn,Baoulé,58,1.0,1.0,0.0,0.0,1.0,1.0,0.9915,1.0,MillID tiny v1.5,0.0,1.0
|
| 183 |
+
bba_Latn,Baatonum,58,0.9913,1.0,0.0,0.0,0.9913,0.9913,0.9831,0.9913,MillID tiny v1.5,-0.0087,0.9913
|
| 184 |
+
bel_Cyrl,Belarusian,58,1.0,1.0,0.8345,0.8345,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.1655
|
| 185 |
+
uig_Arab,Uyghur,58,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 186 |
+
ast_Latn,Asturian,58,1.0,1.0,0.7484,0.7484,0.9735,0.9825,1.0,1.0,MillID large v1.5,0.0,0.2516
|
| 187 |
+
sus_Latn,Susu,58,0.9748,0.9748,0.0,0.0,0.955,0.9643,0.9915,0.9915,MillID large v1.5,0.0167,0.9915
|
| 188 |
+
uzn_Cyrl,Northern Uzbek,58,1.0,1.0,0.0,0.0,0.9913,1.0,1.0,1.0,MillID medium v1,0.0,1.0
|
| 189 |
+
uzn_Latn,Northern Uzbek,58,0.9915,0.9915,0.4056,0.4056,0.9667,0.928,0.9355,0.9667,MillID tiny v1.5,-0.0248,0.5611
|
| 190 |
+
ndo_Latn,Ndonga,58,0.958,0.958,0.0,0.0,0.95,0.9431,0.9587,0.9587,MillID large v1.5,0.0007,0.9587
|
| 191 |
+
nds_Latn,Low German,58,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 192 |
+
qvh_Latn,Huamalíes-Dos de Mayo Huánuco Quechua,58,0.1538,0.1538,0.0,0.0,0.4675,0.2154,0.2154,0.4675,MillID tiny v1.5,0.3137,0.4675
|
| 193 |
+
qvc_Latn,Cajamarca Quechua,58,0.9062,0.9062,0.0,0.0,0.648,0.7891,0.8169,0.8169,MillID large v1.5,-0.0893,0.8169
|
| 194 |
+
dyu_Latn,Dyula,58,0.4337,0.4337,0.0154,0.0154,0.0,0.0,0.0,0.0,MillID tiny v1.5,-0.4337,-0.0154
|
| 195 |
+
eng_Latn,English,58,1.0,1.0,0.5979,0.5979,0.9748,0.9744,0.9831,0.9831,MillID large v1.5,-0.0169,0.3852
|
| 196 |
+
yap_Latn,Yapese,58,1.0,1.0,0.0,0.0,0.9913,0.9915,0.9915,0.9915,MillID medium v1,-0.0085,0.9915
|
| 197 |
+
kmr_Latn,Northern Kurdish,58,1.0,1.0,0.7582,0.7582,1.0,0.9825,0.9455,1.0,MillID tiny v1.5,0.0,0.2418
|
| 198 |
+
kir_Cyrl,Kyrgyz,58,1.0,1.0,0.6237,0.6237,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.3763
|
| 199 |
+
glv_Latn,Manx,58,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 200 |
+
kor_Hang,Korean,58,1.0,1.0,0.8923,0.8923,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.1077
|
| 201 |
+
rus_Cyrl,Russian,58,0.9667,0.9667,0.7838,0.7838,0.9748,0.9573,0.9655,0.9748,MillID tiny v1.5,0.0081,0.191
|
| 202 |
+
glg_Latn,Galician,58,1.0,1.0,0.8286,0.8286,0.9828,0.9913,0.9915,0.9915,MillID large v1.5,-0.0085,0.1629
|
| 203 |
+
xho_Latn,Xhosa,58,1.0,1.0,0.8529,0.8529,0.9402,0.9204,0.9474,0.9474,MillID large v1.5,-0.0526,0.0945
|
| 204 |
+
kaa_Cyrl,Kara-Kalpak,58,1.0,1.0,0.0,0.0,1.0,0.9915,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 205 |
+
kal_Latn,Kalaallisut,58,1.0,1.0,0.76,0.76,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.24
|
| 206 |
+
kea_Latn,Kabuverdianu,58,0.9355,0.9355,0.125,0.1244,0.7917,0.7945,0.7733,0.7945,MillID medium v1,-0.141,0.6701
|
| 207 |
+
kan_Knda,Kannada,58,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 208 |
+
ina_Latn,Interlingua,58,1.0,1.0,0.0,0.0,0.9913,1.0,1.0,1.0,MillID medium v1,0.0,1.0
|
| 209 |
+
ilo_Latn,Iloko,58,0.9913,0.9913,0.8593,0.8593,0.9831,1.0,1.0,1.0,MillID medium v1,0.0087,0.1407
|
| 210 |
+
bum_Latn,Bulu,58,0.0984,0.0984,0.0,0.0,0.2154,0.3188,0.1875,0.3188,MillID medium v1,0.2204,0.3188
|
| 211 |
+
cat_Latn,Catalan,58,1.0,1.0,0.6905,0.6905,0.9748,0.9667,1.0,1.0,MillID large v1.5,0.0,0.3095
|
| 212 |
+
sme_Latn,Northern Sami,58,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 213 |
+
haw_Latn,Hawaiian,58,0.9735,0.9735,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0265,1.0
|
| 214 |
+
hnj_Latn,Hmong Njua,58,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,MillID tiny v1.5,0.0,0.0
|
| 215 |
+
hil_Latn,Hiligaynon,58,1.0,1.0,0.0,0.0,0.9913,0.9739,0.9913,0.9913,MillID tiny v1.5,-0.0087,0.9913
|
| 216 |
+
hrv_Latn,Croatian,58,0.7273,0.7273,0.4737,0.4737,0.5,0.5371,0.5591,0.5591,MillID large v1.5,-0.1682,0.0854
|
| 217 |
+
huu_Latn,Murui Huitoto,58,1.0,1.0,0.0,0.0,1.0,1.0,0.9915,1.0,MillID tiny v1.5,0.0,1.0
|
| 218 |
+
kri_Latn,Krio,58,1.0,1.0,0.0,0.0,1.0,1.0,0.9915,1.0,MillID tiny v1.5,0.0,1.0
|
| 219 |
+
nso_Latn,Northern Sotho,58,1.0,1.0,0.9508,0.9508,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0492
|
| 220 |
+
nld_Latn,Dutch,58,1.0,1.0,0.703,0.703,0.9915,1.0,1.0,1.0,MillID medium v1,0.0,0.297
|
| 221 |
+
niu_Latn,Niuean,58,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 222 |
+
mxv_Latn,Metlatónoc Mixtec,58,0.0,0.0,0.0,0.0,0.129,0.2154,0.0339,0.2154,MillID medium v1,0.2154,0.2154
|
| 223 |
+
lij_Latn,Ligurian,58,0.8593,0.8593,0.4872,0.4872,0.9831,0.9587,0.9508,0.9831,MillID tiny v1.5,0.1238,0.4959
|
| 224 |
+
lit_Latn,Lithuanian,58,1.0,1.0,0.8112,0.8112,1.0,1.0,0.9915,1.0,MillID tiny v1.5,0.0,0.1888
|
| 225 |
+
loz_Latn,Lozi,58,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 226 |
+
quc_Latn,Kʼicheʼ,58,0.9174,0.9174,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0826,1.0
|
| 227 |
+
run_Latn,Rundi,58,0.8992,0.8992,0.8,0.8,0.8571,0.9391,0.9032,0.9391,MillID medium v1,0.0399,0.1391
|
| 228 |
+
crs_Latn,Seselwa Creole French,58,1.0,1.0,0.0,0.0,0.9915,0.9744,0.9915,0.9915,MillID tiny v1.5,-0.0085,0.9915
|
| 229 |
+
nbl_Latn,South Ndebele,58,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,MillID tiny v1.5,0.0,0.0
|
| 230 |
+
lad_Latn,Ladino,58,1.0,1.0,0.0,0.0,0.9643,0.9913,1.0,1.0,MillID large v1.5,0.0,1.0
|
| 231 |
+
lue_Latn,Luvale,58,0.9913,0.9913,0.0,0.0,0.9748,0.9831,0.9667,0.9831,MillID medium v1,-0.0082,0.9831
|
| 232 |
+
mah_Latn,Marshallese,58,1.0,1.0,0.0,0.0,1.0,0.9915,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 233 |
+
lus_Latn,Mizo,58,1.0,0.9915,0.3986,0.3986,0.9913,1.0,0.9913,1.0,MillID medium v1,0.0085,0.6014
|
| 234 |
+
epo_Latn,Esperanto,58,1.0,1.0,0.4234,0.4234,0.9913,0.9825,0.9913,0.9913,MillID tiny v1.5,-0.0087,0.5679
|
| 235 |
+
lua_Latn,Luba-Lulua,58,0.7391,0.7391,0.6386,0.6386,0.7097,0.55,0.7033,0.7097,MillID tiny v1.5,-0.0294,0.0711
|
| 236 |
+
ita_Latn,Italian,58,1.0,1.0,0.6629,0.6629,0.9913,0.9825,0.9735,0.9913,MillID tiny v1.5,-0.0087,0.3284
|
| 237 |
+
mfq_Latn,Moba,58,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 238 |
+
isl_Latn,Icelandic,58,1.0,1.0,0.8345,0.8345,0.8738,0.9455,0.8846,0.9455,MillID medium v1,-0.0545,0.111
|
| 239 |
+
mam_Latn,Mam,58,0.9455,0.9455,0.0,0.0,0.8846,1.0,0.9913,1.0,MillID medium v1,0.0545,1.0
|
| 240 |
+
mai_Deva,Maithili,58,1.0,1.0,1.0,1.0,0.9355,0.8593,0.9508,0.9508,MillID large v1.5,-0.0492,-0.0492
|
| 241 |
+
mkd_Cyrl,Macedonian,58,1.0,1.0,1.0,1.0,0.9915,0.9748,0.9915,0.9915,MillID tiny v1.5,-0.0085,-0.0085
|
| 242 |
+
azj_Latn,North Azerbaijani,58,0.9915,0.9915,0.5949,0.5949,0.9915,0.9915,0.9915,0.9915,MillID tiny v1.5,0.0,0.3966
|
| 243 |
+
ben_Beng,Bangla,58,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 244 |
+
arb_Arab,Arabic (Modern Standard),58,1.0,1.0,1.0,1.0,0.8846,0.9259,0.9358,0.9358,MillID large v1.5,-0.0642,-0.0642
|
| 245 |
+
ami_Latn,Amis,58,0.0,0.0,0.0,0.0,0.0,0.0339,0.0984,0.0984,MillID large v1.5,0.0984,0.0984
|
| 246 |
+
arn_Latn,Mapuche,58,0.8952,0.9057,0.8175,0.8175,0.9739,0.9744,0.9913,0.9913,MillID large v1.5,0.0856,0.1738
|
| 247 |
+
tel_Telu,Telugu,58,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 248 |
+
cnh_Latn,Hakha Chin,58,0.9915,0.9915,0.0,0.0,0.9915,0.9828,0.9915,0.9915,MillID tiny v1.5,0.0,0.9915
|
| 249 |
+
tet_Latn,Tetum,58,0.1667,0.1667,0.0,0.0,0.6377,0.6986,0.7075,0.7075,MillID large v1.5,0.5408,0.7075
|
| 250 |
+
vep_Latn,Veps,58,1.0,1.0,0.0,0.0,0.9825,0.9913,1.0,1.0,MillID large v1.5,0.0,1.0
|
| 251 |
+
tpi_Latn,Tok Pisin,58,1.0,1.0,0.9831,0.9831,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0169
|
| 252 |
+
fuf_Latn,Pular,58,0.4828,0.4828,0.0,0.0,0.9057,0.8163,0.9358,0.9358,MillID large v1.5,0.453,0.9358
|
| 253 |
+
fry_Latn,Western Frisian,58,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 254 |
+
fkv_Latn,Kven Finnish,58,0.6667,0.6667,0.0,0.0,0.6353,0.6966,0.9057,0.9057,MillID large v1.5,0.239,0.9057
|
| 255 |
+
pcd_Latn,Picard,58,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,MillID tiny v1.5,0.0,0.0
|
| 256 |
+
pan_Guru,Punjabi,58,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 257 |
+
pap_Latn,Papiamento,58,0.9915,0.9915,0.725,0.725,1.0,1.0,0.9915,1.0,MillID tiny v1.5,0.0085,0.275
|
| 258 |
+
swh_Latn,Swahili,58,0.9825,0.9825,0.5088,0.5088,0.9074,0.9455,0.8785,0.9455,MillID medium v1,-0.037,0.4367
|
| 259 |
+
fon_Latn,Fon,58,0.9913,0.9913,0.4979,0.4979,0.9492,0.9649,0.9667,0.9667,MillID large v1.5,-0.0246,0.4688
|
| 260 |
+
nno_Latn,Norwegian Nynorsk,58,1.0,1.0,0.9748,0.9748,0.9558,0.9744,0.9913,0.9913,MillID large v1.5,-0.0087,0.0165
|
| 261 |
+
tir_Ethi,Tigrinya,58,1.0,1.0,0.9915,0.9915,0.9915,0.9915,0.9831,0.9915,MillID tiny v1.5,-0.0085,0.0
|
| 262 |
+
fra_Latn,French,58,0.9915,0.9915,0.9134,0.9134,0.8544,0.9655,0.9739,0.9739,MillID large v1.5,-0.0176,0.0605
|
| 263 |
+
tsn_Latn,Tswana,58,0.9915,0.9915,0.9134,0.9134,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0085,0.0866
|
| 264 |
+
tyv_Cyrl,Tuvinian,58,0.9915,0.9915,0.0,0.0,1.0,0.9915,1.0,1.0,MillID tiny v1.5,0.0085,1.0
|
| 265 |
+
zyb_Latn,Yongbei Zhuang,57,0.9828,0.9744,0.0,0.0,0.8769,0.9744,0.9744,0.9744,MillID medium v1,0.0,0.9744
|
| 266 |
+
agr_Latn,Aguaruna,57,0.9739,0.9739,0.0,0.0,0.9565,0.9655,0.9744,0.9744,MillID large v1.5,0.0005,0.9744
|
| 267 |
+
sid_Latn,Sidamo,57,0.9268,0.9268,0.0,0.0,0.9825,1.0,0.9825,1.0,MillID medium v1,0.0732,1.0
|
| 268 |
+
aii_Syrc,Assyrian Neo-Aramaic,57,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,MillID tiny v1.5,0.0,0.0
|
| 269 |
+
abk_Cyrl,Abkhazian,57,1.0,1.0,0.9913,0.9913,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0087
|
| 270 |
+
cbt_Latn,Chayahuita,57,0.9913,0.9913,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0087,1.0
|
| 271 |
+
cbu_Latn,Candoshi-Shapra,57,0.8824,0.8738,0.0,0.0,0.5,0.3714,0.557,0.557,MillID large v1.5,-0.3168,0.557
|
| 272 |
+
cfm_Latn,Falam Chin,57,0.9821,0.9821,0.0,0.0,0.9912,0.9828,0.9739,0.9912,MillID tiny v1.5,0.0091,0.9912
|
| 273 |
+
cha_Latn,Chamorro,57,0.9245,0.9245,0.0,0.0,0.9913,0.9825,1.0,1.0,MillID large v1.5,0.0755,1.0
|
| 274 |
+
kek_Latn,Kekchí,57,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 275 |
+
khk_Cyrl,Halh Mongolian,57,0.9912,0.9912,0.8976,0.8976,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0088,0.1024
|
| 276 |
+
kjh_Cyrl,Khakas,57,0.9913,0.9913,0.0,0.0,0.9744,1.0,0.9913,1.0,MillID medium v1,0.0087,1.0
|
| 277 |
+
koi_Cyrl,Komi-Permyak,57,1.0,1.0,0.0,0.0,0.8367,0.8367,0.8485,0.8485,MillID large v1.5,-0.1515,0.8485
|
| 278 |
+
yor_Latn,Yoruba,57,0.9913,0.9913,0.7808,0.7808,1.0,0.9913,0.9913,1.0,MillID tiny v1.5,0.0087,0.2192
|
| 279 |
+
sah_Cyrl,Yakut,57,0.958,0.958,0.0,0.0,1.0,0.9913,1.0,1.0,MillID tiny v1.5,0.042,1.0
|
| 280 |
+
som_Latn,Somali,57,1.0,1.0,0.6994,0.6994,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.3006
|
| 281 |
+
sot_Latn,Southern Sotho,57,1.0,1.0,0.7863,0.7863,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.2137
|
| 282 |
+
cak_Latn,Kaqchikel,57,1.0,1.0,0.0,0.0,0.9912,1.0,0.9913,1.0,MillID medium v1,0.0,1.0
|
| 283 |
+
lao_Laoo,Lao,57,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 284 |
+
fao_Latn,Faroese,57,1.0,1.0,0.7473,0.7473,1.0,0.9913,0.9828,1.0,MillID tiny v1.5,0.0,0.2527
|
| 285 |
+
fat_Latn,Fanti,57,0.9913,0.9913,0.0,0.0,1.0,0.9744,0.9913,1.0,MillID tiny v1.5,0.0087,1.0
|
| 286 |
+
mri_Latn,Māori,57,1.0,1.0,0.3826,0.3813,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.6187
|
| 287 |
+
qxn_Latn,Northern Conchucos Ancash Quechua,57,0.0345,0.0345,0.0,0.0,0.129,0.0345,0.0,0.129,MillID tiny v1.5,0.0945,0.129
|
| 288 |
+
dag_Latn,Dagbani,57,0.9739,0.9739,0.0,0.0,0.9573,0.9825,0.9828,0.9828,MillID large v1.5,0.0089,0.9828
|
| 289 |
+
div_Thaa,Divehi,57,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 290 |
+
quh_Latn,South Bolivian Quechua,57,0.9369,0.9649,0.0,0.0,0.9655,0.9828,0.9661,0.9828,MillID medium v1,0.0179,0.9828
|
| 291 |
+
pon_Latn,Pohnpeian,57,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 292 |
+
rup_Latn,Aromanian,57,1.0,1.0,0.0,0.0,1.0,0.9913,0.9913,1.0,MillID tiny v1.5,0.0,1.0
|
| 293 |
+
pis_Latn,Pijin,57,0.9913,0.9913,0.0,0.0,1.0,1.0,0.9913,1.0,MillID tiny v1.5,0.0087,1.0
|
| 294 |
+
qvn_Latn,North Junín Quechua,57,0.4595,0.4595,0.0,0.0,0.2188,0.2188,0.0339,0.2188,MillID tiny v1.5,-0.2407,0.2188
|
| 295 |
+
tdt_Latn,Tetun Dili,57,0.6115,0.6115,0.0,0.0,0.4565,0.4884,0.4819,0.4884,MillID medium v1,-0.1231,0.4884
|
| 296 |
+
pau_Latn,Palauan,57,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 297 |
+
tuk_Cyrl,Turkmen,57,1.0,1.0,0.0,0.0,1.0,0.973,0.973,1.0,MillID tiny v1.5,0.0,1.0
|
| 298 |
+
quz_Latn,Cusco Quechua,57,0.8947,0.906,0.0,0.0,0.8,0.8829,0.8462,0.8829,MillID medium v1,-0.0231,0.8829
|
| 299 |
+
tat_Cyrl,Tatar,57,1.0,1.0,0.5938,0.5938,1.0,0.9912,1.0,1.0,MillID tiny v1.5,0.0,0.4062
|
| 300 |
+
tah_Latn,Tahitian,57,0.8125,0.8125,0.9661,0.9661,0.973,0.973,0.9735,0.9735,MillID large v1.5,0.161,0.0074
|
| 301 |
+
toi_Latn,Tonga (Zambia),57,1.0,1.0,0.0,0.0,0.9825,0.9636,0.9643,0.9825,MillID tiny v1.5,-0.0175,0.9825
|
| 302 |
+
toj_Latn,Tojolabal,57,1.0,1.0,0.0,0.0,0.9825,0.9913,1.0,1.0,MillID large v1.5,0.0,1.0
|
| 303 |
+
tgk_Cyrl,Tajik,57,1.0,1.0,0.623,0.623,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.377
|
| 304 |
+
bam_Latn,Bambara,57,0.7519,0.7519,0.1992,0.1969,0.8244,0.6267,0.7027,0.8244,MillID tiny v1.5,0.0725,0.6275
|
| 305 |
+
bem_Latn,Bemba,57,1.0,1.0,0.4385,0.4385,0.9913,0.9828,1.0,1.0,MillID large v1.5,0.0,0.5615
|
| 306 |
+
azj_Cyrl,North Azerbaijani,56,1.0,1.0,0.0,0.0,1.0,0.9912,0.9912,1.0,MillID tiny v1.5,0.0,1.0
|
| 307 |
+
bho_Deva,Bhojpuri,56,0.9333,0.9333,0.8317,0.8317,0.8515,0.6429,0.7416,0.8515,MillID tiny v1.5,-0.0818,0.0198
|
| 308 |
+
pcm_Latn,Nigerian Pidgin,56,0.9402,0.9412,0.8627,0.955,1.0,0.9825,1.0,1.0,MillID tiny v1.5,0.0588,0.045
|
| 309 |
+
pbb_Latn,Páez,56,0.7126,0.7126,0.0,0.0,0.6977,0.8485,0.8247,0.8485,MillID medium v1,0.1359,0.8485
|
| 310 |
+
mya_Mymr,Burmese,56,1.0,1.0,0.6707,0.6707,1.0,0.991,1.0,1.0,MillID tiny v1.5,0.0,0.3293
|
| 311 |
+
ura_Latn,Urarina,56,1.0,1.0,0.0,0.0,1.0,1.0,0.991,1.0,MillID tiny v1.5,0.0,1.0
|
| 312 |
+
gle_Latn,Irish,56,0.9825,0.9825,1.0,1.0,1.0,0.9912,1.0,1.0,MillID tiny v1.5,0.0175,0.0
|
| 313 |
+
ady_Cyrl,Adyghe,56,0.9818,0.9818,0.6707,0.6707,0.963,0.991,1.0,1.0,MillID large v1.5,0.0182,0.3293
|
| 314 |
+
slk_Latn,Slovak,56,1.0,1.0,0.9256,0.9256,0.9825,0.9821,1.0,1.0,MillID large v1.5,0.0,0.0744
|
| 315 |
+
shn_Mymr,Shan,56,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0
|
| 316 |
+
mos_Latn,Mossi,56,1.0,1.0,0.855,0.855,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.145
|
| 317 |
+
lun_Latn,Lunda,56,0.991,0.991,0.0,0.0,0.991,1.0,1.0,1.0,MillID medium v1,0.009,1.0
|
| 318 |
+
npi_Deva,Nepali,55,0.9908,0.9908,0.9565,0.9565,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0092,0.0435
|
| 319 |
+
tem_Latn,Timne,55,1.0,1.0,0.0,0.0,1.0,0.991,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 320 |
+
dyo_Latn,Jola-Fonyi,55,0.9908,1.0,0.0,0.0,0.9908,0.9815,0.955,0.9908,MillID tiny v1.5,-0.0092,0.9908
|
| 321 |
+
mnw_Mymr,Mon,55,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 322 |
+
yrk_Cyrl,Nenets,55,0.972,0.972,0.0,0.0,0.9815,0.9815,0.9815,0.9815,MillID tiny v1.5,0.0095,0.9815
|
| 323 |
+
kbd_Cyrl,Kabardian,55,0.9735,0.9735,0.0,0.0,0.9649,0.991,1.0,1.0,MillID large v1.5,0.0265,1.0
|
| 324 |
+
san_Deva,Sanskrit,55,0.9908,0.9908,0.973,0.973,0.9815,0.9908,0.9908,0.9908,MillID medium v1,0.0,0.0178
|
| 325 |
+
yad_Latn,Yagua,55,0.9815,0.9815,0.0,0.0,0.963,1.0,1.0,1.0,MillID medium v1,0.0185,1.0
|
| 326 |
+
ido_Latn,Ido,55,1.0,1.0,0.0,0.0,0.9636,0.9815,1.0,1.0,MillID large v1.5,0.0,1.0
|
| 327 |
+
koo_Latn,Konzo,54,0.9615,0.9615,0.0,0.0,0.9714,0.9714,0.9714,0.9714,MillID tiny v1.5,0.0099,0.9714
|
| 328 |
+
dje_Latn,Zarma,54,0.9412,0.9515,0.0,0.0,0.898,0.8866,0.9412,0.9412,MillID large v1.5,-0.0103,0.9412
|
| 329 |
+
plt_Latn,Malagasy (Plateau),53,1.0,0.9907,0.955,0.955,0.9907,1.0,1.0,1.0,MillID medium v1,0.0093,0.045
|
| 330 |
+
mag_Deva,Magahi,53,0.9444,0.9444,0.6323,0.6323,0.8679,0.7963,0.8491,0.8679,MillID tiny v1.5,-0.0765,0.2356
|
| 331 |
+
yao_Latn,Yao,52,0.9903,0.9903,0.0,0.0,0.9811,1.0,1.0,1.0,MillID medium v1,0.0097,1.0
|
| 332 |
+
cni_Latn,Asháninka,52,0.963,0.963,0.0,0.0,0.7727,0.7075,0.7172,0.7727,MillID tiny v1.5,-0.1903,0.7727
|
| 333 |
+
alt_Cyrl,Southern Altai,51,0.9623,0.9623,0.6986,0.6986,0.9903,0.9903,1.0,1.0,MillID large v1.5,0.0377,0.3014
|
| 334 |
+
kde_Latn,Makonde,51,0.9703,0.9703,0.0,0.0,0.9278,0.9184,0.8667,0.9278,MillID tiny v1.5,-0.0425,0.9278
|
| 335 |
+
mto_Latn,Totontepec Mixe,51,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,MillID tiny v1.5,0.0,0.0
|
| 336 |
+
miq_Latn,Mískito,51,1.0,1.0,0.0,0.0,1.0,1.0,0.9903,1.0,MillID tiny v1.5,0.0,1.0
|
| 337 |
+
idu_Latn,Idoma,51,0.8791,0.9375,0.0,0.0,0.6027,0.9278,0.9263,0.9278,MillID medium v1,-0.0097,0.9278
|
| 338 |
+
amh_Ethi,Amharic,51,1.0,1.0,0.8409,0.8409,0.9901,0.9901,0.9697,0.9901,MillID tiny v1.5,-0.0099,0.1492
|
| 339 |
+
zdj_Latn,Ngazidja Comorian,50,0.9796,0.9899,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0101,1.0
|
| 340 |
+
nyn_Latn,Nyankole,50,0.9796,0.9796,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0204,1.0
|
| 341 |
+
gug_Latn,Paraguayan Guaraní,49,1.0,1.0,0.2367,0.2367,0.9515,0.9515,0.98,0.98,MillID large v1.5,-0.02,0.7433
|
| 342 |
+
lug_Latn,Ganda,49,1.0,1.0,0.359,0.359,0.9899,0.9897,1.0,1.0,MillID large v1.5,0.0,0.641
|
| 343 |
+
eve_Cyrl,Even,49,0.8864,0.8864,0.0,0.0,1.0,0.9897,1.0,1.0,MillID tiny v1.5,0.1136,1.0
|
| 344 |
+
yue_Hani,Cantonese,49,0.5797,0.5797,0.1108,0.1108,0.7342,0.6522,0.7071,0.7342,MillID tiny v1.5,0.1545,0.6234
|
| 345 |
+
gyr_Latn,Guarayu,49,0.9897,0.9897,0.0,0.0,0.5373,0.339,0.3934,0.5373,MillID tiny v1.5,-0.4524,0.5373
|
| 346 |
+
wuu_Hani,Wu Chinese,48,0.5562,0.5529,0.0,0.0,0.6144,0.6277,0.5481,0.6277,MillID medium v1,0.0748,0.6277
|
| 347 |
+
hak_Hani,Hakka Chinese,48,0.0408,0.0408,0.0,0.0,0.0392,0.0,0.0755,0.0755,MillID large v1.5,0.0347,0.0755
|
| 348 |
+
nan_Hani,Min Nan Chinese,48,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,MillID tiny v1.5,0.0,0.0
|
| 349 |
+
bin_Latn,Bini,47,0.9318,0.9318,0.0,0.0,0.9892,0.9892,1.0,1.0,MillID large v1.5,0.0682,1.0
|
| 350 |
+
lia_Latn,West-Central Limba,45,1.0,1.0,0.0,0.0,1.0,0.9888,0.989,1.0,MillID tiny v1.5,0.0,1.0
|
| 351 |
+
oss_Cyrl,Ossetic,44,1.0,1.0,0.9778,0.9778,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0222
|
| 352 |
+
ike_Cans,Eastern Canadian Inuktitut,44,0.9778,0.9778,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0222,1.0
|
| 353 |
+
ojb_Cans,Northwestern Ojibwa,43,0.9639,0.9639,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0361,1.0
|
| 354 |
+
cbs_Latn,Cashinahua,43,0.9647,0.9647,0.0,0.0,0.9882,0.9885,1.0,1.0,MillID large v1.5,0.0353,1.0
|
| 355 |
+
ese_Latn,Ese Ejja,35,0.9855,0.9855,0.0,0.0,1.0,0.9855,0.9855,1.0,MillID tiny v1.5,0.0145,1.0
|
| 356 |
+
tzh_Latn,Tzeltal,25,0.9804,0.9804,0.0,0.0,0.9259,0.9434,0.9804,0.9804,MillID large v1.5,0.0,0.9804
|
| 357 |
+
chv_Cyrl,Chuvash,25,1.0,1.0,0.9434,0.9434,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,0.0566
|
| 358 |
+
taj_Deva,Eastern Tamang,1,1.0,1.0,0.0,0.0,1.0,1.0,1.0,1.0,MillID tiny v1.5,0.0,1.0
|
| 359 |
+
tzm_Tfng,Central Atlas Tamazight,1,0.0645,0.0645,0.0328,0.0328,0.1667,0.3333,0.1,0.3333,MillID medium v1,0.2688,0.3005
|