# M1 data EDA — UniversalCEFR

Generated: 2026-06-11T14:43:50+00:00
Command: `scripts/eda_universalcefr.py --langs all --report docs/evals/m1_data_eda_all.md`

## Subsets overview

| dataset | lang | rows | categories | formats | licenses | words: median (p10, p90) |
|---|---|---:|---|---|---|---|
| `UniversalCEFR/caes_es` | es | 31149 | {'learner': 31149} | {'document-level': 31149} | {'CC BY-NC 4.0': 31149} | 150 (p10=66, p90=289) |
| `UniversalCEFR/cambridge_exams_en` | en | 331 | {'reference': 331} | {'document-level': 331} | {'CC BY-NC-SA 4.0': 331} | 563 (p10=123, p90=923) |
| `UniversalCEFR/cefr_asag_en` | en | 299 | {'learner': 299} | {'document-level': 299} | {'CC BY-NC-SA 4.0': 299} | 59 (p10=8, p90=150) |
| `UniversalCEFR/cefr_sp_en` | en | 10004 | {'reference': 10004} | {'sentence-level': 10004} | {'CC BY-NC-SA 4.0': 10004} | 14 (p10=8, p90=24) |
| `UniversalCEFR/cople2_pt` | pt | 942 | {'learner': 942} | {'paragraph-level': 942} | {'CC BY-SA-NC 4.0': 942} | 152 (p10=67, p90=264) |
| `UniversalCEFR/elg_cefr_de` | de | 509 | {'reference': 509} | {'document-level': 509} | {'CC BY-NC-SA 4.0': 509} | 140 (p10=124, p90=148) |
| `UniversalCEFR/elg_cefr_en` | en | 712 | {'reference': 712} | {'document-level': 712} | {'CC BY-NC-SA 4.0': 712} | 277 (p10=191, p90=799) |
| `UniversalCEFR/elg_cefr_nl` | nl | 3596 | {'reference': 3596} | {'document-level': 3596} | {'CC BY-NC-SA 4.0': 3596} | 211 (p10=127, p90=230) |
| `UniversalCEFR/elle_et` | et | 1697 | {'learner': 1697} | {'paragraph-level': 420, 'document-level': 1277} | {'CC BY 4.0': 1697} | 186 (p10=61, p90=463) |
| `UniversalCEFR/hablacultura_es` | es | 713 | {'reference': 713} | {'paragraph-level': 713} | {'CC BY NC 4.0': 713} | 61 (p10=23, p90=137) |
| `UniversalCEFR/icle500_en` | en | 495 | {'learner': 495} | {'document-level': 495} | {'CC0 1.0 (Public Domain)': 495} | 585 (p10=476, p90=762) |
| `UniversalCEFR/kwiqiz_es` | es | 206 | {'reference': 206} | {'document-level': 206} | {'CC BY NC 4.0': 206} | 232 (p10=123, p90=447) |
| `UniversalCEFR/kwiqiz_fr` | fr | 344 | {'reference': 344} | {'document-level': 344} | {'CC BY NC 4.0': 344} | 329 (p10=165, p90=631) |
| `UniversalCEFR/learn_welsh_cy` | cy | 1372 | {'reference': 1372} | {'dialogue-level': 115, 'document-level': 41, 'paragraph-level': 109, 'sentence-level': 1107} | {'public': 1372} | 6 (p10=3, p90=61) |
| `UniversalCEFR/merlin_cs` | cs | 441 | {'learner': 441} | {'paragraph-level': 441} | {'CC BY-SA 4.0': 441} | 135 (p10=61, p90=230) |
| `UniversalCEFR/merlin_de` | de | 1033 | {'learner': 1033} | {'paragraph-level': 1033} | {'CC BY-SA 4.0': 1033} | 104 (p10=40, p90=222) |
| `UniversalCEFR/merlin_it` | it | 813 | {'learner': 813} | {'paragraph-level': 813} | {'CC BY-SA 4.0': 813} | 92 (p10=44, p90=198) |
| `UniversalCEFR/peapl2_pt` | pt | 481 | {'learner': 481} | {'paragraph-level': 481} | {'CC BY SA NC 4.0': 481} | 220 (p10=100, p90=319) |
| `UniversalCEFR/readme_ar` | ar | 1945 | {'reference': 1945} | {'sentence-level': 1945} | {'CC BY SA NC 4.0': 1945} | 19 (p10=7, p90=47) |
| `UniversalCEFR/readme_en` | en | 2822 | {'reference': 2822} | {'sentence-level': 2822} | {'CC BY SA NC 4.0': 2822} | 17 (p10=7, p90=36) |
| `UniversalCEFR/readme_fr` | fr | 1669 | {'reference': 1669} | {'sentence-level': 1669} | {'CC BY SA NC 4.0': 1669} | 18 (p10=8, p90=41) |
| `UniversalCEFR/readme_hi` | hi | 1491 | {'reference': 1491} | {'sentence-level': 1491} | {'CC BY SA NC 4.0': 1491} | 17 (p10=8, p90=35) |
| `UniversalCEFR/readme_ru` | ru | 1758 | {'reference': 1758} | {'sentence-level': 1758} | {'CC BY SA NC 4.0': 1758} | 16 (p10=7, p90=33) |
| `UniversalCEFR/zaebuc_ar` | ar | 215 | {'learner': 215} | {'paragraph-level': 215} | {'CC BY-SA 4.0': 215} | 155 (p10=72, p90=244) |

## Level distribution per subset

| dataset | A1 | A2 | B1 | B2 | C1 | C2 | odd labels |
|---|---:|---:|---:|---:|---:|---:|---:|
| `UniversalCEFR/caes_es` | 8261 | 8582 | 6510 | 4729 | 3067 | 0 | 0 |
| `UniversalCEFR/cambridge_exams_en` | 0 | 64 | 60 | 71 | 67 | 69 | 0 |
| `UniversalCEFR/cefr_asag_en` | 18 | 59 | 113 | 74 | 30 | 5 | 0 |
| `UniversalCEFR/cefr_sp_en` | 124 | 1271 | 3305 | 3330 | 1744 | 230 | 0 |
| `UniversalCEFR/cople2_pt` | 236 | 236 | 163 | 163 | 72 | 72 | 0 |
| `UniversalCEFR/elg_cefr_de` | 31 | 90 | 90 | 115 | 88 | 95 | 0 |
| `UniversalCEFR/elg_cefr_en` | 24 | 110 | 158 | 140 | 97 | 69 | 114 |
| `UniversalCEFR/elg_cefr_nl` | 51 | 216 | 782 | 738 | 219 | 85 | 1505 |
| `UniversalCEFR/elle_et` | 0 | 395 | 588 | 407 | 307 | 0 | 0 |
| `UniversalCEFR/hablacultura_es` | 0 | 33 | 271 | 273 | 118 | 0 | 18 |
| `UniversalCEFR/icle500_en` | 0 | 1 | 29 | 99 | 91 | 64 | 211 |
| `UniversalCEFR/kwiqiz_es` | 21 | 33 | 54 | 59 | 39 | 0 | 0 |
| `UniversalCEFR/kwiqiz_fr` | 11 | 24 | 131 | 126 | 52 | 0 | 0 |
| `UniversalCEFR/learn_welsh_cy` | 764 | 608 | 0 | 0 | 0 | 0 | 0 |
| `UniversalCEFR/merlin_cs` | 1 | 188 | 165 | 81 | 4 | 0 | 2 |
| `UniversalCEFR/merlin_de` | 57 | 306 | 331 | 293 | 42 | 4 | 0 |
| `UniversalCEFR/merlin_it` | 29 | 381 | 394 | 2 | 0 | 0 | 7 |
| `UniversalCEFR/peapl2_pt` | 78 | 89 | 204 | 70 | 40 | 0 | 0 |
| `UniversalCEFR/readme_ar` | 81 | 252 | 514 | 565 | 350 | 183 | 0 |
| `UniversalCEFR/readme_en` | 182 | 673 | 623 | 896 | 377 | 71 | 0 |
| `UniversalCEFR/readme_fr` | 140 | 366 | 444 | 352 | 241 | 126 | 0 |
| `UniversalCEFR/readme_hi` | 263 | 283 | 286 | 263 | 222 | 174 | 0 |
| `UniversalCEFR/readme_ru` | 402 | 293 | 409 | 326 | 237 | 91 | 0 |
| `UniversalCEFR/zaebuc_ar` | 0 | 7 | 111 | 80 | 11 | 0 | 6 |

Odd labels detail: `{'UniversalCEFR/elg_cefr_en': {'A1+': 1, 'A2+': 31, 'B1+': 48, 'B2+': 34}, 'UniversalCEFR/elg_cefr_nl': {'B1+': 708, 'A2+': 316, 'B2+': 305, 'A1+': 60, 'C2+': 14, 'C1+': 102}, 'UniversalCEFR/hablacultura_es': {'B': 18}, 'UniversalCEFR/icle500_en': {'B2+': 119, 'B1+': 85, 'NA': 7}, 'UniversalCEFR/merlin_cs': {'EMPTY': 2}, 'UniversalCEFR/merlin_it': {'unrated': 6, 'EMPTY': 1}, 'UniversalCEFR/zaebuc_ar': {'Unassessable': 6}}`

## Reference pool (the M1 reading-classifier candidates)

Subsets whose `category` includes `reference`, i.e. texts written *for* learners rather than *by* them. Learner-production subsets are the M3 candidates (grading learner writing), not M1 training data.

| lang | reference rows | from subsets |
|---|---:|---|
| ar | 1945 | `readme_ar` |
| cy | 1372 | `learn_welsh_cy` |
| de | 509 | `elg_cefr_de` |
| en | 13869 | `cambridge_exams_en`, `cefr_sp_en`, `elg_cefr_en`, `readme_en` |
| es | 919 | `hablacultura_es`, `kwiqiz_es` |
| fr | 2013 | `kwiqiz_fr`, `readme_fr` |
| hi | 1491 | `readme_hi` |
| nl | 3596 | `elg_cefr_nl` |
| ru | 1758 | `readme_ru` |

---
Notes: word counts use whitespace tokenisation (approximate for ar/hi). Licenses are aggregated from the per-row `license` field; decisions and exclusions are recorded in `docs/adr/0003-datasets-and-licensing.md`.