docs/evals/m1_data_eda.md

# M1 data EDA — UniversalCEFR

Generated: 2026-06-11T14:28:37+00:00
Command: `scripts/eda_universalcefr.py`

## Subsets overview

| dataset | lang | rows | categories | formats | licenses | words: median (p10, p90) |
|---|---|---:|---|---|---|---|
| `UniversalCEFR/cambridge_exams_en` | en | 331 | {'reference': 331} | {'document-level': 331} | {'CC BY-NC-SA 4.0': 331} | 563 (p10=123, p90=923) |
| `UniversalCEFR/cefr_asag_en` | en | 299 | {'learner': 299} | {'document-level': 299} | {'CC BY-NC-SA 4.0': 299} | 59 (p10=8, p90=150) |
| `UniversalCEFR/cefr_sp_en` | en | 10004 | {'reference': 10004} | {'sentence-level': 10004} | {'CC BY-NC-SA 4.0': 10004} | 14 (p10=8, p90=24) |
| `UniversalCEFR/elg_cefr_en` | en | 712 | {'reference': 712} | {'document-level': 712} | {'CC BY-NC-SA 4.0': 712} | 277 (p10=191, p90=799) |
| `UniversalCEFR/icle500_en` | en | 495 | {'learner': 495} | {'document-level': 495} | {'CC0 1.0 (Public Domain)': 495} | 585 (p10=476, p90=762) |
| `UniversalCEFR/readme_en` | en | 2822 | {'reference': 2822} | {'sentence-level': 2822} | {'CC BY SA NC 4.0': 2822} | 17 (p10=7, p90=36) |

## Level distribution per subset

| dataset | A1 | A2 | B1 | B2 | C1 | C2 | odd labels |
|---|---:|---:|---:|---:|---:|---:|---:|
| `UniversalCEFR/cambridge_exams_en` | 0 | 64 | 60 | 71 | 67 | 69 | 0 |
| `UniversalCEFR/cefr_asag_en` | 18 | 59 | 113 | 74 | 30 | 5 | 0 |
| `UniversalCEFR/cefr_sp_en` | 124 | 1271 | 3305 | 3330 | 1744 | 230 | 0 |
| `UniversalCEFR/elg_cefr_en` | 24 | 110 | 158 | 140 | 97 | 69 | 114 |
| `UniversalCEFR/icle500_en` | 0 | 1 | 29 | 99 | 91 | 64 | 211 |
| `UniversalCEFR/readme_en` | 182 | 673 | 623 | 896 | 377 | 71 | 0 |

Odd labels detail: `{'UniversalCEFR/elg_cefr_en': {'A1+': 1, 'A2+': 31, 'B1+': 48, 'B2+': 34}, 'UniversalCEFR/icle500_en': {'B2+': 119, 'B1+': 85, 'NA': 7}}`

## Reference pool (the M1 reading-classifier candidates)

Subsets whose `category` includes `reference`, i.e. texts written *for* learners rather than *by* them. Learner-production subsets are the M3 candidates (grading learner writing), not M1 training data.

| lang | reference rows | from subsets |
|---|---:|---|
| en | 13869 | `cambridge_exams_en`, `cefr_sp_en`, `elg_cefr_en`, `readme_en` |

Pooled level distribution for `en` reference subsets (class balance check):

| A1 | A2 | B1 | B2 | C1 | C2 | odd |
|---:|---:|---:|---:|---:|---:|---:|
| 330 | 2118 | 4146 | 4437 | 2285 | 439 | 114 |

---
Notes: word counts use whitespace tokenisation (approximate for ar/hi). Licenses are aggregated from the per-row `license` field; decisions and exclusions are recorded in `docs/adr/0003-datasets-and-licensing.md`.