Spaces:
Sleeping
Sleeping
| title: MERA Reason | |
| emoji: 🥇 | |
| colorFrom: green | |
| colorTo: indigo | |
| sdk: gradio | |
| app_file: app.py | |
| pinned: true | |
| license: other | |
| short_description: Оценка способностей к рассуждению на русском языке | |
| sdk_version: 5.43.1 | |
| tags: | |
| - leaderboard | |
| # MERA Reason Leaderboard | |
| Zip-only leaderboard for **ruAIME**, **T-math**, **Luzitania**, **MMReD** (open-reasoning track). | |
| ## Submission flow | |
| 1. Upload a MERA-style zip on the **Submit** tab (`ruaime.json`, `t_math.json`, `luzitania.json`, `mmred.json`, `logs_public/`). | |
| 2. Metrics are extracted from `results_*.json` inside `logs_public`. | |
| 3. Aggregated scores are saved to the results dataset repo and reloaded on every page visit. | |
| 4. Tables update immediately — no evaluation queue. | |
| ## Metrics | |
| | Benchmark | Main leaderboard metric | | |
| |-----------|-------------------------| | |
| | ruAIME | Exact match | | |
| | T-math | Exact match | | |
| | Luzitania | Exact match | | |
| | MMReD | `em.dc_aggregate` | | |
| The MMReD dataset tab also shows group-level `em.dc_aggregate` and per-subtask `exact_match`. | |
| ## Configuration | |
| - Tasks and column names: `src/about.py` | |
| - Zip parsing: `src/submission/process_zip.py` | |
| - Local storage: `src/leaderboard/store.py`, directory `results/` | |