Upload README.md with huggingface_hub

README.md

@@ -35,11 +35,42 @@ tags:
 ## Datasets
 
 ### Extra training data (for the LoRA adapter)
-| Dataset | Source and licence | Size | Preparation |
+| Dataset | Source / licence | Questions | Train / validation |
 |---|---|---|---|
-| **Global-MMLU, Sinhala** | [`CohereLabs/Global-MMLU`](https://huggingface.co/datasets/CohereLabs/Global-MMLU), config `si`, revision `0e619dbeb34206cd48705a1a0ea7fb21cae09993`, Apache-2.0. MMLU translated into Sinhala; 57 subjects, categories humanities 4,770, STEM 3,245, social science 3,137, other 2,433, business 739. | 14,042 test + 285 dev questions → **14,324** | Merged test and dev, dropped 3 questions with an empty option, removed metadata, converted to the competition format with ids `gsi00001…`, and split per subject into 12,926 training / 1,398 validation questions (10%, seed 42). Code: `training/sinmmlu/gmmlu.py`. |
-| **sinmulu_si** (our own) | Built by our team from 3,486 Sri Lankan exam-paper PDFs (G.C.E. O/L and A/L papers and provincial term tests). Most PDFs use legacy Sinhala fonts, so text extraction was unusable; instead, each page was rendered as an image and transcribed by a large language model, in batches by several agents working in parallel on separate slices. Only Sinhala-medium papers and true multiple-choice questions were kept. Calculation-heavy subjects (Mathematics, Accounting, Statistics) and English, Tamil and other language papers were excluded. Questions that depended on diagrams, or whose scan was too poor to read confidently, were dropped rather than guessed. Stems and options were copied verbatim. Answers come from official marking schemes where available, matched to papers by grade, year and province; otherwise an answer was kept only if the model was highly confident, and left blank if not, so **some answers are model-derived**. Records were then validated, de-duplicated and cleaned. The full corpus holds 15,355 four-option MCQs across about 48 subjects (each with a Sinhala and an English subject name). After the overlap check against the test set (below), 15,190 remain (ids `osi00001`–`osi15190`), under 57 subject names, e.g. e.g. Agricultural Science 990, Home Economics 971, Biology 944, ICT 890, Communication and Media Studies 860, History 779, Geography 672, Buddhism 647. **Not published here; we will publish it separately, and it is available to the organisers on request.** | 15,190 → **15,180** | Removed 10 duplicates (same question and options), checked for overlap with the shared-task test set, and split per subject into 13,749 training / 1,431 validation questions (10%, seed 42). Code: `training/sinmmlu/sinmulu.py`. |
-| **Combined** (used for the final adapter) | union of the two above | **26,675 training / 2,829 validation** | A validation question whose text also appears in training would be moved to training (none did). Questions longer than 1,024 tokens were skipped in training, leaving 25,799 / 2,745. The answer distribution in training is 1: 23.1%, 2: 25.9%, 3: 26.4%, 4: 24.7%. Code: `training/sinmmlu/combined.py`. |
+| **Global-MMLU, Sinhala** | [`CohereLabs/Global-MMLU`](https://huggingface.co/datasets/CohereLabs/Global-MMLU) (`si`), Apache-2.0 | 14,324 | 12,926 / 1,398 |
+| **sinmulu_si** (our own) | Sri Lankan exam papers; not published here (available on request) | 15,180 | 13,749 / 1,431 |
+| **Combined** (final adapter) | union of the two | 29,504 | 26,675 / 2,829 |
+
+Validation sets are 10% per subject (seed 42).
+
+**Global-MMLU, Sinhala** (`training/sinmmlu/gmmlu.py`)
+- MMLU translated into Sinhala: 57 subjects. Categories: humanities 4,770, STEM 3,245, social science 3,137, other 2,433, business 739.
+- Pinned revision `0e619dbeb34206cd48705a1a0ea7fb21cae09993`. We merged the test (14,042) and dev (285) splits and dropped 3 questions with an empty option.
+- We removed the metadata, converted the questions to the competition format with ids `gsi00001…`, and split them per subject.
+
+**sinmulu_si** (`training/sinmmlu/sinmulu.py`)
+- **Source:** built by our team from 3,486 Sri Lankan exam-paper PDFs: G.C.E. O/L and A/L papers and provincial term tests.
+- **Transcription:**
+  - Most PDFs use legacy Sinhala fonts, so text extraction was unusable. Instead, each page was rendered as an image and transcribed by a large language model.
+  - This ran in batches, with several agents working in parallel on separate slices.
+  - Stems and options were copied verbatim.
+- **Selection:**
+  - Only Sinhala-medium papers and true multiple-choice questions were kept.
+  - Calculation-heavy subjects (Mathematics, Accounting, Statistics) and English, Tamil and other language papers were excluded.
+  - Questions that depended on diagrams, or whose scan was too poor to read confidently, were dropped rather than guessed.
+- **Answers:**
+  - These come from official marking schemes where available, matched to papers by grade, year and province.
+  - Otherwise, an answer was kept only if the model was highly confident, and left blank if not, so some answers are model-derived.
+- **Cleaning:** records were validated, de-duplicated and cleaned. That gave 15,355 four-option MCQs across about 48 subjects.
+- **Final set:**
+  - After the overlap check against the test set (below), 15,190 remain (`osi00001`–`osi15190`). Removing 10 more duplicates (same question and options) leaves 15,180.
+  - Largest subjects: Agricultural Science 990, Home Economics 971, Biology 944, ICT 890, Communication and Media Studies 860, History 779.
+- **Availability:** we will publish it separately; it is available to the organisers on request.
+
+**Combined** (`training/sinmmlu/combined.py`)
+- A validation question whose text also appeared in training would have been moved to training; none did.
+- Questions longer than 1,024 tokens were skipped in training, leaving 25,799 / 2,745.
+- Training answers are spread evenly: 1: 23.1%, 2: 25.9%, 3: 26.4%, 4: 24.7%.
 
 ### Overlap check against the test set (sinmulu_si)
 Real exam papers can repeat questions, so before training we checked sinmulu_si (15,355 questions) against all 2,238 shared-task test questions: