Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,210 @@
+---
+base_model: unsloth/gemma-4-E4B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/gemma-4-E4B
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.21.2

adapter_config.json

@@ -0,0 +1,47 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Gemma4ForConditionalGeneration",
+    "parent_library": "unsloth_compiled_module_gemma4",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/gemma-4-E4B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.2",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "(?:.*?(?:language|text).*?(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer).*?\\.(?:k_proj|q_proj|v_proj|o_proj|gate_proj|up_proj|down_proj|per_layer_input_gate|per_layer_projection|linear|embedding_projection|relative_k_proj))|(?:\\bmodel\\.layers\\.[\\d]{1,}(?!.*\\.experts\\.\\d)\\..*?(?:self_attn|attention|attn|mixer|mlp|feed_forward|ffn|dense|mixer).*?\\.(?:(?:k_proj|q_proj|v_proj|o_proj|gate_proj|up_proj|down_proj|per_layer_input_gate|per_layer_projection|linear|embedding_projection|relative_k_proj)))",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:595aa89a198262b8de9332d3de1fe563678ee1566635d9c463a2b3c65154f2a9
+size 293689248

processor_config.json

@@ -0,0 +1,75 @@
+{
+  "audio_ms_per_token": 40,
+  "audio_seq_length": 750,
+  "feature_extractor": {
+    "dither": 0.0,
+    "feature_extractor_type": "Gemma4AudioFeatureExtractor",
+    "feature_size": 128,
+    "fft_length": 512,
+    "fft_overdrive": false,
+    "frame_length": 320,
+    "hop_length": 160,
+    "input_scale_factor": 1.0,
+    "max_frequency": 8000.0,
+    "mel_floor": 0.001,
+    "min_frequency": 0.0,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "per_bin_mean": null,
+    "per_bin_stddev": null,
+    "preemphasis": 0.0,
+    "preemphasis_htk_flavor": true,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_processor_type": "Gemma4ImageProcessor",
+    "image_seq_length": 280,
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 280,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098
+  },
+  "image_seq_length": 280,
+  "processor_class": "Gemma4Processor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 70,
+    "num_frames": 32,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "video_processor_type": "Gemma4VideoProcessor"
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12bac982b793c44b03d52a250a9f0d0b666813da566b910c24a6da0695fd11e6
+size 32170070

tokenizer_config.json

@@ -0,0 +1,248 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<eos>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "processor_class": "Gemma4Processor",
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "46": {
+      "content": "<|tool>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "47": {
+      "content": "<tool|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "48": {
+      "content": "<|tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "49": {
+      "content": "<tool_call|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "50": {
+      "content": "<|tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "51": {
+      "content": "<tool_response|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "52": {
+      "content": "<|\"|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "98": {
+      "content": "<|think|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "100": {
+      "content": "<|channel>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "101": {
+      "content": "<channel|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "105": {
+      "content": "<|turn>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "106": {
+      "content": "<turn|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "255999": {
+      "content": "<|image>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "256000": {
+      "content": "<|audio>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258880": {
+      "content": "<|image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258881": {
+      "content": "<|audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258882": {
+      "content": "<image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258883": {
+      "content": "<audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "258884": {
+      "content": "<|video|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

training/make_predictions.py

@@ -0,0 +1,387 @@
+#!/usr/bin/env python
+"""
+Produce the shared-task submission: predictions.zip containing predictions.jsonl.
+
+Required line format (one JSON object per question):
+    {"q_no":1,"subject":"Arts","category":"humanities","level":"easy","answer":"2"}
+
+`answer` is a STRING holding the 1-based option number.
+
+    python make_predictions.py \
+        --data /workspace/data/TEST \
+        --model /workspace/outputs/mcq_sft_base_v2/best_adapter \
+        --out predictions.jsonl --zip predictions.zip
+
+--data may be either a single folder of *.json files, or a root containing easy/ medium/ hard/
+subfolders; the level comes from metadata.difficulty, falling back to the folder name.
+
+Scoring: for every option, score the joint tokenisation of prompt+" N" and take the highest total
+log-probability ("loglik"). This is deterministic, always yields a valid 1..N answer, and measured
+highest on TEST/easy. Because the task is submitted as PREDICTIONS, we are free to choose the
+decoding method - we are not restricted to the organisers' bare-digit logit comparison.
+
+Reproducibility: the organisers re-run submitted code on their own hardware and only verified runs
+enter the final ranking. Nothing here samples; seeds are fixed and generation is not used. Keep this
+file, the adapter/model reference and the prompt builder together in the submitted code.
+"""
+import argparse
+import glob
+import json
+import os
+import re
+import random
+import sys
+import time
+import zipfile
+
+os.environ.setdefault("HF_HUB_ENABLE_HF_TRANSFER", "1")
+
+# --- the organisers' prompt, verbatim from their util_prompt.py -------------------------------
+SINHALA_INSTRUCTIONS = {
+    "easy": "පහත ප්‍රශ්නයට 1, 2, 3, 4 යන පිළිතුරුවලින් නිවැරදි හෝ ඉතාමත් ගැළපෙන පිළිතුර තෝරන්න.\n\n",
+    "medium": "පහත ප්‍රශ්නයට 1, 2, 3, 4  යන පිළිතුරුවලින් නිවැරදි හෝ ඉතාමත් ගැළපෙන පිළිතුර තෝරාගන්න.\n\n",
+    "hard": "පහත ප්‍රශ්නයට 1, 2, 3, 4, 5 යන පිළිතුරුවලින් නිවැරදි හෝ ඉතාමත් ගැළපෙන පිළිතුර තෝරාගන්න.\n\n",
+}
+ENGLISH_INSTRUCTIONS = {
+    "easy": "Choose the correct or most appropriate answer from 1, 2, 3, 4 for the question below.\n\n",
+    "medium": "Choose the correct or most appropriate answer from 1, 2, 3, 4 for the question below.\n\n",
+    "hard": "Select the correct answer from 1, 2, 3, 4, 5 for the given question.\n\n",
+}
+LEVELS = ("easy", "medium", "hard")
+
+# Prompt variants to ensemble over. Each is a real configuration the organisers' harness supports
+# (sinhala/english x intro true/false), plus option rotations to cancel answer-position bias.
+# "rot" rotates the DISPLAYED option order; predictions are mapped back to the original indices.
+VARIANTS = {
+    "si_intro":    {"lang": "si", "intro": True,  "rot": 0},
+    "si_nointro":  {"lang": "si", "intro": False, "rot": 0},
+    "en_intro":    {"lang": "en", "intro": True,  "rot": 0},
+    "si_rot1":     {"lang": "si", "intro": True,  "rot": 1},
+    "si_rot2":     {"lang": "si", "intro": True,  "rot": 2},
+}
+DEFAULT_ENSEMBLE = ["si_intro", "si_nointro", "en_intro", "si_rot1", "si_rot2"]
+
+
+def fit_numbers(instruction, n):
+    """The organisers hard-code "1, 2, 3, 4, 5" in the HARD instruction because the original
+    SinhalaMMLU hard split had five options. SinhalaMMLU 2.0 dropped one incorrect option, so hard
+    questions now have four. Telling the model to choose from five options when four are shown is
+    strictly worse, so the number run is rewritten to match the actual count."""
+    return re.sub(r"1,\s*2,\s*3,\s*4(?:,\s*5)?",
+                  ", ".join(str(i + 1) for i in range(n)), instruction, count=1)
+
+
+def official_prompt(level, subject_original, question, choices, intro_type=True):
+    intro = f"මෙය {subject_original} විෂයයට අදාළ බහුවරණ ප්‍රශ්නයකි.\n" if intro_type else ""
+    body = "\n".join(f"{i + 1}. {str(c).strip()}" for i, c in enumerate(choices))
+    instr = fit_numbers(SINHALA_INSTRUCTIONS[level], len(choices))
+    return intro + instr + f"ප්‍රශ්නය: {question}\n{body}\nපිළිතුර:"
+
+
+def english_prompt(level, subject, question, choices, intro_type=True):
+    """The organisers' prompt with language="english" (their english_intro uses the ENGLISH subject)."""
+    intro = (f"This is a multiple choice question related to the subject {subject}. "
+             "Given in Sinhala Language.\n") if intro_type else ""
+    body = "\n".join(f"{i + 1}. {str(c).strip()}" for i, c in enumerate(choices))
+    instr = fit_numbers(ENGLISH_INSTRUCTIONS[level], len(choices))
+    return intro + instr + f"Question: {question}\n{body}\nAnswer:"
+
+
+def rotate(choices, rot):
+    """Displayed order; displayed position j holds original option (j + rot) % n."""
+    n = len(choices)
+    r = rot % n if n else 0
+    return choices[r:] + choices[:r], r
+
+
+def variant_prompt(row, spec):
+    shown, r = rotate(row["choices"], spec["rot"])
+    if spec["lang"] == "en":
+        p = english_prompt(row["level"], row["subject"], row["question"], shown, spec["intro"])
+    else:
+        p = official_prompt(row["level"], row["subject_original"], row["question"], shown, spec["intro"])
+    return p, r, len(shown)
+
+
+def level_of(item, path, override=""):
+    """metadata.difficulty, else a top-level 'level'/'difficulty' field, else the file name or any
+    path component containing easy/medium/hard, else 'easy'."""
+    if override:
+        return override
+    md = item.get("metadata") or {}
+    for cand in (md.get("difficulty"), md.get("level"), item.get("level"), item.get("difficulty")):
+        d = str(cand or "").strip().lower()
+        if d in LEVELS:
+            return d
+    low = path.replace("\\", "/").lower()
+    # substring match so "hard.json", "test_hard.jsonl" and ".../hard/..." all work
+    hits = [lv for lv in LEVELS if lv in low]
+    if len(hits) == 1:
+        return hits[0]
+    if hits:                                  # e.g. ".../easy/medium_extra.json": trust the last one
+        return max(hits, key=low.rfind)
+    return "easy"
+
+
+def read_any(path):
+    """A JSON array, a JSONL file, or a dict wrapping the list under a common key."""
+    raw = open(path, encoding="utf-8-sig").read().strip()
+    if not raw:
+        return []
+    if raw[0] == "[":
+        return json.loads(raw)
+    if raw[0] == "{":
+        try:
+            obj = json.loads(raw)
+        except json.JSONDecodeError:
+            obj = None
+        if isinstance(obj, dict):
+            for k in ("data", "questions", "items", "rows", "test"):
+                if isinstance(obj.get(k), list):
+                    return obj[k]
+            return [obj]
+        # not one object -> treat as JSONL
+    out = []
+    for line in raw.splitlines():
+        line = line.strip()
+        if line:
+            out.append(json.loads(line))
+    return out
+
+
+def load_items(data_dir, level_override=""):
+    if os.path.isfile(data_dir):
+        files = [data_dir]
+    else:
+        files = sorted(set(glob.glob(os.path.join(data_dir, "**", "*.json"), recursive=True))
+                       | set(glob.glob(os.path.join(data_dir, "**", "*.jsonl"), recursive=True)))
+    if not files:
+        sys.exit(f"no .json/.jsonl files found under {data_dir}")
+    rows, skipped, per_file = [], 0, []
+    for path in files:
+        try:
+            data = read_any(path)
+        except Exception as e:                                           # noqa: BLE001
+            print(f"[warn] {os.path.basename(path)}: {type(e).__name__}: {e}")
+            continue
+        if not isinstance(data, list) or not data:
+            print(f"[warn] {os.path.basename(path)}: no question list found")
+            continue
+        before = len(rows)
+        for it in data:
+            md = it.get("metadata") or {}
+            ch = [c for c in (it.get("choices") or []) if str(c).strip()]
+            if len(ch) < 2:
+                skipped += 1          # never drop silently: these still need a prediction line
+            rows.append({
+                "file": os.path.basename(path),
+                "q_no": it.get("q_no"),
+                "subject": it.get("subject"),
+                "category": it.get("category"),
+                "level": level_of(it, path, level_override),
+                "question": str(it.get("question") or "").strip(),
+                "choices": ch,
+                "subject_original": md.get("subject_original") or it.get("subject"),
+            })
+        lv = {r["level"] for r in rows[before:]}
+        per_file.append((os.path.basename(path), len(rows) - before, sorted(lv)))
+    by_level = {}
+    for r in rows:
+        by_level[r["level"]] = by_level.get(r["level"], 0) + 1
+    print(f"[data] {len(rows)} questions from {len(files)} file(s); by level: {by_level}")
+    for name, n, lv in per_file:
+        print(f"    {name:<44} {n:>6} questions   level -> {','.join(lv)}")
+    missing = [lv for lv in LEVELS if lv not in by_level]
+    if missing:
+        print(f"[warn] no questions detected for: {missing}. If that is wrong, the level could not be "
+              "read from metadata.difficulty or the file name - check the table above.")
+    if skipped:
+        print(f"[warn] {skipped} question(s) have fewer than 2 usable options; they still get a line")
+    return rows
+
+
+def common_prefix_len(a, b):
+    n = 0
+    for x, y in zip(a, b):
+        if x != y:
+            break
+        n += 1
+    return n
+
+
+def predict(model, tok, rows, max_len, batch_seqs, variants):
+    """Deterministic prompt ensemble. For every (question, variant) the options are scored, converted
+    to a log-softmax over the DISPLAYED positions (so variants with different prompt lengths are on a
+    common scale), mapped back to the original option order, and summed across variants.
+    No sampling, no generation: the same inputs always give the same predictions."""
+    import math
+    import torch
+    model.eval()
+    dev = next(model.parameters()).device
+    pad = tok.pad_token_id if tok.pad_token_id is not None else 0
+
+    items, meta = [], {}                 # (ri, vi, shown_pos, ids, answer_start) ; (ri,vi)->(rot,n)
+    for ri, r in enumerate(rows):
+        if len(r["choices"]) < 2:
+            continue
+        for vi, vname in enumerate(variants):
+            prompt, rot, n = variant_prompt(r, VARIANTS[vname])
+            meta[(ri, vi)] = (rot, n)
+            p_ids = tok(prompt)["input_ids"]
+            for j in range(n):
+                f = tok(prompt + f" {j + 1}")["input_ids"]
+                if len(f) > max_len:
+                    f = f[-max_len:]
+                items.append((ri, vi, j, f, max(1, common_prefix_len(p_ids, f))))
+
+    print(f"[predict] {len(rows)} questions x {len(variants)} variant(s) = {len(items)} candidates")
+    scores, t0 = {}, time.time()
+    with torch.no_grad():
+        for i in range(0, len(items), batch_seqs):
+            chunk = items[i:i + batch_seqs]
+            L = max(len(x[3]) for x in chunk)
+            ids = torch.full((len(chunk), L), pad, dtype=torch.long)
+            mask = torch.zeros((len(chunk), L), dtype=torch.long)
+            for b, (_, _, _, sq, _) in enumerate(chunk):
+                ids[b, :len(sq)] = torch.tensor(sq)
+                mask[b, :len(sq)] = 1
+            logits = model(input_ids=ids.to(dev), attention_mask=mask.to(dev)).logits
+            for b, (ri, vi, j, sq, k) in enumerate(chunk):
+                lp = logits[b, k - 1:len(sq) - 1].float().log_softmax(-1)
+                tgt = ids[b, k:len(sq)].to(dev)
+                scores[(ri, vi, j)] = lp.gather(-1, tgt[:, None]).sum().item()
+            del logits
+            if (i // max(1, batch_seqs)) % 100 == 0:
+                done = min(i + batch_seqs, len(items))
+                print(f"  scored {done}/{len(items)} ({time.time() - t0:.0f}s)", flush=True)
+
+    preds, fallback, agree = [], 0, 0
+    for ri, r in enumerate(rows):
+        n = len(r["choices"])
+        if n < 2:
+            preds.append(1)
+            fallback += 1
+            continue
+        total, used, per_variant = [0.0] * n, 0, []
+        for vi in range(len(variants)):
+            if (ri, vi) not in meta:
+                continue
+            rot, nv = meta[(ri, vi)]
+            raw = [scores.get((ri, vi, j)) for j in range(nv)]
+            if any(x is None for x in raw):
+                continue
+            m = max(raw)                                     # log-softmax over displayed positions
+            z = m + math.log(sum(math.exp(x - m) for x in raw))
+            best_j = max(range(nv), key=lambda j: raw[j])
+            per_variant.append(1 + (best_j + rot) % n)
+            for j in range(nv):
+                total[(j + rot) % n] += raw[j] - z            # displayed j == original (j+rot)%n
+            used += 1
+        if used == 0:
+            preds.append(1)
+            fallback += 1
+            continue
+        preds.append(1 + max(range(n), key=lambda i: total[i]))
+        if len(set(per_variant)) == 1:
+            agree += 1
+
+    if fallback:
+        print(f"[warn] {fallback} question(s) fell back to answer 1")
+    if len(variants) > 1:
+        print(f"[predict] all variants agreed on {agree}/{len(rows)} ({agree / max(1, len(rows)):.1%}) "
+              "- the rest are where the ensemble changes the answer")
+    print(f"[predict] done in {time.time() - t0:.0f}s")
+    return preds, None
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--data", required=True, help="folder of test *.json files, or a root with easy/medium/hard")
+    ap.add_argument("--model", required=True, help="adapter dir, merged dir, or HF repo")
+    ap.add_argument("--out", default="predictions.jsonl")
+    ap.add_argument("--zip", default="predictions.zip", help="'' to skip zipping")
+    ap.add_argument("--level", default="", choices=["", *LEVELS], help="force the level for every item")
+    ap.add_argument("--max_seq_length", type=int, default=1024)
+    ap.add_argument("--eval_batch", type=int, default=16)
+    ap.add_argument("--ensemble", nargs="*", default=None,
+                    help=f"prompt variants to average over (default: {' '.join(DEFAULT_ENSEMBLE)}). "
+                         f"Choices: {' '.join(VARIANTS)}. Pass 'si_intro' alone for single-prompt scoring.")
+    ap.add_argument("--seed", type=int, default=3407)
+    ap.add_argument("--dry_run", action="store_true", help="show the prompt and exit, no model load")
+    args = ap.parse_args()
+    for stream in (sys.stdout, sys.stderr):
+        try:
+            stream.reconfigure(encoding="utf-8")
+        except Exception:                                                # noqa: BLE001
+            pass
+
+    random.seed(args.seed)
+    variants = args.ensemble if args.ensemble else list(DEFAULT_ENSEMBLE)
+    unknown = [v for v in variants if v not in VARIANTS]
+    if unknown:
+        sys.exit(f"unknown variant(s) {unknown}; choose from {list(VARIANTS)}")
+    print(f"[ensemble] {len(variants)} variant(s): {variants}")
+    rows = load_items(args.data, args.level)
+
+    if args.dry_run:
+        r = next((x for x in rows if len(x["choices"]) >= 2), None)
+        if r:
+            for v in variants:
+                prompt, rot, _ = variant_prompt(r, VARIANTS[v])
+                print(f"\n--- variant {v} (rot={rot}) ---")
+                print(prompt)
+            print("--- end ---")
+        return
+
+    import torch
+    torch.manual_seed(args.seed)
+    from unsloth import FastModel
+    print(f"[model] loading {args.model}")
+    model, processor = FastModel.from_pretrained(model_name=args.model, dtype=None,
+                                                 max_seq_length=args.max_seq_length,
+                                                 load_in_4bit=False, full_finetuning=False)
+    tok = getattr(processor, "tokenizer", processor)
+    if tok.pad_token is None:
+        tok.pad_token = tok.eos_token
+
+    preds, _ = predict(model, tok, rows, args.max_seq_length, args.eval_batch, variants)
+
+    with open(args.out, "w", encoding="utf-8") as f:
+        for r, p in zip(rows, preds):
+            f.write(json.dumps({"q_no": r["q_no"], "subject": r["subject"],
+                                "category": r["category"], "level": r["level"],
+                                "answer": str(p)}, ensure_ascii=False) + "\n")
+    print(f"[out] {len(rows)} lines -> {args.out}")
+
+    # self-checks: the submission is rejected or mis-scored if any of these are wrong
+    lines = [json.loads(l) for l in open(args.out, encoding="utf-8") if l.strip()]
+    assert len(lines) == len(rows), "line count does not match the question count"
+    bad_keys = [i for i, d in enumerate(lines)
+                if set(d) != {"q_no", "subject", "category", "level", "answer"}]
+    bad_ans = [i for i, d in enumerate(lines)
+               if not (isinstance(d["answer"], str) and d["answer"] in {"1", "2", "3", "4", "5"})]
+    bad_lvl = [i for i, d in enumerate(lines) if d["level"] not in LEVELS]
+    for name, bad in [("wrong key set", bad_keys), ("answer not a '1'..'5' string", bad_ans),
+                      ("level not easy/medium/hard", bad_lvl)]:
+        print(f"[check] {name}: {len(bad)}" + (f"  e.g. line {bad[0] + 1}" if bad else "  OK"))
+    dist = {}
+    for d in lines:
+        dist[d["answer"]] = dist.get(d["answer"], 0) + 1
+    print(f"[check] answer distribution: {dict(sorted(dist.items()))}")
+    dup = len(lines) - len({(d["level"], d["subject"], d["q_no"]) for d in lines})
+    print(f"[check] duplicate (level, subject, q_no) keys: {dup}")
+    if bad_keys or bad_ans or bad_lvl:
+        sys.exit("fix the problems above before submitting")
+
+    if args.zip:
+        with zipfile.ZipFile(args.zip, "w", zipfile.ZIP_DEFLATED) as z:
+            z.write(args.out, arcname="predictions.jsonl")   # must sit at the archive root
+        with zipfile.ZipFile(args.zip) as z:
+            print(f"[zip] {args.zip} contains: {z.namelist()}")
+    print("\nFirst two lines:")
+    for l in open(args.out, encoding="utf-8").readlines()[:2]:
+        print("  " + l.rstrip())
+
+
+if __name__ == "__main__":
+    main()

training/results_v3b_final_easy.json

@@ -0,0 +1,65 @@
+{
+ "model": "/workspace/outputs/mcq_sft_base_v3b/final_adapter",
+ "data": "/workspace/data/TEST/easy",
+ "n_questions": 1850,
+ "loglik": {
+  "accuracy": 0.5756756756756757,
+  "n": 1850,
+  "no_prediction": 0,
+  "by_category": {
+   "humanities": 0.5327022375215146,
+   "social_science": 0.7127371273712737,
+   "language": 0.5096774193548387,
+   "stem": 0.6341463414634146
+  },
+  "by_subject": {
+   "Arts": 0.49,
+   "Buddhism": 0.5,
+   "Catholicism": 0.6363636363636364,
+   "Christianity": 0.7153846153846154,
+   "Civics": 0.7398373983739838,
+   "Eastern_music": 1.0,
+   "Eastern music": 0.44366197183098594,
+   "Geography": 0.646551724137931,
+   "Health and Physical Science": 0.7461538461538462,
+   "History": 0.47019867549668876,
+   "Islam": 0.6759259259259259,
+   "Sinhala_language_and_literature": 0.0,
+   "Sinhala language and literature": 0.512987012987013,
+   "dancing": 0.3055555555555556,
+   "drama_and_theatre": 0.0,
+   "drama and Theatre ": 0.5590551181102362,
+   "science": 0.6341463414634146
+  }
+ },
+ "official": {
+  "accuracy": 0.5708108108108109,
+  "n": 1850,
+  "no_prediction": 0,
+  "by_category": {
+   "humanities": 0.5258175559380379,
+   "social_science": 0.7046070460704607,
+   "language": 0.5419354838709678,
+   "stem": 0.6158536585365854
+  },
+  "by_subject": {
+   "Arts": 0.48,
+   "Buddhism": 0.47530864197530864,
+   "Catholicism": 0.6363636363636364,
+   "Christianity": 0.7384615384615385,
+   "Civics": 0.7398373983739838,
+   "Eastern_music": 1.0,
+   "Eastern music": 0.4295774647887324,
+   "Geography": 0.6379310344827587,
+   "Health and Physical Science": 0.7307692307692307,
+   "History": 0.4768211920529801,
+   "Islam": 0.6388888888888888,
+   "Sinhala_language_and_literature": 0.0,
+   "Sinhala language and literature": 0.5454545454545454,
+   "dancing": 0.3148148148148148,
+   "drama_and_theatre": 0.0,
+   "drama and Theatre ": 0.5433070866141733,
+   "science": 0.6158536585365854
+  }
+ }
+}

training/sft_gemma4_mcq.py

@@ -0,0 +1,282 @@
+#!/usr/bin/env python
+"""
+Supervised fine-tuning of Gemma 4 E4B for Sinhala multiple-choice QA (SinhalaMMLU-style), Unsloth + LoRA.
+
+Input: the prompt/completion files made by `mcq_dataset.py build-sft` (plain format, with a `candidates` field).
+The loss is computed ONLY on the answer tokens. Accuracy is measured separately by scoring every answer
+option on the held-out validation file (before training = zero-shot baseline, then after every epoch).
+
+  # base model
+  python sft_gemma4_mcq.py --train sft_train.jsonl --val sft_val.jsonl \
+      --model unsloth/gemma-4-E4B --out /workspace/outputs/mcq_sft_base
+  # CPT model (local merged folder from the earlier run)
+  python sft_gemma4_mcq.py --train sft_train.jsonl --val sft_val.jsonl \
+      --model /workspace/outputs/gemma4_e4b_sinhala_cpt/merged_16bit --out /workspace/outputs/mcq_sft_cpt
+  # zero-shot baseline only (no training)
+  python sft_gemma4_mcq.py --val sft_val.jsonl --model unsloth/gemma-4-E4B --eval_only
+
+Outputs: <out>/best_adapter (best validation accuracy), <out>/final_adapter, <out>/history.json
+To merge + upload the best adapter:  python upload_to_hf.py --adapter_dir <out>/best_adapter --make_merged ...
+"""
+import argparse
+import inspect
+import json
+import os
+import random
+import sys
+import time
+
+os.environ.setdefault("HF_HUB_ENABLE_HF_TRANSFER", "1")
+
+
+# ------------------------------------------------------------------ helpers (no GPU needed)
+def load_rows(path):
+    rows = [json.loads(l) for l in open(path, encoding="utf-8") if l.strip()]
+    for r in rows[:3]:
+        if not isinstance(r.get("prompt"), str) or not isinstance(r.get("candidates"), list):
+            sys.exit(f"{path}: need plain prompt/completion rows with a 'candidates' list "
+                     "(re-run mcq_dataset.py build-sft without --format chat)")
+    return rows
+
+
+def common_prefix_len(a, b):
+    n = 0
+    for x, y in zip(a, b):
+        if x != y:
+            break
+        n += 1
+    return n
+
+
+def encode_example(tok, prompt, completion, add_eos=True):
+    """Tokenise prompt+completion jointly (as a test harness would); mask everything before the first
+    token that differs from the prompt-only tokenisation. Returns input_ids, labels."""
+    p_ids = tok(prompt)["input_ids"]
+    f_ids = tok(prompt + completion)["input_ids"]
+    k = max(1, common_prefix_len(p_ids, f_ids))
+    ids = list(f_ids) + ([tok.eos_token_id] if add_eos and tok.eos_token_id is not None else [])
+    labels = [-100] * k + ids[k:]
+    return ids, labels
+
+
+def evaluate(model, tok, rows, max_len=1024, limit=0, batch_seqs=16, verbose=True):
+    """Accuracy of picking the option whose answer tokens get the highest total log-probability.
+    Every (question, candidate) pair is scored; pairs are batched together for speed."""
+    import torch
+    was_training = model.training
+    model.eval()
+    dev = next(model.parameters()).device
+    pad = tok.pad_token_id if tok.pad_token_id is not None else 0
+    use = rows[:limit] if limit else rows
+    t0 = time.time()
+
+    items, skipped = [], 0            # (row_idx, cand_idx, token_ids, answer_start)
+    for ri, r in enumerate(use):
+        p_ids = tok(r["prompt"])["input_ids"]
+        seqs, starts = [], []
+        for c in r["candidates"]:
+            f = tok(r["prompt"] + c)["input_ids"]
+            seqs.append(f)
+            starts.append(max(1, common_prefix_len(p_ids, f)))
+        if max(len(x) for x in seqs) > max_len:
+            skipped += 1
+            continue
+        for ci, (sq, k) in enumerate(zip(seqs, starts)):
+            items.append((ri, ci, sq, k))
+
+    scores = {}
+    with torch.no_grad():
+        for i in range(0, len(items), batch_seqs):
+            chunk = items[i:i + batch_seqs]
+            L = max(len(x[2]) for x in chunk)
+            ids = torch.full((len(chunk), L), pad, dtype=torch.long)
+            mask = torch.zeros((len(chunk), L), dtype=torch.long)
+            for b, (_, _, sq, _) in enumerate(chunk):
+                ids[b, :len(sq)] = torch.tensor(sq)
+                mask[b, :len(sq)] = 1
+            logits = model(input_ids=ids.to(dev), attention_mask=mask.to(dev)).logits
+            for b, (ri, ci, sq, k) in enumerate(chunk):
+                lp = logits[b, k - 1:len(sq) - 1].float().log_softmax(-1)
+                tgt = ids[b, k:len(sq)].to(dev)
+                scores[(ri, ci)] = lp.gather(-1, tgt[:, None]).sum().item()
+            del logits
+
+    n = correct = 0
+    by_subj, gold_pos = {}, {}
+    for ri, r in enumerate(use):
+        cs = [scores.get((ri, ci)) for ci in range(len(r["candidates"]))]
+        if any(c is None for c in cs):
+            continue
+        pred = max(range(len(cs)), key=lambda i: cs[i])
+        gold = r["candidates"].index(r["completion"])
+        n += 1
+        correct += int(pred == gold)
+        gold_pos[gold] = gold_pos.get(gold, 0) + 1
+        sj = r.get("subject") or "?"
+        c_, t_ = by_subj.get(sj, (0, 0))
+        by_subj[sj] = (c_ + int(pred == gold), t_ + 1)
+
+    if was_training:
+        model.train()
+    acc = correct / max(1, n)
+    majority = max(gold_pos.values()) / max(1, n) if gold_pos else 0.0
+    if verbose:
+        print(f"[eval] accuracy {acc:.4f} ({correct}/{n}); always-guess-most-common-position {majority:.4f}; "
+              f"skipped(too long) {skipped}; {time.time() - t0:.0f}s", flush=True)
+    return {"acc": acc, "n": n, "majority_baseline": majority,
+            "by_subject": {k: round(v[0] / v[1], 4) for k, v in sorted(by_subj.items()) if v[1] >= 20}}
+
+
+# ------------------------------------------------------------------ main
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--train", default="")
+    ap.add_argument("--val", required=True)
+    ap.add_argument("--model", default="unsloth/gemma-4-E4B")
+    ap.add_argument("--out", default="/workspace/outputs/mcq_sft")
+    ap.add_argument("--max_seq_length", type=int, default=1024)
+    ap.add_argument("--lora_r", type=int, default=32)
+    ap.add_argument("--lora_alpha", type=int, default=32)
+    ap.add_argument("--lr", type=float, default=1e-4)
+    ap.add_argument("--epochs", type=float, default=3.0)
+    ap.add_argument("--max_steps", type=int, default=-1, help=">0 overrides epochs (smoke test)")
+    ap.add_argument("--batch_size", type=int, default=8)
+    ap.add_argument("--grad_accum", type=int, default=4)
+    ap.add_argument("--warmup_steps", type=int, default=10)
+    ap.add_argument("--eval_limit", type=int, default=0, help="evaluate on the first N val rows only (0 = all)")
+    ap.add_argument("--eval_batch", type=int, default=16, help="candidate sequences scored per forward pass")
+    ap.add_argument("--eval_steps", type=int, default=0,
+                    help="also evaluate every N optimizer steps (0 = only at epoch ends). "
+                         "Use it when the model converges early: the best checkpoint may be mid-epoch.")
+    ap.add_argument("--skip_baseline", action="store_true")
+    ap.add_argument("--eval_only", action="store_true", help="just score the model on --val (zero-shot baseline)")
+    ap.add_argument("--seed", type=int, default=3407)
+    args = ap.parse_args()
+    if not args.eval_only and not args.train:
+        sys.exit("--train is required unless --eval_only")
+
+    from unsloth import FastModel  # must come before transformers / trl / peft
+    import torch
+    from datasets import Dataset
+    from transformers import DataCollatorForSeq2Seq, TrainerCallback
+    from trl import SFTConfig, SFTTrainer
+
+    random.seed(args.seed)
+    os.makedirs(args.out, exist_ok=True)
+    json.dump(vars(args), open(os.path.join(args.out, "run_config.json"), "w"), indent=1)
+    print(f"[model] loading {args.model}")
+    model, processor = FastModel.from_pretrained(model_name=args.model, dtype=None,
+                                                 max_seq_length=args.max_seq_length,
+                                                 load_in_4bit=False, full_finetuning=False)
+    tok = getattr(processor, "tokenizer", processor)
+    if tok.pad_token is None:
+        tok.pad_token = tok.eos_token
+
+    val_rows = load_rows(args.val)
+    print(f"[data] val rows: {len(val_rows)}")
+
+    if args.eval_only:
+        res = evaluate(model, tok, val_rows, args.max_seq_length, args.eval_limit, args.eval_batch)
+        json.dump(res, open(os.path.join(args.out, "eval_only.json"), "w"), ensure_ascii=False, indent=1)
+        return
+
+    model = FastModel.get_peft_model(
+        model, finetune_vision_layers=False, finetune_language_layers=True,
+        finetune_attention_modules=True, finetune_mlp_modules=True,
+        r=args.lora_r, lora_alpha=args.lora_alpha, lora_dropout=0, bias="none", random_state=args.seed,
+        use_gradient_checkpointing="unsloth")
+    try:
+        model.print_trainable_parameters()
+    except Exception:
+        pass
+
+    train_rows = load_rows(args.train)
+    enc, dropped = [], 0
+    for r in train_rows:
+        ids, labels = encode_example(tok, r["prompt"], r["completion"])
+        if len(ids) > args.max_seq_length:
+            dropped += 1
+            continue
+        enc.append({"input_ids": ids, "attention_mask": [1] * len(ids), "labels": labels})
+    print(f"[data] train examples: {len(enc)} (dropped {dropped} longer than {args.max_seq_length} tokens)")
+    train_ds = Dataset.from_list(enc)
+
+    coll_kw = dict(padding=True, label_pad_token_id=-100, pad_to_multiple_of=8)
+    coll_params = inspect.signature(DataCollatorForSeq2Seq.__init__).parameters
+    if "tokenizer" in coll_params:
+        collator = DataCollatorForSeq2Seq(tokenizer=tok, **coll_kw)
+    else:                                   # transformers >=5 renamed it
+        collator = DataCollatorForSeq2Seq(processing_class=tok, **coll_kw)
+    # --- masking sanity check: only the answer tokens may carry a label
+    b = collator([enc[0], enc[min(1, len(enc) - 1)]])
+    frac = float((b["labels"] != -100).float().mean())
+    shown = tok.decode([t for t in b["labels"][0].tolist() if t != -100])
+    print(f"[mask] labelled-token fraction {frac:.3f}; example 0 trains on: {shown!r}")
+    if frac > 0.3:
+        sys.exit("Masking looks wrong (too many labelled tokens). Stop and check the data format.")
+
+    history = []
+    best = {"acc": -1.0}
+
+    def record(tag, res, step):
+        history.append({"tag": tag, "step": step, **res})
+        json.dump(history, open(os.path.join(args.out, "history.json"), "w"), ensure_ascii=False, indent=1)
+
+    if not args.skip_baseline:
+        print("[baseline] zero-shot accuracy before training")
+        record("baseline", evaluate(model, tok, val_rows, args.max_seq_length, args.eval_limit, args.eval_batch), 0)
+
+    class EpochEval(TrainerCallback):
+        last = -1
+
+        def _run(self, tag, state, model):
+            if state.global_step == self.last:
+                return
+            self.last = state.global_step
+            res = evaluate(model, tok, val_rows, args.max_seq_length, args.eval_limit, args.eval_batch)
+            record(tag, res, state.global_step)
+            if res["acc"] > best["acc"]:
+                best.update(res, step=state.global_step)
+                d = os.path.join(args.out, "best_adapter")
+                model.save_pretrained(d)
+                processor.save_pretrained(d)
+                print(f"[save] new best accuracy {res['acc']:.4f} -> {d}")
+
+        def on_step_end(self, a, state, control, model=None, **kw):
+            if args.eval_steps > 0 and state.global_step > 0 and state.global_step % args.eval_steps == 0:
+                self._run(f"step {state.global_step}", state, model)
+
+        def on_epoch_end(self, a, state, control, model=None, **kw):
+            self._run(f"epoch {state.epoch:.2f}", state, model)
+
+        def on_train_end(self, a, state, control, model=None, **kw):
+            self._run("end", state, model)
+
+    cfg_kw = dict(
+        output_dir=os.path.join(args.out, "trainer"), per_device_train_batch_size=args.batch_size,
+        gradient_accumulation_steps=args.grad_accum, learning_rate=args.lr, lr_scheduler_type="cosine",
+        warmup_steps=args.warmup_steps, weight_decay=0.01, max_grad_norm=1.0,
+        num_train_epochs=args.epochs, max_steps=args.max_steps, bf16=True, optim="adamw_8bit",
+        logging_steps=10, save_strategy="no", seed=args.seed, report_to="none",
+        remove_unused_columns=False, dataset_text_field="", max_length=args.max_seq_length)
+    if "dataset_kwargs" in inspect.signature(SFTConfig.__init__).parameters:
+        cfg_kw["dataset_kwargs"] = {"skip_prepare_dataset": True}
+    trainer = SFTTrainer(model=model, processing_class=tok, train_dataset=train_ds,
+                         data_collator=collator, args=SFTConfig(**cfg_kw), callbacks=[EpochEval()])
+    stats = trainer.train()
+    print(f"[done] {stats.metrics.get('train_runtime', 0) / 60:.1f} min; "
+          f"peak VRAM {torch.cuda.max_memory_reserved() / 1024 ** 3:.1f} GB")
+
+    final = os.path.join(args.out, "final_adapter")
+    model.save_pretrained(final)
+    processor.save_pretrained(final)
+    print(f"[save] final adapter -> {final}")
+    print("\nValidation accuracy history:")
+    for h in history:
+        print(f"  {h['tag']:>12s}  step {h['step']:5d}  acc {h['acc']:.4f}  (n={h['n']}, "
+              f"majority-position baseline {h['majority_baseline']:.3f})")
+    print(f"best: {best['acc']:.4f} at step {best.get('step')}  -> {os.path.join(args.out, 'best_adapter')}")
+
+
+if __name__ == "__main__":
+    main()