# Team SinBotics LoRA adapter (`adapter/`)

This is Team SinBotics' LoRA adapter for the **Sinhala MMLU Shared Task @ ICTer 2026**, on top of [`Qwen/Qwen3.5-4B`](https://huggingface.co/Qwen/Qwen3.5-4B) (revision `851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a`). Code, inference script and predictions are in the main folder of this repo, [`SinhalaMMLU-SharedTask/team-sinbotics`](https://huggingface.co/SinhalaMMLU-SharedTask/team-sinbotics).

| | |
|---|---|
| Training data | Global-MMLU Sinhala (`CohereLabs/Global-MMLU`, `si`) + sinmulu_si (our Sri Lankan question collection): 25,799 training / 2,745 validation questions |
| Objective | the answer digit after a plain multiple-choice prompt with the reply pre-filled "Answer: " (loss on that token only) |
| LoRA | rank 16, alpha 32, dropout 0.05; attention, MLP and linear-attention projections of the 32 text layers; 30,474,240 parameters |
| Optimiser | AdamW, lr 1e-4, cosine, 3% warm-up, 32 questions per step, 1 epoch (807 steps), seed 42 |
| Hardware | 1× RTX 3090 24 GB, 8.8 h |
| Validation accuracy | 44.4% (base) → 54.8% |
| Parameters at inference | 4,205,751,296 (Qwen3.5-4B text model) + 30,474,240 (adapter), merged |

Files: `adapter_config.json` and `adapter_model.safetensors` (adapter), the tokenizer files of the base model, `train_config.json` and `train_log.jsonl` (training run).

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "Qwen/Qwen3.5-4B"; rev = "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a"
tok = AutoTokenizer.from_pretrained(base, revision=rev)
model = AutoModelForCausalLM.from_pretrained(base, revision=rev, dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "SinhalaMMLU-SharedTask/team-sinbotics", subfolder="adapter").merge_and_unload()
```
The final system adds Qwen's thinking mode and a teacher-role English reasoning prompt; see `inference.py` in the main repo.