Nyingi101/math-tutor-tinyllama-lora

Math Tutor TinyLlama LoRA — AI Math Tutor for Early Learners

S2.T3.1 · AIMS KTT Hackathon · Tier 3

A QLoRA fine-tuned adapter on TinyLlama-1.1B-Chat-v1.0 for adaptive, multilingual math tutoring of children aged 5–9 (P1–P3 numeracy). Trained on 2,000 instruction pairs in English, French, and Kinyarwanda.

Model Details

Property	Value
Base model	TinyLlama/TinyLlama-1.1B-Chat-v1.0
Fine-tuning method	QLoRA (4-bit NF4 + LoRA rank=8)
Trainable parameters	2,252,800 / 1,102,301,184 (0.204%)
Training loss	0.4768 (converged at epoch 3)
Training time	793 s on Tesla T4 (15.6 GB VRAM)
Adapter size	8.6 MB safetensors
Languages	English · French · Kinyarwanda (+ code-switching)
Target age	5–9 years (P1–P3)
Skills	counting, number sense, addition, subtraction, word problems

Training Details

Dataset

• 2,000 instruction pairs generated by scripts/make_instruction_data.py
• Equal split: ~667 EN / 667 FR / 666 KIN
• Format: system prompt + child context → feedback response
• Skills covered: all 5 numeracy sub-skills across difficulty 1–10

LoRA Configuration

LoraConfig(
    r=8,
    lora_alpha=16,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM,
)

Training Arguments

TrainingArguments(
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    fp16=True,
    lr_scheduler_type="cosine",
    warmup_steps=50,
)

Training Infrastructure

Trained on Modal.com using a Tesla T4 GPU (free tier). Reproducible with:

pip3 install modal && modal token new
modal run scripts/train_modal.py

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, "Nyingi101/math-tutor-tinyllama-lora")

prompt = (
    "<|system|>\nYou are a friendly math tutor for young children.</s>\n"
    "<|user|>\nChild answered 3+2=6 (wrong). Give gentle encouragement.</s>\n"
    "<|assistant|>\n"
)
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=80)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Multilingual Example

# Kinyarwanda feedback
prompt_kin = (
    "<|system|>\nUri umwarimu w'imibare w'abana bato.</s>\n"
    "<|user|>\nUmwana yabaze 4+3=7 (nibyo). Muhe ishimwe.</s>\n"
    "<|assistant|>\n"
)

# French feedback
prompt_fr = (
    "<|system|>\nTu es un tuteur de mathématiques pour les jeunes enfants.</s>\n"
    "<|user|>\nL'enfant a répondu 5-2=3 (correct). Encourage-le.</s>\n"
    "<|assistant|>\n"
)

Full Project

This adapter is part of the AI Math Tutor for Early Learners system:

• GitHub: github.com/Josephnyingi/ai-math-tutor
• Architecture: Gradio UI · BKT/Elo knowledge tracing · Whisper-tiny ASR · AES-256-GCM SQLite · differential privacy sync
• Footprint: 24 MB tutor/ directory (target ≤ 75 MB ✓)
• Offline-first: zero network calls at inference; works without WiFi
• Languages: English, French, Kinyarwanda, code-switched input

Knowledge Tracing Results

Model	AUC	N predictions
BKT	0.5677	2,400
Elo baseline	0.5203	2,400
Delta	+0.0474	BKT wins

Intended Use

• Adaptive math tutoring for children aged 5–9 in Rwanda and similar contexts
• Multilingual feedback generation (EN/FR/KIN)
• Offline deployment on low-cost tablets (≤ 75 MB footprint)

Limitations

• Base model is TinyLlama-1.1B; responses may occasionally be grammatically imperfect in Kinyarwanda
• Trained on synthetic instruction data; real child interaction data would improve performance
• This adapter provides feedback text only; the full system (ASR, knowledge tracing, UI) is in the GitHub repo

License

Apache 2.0 — same as the TinyLlama base model.