Tetsuto/gemma-3-4b-11plus-tutor

Gemma 3 4B — UK 11+ Tutor LoRA (v1)

A LoRA adapter that fine-tunes Gemma 3 4B IT (QAT 4-bit) into a tutor for the UK 11+ exam — primary-school maths, English, and verbal reasoning.

~30 MB adapter. Trained in one overnight run on a 128 GB M5 Max. Designed to run on-device (iPhone, iPad, Mac).

🎮 Try it in the Hugging Face playground →

Headline result

Model	Accuracy on 150-Q holdout	Elo (vs base)
Gemma 3 4B base	43%	1079
Gemma 3 4B + this LoRA	89% (+46 pts)	1036 (−43)

Accuracy: programmatic match against the gold answer key. Elo: pairwise judgement by Qwen 3.6 27B over 180 forward+reverse comparisons.

What this LoRA does well

• Worked solutions for 11+ questions across maths, English, and verbal reasoning
• Outputs in a kid-friendly tone
• Reliably terminates with Answer: X for grading

What it doesn't do

• It's not a full tutoring conversation partner — it answers single questions, not multi-turn dialogues
• It does not give meta-strategies ("for compound-word questions, sound it out") — that's a planned v3
• It can be terse. We tried to fix this with a richer teacher-distillation v2; it actually made things worse. See the writeup for the full ablation story.

How to use

With mlx-lm (Apple Silicon)

pip install mlx-lm
python -m mlx_lm generate \
  --model mlx-community/gemma-3-4b-it-qat-4bit \
  --adapter-path <path-to-this-folder> \
  --prompt "Question: What is 7 × 8?\n\nA. 54  B. 56  C. 58  D. 64\n\nAnswer this 11+ maths question with a worked solution."

With Hugging Face transformers + peft

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("google/gemma-3-4b-it")
tokenizer = AutoTokenizer.from_pretrained("google/gemma-3-4b-it")
model = PeftModel.from_pretrained(base, "Tetsuto/gemma-3-4b-11plus-tutor")

Training details

• Base model: mlx-community/gemma-3-4b-it-qat-4bit (Gemma 3 4B IT, QAT 4-bit)
• Method: LoRA (rank 16, num_layers 16) via mlx-lm lora
• Dataset: ~10K worked solutions to 11+ questions, generated by Gemma 3 4B itself and post-filtered
• Iters: 6,000 with batch size 4, learning rate 1e-4, max_seq_length 1536
• Hardware: M5 Max 128 GB
• Training time: ~40 min

License

This adapter is released under the Gemma Terms of Use. The base model retains its original license; this adapter is a delta-weight derivative subject to the same terms (commercial use OK, attribution required, prohibited-uses clause flows through).

Citation

@misc{gemma3-4b-11plus-tutor,
  author       = {Jon Hammant},
  title        = {Gemma 3 4B — UK 11+ Tutor LoRA},
  year         = 2026,
  url          = {https://huggingface.co/Tetsuto/gemma-3-4b-11plus-tutor},
}

Limitations & ethical notes

• This model is for practice and education, not authoritative answer-keying. Always validate against the official mark-scheme.
• It can be confidently wrong — accuracy is 89%, meaning ~1 in 10 answers will be incorrect. Do not deploy to children without an adult-supervised review path.
• It inherits Gemma 3's known limitations and biases.

Read more

• Project writeup (the engineering story, including the v2/v2.5 ablation): link to GitHub
• Eval methodology + raw results: see REPORT_V2_5.md and results/ in the repo