Minmini-1B

Beats 105-billion-parameter Sarvam on Tamil inference and reading comprehension, at 1/100th the size. IndicXNLI 75.0% vs 58.2%; IndicQA F1 52.1 vs 44.5 (1,000 items each, same prompts).

Minmini (மின்மினி, "firefly") is a 1-billion-parameter Tamil language model built on Google's Gemma-3-1B-IT and fine-tuned entirely on one laptop. On Tamil-Bench's leaderboard items it scores higher than Gemini 3.8 Flash and Claude Sonnet 5.5 on Tamil natural-language inference (79.5%) and reading comprehension (IndicQA F1 55.1). It also translates English↔Tamil (FLORES chrF 46.8 / 45.1) and keeps the base model's step-by-step arithmetic in English.

These are supervised, in-format results: Minmini was fine-tuned on these task formats, while the models it is compared with were not. It is a small specialist, not a general assistant. See Limitations.

What Minmini is best for

Minmini is best at tasks that work from text you give it: answering from a passage, checking a claim against a statement, and translating. It is not a knowledge source, and its reasoning is reliable only for simple problems in English.

Use	Fit	Evidence
Answering questions from a Tamil passage (support docs, FAQs, forms)	Best	IndicQA F1 52.1 (answerable questions 47.8); says "no answer in the passage" reliably (AUC 0.89)
Checking whether a Tamil statement follows from, contradicts or is unrelated to a text	Best	IndicXNLI 75.0%, robust to answer-option order (75.0–75.7%)
English↔Tamil translation of short sentences	Good	FLORES chrF 46.8 / 45.1
Simple arithmetic and logic, in English	Fair	5/5 simple test questions in English, with correct step-by-step working
Simple arithmetic and logic, in Tamil	Weak	3/5 in Tamil; makes arithmetic slips (e.g. writes 12 − 5 = 5)
Drafting short Tamil text (messages, lists, short paragraphs)	Fair	Readable, well-structured Tamil, but with occasional wrong words and invented facts
On-device or offline Tamil NLP	Good	1B parameters, 2 GB in bf16; runs on a laptop
Facts, general knowledge, exam questions	Not suitable	MILU 28.0% (chance is 25%); Belebele 36.9%, above chance but low; invents facts
Long-form writing or multi-turn chat	Not suitable	Not evaluated; loses the thread in longer replies

How to use

The weights are standard Gemma-3 safetensors (bf16) and load in both transformers and MLX. The chat format is Gemma's; a system message is folded into the first user turn. Generation stops at <end_of_turn>.

transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "Felamity-Technologies/Minmini-1B"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16)

msgs = [{"role": "user", "content": "Translate to Tamil:\nThe weather is very hot today."}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
out = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
# இன்று வானிலை மிகவும் சூடாக உள்ளது.

For free-form replies longer than a sentence or two, sampling usually reads better than greedy decoding: do_sample=True, temperature=0.7, top_p=0.95, repetition_penalty=1.1.

MLX (Apple silicon)

from mlx_lm import load, generate

model, tok = load("Felamity-Technologies/Minmini-1B")
ids = tok.apply_chat_template([{"role": "user", "content": "Translate to Tamil:\nThe weather is very hot today."}],
                              add_generation_prompt=True)
print(generate(model, tok, prompt=ids, max_tokens=100))

Answering from a passage (best results)

Minmini was trained on this exact prompt. Send it as a single user message:

You are a helpful assistant that answers reading comprehension questions in Tamil.

Read the following passage and answer the question. Reply ONLY with a short phrase from the passage, in Tamil. No explanation.

Passage: {context}

Question: {question}
Answer:

An empty reply means "the answer is not in the passage". The reported scores use a fixed rule: take the probability of <end_of_turn> (token id 106) as the first generated token; if it is above 0.5, treat the question as unanswerable; otherwise generate with that token blocked for the first step.

Natural-language inference (best results)

You are a careful reasoner. Answer with the single letter A, B, or C.

பின்வரும் வாக்கியத்தைப் படிக்கவும்:

முன்னுரை: {premise}

கூற்று: {hypothesis}

முன்னுரையைப் பொறுத்து, கூற்று எந்த நிலையில் உள்ளது?
A) கண்டிப்பாக உண்மை (முன்னுரையால் உறுதிப்படுத்தப்படுகிறது)
B) கண்டிப்பாக தவறு (முன்னுரையுடன் முரண்படுகிறது)
C) முடிவு செய்ய முடியாது (முன்னுரையில் தெளிவான தகவல் இல்லை)

Reply with ONLY the single letter A, B, or C.

A = entailment, B = contradiction, C = neutral. Comparing the probabilities of the three letter tokens gives a confidence score.

Results

Tamil-Bench leaderboard items

The exact items Tamil-Bench's leaderboard uses: the first 200 IndicXNLI pairs and first 100 IndicQA questions of its fixed shuffle (seed 42). Same prompts and answer parsers for every model.

Model	IndicXNLI accuracy % (n=200)	IndicQA EM / F1 (n=100)
Minmini-1B	79.5	44.0 / 55.1
Gemini 3.8 Flash	75.8	20.0 / 42.4
Claude Sonnet 5.5	75.5	22.0 / 43.1
GLM 5.3 Flash	71.5	19.0 / 36.8
GPT-6 Luna	65.0	16.0 / 40.5
Gemma 4 26B A4B	64.8	20.2 / 39.3
Liquid LFM2.5 2.6B	64.6	18.0 / 35.3
DeepSeek V4.1 Flash	62.0	18.0 / 38.9
gpt-oss-20b	62.0	20.0 / 40.7
GPT-5.6 Luna	61.0	17.0 / 40.7
Sarvam-105B (measured by us)	45.5 strict / 60.0 lenient	20.0 / 38.8

Other models' scores are from the Tamil-Bench leaderboard (generated 2026-10-02). At n=200 the 95% interval on XNLI is about ±6 points, so the XNLI lead is within noise; the IndicQA lead is larger. Much of Minmini's IndicQA edge comes from correctly declining unanswerable questions, which chat models usually attempt.

Larger samples (n=1000)

Task	Minmini-1B	Sarvam-105B
IndicXNLI accuracy	75.0%	43.0% strict / 58.2% lenient
IndicQA EM / F1, all questions	43.9 / 52.1	29.1 / 44.5
IndicQA F1, answerable questions only	47.8	54.0
MILU accuracy (5-shot)	28.0%	—

Translation and general benchmarks

lm-eval 0.4.13, full test sets, translation and MILU 5-shot, others 0-shot, plain prompts (no chat template).

Task	Minmini-1B
FLORES en→ta / ta→en (chrF)	46.8 / 45.1
IN22-Gen en→ta / ta→en (chrF)	41.0 / 42.2
IN22-Conv en→ta / ta→en (chrF)	36.1 / 37.3
MILU / Belebele / MMLU (ta)	28.6 / 36.9 / 24.9
XCOPA / MultiBLiMP (ta)	55.4 / 95.0

Training

All training ran on one 24 GB Apple M5 Pro MacBook with MLX, using LoRA (rank 64 on all attention and MLP projections).

1. Base model: Google's instruction-tuned Gemma-3-1B-IT, which keeps Google's chat and reasoning post-training.
2. Five supervised fine-tuning stages (about 18 hours in all), each added to fix a weakness found by evaluation:
   • chat format, NLI, QA and general instructions (IndicXNLI, IndicSQuAD, Aya, IndicAlign);
   • QA rebuilt so the model learns real "no answer" cases;
   • NLI with shuffled answer options, twice, to remove a letter-position bias;
   • English↔Tamil translation (IndicAlign sentence pairs).
3. Weight blend: the released model is 0.3 × the stage-4 model (NLI and QA) + 0.7 × the stage-5 model (translation), averaged weight by weight.

All training text was n-gram decontaminated against MILU, IndicXNLI-test, IndicQA, FLORES and IN22. No test items were used.

Findings

• The base model matters more than Tamil pre-training for these tasks. An earlier version of Minmini added 77 hours of Tamil continued pre-training to the raw gemma-3-1b-pt model before the same fine-tuning. It modelled raw Tamil text better (1.24 vs 1.53 bits per character), but it lost the base model's reasoning (3–4 of 10 simple questions vs 8 of 10). Fine-tuning gemma-3-1b-it directly matched it on NLI and QA, beat it on translation and answerable-question QA, and kept the reasoning.
• Translated QA data carries a shortcut. After the first fine-tuning stage the model answered "no answer" to almost every IndicQA question. The machine-translated unanswerable questions in IndicSQuAD carry a surface cue that human-written questions lack. Building unanswerables from other paragraphs of the same article fixed it.
• Benchmark scores can hide position bias. Models trained only on the benchmark's fixed answer order learn "entailment means A". Training with shuffled options removed this.

Limitations

• Supervised, in-format: 50–100% of the NLI and 60–75% of the QA fine-tuning examples used Tamil-Bench's prompt formats. The comparison models were evaluated zero-shot. The results show a small specialist can match large general models on these two tasks, not that it is generally stronger.
• Little knowledge, limited reasoning: exam-knowledge benchmarks (MILU, MMLU) are near chance. Simple English arithmetic and logic work; Tamil arithmetic is unreliable. Open-ended factual answers are often wrong.
• Fluency: Tamil text is readable but less natural than a Tamil-pretrained model's, with occasional wrong word choices.
• Not evaluated: safety, toxicity, bias, long-form generation and multi-turn chat. There was no human evaluation of fluency.
• Tokenizer: Gemma's tokenizer uses about 2.4 tokens per Tamil word, so Tamil text is slower and costlier than with a Tamil-specific tokenizer.
• Do not use Minmini for medical, legal, financial or other high-stakes decisions.

Versions

• v2 (current): fine-tuned from google/gemma-3-1b-it, as described above.
• v1: Tamil continued pre-training on google/gemma-3-1b-pt, then the same fine-tuning and blend (IndicXNLI 76.0%, IndicQA F1 52.4, FLORES chrF 44.7 / 45.6). Available on the v1 branch of this repository.

Licence and attribution

Minmini-1B is a derivative of Gemma and is distributed under the Gemma Terms of Use. Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms. Use of this model must comply with the Gemma Prohibited Use Policy, and anyone you share it with is bound by the same terms.

Training data:

• IndicXNLI (CC0)
• IndicSQuAD (CC-BY-4.0)
• IndicAlign (AI4Bharat, CC-BY-4.0)
• Aya Dataset (Cohere For AI, Apache-2.0)

Evaluation uses Tamil-Bench, FLORES-200 and IN22.

Citation

@misc{minmini2026,
  title  = {Minmini-1B: a small Tamil language model trained on a laptop},
  author = {Nafize Ahamed F},
  year   = {2026},
  url    = {https://huggingface.co/Felamity-Technologies/Minmini-1B}
}