prasadvittaldev/gemma4-tamil-e4b-it

Gemma 4 E4B — Tamil Instruction-Tuned

A Tamil-fluent, Tanglish-tolerant instruction-tuned fine-tune of Gemma 4 E4B, trained via QLoRA (Unsloth) on a combined Tamil instruction dataset. Built as the LLM stage of a Tamil voice-assistant pipeline (paired with a Tamil STT front-end and an Orpheus-based Tamil TTS back-end).

How this was built

1. Data collection — four existing public Tamil instruction/conversation datasets were identified, each covering a different gap: a broad multi-task instruction set (Aya_Tamil), conversational/safety-alignment data (indic-align), a general Alpaca/Orca-style instruction set (tamil-alpaca-orca), and a small natural Tamil-English code-switching sample (Chat2Find).
2. Normalization — each source was converted into one unified {"messages": [...]} chat format compatible with Gemma's chat template, with light per-source cleanup (stratified sampling where a source was too large, Tamil-majority filtering for the code-switching source).
3. Deduplication and merge — all four sources were combined and deduplicated on normalized text (to catch near-identical rows that trace back to the same underlying original data across sources), producing one ~300K-row training set.
4. Fine-tuning — QLoRA (rank 16, alpha 32, targeting attention + MLP projections) via Unsloth, 2 epochs, on a single local RTX 5060 Ti (16GB).
5. Merge — the trained LoRA adapter was merged back into the base model to produce one standalone 16-bit checkpoint (this repository).
6. QA — manual sign-off against a small set of representative prompts (pure Tamil, Tanglish + numerals, a refusal check) before release.

Model details

• Base model: unsloth/gemma-4-E4B-it-unsloth-bnb-4bit (Gemma 4 E4B, instruction-tuned, 4-bit)
• Method: QLoRA, r=16, alpha=32, targeting attention + MLP projections, merged into the base at 16-bit precision after training
• Training data: prasadvittaldev/gemma4-tamil-sft — combined and deduplicated from 4 sources:
  • Cognitive-Lab/Aya_Tamil (stratified sample, ~150K rows)
  • ai4bharat/indic-align (Wiki_Chat, HHRLHF_T, Toxic_Matrix configs, Tamil-extracted)
  • abhinand/tamil-alpaca-orca (full, ~145K rows)
  • Chat2Find/Chat2Find-Instruct-Reasoning-Sample (Tamil-majority filtered, Tanglish/code-switching)
• Epochs: 2 (final training loss: 0.031)
• Hardware: local RTX 5060 Ti (16GB)

License

Released under CC-BY-NC-4.0 (non-commercial). This is not the same as Gemma 4's own Apache 2.0 license — the more restrictive non-commercial terms are used here because part of the training mix (abhinand/tamil-alpaca-orca) carries a non-commercial license (its Alpaca-derived half). This model may be used for research, evaluation, and personal projects, but not commercial deployment or resale. If you need a fully commercial-use model, retrain on the same pipeline with that source excluded.

Intended use

Tamil-language instruction following and conversation, with tolerance for natural Tamil-English code-switching ("Tanglish") common in everyday Tamil speech/chat. Designed as the reasoning/response-generation stage of a real-time Tamil voice assistant.

Limitations

• Evaluated only via a small manual QA set (3 representative prompts: pure Tamil, Tanglish + numerals, a refusal check), not comprehensive benchmarking.
• Refusal/safety-decline responses were observed to be functionally correct (no harmful content generated) but occasionally grammatically rougher than the model's general fluency on non-refusal prompts.
• Not evaluated on code, math, or long-document reasoning — the training data is oriented toward conversational/instructional Tamil.

How to use

Step 1 — install dependencies:

pip install transformers torch accelerate bitsandbytes

Step 2 — load the tokenizer and model. The full model is ~16GB in bf16; on a 16GB GPU, load it 4-bit quantized to leave headroom for generation. On a larger GPU (24GB+), you can load it directly in bf16 instead (drop the quantization_config argument and use dtype=torch.bfloat16).

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

MODEL = "prasadvittaldev/gemma4-tamil-e4b-it"

tokenizer = AutoTokenizer.from_pretrained(MODEL)
quant_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)
model = AutoModelForCausalLM.from_pretrained(MODEL, quantization_config=quant_config, device_map="cuda")

Step 3 — format your prompt as a chat message (a list of {"role": ..., "content": ...} dicts):

messages = [{"role": "user", "content": "தமிழ் மொழியின் வரலாறு பற்றி சுருக்கமாக சொல்லுங்கள்."}]

Step 4 — apply the chat template and move inputs to the GPU:

inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to("cuda")

Step 5 — generate a response:

output = model.generate(**inputs, max_new_tokens=200)

Step 6 — decode only the newly generated tokens (skip the prompt you passed in):

response = tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(response)

Contact

For questions, feedback, or further discussion, reach out at prasadvittaldev@gmail.com.