TatarNLPWorld/gpt2-medium-tatar-lora-r16

GPT-2 Medium LoRA Adapter for Tatar Language

This is a LoRA adapter for GPT-2 medium fine-tuned on Tatar language.

📊 Model Details

Property	Value
Base Model	GPT-2 medium
LoRA Rank	16
Training Data	10,000 samples
Test Perplexity	5.31
Training Epochs	3

🚀 Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = "gpt2-medium"
tokenizer = AutoTokenizer.from_pretrained(base_model)
tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(base_model)
model = PeftModel.from_pretrained(model, "TatarNLPWorld/gpt2-medium-tatar-lora-r16")

prompt = "Минем исемем"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0]))

💡 Why GPT-2 medium?

• Lightweight (can run on CPU)
• Good baseline for comparison with 7B models
• Perplexity 5.31 — solid performance for its size

📈 Generation Example

Prompt: "Минем исемем"
Generated: "... Айрат. Мин Казанда яшим."

📊 Performance

Model	Parameters	Perplexity
GPT-2 medium (full)	354M	2.91
GPT-2 medium + LoRA r16	1.09M	5.31
Parameter reduction	325x	+82%

👥 Authors

• Arabov Mullosharaf Kurbonovich

📜 License

MIT (same as original GPT-2)

🤝 Citation

@software{gpt2_medium_tatar_lora_2026,
    title = {{GPT-2 Medium LoRA Adapter for Tatar Language}},
    author = {Arabov Mullosharaf Kurbonovich},
    year = {2026},
    publisher = {Hugging Face},
    url = {https://huggingface.co/TatarNLPWorld/gpt2-medium-tatar-lora-r16}
}