siva-gunasehkaran/cricket-commentary-tinyllama-lora

Cricket Commentary Generator (LoRA Fine-tuned)

Model Description

This is a LoRA fine-tuned version of TinyLlama-1.1B-Chat specifically trained to generate exciting cricket commentary.

Developed by: Dan Shiv (siva-gunasehkaran)
Model type: Causal Language Model with LoRA adapters
Language: English
License: Apache 2.0
Fine-tuned from: TinyLlama/TinyLlama-1.1B-Chat-v1.0

What This Model Does

Given a cricket match situation (batsman, bowler, action, result), the model generates authentic, exciting cricket commentary in the style of professional TV commentators.

Example

Input:

Batsman: Kohli, Bowler: Starc, Action: cover drive, Result: four

Output:

FOUR! Magnificent from Kohli! Starc pitched it up and Kohli has caressed that through the covers with impeccable timing. The ball raced away to the boundary before cover could even move. That's the class of Virat Kohli right there!

Training Details

Training Data

• 24 hand-crafted cricket commentary examples
• Covering: boundaries (4s and 6s), wickets (bowled, caught, LBW), run outs, dropped catches, defensive play, and match-winning moments

Training Procedure

• Method: LoRA (Low-Rank Adaptation)
• LoRA Rank (r): 8
• LoRA Alpha: 16
• Target Modules: q_proj, k_proj, v_proj, o_proj
• Epochs: 3
• Learning Rate: 2e-4
• Batch Size: 2 (with gradient accumulation of 4)

Training Infrastructure

• Hardware: Google Colab (Tesla T4 GPU)
• Training Time: ~6 seconds
• Adapter Size: 8.8 MB

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "siva-gunasehkaran/cricket-commentary-tinyllama-lora")

# Generate commentary
prompt = """<|system|>
You are an exciting cricket commentator.</s>
<|user|>
Generate exciting cricket commentary for this moment

Batsman: Warner, Bowler: Bumrah, Action: pull shot, Result: six</s>
<|assistant|>
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

• Trained on only 24 examples (proof of concept)
• May generate repetitive or inconsistent commentary
• Best suited for common cricket scenarios (batting shots, wickets)
• Based on TinyLlama 1.1B (smaller model, limited reasoning)

Future Improvements

• Train on larger dataset (500+ examples)
• Fine-tune larger base model (Mistral 7B, Llama 3.1 8B)
• Add AFL/football commentary
• Implement evaluation metrics (BLEU, human evaluation)

About the Developer

This model was created by Dan Shiv as part of learning AI/ML fine-tuning techniques.

• Portfolio: Contextory.app
• GitHub: github.com/siva-gunasehkaran
• LinkedIn: Connect with me

Citation

@misc{cricket-commentary-lora,
  author = {Dan Shiv},
  title = {Cricket Commentary Generator - LoRA Fine-tuned TinyLlama},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/siva-gunasehkaran/cricket-commentary-tinyllama-lora}
}