hassan7272/tinyllama-lora-eval

TinyLlama LoRA Fine-Tuning (Evaluation Project)

This repository contains LoRA adapters fine-tuned on top of TinyLlama/TinyLlama-1.1B-Chat-v1.0 using Hugging Face PEFT.

🔧 Training Details

• Method: LoRA (Parameter-Efficient Fine-Tuning)
• Trainable parameters: <1% of base model
• Trainer: TRL SFTTrainer
• Hardware: Google Colab (single GPU)
• Epochs: 2

📊 Evaluation

The model was evaluated against the base model using identical prompts. Results showed:

• Noticeable changes in response style and length
• Improved instruction adherence in some cases
• Sensitivity to dataset quality and size

This project focuses on understanding LoRA behavior and evaluation rather than maximizing benchmark scores.

⚠️ Limitations

• Small base model (1.1B parameters)
• Limited fine-tuning dataset
• Some responses may be inaccurate or off-topic

🚀 Usage

Example usage (see Hugging Face page for full snippet).

📌 Disclaimer

This repository is for educational and evaluation purposes.