---
license: apache-2.0
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
tags:
- lora
- peft
- fine-tuning
- evaluation
- transformers
---

# TinyLlama LoRA Fine-Tuning (Evaluation Project)

This repository contains **LoRA adapters** fine-tuned on top of
`TinyLlama/TinyLlama-1.1B-Chat-v1.0` using Hugging Face PEFT.

## 🔧 Training Details
- **Method**: LoRA (Parameter-Efficient Fine-Tuning)
- **Trainable parameters**: <1% of base model
- **Trainer**: TRL SFTTrainer
- **Hardware**: Google Colab (single GPU)
- **Epochs**: 2

## 📊 Evaluation
The model was evaluated against the base model using identical prompts.
Results showed:
- Noticeable changes in response style and length
- Improved instruction adherence in some cases
- Sensitivity to dataset quality and size

This project focuses on **understanding LoRA behavior and evaluation**
rather than maximizing benchmark scores.

## ⚠️ Limitations
- Small base model (1.1B parameters)
- Limited fine-tuning dataset
- Some responses may be inaccurate or off-topic

## 🚀 Usage

Example usage (see Hugging Face page for full snippet).

## 📌 Disclaimer
This repository is for **educational and evaluation purposes**.