metadata
license: apache-2.0
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
tags:
- lora
- peft
- fine-tuning
- evaluation
- transformers
TinyLlama LoRA Fine-Tuning (Evaluation Project)
This repository contains LoRA adapters fine-tuned on top of
TinyLlama/TinyLlama-1.1B-Chat-v1.0 using Hugging Face PEFT.
🔧 Training Details
- Method: LoRA (Parameter-Efficient Fine-Tuning)
- Trainable parameters: <1% of base model
- Trainer: TRL SFTTrainer
- Hardware: Google Colab (single GPU)
- Epochs: 2
📊 Evaluation
The model was evaluated against the base model using identical prompts. Results showed:
- Noticeable changes in response style and length
- Improved instruction adherence in some cases
- Sensitivity to dataset quality and size
This project focuses on understanding LoRA behavior and evaluation rather than maximizing benchmark scores.
⚠️ Limitations
- Small base model (1.1B parameters)
- Limited fine-tuning dataset
- Some responses may be inaccurate or off-topic
🚀 Usage
Example usage (see Hugging Face page for full snippet).
📌 Disclaimer
This repository is for educational and evaluation purposes.