TinyLlama LoRA Fine-Tuning (Evaluation Project)
This repository contains LoRA adapters fine-tuned on top of
TinyLlama/TinyLlama-1.1B-Chat-v1.0 using Hugging Face PEFT.
🔧 Training Details
- Method: LoRA (Parameter-Efficient Fine-Tuning)
- Trainable parameters: <1% of base model
- Trainer: TRL SFTTrainer
- Hardware: Google Colab (single GPU)
- Epochs: 2
📊 Evaluation
The model was evaluated against the base model using identical prompts. Results showed:
- Noticeable changes in response style and length
- Improved instruction adherence in some cases
- Sensitivity to dataset quality and size
This project focuses on understanding LoRA behavior and evaluation rather than maximizing benchmark scores.
⚠️ Limitations
- Small base model (1.1B parameters)
- Limited fine-tuning dataset
- Some responses may be inaccurate or off-topic
🚀 Usage
Example usage (see Hugging Face page for full snippet).
📌 Disclaimer
This repository is for educational and evaluation purposes.
- Downloads last month
- 1
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for hassan7272/tinyllama-lora-eval
Base model
TinyLlama/TinyLlama-1.1B-Chat-v1.0
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("hassan7272/tinyllama-lora-eval", dtype="auto")