| license: apache-2.0 | |
| base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0 | |
| tags: | |
| - lora | |
| - peft | |
| - fine-tuning | |
| - evaluation | |
| - transformers | |
| # TinyLlama LoRA Fine-Tuning (Evaluation Project) | |
| This repository contains **LoRA adapters** fine-tuned on top of | |
| `TinyLlama/TinyLlama-1.1B-Chat-v1.0` using Hugging Face PEFT. | |
| ## 🔧 Training Details | |
| - **Method**: LoRA (Parameter-Efficient Fine-Tuning) | |
| - **Trainable parameters**: <1% of base model | |
| - **Trainer**: TRL SFTTrainer | |
| - **Hardware**: Google Colab (single GPU) | |
| - **Epochs**: 2 | |
| ## 📊 Evaluation | |
| The model was evaluated against the base model using identical prompts. | |
| Results showed: | |
| - Noticeable changes in response style and length | |
| - Improved instruction adherence in some cases | |
| - Sensitivity to dataset quality and size | |
| This project focuses on **understanding LoRA behavior and evaluation** | |
| rather than maximizing benchmark scores. | |
| ## ⚠️ Limitations | |
| - Small base model (1.1B parameters) | |
| - Limited fine-tuning dataset | |
| - Some responses may be inaccurate or off-topic | |
| ## 🚀 Usage | |
| Example usage (see Hugging Face page for full snippet). | |
| ## 📌 Disclaimer | |
| This repository is for **educational and evaluation purposes**. | |