---
license: apache-2.0
base_model: google/gemma-2b
tags:
- gemma
- lora
- qlora
- instruction-tuning
- unsloth
- transformers
- text-generation
library_name: transformers
---
- **Developed by:** Tushar Kamthe
- **License:** apache-2.0
- **Finetuned from model :** unsloth/gemma-2-2b-it-bnb-4bit
- 
# Gemma-2B QLoRA Fine-tuned Model

## Model Description

This model is a fine-tuned version of **Google's Gemma-2B** using the **QLoRA (Quantized Low-Rank Adaptation)** technique. The model was trained using **parameter-efficient fine-tuning (PEFT)**, where only LoRA adapters were trained while keeping the base model weights frozen.

The model is designed for **instruction-following text generation tasks**.

Fine-tuning was performed using:

- QLoRA (4-bit quantization)
- LoRA adapters
- HuggingFace Transformers
- PEFT
- Unsloth for faster training

---

## Base Model

Base model used for training:

google/gemma-2b

---

## Training Details

### Training Method

The model was trained using **QLoRA**, which enables efficient training of large language models by:

- Loading the base model in **4-bit quantized format**
- Training **LoRA adapter weights only**
- Keeping base model weights frozen

This significantly reduces GPU memory requirements.

---

### Training Configuration

| Parameter | Value |
|--------|--------|
| Method | QLoRA |
| Quantization | 4-bit (NF4) |
| LoRA Rank (r) | 16 |
| LoRA Alpha | 64 |
| LoRA Dropout | 0.05 |
| Optimizer | AdamW |
| Precision | bfloat16 |
| Framework | HuggingFace Transformers |

---

### Hardware

Training was performed on:

- GPU: NVIDIA GPU (Colab / Local GPU)
- Framework: PyTorch
- Libraries:
  - transformers
  - peft
  - datasets
  - unsloth

---

## Dataset

The model was fine-tuned on a **custom instruction dataset** containing prompt-response pairs.

Dataset format: