---
library_name: peft
tags:
- axolotl
- lora
- transformers
- gemma
pipeline_tag: text-generation
base_model: google/gemma-3-27b-it
---

# EmoN1

Fine-tuned from [google/gemma-3-27b-it](https://huggingface.co/google/gemma-3-27b-it) using QLoRA.

## Training Details

- **Base Model:** google/gemma-3-27b-it
- **Method:** QLoRA (4-bit quantization + LoRA)
- **LoRA Rank:** 32
- **LoRA Alpha:** 64
- **Sequence Length:** 8192
- **Epochs:** 3
- **Learning Rate:** 2e-4

### Training Results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 0.9058        | 1.0   | 63   | 0.8959          |
| 0.8279        | 2.0   | 126  | 0.8607          |

### Framework Versions

- PEFT 0.17.1
- Transformers 4.55.4
- Pytorch 2.7.1+cu126