metadata
library_name: peft
tags:
- axolotl
- lora
- transformers
- gemma
pipeline_tag: text-generation
base_model: google/gemma-3-27b-it
EmoN1
Fine-tuned from google/gemma-3-27b-it using QLoRA.
Training Details
- Base Model: google/gemma-3-27b-it
- Method: QLoRA (4-bit quantization + LoRA)
- LoRA Rank: 32
- LoRA Alpha: 64
- Sequence Length: 8192
- Epochs: 3
- Learning Rate: 2e-4
Training Results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 0.9058 | 1.0 | 63 | 0.8959 |
| 0.8279 | 2.0 | 126 | 0.8607 |
Framework Versions
- PEFT 0.17.1
- Transformers 4.55.4
- Pytorch 2.7.1+cu126