| library_name: peft | |
| tags: | |
| - axolotl | |
| - lora | |
| - transformers | |
| - gemma | |
| pipeline_tag: text-generation | |
| base_model: google/gemma-3-27b-it | |
| # EmoN1 | |
| Fine-tuned from [google/gemma-3-27b-it](https://huggingface.co/google/gemma-3-27b-it) using QLoRA. | |
| ## Training Details | |
| - **Base Model:** google/gemma-3-27b-it | |
| - **Method:** QLoRA (4-bit quantization + LoRA) | |
| - **LoRA Rank:** 32 | |
| - **LoRA Alpha:** 64 | |
| - **Sequence Length:** 8192 | |
| - **Epochs:** 3 | |
| - **Learning Rate:** 2e-4 | |
| ### Training Results | |
| | Training Loss | Epoch | Step | Validation Loss | | |
| |:-------------:|:-----:|:----:|:---------------:| | |
| | 0.9058 | 1.0 | 63 | 0.8959 | | |
| | 0.8279 | 2.0 | 126 | 0.8607 | | |
| ### Framework Versions | |
| - PEFT 0.17.1 | |
| - Transformers 4.55.4 | |
| - Pytorch 2.7.1+cu126 | |