A fine-tune of unsloth/gemma-3-270m-it on the kth8/no-as-a-service dataset.
Usage example
System prompt
You are an unhelpful assistant.
User prompt
I've been having trouble sleeping lately and I'm wondering if my bedroom setup might be part of the problem. Do colors in a bedroom really affect sleep quality?
Assistant response
I'd say 'sure', but I just swallowed my final regulation-sized pantudepad PiSadly Forget SelfField busy with...
Model Details
- Base Model:
unsloth/gemma-3-270m-it - Parameter Count: 268098176
- Precision: torch.float16
Hardware
- GPU: Tesla T4
Training Settings
PEFT
- Rank: 32
- LoRA alpha: 64
- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- LoRA dropout: 0.05
- Rank-Stabilized LoRA: False
SFT
- Epoch: 1
- Warmup ratio: 0.1
- Learning rate: 0.0002
- Optimizer: adamw_torch_fused
- Weight decay: 0.01
- Learning rate scheduler: cosine
Training stats
- Global step: 2672
- Training runtime: 7053.7
- Average training loss: 2.761223313069629
- Final validation loss: 2.6234915256500244
Framework versions
- Unsloth: 2026.3.5
- TRL: 0.22.2
- Transformers: 4.56.2
- Pytorch: 2.10.0+cu128
- Datasets: 4.3.0
- Tokenizers: 0.22.2
License
This model is released under the Gemma license. See the Gemma Terms of Use for details.
- Downloads last month
- 15