A fine-tune of unsloth/gemma-3-1b-it on the kth8/multi-turn-conversation-50000x dataset.
Usage example
System prompt
You are a helpful assistant.
User prompt
Hey there! How's it going?
Assistant response
Hey! I'm doing great, thanks for asking! I'm here and ready to help with whatever you need. What's on your mind today?
Model Details
- Base Model:
unsloth/gemma-3-1b-it - Parameter Count: 999885952
- Precision: torch.bfloat16
Training Settings
Hardware
- GPU: NVIDIA RTX PRO 6000 Blackwell Server Edition
PEFT
- Rank: 32
- LoRA alpha: 64
- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Gradient checkpointing: unsloth
SFT
- Epoch: 2
- Batch size: 48
- Gradient Accumulation steps: 1
- Warmup ratio: 0.1
- Learning rate: 0.0002
- Optimizer: adamw_torch_fused
- Learning rate scheduler: cosine
Training stats
- Global step: 1996
- Training runtime (seconds): 6834.1445
- Average training loss: 1.1743444665400442
- Final validation loss: 1.1191450357437134
Framework versions
- Unsloth: 2026.3.8
- TRL: 0.22.2
- Transformers: 4.56.2
- Pytorch: 2.10.0+cu128
- Datasets: 4.8.3
- Tokenizers: 0.22.2
License
This model is released under the Gemma license. See the Gemma Terms of Use and Prohibited Use Policy regarding the use of Gemma-generated content.
- Downloads last month
- 9