unsloth/mistral-7b-v0.3-bnb-4bit Fine-tuned with QLoRA (Unsloth) on Alpaca
This model is a fine-tuned version of unsloth/mistral-7b-v0.3-bnb-4bit using QLoRA and Unsloth for efficient instruction-tuning.
📖 Training Details
- Dataset:
yahma/alpaca-cleaned - QLoRA: 4 bit quantization (NF4) using
bitsandbytes - LoRA Rank: 16 (adjust based on your config)
- LoRA Alpha: 16
- Batch Size: 2 per device
- Gradient Accumulation: 4
- Learning Rate: 2e-4
- Epochs: 1
- Trainer:
trl.SFTTrainer
💡 Notes
- Optimized for memory-efficient fine-tuning with Unsloth
- No evaluation was run during training — please evaluate separately
📝 License
Apache 2.0
- Downloads last month
- 6