Mistral 7B MedQuAD (LoRA r=4)
Training configuration
- Epochs: 1
- Batch size: 1
- Gradient accumulation: 4
- Learning rate: 2e-4
- Optimizer: paged_adamw_8bit
- Quantization: 4-bit (bitsandbytes)
Training Summary
| Step | Training Loss | Validation Loss | Mean Token Accuracy |
|---|---|---|---|
| 20 | 1.1792 | 1.1783 | 0.7006 |
| 100 | 0.7425 | 0.9143 | 0.7697 |
| 200 | 0.9692 | 0.8869 | 0.7738 |
| 300 | 0.9178 | 0.8768 | 0.7758 |
| 400 | 1.0642 | 0.8701 | 0.7773 |
| 500 | 0.9679 | 0.8673 | 0.7783 |
Final metrics
- Final Training Loss: 0.93
- Final Validation Loss: 0.87
- Final Mean Token Accuracy: 0.78
- Epochs: 1
- Total Steps: 500
Model fine-tuned on the MedQuAD dataset for medical QA using PEFT + QLoRA.
- Downloads last month
- 3
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for vlachner/mistral-medquad-r4
Base model
mistralai/Mistral-7B-v0.3 Finetuned
mistralai/Mistral-7B-Instruct-v0.3