CHF0101/medquad-lora-r32

MedQuAD LoRA r=32

Configuración

• Base: mistralai/Mistral-7B-Instruct-v0.3
• LoRA r: 32
• Módulos: q_proj, k_proj, v_proj
• 4-bit NF4
• Early Stopping: patience=3

Entrenamiento

Training logs (manual, Epoch estimado):

Step	Epoch	Training Loss	Validation Loss
100	0.046	0.824200	0.792224
200	0.093	0.766400	0.763532
300	0.139	0.761900	0.749713
400	0.186	0.734200	0.737889
500	0.232	0.728500	0.727421
600	0.279	0.734900	0.719858
700	0.325	0.715400	0.706352
800	0.371	0.722500	0.695117
900	0.418	0.702000	0.687702
1000	0.464	0.686500	0.677177
1100	0.511	0.674400	0.673650
1200	0.557	0.646700	0.663053
1300	0.604	0.651100	0.662174

Uso

from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained('mistralai/Mistral-7B-Instruct-v0.3', load_in_4bit=True)
model = PeftModel.from_pretrained(base, 'CHF0101/medquad-lora-r32')