MedQA-SmolLM2-1.7B

A medical question answering model fine-tuned from SmolLM2-1.7B-Instruct using QLoRA on the ChatDoctor-HealthCareMagic-100k dataset.

Results

Metric	Base Model	Fine-Tuned	Improvement
ROUGE-L	0.122	0.157	+28.9%
Cases Improved	—	37/50	74% of test cases

Training

• Base model: HuggingFaceTB/SmolLM2-1.7B-Instruct
• Dataset: ChatDoctor-HealthCareMagic-100k (4,500 samples)
• Method: QLoRA (4-bit quantization, r=16)
• Hardware: Google Colab T4 GPU

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM2-1.7B-Instruct")
model = PeftModel.from_pretrained(base, "luminoria/MedQA-SmolLM2-1.7B")
tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-1.7B-Instruct")

prompt = "<|user|>\nWhat are the symptoms of diabetes?\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Links

• Training code: https://github.com/Lumiin0us/MedQA-FineTune