---
language: en
license: mit
tags:
  - medical
  - classification
  - distilbert
  - pubmedqa
  - healthcare-rag
datasets:
  - llamafactory/PubMedQA
metrics:
  - f1
pipeline_tag: text-classification
---

# DistilBERT Medical Query Classifier

Fine-tuned distilbert-base-uncased for classifying medical questions into 6 categories.

## Categories
| ID | Category |
|----|----------|
| 0 | Diagnosis |
| 1 | General |
| 2 | Medication |
| 3 | Prevention |
| 4 | Symptoms |
| 5 | Treatment |

## Results
| Metric | Score |
|--------|-------|
| Macro F1 | 0.8670 |
| Weighted F1 | 0.8787 |
| Accuracy | 0.8790 |

## Training Config
| Item | Value |
|------|-------|
| Base model | distilbert-base-uncased |
| Dataset | llamafactory/PubMedQA (10,000 rows) |
| Split | 80/10/10 |
| Epochs | 3 |
| Learning rate | 2e-5 |
| Batch size | 16 |
| Class weights | Balanced (custom WeightedTrainer) |

## Usage
from transformers import DistilBertTokenizer, DistilBertForSequenceClassification
import torch

tokenizer = DistilBertTokenizer.from_pretrained("AbdoMatrix/distilbert-medical-classifier")
model = DistilBertForSequenceClassification.from_pretrained("AbdoMatrix/distilbert-medical-classifier")

text = "What are the symptoms of diabetes?"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)

with torch.no_grad():
    outputs = model(**inputs)

predicted = model.config.id2label[torch.argmax(outputs.logits, dim=1).item()]
print(predicted)  # → Symptoms
## Project
Healthcare RAG-Powered Medical Q&A Assistant
eyouth x DEPI | Microsoft Machine Learning Track | 2026
GitHub: https://github.com/AbdooMatrix/Healthcare-RAG-Powered-Medical-QA-Assistant