---
language: en
license: mit
tags:
- medical
- pharmaceutical
- autocomplete
- distillation
- gpt2
datasets:
- medmcqa
metrics:
- perplexity
model-index:
- name: codehance/distilgpt2-medical-pharma
  results:
  - task:
      type: text-generation
    dataset:
      name: Medical Q&A
      type: medmcqa
    metrics:
    - name: Perplexity
      type: perplexity
      value: 44.07
---

# DistilGPT-2 Medical Pharmaceutical Autocomplete

## Model Description

This is a distilled GPT-2 model fine-tuned for pharmaceutical autocomplete. It suggests drug names and medical terminology based on clinical context.

**Key Features:**
- 34% smaller than base fine-tuned model (81,912,576 parameters)
- 45% faster inference (347.9ms per generation)
- Specialized in pharmaceutical vocabulary

## Training Process

### Stage 1: Fine-Tuning
- Base model: GPT-2 (124M parameters)
- Dataset: Medical Q&A (medmcqa) - 4,500 training examples
- Training: 3 epochs
- Final perplexity: 23.61

### Stage 2: Knowledge Distillation
- Teacher: Fine-tuned GPT-2
- Student: DistilGPT-2
- Training: 2 epochs
- Compression: 34.2% size reduction

## Performance

| Metric | Value |
|--------|-------|
| Parameters | 81,912,576 |
| Perplexity | 44.07 |
| Inference Speed | 347.9ms |
| Quality Retained | 53.6% |

## Usage
```python
from transformers import GPT2LMHeadModel, GPT2Tokenizer

# Load model and tokenizer
model = GPT2LMHeadModel.from_pretrained("codehance/distilgpt2-medical-pharma")
tokenizer = GPT2Tokenizer.from_pretrained("codehance/distilgpt2-medical-pharma")

# Generate pharmaceutical suggestions
prompt = "The patient should take"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=30, num_return_sequences=3)

for output in outputs:
    print(tokenizer.decode(output, skip_special_tokens=True))
```

## Intended Use

**Primary Use Cases:**
- Pharmaceutical autocomplete systems
- Medical documentation assistance
- Clinical note-taking tools
- Drug name suggestion

**Limitations:**
- Not a substitute for medical advice
- May suggest incorrect drugs - always verify with qualified professionals
- Trained on medical exam questions, not real prescriptions
- English language only

## Training Data

- **Source:** MedMCQA dataset (Indian medical entrance exam questions)
- **Size:** 4,500 training examples
- **Content:** Medical questions with pharmaceutical terminology

## Ethical Considerations

⚠️ **Important:** This model is for autocomplete assistance only. It should NOT be used as the sole basis for medical decisions. Always verify suggestions with qualified healthcare professionals.

## Model Card Authors

Created as part of a pharmaceutical autocomplete system tutorial demonstrating transfer learning, fine-tuning, and knowledge distillation.

## Citation
```bibtex
@misc{distilgpt2-medical-pharma,
  author = {codehance},
  title = {DistilGPT-2 Medical Pharmaceutical Autocomplete},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/codehance/distilgpt2-medical-pharma}}
}
```