File size: 3,620 Bytes
b8d45d8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 |
# 🩺 MedicalChatBot
**MedicalChatBot** is a medical domain-focused chatbot fine-tuned using **LoRA (Low-Rank Adaptation)** on top of [`mistralai/Mistral-7B-Instruct`](https://huggingface.co/mistralai/Mistral-7B-Instruct).
It is designed for health education, medical Q&A, and research use only.
---
## 📌 Overview
- 🧠 Based on Mistral-7B-Instruct, a powerful instruction-following LLM
- 🔧 Fine-tuned using [PEFT](https://github.com/huggingface/peft) + LoRA on a medical dataset
- 📚 Trained on: [`kberta2014/medical-chat-dataset`](https://huggingface.co/datasets/kberta2014/medical-chat-dataset)
- ⚡ Efficient: Only trains adapter layers instead of the full model
- 📦 Deployment-ready: Compatible with Hugging Face `transformers`, `Gradio`, and Spaces
---
## 🧠 Prompt Format
Use the model in the following format:
```
### Instruction:
<Your question>
### Input:
<Optional additional context>
### Response:
```
Example:
```
### Instruction:
What are the symptoms of high blood pressure?
### Input:
### Response:
```
---
## 💬 Example Usage
```python
from transformers import pipeline
pipe = pipeline("text-generation", model="kberta2014/MedicalChatBot", tokenizer="kberta2014/MedicalChatBot")
prompt = '''### Instruction:
What are common symptoms of diabetes?
### Input:
### Response:
'''
output = pipe(prompt, max_new_tokens=200, temperature=0.7)
print(output[0]["generated_text"])
```
---
## 🤖 Gradio Chatbot Interface
```python
import gradio as gr
from transformers import pipeline
pipe = pipeline("text-generation", model="kberta2014/MedicalChatBot", tokenizer="kberta2014/MedicalChatBot")
def chat(instruction, input_text=""):
prompt = f"### Instruction:\n{instruction}\n\n### Input:\n{input_text}\n\n### Response:\n"
return pipe(prompt, max_new_tokens=200, temperature=0.7)[0]["generated_text"]
gr.Interface(fn=chat,
inputs=["text", "text"],
outputs="text",
title="🩺 MedicalChatBot",
description="Ask medical questions and get responses from a fine-tuned LLM"
).launch()
```
---
## 🏋️ Training Configuration
- **Model**: `mistralai/Mistral-7B-Instruct`
- **Dataset**: [`kberta2014/medical-chat-dataset`](https://huggingface.co/datasets/kberta2014/medical-chat-dataset)
- **Framework**: Hugging Face `transformers`, `peft`, `datasets`
- **PEFT Config**:
- `r=8`, `lora_alpha=16`, `target_modules=["q_proj", "v_proj"]`
- `lora_dropout=0.05`, `bias="none"`, `task_type="CAUSAL_LM"`
- **Training Time**: ~3 epochs on Colab T4
- **Batch Size**: 2
- **Learning Rate**: 2e-4
- **Precision**: bf16 / float16
---
## 📊 Training Metrics (Sample)
| Metric | Value |
|-------------------|-------------|
| Training loss | ~1.02 |
| Eval loss | ~0.94 |
| Perplexity | ~2.6 |
| Epochs | 3 |
| Trainable params | ~7M (LoRA) |
---
## 🧾 Citation
If you use this model in your research or application, please cite:
```bibtex
@misc{medicalchatbot2025,
title={MedicalChatBot: A LoRA Fine-Tuned Mistral-7B Model for Medical QA},
author={kberta2014},
year={2025},
url={https://huggingface.co/kberta2014/MedicalChatBot},
note={Hugging Face model repository}
}
```
---
## ⚠️ Disclaimer
This model is intended for **research and educational purposes only**.
It is **not a replacement for professional medical advice or diagnosis**.
Always consult a licensed healthcare provider for real medical concerns.
---
## 📄 License
Apache 2.0 — same as the base model `mistralai/Mistral-7B-Instruct`.
|