| # BioMistral Medical Chatbot - Hugging Face Space | |
| A medical chatbot that runs locally in Hugging Face Spaces, designed to answer medical questions using the BioMistral medical model. | |
| ## 🚀 Features | |
| - **BioMistral Medical Model**: Uses the specialized BioMistral-7b-Chat-Doctor model | |
| - **Doctor-Patient Conversations**: Designed for multi-turn medical dialogue | |
| - **Local Model Loading**: Loads the medical model directly in the Space | |
| - **Device Compatibility**: Handles GPU/CPU device switching automatically | |
| - **Error Recovery**: Graceful fallback to CPU if GPU issues occur | |
| - **Medical Focus**: Specialized for medical questions and responses | |
| ## 🔧 Recent Updates | |
| ### Model Switch to BioMistral-7b-Chat-Doctor | |
| **Previous Issue**: The original model had architecture mismatches and device placement problems. | |
| **New Solution**: Switched to [BioMistral-7b-Chat-Doctor](https://huggingface.co/siyah1/BioMistral-7b-Chat-Doctor) which offers: | |
| - **Proper medical training**: Fine-tuned on 257k medical conversations | |
| - **Stable architecture**: Based on BioMistral-7B foundation | |
| - **Multi-turn conversations**: Designed for ongoing doctor-patient dialogue | |
| - **Apache-2.0 license**: Open source and free to use | |
| ### Device Handling Improvements | |
| - Proper device detection and mapping | |
| - Consistent tensor placement on the same device | |
| - Automatic fallback to CPU if GPU issues occur | |
| - Better error handling during generation | |
| ## 📦 Dependencies | |
| - `gradio>=4.44.0` - Web interface | |
| - `torch>=2.1.0,<3.0.0` - PyTorch for model operations | |
| - `transformers>=4.35.0,<5.0.0` - Hugging Face transformers | |
| - `accelerate>=0.24.0` - Model acceleration | |
| - `spaces>=0.1.0` - HF Spaces support | |
| ## 🏃♂️ Usage | |
| 1. **Deploy to HF Space**: Upload the files to your Hugging Face Space | |
| 2. **Automatic Setup**: The Space will automatically install dependencies | |
| 3. **Model Loading**: The BioMistral medical model will be downloaded and loaded | |
| 4. **Start Chatting**: Use the web interface to ask medical questions | |
| ## 🧪 Testing | |
| Run the test script to verify device handling: | |
| ```bash | |
| python test_device_fix.py | |
| ``` | |
| ## 🔍 Troubleshooting | |
| ### Common Issues | |
| 1. **Device Mismatch Error**: | |
| - ✅ Fixed with improved tensor handling | |
| - Automatic fallback to CPU if GPU fails | |
| 2. **Memory Issues**: | |
| - Uses `low_cpu_mem_usage=True` | |
| - Float16 precision on GPU for memory efficiency | |
| 3. **Model Loading Failures**: | |
| - Multiple fallback strategies | |
| - Detailed error logging | |
| ### Logs | |
| Check the Space logs for detailed information about: | |
| - Device detection | |
| - Model loading progress | |
| - Generation errors | |
| - Memory usage | |
| ## 📝 Example Questions | |
| - "What are the symptoms of diabetes?" | |
| - "How is hypertension treated?" | |
| - "What are the risk factors for heart disease?" | |
| - "I have a headache, what should I do?" | |
| - "What are the common symptoms of COVID-19?" | |
| ## 🤝 Contributing | |
| Feel free to submit issues or improvements to the device handling logic. | |
| ## 📄 License | |
| This project is open source and available under the MIT License. | |
| ## 🙏 Acknowledgments | |
| - **BioMistral Team**: For the excellent medical foundation model | |
| - **Siyahul Haque T P**: For fine-tuning the chat-doctor model | |
| - **Hugging Face**: For providing the platform and infrastructure |