README.md

BioMistral Medical Chatbot - Hugging Face Space

A medical chatbot that runs locally in Hugging Face Spaces, designed to answer medical questions using the BioMistral medical model.

🚀 Features

• BioMistral Medical Model: Uses the specialized BioMistral-7b-Chat-Doctor model
• Doctor-Patient Conversations: Designed for multi-turn medical dialogue
• Local Model Loading: Loads the medical model directly in the Space
• Device Compatibility: Handles GPU/CPU device switching automatically
• Error Recovery: Graceful fallback to CPU if GPU issues occur
• Medical Focus: Specialized for medical questions and responses

🔧 Recent Updates

Model Switch to BioMistral-7b-Chat-Doctor

Previous Issue: The original model had architecture mismatches and device placement problems.

New Solution: Switched to BioMistral-7b-Chat-Doctor which offers:

• Proper medical training: Fine-tuned on 257k medical conversations
• Stable architecture: Based on BioMistral-7B foundation
• Multi-turn conversations: Designed for ongoing doctor-patient dialogue
• Apache-2.0 license: Open source and free to use

Device Handling Improvements

• Proper device detection and mapping
• Consistent tensor placement on the same device
• Automatic fallback to CPU if GPU issues occur
• Better error handling during generation

📦 Dependencies

• gradio>=4.44.0 - Web interface
• torch>=2.1.0,<3.0.0 - PyTorch for model operations
• transformers>=4.35.0,<5.0.0 - Hugging Face transformers
• accelerate>=0.24.0 - Model acceleration
• spaces>=0.1.0 - HF Spaces support

🏃‍♂️ Usage

1. Deploy to HF Space: Upload the files to your Hugging Face Space
2. Automatic Setup: The Space will automatically install dependencies
3. Model Loading: The BioMistral medical model will be downloaded and loaded
4. Start Chatting: Use the web interface to ask medical questions

🧪 Testing

Run the test script to verify device handling:

python test_device_fix.py

🔍 Troubleshooting

Common Issues

1. Device Mismatch Error:

   • ✅ Fixed with improved tensor handling
   • Automatic fallback to CPU if GPU fails

2. Memory Issues:

   • Uses low_cpu_mem_usage=True
   • Float16 precision on GPU for memory efficiency

3. Model Loading Failures:

   • Multiple fallback strategies
   • Detailed error logging

Logs

Check the Space logs for detailed information about:

• Device detection
• Model loading progress
• Generation errors
• Memory usage

📝 Example Questions

• "What are the symptoms of diabetes?"
• "How is hypertension treated?"
• "What are the risk factors for heart disease?"
• "I have a headache, what should I do?"
• "What are the common symptoms of COVID-19?"

🤝 Contributing

Feel free to submit issues or improvements to the device handling logic.

📄 License

This project is open source and available under the MIT License.

🙏 Acknowledgments

• BioMistral Team: For the excellent medical foundation model
• Siyahul Haque T P: For fine-tuning the chat-doctor model
• Hugging Face: For providing the platform and infrastructure