File size: 1,645 Bytes
9711fd0 0d2677e 0fb9a51 9711fd0 0fb9a51 9711fd0 0fb9a51 9711fd0 0fb9a51 9711fd0 0fb9a51 9711fd0 0fb9a51 9711fd0 0fb9a51 9711fd0 0fb9a51 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | ---
library_name: transformers
tags:
- medical
license: apache-2.0
language:
- fr
- en
base_model:
- ik-ram28/MedMistralInstruct-CPT-7B
- mistralai/Mistral-7B-Instruct-v0.1
---
## MedMistralInstruct-CPT-SFT-7B
### Model Description
MedMistralInstruct-CPT-SFT-7B is a French medical language model based on Mistral-7B-Instruct-v0.1, adapted through Continual Pre-Training followed by Supervised Fine-Tuning.
### Model Details
- **Model Type**: Causal Language Model
- **Base Model**: Mistral-7B-Instruct-v0.1
- **Language**: French
- **Domain**: Medical/Healthcare
- **Parameters**: 7 billion
- **License**: Apache 2.0
### Training Details
**Continual Pre-Training (CPT)**
- **Dataset**: NACHOS corpus (7.4 GB French medical texts)
- **Training Duration**: 2.8 epochs
- **Hardware**: 32 NVIDIA A100 80GB GPUs
- **Training Time**: ~40 hours
**Supervised Fine-Tuning (SFT)**
- **Dataset**: 30K French medical question-answer pairs
- **Method**: DoRA (Weight-Decomposed Low-Rank Adaptation)
- **Training Duration**: 10 epochs
- **Hardware**: 1 NVIDIA H100 80GB GPU
- **Training Time**: ~42 hours
### Computational Requirements
- **Carbon Emissions**: 33.96 kgCO2e (CPT+SFT)
- **Training Time**: 82 hours total (CPT+SFT)
### Ethical Considerations
- **Medical Accuracy**: For research and educational purposes only
- **Professional Oversight**: Requires verification by qualified medical professionals
- **Bias Awareness**: May contain biases from training data
- **Privacy**: Do not input private health information
### Citation
```bibtex
```
### Contact
For questions about these models, please contact: ikram.belmadani@lis-lab.fr |