MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
Paper
•
2311.16079
•
Published
•
19
YAML Metadata
Warning:
The pipeline tag "text2text-generation" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other
Model Card for MediNote-13B-v1.0
MediNote is a suite of open-source medical Large Language Models (LLMs) fine-tuned for clinical note generation from the MediTron foundation model.
MediNote-13B is a 13 billion parameters model trained to generate clinical notes from doctor-patient conversations.
Model Details
- Developed by: Antoine Bonnet and Paul Boulenger
- Model type: Causal decoder-only transformer language model
- Language(s): English only
- Model License: LLAMA 2 COMMUNITY LICENSE AGREEMENT
- Code License: MIT
- Fine-tuned from model: Llama-2-13B with continued pre-training on PubMed Central (MediTron-13B equivalent)
- Context length: 2K tokens
- Input: Text-only data
- Output: Model generates text only
Model Sources
- Repository: EPFL-IC-Make-Team/ClinicalNotes
- Trainer: epflLLM/Megatron-LLM
- Paper: MediNote: Automatic Clinical Notes
Uses
Direct Use
It is possible to use this model to generate clinical notes, which is useful for experimentation and understanding its capabilities. It should not be used directly for production or work that may impact people.
Downstream Use
Out-of-Scope Use
We do not recommend using this model for natural language generation in a production environment, finetuned or otherwise.
Recommendations
Citation
BibTeX: If you use MediNote or its training data, please cite our work:
ADD CITATION
- Downloads last month
- 2