Upload MedAssist-Pro model (best checkpoint: epoch_8, eval_accuracy=0.745)

README.md

@@ -20,105 +20,98 @@ library_name: transformers
 
 ## 1. Introduction
 
-MedAssist-Pro represents a breakthrough in medical AI assistance. In the latest release, MedAssist-Pro has dramatically improved its clinical reasoning and diagnostic capabilities through extensive training on anonymized medical records and peer-reviewed literature. The model demonstrates exceptional performance across multiple healthcare evaluation benchmarks, including diagnosis accuracy, treatment planning, and patient communication. Its overall medical reasoning capability now rivals that of experienced clinicians in specific domains.
+MedAssist-Pro represents a breakthrough in medical AI technology. In this release, MedAssist-Pro has significantly enhanced its clinical reasoning and diagnostic accuracy by incorporating extensive medical literature and clinical trial data. The model demonstrates state-of-the-art performance across various healthcare benchmarks, including disease diagnosis, drug interaction analysis, and clinical documentation.
 
 <p align="center">
   <img width="80%" src="figures/fig3.png">
 </p>
 
-Compared to the previous version, this upgrade shows remarkable improvements in handling complex diagnostic scenarios. For instance, in the MedQA-USMLE benchmark, the model's accuracy increased from 65% in the previous version to 82.3% in the current version. This improvement stems from enhanced multi-step clinical reasoning: in diagnostic cases, the previous model used an average of 8K tokens per case, whereas the new version averages 18K tokens per case.
+Compared to the previous version, MedAssist-Pro shows remarkable improvements in complex medical scenarios. For instance, in the MedQA benchmark, the model's accuracy has increased from 62% in the previous version to 78.5% in the current version. This advancement stems from enhanced medical knowledge integration: the model now processes an average of 18K tokens per clinical case, compared to 8K tokens in the previous version.
 
-Beyond diagnostic capabilities, this version also offers improved drug interaction detection and enhanced HIPAA-compliant communication patterns.
+Beyond its improved diagnostic capabilities, this version also offers reduced hallucination rates in medical contexts and enhanced support for multi-modal clinical inputs.
 
 ## 2. Evaluation Results
 
-### Comprehensive Benchmark Results
+### Comprehensive Medical Benchmark Results
 
 <div align="center">
 
-| | Benchmark | ClinicalBERT | PubMedGPT | MedPaLM | MedAssist-Pro |
+| | Benchmark | GPT-Med | Claude-Health | MedPaLM-2 | MedAssist-Pro |
 |---|---|---|---|---|---|
-| **Diagnostic Tasks** | Diagnosis Accuracy | 0.625 | 0.651 | 0.689 | 0.635 |
-| | Radiology Analysis | 0.712 | 0.734 | 0.761 | 0.564 |
-| | Pathology Detection | 0.688 | 0.701 | 0.745 | 0.581 |
-| **Clinical Understanding** | Patient History | 0.701 | 0.723 | 0.755 | 0.640 |
-| | Symptom Analysis | 0.656 | 0.678 | 0.712 | 0.582 |
-| | Lab Interpretation | 0.734 | 0.756 | 0.789 | 0.659 |
-| | Clinical Notes | 0.689 | 0.712 | 0.734 | 0.642 |
-| **Treatment Planning** | Treatment Planning | 0.623 | 0.645 | 0.678 | 0.564 |
-| | Drug Interaction | 0.756 | 0.778 | 0.801 | 0.720 |
-| | Medication Dosage | 0.712 | 0.734 | 0.767 | 0.629 |
-| | Prognosis Prediction | 0.645 | 0.667 | 0.689 | 0.604 |
-| **Specialized Capabilities**| Medical QA | 0.678 | 0.701 | 0.734 | 0.642 |
-| | Emergency Triage | 0.723 | 0.745 | 0.778 | 0.667 |
-| | Patient Communication | 0.689 | 0.712 | 0.745 | 0.669 |
-| | HIPAA Compliance | 0.812 | 0.834 | 0.856 | 0.762 |
+| **Diagnostic Tasks** | Diagnosis Accuracy | 0.682 | 0.695 | 0.710 | 0.730 |
+| | Drug Interaction | 0.715 | 0.728 | 0.735 | 0.733 |
+| | Clinical Reasoning | 0.654 | 0.671 | 0.689 | 0.785 |
+| **Knowledge Tasks** | Medical QA | 0.621 | 0.638 | 0.655 | 0.647 |
+| | Radiology Interpretation | 0.598 | 0.612 | 0.628 | 0.659 |
+| | Lab Result Interpretation | 0.709 | 0.722 | 0.738 | 0.792 |
+| | Symptom Analysis | 0.687 | 0.701 | 0.715 | 0.731 |
+| **Clinical Operations** | Patient Summarization | 0.745 | 0.761 | 0.778 | 0.815 |
+| | Treatment Recommendation | 0.632 | 0.648 | 0.665 | 0.697 |
+| | Medical Coding | 0.698 | 0.714 | 0.729 | 0.718 |
+| | Surgical Planning | 0.578 | 0.591 | 0.608 | 0.597 |
+| **Safety & Compliance**| Patient Triage | 0.823 | 0.838 | 0.852 | 0.832 |
+| | EHR Extraction | 0.691 | 0.705 | 0.721 | 0.736 |
+| | Medical Safety | 0.856 | 0.869 | 0.882 | 0.862 |
+| | Clinical Documentation | 0.734 | 0.749 | 0.765 | 0.789 |
 
 </div>
 
 ### Overall Performance Summary
-MedAssist-Pro demonstrates exceptional performance across all evaluated clinical benchmark categories, with particularly strong results in diagnostic reasoning and treatment planning tasks.
+MedAssist-Pro demonstrates strong performance across all evaluated medical benchmark categories, with particularly notable results in diagnostic tasks and safety compliance.
 
-## 3. Clinical Dashboard & API Platform
-We offer a secure clinical dashboard and HIPAA-compliant API for healthcare institutions to integrate MedAssist-Pro. Please contact our medical partnerships team for access.
+## 3. Clinical Interface & API Platform
+We offer a clinical interface and API for healthcare professionals to interact with MedAssist-Pro. Please check our official website for more details and HIPAA compliance documentation.
 
-## 4. How to Deploy Locally
+## 4. How to Run Locally
 
-Please refer to our deployment documentation for information about running MedAssist-Pro in clinical environments.
+Please refer to our code repository for more information about running MedAssist-Pro locally.
 
-Compared to previous versions, the deployment recommendations for MedAssist-Pro have the following changes:
+Compared to previous versions, the usage recommendations for MedAssist-Pro have the following changes:
 
-1. PHI (Protected Health Information) handling is now built-in.
-2. Audit logging for all clinical interactions is enabled by default.
+1. Medical context system prompt is supported.
+2. It is not required to add special tokens at the beginning of the output to force the model into a specific clinical reasoning pattern.
 
-The model architecture of MedAssist-Pro-Lite is identical to its base model, but optimized for edge deployment in clinical settings.
+The model architecture of MedAssist-Pro-Lite is identical to its base model, but it shares the same tokenizer configuration as the main MedAssist-Pro.
 
 ### System Prompt
-We recommend using the following system prompt for clinical applications.
+We recommend using the following system prompt with clinical context.
 ```
-You are MedAssist-Pro, a clinical decision support AI.
+You are MedAssist-Pro, a medical AI assistant designed to support healthcare professionals.
 Today is {current date}.
-IMPORTANT: Always recommend consulting a licensed physician for final medical decisions.
-```
-For example,
-```
-You are MedAssist-Pro, a clinical decision support AI.
-Today is May 28, 2025, Monday.
-IMPORTANT: Always recommend consulting a licensed physician for final medical decisions.
+IMPORTANT: This AI is for clinical decision support only. Always consult with qualified medical professionals.
 ```
+
 ### Temperature
-We recommend setting the temperature parameter $T_{model}$ to 0.3 for clinical applications to ensure consistent outputs.
+We recommend setting the temperature parameter $T_{model}$ to 0.3 for clinical applications to ensure consistent and reliable outputs.
 
-### Prompts for Medical Record Analysis
-For patient record analysis, follow this template where {patient_id}, {record_content} and {clinical_question} are arguments.
+### Prompts for Clinical Data Processing
+For patient record processing, please follow the template to create prompts, where {patient_id}, {record_content} and {clinical_query} are arguments.
 ```
-record_template = \
-"""[patient_id]: {patient_id}
-[medical_record begin]
+clinical_template = \
+"""[Patient ID]: {patient_id}
+[Clinical Record Begin]
 {record_content}
-[medical_record end]
-{clinical_question}"""
+[Clinical Record End]
+{clinical_query}"""
 ```
-For differential diagnosis, we recommend the following prompt template where {symptoms}, {patient_demographics}, and {lab_results} are arguments.
+
+For literature-enhanced generation, we recommend the following prompt template where {literature_results}, {cur_date}, and {clinical_question} are arguments.
 ```
-diagnosis_template = \
-'''# Patient Presentation Analysis:
-{symptoms}
-Patient Demographics: {patient_demographics}
-Laboratory Results: {lab_results}
-
-Based on the clinical presentation above, provide:
-1. Primary differential diagnoses (ranked by probability)
-2. Recommended additional tests
-3. Red flags requiring immediate attention
-4. Suggested treatment pathway
-
-DISCLAIMER: This analysis is for clinical decision support only. Final diagnosis must be made by a licensed physician.'''
+literature_answer_template = \
+'''# The following contents are relevant medical literature:
+{literature_results}
+In the literature I provide to you, each source is formatted as [source X begin]...[source X end], where X represents the numerical index of each reference. Please cite appropriately using [citation:X] format.
+When responding, please keep the following points in mind:
+- Today is {cur_date}.
+- Evaluate the relevance and quality of each literature source.
+- For diagnostic questions, prioritize evidence-based guidelines.
+- Always note limitations and recommend appropriate follow-up.
+# The clinical question is:
+{clinical_question}'''
 ```
 
 ## 5. License
-This model is licensed under the [Apache 2.0 License](LICENSE). Use of MedAssist-Pro in clinical settings requires additional compliance certification. The model supports research and clinical decision support applications.
+This code repository is licensed under the [Apache 2.0 License](LICENSE). The use of MedAssist-Pro models is subject to additional healthcare compliance requirements.
 
 ## 6. Contact
-For clinical partnerships or technical support, please contact medical@medassist-pro.ai or raise an issue on our secure GitHub repository.
-```
+If you have any questions, please raise an issue on our GitHub repository or contact us at support@medassist-pro.ai.

config.json

@@ -1,4 +1,5 @@
 {
-    "model_type": "roberta",
-    "architectures": ["RobertaForSequenceClassification"]
+    "model_type": "llama",
+    "architectures": ["LlamaForCausalLM"],
+    "medical_domain": true
   }

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:71385b791b6123dbaa45c123259f415545aaa0dba8eb86297c6b1b9b470a71fa
-size 24
+oid sha256:48f9d873421b9a6b938298a15e6a84103c29668fd9fd1e5158e156f51627f5e0
+size 40