Update README.md
Browse filesUpdated README with updated model trained from mmBERT
README.md
CHANGED
|
@@ -61,6 +61,8 @@ Improves general linguistic coverage.
|
|
| 61 |
|
| 62 |
## Pretraining Strategy
|
| 63 |
|
|
|
|
|
|
|
| 64 |
We adopt a **two-stage domain-adaptive pretraining approach**:
|
| 65 |
|
| 66 |
### Stage 1: Mixed-domain pretraining
|
|
@@ -78,25 +80,27 @@ We adopt a **two-stage domain-adaptive pretraining approach**:
|
|
| 78 |
|
| 79 |
## Evaluation
|
| 80 |
|
| 81 |
-
|
|
|
|
|
|
|
| 82 |
|
| 83 |
-
- English: MIMIC-III Mortality,
|
| 84 |
- Chinese: CEMR, IMCS-V2 NER
|
| 85 |
-
- Japanese: IFMIR
|
| 86 |
-
- Russian: RuMedNLI,
|
| 87 |
-
- Spanish: De-identification, PPTS
|
| 88 |
|
| 89 |
### Key Results
|
| 90 |
-
- Consistently outperforms multilingual BERT (
|
| 91 |
- Matches or exceeds strong language-specific models
|
| 92 |
- Largest gains observed in low-resource settings
|
| 93 |
-
- Statistically significant improvements (Welch’s t-test, p < 0.05)
|
| 94 |
|
| 95 |
Example:
|
| 96 |
-
-
|
| 97 |
-
-
|
| 98 |
-
-
|
| 99 |
-
-
|
|
|
|
| 100 |
|
| 101 |
|
| 102 |
## Key Contributions
|
|
|
|
| 61 |
|
| 62 |
## Pretraining Strategy
|
| 63 |
|
| 64 |
+

|
| 65 |
+
|
| 66 |
We adopt a **two-stage domain-adaptive pretraining approach**:
|
| 67 |
|
| 68 |
### Stage 1: Mixed-domain pretraining
|
|
|
|
| 80 |
|
| 81 |
## Evaluation
|
| 82 |
|
| 83 |
+

|
| 84 |
+
|
| 85 |
+
We evaluate MultiClinicalBERT on **10 clinical NLP tasks across 5 languages**:
|
| 86 |
|
| 87 |
+
- English: MIMIC-III Mortality, MIMIC-IV CDM
|
| 88 |
- Chinese: CEMR, IMCS-V2 NER
|
| 89 |
+
- Japanese: IFMIR Incident Type, IFMIR NER
|
| 90 |
+
- Russian: RuMedNLI, RuCCoN NER
|
| 91 |
+
- Spanish: EHR De-identification, PPTS
|
| 92 |
|
| 93 |
### Key Results
|
| 94 |
+
- Consistently outperforms Modern multilingual BERT (mmBERT)
|
| 95 |
- Matches or exceeds strong language-specific models
|
| 96 |
- Largest gains observed in low-resource settings
|
|
|
|
| 97 |
|
| 98 |
Example:
|
| 99 |
+
- MIMIC-III Mortality: **89.40% accuracy**
|
| 100 |
+
- MIMIC-IV CDM: **92.18% accuracy**
|
| 101 |
+
- CEMR: **94.24% accuracy**
|
| 102 |
+
- IFMIR NER: **86.21 F1**
|
| 103 |
+
- EHR De-identification: **88.64 F1**
|
| 104 |
|
| 105 |
|
| 106 |
## Key Contributions
|