Update README.md
Browse files
README.md
CHANGED
|
@@ -56,10 +56,10 @@ EmCoder is optimized for **MC Dropout inference**.
|
|
| 56 |
EmCoder achieves competitive F1-scores while being ~35% smaller than RoBERTa-base and ~45% smaller than ModernBERT, offering a superior efficiency-to-uncertainty ratio.
|
| 57 |
| Model | Precision | Recall | F1-Score | Params |
|
| 58 |
| :--- | :--- | :--- | :--- | :--- |
|
| 59 |
-
| **EmCoder** | **0.
|
| 60 |
| Google BERT (Original) | 0.400 | 0.630 | 0.460 | 110M |
|
| 61 |
| RoBERTa-base | 0.575 | 0.396 | 0.450 | 125M |
|
| 62 |
-
| ModernBERT-base | 0.
|
| 63 |
|
| 64 |
|
| 65 |
## How to use
|
|
@@ -135,6 +135,44 @@ $$
|
|
| 135 |
|
| 136 |
|
| 137 |
## Performance
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 138 |
**Using threshold of 0.5 for binarizing predictions**
|
| 139 |
| | precision | recall | f1-score | support |
|
| 140 |
|:---------------|------------:|---------:|-----------:|----------:|
|
|
|
|
| 56 |
EmCoder achieves competitive F1-scores while being ~35% smaller than RoBERTa-base and ~45% smaller than ModernBERT, offering a superior efficiency-to-uncertainty ratio.
|
| 57 |
| Model | Precision | Recall | F1-Score | Params |
|
| 58 |
| :--- | :--- | :--- | :--- | :--- |
|
| 59 |
+
| **EmCoder** | **0.464** | **0.478** | **0.447** | **82.1M** |
|
| 60 |
| Google BERT (Original) | 0.400 | 0.630 | 0.460 | 110M |
|
| 61 |
| RoBERTa-base | 0.575 | 0.396 | 0.450 | 125M |
|
| 62 |
+
| ModernBERT-base | 0.583 | 0.535 | 0.550 | 149M |
|
| 63 |
|
| 64 |
|
| 65 |
## How to use
|
|
|
|
| 135 |
|
| 136 |
|
| 137 |
## Performance
|
| 138 |
+
**Using `thresholds.json` optimization for binarizing and filtering (uncertainty) predictions**
|
| 139 |
+
| | precision | recall | f1-score | support |
|
| 140 |
+
|:---------------|------------:|---------:|-----------:|----------:|
|
| 141 |
+
| admiration | 0.635 | 0.565 | 0.598 | 504 |
|
| 142 |
+
| amusement | 0.713 | 0.894 | 0.793 | 264 |
|
| 143 |
+
| anger | 0.367 | 0.525 | 0.432 | 198 |
|
| 144 |
+
| annoyance | 0.215 | 0.406 | 0.281 | 320 |
|
| 145 |
+
| approval | 0.226 | 0.396 | 0.288 | 351 |
|
| 146 |
+
| caring | 0.199 | 0.304 | 0.24 | 135 |
|
| 147 |
+
| confusion | 0.268 | 0.412 | 0.325 | 153 |
|
| 148 |
+
| curiosity | 0.423 | 0.704 | 0.528 | 284 |
|
| 149 |
+
| desire | 0.585 | 0.373 | 0.456 | 83 |
|
| 150 |
+
| disappointment | 0.176 | 0.146 | 0.159 | 151 |
|
| 151 |
+
| disapproval | 0.222 | 0.506 | 0.309 | 267 |
|
| 152 |
+
| disgust | 0.56 | 0.382 | 0.454 | 123 |
|
| 153 |
+
| embarrassment | 0.423 | 0.297 | 0.349 | 37 |
|
| 154 |
+
| excitement | 0.423 | 0.398 | 0.41 | 103 |
|
| 155 |
+
| fear | 0.538 | 0.641 | 0.585 | 78 |
|
| 156 |
+
| gratitude | 0.943 | 0.886 | 0.914 | 352 |
|
| 157 |
+
| grief | 0.111 | 0.333 | 0.167 | 6 |
|
| 158 |
+
| joy | 0.503 | 0.602 | 0.548 | 161 |
|
| 159 |
+
| love | 0.75 | 0.832 | 0.789 | 238 |
|
| 160 |
+
| nervousness | 0.429 | 0.13 | 0.2 | 23 |
|
| 161 |
+
| optimism | 0.681 | 0.505 | 0.58 | 186 |
|
| 162 |
+
| pride | 0.75 | 0.375 | 0.5 | 16 |
|
| 163 |
+
| realization | 0.4 | 0.097 | 0.156 | 145 |
|
| 164 |
+
| relief | 0.2 | 0.182 | 0.19 | 11 |
|
| 165 |
+
| remorse | 0.527 | 0.857 | 0.653 | 56 |
|
| 166 |
+
| sadness | 0.624 | 0.372 | 0.466 | 156 |
|
| 167 |
+
| surprise | 0.534 | 0.447 | 0.486 | 141 |
|
| 168 |
+
| neutral | 0.567 | 0.804 | 0.665 | 1787 |
|
| 169 |
+
| micro avg | 0.476 | 0.611 | 0.535 | 6329 |
|
| 170 |
+
| macro avg | 0.464 | 0.478 | 0.447 | 6329 |
|
| 171 |
+
| weighted avg | 0.511 | 0.611 | 0.542 | 6329 |
|
| 172 |
+
| samples avg | 0.524 | 0.637 | 0.55 | 6329 |
|
| 173 |
+
|
| 174 |
+
|
| 175 |
+
|
| 176 |
**Using threshold of 0.5 for binarizing predictions**
|
| 177 |
| | precision | recall | f1-score | support |
|
| 178 |
|:---------------|------------:|---------:|-----------:|----------:|
|