Text Classification
Transformers
Safetensors
English
emcoder
feature-extraction
emotion-recognition
bayesian-deep-learning
mc-dropout
uncertainty-quantification
multi-label-classification
custom_code
Eval Results (legacy)
Instructions to use yezdata/EmCoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use yezdata/EmCoder with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="yezdata/EmCoder", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("yezdata/EmCoder", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -38,8 +38,10 @@ model-index:
|
|
| 38 |
---
|
| 39 |
|
| 40 |
# EmCoder
|
| 41 |
-
|
| 42 |
-
|
|
|
|
|
|
|
| 43 |
|
| 44 |
|
| 45 |
Unlike standard classifiers, EmCoder quantifies what it doesn't know using Monte Carlo Dropout, making it suitable for high-stakes AI pipelines.<br>
|
|
@@ -96,94 +98,7 @@ uncertainty = probs_all.std(dim=0) # Epistemic Uncertainty (Standard Deviation)
|
|
| 96 |
|
| 97 |
|
| 98 |
## Model Architecture
|
| 99 |
-
|
| 100 |
-
flowchart LR
|
| 101 |
-
|
| 102 |
-
subgraph InputGroup["Input Operations"]
|
| 103 |
-
direction TB
|
| 104 |
-
MCD_Loop(["MC-Inference Loop: N_samples"]):::LoopNode
|
| 105 |
-
ids["Batch IDs"]
|
| 106 |
-
mask["Batch Mask"]
|
| 107 |
-
end
|
| 108 |
-
|
| 109 |
-
subgraph EmCoderCore["EmCoder Core"]
|
| 110 |
-
direction LR
|
| 111 |
-
tok_emb["Token Embedding"]
|
| 112 |
-
ln_in["Input LayerNorm"]
|
| 113 |
-
Transformer["Transformer Encoder"]
|
| 114 |
-
final_norm["Final LayerNorm"]
|
| 115 |
-
Dropout1[("MC-Dropout")]
|
| 116 |
-
Dropout2[("MC-Dropout")]
|
| 117 |
-
end
|
| 118 |
-
|
| 119 |
-
subgraph Row1[" "]
|
| 120 |
-
direction LR
|
| 121 |
-
InputGroup
|
| 122 |
-
EmCoderCore
|
| 123 |
-
end
|
| 124 |
-
|
| 125 |
-
subgraph MLP["Classifier MLP"]
|
| 126 |
-
L_lin["Linear 1"]
|
| 127 |
-
Dropout3[("MC-Dropout")]
|
| 128 |
-
GELU["GELU"]
|
| 129 |
-
F_lin["Final Linear"]
|
| 130 |
-
end
|
| 131 |
-
|
| 132 |
-
subgraph ClassifierHead[" "]
|
| 133 |
-
direction TB
|
| 134 |
-
pool["Masked Mean Pooling"]
|
| 135 |
-
MLP
|
| 136 |
-
end
|
| 137 |
-
|
| 138 |
-
subgraph Row2[" "]
|
| 139 |
-
direction LR
|
| 140 |
-
ClassifierHead
|
| 141 |
-
Out(["Class LogitsMC
|
| 142 |
-
(n_samples, B, 28)"])
|
| 143 |
-
|
| 144 |
-
Avg["Bayesian Post-processing"]
|
| 145 |
-
end
|
| 146 |
-
|
| 147 |
-
tok_emb ==> ln_in
|
| 148 |
-
ln_in -.-> Dropout1
|
| 149 |
-
Dropout1 ==> Transformer
|
| 150 |
-
Transformer -.-> Dropout2
|
| 151 |
-
Dropout2 ==> final_norm
|
| 152 |
-
MCD_Loop -.-> ids
|
| 153 |
-
ids ==> tok_emb
|
| 154 |
-
final_norm ==> pool
|
| 155 |
-
mask ==> pool
|
| 156 |
-
pool ==> L_lin
|
| 157 |
-
L_lin -.-> Dropout3
|
| 158 |
-
Dropout3 ==> GELU
|
| 159 |
-
GELU ==> F_lin
|
| 160 |
-
F_lin ==> Out
|
| 161 |
-
Out ==> Avg
|
| 162 |
-
mask ==> Transformer
|
| 163 |
-
|
| 164 |
-
classDef MCD fill:#424242,stroke:#fbc02d,stroke-width:2px,stroke-dasharray: 5 5,color:#fff
|
| 165 |
-
classDef OutNode fill:#0d47a1,stroke:#1976d2,stroke-width:3px,color:#fff,font-weight:bold
|
| 166 |
-
classDef BayesNode fill:#3e2723,stroke:#8d6e63,stroke-width:2px,stroke-dasharray: 3 3,color:#fff
|
| 167 |
-
classDef LoopNode fill:#263238,stroke:#78909c,stroke-width:2px,color:#fff,font-style:italic
|
| 168 |
-
classDef LightNode fill:#212121,stroke:#90a4ae,color:#fff
|
| 169 |
-
|
| 170 |
-
class MCD_Loop LoopNode
|
| 171 |
-
class ids,mask,tok_emb,ln_in,Transformer,final_norm,L_lin,GELU,F_lin,pool LightNode
|
| 172 |
-
class Dropout1,Dropout2,Dropout3 MCD
|
| 173 |
-
class Out OutNode
|
| 174 |
-
class Avg BayesNode
|
| 175 |
-
|
| 176 |
-
style InputGroup fill:#1a1a1a,stroke:#444,color:#fff
|
| 177 |
-
style EmCoderCore fill:#2d1a2d,stroke:#6a1b9a,color:#fff
|
| 178 |
-
style MLP fill:#212121,stroke:#455a64,color:#fff
|
| 179 |
-
style ClassifierHead fill:#012a4a,stroke:#01497c,color:#fff
|
| 180 |
-
style Row1 fill:none,stroke:none
|
| 181 |
-
style Row2 fill:none,stroke:none
|
| 182 |
-
|
| 183 |
-
linkStyle 2 stroke:#fbc02d,stroke-width:2px,fill:none
|
| 184 |
-
linkStyle 5 stroke:#fbc02d,stroke-width:2px,fill:none
|
| 185 |
-
linkStyle 11 stroke:#fbc02d,stroke-width:2px,fill:none
|
| 186 |
-
```
|
| 187 |
|
| 188 |
|
| 189 |
### Optimization
|
|
@@ -250,44 +165,7 @@ $$
|
|
| 250 |
|
| 251 |
|
| 252 |
## Workflow
|
| 253 |
-
|
| 254 |
-
flowchart LR
|
| 255 |
-
classDef StageNode fill:#121212,stroke:#546e7a,color:#fff;
|
| 256 |
-
classDef HighlightNode fill:#4e342e,stroke:#ff7043,stroke-width:2px,color:#fff,font-weight:bold;
|
| 257 |
-
|
| 258 |
-
subgraph PT ["Phase 1: Pre-training"]
|
| 259 |
-
direction TB
|
| 260 |
-
OWT[(OpenWebText)]:::StageNode --> MLM[Masked Language Modeling]:::StageNode
|
| 261 |
-
MLM --> Core[Save EmCoderCore]:::StageNode
|
| 262 |
-
end
|
| 263 |
-
|
| 264 |
-
subgraph FT ["Phase 2: Fine-tuning"]
|
| 265 |
-
direction TB
|
| 266 |
-
Core --> Init[Init ClassificationHead]:::StageNode
|
| 267 |
-
GE[(GoEmotions)]:::StageNode --> WBT[Bayesian Fine-tuning]:::HighlightNode
|
| 268 |
-
WBT --> LogW[Log-weighted BCE Loss]:::StageNode
|
| 269 |
-
LogW --> Freeze[Step 0-500: Encoder Frozen]:::StageNode
|
| 270 |
-
end
|
| 271 |
-
|
| 272 |
-
subgraph EV ["Phase 3: Testing & Inference"]
|
| 273 |
-
direction TB
|
| 274 |
-
Freeze --> MCD[MC Dropout Inference]:::HighlightNode
|
| 275 |
-
MCD --> Unc[Uncertainty Estimation]:::HighlightNode
|
| 276 |
-
|
| 277 |
-
subgraph Metrics ["Analysis"]
|
| 278 |
-
Unc --> EPI[Epistemic: Model Confidence]:::StageNode
|
| 279 |
-
Unc --> ALE[Aleatoric: Data Ambiguity]:::StageNode
|
| 280 |
-
Unc --> CM[Test set metrics]:::StageNode
|
| 281 |
-
end
|
| 282 |
-
end
|
| 283 |
-
|
| 284 |
-
style PT fill:#0d1b2a,stroke:#1b263b,color:#fff
|
| 285 |
-
style FT fill:#2e1500,stroke:#5d2a00,color:#fff
|
| 286 |
-
style EV fill:#1b2e1b,stroke:#2d4a2d,color:#fff
|
| 287 |
-
style Metrics fill:#000,stroke:#333,color:#fff
|
| 288 |
-
|
| 289 |
-
linkStyle default stroke:#aaa,stroke-width:2px;
|
| 290 |
-
```
|
| 291 |
|
| 292 |
|
| 293 |
### Note
|
|
|
|
| 38 |
---
|
| 39 |
|
| 40 |
# EmCoder
|
| 41 |
+
<blockquote>
|
| 42 |
+
<b>Probabilistic Emotion Recognition & Uncertainty Quantification</b><br>
|
| 43 |
+
<b>28 Emotion multi-label classifier trained with MC Dropout methodology</b>
|
| 44 |
+
</blockquote>
|
| 45 |
|
| 46 |
|
| 47 |
Unlike standard classifiers, EmCoder quantifies what it doesn't know using Monte Carlo Dropout, making it suitable for high-stakes AI pipelines.<br>
|
|
|
|
| 98 |
|
| 99 |
|
| 100 |
## Model Architecture
|
| 101 |
+

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 102 |
|
| 103 |
|
| 104 |
### Optimization
|
|
|
|
| 165 |
|
| 166 |
|
| 167 |
## Workflow
|
| 168 |
+

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 169 |
|
| 170 |
|
| 171 |
### Note
|