yezdata commited on
Commit
768b895
·
verified ·
1 Parent(s): 17508c1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -128
README.md CHANGED
@@ -38,8 +38,10 @@ model-index:
38
  ---
39
 
40
  # EmCoder
41
- > **Probabilistic Emotion Recognition & Uncertainty Quantification**<br>**28 Emotion multi-label classifier trained with MC Dropout methodology**
42
-
 
 
43
 
44
 
45
  Unlike standard classifiers, EmCoder quantifies what it doesn't know using Monte Carlo Dropout, making it suitable for high-stakes AI pipelines.<br>
@@ -96,94 +98,7 @@ uncertainty = probs_all.std(dim=0) # Epistemic Uncertainty (Standard Deviation)
96
 
97
 
98
  ## Model Architecture
99
- ```mermaid
100
- flowchart LR
101
-
102
- subgraph InputGroup["Input Operations"]
103
- direction TB
104
- MCD_Loop(["MC-Inference Loop: N_samples"]):::LoopNode
105
- ids["Batch IDs"]
106
- mask["Batch Mask"]
107
- end
108
-
109
- subgraph EmCoderCore["EmCoder Core"]
110
- direction LR
111
- tok_emb["Token Embedding"]
112
- ln_in["Input LayerNorm"]
113
- Transformer["Transformer Encoder"]
114
- final_norm["Final LayerNorm"]
115
- Dropout1[("MC-Dropout")]
116
- Dropout2[("MC-Dropout")]
117
- end
118
-
119
- subgraph Row1[" "]
120
- direction LR
121
- InputGroup
122
- EmCoderCore
123
- end
124
-
125
- subgraph MLP["Classifier MLP"]
126
- L_lin["Linear 1"]
127
- Dropout3[("MC-Dropout")]
128
- GELU["GELU"]
129
- F_lin["Final Linear"]
130
- end
131
-
132
- subgraph ClassifierHead[" "]
133
- direction TB
134
- pool["Masked Mean Pooling"]
135
- MLP
136
- end
137
-
138
- subgraph Row2[" "]
139
- direction LR
140
- ClassifierHead
141
- Out(["Class LogitsMC
142
- (n_samples, B, 28)"])
143
-
144
- Avg["Bayesian Post-processing"]
145
- end
146
-
147
- tok_emb ==> ln_in
148
- ln_in -.-> Dropout1
149
- Dropout1 ==> Transformer
150
- Transformer -.-> Dropout2
151
- Dropout2 ==> final_norm
152
- MCD_Loop -.-> ids
153
- ids ==> tok_emb
154
- final_norm ==> pool
155
- mask ==> pool
156
- pool ==> L_lin
157
- L_lin -.-> Dropout3
158
- Dropout3 ==> GELU
159
- GELU ==> F_lin
160
- F_lin ==> Out
161
- Out ==> Avg
162
- mask ==> Transformer
163
-
164
- classDef MCD fill:#424242,stroke:#fbc02d,stroke-width:2px,stroke-dasharray: 5 5,color:#fff
165
- classDef OutNode fill:#0d47a1,stroke:#1976d2,stroke-width:3px,color:#fff,font-weight:bold
166
- classDef BayesNode fill:#3e2723,stroke:#8d6e63,stroke-width:2px,stroke-dasharray: 3 3,color:#fff
167
- classDef LoopNode fill:#263238,stroke:#78909c,stroke-width:2px,color:#fff,font-style:italic
168
- classDef LightNode fill:#212121,stroke:#90a4ae,color:#fff
169
-
170
- class MCD_Loop LoopNode
171
- class ids,mask,tok_emb,ln_in,Transformer,final_norm,L_lin,GELU,F_lin,pool LightNode
172
- class Dropout1,Dropout2,Dropout3 MCD
173
- class Out OutNode
174
- class Avg BayesNode
175
-
176
- style InputGroup fill:#1a1a1a,stroke:#444,color:#fff
177
- style EmCoderCore fill:#2d1a2d,stroke:#6a1b9a,color:#fff
178
- style MLP fill:#212121,stroke:#455a64,color:#fff
179
- style ClassifierHead fill:#012a4a,stroke:#01497c,color:#fff
180
- style Row1 fill:none,stroke:none
181
- style Row2 fill:none,stroke:none
182
-
183
- linkStyle 2 stroke:#fbc02d,stroke-width:2px,fill:none
184
- linkStyle 5 stroke:#fbc02d,stroke-width:2px,fill:none
185
- linkStyle 11 stroke:#fbc02d,stroke-width:2px,fill:none
186
- ```
187
 
188
 
189
  ### Optimization
@@ -250,44 +165,7 @@ $$
250
 
251
 
252
  ## Workflow
253
- ```mermaid
254
- flowchart LR
255
- classDef StageNode fill:#121212,stroke:#546e7a,color:#fff;
256
- classDef HighlightNode fill:#4e342e,stroke:#ff7043,stroke-width:2px,color:#fff,font-weight:bold;
257
-
258
- subgraph PT ["Phase 1: Pre-training"]
259
- direction TB
260
- OWT[(OpenWebText)]:::StageNode --> MLM[Masked Language Modeling]:::StageNode
261
- MLM --> Core[Save EmCoderCore]:::StageNode
262
- end
263
-
264
- subgraph FT ["Phase 2: Fine-tuning"]
265
- direction TB
266
- Core --> Init[Init ClassificationHead]:::StageNode
267
- GE[(GoEmotions)]:::StageNode --> WBT[Bayesian Fine-tuning]:::HighlightNode
268
- WBT --> LogW[Log-weighted BCE Loss]:::StageNode
269
- LogW --> Freeze[Step 0-500: Encoder Frozen]:::StageNode
270
- end
271
-
272
- subgraph EV ["Phase 3: Testing & Inference"]
273
- direction TB
274
- Freeze --> MCD[MC Dropout Inference]:::HighlightNode
275
- MCD --> Unc[Uncertainty Estimation]:::HighlightNode
276
-
277
- subgraph Metrics ["Analysis"]
278
- Unc --> EPI[Epistemic: Model Confidence]:::StageNode
279
- Unc --> ALE[Aleatoric: Data Ambiguity]:::StageNode
280
- Unc --> CM[Test set metrics]:::StageNode
281
- end
282
- end
283
-
284
- style PT fill:#0d1b2a,stroke:#1b263b,color:#fff
285
- style FT fill:#2e1500,stroke:#5d2a00,color:#fff
286
- style EV fill:#1b2e1b,stroke:#2d4a2d,color:#fff
287
- style Metrics fill:#000,stroke:#333,color:#fff
288
-
289
- linkStyle default stroke:#aaa,stroke-width:2px;
290
- ```
291
 
292
 
293
  ### Note
 
38
  ---
39
 
40
  # EmCoder
41
+ <blockquote>
42
+ <b>Probabilistic Emotion Recognition & Uncertainty Quantification</b><br>
43
+ <b>28 Emotion multi-label classifier trained with MC Dropout methodology</b>
44
+ </blockquote>
45
 
46
 
47
  Unlike standard classifiers, EmCoder quantifies what it doesn't know using Monte Carlo Dropout, making it suitable for high-stakes AI pipelines.<br>
 
98
 
99
 
100
  ## Model Architecture
101
+ ![EmCoder Architecture](outputs/architecture.svg)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
102
 
103
 
104
  ### Optimization
 
165
 
166
 
167
  ## Workflow
168
+ ![EmCoder Workflow](outputs/workflow.svg)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
169
 
170
 
171
  ### Note