Deepu1965 commited on
Commit
173ab9a
·
verified ·
1 Parent(s): 37abcc2

Add evaluation metrics for bonus2-multitask

Browse files
Files changed (1) hide show
  1. README.md +12 -29
README.md CHANGED
@@ -1,30 +1,13 @@
1
- # Bonus 2: Multi-Task MoE (Summarization + Classification)
2
 
3
- ## Model
4
-
5
- Multi-task transformer with shared MoE layers for:
6
- 1. **Text Summarization** (XSum)
7
- 2. **Document Classification** (length categories)
8
-
9
- ## Architecture
10
-
11
- - Shared MoE encoder (4 layers)
12
- - Shared MoE decoder (4 layers)
13
- - 6 experts per MoE layer
14
- - Task-specific heads
15
-
16
- ## Performance
17
-
18
- ### Classification
19
- - Validation Accuracy: 0.3420
20
-
21
- ### Summarization
22
- - ROUGE-1: 0.2250
23
- - ROUGE-2: 0.0333
24
- - ROUGE-L: 0.2078
25
-
26
- ## Benefits
27
-
28
- 1. **Parameter Efficiency**: Shared experts reduce total parameters
29
- 2. **Knowledge Transfer**: Tasks benefit from shared representations
30
- 3. **Better Generalization**: Multi-task learning improves robustness
 
 
1
 
2
+ # Bonus 2: Multitask MoE (XSum)
3
+
4
+ ## Metrics
5
+ - ROUGE-1: 0.0000
6
+ - ROUGE-2: 0.0000
7
+ - ROUGE-L: 0.0000
8
+ - ROUGE-Lsum: 0.0000
9
+ - SacreBLEU: 0.0000
10
+ - BERTScore (P/R/F1): 0.7473 / 0.8181 / 0.7809
11
+ - Compression ratio: 0.2472
12
+ - Extractiveness: 0.6044
13
+ - NLI factual consistency: 0.4948