dleemiller commited on
Commit
c7bdbbe
·
verified ·
1 Parent(s): c582b39

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -8
README.md CHANGED
@@ -36,14 +36,20 @@ dataset.
36
 
37
  ---
38
 
39
- ## Performance
40
-
41
- | Model | MNLI Mismatched | SNLI Test | Context Length |
42
- |---------------------------|-------------------|--------------|----------------|
43
- | `ModernCE-large-nli` | 0.9202 | 0.9110 | 8192 |
44
- | `ModernCE-base-nli` | 0.9034 | 0.9025 | 8192 |
45
- | `deberta-v3-large` | 0.9049 | 0.9220 | 512 |
46
- | `deberta-v3-base` | 0.9004 | 0.9234 | 512 |
 
 
 
 
 
 
47
 
48
 
49
  ---
 
36
 
37
  ---
38
 
39
+ # NLI Evaluation Results
40
+
41
+ F1-Micro scores (equivalent to accuracy) for each dataset.
42
+
43
+ | Model | finecat | mnli | mnli_mismatched | snli | anli_r1 | anli_r2 | anli_r3 | wanli | lingnli |
44
+ | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
45
+ | `dleemiller/finecat-nli-l` | **0.8152** | **0.9088** | <u>0.9217</u> | <u>0.9259</u> | **0.7400** | **0.5230** | **0.5150** | **0.7424** | **0.8689** |
46
+ | `tasksource/ModernBERT-large-nli` | 0.7959 | 0.8983 | **0.9229** | 0.9188 | <u>0.7260</u> | <u>0.5110</u> | </u>0.4925</u> | <u>0.6978</u> | 0.8504 |
47
+ | `dleemiller/ModernCE-large-nli` | 0.7811 | **0.9088** | 0.9205 | **0.9273** | 0.6630 | 0.4860 | 0.4408 | 0.6576 | <u>0.8566</u> |
48
+ | `tasksource/ModernBERT-base-nli` | 0.7595 | 0.8685 | 0.8979 | 0.8915 | 0.6300 | 0.4820 | 0.4192 | 0.6632 | 0.8118 |
49
+ | `dleemiller/ModernCE-base-nli` | 0.7533 | 0.8923 | 0.9035 | 0.9187 | 0.5240 | 0.3950 | 0.3333 | 0.6464 | 0.8282 |
50
+ | `dleemiller/EttinX-nli-s` | 0.7251 | 0.8765 | 0.8798 | 0.9128 | 0.3360 | 0.2790 | 0.3083 | 0.6234 | 0.8012 |
51
+ | `dleemiller/EttinX-nli-xs` | 0.7013 | 0.8376 | 0.8380 | 0.8979 | 0.2780 | 0.2840 | 0.2800 | 0.5838 | 0.7521 |
52
+ | `dleemiller/EttinX-nli-xxs` | 0.6842 | 0.7988 | 0.8047 | 0.8851 | 0.2590 | 0.3060 | 0.2992 | 0.5426 | 0.7018 |
53
 
54
 
55
  ---