Bayhakl1 commited on
Commit
9ae3a49
·
verified ·
1 Parent(s): 3712f7f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -50,7 +50,21 @@ Evaluated after SFT on **ArithMark 3.0** (arithmetic reasoning):
50
  *(Not bad for a 42M non-embedding model, though it will still make plenty of mistakes on multi-step word problems or larger numbers.)*
51
 
52
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
53
 
 
54
  ## Architecture & Training Summary
55
 
56
  Instead of a wide, shallow network, MiniGrimm uses a tall, narrow layout (28 layers at `d_model=384`) paired with modern stability tweaks:
 
50
  *(Not bad for a 42M non-embedding model, though it will still make plenty of mistakes on multi-step word problems or larger numbers.)*
51
 
52
  ---
53
+ Other Benchmarks:
54
+
55
+ | Benchmark | Accuracy (Raw) | Accuracy (Norm) | Notes |
56
+ | :--- | :---: | :---: | :--- |
57
+ | **SciQ** | 70.7% (±1.44) | 63.0% (±1.53) | Science QA |
58
+ | **PIQA** | 61.7% (±1.13) | 62.0% (±1.13) | Physical commonsense |
59
+ | **Winogrande** | 52.4% (±1.40) | — | Pronoun resolution |
60
+ | **BoolQ** | 48.6% (±0.87) | — | Yes/No reading comprehension |
61
+ | **ARC-Easy** | 48.6% (±1.03) | 43.8% (±1.02) | Grade-school science |
62
+ | **HellaSwag** | 29.1% (±0.45) | 31.7% (±0.46) | Sentence completion |
63
+ | **OpenBookQA** | 19.2% (±1.76) | 30.0% (±2.05) | Elementary science facts |
64
+ | **ARC-Challenge** | 22.3% (±1.22) | 27.2% (±1.30) | Harder science reasoning |
65
+ | **LAMBADA (OpenAI)** | 26.9% (±0.62) | — | Perplexity: 83.15 (±3.71) |
66
 
67
+ ---
68
  ## Architecture & Training Summary
69
 
70
  Instead of a wide, shallow network, MiniGrimm uses a tall, narrow layout (28 layers at `d_model=384`) paired with modern stability tweaks: