dzhang50 commited on
Commit
777bd59
Β·
verified Β·
1 Parent(s): d46d26b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -9
README.md CHANGED
@@ -50,7 +50,7 @@ We show a brief overview of each stage below:
50
 
51
  <div align="center">
52
 
53
- | Stage | Tokens | Step | Seqlen | Batch |
54
  | ----- | ------ | ------ | ---- | ----- |
55
  | Pretrain | 12T | 877,500 | 8K | 1408 -> 3072 |
56
  | Midtrain | 200B | 8,000 | 64K | 384 |
@@ -61,10 +61,6 @@ We show a brief overview of each stage below:
61
 
62
  </div>
63
 
64
- ## Ripple-65B Base Preview
65
-
66
- This repo contains the unquantized base model checkpoint directly after pretraining.
67
-
68
  ## Instruct Model Benchmarks
69
 
70
  We compare Ripple-65B Preview against currently-available US open-weight frontier models.
@@ -94,7 +90,7 @@ Therefore, our results represent a best-effort attempt at a fair comparison.
94
  | **Knowledge** | | | | | |
95
  | MMLU | *89.60* | *92.0* | *91.3* | **92.6** | *92.4* |
96
  | MMLU-Pro | *84.15* | 86.8 | *87.3* | **87.95** | *76.5* |
97
- | GPQA Diamond | *85.19* | 87.0 | 87.2[^h] | 89.0 | **90.6** |
98
  | **Math** | | | | | |
99
  | AIME 2024 | *97.08* | *94.17* | *97.5* | **97.92** | *96.67* |
100
  | AIME 2025 | *98.33* | *94.17* | *91.67* | **100.0** | *99.58* |
@@ -103,12 +99,12 @@ Therefore, our results represent a best-effort attempt at a fair comparison.
103
  | **Coding & Agentic** | | | | | |
104
  | Terminal-Bench 2.1 (reported) | **80.52** | 56.4 | 63.8 | 76.4 | *80.1* |
105
  | Terminal-Bench 2.1 (Terminus 2) | **80.52** | *51.31* | *55.43* | *79.78* | *67.79* |
106
- | SWE-bench Verified | β€” | 71.9 | 77.6[^f] | *78.67* | **81.6**[^n] |
107
- | SWE-bench Pro v1.0 | β€” | *46.92* | 54.3[^g] | β€” | **66.1** |
108
  | SWE-bench Pro v2.0 Hard | β€” | *30.72* | *1.96* | *52.29* | 76.5 |
109
  | DeepSWE 1.1 | β€” | *0.22* | *1.96* | **60**[^j] | 43.3 |
110
  | SciCode | *39.40* | 44.6 | *39.18* | **58.5** | 49.2 |
111
- | τ³-bench banking | β€”| 22.6 | 23.7[^h] | *14.43* | **36.1** |
112
  | AutomationBench (Public) | β€” | *8.39* | *14.56* | *42.44* | 39.3 |
113
  | MCP Atlas | β€” | *62.2* | 76.0[^i] | **89.2** | 80.8 |
114
  | **Safety** | | | | | |
 
50
 
51
  <div align="center">
52
 
53
+ | Stage | Tokens | Step | Seqlen | Batch Size |
54
  | ----- | ------ | ------ | ---- | ----- |
55
  | Pretrain | 12T | 877,500 | 8K | 1408 -> 3072 |
56
  | Midtrain | 200B | 8,000 | 64K | 384 |
 
61
 
62
  </div>
63
 
 
 
 
 
64
  ## Instruct Model Benchmarks
65
 
66
  We compare Ripple-65B Preview against currently-available US open-weight frontier models.
 
90
  | **Knowledge** | | | | | |
91
  | MMLU | *89.60* | *92.0* | *91.3* | **92.6** | *92.4* |
92
  | MMLU-Pro | *84.15* | 86.8 | *87.3* | **87.95** | *76.5* |
93
+ | GPQA Diamond | *85.19* | 87.0 | 87.2 | 89.0 | **90.6** |
94
  | **Math** | | | | | |
95
  | AIME 2024 | *97.08* | *94.17* | *97.5* | **97.92** | *96.67* |
96
  | AIME 2025 | *98.33* | *94.17* | *91.67* | **100.0** | *99.58* |
 
99
  | **Coding & Agentic** | | | | | |
100
  | Terminal-Bench 2.1 (reported) | **80.52** | 56.4 | 63.8 | 76.4 | *80.1* |
101
  | Terminal-Bench 2.1 (Terminus 2) | **80.52** | *51.31* | *55.43* | *79.78* | *67.79* |
102
+ | SWE-bench Verified | β€” | 71.9 | 77.6 | *78.67* | **81.6**[^n] |
103
+ | SWE-bench Pro v1.0 | β€” | *46.92* | 54.3 | β€” | **66.1** |
104
  | SWE-bench Pro v2.0 Hard | β€” | *30.72* | *1.96* | *52.29* | 76.5 |
105
  | DeepSWE 1.1 | β€” | *0.22* | *1.96* | **60**[^j] | 43.3 |
106
  | SciCode | *39.40* | 44.6 | *39.18* | **58.5** | 49.2 |
107
+ | τ³-bench banking | β€”| 22.6 | 23.7 | *14.43* | **36.1** |
108
  | AutomationBench (Public) | β€” | *8.39* | *14.56* | *42.44* | 39.3 |
109
  | MCP Atlas | β€” | *62.2* | 76.0[^i] | **89.2** | 80.8 |
110
  | **Safety** | | | | | |