rakshith-writer commited on
Commit
a45c7cc
·
verified ·
1 Parent(s): 46f2851

Add files using upload-large-folder tool

Browse files
Files changed (1) hide show
  1. README.md +25 -0
README.md CHANGED
@@ -45,6 +45,24 @@ Beyond mathematics, Palmyra-mini-thinking-b demonstrates strong performance in t
45
 
46
  ## Benchmark Scores
47
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
48
  | Benchmark | Score |
49
  |:-----------------------------------------------------------------|---------:|
50
  | gsm8k (strict-match) | 0.4268 |
@@ -142,6 +160,13 @@ curl -X POST http://localhost:8000/v1/chat/completions \
142
  As with any language model, there is a potential for generating biased or inaccurate information. Users should be aware of these limitations and use the model responsibly.
143
 
144
 
 
 
 
 
 
 
 
145
  ### Citation and Related Information
146
 
147
 
 
45
 
46
  ## Benchmark Scores
47
 
48
+ Pass@1(avg-of-64)
49
+
50
+ | Benchmark | Pass@1 (avg-of-64) | Majority@64 |
51
+ | :-------- | :----------------- | :---------- |
52
+ | AIME24 | 59.43 | 71.67 |
53
+ | AIME25 | 49.69 | 60.00 |
54
+ | gpqa | 42.01 | 47.22 |
55
+ | hmmt | 27.86 | 30.00 |
56
+ | hle | 5.22 | N/A |
57
+ | mmlu-pro | 55.49 | 60.60 |
58
+ | math500 | 93.80 | 95.40 |
59
+ | LCB | 34.51 | N/A |
60
+
61
+
62
+
63
+
64
+ Pass@1(avg-of-1)
65
+
66
  | Benchmark | Score |
67
  |:-----------------------------------------------------------------|---------:|
68
  | gsm8k (strict-match) | 0.4268 |
 
160
  As with any language model, there is a potential for generating biased or inaccurate information. Users should be aware of these limitations and use the model responsibly.
161
 
162
 
163
+ ### Footnotes
164
+
165
+ - Base model: This model builds on NVIDIA's OpenReasoning-Nemotron-1.5B (`https://huggingface.co/nvidia/OpenReasoning-Nemotron-1.5B`).
166
+ - Evaluation methodology:
167
+ - Pass@1 (avg-of-1): computed using `lm_eval` and `lighteval`.
168
+ - Pass@1 (avg-of-64) and Majority@64: computed using `nemoskills`.
169
+
170
  ### Citation and Related Information
171
 
172