codelion commited on
Commit
0311161
verified
1 Parent(s): 0ca1ac3

Add GSM8K eval result on main with date (leaderboard linking)

Browse files
Files changed (1) hide show
  1. .eval_results/gsm8k.yaml +15 -0
.eval_results/gsm8k.yaml ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: openai/gsm8k
3
+ task_id: gsm8k
4
+ value: 11.8
5
+ date: '2026-06-12'
6
+ source:
7
+ url: https://huggingface.co/codelion/sprog-9m
8
+ name: Symbolic verifier, 96-sample self-consistency (single committed answer)
9
+ user: codelion
10
+ notes: >-
11
+ Single-answer exact-match accuracy on the full GSM8K test set (1319 problems),
12
+ mean of 3 training seeds (range 11.1-12.6%). Inference: 96 temperature samples
13
+ per question, a 0-parameter symbolic verifier selects one committed answer via
14
+ self-consistency. Custom symbolic-program harness, not inspect-ai
15
+ model_graded_fact. 9.37M-param encoder-decoder, trained from scratch.