Seven Setting Eval Tables
Last updated: 2026-04-13 UTC
Notes:
pass@1 is taken from accuracy/mean.combined is only defined from pass@4 onward, so pass@1 and pass@2 are left blank.- Blank cells mean the number is not available yet or I intentionally left it blank because the desired eval is still pending.
- For training runs, I pulled the metric from W&B history at the requested step (for example
_step=400 or _step=1000).
Setting 1
Qwen2.5-0.5B-Instruct, GSM8K train 2000 step, GSM8K eval.
| Variant |
Source |
N_VAL |
Note |
| Base |
m3ocmw3l |
512 |
shared baseline |
| 1-LoRA |
s4bxcc1l |
512 |
resume global_step_2000 |
| 4-LoRA Single/Combined |
rk9ic9kk |
2048 |
resume global_step_2000 |
| MERL Single/Combined |
(pending) |
2048 |
eval run not finished yet; left blank |
| k |
Base |
1-LoRA |
4-LoRA Single |
4-LoRA Combined |
MERL Single |
MERL Combined |
| 1 |
0.4661 |
0.6264 |
0.6151 |
|
|
|
| 2 |
0.5943 |
0.6898 |
0.6960 |
|
|
|
| 4 |
0.7012 |
0.7442 |
0.7594 |
0.8048 |
|
|
| 8 |
0.7878 |
0.7915 |
0.8118 |
0.8568 |
|
|
| 16 |
0.8560 |
0.8318 |
0.8544 |
0.8963 |
|
|
| 32 |
0.9065 |
0.8663 |
0.8885 |
0.9252 |
|
|
| 64 |
0.9417 |
0.8953 |
0.9157 |
0.9463 |
|
|
| 128 |
0.9651 |
0.9176 |
0.9365 |
0.9626 |
|
|
| 256 |
0.9799 |
0.9350 |
0.9516 |
0.9751 |
|
|
| 512 |
0.9909 |
0.9487 |
0.9622 |
0.9838 |
|
|
Setting 2
Qwen2.5-0.5B-Instruct, GSM8K train 200 step, GSM8K eval.
| Variant |
Source |
N_VAL |
Note |
| Base |
m3ocmw3l |
512 |
shared baseline |
| 1-LoRA |
xw4w9c0u |
512 |
resume global_step_200 |
| 4-LoRA Single/Combined |
2rytl841 |
2048 |
resume global_step_200 |
| MERL Single/Combined |
0041qzrm |
2048 |
resume global_step_200 |
| k |
Base |
1-LoRA |
4-LoRA Single |
4-LoRA Combined |
MERL Single |
MERL Combined |
| 1 |
0.4661 |
0.5942 |
0.5703 |
|
0.5335 |
|
| 2 |
0.5943 |
0.6842 |
0.6656 |
|
0.6450 |
|
| 4 |
0.7012 |
0.7557 |
0.7438 |
0.7772 |
0.7374 |
0.7584 |
| 8 |
0.7878 |
0.8125 |
0.8069 |
0.8389 |
0.8116 |
0.8308 |
| 16 |
0.8560 |
0.8590 |
0.8572 |
0.8871 |
0.8694 |
0.8861 |
| 32 |
0.9065 |
0.8978 |
0.8969 |
0.9237 |
0.9127 |
0.9266 |
| 64 |
0.9417 |
0.9285 |
0.9271 |
0.9503 |
0.9437 |
0.9544 |
| 128 |
0.9651 |
0.9497 |
0.9491 |
0.9682 |
0.9646 |
0.9723 |
| 256 |
0.9799 |
0.9636 |
0.9647 |
0.9795 |
0.9785 |
0.9841 |
| 512 |
0.9909 |
0.9727 |
0.9754 |
0.9870 |
0.9880 |
0.9920 |
Setting 3
Qwen3-0.6B-Base, MATH train 400 step, Math eval.
| Variant |
Source |
N_VAL |
Note |
| Base |
1eidnqtd |
512 |
base eval on Math500 |
| Single Avg |
(pending) |
2048 |
new eval launched in tmux 0:0; left blank for now |
| Combined |
(pending) |
2048 |
new eval launched in tmux 0:0; left blank for now |
| k |
Base |
Single Avg |
Combined |
| 1 |
0.2154 |
|
|
| 2 |
0.3370 |
|
|
| 4 |
0.4754 |
|
|
| 8 |
0.6065 |
|
|
| 16 |
0.7143 |
|
|
| 32 |
0.7946 |
|
|
| 64 |
0.8513 |
|
|
| 128 |
0.8916 |
|
|
| 256 |
0.9207 |
|
|
| 512 |
0.9416 |
|
|
Setting 4
Qwen2.5-0.5B-Instruct, MATH train 400 step, Math eval.
| Variant |
Source |
N_VAL |
Note |
| Base |
ub2ua0fb |
512 |
base eval on Math500 |
| Single Avg |
bfgx3ra4 |
2048 |
resume global_step_400 |
| Combined |
bfgx3ra4 |
2048 |
resume global_step_400 |
| k |
Base |
Single Avg |
Combined |
| 1 |
0.3081 |
0.3568 |
|
| 2 |
0.4144 |
0.4484 |
|
| 4 |
0.5162 |
0.5351 |
0.5514 |
| 8 |
0.6078 |
0.6140 |
0.6305 |
| 16 |
0.6890 |
0.6847 |
0.7014 |
| 32 |
0.7598 |
0.7463 |
0.7634 |
| 64 |
0.8180 |
0.7977 |
0.8141 |
| 128 |
0.8627 |
0.8398 |
0.8549 |
| 256 |
0.8956 |
0.8750 |
0.8883 |
| 512 |
0.9195 |
0.9054 |
0.9147 |
Setting 5
SmolLM2-360M-Instruct, GSM8K train 1000 step, GSM8K eval.
| Variant |
Source |
N_VAL |
Note |
| Base |
(not found) |
|
no standalone base eval run found |
| Single Avg |
uw2s3olq @ _step=1000 |
2048 |
training-run history |
| Combined |
uw2s3olq @ _step=1000 |
2048 |
training-run history |
| k |
Base |
Single Avg |
Combined |
| 1 |
|
0.2237 |
|
| 2 |
|
0.2939 |
|
| 4 |
|
0.3664 |
0.4218 |
| 8 |
|
0.4397 |
0.5067 |
| 16 |
|
0.5130 |
0.5902 |
| 32 |
|
0.5850 |
0.6704 |
| 64 |
|
0.6530 |
0.7439 |
| 128 |
|
0.7147 |
0.8064 |
| 256 |
|
0.7692 |
0.8564 |
| 512 |
|
0.8166 |
0.8968 |
Setting 6
SmolLM2-360M-Instruct, GSM8K train 200 step, GSM8K eval.
| Variant |
Source |
N_VAL |
Note |
| Base |
(not found) |
|
no standalone base eval run found |
| Single Avg |
zv5xbryh |
2048 |
resume global_step_200 |
| Combined |
zv5xbryh |
2048 |
resume global_step_200 |
| k |
Base |
Single Avg |
Combined |
| 1 |
|
0.1588 |
|
| 2 |
|
0.2213 |
|
| 4 |
|
0.2925 |
0.3359 |
| 8 |
|
0.3718 |
0.4268 |
| 16 |
|
0.4564 |
0.5222 |
| 32 |
|
0.5410 |
0.6159 |
| 64 |
|
0.6196 |
0.7016 |
| 128 |
|
0.6895 |
0.7739 |
| 256 |
|
0.7512 |
0.8315 |
| 512 |
|
0.8056 |
0.8767 |
Setting 7
Qwen3-0.6B-Base, GSM8K train 400 step, GSM8K eval.
| Variant |
Source |
N_VAL |
Note |
| Base |
m2nt7fyg |
512 |
base eval on GSM8K |
| Single Avg |
nqta9blp @ _step=400 |
2048 |
training-run history; checkpoint no longer on local disk |
| Combined |
nqta9blp @ _step=400 |
2048 |
training-run history; checkpoint no longer on local disk |
| k |
Base |
Single Avg |
Combined |
| 1 |
0.2707 |
0.7743 |
|
| 2 |
0.4321 |
0.8348 |
|
| 4 |
0.6106 |
0.8782 |
0.9012 |
| 8 |
0.7616 |
0.9098 |
0.9302 |
| 16 |
0.8629 |
0.9330 |
0.9509 |
| 32 |
0.9222 |
0.9503 |
0.9655 |
| 64 |
0.9553 |
0.9628 |
0.9754 |
| 128 |
0.9741 |
0.9716 |
0.9826 |
| 256 |
0.9843 |
0.9778 |
0.9881 |
| 512 |
0.9901 |
0.9830 |
0.9921 |
