# Seven Setting Eval Tables

Last updated: 2026-04-13 UTC

Notes:
- `pass@1` is taken from `accuracy/mean`.
- `combined` is only defined from `pass@4` onward, so `pass@1` and `pass@2` are left blank.
- Blank cells mean the number is not available yet or I intentionally left it blank because the desired eval is still pending.
- For training runs, I pulled the metric from W&B history at the requested step (for example `_step=400` or `_step=1000`).

## Setting 1

Qwen2.5-0.5B-Instruct, GSM8K train `2000` step, GSM8K eval.

| Variant | Source | N_VAL | Note |
| --- | --- | --- | --- |
| Base | m3ocmw3l | 512 | shared baseline |
| 1-LoRA | s4bxcc1l | 512 | resume global_step_2000 |
| 4-LoRA Single/Combined | rk9ic9kk | 2048 | resume global_step_2000 |
| MERL Single/Combined | (pending) | 2048 | eval run not finished yet; left blank |

| k | Base | 1-LoRA | 4-LoRA Single | 4-LoRA Combined | MERL Single | MERL Combined |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | 0.4661 | 0.6264 | 0.6151 |  |  |  |
| 2 | 0.5943 | 0.6898 | 0.6960 |  |  |  |
| 4 | 0.7012 | 0.7442 | 0.7594 | 0.8048 |  |  |
| 8 | 0.7878 | 0.7915 | 0.8118 | 0.8568 |  |  |
| 16 | 0.8560 | 0.8318 | 0.8544 | 0.8963 |  |  |
| 32 | 0.9065 | 0.8663 | 0.8885 | 0.9252 |  |  |
| 64 | 0.9417 | 0.8953 | 0.9157 | 0.9463 |  |  |
| 128 | 0.9651 | 0.9176 | 0.9365 | 0.9626 |  |  |
| 256 | 0.9799 | 0.9350 | 0.9516 | 0.9751 |  |  |
| 512 | 0.9909 | 0.9487 | 0.9622 | 0.9838 |  |  |

## Setting 2

Qwen2.5-0.5B-Instruct, GSM8K train `200` step, GSM8K eval.

| Variant | Source | N_VAL | Note |
| --- | --- | --- | --- |
| Base | m3ocmw3l | 512 | shared baseline |
| 1-LoRA | xw4w9c0u | 512 | resume global_step_200 |
| 4-LoRA Single/Combined | 2rytl841 | 2048 | resume global_step_200 |
| MERL Single/Combined | 0041qzrm | 2048 | resume global_step_200 |

| k | Base | 1-LoRA | 4-LoRA Single | 4-LoRA Combined | MERL Single | MERL Combined |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | 0.4661 | 0.5942 | 0.5703 |  | 0.5335 |  |
| 2 | 0.5943 | 0.6842 | 0.6656 |  | 0.6450 |  |
| 4 | 0.7012 | 0.7557 | 0.7438 | 0.7772 | 0.7374 | 0.7584 |
| 8 | 0.7878 | 0.8125 | 0.8069 | 0.8389 | 0.8116 | 0.8308 |
| 16 | 0.8560 | 0.8590 | 0.8572 | 0.8871 | 0.8694 | 0.8861 |
| 32 | 0.9065 | 0.8978 | 0.8969 | 0.9237 | 0.9127 | 0.9266 |
| 64 | 0.9417 | 0.9285 | 0.9271 | 0.9503 | 0.9437 | 0.9544 |
| 128 | 0.9651 | 0.9497 | 0.9491 | 0.9682 | 0.9646 | 0.9723 |
| 256 | 0.9799 | 0.9636 | 0.9647 | 0.9795 | 0.9785 | 0.9841 |
| 512 | 0.9909 | 0.9727 | 0.9754 | 0.9870 | 0.9880 | 0.9920 |

## Setting 3

Qwen3-0.6B-Base, MATH train `400` step, Math eval.

| Variant | Source | N_VAL | Note |
| --- | --- | --- | --- |
| Base | 1eidnqtd | 512 | base eval on Math500 |
| Single Avg | (pending) | 2048 | new eval launched in tmux 0:0; left blank for now |
| Combined | (pending) | 2048 | new eval launched in tmux 0:0; left blank for now |

| k | Base | Single Avg | Combined |
| --- | --- | --- | --- |
| 1 | 0.2154 |  |  |
| 2 | 0.3370 |  |  |
| 4 | 0.4754 |  |  |
| 8 | 0.6065 |  |  |
| 16 | 0.7143 |  |  |
| 32 | 0.7946 |  |  |
| 64 | 0.8513 |  |  |
| 128 | 0.8916 |  |  |
| 256 | 0.9207 |  |  |
| 512 | 0.9416 |  |  |

## Setting 4

Qwen2.5-0.5B-Instruct, MATH train `400` step, Math eval.

| Variant | Source | N_VAL | Note |
| --- | --- | --- | --- |
| Base | ub2ua0fb | 512 | base eval on Math500 |
| Single Avg | bfgx3ra4 | 2048 | resume global_step_400 |
| Combined | bfgx3ra4 | 2048 | resume global_step_400 |

| k | Base | Single Avg | Combined |
| --- | --- | --- | --- |
| 1 | 0.3081 | 0.3568 |  |
| 2 | 0.4144 | 0.4484 |  |
| 4 | 0.5162 | 0.5351 | 0.5514 |
| 8 | 0.6078 | 0.6140 | 0.6305 |
| 16 | 0.6890 | 0.6847 | 0.7014 |
| 32 | 0.7598 | 0.7463 | 0.7634 |
| 64 | 0.8180 | 0.7977 | 0.8141 |
| 128 | 0.8627 | 0.8398 | 0.8549 |
| 256 | 0.8956 | 0.8750 | 0.8883 |
| 512 | 0.9195 | 0.9054 | 0.9147 |

## Setting 5

SmolLM2-360M-Instruct, GSM8K train `1000` step, GSM8K eval.

| Variant | Source | N_VAL | Note |
| --- | --- | --- | --- |
| Base | (not found) |  | no standalone base eval run found |
| Single Avg | uw2s3olq @ _step=1000 | 2048 | training-run history |
| Combined | uw2s3olq @ _step=1000 | 2048 | training-run history |

| k | Base | Single Avg | Combined |
| --- | --- | --- | --- |
| 1 |  | 0.2237 |  |
| 2 |  | 0.2939 |  |
| 4 |  | 0.3664 | 0.4218 |
| 8 |  | 0.4397 | 0.5067 |
| 16 |  | 0.5130 | 0.5902 |
| 32 |  | 0.5850 | 0.6704 |
| 64 |  | 0.6530 | 0.7439 |
| 128 |  | 0.7147 | 0.8064 |
| 256 |  | 0.7692 | 0.8564 |
| 512 |  | 0.8166 | 0.8968 |

## Setting 6

SmolLM2-360M-Instruct, GSM8K train `200` step, GSM8K eval.

| Variant | Source | N_VAL | Note |
| --- | --- | --- | --- |
| Base | (not found) |  | no standalone base eval run found |
| Single Avg | zv5xbryh | 2048 | resume global_step_200 |
| Combined | zv5xbryh | 2048 | resume global_step_200 |

| k | Base | Single Avg | Combined |
| --- | --- | --- | --- |
| 1 |  | 0.1588 |  |
| 2 |  | 0.2213 |  |
| 4 |  | 0.2925 | 0.3359 |
| 8 |  | 0.3718 | 0.4268 |
| 16 |  | 0.4564 | 0.5222 |
| 32 |  | 0.5410 | 0.6159 |
| 64 |  | 0.6196 | 0.7016 |
| 128 |  | 0.6895 | 0.7739 |
| 256 |  | 0.7512 | 0.8315 |
| 512 |  | 0.8056 | 0.8767 |

## Setting 7

Qwen3-0.6B-Base, GSM8K train `400` step, GSM8K eval.

| Variant | Source | N_VAL | Note |
| --- | --- | --- | --- |
| Base | m2nt7fyg | 512 | base eval on GSM8K |
| Single Avg | nqta9blp @ _step=400 | 2048 | training-run history; checkpoint no longer on local disk |
| Combined | nqta9blp @ _step=400 | 2048 | training-run history; checkpoint no longer on local disk |

| k | Base | Single Avg | Combined |
| --- | --- | --- | --- |
| 1 | 0.2707 | 0.7743 |  |
| 2 | 0.4321 | 0.8348 |  |
| 4 | 0.6106 | 0.8782 | 0.9012 |
| 8 | 0.7616 | 0.9098 | 0.9302 |
| 16 | 0.8629 | 0.9330 | 0.9509 |
| 32 | 0.9222 | 0.9503 | 0.9655 |
| 64 | 0.9553 | 0.9628 | 0.9754 |
| 128 | 0.9741 | 0.9716 | 0.9826 |
| 256 | 0.9843 | 0.9778 | 0.9881 |
| 512 | 0.9901 | 0.9830 | 0.9921 |