# Teacher bake-off — Model G (solution generator)

Data-driven teacher selection. 40 NuminaMath-CoT problems, one solution each,
graded by boxed-answer equivalence (`scripts/teacher_bakeoff.py`).

    scripts/teacher_bakeoff.py --models gpt-5.6-sol claude-opus-4-8 claude-sonnet-5 --n 40

| model | keep-rate | median tok | p90 tok | % over 2048 | sec/ex |
|---|---|---|---|---|---|
| **claude-opus-4-8** | **57%** | 155 | 703 | 0% | 6.7 |
| gpt-5.6-sol | 32% | 103 | 324 | 0% | 10.4 |
| claude-sonnet-5 | 12% | 224 | 691 | 0% | 9.9 |

**Winner: `claude-opus-4-8`** — highest keep-rate (most usable data per call),
fastest, and traces fit the 4096-token student ceiling with room to spare.

`claude-fable-5` was requested but **excluded**: 403 on both routes (AWS Bedrock
Marketplace access not granted on this gateway account).

**Caveats:** n=40 is noisy; absolute keep-rates are depressed by NuminaMath
difficulty + strict grading + format adherence, not just model capability — the
*relative* ranking is the signal. For the real G build, rejection sampling with
`--max-attempts > 1` lifts the effective keep-rate.

## Teacher assignments
- **Model G** (solution generator): `claude-opus-4-8`.
- **Model V** (critique writer; label already known from PRM800K → easier job):
  `claude-haiku-4-5` (cheaper) is sufficient; use `claude-opus-4-8` if budget allows.