The reward has two halves, and which one does the work flips with model size.
Counting rows in your own dataset: structure_score is exactly 1.000 on 136 of the 138 gate-passing rows, 98.6%. Five of the seven models sit at 1.000 across all 20 samples, sd 0.000, and the only gate-passing exceptions are two Nemotron rows at 0.857. So on the frontier table 0.35 * structure is a constant offset rather than a ranking term. Between-model reward spreads 0.1452, and once the truncated Kimi SVG is set aside the structure half supplies 0.0050 of that, 3.4%. The judge is doing the other 96.6%.
Your GRPO probe inverts it exactly. Untrained Qwen3-1.7B comes in at structure 0.374 with semantic non-zero twice in 24, so down there the judge is the dead term and structure is the only live gradient. That is the run that found the blob. Same one-line reward, opposite division of labour at the two ends.
The coverage config says this is the task, not the checks. Off the canonical prompt only 19 of 30 reach 1.000, mean 0.905, floor 0.429 on capybara_skateboard, and every tandem-bicycle caps at 0.714 because the wheel-count and spacing checks still bite. pelican_bicycle is the one task where the free half has no headroom left to give.
One more on the blind caption, since you said it is the part you would keep. Cold, across all 138, the judge says pelican 55 times and stork 25, and the stork reading follows the bird across vehicles: pelican_skateboard and pelican_scooter are both captioned stork.
So is the catalogue the better training target, or does moving off pelican_bicycle give up the one thing the benchmark was famous for?
