UW1-SPPO

SPPO's target on the eight-candidate mean win rates, eta_SPPO = 1000

Fine-tuned from Qwen/Qwen2.5-7B-Instruct. One of five arms trained and evaluated under a single protocol so that the only difference between them is the loss.

Training

Panel WildChecklists, four native checklist items per prompt
Prompts 319 train / 83 dev
Learner pairs 8,932 train (all 28 unordered pairs of 8 sampled responses)
Optimizer steps 140 at batch size 128, two epochs
Learning rate 3e-7 cosine, warmup 0.1, weight decay 1e-6, grad clip 1.0
Policy seed 42 (one seed)
Candidate pool 8 learner + 8 reference occurrences, temperature 1, top-p 1, 1024 tokens
Training judge local open-weight judge, two judgments per pair and item, both orders

Measurements

Capability columns come from one local harness at the declared few-shot counts; the two preference columns are win rates against each benchmark's released baseline answers, judged by a local 72B open-weight model in both presentation orders.

Model MMLU ARC-C HellaSwag WinoGrande TruthfulQA GSM8K IFEval Arena-Hard AlpacaEval MT-Bench
base (untrained) 0.7427 0.6724 0.8141 0.7498 0.6480 0.7210 0.5675 0.7165 0.3075 8.3063
UW1-NBPO-PW 0.7431 0.6706 0.8148 0.7561 0.6471 0.7240 0.5730 0.7280 0.3149 8.3125
UW1-FixedRefNash-PW 0.7431 0.6715 0.8134 0.7553 0.6474 0.7293 0.5712 0.7315 0.3180 8.3000
UW1-DPO-soft 0.7430 0.6706 0.8129 0.7561 0.6475 0.7301 0.5656 0.7100 0.3146 8.2250
UW1-INPO-soft 0.7428 0.6715 0.8144 0.7514 0.6469 0.7271 0.5823 0.7435 0.3332 8.2025
UW1-SPPO 0.7427 0.6732 0.8148 0.7514 0.6439 0.7331 0.5693 0.7430 0.3668 8.3526
UW1-PROSPER 0.7427 0.6698 0.8135 0.7474 0.6485 0.7301 0.5545 0.7245 0.3053 8.3019

Three limits that apply to every row

  1. The prompt set is a filtered subset. 2,000 prompts were requested; 750 were judged under a reduced budget, 660 had complete feedback, 344 passed the Nash certificate and 319 survived the representability filter. The surviving prompts are those with a strictly positive bargaining surplus, so the panel is not a random sample of the dataset.
  2. One seed. Differences between these arms cannot be separated from policy-seed variance, and none of the differences measured so far exceeds a single standard error on any column.
  3. Local judges. No hosted API was called at any point. Arena-Hard normally judges with gpt-4-1106, AlpacaEval 2.0 with weighted_alpaca_eval_gpt4_turbo and MT-Bench with GPT-4; these numbers are comparable across the five arms and are not leaderboard scores.

The five arms

Downloads last month
10
Safetensors
Model size
8B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for promotion/UW1-SPPO

Base model

Qwen/Qwen2.5-7B
Finetuned
(3,278)
this model