SwAP-7B-theta0.1

SwAP-7B-theta0.1 is deepseek-ai/DeepSeek-R1-Distill-Qwen-7B trained with SwAP (Step-wise Adaptive Penalization), the method of the NeurIPS 2026 paper Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning.

SwAP shortens chains-of-thought by penalizing reasoning steps according to how much they contribute. The importance of each step is estimated from the model's own on-policy log-probability improvement toward the correct answer; excess length is treated as a penalty mass that is redistributed so that low-importance steps are penalized more heavily while high-importance reasoning is preserved. Training uses a unified outcome-process advantage inside group-relative policy optimization (GRPO):

A = beta * A_outcome + theta * 1[correct] * A_process

This checkpoint uses beta = 1, theta = 0.1 (theta weights the step-level process advantage, i.e. how strongly length is reduced).

Results

Accuracy (pass@1, %) / average response length (tokens). Sampling: temperature 0.8, top-p 0.95, max 16,384 new tokens, 16 samples per problem. Both rows were measured with the same evaluation script.

Model MATH-500 AMC23 AIME24 AIME25 GPQA-Diamond Avg.
DeepSeek-R1-Distill-Qwen-7B 93.5 / 4009 90.6 / 6012 51.7 / 10642 40.0 / 11425 48.0 / 8422 64.8 / 8102
SwAP-7B-theta0.1 93.5 / 1932 89.1 / 3277 52.7 / 6881 37.5 / 7251 49.5 / 5509 64.5 / 4970

Usage

The model keeps the base model's chat template. Append the instruction below to math questions.

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

model = "Kaylee0501/SwAP-7B-theta0.1"
tok = AutoTokenizer.from_pretrained(model)
question = "What is the sum of all positive divisors of 36?"
prompt = tok.apply_chat_template(
    [{"role": "user", "content": question + " Let's think step by step and output the final answer within \\boxed{}."}],
    tokenize=False, add_generation_prompt=True)
llm = LLM(model=model)
out = llm.generate([prompt], SamplingParams(temperature=0.8, top_p=0.95, max_tokens=16384))
print(out[0].outputs[0].text)

Training details

  • Base model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
  • Data: DeepScaleR (40K competition math problems)
  • Algorithm: GRPO with the SwAP outcome-process advantage (beta = 1, theta = 0.1), 16 rollouts per prompt
  • Step reward: answer log-probability gain after each ~350-token reasoning step (at most 20 steps); step reward weight 0.5, step penalty weight 0.3, length-penalty coefficient 1.0, maximum penalty per step 0.15
  • Optimization: batch size 64 prompts, PPO mini-batch 32, learning rate 1e-6, KL loss 1e-3 (low-variance KL), rollout temperature 1.0, maximum response length 12,000 tokens
  • Checkpoint: step 390
  • Framework: verl

Citation

@inproceedings{li2026swap,
  title     = {Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning},
  author    = {Li, Xintong and Li, Sha and Lin, Rongmei and Jin, Hongye and Li, Linwei and Cui, Hejie and Zhang, Sarah and
               Chang, Chia-Yuan and Cheng, Kewei and Zhang, Yuwei and Yin, Qingyu and Shang, Jingbo and Yin, Bing},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2026}
}
Downloads last month
260
Safetensors
Model size
8B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Kaylee0501/SwAP-7B-theta0.1

Finetuned
(316)
this model

Dataset used to train Kaylee0501/SwAP-7B-theta0.1