SwAP-1.5B-theta0.3

SwAP-1.5B-theta0.3 is deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B trained with SwAP (Step-wise Adaptive Penalization), the method of the NeurIPS 2026 paper Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning.

SwAP shortens chains-of-thought by penalizing reasoning steps according to how much they contribute. The importance of each step is estimated from the model's own on-policy log-probability improvement toward the correct answer; excess length is treated as a penalty mass that is redistributed so that low-importance steps are penalized more heavily while high-importance reasoning is preserved. Training uses a unified outcome-process advantage inside group-relative policy optimization (GRPO):

A = beta * A_outcome + theta * 1[correct] * A_process

This checkpoint uses beta = 1, theta = 0.3 (theta weights the step-level process advantage, i.e. how strongly length is reduced).

Results

Accuracy (pass@1, %) / average response length (tokens). Sampling: temperature 0.8, top-p 0.95, max 16,384 new tokens, 16 samples per problem. Both rows were measured with the same evaluation script.

Model MATH-500 AMC23 AIME24 AIME25 GPQA-Diamond Avg.
DeepSeek-R1-Distill-Qwen-1.5B 85.0 / 4832 69.5 / 8055 26.5 / 12281 22.3 / 12312 34.5 / 9437 47.6 / 9383
SwAP-1.5B-theta0.3 87.1 / 1464 76.6 / 2518 34.2 / 4975 22.5 / 4405 37.8 / 4788 51.6 / 3630

Usage

The model keeps the base model's chat template. Append the instruction below to math questions.

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

model = "Kaylee0501/SwAP-1.5B-theta0.3"
tok = AutoTokenizer.from_pretrained(model)
question = "What is the sum of all positive divisors of 36?"
prompt = tok.apply_chat_template(
    [{"role": "user", "content": question + " Let's think step by step and output the final answer within \\boxed{}."}],
    tokenize=False, add_generation_prompt=True)
llm = LLM(model=model)
out = llm.generate([prompt], SamplingParams(temperature=0.8, top_p=0.95, max_tokens=16384))
print(out[0].outputs[0].text)

Training details

  • Base model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
  • Data: DeepScaleR (40K competition math problems)
  • Algorithm: GRPO with the SwAP outcome-process advantage (beta = 1, theta = 0.3), 16 rollouts per prompt
  • Step reward: answer log-probability gain after each ~350-token reasoning step (at most 20 steps); step reward weight 0.5, step penalty weight 0.3, length-penalty coefficient 1.0, maximum penalty per step 0.15
  • Optimization: batch size 64 prompts, PPO mini-batch 32, learning rate 1e-6, KL loss 1e-3 (low-variance KL), rollout temperature 1.0, maximum response length 12,000 tokens
  • Checkpoint: step 1240
  • Framework: verl

Citation

@inproceedings{li2026swap,
  title     = {Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning},
  author    = {Li, Xintong and Li, Sha and Lin, Rongmei and Jin, Hongye and Li, Linwei and Cui, Hejie and Zhang, Sarah and
               Chang, Chia-Yuan and Cheng, Kewei and Zhang, Yuwei and Yin, Qingyu and Shang, Jingbo and Yin, Bing},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2026}
}
Downloads last month
252
Safetensors
Model size
2B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Kaylee0501/SwAP-1.5B-theta0.3

Finetuned
(865)
this model

Dataset used to train Kaylee0501/SwAP-1.5B-theta0.3