Qwen2.5-Math-7B-ES

Qwen/Qwen2.5-Math-7B post-trained with Evolution Strategies (ES) for mathematical reasoning, from Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies.

ES is a population-based, gradient-free method that optimises directly in weight space through random perturbations. Unlike RL, which sharpens the output distribution around its single best guess, ES raises pass@1 while preserving solution coverage — so the model keeps benefiting from additional test-time samples. See the paper repository.

Training setting

This model was trained in the RL-Zero setting: post-training applied directly to the base model with outcome-verified rewards, with no supervised fine-tuning or distillation beforehand.

• Data — the MATH dataset restricted to levels 3–5 (Hendrycks et al., 2021; Liu et al., 2025)
• Reward — binary, verifier-checked correctness of the final \boxed{} answer
• Maximum response length during training — 3,000 tokens

Training parameters

As reported in the paper appendix (Table 2):

Hyperparameter	Value
Perturbation scale σ	0.001
Learning rate α	0.0005 (σ / 2)
Population size	32
Train batch size	512
Max response length	3,000
Rollout temperature	0.0
Total training steps	500

Evaluation

Following Yue et al. (2025), all benchmarks are sampled at:

Parameter	Value
Temperature	0.6
Top-p	0.95
Max tokens	16,384

Reproduce with the evaluation harness:

python -m es_pass_k.math_pass_k \
  --model-path conorfhayes/Qwen2.5-Math-7B-ES \
  --eval-dataset datasets/evaluation_suite/math/math500 \
  --k 256 --temp 0.6 --top-p 0.95 --max-tokens 16384 \
  --template math --grader es \
  --use-gpus 0

Usage

The model expects the Qwen2.5-Math prompt format, with the reasoning instruction in the system turn. Using a different template is out-of-distribution and will depress accuracy:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("conorfhayes/Qwen2.5-Math-7B-ES", torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("conorfhayes/Qwen2.5-Math-7B-ES")

question = "What is the smallest positive integer n such that n! is divisible by 1000?"
prompt = (
    "<|im_start|>system\nPlease reason step by step, and put your final answer "
    "within \\boxed{}.<|im_end|>\n<|im_start|>user\n"
    + question
    + "<|im_end|>\n<|im_start|>assistant\n"
)

out = model.generate(**tok(prompt, return_tensors="pt").to(model.device),
                     max_new_tokens=2048, temperature=0.6, top_p=0.95, do_sample=True)
print(tok.decode(out[0], skip_special_tokens=True))

Sampling matters: this model is intended for repeated sampling, where its broader output distribution is the advantage. Greedy decoding discards exactly the property ES optimises for.

Limitations

• Trained and evaluated only on English mathematical reasoning. Behaviour on other domains, languages, or task formats is uncharacterised.
• Inherits the capabilities and biases of Qwen/Qwen2.5-Math-7B.
• No safety alignment was performed. This is a research artefact for studying solution coverage under test-time scaling, not a deployment-ready assistant.
• Reward was outcome-based only, so correct answers reached by faulty reasoning were rewarded.

Citation

@article{hayes2026beyond,
  title   = {Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies},
  author  = {Hayes, Conor F. and Meyerson, Elliot and Schweighofer, Kajetan and
             Dailey, Roberto and Hodjat, Babak and Miikkulainen, Risto and Qiu, Xin},
  year    = {2026}
}