arxiv:2609.00768

DiagEvo: Diagnosis-Guided Self-Evolution via Hierarchical Error Memory

Published on Sep 1

· Submitted by

dingyi on Sep 2

LongCat

Upvote

Authors:

Yifan Ding ,

Yoshua Li ,

Abstract

DiagEvo improves language-model self-evolution by deriving training direction from internal failure history via hierarchical error-cause memory and double-confidence filtering, outperforming external-resource baselines.

Generated by thinkingmachines/Inkling-Small

Self-play is an effective paradigm for language-model self-evolution, but without guidance, solver performance can plateau or decline across rounds. Unguided methods steer question generation with signals such as difficulty, learnability, or diversity. These signals keep questions challenging and varied but do not specify which unresolved reasoning weaknesses later rounds should target. Guided methods obtain direction from external task resources, including human examples, document corpora, or specified difficulty targets, and therefore rely on task information supplied outside the self-play loop. We show that the needed direction can instead be derived from the solver's own failure history. We introduce DiagEvo, whose diagnostician extracts recurring error causes from this history and stores them in a hierarchical error-cause memory. The memory groups related causes under skill nodes and tracks each as Active or Mastered according to self-consistency on targeted questions. The challenger uses these states and recurrence counts to balance cause-targeted generation with free exploration. Double-confidence filtering retains intermediate-difficulty questions only when the most common solver answer has a clear vote lead. DiagEvo derives its curriculum from information produced during self-play, without external task resources. With the default 4B diagnostician, DiagEvo outperforms every baseline in mean accuracy across all nine benchmarks for each of the three solvers: Qwen3-4B, Qwen3-8B, and OctoThinker-8B. On Qwen3-8B, it reaches 72.3% mean accuracy across five mathematical reasoning benchmarks, 4.5 percentage points above R-Zero. Its mean accuracy across all nine benchmarks is 57.4%, 1.1 percentage points above DARC. Ablations show that the hierarchical error-cause memory and double-confidence filtering both contribute to these gains.

View arXiv page View PDF Project page Add to collection

Community

dingyii

Paper author Paper submitter about 12 hours ago

Hi everyone! We’re excited to introduce our new work, DiagEvo.
Reasoning self-improvement (RSI) has recently attracted growing attention. One practical question is: How can a model identify its next direction for improvement from its own training process?
In self-play, a Challenger continually generates problems while a Solver attempts to answer them. But difficulty alone is not enough. It tells the system how hard a problem is, but not why the Solver failed. As training continues, generated problems may simply become longer without targeting the Solver’s actual reasoning weaknesses, causing performance to plateau prematurely.
DiagEvo discovers training directions from the Solver’s failure history. It distills recurring, transferable causes of error from these records.
🚀 Key Highlights
✦ Hierarchical Memory of Error Causes
Related error causes are organized under the same skill node. Active indicates that an error still requires direct training, while Mastered means that the Solver has achieved high self-consistency on related problems. If an error reappears, its status changes from Mastered back to Active. The more frequently an error recurs, the more likely it is to be selected for the next training round.
✦ Targeted Training Alongside Free Exploration
The Challenger generates problems around Active error causes while retaining free exploration. It can also combine Active and Mastered causes from the same skill node into a single new problem. This allows the system both to train current weaknesses and to test whether related abilities can be applied together.
✦ Dual-Confidence Filtering
Majority voting is not always reliable. When the two leading answers receive similar vote counts, the pseudo-label can easily change. DiagEvo retains only problems that are moderately difficult and whose top-ranked answer clearly outperforms the runner-up.
✦ No External Task Resources Required
The training curriculum uses no human-written examples, external documents, or externally provided difficulty labels. The default configuration requires only a 4B diagnostic model. Diagnosis and memory management account for just 5.7% of the total runtime.
📊 Main Results
We evaluated DiagEvo on Qwen3-4B, Qwen3-8B, and OctoThinker-8B. Using the default 4B diagnostic model, DiagEvo outperformed all baselines for all three Solvers when averaged across nine benchmarks.
On Qwen3-8B, it achieved a 72.3% average on mathematical reasoning, outperforming R-Zero by 4.5 percentage points. Its overall average across nine benchmarks reached 57.4%, exceeding DARC by 1.1 points.
Although the training curriculum contained only math problems, Qwen3-8B’s average performance on general reasoning benchmarks still improved from 33.2% to 38.8%.
💡 We also found that longer generated problems do not necessarily produce a more effective curriculum.
Under pure free exploration, the average problem length increased from 45.6 words to 168.9 words by Round 5, yet Solver performance plateaued after Round 3. In contrast, DiagEvo stabilized problem length at approximately 73 words after Round 2 and continued improving for five consecutive rounds.
The complete method also increased the agreement between Round 5 pseudo-labels and external reference answers from 65% to 75%.
Version 1 of the paper is now publicly available, and we are currently preparing the code for release. We welcome your interest, feedback, and criticism—and would greatly appreciate your support on Hugging Face and alphaXiv!
Paper: https://arxiv.org/abs/2609.00768
alphaXiv: https://www.alphaxiv.org/abs/2609.00768