Self-Evaluating Recursive Agents
Abstract
SERA trains recursive language agents to generate evaluation rubrics for subtasks, improving decomposition, solving, and self-evaluation with reduced external supervision.
Recursive language-model agents decompose tasks and delegate subtasks to child instances of the same policy, forming a tree of work. Training them, however, is hard: the final outcome is verifiable, but the self-invented intermediate subtasks are numerous and carry no ground truth. Existing methods score each node with a verifier or judge, which is costly at scale and blind to decomposition quality. We argue that a recursive agent must learn three coupled capabilities within one set of weights: decomposing problems into subtasks, solving them, and evaluating the outcomes, each requiring its own training signal. SERA (Self-Evaluating Recursive Agents) turns evaluation into a learned capability of the policy itself. Before delegating, the parent writes a rubric of weighted success criteria for each child subtask; a ranking objective against verified outcomes then trains rubric generation so that the criteria track genuine subtask success. In addition, a complementary leaf-coverage signal provides direct credit for task decomposition. Our central finding is that training the policy to generate aligned rubrics is what drives the gains: because the same weights both evaluate and execute, learning to judge subtasks sharpens the agent's ability to solve them. Notably, external supervision is also reduced: the judge is consulted only to train the rubric generator, while solving is trained against the agent's own rubric scores, which outperform direct use of the judge. Beyond training, the learned rubric doubles as an inference-time selector for tree search. On TextCraft-Synth and TextWorld-Sync, SERA improves over strong recursive-agent baselines by 5.38 and 13.14 points on average, and rubric-guided tree search at inference adds a further 2.43 points on TextWorld-Sync.
Community
🚀 Excited to share our latest work: Self-Evaluating Recursive Agents (SERA)!
As coding agents such as Claude Code and OpenAI Codex increasingly adopt task decomposition and subagent delegation, recursive agent architectures are emerging as a promising paradigm for tackling complex, long-horizon tasks.
However, effectively training recursive agents remains a fundamental challenge. While final task outcomes are often verifiable, the intermediate subtasks generated by agents lack ground-truth supervision, making it difficult to assign credit for subtask execution and even harder to evaluate decomposition quality. Existing approaches typically rely on external verifiers or LLM judges, which can be costly and provide limited supervision for learning effective decomposition.
To address this challenge, we introduce SERA (Self-Evaluating Recursive Agents), a framework that jointly trains a single policy to decompose, execute, and evaluate subtasks.
SERA enables agents to generate task-specific evaluation rubrics before delegation and learns to align these rubrics with verifiable task outcomes, providing finer-grained feedback for subtask execution. We also introduce a Leaf-Coverage Reward to directly improve decomposition quality.
🔍 Our key finding: Learning to Evaluate Improves Solving.
Because execution and evaluation share the same model parameters, training agents to recognize successful subtask outcomes also improves their ability to solve those subtasks. In other words, evaluation is not merely a source of supervision; it can itself become a learned capability that strengthens agent reasoning and execution.
Experimentally, SERA outperforms strong recursive-agent baselines by 5.38 and 13.14 percentage points on TextCraft-Synth and TextWorld-Sync, respectively. Moreover, the learned rubrics can guide tree search at inference time, yielding an additional 2.43-point improvement on TextWorld-Sync.
Our results suggest a broader perspective: rather than relying solely on external judges, we can train agents to evaluate their own work, and in doing so, make them better problem solvers.
Get this paper in your agent:
hf papers read 2610.04902 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper