Abstract
Coding agents often find correct patches early but continue working, sometimes undoing correct solutions, with termination lag varying by model and benchmark.
Coding agents can solve a task well before they stop working on it. We characterize termination lag--the number of steps between an agent's first correct patch and the end of its run--across five models, two agent frameworks, and three SWE-bench datasets by recording and testing every distinct repository patch generated during execution. Our analysis reveals that 38--82% of successful runs achieve a correct solution on their initial source-code edit. However, agents continue to expend substantial effort after correct patches, with post-solution steps accounting for a median of 35--37% for Qwen3-Coder and 54--60% for Laguna-XS.2 in successful runs on SWE-bench Verified and Multilingual. Agents spend most of these additional steps verifying their work and exploring the repository. In a small fraction of cases (1--8%), this continued activity leads agents to undo their earlier correct patch, leaving the final solution incorrect.
Community
What if coding agents already solved the problem but don't know when to stop?
We studied 5 LLMs, 2 agent frameworks, and 3 SWE-bench datasets, tracking and evaluating intermediate code patches throughout agent execution.
๐ Key findings:
- 38โ82% of runs that reach a correct patch get it right on their first source-code edit.
- A median of 35โ60% of steps in successful runs occur AFTER the correct patch is produced in our core settings.
- Agents spend most of these extra steps verifying and exploring.
- In 1โ8% of cases, agents even break an already-correct solution and finish with an incorrect patch.
These findings suggest an important opportunity: making coding agents recognize when to stop could significantly reduce unnecessary computation.
๐ฌ How should coding agents decide when a solution is good enough to submit?
Github:
https://github.com/musacim/termination-lag
HF:
https://huggingface.co/datasets/musacim/termination-lag-trajectories
Get this paper in your agent:
hf papers read 2611.14960 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper