Papers
arxiv:2611.14960

Termination Lag: Coding Agents Solve Early but Stop Late

Published on Nov 11
ยท Submitted by
musa cim
on Nov 17
Authors:
,
,

Abstract

Coding agents often find correct patches early but continue working, sometimes undoing correct solutions, with termination lag varying by model and benchmark.

Coding agents can solve a task well before they stop working on it. We characterize termination lag--the number of steps between an agent's first correct patch and the end of its run--across five models, two agent frameworks, and three SWE-bench datasets by recording and testing every distinct repository patch generated during execution. Our analysis reveals that 38--82% of successful runs achieve a correct solution on their initial source-code edit. However, agents continue to expend substantial effort after correct patches, with post-solution steps accounting for a median of 35--37% for Qwen3-Coder and 54--60% for Laguna-XS.2 in successful runs on SWE-bench Verified and Multilingual. Agents spend most of these additional steps verifying their work and exploring the repository. In a small fraction of cases (1--8%), this continued activity leads agents to undo their earlier correct patch, leaving the final solution incorrect.

Community

Paper submitter

What if coding agents already solved the problem but don't know when to stop?

We studied 5 LLMs, 2 agent frameworks, and 3 SWE-bench datasets, tracking and evaluating intermediate code patches throughout agent execution.

๐Ÿ” Key findings:

  • 38โ€“82% of runs that reach a correct patch get it right on their first source-code edit.
  • A median of 35โ€“60% of steps in successful runs occur AFTER the correct patch is produced in our core settings.
  • Agents spend most of these extra steps verifying and exploring.
  • In 1โ€“8% of cases, agents even break an already-correct solution and finish with an incorrect patch.

These findings suggest an important opportunity: making coding agents recognize when to stop could significantly reduce unnecessary computation.

๐Ÿ’ฌ How should coding agents decide when a solution is good enough to submit?

Github:
https://github.com/musacim/termination-lag

HF:
https://huggingface.co/datasets/musacim/termination-lag-trajectories

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.14960
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.14960 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.14960 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.14960 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.