TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents
Abstract
TRACE is a training-free framework that ranks visual evidence by future utility and diversity, reserves native tokens for spatial coverage, and contracts retired frames to reduce latency and memory in GUI agents.
GUI agents accumulate high-resolution screenshots as the trajectory unfolds, increasing inference latency and memory usage. Training-free visual token pruning can reduce this cost, but cache reuse introduces a fundamental constraint. Once tokens are discarded, the corresponding visual evidence cannot be recovered without re-encoding. Pruning therefore becomes an irreversible admission decision that must remain useful for unknown future targets while preserving coverage of operable regions under tight budgets. To address these challenges, we propose \method{}, a training-free framework for \textbf{Trajectory-robust Admission and Coverage-aware Evidence ordering}. Specifically, we combine a query-independent layout-derived interaction prior with instruction relevance and feature novelty to rank visual evidence according to both potential future utility and diversity. Then, we reserve part of the budget for native visual tokens distributed across the screen, repairing missing spatial coverage without breaking the ordering. Together, these mechanisms produce a nested token order, allowing retained visual evidence to shrink monotonically across budgets while remaining reusable throughout the trajectory. Finally, our monotone KV contraction incrementally contracts retired frames into compact session state, avoiding repeated visual encoding or pruning. Extensive experiments across six GUI benchmarks and diverse models verify the effectiveness of our proposed under tight budgets. The source code will be released.
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding (2026)
- VisCache: Visual KV Cache Pruning for Efficient Vision Large Language Model Inference (2026)
- SimpleMemVLA: A Simple but Effective Native-Video Memory for Vision-Language-Action Models (2026)
- Dual-Latent Memory Routing for Vision-Language Reasoning (2026)
- MEDR: Query-Independent Frame Selection via Multi-Signal Event Modeling and Dynamic Rescoring (2026)
- Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization (2026)
- Concentrate After Imagination: Text-Conditioned Evidence Grounding for Partially Relevant Video Retrieval (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2609.10297 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper



