Note Paper: https://arxiv.org/abs/2604.08523 · code: https://github.com/TIGER-AI-Lab/ClawBench · project: https://claw-bench.com/

Note V1/V2 tasks and rubrics · repo: https://github.com/TIGER-AI-Lab/ClawBench

Note Full V1 execution traces for regrading and analysis.

Note Full V2 execution traces for regrading and analysis.

Note GAIA: general AI assistant benchmark with real-world questions.

Note TheAgentCompany: consequential real-world tasks for LLM agents.

Note WebArena: realistic websites for autonomous web agents.

Note OSWorld: open-ended tasks in real computer environments.

Note The BrowserGym Ecosystem for Web Agent Research.

Note Interactive leaderboard: https://huggingface.co/spaces/TIGER-Lab/ClawBench · project: https://claw-bench.com/

Note WorkArena: enterprise knowledge-work tasks for web agents.

Note WorkArena++: compositional planning and reasoning for web-agent tasks.

Note AssistantBench: realistic, time-consuming web-agent tasks.

Note Mind2Web: generalist web-agent benchmark and dataset.

Note Mind2Web 2: long-horizon agentic-search benchmark.

Note WebVoyager: end-to-end multimodal web-agent benchmark.

Note WebShop: grounded language agents on real-world shopping tasks.

Note OSWorld-MCP: benchmark for MCP tool invocation in computer-use agents.

Note BrowseComp: persistent browsing questions requiring hard-to-find web information.

Note BrowseComp-V3: visual, vertical, and verifiable multimodal browsing benchmark.

Note K-BrowseComp: web-browsing benchmark grounded in Korean contexts.

Note AndroidWorld: dynamic benchmark for multimodal mobile agents.

Note AgentWebBench: multi-agent coordination in agentic web settings.

Note BEARCUBS: compact benchmark for computer-using web agents.

Note WebCanvas: web agents in online environments with live evaluation.

Note Online-Mind2Web: live evaluation across 300 realistic tasks and 136 websites.

Note WebLINX: multi-turn conversational website navigation benchmark.

Note ST-WebAgentBench: safety and trustworthiness benchmark for web agents.

Note WebRetriever: comprehensive benchmark for efficient web-agent evaluation.

Note Mobile-Bench: evaluation benchmark for LLM-based mobile agents.

Note SPA-Bench: smartphone-agent benchmark in interactive environments.

Note MobileWorld: agent-user interactive and MCP-augmented mobile benchmark.

Note MobileWorldBench: semantic world-model benchmark for mobile agents.

Note PhoneWorld: scalable phone-use agent environments.

Note AIRS-Bench: end-to-end AI research science benchmark for LLM agents.

Note PaperBench: evaluating agents on reproducing AI research.

Note AgencyBench: broad real-world autonomous-agent benchmark.

Note MLE-bench: machine-learning engineering benchmark for agents.

Note MLGym: framework and benchmark for AI research agents.

Note AppWorld: controllable multi-app environment and benchmark for interactive coding agents. Project: https://appworld.dev/ · code: https://github.com/StonyBrookNLP/appworld

Note Tool-agent-user benchmark with state-based evaluation and pass^k reliability. Paper: https://arxiv.org/abs/2406.12045 · code: https://github.com/sierra-research/tau-bench

Note TUA-Bench evaluates general-purpose terminal-use agents across real-world, scientific, and engineering workflows. Paper: https://arxiv.org/abs/2606.28480

Note Computer-use agent robustness benchmark under realistic environment corruptions. Paper: https://arxiv.org/abs/2605.25707 · project: https://AgentHijack.github.io

Note Long-horizon real-world computer-use workflows with authentic artifacts and stateful tasks. Paper: https://arxiv.org/abs/2606.29537

Note Dynamic desktop GUI benchmark with varied initial states across 10 applications. Paper: https://arxiv.org/abs/2502.08047 · code: https://github.com/showlab/WorldGUI

Note Multimodal desktop GUI-navigation benchmark with multi-application tasks and automated validation. Paper: https://arxiv.org/abs/2505.03570 · code: https://github.com/agentsea/osuniverse

Note Web-agent benchmark built from natural crowdsourcing HTML pages, with 32.2K instances across 158 tasks. Paper: https://arxiv.org/abs/2403.11905

Note Diagnostic web-agent benchmark organized around an action taxonomy to analyze failure modes. Paper: https://arxiv.org/abs/2406.01623

Note MCP-Bench evaluates multi-step real-world tasks across 28 MCP servers and 250 tools. Paper: https://arxiv.org/abs/2508.20453 · code: https://github.com/Accenture/mcp-bench

Note Stateful, conversational tool-use benchmark with implicit dependencies and milestone-based evaluation. Paper: https://arxiv.org/abs/2408.04682 · code: https://github.com/apple/ToolSandbox

Note Large-scale MCP tool-use benchmark spanning thousands of servers and multi-step calls. Paper: https://arxiv.org/abs/2508.07575

Note Multi-environment agent benchmark spanning web browsing, web shopping, operating systems, databases, and other interactive settings. Paper: https://arxiv.org/abs/2308.03688 · code: https://github.com/THUDM/AgentBench

Note Tool-augmented LLM benchmark covering API invocation, tool selection, and execution. Paper: https://arxiv.org/abs/2304.08244

Note Live, contamination-resistant benchmark of issue resolution on recently created GitHub repositories. Paper: https://arxiv.org/abs/2505.23419 · project: https://www.swebench.com/

Note Code-agent benchmark for solving real-world tasks through repository-level leverage. Paper: https://arxiv.org/abs/2508.18993 · code: https://github.com/QuantaAlpha/GitTaskBench

Note Updated τ-bench implementation for tool-agent-user interaction and reliability evaluation. Paper: https://arxiv.org/abs/2603.04370 · code: https://github.com/sierra-research/tau2-bench

Note Professional high-resolution GUI grounding benchmark covering 23 applications, five industries, and three operating systems. Paper: https://arxiv.org/abs/2504.07981 · leaderboard: https://gui-agent.github.io/grounding-leaderboard

Note Mobile GUI benchmark that evaluates both final state and intermediate process information across 200+ tasks. Paper: https://arxiv.org/abs/2511.09157

Note Multimodal web-agent benchmark extending WebArena with visual observations and tasks. Paper: https://arxiv.org/abs/2401.13649 · code: https://github.com/web-arena-x/visualwebarena

Note Interactive web-interface benchmark with 100+ browser environments and programmatic rewards. Paper: https://arxiv.org/abs/1802.08802 · code: https://github.com/Farama-Foundation/miniwob-plusplus

Note Dynamic benchmark for prompt-injection attacks and defenses in tool-using agents. Paper: https://arxiv.org/abs/2406.13352 · code: https://github.com/ethz-spylab/agentdojo

Note Comprehensive safety benchmark for LLM agents across interactive environments and risk categories. Paper: https://arxiv.org/abs/2412.14470 · code: https://github.com/thu-coai/Agent-SafetyBench

Note LM-emulated sandbox benchmark for identifying risks in tool-using agents, covering 36 toolkits and 144 tests. Paper: https://arxiv.org/abs/2309.15817 · code: https://github.com/ryoungj/ToolEmu

Note Multi-turn tool-using agent safety benchmark with taxonomy of risks and defenses. Paper: https://arxiv.org/abs/2602.13379 · code: https://github.com/CHATS-lab/ToolShield

Note Machine-learning experimentation benchmark for evaluating language agents on realistic ML workflows. Paper: https://arxiv.org/abs/2310.03302

Note Multi-agent benchmark evaluating collaboration and competition dynamics across interactive tasks. Paper: https://arxiv.org/abs/2503.01935

Note Multimodal desktop-and-web benchmark measuring executable program generation for computer tasks. Paper: https://arxiv.org/abs/2402.17553

Note Large-scale Android device-control dataset with natural-language instructions, screenshots, and action traces. Paper: https://arxiv.org/abs/2307.10088 · code: https://github.com/google-research/google-research/tree/master/android_in_the_wild

Note Reproducible Android-agent benchmark with 138 tasks across nine apps and predefined virtual devices. Paper: https://arxiv.org/abs/2410.24024 · code: https://github.com/THUDM/Android-Lab

Note HF dataset release of WebArena-Verified, a curated and reproducibly evaluated WebArena task set. Dataset: https://huggingface.co/datasets/AmineHA/WebArena-Verified · code: https://github.com/ServiceNow/webarena-verified

Note Reproducible web benchmark with 532 tedious, memory-heavy, calculation, and long-term memory tasks. Paper: https://arxiv.org/abs/2506.01952 · project: https://webchorearena.github.io/

Note Hermetic benchmark suite of 50+ interactive browser challenges with verifiable ground-truth solutions. Paper: https://arxiv.org/abs/2502.18356 · project: https://webgames.convergence.ai

Note Interactive text environment benchmark for grounded scientific reasoning and multi-step experimentation. Paper: https://arxiv.org/abs/2203.07540 · code: https://github.com/allenai/ScienceWorld

Note Interactive coding benchmark with execution feedback across Bash, SQL, Python, and other environments. Paper: https://arxiv.org/abs/2306.14898 · project: https://intercode-benchmark.github.io

Note Process-level web-agent benchmark with 1,800 instances and automatic semantic state tracking. Paper: https://arxiv.org/abs/2606.15673 · project: https://jiwanchung.github.io/webstep/

Note Benchmark for data-driven scientific agents with expert analyses across 12 datasets and research questions. Paper: https://arxiv.org/abs/2408.09667

Note Science-agent benchmark with 102 expert-validated tasks drawn from 44 peer-reviewed publications. Paper: https://arxiv.org/abs/2410.05080

Note Realistic data-science-agent benchmark covering data analysis tasks and relative performance gaps. Paper: https://arxiv.org/abs/2409.07703

Note Scientific research-agent benchmark covering literature review, experiment reproduction, data analysis, and research planning. Paper: https://arxiv.org/abs/2510.21652

Note Comprehensive benchmark for data agents with realistic tasks, fine-grained ground truth, and skill coverage across data workflows. Paper: https://arxiv.org/abs/2607.01647

Note Web-agent safety benchmark with 250 safe and 250 harmful tasks across four websites. Paper: https://arxiv.org/abs/2503.04957 · project: https://safearena.github.io

Note Enterprise benchmark distilled from real workplace agent sessions into 852 reproducible tasks with artifacts and rubrics. Paper: https://arxiv.org/abs/2606.23654 · code: https://github.com/FrontisAI/EnterpriseClawBench

Note General Tool Agent benchmark with 229 human-written multimodal tasks and executable tool chains. Paper: https://arxiv.org/abs/2407.08713 · code: https://github.com/open-compass/GTA

Note Hierarchical tool-agent benchmark spanning atomic tool use and long-horizon deliverable workflows. Paper: https://arxiv.org/abs/2604.15715 · code: https://github.com/open-compass/GTA

Note Benchmark isolating model-versus-harness effects across realistic agent workflows and shared task environments. Paper: https://arxiv.org/abs/2605.27922 · project: https://www.harness-bench.ai/

Note Long-horizon benchmark running CLI agent harnesses with real tools in reproducible Docker tasks. Paper: https://arxiv.org/abs/2605.10912

Note Benchmark and flexible harness for multi-turn agents on deterministic simulations of real-world websites. Paper: https://arxiv.org/abs/2504.11543

Note Professional post-production workflow benchmark with 100 agentic tasks from 20 industry experts; analyzes harness effects and deliverable quality. Paper: https://arxiv.org/abs/2605.27705

Note Open-ended machine-learning research-agent benchmark with 201 tasks, rubric-based judging, and modular agent scaffold. Paper: https://arxiv.org/abs/2505.19955

Note Scalable Windows desktop-agent benchmark with 150+ diverse OS tasks and reproducible parallel evaluation. Paper: https://arxiv.org/abs/2409.08264 · code: https://github.com/microsoft/WindowsAgentArena

Note Cross-environment benchmark for multimodal agents across websites, desktop computers, and mobile phones. Paper: https://arxiv.org/abs/2407.01511

Note Scientific-literature benchmark requiring cross-paper synthesis and multi-tool orchestration. Paper: https://arxiv.org/abs/2510.10909 · project: https://paperarena-ai.github.io/

Note Real-world MCP tool-use benchmark with simulated tools, distractor selection, multi-step tasks, and execution-efficiency metrics. Paper: https://arxiv.org/abs/2512.24565

Note Trajectory-aware tool-use benchmark reporting intermediate tool selection and execution metrics beyond final accuracy. Paper: https://arxiv.org/abs/2510.04550 · dataset: https://huggingface.co/datasets/bigboss24/TRAJECT-Bench

Note Holistic offline benchmark decomposing agent capabilities into understanding, reasoning, planning, problem-solving, and self-correction. Paper: https://arxiv.org/abs/2407.18961 · project: https://machinelearning.apple.com/research/mmau

Note Holistic agent evaluation leaderboard with large-scale logs, cost-aware comparisons, and behavior analysis. Paper: https://arxiv.org/abs/2510.11977 · project: https://hal.cs.princeton.edu/

Note Diagnostic planning benchmark with multimodal cases, feedback-conditioned planning, and robustness to broken or extraneous tools. Paper: https://arxiv.org/abs/2606.04874

Note Mobile-agent framework releasing the MobiFlow benchmarking suite for real-world mobile scenarios. Paper: https://arxiv.org/abs/2509.00531

Note Verifiable, highly parallel mobile GUI-agent simulation platform with deterministic state-diff evaluation. Paper: https://arxiv.org/abs/2605.26114 · code: https://github.com/Purewhiter/mobilegym

Note Omni-modal smartphone GUI benchmark combining audio, video, and image inputs. Paper: https://arxiv.org/abs/2605.18758 · project: https://omni-gui.github.io

Note Memory-centric mobile GUI benchmark with 128 tasks across 26 apps and cross-session evaluation. Paper: https://arxiv.org/abs/2602.06075 · project: https://lgy0404.github.io/MemGUI-Bench/

Note Hierarchical multi-platform GUI-agent benchmark across Windows, macOS, Linux, iOS, Android, and Web. Paper: https://arxiv.org/abs/2507.19478

Note Paper: https://arxiv.org/abs/2605.18652 · introduces MementoGUI-Bench for long-horizon GUI-agent decision-making · project: https://zzzmyyzeng.github.io/MementoGUI

Note Paper: https://arxiv.org/abs/2604.25380 · introduces DynamicGUIBench for high-dynamic GUI environments

Note Paper: https://arxiv.org/abs/2606.06560 · MacArena: 421 manually verified macOS computer-use tasks across 50 applications.

Note Paper: https://arxiv.org/abs/2606.09426 · project: https://weavebench.github.io/ · code: https://github.com/weavebench/WeaveBench · long-horizon GUI+CLI benchmark.

Note Paper: https://arxiv.org/abs/2605.22535 · project: https://terminalworld.ai/ · code: https://github.com/EuniAI/TerminalWorld · real-world terminal-agent benchmark.

Note Paper: https://arxiv.org/abs/2504.10445 · project: https://scai.cs.jhu.edu/projects/RealWebAssist/ · long-horizon web assistance benchmark.

Note Paper: https://arxiv.org/abs/2603.25864 · project: https://guide-bench.github.io/ · GUIDE evaluates GUI behavior understanding, intent prediction, and assistance.

Note Paper: https://arxiv.org/abs/2603.08013 · PIRA-Bench evaluates proactive intent recommendation from continuous GUI observations.

Note Paper: https://arxiv.org/abs/2606.16748 · project: https://mypcbench.com/ · code: https://github.com/ljang0/MyPCBench · benchmark for personalized computer-use agents.

Note Paper: https://arxiv.org/abs/2505.05026 · G-FOCUS studies robust assessment of UI design persuasiveness and the WiserUI-Bench evaluation setting.

Note Paper: https://arxiv.org/abs/2503.15661 · UI-Vision is a desktop-centric GUI benchmark for contextual visual interaction understanding.

Note Paper: https://arxiv.org/abs/2606.29445 · project: https://vg-gui-tasker.github.io/ · code: https://github.com/VG-GUI-TASKER/VG-GUI-TASKER · VG-GUIBench evaluates video-guided long-horizon GUI tasks.

Note Paper: https://arxiv.org/abs/2602.14337 · LongCLI-Bench evaluates long-horizon agentic programming workflows in command-line environments.

Note Paper: https://arxiv.org/abs/2607.08964 · Long-Horizon-Terminal-Bench evaluates 46 long-horizon terminal tasks across nine categories.

Note Paper: https://arxiv.org/abs/2605.12501 · Microsoft-led benchmark and data-synthesis pipeline covering human action space for computer use.

Note Paper: https://arxiv.org/abs/2604.02947 · AgentHazard benchmarks harmful behavior and robustness of computer-use agents.

Note Paper: https://arxiv.org/abs/2602.03255 · code: https://github.com/tychenn/LPS-Bench · LPS-Bench evaluates long-horizon planning-time safety awareness.

Note Paper: https://arxiv.org/abs/2510.06607 · AdvCUA benchmarks real-world security threats of computer-use agents in multi-host environments.

Note Paper: https://arxiv.org/abs/2604.18847 · BackBench evaluates recovery from harmful states in computer-use tasks.

Note Paper: https://arxiv.org/abs/2606.24551 · matched 440-task benchmark comparing GUI and CLI execution layers for computer-use agents.

Note Paper: https://arxiv.org/abs/2510.02418 · BrowserArena evaluates web agents on live real-world navigation tasks with human step-level feedback.

Note Paper: https://arxiv.org/abs/2506.15677 · project: https://embodied-web-agent.github.io/ · Embodied Web Agents Benchmark spans coordinated physical and digital web tasks.

Note Paper: https://arxiv.org/abs/2505.15277 · WebRewardBench evaluates process reward models for web-agent trajectory assessment.

Note Paper: https://arxiv.org/abs/2509.06477 · project: https://pengxiang-zhao.github.io/MAS-Bench · MAS-Bench evaluates shortcut-augmented hybrid mobile GUI agents.

Note Paper: https://arxiv.org/abs/2504.18575 · code: https://github.com/facebookresearch/wasp · WASP benchmarks end-to-end web-agent security against prompt injection.

Note Paper: https://arxiv.org/abs/2604.06182 · code: https://github.com/inclusionAI/UI-Venus/tree/VenusBench-Mobile · VenusBench-Mobile evaluates user-centric mobile GUI agents under realistic environment variation.

Note Paper: https://arxiv.org/abs/2510.01354 · code: https://github.com/Norrrrrrr-lyn/WAInjectBench · WAInjectBench evaluates prompt-injection detection for web agents.

Note Paper: https://arxiv.org/abs/2511.20597 · BrowseSafe benchmarks prompt-injection risks and defenses for AI browser agents.

Note Paper: https://arxiv.org/abs/2605.25160 · SimuWoB benchmarks mobile GUI agents in high-fidelity synthetic app environments with automatic rewards.

Note Paper: https://arxiv.org/abs/2603.15039 · GUI-CEval evaluates mobile GUI agents across perception, planning, reflection, execution, and evaluation.

Note Paper: https://arxiv.org/abs/2604.09574 · Turing Test on Screen introduces the Agent Humanization Benchmark for mobile GUI interaction behavior.

Note Paper: https://arxiv.org/abs/2507.14293 · WebGuard provides a dataset and evaluation framework for web-agent action-risk guardrails.

Note Paper: https://arxiv.org/abs/2603.04949 · TimeWarp evaluates web-agent robustness across historical UI and layout versions.

Note Paper: https://arxiv.org/abs/2501.01149 · project: https://yuxiangchai.github.io/Android-Agent-Arena/ · A3 evaluates mobile GUI agents on 21 apps and real-world tasks.

Note Paper: https://arxiv.org/abs/2604.16385 · StressWeb diagnoses web-agent robustness under realistic interaction variability.

Note Paper: https://arxiv.org/abs/2510.20333 · GhostEI-Bench evaluates mobile-agent resilience to adversarial environmental UI injections.

Note Paper: https://arxiv.org/abs/2511.07413 · project: https://facebookresearch.github.io/DigiData/ · DigiData-Bench evaluates general-purpose mobile control agents on complex real-world tasks.

Note Paper: https://arxiv.org/abs/2604.24441 · AutoGUI-v2 evaluates GUI functionality understanding and interaction outcome prediction across six operating systems.

Note Paper: https://arxiv.org/abs/2603.05295 · dataset: https://huggingface.co/datasets/webagentlab/WebChain · WebChain and WebChainBench provide large-scale human-annotated real-world web interaction trajectories.

Note Paper: https://arxiv.org/abs/2603.22529 · project: https://ego2web.github.io/ · Ego2Web benchmarks web agents grounded in egocentric video and online task execution.

Note Paper: https://arxiv.org/abs/2603.25226 · WebTestBench evaluates computer-use agents on end-to-end automated web testing workflows.

Note Paper: https://arxiv.org/abs/2605.29447 · GUI-RobustEval benchmarks error awareness and recovery across policy-induced GUI failures.

Note Paper: https://arxiv.org/abs/2606.11042 · Workflow-GYM evaluates long-horizon computer-use workflows in real-world professional software environments.

Note Paper: https://arxiv.org/abs/2604.27776 · code: https://github.com/HITsz-TMG/WindowsWorld · WindowsWorld benchmarks process-aware cross-application professional GUI workflows.

Note Paper: https://arxiv.org/abs/2604.06126 · project: https://cmu-l3.github.io/gym-anything/ · Gym-Anything produces CUA-World, 10K+ tasks across 200+ software environments.

Note Paper: https://arxiv.org/abs/2604.09937 · HealthAdminBench evaluates computer-use agents on healthcare administration workflows.

Note Paper: https://arxiv.org/abs/2606.31179 · HealthAgentBench evaluates agentic healthcare workflows across seven environment categories.

Note CUA-Gym: 32,112 verified computer-use training tuples across 110 environments; code/data/project links: https://arxiv.org/abs/2605.25624

Note TRIP-Bench evaluates long-horizon interactive agents in realistic travel-planning scenarios with multi-turn constraints and tool use; paper: https://arxiv.org/abs/2602.01675

Note OS-Harm: 150 computer-use safety tasks; project/repo: https://github.com/tml-epfl/os-harm · paper: https://arxiv.org/abs/2506.14866

Note DashboardQA benchmarks GUI agents on interactive dashboards; repo: https://github.com/vis-nlp/DashboardQA · paper: https://arxiv.org/abs/2508.17398

Note General AgentBench is a unified benchmark spanning search, coding, reasoning, and tool use; repo: https://github.com/cxcscmu/General-AgentBench · paper: https://arxiv.org/abs/2602.18998

Note Data Agent Benchmark (DAB) evaluates agents over multi-database enterprise data workflows; repo: https://github.com/ucbepic/DataAgentBench · paper: https://arxiv.org/abs/2603.20576

Note UI-CUBE evaluates enterprise computer-use reliability across 226 UI tasks and workflow scenarios; paper: https://arxiv.org/abs/2511.17131

Note OS-Harm dataset: 150 computer-use safety tasks; source project: https://github.com/tml-epfl/os-harm

Note DashboardQA dataset for interactive dashboard question answering and GUI-agent evaluation; source repo: https://github.com/vis-nlp/DashboardQA

Note General AgentBench dataset spanning search, coding, reasoning, and tool-use tasks; source repo: https://github.com/cxcscmu/General-AgentBench

Note DataAgentBench data for multi-database enterprise data-agent workflows; source repo: https://github.com/ucbepic/DataAgentBench

Note General Agent Evaluation proposes a unified protocol and open leaderboard across six environments; HF paper page: https://huggingface.co/papers/2602.22953

Note MedBrowseComp benchmarks medical deep research and computer use over live domain-specific knowledge bases; project: https://moreirap12.github.io/mbc-browse-app/ · paper: https://arxiv.org/abs/2505.14963

Note Computer Use at the Edge of the Statistical Precipice analyzes evaluation pitfalls and statistical reliability in computer-use benchmarks; paper: https://arxiv.org/abs/2605.08261

Note MacAgentBench evaluates 676 macOS tasks across 25 applications with deterministic and fine-grained scoring; code: https://github.com/JetAstra/MacAgentBench · paper: https://arxiv.org/abs/2606.22557

Note Hardening Agent Benchmarks with Adversarial Hacker-Fixer Loops studies exploitable verifiers across terminal-agent benchmarks; paper: https://arxiv.org/abs/2606.08960

Note WebForge-Bench contains 934 reproducible browser-agent tasks with controlled multidimensional difficulty; code: https://github.com/yuandaxia2001/WebForge · paper: https://arxiv.org/abs/2604.10988

Note AgentRewardBench evaluates automatic judges for web-agent trajectories using 1,302 expert-reviewed trajectories; project: https://agent-reward-bench.github.io/ · paper: https://arxiv.org/abs/2504.08942

Note CORE-Bench evaluates computational reproducibility agents on 270 tasks based on 90 scientific papers; code: https://github.com/siegelz/core-bench · paper: https://arxiv.org/abs/2409.11363

Note CORE-Bench dataset for computational reproducibility agents; source repo: https://github.com/siegelz/core-bench

Note SafeArena dataset for safety evaluation of autonomous web agents; source project: https://safearena.github.io

Note AssistantBench dataset for realistic, time-consuming web-agent tasks; paper: https://arxiv.org/abs/2407.15711

Note Mind2Web dataset for generalist web-agent tasks and trajectories; benchmark paper: https://arxiv.org/abs/2306.06070

Note OSWorld 2.0 long-horizon desktop task set; benchmark paper: https://arxiv.org/abs/2606.29537

Note WebLINX/BrowserGym resources for browser-agent interaction; project: https://mcgill-nlp.github.io/weblinx/

Note Agentic Benchmark Checklist (ABC) provides rigorous guidelines for task setup and reward design in agentic benchmarks; paper: https://arxiv.org/abs/2507.02825

Note AgentIF benchmarks instruction following in realistic agentic scenarios with 707 human-annotated instructions; dataset: https://huggingface.co/datasets/THU-KEG/AgentIF · paper: https://arxiv.org/abs/2505.16944

Note PACE predicts expensive agentic benchmark performance from a small atomic subset; paper: https://arxiv.org/abs/2607.02032

Note AgentIF dataset: 707 human-annotated instructions for realistic agentic scenarios; paper: https://arxiv.org/abs/2505.16944

Note OS-MAP evaluates 416 computer-use tasks across 15 applications by automation level and generalization scope; code: https://github.com/OS-Copilot/OS-Map · paper: https://arxiv.org/abs/2507.19132

Note CUAHarm benchmarks misuse risks of computer-using agents on 104 expert-written scenarios; code: https://github.com/db-ol/CUAHarm · paper: https://arxiv.org/abs/2508.00935

Note CUAHarm dataset: expert-written computer-use misuse scenarios; paper: https://arxiv.org/abs/2508.00935

Note EconWebArena benchmarks 360 multimodal economic web tasks across 82 authoritative websites; dataset: https://huggingface.co/datasets/EconWebArena/EconWebArena · paper: https://arxiv.org/abs/2506.08136

Note WebArXiv provides 275 static, time-invariant web-agent tasks with deterministic ground truths; paper: https://arxiv.org/abs/2507.00938

Note EconWebArena dataset: 360 curated economic web-agent tasks; paper: https://arxiv.org/abs/2506.08136

Note WARC-Bench provides 438 sandboxed web-archive GUI subtasks with deterministic rewards; ICLR 2026 paper: https://arxiv.org/abs/2510.09872

Note Workspace-Bench evaluates 388 tasks over large-scale file dependencies, with 20,476 files and 7,399 rubrics; paper: https://arxiv.org/abs/2605.03596

Note DeskCraft benchmarks 538 professional desktop tasks with proactive human-agent interaction; code: https://github.com/mrwwk/DeskCraft · paper: https://arxiv.org/abs/2606.03103

Note ProSoftArena evaluates 436 professional-software GUI tasks across 13 applications; project: https://prosoftarena.github.io · paper: https://arxiv.org/abs/2601.02399

Note Workspace-Bench full task dataset; paper: https://arxiv.org/abs/2605.03596

Note Workspace-Bench-Lite 100-task subset for lower-cost workspace-agent evaluation; paper: https://arxiv.org/abs/2605.03596

Note Workspace-Bench workspace fixtures and file-dependency environments; paper: https://arxiv.org/abs/2605.03596

Note ClawsBench (distinct from ClawBench) evaluates productivity-agent capability and safety in simulated multi-service workspaces; paper: https://arxiv.org/abs/2604.05172

Note ClawsBench dataset with agent traces for simulated productivity workflows; paper: https://arxiv.org/abs/2604.05172

Note Claw-Eval-Live is a live workflow-agent benchmark with refreshable demand signals and verifiable execution evidence; project: https://claw-eval-live.github.io · paper: https://arxiv.org/abs/2604.28139

Note Claw-Eval-Live release snapshot with 105 workflow tasks, mock services, fixtures, and graders; paper: https://arxiv.org/abs/2604.28139

Note EnterpriseBench evaluates 500 enterprise tasks spanning software engineering, HR, finance, and administration; paper: https://arxiv.org/abs/2510.27287

Note EnterpriseBench CoreCraft provides a high-fidelity customer-support simulation with 2,500 entities and 23 tools for agent training/evaluation; paper: https://arxiv.org/abs/2602.16179

Note EXP-Bench evaluates 461 end-to-end AI research experiment tasks sourced from 51 top-tier papers; code: https://github.com/Just-Curieous/Curie/tree/main/benchmark/exp_bench

Note FDABench evaluates 2,007 data-agent analytical tasks over heterogeneous sources; paper: https://arxiv.org/abs/2509.02473

Note AgencyBench dataset for long-context, long-horizon autonomous-agent evaluation; paper: https://arxiv.org/abs/2601.11044

Note FDABench full dataset for heterogeneous data-agent analytical tasks; paper: https://arxiv.org/abs/2509.02473

Note DeepResearch Bench evaluates 100 PhD-level research tasks across 22 fields with retrieval and report-quality metrics; code: https://github.com/Ayanami0730/deep_research_bench

Note EvoAgentBench evaluates procedural self-evolution and ability transfer across web research, reasoning, software engineering, and knowledge-work domains; dataset: https://huggingface.co/datasets/EverMind-AI/EvoAgentBench

Note AgentSearchBench evaluates execution-grounded retrieval and ranking of nearly 10,000 real-world AI agents; code: https://github.com/Bingo-W/AgentSearchBench

Note EvoAgentBench dataset for ability-guided agent self-evolution evaluation; paper: https://arxiv.org/abs/2607.05202

Note AgentSearchBench task split for execution-grounded agent discovery; paper: https://arxiv.org/abs/2604.22436

Note DeepScholar-Bench evaluates live research synthesis, retrieval quality, citation verifiability, and knowledge synthesis; code: https://github.com/guestrin-lab/deepscholar-bench

Note DeepResearch Bench II evaluates 132 cross-domain research tasks with 9,430 expert-derived fine-grained rubrics; paper: https://arxiv.org/abs/2601.08536

Note Enterprise Deep Research (Salesforce AI Research) provides a steerable multi-agent research system with public code and EDR-200 data; code: https://github.com/SalesforceAIResearch/enterprise-deep-research

Note DRBench evaluates 15 realistic enterprise deep-research tasks spanning public web and private company knowledge; code: https://github.com/ServiceNow/drbench

Note DeepScholar-Bench dataset for related-work synthesis and citation analysis; paper: https://arxiv.org/abs/2508.20033

Note Salesforce Enterprise Deep Research trajectory dataset; paper: https://arxiv.org/abs/2510.17797

Note ServiceNow DRBench dataset for enterprise deep-research evaluation; paper: https://arxiv.org/abs/2510.00172

Note Salesforce AI Research; live web deep-research benchmark with 100 expert-curated tasks and citation-grounded evaluation. Paper: https://arxiv.org/abs/2510.14240 · code: https://github.com/SalesforceAIResearch/LiveResearchBench · dataset: https://huggingface.co/datasets/Salesforce/LiveResearchBench · project: https://livedeepresearch.github.io/

Note Multimodal deep-research benchmark with visual evidence, citation alignment, and 140 expert-crafted tasks. Paper: https://arxiv.org/abs/2601.12346 · code: https://github.com/AIoT-MLSys-Lab/MMDeepResearch-Bench · dataset: https://huggingface.co/datasets/MMDR-2025/MMdeepresearch · project: https://mmdeepresearch-bench.github.io/

Note FINDER deep-research benchmark with 100 human-curated tasks, 419 checklist items, and a failure taxonomy for evidence integration. Paper: https://arxiv.org/abs/2512.01948

Note ResearchArena evaluates end-to-end agentic research workflows with artifact-aware and manuscript-level review. Paper: https://arxiv.org/abs/2605.19156

Note Tencent WorkBuddy Bench: multi-domain coding-agent benchmark spanning code, web, office, and security tasks with contamination-resistant construction. Paper: https://arxiv.org/abs/2607.20911 · dataset: https://huggingface.co/datasets/tencent/workbuddy-bench

Note Agentic-MME evaluates multimodal agent capability through tool usage, process efficiency, and outcome verification. Paper: https://arxiv.org/abs/2604.03016

Note TASTE improves agent-benchmark coverage and difficulty through adaptive tool-sequence generation. Paper: https://arxiv.org/abs/2605.28556

Note DREAM evaluates deep-research agents with agentic, temporally aware, citation-grounded metrics. Paper: https://arxiv.org/abs/2602.18940

Note SGR-Bench evaluates web search agents on state-gated retrieval across 100 expert-curated tasks and 12 public data ecosystems. Paper: https://arxiv.org/abs/2605.22219 · dataset: https://huggingface.co/datasets/PKUAIWeb/SGR-BENCH

Note Official SGR-Bench dataset for state-gated retrieval web-agent evaluation. Paper: https://arxiv.org/abs/2605.22219

Note DeepWeb-Bench public dataset for long-horizon, cross-source deep research evaluation. Paper: https://arxiv.org/abs/2605.21482 · project: https://sixiongxie1001-dot.github.io/deep-research-benchmark2.0/

Note CutVerse evaluates GUI agents on 186 long-horizon professional media post-production tasks across seven applications. Paper: https://arxiv.org/abs/2605.19484

Note π-Bench evaluates proactive personal assistant agents on 100 multi-turn tasks with hidden intents, inter-task dependencies, and cross-session continuity. Paper: https://arxiv.org/abs/2605.14678

Note CLI-Universe synthesizes verifiable terminal-agent tasks with Dockerized environments and strict execution checks. Paper: https://arxiv.org/abs/2606.22883

Note PRDBench evaluates project-level coding agents on 50 real-world Python projects with PRD-based and agent-as-judge criteria. Paper: https://arxiv.org/abs/2510.24358

Note PlanBench-XL evaluates long-horizon planning over 327 retail tasks and 1,665 tools, including missing, failing, and distracting tool conditions. Paper: https://arxiv.org/abs/2606.22388 · dataset: https://huggingface.co/datasets/JiayuJeff/PlanBench-XL

Note ResearchClawBench evaluates end-to-end autonomous scientific research across 40 tasks and 10 scientific domains with multimodal expert rubrics. Paper: https://arxiv.org/abs/2606.07591 · dataset: https://huggingface.co/datasets/InternScience/ResearchClawBench

Note Official ResearchClawBench benchmark dataset for autonomous scientific research evaluation. Paper: https://arxiv.org/abs/2606.07591

Note EvoBrowseComp is an evolving web-search benchmark with 800 contamination-free questions synthesized from live-web traversal. Paper: https://arxiv.org/abs/2606.13120 · dataset: https://huggingface.co/datasets/Krystalan/EvoBrowseComp

Note Official EvoBrowseComp dataset for contamination-resistant, evolving web-search-agent evaluation. Paper: https://arxiv.org/abs/2606.13120

Note DataClawBench evaluates exploratory real-world financial data analysis with 492 cross-domain tasks and native data noise. Paper: https://arxiv.org/abs/2605.02503 · dataset: https://huggingface.co/datasets/GTML-LAB/DataClaw

Note Official DataClaw dataset for exploratory agentic data-analysis evaluation. Paper: https://arxiv.org/abs/2605.02503

Note StateAct, from Salesforce AI Research, evaluates a program-state-first harness for long-horizon computer-use agents and reports results on OSWorld 2.0 and related suites. Paper: https://arxiv.org/abs/2607.22798

Note Claw-SWE-Bench evaluates OpenClaw-style agent harnesses on 350 multilingual software-engineering tasks with standardized adapters and cost accounting. Paper: https://arxiv.org/abs/2606.12344 · code: https://github.com/opensquilla/claw-swe-bench · dataset: https://huggingface.co/datasets/TokenRhythm/Claw-SWE-Bench

Note Official Claw-SWE-Bench dataset for reproducible OpenClaw-style coding-agent harness evaluation. Paper: https://arxiv.org/abs/2606.12344

Note HiViG studies history-aware, visually grounded critics for long-horizon GUI-agent evaluation across web, mobile, and desktop benchmarks. Paper: https://arxiv.org/abs/2606.11078

Note AutoElicit systematically evaluates unsafe unintended behaviors of computer-use agents under benign inputs. Paper: https://arxiv.org/abs/2602.08235 · code: https://github.com/OSU-NLP-Group/AutoElicit

Note AutoElicit benchmark for realistic unintended-behavior testing of computer-use agents. Paper: https://arxiv.org/abs/2602.08235

Note Execution traces for AutoElicit computer-use safety evaluation. Paper: https://arxiv.org/abs/2602.08235

Note CUA-Suite from ServiceNow provides continuous expert demonstrations, UI-Vision benchmark resources, and GroundCUA annotations for desktop computer-use agents. Paper: https://arxiv.org/abs/2603.24440 · dataset: https://huggingface.co/datasets/ServiceNow/VideoCUA · project: https://cua-suite.github.io

Note Official VideoCUA continuous human-demonstration dataset from CUA-Suite. Paper: https://arxiv.org/abs/2603.24440

Note MedCUA-Bench evaluates clinical computer-use agents across 18 scenarios, 10 medical domains, deterministic completion checks, and five clinical-safety dimensions. Paper: https://arxiv.org/abs/2606.03203

Note RedTeamCUA and RTC-Bench evaluate indirect prompt injection in hybrid web-OS computer-use environments. Paper: https://arxiv.org/abs/2505.21936

Note WebPII benchmarks visual PII detection for computer-use agents with 44,865 annotated e-commerce UI images. Paper: https://arxiv.org/abs/2603.17357

Note A11y-CUA characterizes the accessibility gap between blind/low-vision users, sighted users, and computer-use agents across 60 everyday desktop tasks. Paper: https://arxiv.org/abs/2602.09310

Note Full multimodal A11y-CUA dataset with synchronized interaction traces, accessibility trees, screen video, and audio. Paper: https://arxiv.org/abs/2602.09310

Note Structured-data-only A11y-CUA subset for accessibility-focused computer-use research. Paper: https://arxiv.org/abs/2602.09310

Note AgentCIBench evaluates contextual-integrity and cross-application privacy failures in computer-use agents with executable, deterministic scenarios. Paper: https://arxiv.org/abs/2606.23189 · dataset: https://huggingface.co/datasets/UKPLab/agentcibench

Note Official AgentCIBench dataset for contextual-integrity evaluation of computer-use agents. Paper: https://arxiv.org/abs/2606.23189

Note Official AgentHazard benchmark dataset with 2,653 execution-level safety instances. Paper: https://arxiv.org/abs/2604.02947

Note BLIND-ACT evaluates blind goal-directedness in computer-use agents with 90 OSWorld-based tasks covering feasibility, safety, ambiguity, and context. Paper: https://arxiv.org/abs/2510.01670 · Microsoft Research publication: https://www.microsoft.com/en-us/research/publication/just-do-it-computer-use-agents-exhibit-blind-goal-directedness/

Note Paper: https://arxiv.org/abs/2605.17554 · public paper page: https://huggingface.co/papers/2605.17554

Note Paper: https://arxiv.org/abs/2603.14465 · code: https://github.com/RUCBM/AgentProcessBench · project: https://rucbm.github.io/AgentProcessBench-Homepage/

Note Dataset: https://huggingface.co/datasets/LulaCola/AgentProcessBench · paper: https://arxiv.org/abs/2603.14465

Note Dataset: https://huggingface.co/datasets/AIM-Harvard/MedBrowseComp · paper: https://arxiv.org/abs/2505.14963

Note Paper: https://arxiv.org/abs/2601.12369 · code: https://github.com/KongLongGeFDU/TaxoBench

Note Paper: https://arxiv.org/abs/2603.01152 · dataset: https://huggingface.co/datasets/artillerywu/DeepResearch-9K · code: https://github.com/Applied-Machine-Learning-Lab/DeepResearch-R1

Note Dataset: https://huggingface.co/datasets/artillerywu/DeepResearch-9K · paper: https://arxiv.org/abs/2603.01152

Note Paper: https://arxiv.org/abs/2607.14989 · OmniaBench evaluates general agents across diverse application scenarios.

Note Paper: https://arxiv.org/abs/2605.27820 · EgoBench evaluates interactive egocentric multimodal tool-using agents.

Note Dataset: https://huggingface.co/datasets/Halluminate/WebBench · browser-agent benchmark with realistic web workflows.

Note Dataset: https://huggingface.co/datasets/wanlilll/WeaveBench · paper: https://arxiv.org/abs/2606.09426 · project: https://weavebench.github.io/

Note Dataset/container resources associated with SaaS-Bench; paper: https://arxiv.org/abs/2605.15777 · code: https://github.com/UniPat-AI/SaaS-Bench

Note Public dataset for agent security benchmark research: https://huggingface.co/datasets/Lakera/b3-agent-security-benchmark-weak

Note Public MCP agent trajectory benchmark dataset: https://huggingface.co/datasets/obaydata/mcp-agent-trajectory-benchmark

Note Dataset for screen-agent humanization evaluation; related paper: https://arxiv.org/abs/2604.09574

Note APEX-Agents benchmark dataset for deep-research agents: https://huggingface.co/datasets/mercor/apex-agents

Note Terminal-Bench 2.0 benchmark dataset for terminal agents: https://huggingface.co/datasets/harborframework/terminal-bench-2.0

Note ScreenSpot-Pro benchmark dataset for GUI grounding and computer-use agents: https://huggingface.co/datasets/likaixin/ScreenSpot-Pro

Note GAIA benchmark dataset for general AI assistants: https://huggingface.co/datasets/gaia-benchmark/GAIA · benchmark paper: https://arxiv.org/abs/2311.12983

Note OpenAI BrowseComp long-context dataset: https://huggingface.co/datasets/openai/BrowseCompLongContext · benchmark context: https://openai.com/index/browsecomp/

Note BrowseComp dataset mirror for browsing-agent evaluation: https://huggingface.co/datasets/smolagents/browse_comp · benchmark context: https://openai.com/index/browsecomp/

Note APEX-v1 extended benchmark dataset for deep-research agents: https://huggingface.co/datasets/mercor/APEX-v1-extended

Note ServiceNow WorkArena benchmark instances for web agents: https://huggingface.co/datasets/ServiceNow/WorkArena-Instances · paper: https://arxiv.org/abs/2403.07718

Note WebLINX real-world conversational web navigation benchmark: https://huggingface.co/datasets/McGill-NLP/WebLINX-full · paper: https://arxiv.org/abs/2402.05930 · code: https://github.com/mcgill-nlp/weblinx

Note Multimodal Mind2Web dataset for web-agent evaluation: https://huggingface.co/datasets/osunlp/Multimodal-Mind2Web · Mind2Web paper: https://arxiv.org/abs/2306.06070

Note LAB-Bench scientific research agent evaluation dataset: https://huggingface.co/datasets/futurehouse/lab-bench · benchmark paper: https://arxiv.org/abs/2407.10362

Note LABBench2 dataset for biology research agent evaluation: https://huggingface.co/datasets/EdisonScientific/labbench2 · paper: https://arxiv.org/abs/2604.09554

Note Meta FAIR AIRS-Bench dataset for AI research science agents: https://huggingface.co/datasets/facebook/airs-bench · paper: https://arxiv.org/abs/2602.06855 · code: https://github.com/facebookresearch/airs-bench

Note AgentHarm harmfulness benchmark for LLM agents: https://huggingface.co/datasets/ai-safety-institute/AgentHarm · paper: https://arxiv.org/abs/2410.09024

Note ETH SRI AgentBench dataset for agentic software-engineering workflows: https://huggingface.co/datasets/eth-sri/agentbench · code: https://github.com/eth-sri/agentbench

Note WebRetriever large-scale web-agent evaluation dataset: https://huggingface.co/datasets/Mininglamp-2718/WebRetriever · paper: https://arxiv.org/abs/2607.06118

Note Microsoft WebTailBench computer-use benchmark: https://huggingface.co/datasets/microsoft/WebTailBench · paper: https://arxiv.org/abs/2511.19663 · code: https://github.com/microsoft/fara

Note Computer Agent Arena interaction-trajectory dataset: https://huggingface.co/datasets/xlangai/computer-agent-arena · project: https://arena.xlang.ai/

Note StressWeb web-agent robustness benchmark: https://huggingface.co/datasets/HuggingSelf/StressWeb · paper: https://arxiv.org/abs/2604.16385

Note Large-scale desktop computer-use agent trajectory dataset: https://huggingface.co/datasets/xlangai/AgentNet

Note Safety and trustworthiness benchmark for web agents: https://huggingface.co/datasets/ST-WebAgentBench/st-webagentbench · paper: https://arxiv.org/abs/2410.06703

Note DEER benchmark for deep-research expert reports: https://arxiv.org/abs/2512.17776 · LG AI Research

Note DEER deep-research benchmark dataset: https://huggingface.co/datasets/LG-AI-Research/DEER-Deep-Research-Benchmark · paper: https://arxiv.org/abs/2512.17776

Note Community benchmark results for local deep-research agents: https://huggingface.co/datasets/local-deep-research/ldr-benchmarks · code: https://github.com/LearningCircuit/ldr-benchmarks

Note Deep research agent benchmark dataset: https://huggingface.co/datasets/muset-ai/DeepResearch-Bench-Dataset · paper: https://arxiv.org/abs/2506.11763

Note Deep research agent benchmark dataset: https://huggingface.co/datasets/muset-ai/DeepResearch-Bench-II-Dataset · paper: https://arxiv.org/abs/2601.08536

Note ParseBench document-parsing benchmark for AI agents: https://arxiv.org/abs/2604.08538 · code: https://github.com/run-llama/ParseBench

Note ParseBench dataset for agent-facing document parsing: https://huggingface.co/datasets/llamaindex/ParseBench · paper: https://arxiv.org/abs/2604.08538 · code: https://github.com/run-llama/ParseBench

Note OSWorld-Verified computer-use trajectories: https://huggingface.co/datasets/xlangai/ubuntu_osworld_verified_trajs · project: https://xlang.ai/blog/osworld-verified

Note Lite CUA-World computer-use benchmark assets: https://huggingface.co/datasets/cua-lite/Lite.CUAWorld

Note OSWorld-Verified evaluation dataset for computer-use agents: https://huggingface.co/datasets/hud-evals/OSWorld-Verified

Note Hasura Agentic Data Access Benchmark for closed-domain, multi-step data retrieval: https://huggingface.co/datasets/hasura/agentic-data-access-benchmark · code: https://github.com/hasura/agentic-data-access-benchmark

Note Google DeepSearchQA benchmark for difficult multi-step information seeking: https://huggingface.co/datasets/google/deepsearchqa

Note Salesforce LiveResearchBench for user-centric deep research in the wild: https://huggingface.co/datasets/Salesforce/LiveResearchBench · paper: https://arxiv.org/abs/2510.14240 · project: https://livedeepresearch.github.io/

Note AutoResearchBench scientific literature discovery benchmark: https://huggingface.co/datasets/Lk123/AutoResearchBench · code: https://github.com/CherYou/AutoResearchBench

Note AgentWebBench corpus for multi-agent coordination in agentic web: https://huggingface.co/datasets/cx-cmu/AgentWebBench-corpus · paper: https://arxiv.org/abs/2604.10938

Note Persona2Web personalized web-agent benchmark dataset: https://huggingface.co/datasets/yonsei-dli/Persona2Web · paper: https://arxiv.org/abs/2602.17003

Note AgentSynth scalable synthesis pipeline for generalist computer-use agent tasks and trajectories: https://huggingface.co/datasets/sunblaze-ucb/AgentSynth

Note Successful OSWorld computer-use agent trajectories: https://huggingface.co/datasets/markov-ai/computer-use

Note Agents Last Exam task-card metadata: https://huggingface.co/datasets/agents-last-exam/agents-last-exam · source: https://github.com/rdi-berkeley/agents-last-exam · site: https://agents-last-exam.org

Note Agents Last Exam open task-input data: https://huggingface.co/datasets/agents-last-exam/agents-last-exam-data · source: https://github.com/rdi-berkeley/agents-last-exam

Note AJ-Bench evaluates agent-as-a-judge for environment-aware evaluation: https://arxiv.org/abs/2604.18240

Note Persona2Web personalized open-web agent benchmark: https://arxiv.org/abs/2602.17003

Note Microsoft synthetic computer-use trajectories dataset: https://huggingface.co/datasets/microsoft/synthetic-computers-at-scale

Note SATraj-OS computer-use trajectory dataset: https://huggingface.co/datasets/AI45Research/SATraj-OS

Note AIBA benchmark for computer-use/browser-agent atomic-ability and anti-cheat evaluation: https://huggingface.co/datasets/anon-aiba-2026/aiba-benchmark

Note MAD-Bench for deceptive behavior in multimodal computer-use agents: https://huggingface.co/datasets/goldenash/MAD-Bench

Note Dojo-Bench-Mini public computer-use benchmark: https://huggingface.co/datasets/chakra-labs/dojo-bench-mini

Note Claw-Eval benchmark dataset for evaluating agentic systems: https://huggingface.co/datasets/claw-eval/Claw-Eval

Note WildClawBench real-world long-horizon agent evaluation benchmark: https://huggingface.co/datasets/internlm/WildClawBench

Note Mercor ACE benchmark for long-horizon consumer application agent tasks: https://huggingface.co/datasets/mercor/ACE

Note Lite OSWorld computer-use benchmark tasks: https://huggingface.co/datasets/cua-lite/Lite.OSWorld

Note Windows OSWorld computer-use task dataset: https://huggingface.co/datasets/xlangai/windows_osworld

Note macOS OSWorld computer-use task dataset: https://huggingface.co/datasets/xlangai/macos_osworld

Note Ubuntu OSWorld computer-use task dataset: https://huggingface.co/datasets/xlangai/ubuntu_osworld

Note OSWorld-G GUI grounding dataset associated with computer-use evaluation: https://huggingface.co/datasets/MMInstruction/OSWorld-G

Note TESS AgentNet computer-use trajectories for VLA/GUI-agent evaluation: https://huggingface.co/datasets/TESS-Computer/tess-agentnet · source AgentNet: https://huggingface.co/datasets/xlangai/AgentNet

Note VPI-Bench visual prompt-injection benchmark for computer-use agents: https://huggingface.co/datasets/VPI-Bench/vpi-bench · paper: https://arxiv.org/abs/2506.02456

Note DragOn benchmark and dataset for drag-based GUI interactions: https://huggingface.co/datasets/Hcompany/DragOn

Note UI grounding benchmark preview for computer-use agents: https://huggingface.co/datasets/ai-multiple/aim-ui-grounding · methodology: https://research.aimultiple.com/computer-use-agents/

Note MMBench-GUI dataset: https://huggingface.co/datasets/OpenGVLab/MMBench-GUI · paper: https://arxiv.org/abs/2507.19478 · code: https://github.com/open-compass/MMBench-GUI

Note Video-Guided GUI Agent benchmark dataset: https://huggingface.co/datasets/Aoraku/VG-GUI-Bench · code: https://github.com/VG-GUI-TASKER/VG-GUI-TASKER

Note GUI-360 comprehensive computer-use agent dataset and benchmark: https://huggingface.co/datasets/vyokky/GUI-360 · paper: https://arxiv.org/abs/2511.04307

Note GUI Agents Embedding Benchmark Lite: https://huggingface.co/datasets/iMeanAI/GAE-Bench-lite

Note LearnGUI mobile GUI-agent demonstration benchmark: https://huggingface.co/datasets/lgy0404/LearnGUI · paper: https://arxiv.org/abs/2504.13805

Note Official TIGER-Lab browser-agent trajectories and data, directly relevant to real-world web-agent evaluation.

Note Official TIGER-Lab seed tasks/data for browser-agent evaluation and training.

Note Salesforce benchmark data for evaluating agents on CRM workflows.

Note OpenHands community WebArena evaluation outputs for reproducible web-agent comparisons.

Note Computer-vision agent benchmark resources for evaluating visual interaction capabilities.

Note WebArena-X trajectories for long-horizon browser-agent evaluation.

Note PARCS agent benchmark resources for evaluating agent task completion.

Note Out-of-distribution WebArena scenarios for memory-aware web-agent evaluation.

Note GAIA benchmark validation resources for general assistant evaluation.

Note Agent benchmark dataset covering task-oriented AI-agent evaluation.