---
title: Computer Agent v2.0
emoji: 🤖
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: "5.0.0"
app_file: app.py
pinned: false
license: apache-2.0
short_description: "Enhanced universal computer agent with planner, MCP, memory & voice"
---

# 🤖 Open Computer Agent v2.0

An **enhanced** universal computer-use agent built on [smolagents](https://github.com/huggingface/smolagents), [E2B Desktop](https://e2b.dev), and [Playwright](https://playwright.dev).

## What's New in v2.0

| Feature | Description |
|---------|-------------|
| 🧠 **Hierarchical Planner** | Breaks goals into subtasks before execution using a cheap text model |
| 🔌 **Playwright MCP** | Semantic browser control (click by text/role, extract tables/links, evaluate JS) |
| 🎯 **Multi-Model Router** | Auto-selects the cheapest capable model (fast vision ↔ powerful vision ↔ fast text ↔ powerful text) |
| 🧩 **Set-of-Marks Vision** | Overlays numbered bounding boxes on UI elements for coordinate-free interaction |
| 🗄️ **Long-Term Memory** | ChromaDB vector store retrieves similar past tasks and proven strategies |
| 🔍 **Verifier Agent** | Checks subtask completion and triggers recovery loops |
| 🛑 **Human-in-the-Loop** | Pauses on sensitive actions (payments, emails, deletes) for user approval |
| 🎙️ **Voice I/O** | Speak tasks and hear responses via Whisper STT + Kokoro TTS |
| 💰 **Cost Dashboard** | Real-time $/task, token usage, and latency tracking |
| 📹 **Session Recording** | Saves every step as replayable macros with GIF/MP4 export potential |
| 🧪 **Enhanced Eval** | Built-in benchmark suite with LLM-as-a-Judge grading and A/B testing |

## Architecture

```
User Input (Text / Voice / File)
       |
       v
[Intelligence Router] ----> Planner (JSON DAG)
       |
       v
[Memory Retrieval] (ChromaDB)
       |
       v
[Plan Executor]
       |
       +---> [Browser Sub-Agent] (Playwright MCP)
       +---> [Desktop Sub-Agent] (E2B + SoM Vision)
       +---> [Coder Sub-Agent] (Code Interpreter)
       +---> [HF Hub Sub-Agent] (Search / Upload)
       |
       v
[Verifier] -> Retry / Alternative / Continue
       |
       v
[Macro Saver] + Cost Report + Session Recording
```

## Quick Start

1. Set your **HF_TOKEN** and **E2B_API_KEY** in the Space Secrets.
2. Type a task (or speak it) and hit **🚀 Let's go!**.
3. Watch the agent plan, execute, verify, and report costs.

## Sensitive Actions

By default, the agent pauses before:
- Payments, purchases, subscriptions
- Sending emails/messages/posts
- Deleting files or uninstalling software
- Password/credit-card fields

Enable **Auto-approve all actions** in Advanced Options to disable HITL.

## Cost Budget

Default budget is **$2.00 USD per session**. The router automatically downgrades to cheaper models as the budget is consumed.

## Benchmarks

Run the built-in eval suite:
```python
from eval_harness import EvaluationHarness
# See eval_harness.py for usage
```

## Credits

- [smolagents](https://github.com/huggingface/smolagents) by Hugging Face
- [E2B](https://e2b.dev) for secure sandboxed desktops
- [Playwright](https://playwright.dev) for browser automation
- [Qwen2.5-VL](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct) for vision reasoning