README.md

---
license: cc-by-4.0
language:
- en
tags:
- computer-use-agents
- agent-benchmark
- benchmark
- evaluation
pretty_name: Agents Last Exam — Task Card Metadata
configs:
- config_name: default
  data_files:
  - split: v1.0
    path: task_cards.parquet
---

# Agents Last Exam — Task Card Metadata (v1.0)

A metadata-only release (v1.0) of 153 tasks from the Agents Last Exam (ALE)
benchmark for evaluating computer-use agents on long-horizon professional work.

## The Agents Last Exam dataset family

ALE is published as three companion HuggingFace datasets:

| Dataset | Contents | Access |
|---|---|---|
| [Task Card Metadata](https://huggingface.co/datasets/agents-last-exam/agents-last-exam) | One row per task: titles, prompts, taxonomy, input-file descriptors | Open |
| [Task Input Data](https://huggingface.co/datasets/agents-last-exam/agents-last-exam-data) | The `input/` files each task hands the agent at run start | Open |
| [Reference (Ground-Truth) Data](https://huggingface.co/datasets/agents-last-exam/agents-last-exam-reference) | The `reference/` outputs used to score runs | ⚠️ Gated (manual approval) |

**You are viewing: Task Card Metadata.**

- **Source repository:** https://github.com/rdi-berkeley/agents-last-exam
- **Official site:** [https://agents-last-exam.org](https://agents-last-exam.org)


This dataset contains **task cards only** (one row per task). No input data,
software binaries, reference outputs, scoring fixtures, or VM images are
included here. Use the HuggingFace Dataset Viewer above to browse, filter,
and search the table; fetch the input files from the companion data repo and
the ground-truth from the gated reference repo.

## What's Included

| Column | Type | Notes |
|---|---|---|
| `task_id` | string | Stable identifier, `<category>/<task_name>`. |
| `title` | string | Human-readable task title. |
| `summary` | string | One-paragraph summary. |
| `category` | string | Top-level domain (e.g. `business_finance`, `health_medicine`). |
| `subdomain` | string | Subdomain name (e.g. `Quantitative Finance & Trading`). |
| `task_split` | string | One of `near-term`, `last-exam`, `full-spectrum`. |
| `task_prompt` | string | Full task instruction the agent receives. |
| `agent_must_do` | array<string> | Required action checklist. |
| `software` | array<string> | Software the task expects the VM to provide. |
| `input_files` | array<struct> | Descriptors for staged input files: `name`, `format`, `path`, `description`. No file contents. |
| `taxonomy` | struct | Full taxonomy: `domain_id`, `domain_code`, `subdomain_id`, `subdomain_code`, `subdomain_name`. |
| `source_repo_path` | string | Path inside the source repository for this task. |

## What's Excluded

- Input data, reference outputs, software binaries, VM images.
- Private evaluation logic (`evaluation`), reference file lists (`referenceFiles`), VM provisioning details (`vm`).
- Internal audit trails (the classifier model run, submission IDs, internal file paths).