README.md

---
title: SiriBench — iOS Agent Tasks & Trajectories
emoji: 🧠
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.19.0
app_file: app.py
python_version: "3.10"
pinned: false
license: mit
---

# SiriBench — creating iOS tasks & capturing the trajectory

A demo of how we **author tasks for an on-device iOS agent** and **capture each run as a
verifiable trajectory**.

The agent is **Apple's on-device Foundation Model** (the "Siri model", via the `FoundationModels`
framework) driving the **real** iOS system apps — Reminders, Calendar, Contacts, Messages — on the
iOS 26.4 Simulator. It is given the **same 11 tools** for every task and must choose the right
one(s) itself (neutral instructions, no hardcoding).

**This Space does not run the model.** Apple's on-device model runs only on a Mac + iOS Simulator,
so this app replays the **recorded artifacts** each run produced:

- `run.mp4` — screen recording of the simulator while the task ran
- `trajectory.txt` — the model's turn-grouped **INPUT → OUTPUT** transcript
- `trajectory.jsonl` — one JSON event per line

For each task you can see **how it was created** (seeded iOS world + prompt + pass criteria) and
**what was captured** (the run + transcript + PASS/FAIL verdict).

## Run locally
```bash
pip install -r requirements.txt
python app.py
```

## Deploy to Hugging Face Spaces
1. Create a new **Gradio** Space.
2. Add `app.py`, `requirements.txt`, this `README.md`.
3. Add the **`trajectories/`** folder (the per-task `run.mp4` + `trajectory.txt` + `trajectory.jsonl`).
   Track the `.mp4` files with **Git LFS** (`git lfs track "*.mp4"`).

## Scope
Tasks run on the **iOS 26.4** text-only on-device model. **Visual tasks** (image QA, receipt
parsing, image editing) require **image input to the model — an iOS 27 / macOS 27 capability** —
and unblock once the host Mac is on macOS 27.