Add README.md

README.md

@@ -0,0 +1,103 @@
+---
+title: Tiny Computer Agent
+emoji: 🖱️
+colorFrom: indigo
+colorTo: green
+license: mit
+tags:
+  - computer-use
+  - gui-agent
+  - agentic-harness
+  - local-first
+  - tiny-models
+  - offline
+  - marimo
+---
+
+# 🖱️ Tiny Computer Agent
+
+**An agentic harness that drives a real computer with a tiny, fully local model.**
+No cloud API, no API keys, no telemetry — the model is small enough to run on the
+same machine it is controlling.
+
+> **Status: project scaffold.** This card pins down the harness contract
+> (observe → think → act → verify), the intended layout, and the safety model.
+> Implementation lands next; see the [roadmap](#-roadmap).
+
+## The idea, in one loop
+
+```
+   observe          think            act            verify
+  ┌────────┐      ┌─────────┐     ┌─────────┐     ┌─────────┐
+  │ screen │──────▶│  tiny   │─────▶│ mouse / │─────▶│ next    │
+  │ frame  │      │  model  │     │ keyboard│     │ frame   │
+  └────────┘      └─────────┘     └─────────┘     └────┬────┘
+                                                              │
+                          ◀──────────────────────────────────────┘
+```
+
+1. **Observe** — grab the screen, downscale it, optionally attach OCR text or the
+   accessibility tree so the model never has to guess where things are.
+2. **Think** — a tiny (≤ 3B, quantised) vision-language model emits **exactly one
+   action** from a closed vocabulary.
+3. **Act** — the harness executes it with OS-level input (mouse, keyboard, scroll).
+4. **Verify** — the next frame shows whether it worked. Repeat, bounded by a step
+   budget and a kill switch.
+
+## Why "tiny" is the whole point
+
+The harness does the heavy lifting, so the model does not have to be clever:
+
+| Job | Who does it |
+|---|---|
+| Finding *where* a button is | OCR / accessibility tree / element index — not the model |
+| Choosing *what* to do | tiny VLM over a **closed action menu** (click, type, key, scroll, wait, done) |
+| Knowing if it worked | the next screenshot, plus harness-side checks |
+| Recovering from a bad step | retry policy in `loop.py`, not in the weights |
+
+That keeps the agent offline, instant to start, and small enough that a laptop
+GPU (or plain CPU) is enough to run it — on the same box it is controlling.
+
+## Candidate backends
+
+Pluggable behind one interface; nothing here requires a network call at run time.
+
+- **Moondream 2** / **SmolVLM-256M / 500M** — ~hundreds of MB, safe default
+- **Qwen2-VL-2B-Instruct**, **Florence-2-base** — stronger grounding, still tiny
+- **OCR + text-only planner** — no VLM at all; the cheapest credible baseline
+- **mock brain** — deterministic scripted actions, used for harness tests
+
+## Intended layout
+
+```
+tiny-computer-agent/
+├─ README.md
+├─ tiny_computer_agent.py          # marimo notebook: drive/watch the loop
+├─ src/tiny_computer_agent/
+│  ├─ observe.py   # screen capture + OCR / accessibility tree
+│  ├─ brain.py     # model backends behind one interface
+│  ├─ act.py       # action vocabulary + pyautogui / xdotool executor
+│  ├─ loop.py      # observe → think → act → verify
+│  └─ safety.py    # dry-run, allow/deny lists, kill switch, step budget
+└─ LICENSE
+```
+
+## Safety, because this thing has your mouse
+
+- Run it in a **VM, container, or throwaway desktop account**, never your real session.
+- `--dry-run` logs every action without executing it, so you can read the plan first.
+- Deny-list for shells, terminals, payment flows and password managers.
+- Step budget + a global hotkey kill switch before anything touches the pointer.
+- **Local-first, for real:** screen pixels and keystrokes never leave the machine.
+
+## Roadmap
+
+- [ ] `loop.py` + `act.py` with the mock brain — the harness is testable before any model download
+- [ ] OCR / AT-SPI element index, and click-by-element instead of click-by-pixel
+- [ ] tiny-VLM grounding backend (transformers + llama.cpp GGUF)
+- [ ] marimo UI: live screen preview, action log, pause / resume / step
+- [ ] append-only run traces so failed episodes can be replayed and scored
+
+## License
+
+MIT — see [LICENSE](LICENSE).