Card: 2.0 - measured test counts, gallery coverage, watchdog and audit

README.md

@@ -94,7 +94,7 @@ X server. First the suite, headless:
 ```bash
 python -m pip install -e ".[desktop,ocr,dev]"
 pytest tests -q
-# 227 passed, 2 skipped in 18.7s          (that run was 0.3.0; 2.0 has grown this to 284)
+# 227 passed, 2 skipped in 18.7s          (that run was 0.3.0; 2.0 has grown this to 327)
 ```
 
 The two skips are the live-desktop tests. Give them a display and they run for real:
@@ -266,7 +266,7 @@ open problem: it is reliable on a clean screenshot and unreliable on a busy live
   `warning: no text was found on screen, so the index is 96 numbered grid cells and the
   agent is choosing blind` — instead of silently guessing.
 
-**284 tests pass**, including real OCR over a real image and (opt-in) live-desktop tests
+**327 tests collect** - 315 pass, 12 skip for want of a display or an AT-SPI typelib - including real OCR over a real image and (opt-in) live-desktop tests
 that click a real button.
 
 ## Verified on real Windows
@@ -314,7 +314,7 @@ the harness itself, installed into site-packages
 
 None of the three failures in the first row is a harness defect: two are
 `ModuleNotFoundError: numpy` and the third is the guard bug documented below. With the
-OCR engine installed, the same suite exits **0**. (Those Windows counts predate `trace.py` and 2.0; the harness suite has since grown to 284 tests.)
+OCR engine installed, the same suite exits **0**. (Those Windows counts predate `trace.py` and 2.0; the harness suite has since grown to 327 tests.)
 
 **The emulated-OCR caveat.** onnxruntime requires AVX2, and QEMU's software emulation of
 AVX2 is roughly two orders of magnitude slower than native. A suite that finishes in 17
@@ -340,7 +340,7 @@ exactly the kind of thing that only surfaces on a machine without an OCR engine.
 
 ## The test suite, in full
 
-`pytest tests` collects **284 tests**. 282 run headless — no display, no network, no
+`pytest tests` collects **327 tests**. 325 run headless — no display, no network, no
 weights. The last 2 are the opt-in live-desktop tests, skipped unless
 `TINY_COMPUTER_AGENT_LIVE_TEST=1`. On a box without PyGObject, two of the 282 *fail* rather
 than skip: they assert the AT-SPI reader is reachable, and that is the honest outcome.
@@ -356,19 +356,22 @@ than skip: they assert the AT-SPI reader is reachable, and that is the honest ou
 | `tests/test_verify.py` | 7 | success is only claimed when a verifier can actually see it |
 | `tests/test_safety.py` | 11 | deny lists, the kill switch, the blind-grid-click guard |
 | `tests/test_grid_fallback.py` | 11 | the fallback index announces itself instead of guessing |
-| `tests/test_desktop.py` | 10 | display detection, the preflight checks, `doctor` |
-| `tests/test_cli.py` | 16 | the command line: `--doctor`, `--image`, `--live`, and the 2.0 `skills` / `batch` subcommands |
+| `tests/test_desktop.py` | 12 | display detection, the preflight checks, `doctor` |
+| `tests/test_cli.py` | 22 | the command line: `--doctor`, `--image`, `--live`, and the 2.0 `skills` / `batch` subcommands |
 | `tests/test_trace.py` | 28 | traces: a run recorded with no pixels, replayed against another brain, and scored against ground truth |
 | `tests/test_models.py` | 7 | the model backends: lazy loading, device choice, honest refusals |
 | `tests/test_api_surface.py` | 4 | the public API is a promise: everything advertised is importable, and nothing public is left out |
 | `tests/test_decoding.py` | 24 | the mask, the answer space, the retry ladder, the vote - and that an echo is refused |
-| `tests/test_accessibility.py` | 26 | role filtering, the coordinate contract, the fallback chain, and real AT-SPI |
+| `tests/test_accessibility.py` | 27 | role filtering, the coordinate contract, the fallback chain, and real AT-SPI |
 | `tests/test_diff.py` | 8 | what changed between two frames: added, removed, moved, and the verifier that remembers |
 | `tests/test_stepwise.py` | 11 | begin / step / pause / resume, and that it equals the batch loop |
 | `tests/test_ocr_guard.py` | 4 | the OCR guard really skips with no engine - the guard that never fired |
-| `tests/test_gates.py` | 8 | the operator's yes: whitelists, ask-once gates, and a refusal that is a step |
+| `tests/test_gates.py` | 14 | the operator's yes: whitelists, ask-once gates, and a refusal that is a step |
 | `tests/test_skills.py` | 17 | only a verified run becomes a skill; re-grounding by label; aiming at nothing raises |
 | `tests/test_runner.py` | 10 | a queue, a JSONL resume file, and a report counts what was verified |
+| `tests/test_watchdog.py` | 12 | a hung call becomes a named step instead of a stall - and the thread it leaves behind is admitted |
+| `tests/test_audit.py` | 12 | the chain verifies, and editing, deleting or reordering a line is caught by number |
+| `tests/test_atspi_guard.py` | 4 | the AT-SPI guard really skips without the typelib - the same trap as the OCR guard |
 | `tests/test_live_desktop.py` | 2 | **needs a display, opt-in**: OCR finds a real button at the pixels it was drawn at, and a real click makes the app react |
 
 ```bash
@@ -383,7 +386,7 @@ TINY_COMPUTER_AGENT_LIVE_TEST=1 pytest tests -q  # + the 2 live-desktop tests, o
 pip install -e ".[ocr]"              # labels from pixels (RapidOCR: no system binary)
 pip install -e ".[desktop,ocr]"      # + real capture and real input
 pip install -e ".[vision,ocr]"       # + the tiny model brains
-pytest                               # 284 tests, no display or network needed
+pytest                               # 327 tests, no display or network needed
 ```
 
 The accessibility detector needs system packages rather than an extra:
@@ -499,6 +502,8 @@ this: an answer file with non-canonical XML namespaces is **silently ignored** b
 | `skills.py` | **2.0** - Skills: a *verified* run written down once, replayed by `SkillBrain` with no model. Labels, never coordinates |
 | `gates.py` | **2.0** - Approval gates: `AllowOnlyLabels`, `InteractiveGate` (yes / no / always / never), `AllOf` / `AnyOf`. A refusal is a step |
 | `runner.py` | **2.0** - `TaskRunner`: a queue of goals, a JSONL resume file, and a `RunReport` that counts what was *verified* |
+| `watchdog.py` | **2.0** - `Watchdog`: a deadline on `decide` / `execute` / `capture`, so a hung call is a step with a reason, not a stall |
+| `audit.py` | **2.0** - `AuditLog`: an append-only, hash-chained record of a run. Edit, delete or reorder a line and `verify_audit` names it |
 | `showcase.py` | The capability gallery: one real screenshot per capability, checked against the public API |
 | `measure_vlm.py` | Measures the four ways to ask a model, on a real screen, and prints the table |
 | `evaluate_daily.py` | A day's tasks against a real app, verified by the app's own log |
@@ -549,9 +554,10 @@ That click was found **without reading a single pixel**: the window's own status
 reads `last action: Save` afterwards, and the app wrote `Save` to its marker file.
 `--detector best` tries the tree first, then OCR, then the numbered grid.
 
-**Honest limits.** AT-SPI is Linux-only here; **UI Automation is not implemented**, and
-`accessibility_available("uia")` says exactly that rather than pretending - so an agent
-on Windows keeps using OCR. Tk barely publishes a tree and a canvas publishes none,
+**Honest limits.** AT-SPI is Linux-only here. UI Automation *is* implemented - `UiaReader` and
+`UiaDetector`, unit-tested against a synthetic control tree - but it has never run against a
+real Windows desktop, and `accessibility_available("uia")` reports the truth on any machine
+rather than pretending, so an agent on Windows still falls back to OCR. Tk barely publishes a tree and a canvas publishes none,
 which is why this is a detector and not a replacement.
 
 ## Can it do a day's work?
@@ -605,7 +611,7 @@ eleven tasks are a sample, not a survey.
 
 ## Capability gallery
 
-**74 capabilities, one screenshot each - and 173 of the package's 180 pre-2.0 public callables (functions, methods, classes and protocols) appear in at least one of them**, together with 10 of its public constants. The 2.0 additions are not pictured yet: **84 new public callables** (skills, approval gates, the batch runner and the structural diff) plus four new CLI entry points. The counts are read off `gallery/index.json`, which records what each picture shows.
+**74 capabilities, one screenshot each - and 163 of the package's 270 public callables (functions, methods, classes and protocols) appear in at least one of them**, together with 8 of its 26 public constants. **Not yet pictured: 107 callables** - 103 in the 2.0 operator layer (skills 29, the batch runner 20, approval gates 19, the structural diff 14, the audit log 13, the watchdog 8) and four CLI entry points. They are covered by tests rather than by pictures, and `showcase.py` prints the exact list. The counts are read off `gallery/index.json`, which records what each picture shows, and checked against the package by `showcase.api_surface()`.
 
 Nothing here is a mock-up. Each picture is produced by [`showcase.py`](showcase.py): a command or a call is run for real, and its actual output - or the real desktop it acted on - is what you see. The desktop shots are real screen captures on a real X server (Xvfb, 1280x800), including a real mouse click verified by the target application's own reaction.
 
@@ -1233,15 +1239,15 @@ success.
 **Works:** real capture, a real OCR index with coordinates that land on the right pixel,
 a real click an app reacts to, verified success, the closed vocabulary, the policy gates,
 the no-progress guard, echo detection, the doctor, the CLI, portable traces that score a brain's choices offline, skills learned from verified runs,
-a resumable task runner, operator approval gates, 284 tests, a 256M VLM that
+a resumable task runner, operator approval gates, a watchdog that refuses to wait forever rather than stalling silently, a hash-chained audit log, 327 tests, a 256M VLM that
 reads a button label from pixels on a clean screenshot — and the same suite passing inside
 a real Windows Server 2022 guest with no KVM (109 passed, exit 0, with the OCR-inference
 tests excluded as impractically slow under emulation).
 
 **Does not work yet:** reliability on a *live* screen. The 256M VLM echoed the prompt
-there; the 0.5B text planner picks wrong actions; the accessibility-tree detector is not
-implemented, so labels come from OCR only; and the harness still trusts a model's `done`
-when no verifier is given.
+there; the 0.5B text planner picks wrong actions; UI Automation has never been exercised
+against a real Windows desktop, so an agent there still reads labels from OCR; and the
+harness still trusts a model's `done` when no verifier is given.
 
 ## Roadmap
 
@@ -1266,7 +1272,15 @@ when no verifier is given.
 - [x] **2.0** Operator approval gates - a label whitelist, or ask before every action
 - [x] **2.0** A resumable task queue with a report, and structural verification of "it worked"
 - [x] **2.0** `skills` and `batch` subcommands, and an example queue at `tasks/day.json`
-- [ ] A gallery run that pictures the 2.0 additions: the 84 new public callables have no screenshot yet
+- [x] **2.0** A watchdog: a per-call and whole-episode deadline, so a call that never returns
+      becomes a named step instead of a console that looks like it is thinking
+- [x] **2.0** An audit log: an append-only, hash-chained record of a run, and `--verify-audit`
+      to check it - so the evidence survives the container that produced it
+- [x] **2.0** `--gate`, `--audit` and `--verify-audit` on the command line, and the shared
+      AT-SPI guard, so a machine without the typelib skips rather than failing
+- [ ] A gallery run that pictures the 2.0 additions: **107 of the package's 270 public callables**
+      have no screenshot yet - 103 in the operator layer and four CLI entry points. Adding the
+      entries to `showcase_catalog.py` is what closes it
 - [ ] **Verify UI Automation** (Windows) against a real Windows desktop. The reader and the
       detector are implemented and unit-tested against a synthetic control tree, and
       `accessibility_available("uia")` reports the truth on any machine - but no Windows