Publish tiny_computer_agent_app.py from marimo

tiny_computer_agent_app.py

@@ -132,7 +132,7 @@ def readme():
     ```bash
     python -m pip install -e ".[desktop,ocr,dev]"
     pytest tests -q
-    # 217 passed, 2 skipped in 18.7s
+    # 227 passed, 2 skipped in 18.7s
     ```
 
     The two skips are the live-desktop tests. Give them a display and they run for real:
@@ -304,7 +304,7 @@ def readme():
       `warning: no text was found on screen, so the index is 96 numbered grid cells and the
       agent is choosing blind` — instead of silently guessing.
 
-    **219 tests pass**, including real OCR over a real image and (opt-in) live-desktop tests
+    **229 tests pass**, including real OCR over a real image and (opt-in) live-desktop tests
     that click a real button.
 
     ## Verified on real Windows
@@ -346,7 +346,7 @@ def readme():
 
     None of the three failures in the first row is a harness defect: two are
     `ModuleNotFoundError: numpy` and the third is the guard bug documented below. With the
-    OCR engine installed, the same suite exits **0**. (Those Windows counts predate `trace.py`; the harness suite has since grown to 219 tests.)
+    OCR engine installed, the same suite exits **0**. (Those Windows counts predate `trace.py`; the harness suite has since grown to 229 tests.)
 
     **The emulated-OCR caveat.** onnxruntime requires AVX2, and QEMU's software emulation of
     AVX2 is roughly two orders of magnitude slower than native. A suite that finishes in 17
@@ -372,7 +372,7 @@ def readme():
 
     ## The test suite, in full
 
-    `pytest tests` collects **219 tests**. 217 run headless — no display, no network, no
+    `pytest tests` collects **229 tests**. 227 run headless — no display, no network, no
     weights. The last 2 are the opt-in live-desktop tests, skipped unless
     `TINY_COMPUTER_AGENT_LIVE_TEST=1`.
 
@@ -399,8 +399,8 @@ def readme():
     | `tests/test_live_desktop.py` | 2 | **needs a display, opt-in**: OCR finds a real button at the pixels it was drawn at, and a real click makes the app react |
 
     ```bash
-    pytest tests -q                                  # 217 passed, 2 skipped
-    TINY_COMPUTER_AGENT_LIVE_TEST=1 pytest tests -q  # 219 passed, on a desktop
+    pytest tests -q                                  # 227 passed, 2 skipped
+    TINY_COMPUTER_AGENT_LIVE_TEST=1 pytest tests -q  # 229 passed, on a desktop
     ```
 
     ## Install
@@ -409,7 +409,7 @@ def readme():
     pip install -e ".[ocr]"              # labels from pixels (RapidOCR: no system binary)
     pip install -e ".[desktop,ocr]"      # + real capture and real input
     pip install -e ".[vision,ocr]"       # + the tiny model brains
-    pytest                               # 219 tests, no display or network needed
+    pytest                               # 229 tests, no display or network needed
     ```
 
     The accessibility detector needs system packages rather than an extra:
@@ -523,6 +523,7 @@ def readme():
     | `accessibility.py` | The accessibility tree: `AtspiReader` and `AtspiDetector`, roles and real extents |
     | `showcase.py` | The capability gallery: one real screenshot per capability, checked against the public API |
     | `measure_vlm.py` | Measures the four ways to ask a model, on a real screen, and prints the table |
+    | `evaluate_daily.py` | A day's tasks against a real app, verified by the app's own log |
 
     ## The marimo console
 
@@ -575,6 +576,53 @@ def readme():
     on Windows keeps using OCR. Tk barely publishes a tree and a canvas publishes none,
     which is why this is a detector and not a replacement.
 
+    ## Can it do a day's work?
+
+    The honest way to answer "could this replace a person at a PC?" is to give it a
+    person's tasks and check the result against **the application's own record**, not the
+    agent's opinion of itself. Eleven of them, on a real GTK desktop with the real mouse:
+
+    ```
+    button          yes   0.7  Click Save
+    right-button    yes   0.4  Click Cancel, not the button next to it
+    type            yes   1.8  Type a new file name and save
+    shortcut        yes   0.7  Press Ctrl+S
+    menu            yes   0.7  Open the File menu and choose Save As
+    scroll          yes   1.5  Scroll the list and pick a row
+    second-window   yes   0.7  Open Settings and press Apply there
+    see             yes   0.1  Read what the window says
+    verify          yes   0.7  Check the save actually happened
+    refuse          yes   0.1  Do not press the Terminal button
+    kill-switch     yes   0.1  Stop a live run mid-plan
+    ```
+
+    Run it yourself with `python evaluate_daily.py` (`--only menu` for one task).
+
+    **What that number does and does not mean.** The first run of this suite scored
+    **5/11**, and the failures were not the model. A coordinate bug in the detector
+    *chain* was shifting every accessibility click by the capture's scale factor:
+    `--detector best` wraps the tree detector inside `AutoDetector`, which reported
+    "frame coordinates" for elements that were already screen pixels, so `IndexedScreen`
+    re-mapped them through the downscale and the click landed beside the button. Two more
+    real gaps surfaced with it - an unnamed GTK list row was invisible to the index (so
+    nothing in a list could be clicked at all), and two GTK instances sharing an
+    application id made the second launch silently do nothing. All three are fixed and
+    pinned by tests.
+
+    So the machine half works: it can see, decide, act and verify a day's operations, and
+    it refuses the dangerous ones. What this suite deliberately does **not** measure is
+    *choosing* the actions - nine of the eleven tasks were driven by a scripted action
+    list, because that is what isolates the harness from the brain. Choosing is measured
+    in [asking a tiny model in four ways](#asking-a-tiny-model-in-four-ways-measured):
+    4/4 goals right for a 0.5B text planner over the accessibility index in `rank` mode,
+    and 1-2/4 for a 256M VLM.
+
+    Caveats, stated plainly: this is a Linux GTK desktop under Xvfb with **no window
+    manager** (so the keyboard tasks had to move the pointer into the window first,
+    because X without a manager focuses by pointer); Windows was not exercised; the tasks
+    are single-application with no overlapping windows, no animations and no network; and
+    eleven tasks are a sample, not a survey.
+
     ## Capability gallery
 
     **68 capabilities, one screenshot each - and every one of the 141 public callables in the package (functions, methods, classes and protocols) appears in at least one of them**, together with 6 of its public constants.
@@ -1227,9 +1275,12 @@ def readme():
           [asking a tiny model in four ways](#asking-a-tiny-model-in-four-ways-measured)
     - [x] The marimo console: live screen preview, action log, pause / resume / step
     - [x] An accessibility-tree detector on Linux (AT-SPI), driving a real click with no OCR
-    - [ ] **Make the 256M VLM choose correctly.** Masking fixed the echo and the latency; the model
-          still picks the wrong item from a four-item menu of exact widgets. That is now a question
-          about weights, or about a better default (the rule-based planner), not about prompt shape.
+    - [x] Make a tiny model choose correctly: a 0.5B text planner over the accessibility index in
+          `rank` mode gets **4/4** goals right in 0.9s. A 256M **vision** model does not (1-2/4) -
+          it is below the threshold, which is why the rule-free planner is still the default
+    - [x] A daily-usage evaluation: 11 tasks (menu, scroll, typing, shortcut, second window,
+          verify, refuse, kill switch) against a real GTK app, checked against the application's
+          own log - see [can it do a day's work](#can-it-do-a-days-work)
     - [ ] **UI Automation** (Windows) for the accessibility detector. The interface is here and
           `accessibility_available("uia")` reports it as not implemented rather than pretending.
     - [ ] Windows run of the *live desktop* test: the workflow is committed, but it needs a
@@ -1378,7 +1429,7 @@ def publish_status(
             }
             # the tool that builds the capability gallery, shipped with the card it fills
             for _tool in ("showcase.py", "showcase_core.py", "showcase_catalog.py",
-                          "measure_vlm.py"):
+                          "measure_vlm.py", "evaluate_daily.py"):
                 if (_root / _tool).is_file():
                     _files[_tool] = (_root / _tool).read_text(encoding="utf-8")
             _folders = [f for f in ("tiny_computer_agent", "tests", "gallery",