diff --git a/gallery/README.md b/gallery/README.md new file mode 100644 index 0000000000000000000000000000000000000000..70537ff1523ecc1cb2565ef8ba24e17ed1f7d454 --- /dev/null +++ b/gallery/README.md @@ -0,0 +1,602 @@ +--- +title: Capability gallery +--- + +## Capability gallery + +**68 capabilities, one screenshot each - and every one of the 141 public callables in the package (functions, methods, classes and protocols) appears in at least one of them**, together with 6 of its public constants. + +Nothing here is a mock-up. Each picture is produced by [`showcase.py`](showcase.py): a command or a call is run for real, and its actual output - or the real desktop it acted on - is what you see. The desktop shots are real screen captures on a real X server (Xvfb, 1280x800), including a real mouse click verified by the target application's own reaction. + +> **An honest note on the platform.** These were captured on Linux, because the sandbox that builds this card has no Windows: no KVM, no QEMU and no Windows image. The harness is OS-agnostic (pyautogui uses `SendInput` on Windows, and `GrabScreen` falls back to Pillow's `ImageGrab`), so these are the same code paths - but they are not Windows pixels. The Windows twins are generated by the same script in [`.github/workflows/live-desktop.yml`](.github/workflows/live-desktop.yml). + +Regenerate it with `python showcase.py` (`--list` to see the ids, `--only ` for a subset, `--card` to rewrite this section). It runs the calls for real, so a broken capability fails loudly instead of shipping a stale picture. + +| area | capabilities | +|---|---:| +| Action vocabulary | 3 | +| Decisions | 7 | +| Execution | 2 | +| The loop | 7 | +| Safety | 4 | +| Screens | 6 | +| Perception | 6 | +| Preflight | 4 | +| Models | 5 | +| Traces & scoring | 7 | +| Command line | 9 | +| Real desktop | 7 | +| Extensibility | 1 | + +--- + +### Action vocabulary + +**The six verbs, and only six** + +*Every action is validated on construction, before anything touches the machine.* + + + +`shows:` `ACTION_KINDS`, `ACTION_MENU`, `Action`, `Action.describe`, `Action.to_dict`, `parse_action` + +**Malformed actions are refused, not guessed at** + +*An unknown verb or a missing target raises before the machine is touched.* + + + +`shows:` `ActionError`, `Action`, `parse_action` + +**The contract handed to a model** + +*The prompt-side menu: one JSON object, six shapes, no invented verbs.* + + + +`shows:` `ACTION_MENU` + +### Decisions + +**A short numbered menu, in reading order** + +*A 256M model cannot emit coordinates; it can pick a number from twelve.* + + + +`shows:` `DecisionMenu`, `DecisionMenu.render`, `MenuItem`, `MenuItem.label` + +**Menu lookup tolerates what models actually send** + +*1, "2", 2.0 and "Save." all resolve; a stray word resolves to nothing.* + + + +`shows:` `DecisionMenu.item`, `DecisionMenu.find`, `DecisionMenu.mentioned_in` + +**Reading a decision out of a small model's reply** + +*The LAST JSON object wins, because a model that echoes its prompt answers last.* + + + +`shows:` `parse_decision`, `extract_json_object`, `extract_json_objects`, `DecisionMenu` + +**An echoed prompt is refused; a right answer in the wrong field is recovered** + +*Real 256M/0.5B failure modes, both caught: no blind click either way.* + + + +`shows:` `parse_decision`, `BrainError`, `DecisionMenu.mentioned_in` + +**A garbled verb with a valid label is still usable** + +*A 256M model really does emit {"kind": "file", "label": "Save"}.* + + + +`shows:` `parse_decision`, `BrainError` + +**MockBrain: replay a fixed script, exactly once** + +*Deterministic and offline - the workhorse for tests, demos and replays.* + + + +`shows:` `MockBrain`, `MockBrain.decide`, `MockBrain.remaining` + +**PlannerBrain: no weights, no download, and it gives up honestly** + +*The proof the harness works end to end, and a real fallback when no model is wanted.* + + + +`shows:` `PlannerBrain`, `PlannerBrain.decide`, `GoalUnreachable` + +### Execution + +**Dry run resolves every click to pixels - and touches nothing** + +*The default everywhere: read a whole episode before anything clicks.* + + + +`shows:` `DryRunExecutor`, `DryRunExecutor.execute`, `resolve_point`, `ActionResult` + +**Real input refuses to guess a pixel for a label it cannot find** + +*This is the gate between a hallucinated label and 'Delete account'.* + + + +`shows:` `ActionResult.summary`, `InputUnavailable`, `PyAutoGuiExecutor`, `PyAutoGuiExecutor.execute` + +### The loop + +**observe -> think -> act -> verify, traced end to end** + +*One call runs a whole episode; every step is kept for audit.* + + + +`shows:` `AgentLoop`, `AgentLoop.run`, `Episode`, `Episode.render`, `Episode.to_rows`, `Step`, `Step.line`, `Step.ok` + +**A hard step budget, so a run cannot wander** + +*The budget ends the episode and says so; nothing executes past it.* + + + +`shows:` `Policy`, `Policy.step_budget`, `AgentLoop.run` + +**Repeating one action against an unchanged screen is stopped** + +*A stuck agent is not a persistent one - it is stopped and told why.* + + + +`shows:` `AgentLoop.run`, `Step`, `Step.error` + +**A kill-switch file aborts before the next action** + +*Checked before every single action: create the file, the run stops.* + + + +`shows:` `Policy.kill_switch_path`, `Policy.should_stop`, `AgentLoop.run` + +**A dead screen or a broken brain is an episode, not a traceback** + +*Every failure path is recorded as a failed step with a reason.* + + + +`shows:` `AgentLoop.run`, `Episode.stop_reason`, `GoalUnreachable`, `BrainError` + +**Success is evidence, not the model's word** + +*Without a verifier, a successful action that removes its own target looks like a failure.* + + + +`shows:` `AgentLoop.run`, `Episode.success`, `Observation.text` + +**Every step can be streamed out as it happens** + +*This is the hook a live console, a progress bar or a websocket uses.* + + + +`shows:` `AgentLoop.run`, `Step.line`, `on_step` + +### Safety + +**Typing from the deny list never reaches the keyboard** + +*Checked before execution; a refusal ends the episode and clicks nothing.* + + + +`shows:` `Policy`, `Policy.check`, `PolicyViolation`, `DEFAULT_DENY_TEXT` + +**Clicking a denied widget is refused, however the label is spelled** + +*Terminal, keychain, wallet, password manager, bank - matched on the label.* + + + +`shows:` `Policy.deny_labels`, `Policy.check`, `Observation.find` + +**Key chords on the deny list are never pressed** + +*ctrl+alt+t, ctrl+alt+delete and ctrl+shift+escape are refused by default.* + + + +`shows:` `Policy.deny_keys`, `Policy.check`, `PolicyViolation` + +**A numbered grid cell is a blind click: allowed to plan, refused for real** + +*With real input the blind guess is refused unless you pass --allow-grid.* + + + +`shows:` `Policy.allow_grid_clicks`, `Policy.check`, `GridDetector` + +### Screens + +**A scripted screen, for testing the loop with no display** + +*The final frame repeats forever, so a runaway agent cannot exhaust the script.* + + + +`shows:` `FakeScreen`, `FakeScreen.capture`, `Observation` + +**A still image (or a sequence) replayed as a screen** + +*Lets the whole pipeline be exercised with no display, and a failure be replayed.* + + + +`shows:` `ImageScreen`, `ImageScreen.capture`, `fake_desktop` + +**Real capture: monitors, downscaling, and HiDPI mapping** + +*Frame pixel -> screen pixel is preserved, so a click cannot drift.* + + + +`shows:` `GrabScreen`, `GrabScreen.capture`, `GrabScreen.map_to_screen`, `ScreenUnavailable` + +**Labels are compared on their alphanumeric core** + +*OCR loses spaces, models add punctuation - so nothing matches by exact string.* + + + +`shows:` `Element`, `Element.center`, `Element.matches`, `Observation.find`, `normalise_label` + +**The compact text a model is shown** + +*This is the whole world as the brain sees it - capped, and honest when empty.* + + + +`shows:` `Observation.summary` + +**The built-in synthetic desktop** + +*Draws a screenshot-like desktop offline - the same image --image example replays.* + + + +`shows:` `fake_desktop`, `write_fake_desktop`, `OcrDetector` + +### Perception + +**Probing for a detector, and building one by name** + +*The doctor and the CLI both ask this before promising anything.* + + + +`shows:` `detector_available`, `make_detector`, `DetectorUnavailable`, `Detector` + +**OCR turns a screenshot into labelled, clickable elements** + +*The model never has to guess where anything is: this is the grounding.* + + + +`shows:` `OcrDetector`, `OcrDetector.detect`, `Element` + +**Three knobs keep dialog prose out of the menu** + +*Every extra line of noise is one more chance for a small model to click prose.* + + + +`shows:` `OcrDetector.min_score`, `OcrDetector.max_label_chars`, `OcrDetector.min_label_chars` + +**A numbered grid, for screens with no text at all** + +*An agent with an empty index has no legal moves; a grid always has moves.* + + + +`shows:` `GridDetector`, `GridDetector.detect` + +**OCR when there is text, a grid when there is none** + +*The default: useful with nothing installed, better the moment OCR appears.* + + + +`shows:` `AutoDetector`, `AutoDetector.detect` + +**IndexedScreen: capture + detect, composed** + +*One capture() call gives the loop an index it can act on.* + + + +`shows:` `IndexedScreen`, `IndexedScreen.capture`, `Observation.summary` + +### Preflight + +**Is there a desktop session to drive at all?** + +*The first question the CLI and the doctor both ask.* + + + +`shows:` `display_available` + +**Every precondition, one list, each with the fix** + +*Eight checks; a required one that fails is a blocking problem.* + + + +`shows:` `run_checks`, `Check`, `Check.render` + +**python -m tiny_computer_agent --doctor** + +*The first thing to run on a new machine. Exits non-zero when not ready.* + + + +`shows:` `detector_available`, `display_available`, `doctor`, `report` + +**One call wires screen + detector + brain + executor + policy** + +*Defaults are the safe ones: dry run, a step budget, and a kill-switch file.* + + + +`shows:` `build_agent`, `Policy`, `AgentLoop` + +### Models + +**Which model runtime is present, and are the weights cached?** + +*Weights are loaded lazily, so asking is free.* + + + +`shows:` `model_dependencies`, `model_cached`, `MODEL_BRAINS`, `make_model_brain`, `DEFAULT_TEXT_MODEL`, `DEFAULT_VISION_MODEL`, `BrainError` + +**Device and dtype selection** + +*CUDA if it is there, CPU otherwise; the harness never needs a GPU.* + + + +`shows:` `resolve_device`, `pick_dtype` + +**The prompt a model actually receives - and how it learns from a dead end** + +*Placeholders, not worked examples: a weak model echoes the prompt, and an echo fails validation.* + + + +`shows:` `ModelBrain`, `ModelBrain.build_prompt`, `ModelBrain.decide` + +**A second decision is told what already failed** + +*'you already tried this and the screen did NOT change' is the most useful line a small model gets.* + + + +`shows:` `ModelBrain`, `ModelBrain.build_prompt`, `ModelBrain.history` + +**The two real model backends, and lazy loading** + +*Constructing a brain costs nothing; the weights arrive on the first decide().* + + + +`shows:` `TextPlannerBrain`, `VLMBrain`, `TextPlannerBrain.unload`, `VLMBrain.unload`, `ModelBrain.loaded`, `make_model_brain` + +### Traces & scoring + +**A task with one right answer, and the frames to show** + +*Ground truth lives with the task, so a run can be scored at all.* + + + +`shows:` `Scenario`, `Scenario.label`, `Scenario.screen`, `Trace.render`, `run_scenario` + +**An episode becomes a portable, pixel-free trace** + +*JSON, so it diffs in review and replays on a machine with nothing installed.* + + + +`shows:` `Trace`, `Trace.blocked`, `Trace.choice`, `Trace.decided`, `Trace.decision_step`, `Trace.frames`, `Trace.load`, `Trace.name`, `Trace.record`, `Trace.save`, `Trace.to_dict`, `Trace.to_json`, `TraceStep`, `TraceStep.observation`, `TraceStep.to_dict` + +**Replay a recorded trace against a different brain** + +*The frames are the fixture; the brain is the only variable. Nothing is captured or clicked.* + + + +`shows:` `Trace.replay`, `Trace.observations`, `score_trace` + +**Counting how often the choice was right** + +*Scored on normalised text, so OCR spacing is forgiven - the same bargain the index makes.* + + + +`shows:` `score_trace`, `score`, `score_replay`, `Verdict`, `Verdict.line` + +**Verdicts collected into an accuracy** + +*The number the whole module exists for: which brain is right, and how often.* + + + +`shows:` `Scoreboard`, `Scoreboard.accuracy`, `Scoreboard.accuracy_for`, `Scoreboard.add`, `Scoreboard.blocked`, `Scoreboard.brains`, `Scoreboard.by_brain`, `Scoreboard.confusion`, `Scoreboard.earned`, `Scoreboard.render`, `Scoreboard.to_rows`, `Scoreboard.total` + +**Every scenario through every brain, in one table** + +*A fresh brain per task, so state like 'already clicked this' cannot leak between them.* + + + +`shows:` `benchmark`, `Scoreboard.to_rows` + +**Replay a whole recorded suite against one brain** + +*The literal roadmap step: the frames are fixed, nothing new is captured.* + + + +`shows:` `benchmark_traces`, `load_traces`, `Trace.save` + +### Command line + +**--image example (replay a screenshot, dry run)** + +*No display needed: the whole pipeline over a still image.* + + + +`shows:` `main`, `build_parser`, `ImageScreen`, `detector_available` + +**--script (replay a fixed action list instead of a brain)** + +*The script is validated exactly like a model's answer: nothing skips the gates.* + + + +`shows:` `MockBrain`, `demo_screen`, `parse_action` + +**--verify-text (the harness decides success, not the brain)** + +*Without it, an action that removes its own target looks like a failure.* + + + +`shows:` `AgentLoop.run`, `Episode.success` + +**--trace / --expected (record the run, then score it)** + +*Writes JSON next to the code, and prints the score straight away.* + + + +`shows:` `Trace.record`, `Trace.save`, `score_trace` + +**--no-ocr (index a numbered grid instead)** + +*Useful on a machine with no OCR engine, or for a canvas app.* + + + +`shows:` `GridDetector`, `make_detector`, `detector_available` + +**Exit codes: 0 success, 1 failure, 2 bad usage, 3 no screen** + +*A shell can tell 'it worked' from 'it gave up' without parsing any text.* + + + +`shows:` `main` + +**The two deliberate acts before the mouse can move** + +*--real-input without the env var, and --brain vlm with no pixels: both refuse.* + + + +`shows:` `main`, `ALLOW_REAL_INPUT_ENV` + +**--live with no display is refused, with the fix** + +*It tells you what to do instead of dying inside an episode.* + + + +`shows:` `display_available`, `main` + +**--live (your real screen, dry run)** + +*Real capture, real OCR, a real element index - and not one click.* + + + +`shows:` `GrabScreen`, `IndexedScreen`, `AutoDetector`, `main` + +### Real desktop + +**GrabScreen: capture a real desktop** + +*1280x800 of a real X server. mss first, Pillow's ImageGrab as the fallback.* + + + +`shows:` `GrabScreen`, `GrabScreen.capture`, `ScreenUnavailable` + +**The element index, drawn on the pixels it came from** + +*Every box is a real OCR word with a real screen coordinate: the model picks a label, not a pixel.* + + + +`shows:` `OcrDetector`, `OcrDetector.detect`, `IndexedScreen`, `Element.center` + +**A real click, and the window's own reaction** + +*Pixels in, a label chosen, a pixel resolved, a button pressed - and the dialog closed itself.* + + + +`shows:` `PyAutoGuiExecutor`, `AgentLoop.run`, `Policy.dry_run`, `resolve_point` + +**Success only when the harness can see it** + +*The app changed its own title and wrote a marker file; the verifier saw the change.* + + + +`shows:` `AgentLoop.run`, `Episode.success`, `Episode.stop_reason` + +**A screen with no text still has legal moves** + +*OCR finds nothing on a canvas, so the index degrades to numbered cells - and says so.* + + + +`shows:` `AutoDetector`, `GridDetector`, `OcrDetector` + +**The safety gate, on a real button, in a real window** + +*The Terminal button is visible and labelled - and the policy refuses to press it.* + + + +`shows:` `Policy.check`, `Policy.deny_labels`, `PolicyViolation`, `Observation.find` + +**The same click, driven end to end from the CLI** + +*--live --real-input --verify-text Saved: capture, OCR, click, verify, exit code.* + + + +`shows:` `main`, `PyAutoGuiExecutor`, `AgentLoop.run` + +### Extensibility + +**Every stage is a Protocol: bring your own screen, brain, detector, executor** + +*The loop depends on four one-method contracts, not on the shipped implementations.* + + + +`shows:` `Brain`, `Brain.decide`, `ScreenSource`, `ScreenSource.capture`, `Detector`, `Detector.detect`, `Executor`, `Executor.execute`, `ActionResult` + diff --git a/gallery/_card_section.md b/gallery/_card_section.md new file mode 100644 index 0000000000000000000000000000000000000000..77fb7fbb06e43ce5aa21406a26497388b74e4caf --- /dev/null +++ b/gallery/_card_section.md @@ -0,0 +1,597 @@ +## Capability gallery + +**68 capabilities, one screenshot each - and every one of the 141 public callables in the package (functions, methods, classes and protocols) appears in at least one of them**, together with 6 of its public constants. + +Nothing here is a mock-up. Each picture is produced by [`showcase.py`](showcase.py): a command or a call is run for real, and its actual output - or the real desktop it acted on - is what you see. The desktop shots are real screen captures on a real X server (Xvfb, 1280x800), including a real mouse click verified by the target application's own reaction. + +> **An honest note on the platform.** These were captured on Linux, because the sandbox that builds this card has no Windows: no KVM, no QEMU and no Windows image. The harness is OS-agnostic (pyautogui uses `SendInput` on Windows, and `GrabScreen` falls back to Pillow's `ImageGrab`), so these are the same code paths - but they are not Windows pixels. The Windows twins are generated by the same script in [`.github/workflows/live-desktop.yml`](.github/workflows/live-desktop.yml). + +Regenerate it with `python showcase.py` (`--list` to see the ids, `--only ` for a subset, `--card` to rewrite this section). It runs the calls for real, so a broken capability fails loudly instead of shipping a stale picture. + +| area | capabilities | +|---|---:| +| Action vocabulary | 3 | +| Decisions | 7 | +| Execution | 2 | +| The loop | 7 | +| Safety | 4 | +| Screens | 6 | +| Perception | 6 | +| Preflight | 4 | +| Models | 5 | +| Traces & scoring | 7 | +| Command line | 9 | +| Real desktop | 7 | +| Extensibility | 1 | + +--- + +### Action vocabulary + +**The six verbs, and only six** + +*Every action is validated on construction, before anything touches the machine.* + + + +`shows:` `ACTION_KINDS`, `ACTION_MENU`, `Action`, `Action.describe`, `Action.to_dict`, `parse_action` + +**Malformed actions are refused, not guessed at** + +*An unknown verb or a missing target raises before the machine is touched.* + + + +`shows:` `ActionError`, `Action`, `parse_action` + +**The contract handed to a model** + +*The prompt-side menu: one JSON object, six shapes, no invented verbs.* + + + +`shows:` `ACTION_MENU` + +### Decisions + +**A short numbered menu, in reading order** + +*A 256M model cannot emit coordinates; it can pick a number from twelve.* + + + +`shows:` `DecisionMenu`, `DecisionMenu.render`, `MenuItem`, `MenuItem.label` + +**Menu lookup tolerates what models actually send** + +*1, "2", 2.0 and "Save." all resolve; a stray word resolves to nothing.* + + + +`shows:` `DecisionMenu.item`, `DecisionMenu.find`, `DecisionMenu.mentioned_in` + +**Reading a decision out of a small model's reply** + +*The LAST JSON object wins, because a model that echoes its prompt answers last.* + + + +`shows:` `parse_decision`, `extract_json_object`, `extract_json_objects`, `DecisionMenu` + +**An echoed prompt is refused; a right answer in the wrong field is recovered** + +*Real 256M/0.5B failure modes, both caught: no blind click either way.* + + + +`shows:` `parse_decision`, `BrainError`, `DecisionMenu.mentioned_in` + +**A garbled verb with a valid label is still usable** + +*A 256M model really does emit {"kind": "file", "label": "Save"}.* + + + +`shows:` `parse_decision`, `BrainError` + +**MockBrain: replay a fixed script, exactly once** + +*Deterministic and offline - the workhorse for tests, demos and replays.* + + + +`shows:` `MockBrain`, `MockBrain.decide`, `MockBrain.remaining` + +**PlannerBrain: no weights, no download, and it gives up honestly** + +*The proof the harness works end to end, and a real fallback when no model is wanted.* + + + +`shows:` `PlannerBrain`, `PlannerBrain.decide`, `GoalUnreachable` + +### Execution + +**Dry run resolves every click to pixels - and touches nothing** + +*The default everywhere: read a whole episode before anything clicks.* + + + +`shows:` `DryRunExecutor`, `DryRunExecutor.execute`, `resolve_point`, `ActionResult` + +**Real input refuses to guess a pixel for a label it cannot find** + +*This is the gate between a hallucinated label and 'Delete account'.* + + + +`shows:` `ActionResult.summary`, `InputUnavailable`, `PyAutoGuiExecutor`, `PyAutoGuiExecutor.execute` + +### The loop + +**observe -> think -> act -> verify, traced end to end** + +*One call runs a whole episode; every step is kept for audit.* + + + +`shows:` `AgentLoop`, `AgentLoop.run`, `Episode`, `Episode.render`, `Episode.to_rows`, `Step`, `Step.line`, `Step.ok` + +**A hard step budget, so a run cannot wander** + +*The budget ends the episode and says so; nothing executes past it.* + + + +`shows:` `Policy`, `Policy.step_budget`, `AgentLoop.run` + +**Repeating one action against an unchanged screen is stopped** + +*A stuck agent is not a persistent one - it is stopped and told why.* + + + +`shows:` `AgentLoop.run`, `Step`, `Step.error` + +**A kill-switch file aborts before the next action** + +*Checked before every single action: create the file, the run stops.* + + + +`shows:` `Policy.kill_switch_path`, `Policy.should_stop`, `AgentLoop.run` + +**A dead screen or a broken brain is an episode, not a traceback** + +*Every failure path is recorded as a failed step with a reason.* + + + +`shows:` `AgentLoop.run`, `Episode.stop_reason`, `GoalUnreachable`, `BrainError` + +**Success is evidence, not the model's word** + +*Without a verifier, a successful action that removes its own target looks like a failure.* + + + +`shows:` `AgentLoop.run`, `Episode.success`, `Observation.text` + +**Every step can be streamed out as it happens** + +*This is the hook a live console, a progress bar or a websocket uses.* + + + +`shows:` `AgentLoop.run`, `Step.line`, `on_step` + +### Safety + +**Typing from the deny list never reaches the keyboard** + +*Checked before execution; a refusal ends the episode and clicks nothing.* + + + +`shows:` `Policy`, `Policy.check`, `PolicyViolation`, `DEFAULT_DENY_TEXT` + +**Clicking a denied widget is refused, however the label is spelled** + +*Terminal, keychain, wallet, password manager, bank - matched on the label.* + + + +`shows:` `Policy.deny_labels`, `Policy.check`, `Observation.find` + +**Key chords on the deny list are never pressed** + +*ctrl+alt+t, ctrl+alt+delete and ctrl+shift+escape are refused by default.* + + + +`shows:` `Policy.deny_keys`, `Policy.check`, `PolicyViolation` + +**A numbered grid cell is a blind click: allowed to plan, refused for real** + +*With real input the blind guess is refused unless you pass --allow-grid.* + + + +`shows:` `Policy.allow_grid_clicks`, `Policy.check`, `GridDetector` + +### Screens + +**A scripted screen, for testing the loop with no display** + +*The final frame repeats forever, so a runaway agent cannot exhaust the script.* + + + +`shows:` `FakeScreen`, `FakeScreen.capture`, `Observation` + +**A still image (or a sequence) replayed as a screen** + +*Lets the whole pipeline be exercised with no display, and a failure be replayed.* + + + +`shows:` `ImageScreen`, `ImageScreen.capture`, `fake_desktop` + +**Real capture: monitors, downscaling, and HiDPI mapping** + +*Frame pixel -> screen pixel is preserved, so a click cannot drift.* + + + +`shows:` `GrabScreen`, `GrabScreen.capture`, `GrabScreen.map_to_screen`, `ScreenUnavailable` + +**Labels are compared on their alphanumeric core** + +*OCR loses spaces, models add punctuation - so nothing matches by exact string.* + + + +`shows:` `Element`, `Element.center`, `Element.matches`, `Observation.find`, `normalise_label` + +**The compact text a model is shown** + +*This is the whole world as the brain sees it - capped, and honest when empty.* + + + +`shows:` `Observation.summary` + +**The built-in synthetic desktop** + +*Draws a screenshot-like desktop offline - the same image --image example replays.* + + + +`shows:` `fake_desktop`, `write_fake_desktop`, `OcrDetector` + +### Perception + +**Probing for a detector, and building one by name** + +*The doctor and the CLI both ask this before promising anything.* + + + +`shows:` `detector_available`, `make_detector`, `DetectorUnavailable`, `Detector` + +**OCR turns a screenshot into labelled, clickable elements** + +*The model never has to guess where anything is: this is the grounding.* + + + +`shows:` `OcrDetector`, `OcrDetector.detect`, `Element` + +**Three knobs keep dialog prose out of the menu** + +*Every extra line of noise is one more chance for a small model to click prose.* + + + +`shows:` `OcrDetector.min_score`, `OcrDetector.max_label_chars`, `OcrDetector.min_label_chars` + +**A numbered grid, for screens with no text at all** + +*An agent with an empty index has no legal moves; a grid always has moves.* + + + +`shows:` `GridDetector`, `GridDetector.detect` + +**OCR when there is text, a grid when there is none** + +*The default: useful with nothing installed, better the moment OCR appears.* + + + +`shows:` `AutoDetector`, `AutoDetector.detect` + +**IndexedScreen: capture + detect, composed** + +*One capture() call gives the loop an index it can act on.* + + + +`shows:` `IndexedScreen`, `IndexedScreen.capture`, `Observation.summary` + +### Preflight + +**Is there a desktop session to drive at all?** + +*The first question the CLI and the doctor both ask.* + + + +`shows:` `display_available` + +**Every precondition, one list, each with the fix** + +*Eight checks; a required one that fails is a blocking problem.* + + + +`shows:` `run_checks`, `Check`, `Check.render` + +**python -m tiny_computer_agent --doctor** + +*The first thing to run on a new machine. Exits non-zero when not ready.* + + + +`shows:` `detector_available`, `display_available`, `doctor`, `report` + +**One call wires screen + detector + brain + executor + policy** + +*Defaults are the safe ones: dry run, a step budget, and a kill-switch file.* + + + +`shows:` `build_agent`, `Policy`, `AgentLoop` + +### Models + +**Which model runtime is present, and are the weights cached?** + +*Weights are loaded lazily, so asking is free.* + + + +`shows:` `model_dependencies`, `model_cached`, `MODEL_BRAINS`, `make_model_brain`, `DEFAULT_TEXT_MODEL`, `DEFAULT_VISION_MODEL`, `BrainError` + +**Device and dtype selection** + +*CUDA if it is there, CPU otherwise; the harness never needs a GPU.* + + + +`shows:` `resolve_device`, `pick_dtype` + +**The prompt a model actually receives - and how it learns from a dead end** + +*Placeholders, not worked examples: a weak model echoes the prompt, and an echo fails validation.* + + + +`shows:` `ModelBrain`, `ModelBrain.build_prompt`, `ModelBrain.decide` + +**A second decision is told what already failed** + +*'you already tried this and the screen did NOT change' is the most useful line a small model gets.* + + + +`shows:` `ModelBrain`, `ModelBrain.build_prompt`, `ModelBrain.history` + +**The two real model backends, and lazy loading** + +*Constructing a brain costs nothing; the weights arrive on the first decide().* + + + +`shows:` `TextPlannerBrain`, `VLMBrain`, `TextPlannerBrain.unload`, `VLMBrain.unload`, `ModelBrain.loaded`, `make_model_brain` + +### Traces & scoring + +**A task with one right answer, and the frames to show** + +*Ground truth lives with the task, so a run can be scored at all.* + + + +`shows:` `Scenario`, `Scenario.label`, `Scenario.screen`, `Trace.render`, `run_scenario` + +**An episode becomes a portable, pixel-free trace** + +*JSON, so it diffs in review and replays on a machine with nothing installed.* + + + +`shows:` `Trace`, `Trace.blocked`, `Trace.choice`, `Trace.decided`, `Trace.decision_step`, `Trace.frames`, `Trace.load`, `Trace.name`, `Trace.record`, `Trace.save`, `Trace.to_dict`, `Trace.to_json`, `TraceStep`, `TraceStep.observation`, `TraceStep.to_dict` + +**Replay a recorded trace against a different brain** + +*The frames are the fixture; the brain is the only variable. Nothing is captured or clicked.* + + + +`shows:` `Trace.replay`, `Trace.observations`, `score_trace` + +**Counting how often the choice was right** + +*Scored on normalised text, so OCR spacing is forgiven - the same bargain the index makes.* + + + +`shows:` `score_trace`, `score`, `score_replay`, `Verdict`, `Verdict.line` + +**Verdicts collected into an accuracy** + +*The number the whole module exists for: which brain is right, and how often.* + + + +`shows:` `Scoreboard`, `Scoreboard.accuracy`, `Scoreboard.accuracy_for`, `Scoreboard.add`, `Scoreboard.blocked`, `Scoreboard.brains`, `Scoreboard.by_brain`, `Scoreboard.confusion`, `Scoreboard.earned`, `Scoreboard.render`, `Scoreboard.to_rows`, `Scoreboard.total` + +**Every scenario through every brain, in one table** + +*A fresh brain per task, so state like 'already clicked this' cannot leak between them.* + + + +`shows:` `benchmark`, `Scoreboard.to_rows` + +**Replay a whole recorded suite against one brain** + +*The literal roadmap step: the frames are fixed, nothing new is captured.* + + + +`shows:` `benchmark_traces`, `load_traces`, `Trace.save` + +### Command line + +**--image example (replay a screenshot, dry run)** + +*No display needed: the whole pipeline over a still image.* + + + +`shows:` `main`, `build_parser`, `ImageScreen`, `detector_available` + +**--script (replay a fixed action list instead of a brain)** + +*The script is validated exactly like a model's answer: nothing skips the gates.* + + + +`shows:` `MockBrain`, `demo_screen`, `parse_action` + +**--verify-text (the harness decides success, not the brain)** + +*Without it, an action that removes its own target looks like a failure.* + + + +`shows:` `AgentLoop.run`, `Episode.success` + +**--trace / --expected (record the run, then score it)** + +*Writes JSON next to the code, and prints the score straight away.* + + + +`shows:` `Trace.record`, `Trace.save`, `score_trace` + +**--no-ocr (index a numbered grid instead)** + +*Useful on a machine with no OCR engine, or for a canvas app.* + + + +`shows:` `GridDetector`, `make_detector`, `detector_available` + +**Exit codes: 0 success, 1 failure, 2 bad usage, 3 no screen** + +*A shell can tell 'it worked' from 'it gave up' without parsing any text.* + + + +`shows:` `main` + +**The two deliberate acts before the mouse can move** + +*--real-input without the env var, and --brain vlm with no pixels: both refuse.* + + + +`shows:` `main`, `ALLOW_REAL_INPUT_ENV` + +**--live with no display is refused, with the fix** + +*It tells you what to do instead of dying inside an episode.* + + + +`shows:` `display_available`, `main` + +**--live (your real screen, dry run)** + +*Real capture, real OCR, a real element index - and not one click.* + + + +`shows:` `GrabScreen`, `IndexedScreen`, `AutoDetector`, `main` + +### Real desktop + +**GrabScreen: capture a real desktop** + +*1280x800 of a real X server. mss first, Pillow's ImageGrab as the fallback.* + + + +`shows:` `GrabScreen`, `GrabScreen.capture`, `ScreenUnavailable` + +**The element index, drawn on the pixels it came from** + +*Every box is a real OCR word with a real screen coordinate: the model picks a label, not a pixel.* + + + +`shows:` `OcrDetector`, `OcrDetector.detect`, `IndexedScreen`, `Element.center` + +**A real click, and the window's own reaction** + +*Pixels in, a label chosen, a pixel resolved, a button pressed - and the dialog closed itself.* + + + +`shows:` `PyAutoGuiExecutor`, `AgentLoop.run`, `Policy.dry_run`, `resolve_point` + +**Success only when the harness can see it** + +*The app changed its own title and wrote a marker file; the verifier saw the change.* + + + +`shows:` `AgentLoop.run`, `Episode.success`, `Episode.stop_reason` + +**A screen with no text still has legal moves** + +*OCR finds nothing on a canvas, so the index degrades to numbered cells - and says so.* + + + +`shows:` `AutoDetector`, `GridDetector`, `OcrDetector` + +**The safety gate, on a real button, in a real window** + +*The Terminal button is visible and labelled - and the policy refuses to press it.* + + + +`shows:` `Policy.check`, `Policy.deny_labels`, `PolicyViolation`, `Observation.find` + +**The same click, driven end to end from the CLI** + +*--live --real-input --verify-text Saved: capture, OCR, click, verify, exit code.* + + + +`shows:` `main`, `PyAutoGuiExecutor`, `AgentLoop.run` + +### Extensibility + +**Every stage is a Protocol: bring your own screen, brain, detector, executor** + +*The loop depends on four one-method contracts, not on the shipped implementations.* + + + +`shows:` `Brain`, `Brain.decide`, `ScreenSource`, `ScreenSource.capture`, `Detector`, `Detector.detect`, `Executor`, `Executor.execute`, `ActionResult` diff --git a/gallery/auto-detector.png b/gallery/auto-detector.png new file mode 100644 index 0000000000000000000000000000000000000000..b03d7472dfff6cc9429381ba7a7686d4d4aa388d Binary files /dev/null and b/gallery/auto-detector.png differ diff --git a/gallery/benchmark-traces.png b/gallery/benchmark-traces.png new file mode 100644 index 0000000000000000000000000000000000000000..b74e1fb3ea352994abdcd0eca11c194b8d5c52aa Binary files /dev/null and b/gallery/benchmark-traces.png differ diff --git a/gallery/benchmark.png b/gallery/benchmark.png new file mode 100644 index 0000000000000000000000000000000000000000..101f72e0989eb7807bf854563a4031c17df01e47 Binary files /dev/null and b/gallery/benchmark.png differ diff --git a/gallery/brain-mock.png b/gallery/brain-mock.png new file mode 100644 index 0000000000000000000000000000000000000000..58c06a2e35d8faf82a46370ddade2b68a0398589 Binary files /dev/null and b/gallery/brain-mock.png differ diff --git a/gallery/brain-planner.png b/gallery/brain-planner.png new file mode 100644 index 0000000000000000000000000000000000000000..eb268cfc2db4b5fe3266b821bdcc2270e5c274ec Binary files /dev/null and b/gallery/brain-planner.png differ diff --git a/gallery/build-agent.png b/gallery/build-agent.png new file mode 100644 index 0000000000000000000000000000000000000000..1ac3c45520fc868b781d60fa041336acdd4f1bff Binary files /dev/null and b/gallery/build-agent.png differ diff --git a/gallery/capture-options.png b/gallery/capture-options.png new file mode 100644 index 0000000000000000000000000000000000000000..bfe97e0716e0a464c6f69967a9be78a300d17e5c Binary files /dev/null and b/gallery/capture-options.png differ diff --git a/gallery/cli-detector.png b/gallery/cli-detector.png new file mode 100644 index 0000000000000000000000000000000000000000..9a4f0ab9cc62f408946a86d812ca8538e9c26795 Binary files /dev/null and b/gallery/cli-detector.png differ diff --git a/gallery/cli-doctor.png b/gallery/cli-doctor.png new file mode 100644 index 0000000000000000000000000000000000000000..2135b0dbd8248467aab0889bc807b7e8e7737bff Binary files /dev/null and b/gallery/cli-doctor.png differ diff --git a/gallery/cli-exit-codes.png b/gallery/cli-exit-codes.png new file mode 100644 index 0000000000000000000000000000000000000000..5d62335f2c50e98fafbcc0b697c7e1261d2db8ee Binary files /dev/null and b/gallery/cli-exit-codes.png differ diff --git a/gallery/cli-image.png b/gallery/cli-image.png new file mode 100644 index 0000000000000000000000000000000000000000..6be95e4226f14caed66325a0748bc5787220b465 Binary files /dev/null and b/gallery/cli-image.png differ diff --git a/gallery/cli-live-dry-run.png b/gallery/cli-live-dry-run.png new file mode 100644 index 0000000000000000000000000000000000000000..cb2667be48746e77f6c50ad20858a118bfe9dd0c Binary files /dev/null and b/gallery/cli-live-dry-run.png differ diff --git a/gallery/cli-live-headless.png b/gallery/cli-live-headless.png new file mode 100644 index 0000000000000000000000000000000000000000..226ab74f84297e8d4688d64b39e3f6c190eb2f86 Binary files /dev/null and b/gallery/cli-live-headless.png differ diff --git a/gallery/cli-refusals.png b/gallery/cli-refusals.png new file mode 100644 index 0000000000000000000000000000000000000000..0ad55a7ff71ed945ba96370a02bf2766febfdda4 Binary files /dev/null and b/gallery/cli-refusals.png differ diff --git a/gallery/cli-script.png b/gallery/cli-script.png new file mode 100644 index 0000000000000000000000000000000000000000..f00f8dac32e06b660e2ccc0457eb0bd581891b6b Binary files /dev/null and b/gallery/cli-script.png differ diff --git a/gallery/cli-trace.png b/gallery/cli-trace.png new file mode 100644 index 0000000000000000000000000000000000000000..510b8a39f5c23afcaf39ac933a5d659002df8f36 Binary files /dev/null and b/gallery/cli-trace.png differ diff --git a/gallery/cli-verify-text.png b/gallery/cli-verify-text.png new file mode 100644 index 0000000000000000000000000000000000000000..49b9b3f1272c4d32d86b52e5c3cb5dd553c48d27 Binary files /dev/null and b/gallery/cli-verify-text.png differ diff --git a/gallery/decision-echo-guard.png b/gallery/decision-echo-guard.png new file mode 100644 index 0000000000000000000000000000000000000000..538000a83cd5260b6fd1e0ec6402ca8b1989c780 Binary files /dev/null and b/gallery/decision-echo-guard.png differ diff --git a/gallery/decision-garbled.png b/gallery/decision-garbled.png new file mode 100644 index 0000000000000000000000000000000000000000..63a460ef749f2d2780cbcf8e42a99e9d8acf6dad Binary files /dev/null and b/gallery/decision-garbled.png differ diff --git a/gallery/decision-json.png b/gallery/decision-json.png new file mode 100644 index 0000000000000000000000000000000000000000..f30196724bf7421a4305bb9b9a5b94b7518cff4c Binary files /dev/null and b/gallery/decision-json.png differ diff --git a/gallery/desk-capture.png b/gallery/desk-capture.png new file mode 100644 index 0000000000000000000000000000000000000000..4684a30312df900a30f1c60a55d2e4a84843d37a Binary files /dev/null and b/gallery/desk-capture.png differ diff --git a/gallery/desk-cli-real-input.png b/gallery/desk-cli-real-input.png new file mode 100644 index 0000000000000000000000000000000000000000..34931f9853dddba5668a95889e3b655167b1308e Binary files /dev/null and b/gallery/desk-cli-real-input.png differ diff --git a/gallery/desk-click.png b/gallery/desk-click.png new file mode 100644 index 0000000000000000000000000000000000000000..1178500c430a9577f7643258d714303c5b37a4cf Binary files /dev/null and b/gallery/desk-click.png differ diff --git a/gallery/desk-grid-fallback.png b/gallery/desk-grid-fallback.png new file mode 100644 index 0000000000000000000000000000000000000000..36b3d25b35019fa7a8ad3d8ed4117f8a85d3187e Binary files /dev/null and b/gallery/desk-grid-fallback.png differ diff --git a/gallery/desk-ocr-index.png b/gallery/desk-ocr-index.png new file mode 100644 index 0000000000000000000000000000000000000000..2af20a3f2152501beb87f9537cb4a134f4c7edde Binary files /dev/null and b/gallery/desk-ocr-index.png differ diff --git a/gallery/desk-policy-refusal.png b/gallery/desk-policy-refusal.png new file mode 100644 index 0000000000000000000000000000000000000000..901a7e1b4c5e5d09d3c9935bc5a899d5fbc89ea2 Binary files /dev/null and b/gallery/desk-policy-refusal.png differ diff --git a/gallery/desk-verify.png b/gallery/desk-verify.png new file mode 100644 index 0000000000000000000000000000000000000000..b1a2bee4e39e8ab2754227c20b50b0253e16f4c0 Binary files /dev/null and b/gallery/desk-verify.png differ diff --git a/gallery/detector-available.png b/gallery/detector-available.png new file mode 100644 index 0000000000000000000000000000000000000000..de99494bcae97061ee5393b84f57af0ea7887c38 Binary files /dev/null and b/gallery/detector-available.png differ diff --git a/gallery/display-available.png b/gallery/display-available.png new file mode 100644 index 0000000000000000000000000000000000000000..fe02ced0d6096ccbb09dd4d89ba4f0361c24a531 Binary files /dev/null and b/gallery/display-available.png differ diff --git a/gallery/element-matching.png b/gallery/element-matching.png new file mode 100644 index 0000000000000000000000000000000000000000..03c62d7f22af9f1cb4db30ae25e9119c8bb8f51b Binary files /dev/null and b/gallery/element-matching.png differ diff --git a/gallery/exec-dry-run.png b/gallery/exec-dry-run.png new file mode 100644 index 0000000000000000000000000000000000000000..4c7fc81a9772abc23613404b8a7e3fe5ab66b917 Binary files /dev/null and b/gallery/exec-dry-run.png differ diff --git a/gallery/exec-refuses-unresolved.png b/gallery/exec-refuses-unresolved.png new file mode 100644 index 0000000000000000000000000000000000000000..bea8beacb17096ca9373bb3525e06080d95cd641 Binary files /dev/null and b/gallery/exec-refuses-unresolved.png differ diff --git a/gallery/fake-desktop.png b/gallery/fake-desktop.png new file mode 100644 index 0000000000000000000000000000000000000000..66c9445e078b3e7fc750b867084245fc25007eda Binary files /dev/null and b/gallery/fake-desktop.png differ diff --git a/gallery/grid-detector.png b/gallery/grid-detector.png new file mode 100644 index 0000000000000000000000000000000000000000..5b1e9c50e4631b1306b6fdece5a425e9822aa288 Binary files /dev/null and b/gallery/grid-detector.png differ diff --git a/gallery/index.json b/gallery/index.json new file mode 100644 index 0000000000000000000000000000000000000000..c57a1260a0587e01f56450fb570fde8ed0397543 --- /dev/null +++ b/gallery/index.json @@ -0,0 +1,864 @@ +[ + { + "id": "vocab-verbs", + "category": "Action vocabulary", + "title": "The six verbs, and only six", + "shows": [ + "ACTION_KINDS", + "ACTION_MENU", + "Action", + "Action.describe", + "Action.to_dict", + "parse_action" + ], + "note": "Every action is validated on construction, before anything touches the machine.", + "kind": "terminal" + }, + { + "id": "vocab-refusals", + "category": "Action vocabulary", + "title": "Malformed actions are refused, not guessed at", + "shows": [ + "ActionError", + "Action", + "parse_action" + ], + "note": "An unknown verb or a missing target raises before the machine is touched.", + "kind": "terminal" + }, + { + "id": "vocab-menu", + "category": "Action vocabulary", + "title": "The contract handed to a model", + "shows": [ + "ACTION_MENU" + ], + "note": "The prompt-side menu: one JSON object, six shapes, no invented verbs.", + "kind": "terminal" + }, + { + "id": "menu-numbered", + "category": "Decisions", + "title": "A short numbered menu, in reading order", + "shows": [ + "DecisionMenu", + "DecisionMenu.render", + "MenuItem", + "MenuItem.label" + ], + "note": "A 256M model cannot emit coordinates; it can pick a number from twelve.", + "kind": "terminal" + }, + { + "id": "menu-lookup", + "category": "Decisions", + "title": "Menu lookup tolerates what models actually send", + "shows": [ + "DecisionMenu.item", + "DecisionMenu.find", + "DecisionMenu.mentioned_in" + ], + "note": "1, \"2\", 2.0 and \"Save.\" all resolve; a stray word resolves to nothing.", + "kind": "terminal" + }, + { + "id": "decision-json", + "category": "Decisions", + "title": "Reading a decision out of a small model's reply", + "shows": [ + "parse_decision", + "extract_json_object", + "extract_json_objects", + "DecisionMenu" + ], + "note": "The LAST JSON object wins, because a model that echoes its prompt answers last.", + "kind": "terminal" + }, + { + "id": "decision-echo-guard", + "category": "Decisions", + "title": "An echoed prompt is refused; a right answer in the wrong field is recovered", + "shows": [ + "parse_decision", + "BrainError", + "DecisionMenu.mentioned_in" + ], + "note": "Real 256M/0.5B failure modes, both caught: no blind click either way.", + "kind": "terminal" + }, + { + "id": "decision-garbled", + "category": "Decisions", + "title": "A garbled verb with a valid label is still usable", + "shows": [ + "parse_decision", + "BrainError" + ], + "note": "A 256M model really does emit {\"kind\": \"file\", \"label\": \"Save\"}.", + "kind": "terminal" + }, + { + "id": "exec-dry-run", + "category": "Execution", + "title": "Dry run resolves every click to pixels - and touches nothing", + "shows": [ + "DryRunExecutor", + "DryRunExecutor.execute", + "resolve_point", + "ActionResult" + ], + "note": "The default everywhere: read a whole episode before anything clicks.", + "kind": "terminal" + }, + { + "id": "exec-refuses-unresolved", + "category": "Execution", + "title": "Real input refuses to guess a pixel for a label it cannot find", + "shows": [ + "ActionResult.summary", + "InputUnavailable", + "PyAutoGuiExecutor", + "PyAutoGuiExecutor.execute" + ], + "note": "This is the gate between a hallucinated label and 'Delete account'.", + "kind": "terminal" + }, + { + "id": "loop-end-to-end", + "category": "The loop", + "title": "observe -> think -> act -> verify, traced end to end", + "shows": [ + "AgentLoop", + "AgentLoop.run", + "Episode", + "Episode.render", + "Episode.to_rows", + "Step", + "Step.line", + "Step.ok" + ], + "note": "One call runs a whole episode; every step is kept for audit.", + "kind": "terminal" + }, + { + "id": "loop-budget", + "category": "The loop", + "title": "A hard step budget, so a run cannot wander", + "shows": [ + "Policy", + "Policy.step_budget", + "AgentLoop.run" + ], + "note": "The budget ends the episode and says so; nothing executes past it.", + "kind": "terminal" + }, + { + "id": "loop-no-progress-guard", + "category": "The loop", + "title": "Repeating one action against an unchanged screen is stopped", + "shows": [ + "AgentLoop.run", + "Step", + "Step.error" + ], + "note": "A stuck agent is not a persistent one - it is stopped and told why.", + "kind": "terminal" + }, + { + "id": "loop-kill-switch", + "category": "The loop", + "title": "A kill-switch file aborts before the next action", + "shows": [ + "Policy.kill_switch_path", + "Policy.should_stop", + "AgentLoop.run" + ], + "note": "Checked before every single action: create the file, the run stops.", + "kind": "terminal" + }, + { + "id": "loop-survives-failures", + "category": "The loop", + "title": "A dead screen or a broken brain is an episode, not a traceback", + "shows": [ + "AgentLoop.run", + "Episode.stop_reason", + "GoalUnreachable", + "BrainError" + ], + "note": "Every failure path is recorded as a failed step with a reason.", + "kind": "terminal" + }, + { + "id": "loop-verifier", + "category": "The loop", + "title": "Success is evidence, not the model's word", + "shows": [ + "AgentLoop.run", + "Episode.success", + "Observation.text" + ], + "note": "Without a verifier, a successful action that removes its own target looks like a failure.", + "kind": "terminal" + }, + { + "id": "loop-on-step", + "category": "The loop", + "title": "Every step can be streamed out as it happens", + "shows": [ + "AgentLoop.run", + "Step.line", + "on_step" + ], + "note": "This is the hook a live console, a progress bar or a websocket uses.", + "kind": "terminal" + }, + { + "id": "safety-deny-text", + "category": "Safety", + "title": "Typing from the deny list never reaches the keyboard", + "shows": [ + "Policy", + "Policy.check", + "PolicyViolation", + "DEFAULT_DENY_TEXT" + ], + "note": "Checked before execution; a refusal ends the episode and clicks nothing.", + "kind": "terminal" + }, + { + "id": "safety-deny-label", + "category": "Safety", + "title": "Clicking a denied widget is refused, however the label is spelled", + "shows": [ + "Policy.deny_labels", + "Policy.check", + "Observation.find" + ], + "note": "Terminal, keychain, wallet, password manager, bank - matched on the label.", + "kind": "terminal" + }, + { + "id": "safety-deny-keys", + "category": "Safety", + "title": "Key chords on the deny list are never pressed", + "shows": [ + "Policy.deny_keys", + "Policy.check", + "PolicyViolation" + ], + "note": "ctrl+alt+t, ctrl+alt+delete and ctrl+shift+escape are refused by default.", + "kind": "terminal" + }, + { + "id": "safety-blind-grid", + "category": "Safety", + "title": "A numbered grid cell is a blind click: allowed to plan, refused for real", + "shows": [ + "Policy.allow_grid_clicks", + "Policy.check", + "GridDetector" + ], + "note": "With real input the blind guess is refused unless you pass --allow-grid.", + "kind": "terminal" + }, + { + "id": "screen-fake", + "category": "Screens", + "title": "A scripted screen, for testing the loop with no display", + "shows": [ + "FakeScreen", + "FakeScreen.capture", + "Observation" + ], + "note": "The final frame repeats forever, so a runaway agent cannot exhaust the script.", + "kind": "terminal" + }, + { + "id": "screen-image", + "category": "Screens", + "title": "A still image (or a sequence) replayed as a screen", + "shows": [ + "ImageScreen", + "ImageScreen.capture", + "fake_desktop" + ], + "note": "Lets the whole pipeline be exercised with no display, and a failure be replayed.", + "kind": "terminal" + }, + { + "id": "capture-options", + "category": "Screens", + "title": "Real capture: monitors, downscaling, and HiDPI mapping", + "shows": [ + "GrabScreen", + "GrabScreen.capture", + "GrabScreen.map_to_screen", + "ScreenUnavailable" + ], + "note": "Frame pixel -> screen pixel is preserved, so a click cannot drift.", + "kind": "terminal" + }, + { + "id": "element-matching", + "category": "Screens", + "title": "Labels are compared on their alphanumeric core", + "shows": [ + "Element", + "Element.center", + "Element.matches", + "Observation.find", + "normalise_label" + ], + "note": "OCR loses spaces, models add punctuation - so nothing matches by exact string.", + "kind": "terminal" + }, + { + "id": "observation-summary", + "category": "Screens", + "title": "The compact text a model is shown", + "shows": [ + "Observation.summary" + ], + "note": "This is the whole world as the brain sees it - capped, and honest when empty.", + "kind": "terminal" + }, + { + "id": "detector-available", + "category": "Perception", + "title": "Probing for a detector, and building one by name", + "shows": [ + "detector_available", + "make_detector", + "DetectorUnavailable", + "Detector" + ], + "note": "The doctor and the CLI both ask this before promising anything.", + "kind": "terminal" + }, + { + "id": "ocr-labels", + "category": "Perception", + "title": "OCR turns a screenshot into labelled, clickable elements", + "shows": [ + "OcrDetector", + "OcrDetector.detect", + "Element" + ], + "note": "The model never has to guess where anything is: this is the grounding.", + "kind": "terminal" + }, + { + "id": "ocr-filters", + "category": "Perception", + "title": "Three knobs keep dialog prose out of the menu", + "shows": [ + "OcrDetector.min_score", + "OcrDetector.max_label_chars", + "OcrDetector.min_label_chars" + ], + "note": "Every extra line of noise is one more chance for a small model to click prose.", + "kind": "terminal" + }, + { + "id": "grid-detector", + "category": "Perception", + "title": "A numbered grid, for screens with no text at all", + "shows": [ + "GridDetector", + "GridDetector.detect" + ], + "note": "An agent with an empty index has no legal moves; a grid always has moves.", + "kind": "terminal" + }, + { + "id": "auto-detector", + "category": "Perception", + "title": "OCR when there is text, a grid when there is none", + "shows": [ + "AutoDetector", + "AutoDetector.detect" + ], + "note": "The default: useful with nothing installed, better the moment OCR appears.", + "kind": "terminal" + }, + { + "id": "indexed-screen", + "category": "Perception", + "title": "IndexedScreen: capture + detect, composed", + "shows": [ + "IndexedScreen", + "IndexedScreen.capture", + "Observation.summary" + ], + "note": "One capture() call gives the loop an index it can act on.", + "kind": "terminal" + }, + { + "id": "display-available", + "category": "Preflight", + "title": "Is there a desktop session to drive at all?", + "shows": [ + "display_available" + ], + "note": "The first question the CLI and the doctor both ask.", + "kind": "terminal" + }, + { + "id": "run-checks", + "category": "Preflight", + "title": "Every precondition, one list, each with the fix", + "shows": [ + "run_checks", + "Check", + "Check.render" + ], + "note": "Eight checks; a required one that fails is a blocking problem.", + "kind": "terminal" + }, + { + "id": "cli-doctor", + "category": "Preflight", + "title": "python -m tiny_computer_agent --doctor", + "shows": [ + "detector_available", + "display_available", + "doctor", + "report" + ], + "note": "The first thing to run on a new machine. Exits non-zero when not ready.", + "kind": "terminal" + }, + { + "id": "build-agent", + "category": "Preflight", + "title": "One call wires screen + detector + brain + executor + policy", + "shows": [ + "build_agent", + "Policy", + "AgentLoop" + ], + "note": "Defaults are the safe ones: dry run, a step budget, and a kill-switch file.", + "kind": "terminal" + }, + { + "id": "models-deps", + "category": "Models", + "title": "Which model runtime is present, and are the weights cached?", + "shows": [ + "model_dependencies", + "model_cached", + "MODEL_BRAINS", + "make_model_brain", + "DEFAULT_TEXT_MODEL", + "DEFAULT_VISION_MODEL", + "BrainError" + ], + "note": "Weights are loaded lazily, so asking is free.", + "kind": "terminal" + }, + { + "id": "models-device", + "category": "Models", + "title": "Device and dtype selection", + "shows": [ + "resolve_device", + "pick_dtype" + ], + "note": "CUDA if it is there, CPU otherwise; the harness never needs a GPU.", + "kind": "terminal" + }, + { + "id": "models-prompt", + "category": "Models", + "title": "The prompt a model actually receives - and how it learns from a dead end", + "shows": [ + "ModelBrain", + "ModelBrain.build_prompt", + "ModelBrain.decide" + ], + "note": "Placeholders, not worked examples: a weak model echoes the prompt, and an echo fails validation.", + "kind": "terminal" + }, + { + "id": "models-prompt-history", + "category": "Models", + "title": "A second decision is told what already failed", + "shows": [ + "ModelBrain", + "ModelBrain.build_prompt", + "ModelBrain.history" + ], + "note": "'you already tried this and the screen did NOT change' is the most useful line a small model gets.", + "kind": "terminal" + }, + { + "id": "scenario", + "category": "Traces & scoring", + "title": "A task with one right answer, and the frames to show", + "shows": [ + "Scenario", + "Scenario.label", + "Scenario.screen", + "Trace.render", + "run_scenario" + ], + "note": "Ground truth lives with the task, so a run can be scored at all.", + "kind": "terminal" + }, + { + "id": "trace-record", + "category": "Traces & scoring", + "title": "An episode becomes a portable, pixel-free trace", + "shows": [ + "Trace", + "Trace.blocked", + "Trace.choice", + "Trace.decided", + "Trace.decision_step", + "Trace.frames", + "Trace.load", + "Trace.name", + "Trace.record", + "Trace.save", + "Trace.to_dict", + "Trace.to_json", + "TraceStep", + "TraceStep.observation", + "TraceStep.to_dict" + ], + "note": "JSON, so it diffs in review and replays on a machine with nothing installed.", + "kind": "terminal" + }, + { + "id": "trace-replay", + "category": "Traces & scoring", + "title": "Replay a recorded trace against a different brain", + "shows": [ + "Trace.replay", + "Trace.observations", + "score_trace" + ], + "note": "The frames are the fixture; the brain is the only variable. Nothing is captured or clicked.", + "kind": "terminal" + }, + { + "id": "score-trace", + "category": "Traces & scoring", + "title": "Counting how often the choice was right", + "shows": [ + "score_trace", + "score", + "score_replay", + "Verdict", + "Verdict.line" + ], + "note": "Scored on normalised text, so OCR spacing is forgiven - the same bargain the index makes.", + "kind": "terminal" + }, + { + "id": "scoreboard", + "category": "Traces & scoring", + "title": "Verdicts collected into an accuracy", + "shows": [ + "Scoreboard", + "Scoreboard.accuracy", + "Scoreboard.accuracy_for", + "Scoreboard.add", + "Scoreboard.blocked", + "Scoreboard.brains", + "Scoreboard.by_brain", + "Scoreboard.confusion", + "Scoreboard.earned", + "Scoreboard.render", + "Scoreboard.to_rows", + "Scoreboard.total" + ], + "note": "The number the whole module exists for: which brain is right, and how often.", + "kind": "terminal" + }, + { + "id": "benchmark", + "category": "Traces & scoring", + "title": "Every scenario through every brain, in one table", + "shows": [ + "benchmark", + "Scoreboard.to_rows" + ], + "note": "A fresh brain per task, so state like 'already clicked this' cannot leak between them.", + "kind": "terminal" + }, + { + "id": "benchmark-traces", + "category": "Traces & scoring", + "title": "Replay a whole recorded suite against one brain", + "shows": [ + "benchmark_traces", + "load_traces", + "Trace.save" + ], + "note": "The literal roadmap step: the frames are fixed, nothing new is captured.", + "kind": "terminal" + }, + { + "id": "cli-image", + "category": "Command line", + "title": "--image example (replay a screenshot, dry run)", + "shows": [ + "main", + "build_parser", + "ImageScreen", + "detector_available" + ], + "note": "No display needed: the whole pipeline over a still image.", + "kind": "terminal" + }, + { + "id": "cli-script", + "category": "Command line", + "title": "--script (replay a fixed action list instead of a brain)", + "shows": [ + "MockBrain", + "demo_screen", + "parse_action" + ], + "note": "The script is validated exactly like a model's answer: nothing skips the gates.", + "kind": "terminal" + }, + { + "id": "cli-verify-text", + "category": "Command line", + "title": "--verify-text (the harness decides success, not the brain)", + "shows": [ + "AgentLoop.run", + "Episode.success" + ], + "note": "Without it, an action that removes its own target looks like a failure.", + "kind": "terminal" + }, + { + "id": "cli-trace", + "category": "Command line", + "title": "--trace / --expected (record the run, then score it)", + "shows": [ + "Trace.record", + "Trace.save", + "score_trace" + ], + "note": "Writes JSON next to the code, and prints the score straight away.", + "kind": "terminal" + }, + { + "id": "cli-detector", + "category": "Command line", + "title": "--no-ocr (index a numbered grid instead)", + "shows": [ + "GridDetector", + "make_detector", + "detector_available" + ], + "note": "Useful on a machine with no OCR engine, or for a canvas app.", + "kind": "terminal" + }, + { + "id": "cli-exit-codes", + "category": "Command line", + "title": "Exit codes: 0 success, 1 failure, 2 bad usage, 3 no screen", + "shows": [ + "main" + ], + "note": "A shell can tell 'it worked' from 'it gave up' without parsing any text.", + "kind": "terminal" + }, + { + "id": "cli-refusals", + "category": "Command line", + "title": "The two deliberate acts before the mouse can move", + "shows": [ + "main", + "ALLOW_REAL_INPUT_ENV" + ], + "note": "--real-input without the env var, and --brain vlm with no pixels: both refuse.", + "kind": "terminal" + }, + { + "id": "cli-live-headless", + "category": "Command line", + "title": "--live with no display is refused, with the fix", + "shows": [ + "display_available", + "main" + ], + "note": "It tells you what to do instead of dying inside an episode.", + "kind": "terminal" + }, + { + "id": "cli-live-dry-run", + "category": "Command line", + "title": "--live (your real screen, dry run)", + "shows": [ + "GrabScreen", + "IndexedScreen", + "AutoDetector", + "main" + ], + "note": "Real capture, real OCR, a real element index - and not one click.", + "kind": "terminal" + }, + { + "id": "desk-capture", + "category": "Real desktop", + "title": "GrabScreen: capture a real desktop", + "shows": [ + "GrabScreen", + "GrabScreen.capture", + "ScreenUnavailable" + ], + "note": "1280x800 of a real X server. mss first, Pillow's ImageGrab as the fallback.", + "kind": "desktop" + }, + { + "id": "desk-ocr-index", + "category": "Real desktop", + "title": "The element index, drawn on the pixels it came from", + "shows": [ + "OcrDetector", + "OcrDetector.detect", + "IndexedScreen", + "Element.center" + ], + "note": "Every box is a real OCR word with a real screen coordinate: the model picks a label, not a pixel.", + "kind": "desktop" + }, + { + "id": "desk-click", + "category": "Real desktop", + "title": "A real click, and the window's own reaction", + "shows": [ + "PyAutoGuiExecutor", + "AgentLoop.run", + "Policy.dry_run", + "resolve_point" + ], + "note": "Pixels in, a label chosen, a pixel resolved, a button pressed - and the dialog closed itself.", + "kind": "desktop" + }, + { + "id": "desk-verify", + "category": "Real desktop", + "title": "Success only when the harness can see it", + "shows": [ + "AgentLoop.run", + "Episode.success", + "Episode.stop_reason" + ], + "note": "The app changed its own title and wrote a marker file; the verifier saw the change.", + "kind": "desktop" + }, + { + "id": "desk-grid-fallback", + "category": "Real desktop", + "title": "A screen with no text still has legal moves", + "shows": [ + "AutoDetector", + "GridDetector", + "OcrDetector" + ], + "note": "OCR finds nothing on a canvas, so the index degrades to numbered cells - and says so.", + "kind": "desktop" + }, + { + "id": "desk-policy-refusal", + "category": "Real desktop", + "title": "The safety gate, on a real button, in a real window", + "shows": [ + "Policy.check", + "Policy.deny_labels", + "PolicyViolation", + "Observation.find" + ], + "note": "The Terminal button is visible and labelled - and the policy refuses to press it.", + "kind": "desktop" + }, + { + "id": "desk-cli-real-input", + "category": "Real desktop", + "title": "The same click, driven end to end from the CLI", + "shows": [ + "main", + "PyAutoGuiExecutor", + "AgentLoop.run" + ], + "note": "--live --real-input --verify-text Saved: capture, OCR, click, verify, exit code.", + "kind": "desktop" + }, + { + "id": "protocols", + "category": "Extensibility", + "title": "Every stage is a Protocol: bring your own screen, brain, detector, executor", + "shows": [ + "Brain", + "Brain.decide", + "ScreenSource", + "ScreenSource.capture", + "Detector", + "Detector.detect", + "Executor", + "Executor.execute", + "ActionResult" + ], + "note": "The loop depends on four one-method contracts, not on the shipped implementations.", + "kind": "terminal" + }, + { + "id": "brain-mock", + "category": "Decisions", + "title": "MockBrain: replay a fixed script, exactly once", + "shows": [ + "MockBrain", + "MockBrain.decide", + "MockBrain.remaining" + ], + "note": "Deterministic and offline - the workhorse for tests, demos and replays.", + "kind": "terminal" + }, + { + "id": "brain-planner", + "category": "Decisions", + "title": "PlannerBrain: no weights, no download, and it gives up honestly", + "shows": [ + "PlannerBrain", + "PlannerBrain.decide", + "GoalUnreachable" + ], + "note": "The proof the harness works end to end, and a real fallback when no model is wanted.", + "kind": "terminal" + }, + { + "id": "models-swap", + "category": "Models", + "title": "The two real model backends, and lazy loading", + "shows": [ + "TextPlannerBrain", + "VLMBrain", + "TextPlannerBrain.unload", + "VLMBrain.unload", + "ModelBrain.loaded", + "make_model_brain" + ], + "note": "Constructing a brain costs nothing; the weights arrive on the first decide().", + "kind": "terminal" + }, + { + "id": "fake-desktop", + "category": "Screens", + "title": "The built-in synthetic desktop", + "shows": [ + "fake_desktop", + "write_fake_desktop", + "OcrDetector" + ], + "note": "Draws a screenshot-like desktop offline - the same image --image example replays.", + "kind": "terminal" + } +] \ No newline at end of file diff --git a/gallery/indexed-screen.png b/gallery/indexed-screen.png new file mode 100644 index 0000000000000000000000000000000000000000..0c6dedaf04391101e7df36c9b42d2ada650f52b9 Binary files /dev/null and b/gallery/indexed-screen.png differ diff --git a/gallery/loop-budget.png b/gallery/loop-budget.png new file mode 100644 index 0000000000000000000000000000000000000000..1b714e838feed1449eee1b109b82cf253f76d448 Binary files /dev/null and b/gallery/loop-budget.png differ diff --git a/gallery/loop-end-to-end.png b/gallery/loop-end-to-end.png new file mode 100644 index 0000000000000000000000000000000000000000..e3c68e3f478cf4c5c2e85726944fbdbc8e36c9b9 Binary files /dev/null and b/gallery/loop-end-to-end.png differ diff --git a/gallery/loop-kill-switch.png b/gallery/loop-kill-switch.png new file mode 100644 index 0000000000000000000000000000000000000000..c6315db29f15a779652a223772400b9586cb3305 Binary files /dev/null and b/gallery/loop-kill-switch.png differ diff --git a/gallery/loop-no-progress-guard.png b/gallery/loop-no-progress-guard.png new file mode 100644 index 0000000000000000000000000000000000000000..a47af486b56c556eb42689f466c9ef3abb35c6c9 Binary files /dev/null and b/gallery/loop-no-progress-guard.png differ diff --git a/gallery/loop-on-step.png b/gallery/loop-on-step.png new file mode 100644 index 0000000000000000000000000000000000000000..356e041a4ea47ab876a9b6f5973f175d345847d1 Binary files /dev/null and b/gallery/loop-on-step.png differ diff --git a/gallery/loop-survives-failures.png b/gallery/loop-survives-failures.png new file mode 100644 index 0000000000000000000000000000000000000000..0775c6104481ed5deaae2ea3ac6a2940d67d44c1 Binary files /dev/null and b/gallery/loop-survives-failures.png differ diff --git a/gallery/loop-verifier.png b/gallery/loop-verifier.png new file mode 100644 index 0000000000000000000000000000000000000000..c2aec9b8f877387da525d7b72679c161128b89df Binary files /dev/null and b/gallery/loop-verifier.png differ diff --git a/gallery/menu-lookup.png b/gallery/menu-lookup.png new file mode 100644 index 0000000000000000000000000000000000000000..4c02d78e6d4b33290377250661b763c70943a9e7 Binary files /dev/null and b/gallery/menu-lookup.png differ diff --git a/gallery/menu-numbered.png b/gallery/menu-numbered.png new file mode 100644 index 0000000000000000000000000000000000000000..edc2507e31f3f6ca36e278659b07d8182b28e22d Binary files /dev/null and b/gallery/menu-numbered.png differ diff --git a/gallery/models-deps.png b/gallery/models-deps.png new file mode 100644 index 0000000000000000000000000000000000000000..20b2ce5a8cfba105e3778089d04131dd6517ec25 Binary files /dev/null and b/gallery/models-deps.png differ diff --git a/gallery/models-device.png b/gallery/models-device.png new file mode 100644 index 0000000000000000000000000000000000000000..e8429fa06cfe9f0ddcd67ebf6ddff8e713d4c12c Binary files /dev/null and b/gallery/models-device.png differ diff --git a/gallery/models-prompt-history.png b/gallery/models-prompt-history.png new file mode 100644 index 0000000000000000000000000000000000000000..0790068653ca41cb678be2cd5ac053294af99442 Binary files /dev/null and b/gallery/models-prompt-history.png differ diff --git a/gallery/models-prompt.png b/gallery/models-prompt.png new file mode 100644 index 0000000000000000000000000000000000000000..141a31afcbf171ad58c6db777fd76c74faa25552 Binary files /dev/null and b/gallery/models-prompt.png differ diff --git a/gallery/models-swap.png b/gallery/models-swap.png new file mode 100644 index 0000000000000000000000000000000000000000..6c030e4e8ca505499351afd2b7b5b1341b86bef6 Binary files /dev/null and b/gallery/models-swap.png differ diff --git a/gallery/observation-summary.png b/gallery/observation-summary.png new file mode 100644 index 0000000000000000000000000000000000000000..d850ef785256667ba7995b6041395559b549eb31 Binary files /dev/null and b/gallery/observation-summary.png differ diff --git a/gallery/ocr-filters.png b/gallery/ocr-filters.png new file mode 100644 index 0000000000000000000000000000000000000000..adea4a1012309bc4ed269525df2f13895136b844 Binary files /dev/null and b/gallery/ocr-filters.png differ diff --git a/gallery/ocr-labels.png b/gallery/ocr-labels.png new file mode 100644 index 0000000000000000000000000000000000000000..bd50cc96a20fbf2c809f17879948af8acc181945 Binary files /dev/null and b/gallery/ocr-labels.png differ diff --git a/gallery/protocols.png b/gallery/protocols.png new file mode 100644 index 0000000000000000000000000000000000000000..c9c59966489f25f8bb302f6dbfe6242992ed99e2 Binary files /dev/null and b/gallery/protocols.png differ diff --git a/gallery/run-checks.png b/gallery/run-checks.png new file mode 100644 index 0000000000000000000000000000000000000000..d440405e6e78c2f04ec1e691a1a8dc700193712c Binary files /dev/null and b/gallery/run-checks.png differ diff --git a/gallery/safety-blind-grid.png b/gallery/safety-blind-grid.png new file mode 100644 index 0000000000000000000000000000000000000000..cf2f85da743b5c86e08244ae3011a243c690e618 Binary files /dev/null and b/gallery/safety-blind-grid.png differ diff --git a/gallery/safety-deny-keys.png b/gallery/safety-deny-keys.png new file mode 100644 index 0000000000000000000000000000000000000000..9ab827e22819765cda428abc2e036cc220baec18 Binary files /dev/null and b/gallery/safety-deny-keys.png differ diff --git a/gallery/safety-deny-label.png b/gallery/safety-deny-label.png new file mode 100644 index 0000000000000000000000000000000000000000..f3a4aef1b91da3c4818fbdd5d5a4dc226e9d5585 Binary files /dev/null and b/gallery/safety-deny-label.png differ diff --git a/gallery/safety-deny-text.png b/gallery/safety-deny-text.png new file mode 100644 index 0000000000000000000000000000000000000000..5644434f584b67df862909b47bf743df13726434 Binary files /dev/null and b/gallery/safety-deny-text.png differ diff --git a/gallery/scenario.png b/gallery/scenario.png new file mode 100644 index 0000000000000000000000000000000000000000..1de137182c539b99521667755baa1308307362e6 Binary files /dev/null and b/gallery/scenario.png differ diff --git a/gallery/score-trace.png b/gallery/score-trace.png new file mode 100644 index 0000000000000000000000000000000000000000..fe26e9fb97308dabc5e08404b6062d1c25b9523e Binary files /dev/null and b/gallery/score-trace.png differ diff --git a/gallery/scoreboard.png b/gallery/scoreboard.png new file mode 100644 index 0000000000000000000000000000000000000000..25dc800abba2186eae43bf195dd5216c97e963d4 Binary files /dev/null and b/gallery/scoreboard.png differ diff --git a/gallery/screen-fake.png b/gallery/screen-fake.png new file mode 100644 index 0000000000000000000000000000000000000000..20056ab74eedb016fb27d3862b56bf9ef60e3a4c Binary files /dev/null and b/gallery/screen-fake.png differ diff --git a/gallery/screen-image.png b/gallery/screen-image.png new file mode 100644 index 0000000000000000000000000000000000000000..f3fda22cd95e95266e40680ad0b5fc5f8766a9fa Binary files /dev/null and b/gallery/screen-image.png differ diff --git a/gallery/trace-record.png b/gallery/trace-record.png new file mode 100644 index 0000000000000000000000000000000000000000..61cbf05b03f734127021cf288cabc7893fe1228b Binary files /dev/null and b/gallery/trace-record.png differ diff --git a/gallery/trace-replay.png b/gallery/trace-replay.png new file mode 100644 index 0000000000000000000000000000000000000000..3048b9d037987225d42d9eda1a009530e5e02113 Binary files /dev/null and b/gallery/trace-replay.png differ diff --git a/gallery/vocab-menu.png b/gallery/vocab-menu.png new file mode 100644 index 0000000000000000000000000000000000000000..c2ab876f4f89caa845188dd3dceb05061f6b407c Binary files /dev/null and b/gallery/vocab-menu.png differ diff --git a/gallery/vocab-refusals.png b/gallery/vocab-refusals.png new file mode 100644 index 0000000000000000000000000000000000000000..4ed246b44aca25769446ee32202ee4b84673a260 Binary files /dev/null and b/gallery/vocab-refusals.png differ diff --git a/gallery/vocab-verbs.png b/gallery/vocab-verbs.png new file mode 100644 index 0000000000000000000000000000000000000000..57520efb2ccb923bdc1e6c103b309283efd83007 Binary files /dev/null and b/gallery/vocab-verbs.png differ