Spaces:
Runtime error
Runtime error
Commit ·
6dafaf5
1
Parent(s): c4fd26d
docs(plan): web LLM spectating — 7-task TDD implementation plan
Browse files
docs/superpowers/plans/2026-06-02-proteus-web-spectate.md
ADDED
|
@@ -0,0 +1,753 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# PROTEUS Web LLM Spectating Implementation Plan
|
| 2 |
+
|
| 3 |
+
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
| 4 |
+
|
| 5 |
+
**Goal:** Let a person watch an LLM play the arena in the browser (color grid + live reasoning/analysis), auto-advancing turn by turn with pause/step, always saving a `proteus compare`-compatible LLM trace.
|
| 6 |
+
|
| 7 |
+
**Architecture:** A new threadless `SpectateSession` (agent-driven sibling of `InteractiveSession`) advances exactly one LLM turn per HTTP request via the shared `_session_core`, and is pinned to `SessionRunner(VanillaAgent(...))` by a golden equivalence test. The stdlib `http.server` gains `/spectate*` routes; providers are **lazy-imported inside the create handler** so the offline import-safety invariant holds. The browser auto-loops `POST /spectate/{id}/next` (pause/step), showing each turn's grid + reasoning + optimal/habit/reward (rich disclosure — a spectator is not the scored subject).
|
| 8 |
+
|
| 9 |
+
**Tech Stack:** Python 3.12 stdlib (`http.server`) + existing providers/agents + pydantic v2. Vanilla HTML/JS (no build, no CDN). **Zero new dependencies** (providers already exist; lazy-imported — `.venv` import surface unchanged).
|
| 10 |
+
|
| 11 |
+
**Reference spec:** `docs/superpowers/specs/2026-06-02-proteus-web-spectate-design.md`
|
| 12 |
+
|
| 13 |
+
**Run tests with:** `.venv/bin/python -m pytest -q` (`python` is NOT on PATH; use the venv).
|
| 14 |
+
|
| 15 |
+
**IMPORTANT — isolation & parallelism:** Execute in an **isolated git worktree** (superpowers:using-git-worktrees) branched from current `master`. This plan's writable files (`proteus/runtime/spectate.py`, `proteus/web/server.py`, `proteus/web/static/index.html`, new spectate tests) are **disjoint** from the `pack_evade` plan's writable files (grid + runtime-core), so the two can be implemented in parallel on separate branches and merged without cross-conflict. `SpectateSession` reads `default_memory` via `getattr(..., None)` so it works whether or not pack_evade has merged yet.
|
| 16 |
+
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
## File Structure
|
| 20 |
+
|
| 21 |
+
| File | Responsibility |
|
| 22 |
+
|---|---|
|
| 23 |
+
| `proteus/runtime/spectate.py` | **Create.** `SpectateSession`: agent-driven `state()`/`advance()`/`finish()` over `_session_core`; rich (non-fair) live disclosure. |
|
| 24 |
+
| `proteus/web/server.py` | **Modify.** `/spectate`, `/spectate/{id}/next`, `/spectate/{id}` (GET), `/spectate/{id}/finish`; `/config` += providers/default_model; providers lazy-imported in the create handler. |
|
| 25 |
+
| `proteus/web/static/index.html` | **Modify.** Mode toggle (Play / Spectate), model input, analysis panel, auto-play/pause/step. |
|
| 26 |
+
| `tests/runtime/test_spectate.py` | **Create.** advance / rich live disclosure / finish / advance-after-done (FakeProvider, offline). |
|
| 27 |
+
| `tests/runtime/test_spectate_equivalence.py` | **Create.** Golden: `SpectateSession` trace == `SessionRunner(VanillaAgent(same FakeProvider))`. |
|
| 28 |
+
| `tests/web/test_spectate_routes.py` | **Create.** `/spectate` + `/next` + `/finish` + errors (fake:demo, no socket). |
|
| 29 |
+
| `tests/web/test_import_safety.py` | **Modify.** Extend the existing guard: spectate additions still pull no provider SDK at import. |
|
| 30 |
+
|
| 31 |
+
---
|
| 32 |
+
|
| 33 |
+
## Task 1: `SpectateSession` — agent-driven stepwise driver
|
| 34 |
+
|
| 35 |
+
**Files:**
|
| 36 |
+
- Create: `proteus/runtime/spectate.py`
|
| 37 |
+
- Test: `tests/runtime/test_spectate.py`
|
| 38 |
+
|
| 39 |
+
- [ ] **Step 1: Write the failing test**
|
| 40 |
+
|
| 41 |
+
```python
|
| 42 |
+
# tests/runtime/test_spectate.py
|
| 43 |
+
"""Unit tests for SpectateSession (agent-driven, stepwise, rich disclosure)."""
|
| 44 |
+
from __future__ import annotations
|
| 45 |
+
|
| 46 |
+
import pytest
|
| 47 |
+
|
| 48 |
+
import proteus.grid # noqa: F401
|
| 49 |
+
from proteus.agents.vanilla import VanillaAgent
|
| 50 |
+
from proteus.grid.difficulty import Difficulty
|
| 51 |
+
from proteus.providers.fake import FakeProvider
|
| 52 |
+
from proteus.runtime._session_core import SessionFinishedError
|
| 53 |
+
from proteus.runtime.spectate import SpectateSession
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
def _agent(n=10):
|
| 57 |
+
return VanillaAgent(FakeProvider(responses=["ACTION: stay"] * n, model_name="demo"))
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
def _new(play_turns=5):
|
| 61 |
+
return SpectateSession(
|
| 62 |
+
"predator_evade", agent=_agent(), model_name="fake:demo",
|
| 63 |
+
difficulty=Difficulty.EASY, seed=42, play_turns=play_turns, use_probe=False,
|
| 64 |
+
)
|
| 65 |
+
|
| 66 |
+
|
| 67 |
+
def test_initial_state_is_cut_intro_int_grid():
|
| 68 |
+
s = _new()
|
| 69 |
+
st = s.state()
|
| 70 |
+
assert st["phase"] == "cut_intro"
|
| 71 |
+
assert st["turn_idx"] == 0
|
| 72 |
+
assert isinstance(st["grid"][0][0], int)
|
| 73 |
+
assert st["turns_so_far"] == []
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
def test_advance_runs_one_llm_turn_with_rich_disclosure():
|
| 77 |
+
s = _new()
|
| 78 |
+
st = s.advance()
|
| 79 |
+
assert st["turn_idx"] == 1
|
| 80 |
+
last = st["turns_so_far"][-1]
|
| 81 |
+
# rich: optimal/habit/reward/reasoning ARE exposed live (spectator, not scored).
|
| 82 |
+
for k in ("action", "motive_action", "habit_action", "reward",
|
| 83 |
+
"is_diagnostic", "was_congruent", "reasoning"):
|
| 84 |
+
assert k in last
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
def test_advance_to_budget_then_done_and_finish():
|
| 88 |
+
s = _new(play_turns=2)
|
| 89 |
+
while s.state()["outcome"] is None:
|
| 90 |
+
s.advance()
|
| 91 |
+
st = s.state()
|
| 92 |
+
assert st["phase"] == "done" and st["outcome"] in ("survived", "eliminated")
|
| 93 |
+
trace = s.finish()
|
| 94 |
+
assert trace.model == "fake:demo" and trace.scenario == "predator_evade"
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
def test_advance_after_done_raises():
|
| 98 |
+
s = _new(play_turns=1)
|
| 99 |
+
s.advance()
|
| 100 |
+
with pytest.raises(SessionFinishedError):
|
| 101 |
+
s.advance()
|
| 102 |
+
```
|
| 103 |
+
|
| 104 |
+
- [ ] **Step 2: Run test to verify it fails**
|
| 105 |
+
|
| 106 |
+
Run: `.venv/bin/python -m pytest tests/runtime/test_spectate.py -q`
|
| 107 |
+
Expected: FAIL — `ModuleNotFoundError: proteus.runtime.spectate`.
|
| 108 |
+
|
| 109 |
+
- [ ] **Step 3: Write the implementation**
|
| 110 |
+
|
| 111 |
+
```python
|
| 112 |
+
# proteus/runtime/spectate.py
|
| 113 |
+
"""SpectateSession — a threadless, agent-driven driver for watching an LLM play.
|
| 114 |
+
|
| 115 |
+
Sibling of InteractiveSession, but the server (not a human) supplies each action
|
| 116 |
+
by calling the agent once per HTTP request. Built on the same ``_session_core``
|
| 117 |
+
helpers, so its trace matches SessionRunner for the same agent (pinned by
|
| 118 |
+
tests/runtime/test_spectate_equivalence.py).
|
| 119 |
+
|
| 120 |
+
Disclosure: a spectator is NOT the scored subject, so ``state()`` exposes the per
|
| 121 |
+
turn answer keys (optimal/habit), reward, and the model's reasoning LIVE. (The
|
| 122 |
+
model itself still only ever receives the observation passed to ``agent.act``.)
|
| 123 |
+
"""
|
| 124 |
+
from __future__ import annotations
|
| 125 |
+
|
| 126 |
+
from proteus.agents.base import Agent
|
| 127 |
+
from proteus.grid.difficulty import Difficulty
|
| 128 |
+
from proteus.runtime import _session_core as core
|
| 129 |
+
from proteus.runtime.trace import SessionTrace, TurnTrace
|
| 130 |
+
|
| 131 |
+
|
| 132 |
+
class SpectateSession:
|
| 133 |
+
def __init__(
|
| 134 |
+
self,
|
| 135 |
+
scenario_name: str,
|
| 136 |
+
*,
|
| 137 |
+
agent: Agent,
|
| 138 |
+
model_name: str,
|
| 139 |
+
difficulty: Difficulty = Difficulty.EASY,
|
| 140 |
+
seed: int | None = None,
|
| 141 |
+
play_turns: int = 15,
|
| 142 |
+
use_probe: bool = False,
|
| 143 |
+
motive_category: str = "survival",
|
| 144 |
+
) -> None:
|
| 145 |
+
self._scenario_name = scenario_name
|
| 146 |
+
self._agent = agent
|
| 147 |
+
self._model_name = model_name
|
| 148 |
+
self._difficulty = difficulty
|
| 149 |
+
self._seed = seed
|
| 150 |
+
self._play_turns = play_turns
|
| 151 |
+
self._use_probe = use_probe
|
| 152 |
+
self._motive_category = motive_category
|
| 153 |
+
|
| 154 |
+
built = core.build_session(scenario_name, seed, difficulty, play_turns)
|
| 155 |
+
self._scenario = built.scenario
|
| 156 |
+
self._game = built.game
|
| 157 |
+
self._cut_frames = built.cut_frames
|
| 158 |
+
self._cut_grids = built.cut_grids
|
| 159 |
+
# Forward-compatible with pack_evade's default_memory (absent -> None).
|
| 160 |
+
self._memory = getattr(built, "default_memory", None)
|
| 161 |
+
self._system_prompt = self._scenario.rules_text + core._HANDOVER_FRAMING
|
| 162 |
+
self._turns: list[TurnTrace] = []
|
| 163 |
+
self._trace: SessionTrace | None = None
|
| 164 |
+
|
| 165 |
+
def _is_done(self) -> bool:
|
| 166 |
+
return (
|
| 167 |
+
self._game.eliminated
|
| 168 |
+
or self._game.survived
|
| 169 |
+
or len(self._turns) >= self._play_turns
|
| 170 |
+
)
|
| 171 |
+
|
| 172 |
+
def state(self) -> dict:
|
| 173 |
+
done = self._is_done()
|
| 174 |
+
played = len(self._turns)
|
| 175 |
+
phase = "done" if done else ("cut_intro" if played == 0 else "play")
|
| 176 |
+
st: dict = {
|
| 177 |
+
"phase": phase,
|
| 178 |
+
"turn_idx": played,
|
| 179 |
+
"play_turns": self._play_turns,
|
| 180 |
+
"model": self._model_name,
|
| 181 |
+
"grid": core.grid_to_list(self._game.current_grid()),
|
| 182 |
+
"legend": {str(k): v for k, v in self._scenario.legend().items()},
|
| 183 |
+
"actions": list(core._ACTIONS),
|
| 184 |
+
"outcome": None,
|
| 185 |
+
"cut_frames": self._cut_grids if played == 0 else None,
|
| 186 |
+
# RICH disclosure (spectator): per-turn answer keys + reward + reasoning.
|
| 187 |
+
"turns_so_far": [
|
| 188 |
+
{
|
| 189 |
+
"turn_idx": t.turn_idx,
|
| 190 |
+
"action": t.action,
|
| 191 |
+
"motive_action": t.motive_action,
|
| 192 |
+
"habit_action": t.habit_action,
|
| 193 |
+
"reward": t.reward,
|
| 194 |
+
"is_diagnostic": t.is_diagnostic,
|
| 195 |
+
"was_congruent": t.was_congruent,
|
| 196 |
+
"reasoning": t.reasoning,
|
| 197 |
+
}
|
| 198 |
+
for t in self._turns
|
| 199 |
+
],
|
| 200 |
+
"review": None,
|
| 201 |
+
}
|
| 202 |
+
if done:
|
| 203 |
+
trace = self.finish()
|
| 204 |
+
st["outcome"] = trace.outcome
|
| 205 |
+
st["review"] = {"outcome": trace.outcome, "metrics": trace.metrics}
|
| 206 |
+
return st
|
| 207 |
+
|
| 208 |
+
def advance(self) -> dict:
|
| 209 |
+
if self._is_done():
|
| 210 |
+
raise core.SessionFinishedError("session already finished")
|
| 211 |
+
turn_idx = len(self._turns) + 1
|
| 212 |
+
observation = core.build_observation(
|
| 213 |
+
self._scenario, self._game, self._cut_frames, turn_idx,
|
| 214 |
+
memory=self._memory,
|
| 215 |
+
)
|
| 216 |
+
result = self._agent.act(observation, list(core._ACTIONS), self._system_prompt)
|
| 217 |
+
self._turns.append(core.make_turn_trace(
|
| 218 |
+
self._scenario, self._game,
|
| 219 |
+
turn_idx=turn_idx, observation=observation,
|
| 220 |
+
action=result.action, reasoning=result.reasoning, raw_text=result.raw_text,
|
| 221 |
+
input_tokens=result.input_tokens, output_tokens=result.output_tokens,
|
| 222 |
+
thinking_tokens=result.thinking_tokens,
|
| 223 |
+
))
|
| 224 |
+
return self.state()
|
| 225 |
+
|
| 226 |
+
def finish(self) -> SessionTrace:
|
| 227 |
+
if self._trace is not None:
|
| 228 |
+
return self._trace
|
| 229 |
+
self._trace = core.finalize(
|
| 230 |
+
self._scenario_name, self._scenario, self._game,
|
| 231 |
+
seed=self._seed, difficulty=self._difficulty,
|
| 232 |
+
play_turns=self._play_turns, turns=self._turns,
|
| 233 |
+
cut_frames=self._cut_frames, motive_category=self._motive_category,
|
| 234 |
+
model=self._model_name,
|
| 235 |
+
)
|
| 236 |
+
return self._trace
|
| 237 |
+
```
|
| 238 |
+
|
| 239 |
+
NOTE: `core.build_observation(..., memory=...)` is the CP7-merged signature already on
|
| 240 |
+
`master`. If `build_observation` in this tree does not yet accept `memory`, drop the
|
| 241 |
+
`memory=self._memory` kwarg (predator_evade does not use it); the pack_evade plan
|
| 242 |
+
adds/relies on it.
|
| 243 |
+
|
| 244 |
+
- [ ] **Step 4: Run test to verify it passes**
|
| 245 |
+
|
| 246 |
+
Run: `.venv/bin/python -m pytest tests/runtime/test_spectate.py -q`
|
| 247 |
+
Expected: PASS (4 passed).
|
| 248 |
+
|
| 249 |
+
- [ ] **Step 5: Commit**
|
| 250 |
+
|
| 251 |
+
```bash
|
| 252 |
+
git add proteus/runtime/spectate.py tests/runtime/test_spectate.py
|
| 253 |
+
git commit -m "feat(spectate): SpectateSession — agent-driven stepwise LLM driver"
|
| 254 |
+
```
|
| 255 |
+
|
| 256 |
+
---
|
| 257 |
+
|
| 258 |
+
## Task 2: Golden equivalence — SpectateSession == SessionRunner(VanillaAgent)
|
| 259 |
+
|
| 260 |
+
**Files:**
|
| 261 |
+
- Test: `tests/runtime/test_spectate_equivalence.py`
|
| 262 |
+
|
| 263 |
+
- [ ] **Step 1: Write the test**
|
| 264 |
+
|
| 265 |
+
```python
|
| 266 |
+
# tests/runtime/test_spectate_equivalence.py
|
| 267 |
+
"""Golden: the same scripted FakeProvider produces an identical SessionTrace
|
| 268 |
+
whether driven through SpectateSession (one advance per turn) or SessionRunner +
|
| 269 |
+
VanillaAgent (batch). Pins the spectate path to the runner path."""
|
| 270 |
+
from __future__ import annotations
|
| 271 |
+
|
| 272 |
+
import proteus.grid # noqa: F401
|
| 273 |
+
from proteus.agents.vanilla import VanillaAgent
|
| 274 |
+
from proteus.grid.difficulty import Difficulty
|
| 275 |
+
from proteus.providers.fake import FakeProvider
|
| 276 |
+
from proteus.runtime.session import SessionRunner
|
| 277 |
+
from proteus.runtime.spectate import SpectateSession
|
| 278 |
+
|
| 279 |
+
RESPONSES = ["ACTION: up", "ACTION: up", "ACTION: left",
|
| 280 |
+
"ACTION: stay", "ACTION: right", "ACTION: up"]
|
| 281 |
+
|
| 282 |
+
|
| 283 |
+
def _agent():
|
| 284 |
+
return VanillaAgent(FakeProvider(responses=list(RESPONSES), model_name="demo"))
|
| 285 |
+
|
| 286 |
+
|
| 287 |
+
def test_spectate_matches_session_runner():
|
| 288 |
+
runner = SessionRunner(
|
| 289 |
+
"predator_evade", _agent(), difficulty=Difficulty.EASY, seed=42,
|
| 290 |
+
play_turns=len(RESPONSES), use_probe=False,
|
| 291 |
+
)
|
| 292 |
+
batch_trace = runner.run()
|
| 293 |
+
|
| 294 |
+
sess = SpectateSession(
|
| 295 |
+
"predator_evade", agent=_agent(), model_name="demo",
|
| 296 |
+
difficulty=Difficulty.EASY, seed=42, play_turns=len(RESPONSES), use_probe=False,
|
| 297 |
+
)
|
| 298 |
+
while sess.state()["outcome"] is None:
|
| 299 |
+
sess.advance()
|
| 300 |
+
watch_trace = sess.finish()
|
| 301 |
+
|
| 302 |
+
assert watch_trace.model == batch_trace.model == "demo"
|
| 303 |
+
assert watch_trace.model_dump() == batch_trace.model_dump()
|
| 304 |
+
```
|
| 305 |
+
|
| 306 |
+
- [ ] **Step 2: Run the test**
|
| 307 |
+
|
| 308 |
+
Run: `.venv/bin/python -m pytest tests/runtime/test_spectate_equivalence.py -q`
|
| 309 |
+
Expected: PASS (1 passed). If a field differs, the spectate path diverges from the
|
| 310 |
+
runner — fix `spectate.py` (system_prompt, memory resolution, finalize args) until the
|
| 311 |
+
`model_dump()` is byte-identical. Do not weaken the assert.
|
| 312 |
+
|
| 313 |
+
- [ ] **Step 3: Commit**
|
| 314 |
+
|
| 315 |
+
```bash
|
| 316 |
+
git add tests/runtime/test_spectate_equivalence.py
|
| 317 |
+
git commit -m "test(spectate): golden — SpectateSession trace == SessionRunner(VanillaAgent)"
|
| 318 |
+
```
|
| 319 |
+
|
| 320 |
+
---
|
| 321 |
+
|
| 322 |
+
## Task 3: Server routes (`/spectate*`) + provider config
|
| 323 |
+
|
| 324 |
+
**Files:**
|
| 325 |
+
- Modify: `proteus/web/server.py`
|
| 326 |
+
- Test: `tests/web/test_spectate_routes.py`
|
| 327 |
+
|
| 328 |
+
- [ ] **Step 1: Write the failing test**
|
| 329 |
+
|
| 330 |
+
```python
|
| 331 |
+
# tests/web/test_spectate_routes.py
|
| 332 |
+
"""Route-level tests for spectating (offline via fake:demo, no socket)."""
|
| 333 |
+
from __future__ import annotations
|
| 334 |
+
|
| 335 |
+
import proteus.grid # noqa: F401
|
| 336 |
+
from proteus.web import server
|
| 337 |
+
|
| 338 |
+
|
| 339 |
+
def _reg():
|
| 340 |
+
return {}
|
| 341 |
+
|
| 342 |
+
|
| 343 |
+
def test_config_lists_providers_and_default_model():
|
| 344 |
+
_, payload, _ = server.handle_request("GET", "/config", None, _reg())
|
| 345 |
+
assert "fake" in payload["providers"]
|
| 346 |
+
assert payload["default_model"] == "fake:demo"
|
| 347 |
+
|
| 348 |
+
|
| 349 |
+
def test_spectate_create_next_finish(tmp_path):
|
| 350 |
+
reg = _reg()
|
| 351 |
+
status, created, _ = server.handle_request(
|
| 352 |
+
"POST", "/spectate",
|
| 353 |
+
{"scenario": "predator_evade", "difficulty": "easy", "seed": 42,
|
| 354 |
+
"play_turns": 3, "model": "fake:demo", "probe": False}, reg)
|
| 355 |
+
assert status == 200
|
| 356 |
+
sid = created["session_id"]
|
| 357 |
+
assert created["state"]["phase"] == "cut_intro"
|
| 358 |
+
|
| 359 |
+
# advance to terminal
|
| 360 |
+
last = None
|
| 361 |
+
for _ in range(3):
|
| 362 |
+
status, payload, _ = server.handle_request(
|
| 363 |
+
"POST", f"/spectate/{sid}/next", None, reg)
|
| 364 |
+
assert status == 200
|
| 365 |
+
last = payload["state"]
|
| 366 |
+
if last["outcome"] is not None:
|
| 367 |
+
break
|
| 368 |
+
assert last["turns_so_far"] # rich disclosure present
|
| 369 |
+
|
| 370 |
+
out = tmp_path / "spectate.jsonl"
|
| 371 |
+
status, payload, _ = server.handle_request(
|
| 372 |
+
"POST", f"/spectate/{sid}/finish", {"out": str(out)}, reg)
|
| 373 |
+
assert status == 200 and out.exists()
|
| 374 |
+
assert payload["trace_path"] == str(out) and "survival_fraction" in payload["metrics"]
|
| 375 |
+
|
| 376 |
+
|
| 377 |
+
def test_unknown_model_is_400():
|
| 378 |
+
status, payload, _ = server.handle_request(
|
| 379 |
+
"POST", "/spectate",
|
| 380 |
+
{"scenario": "predator_evade", "difficulty": "easy", "seed": 1,
|
| 381 |
+
"play_turns": 3, "model": "nope:x", "probe": False}, _reg())
|
| 382 |
+
assert status == 400 and "error" in payload
|
| 383 |
+
|
| 384 |
+
|
| 385 |
+
def test_next_after_done_is_409():
|
| 386 |
+
reg = _reg()
|
| 387 |
+
_, created, _ = server.handle_request(
|
| 388 |
+
"POST", "/spectate",
|
| 389 |
+
{"scenario": "predator_evade", "difficulty": "easy", "seed": 42,
|
| 390 |
+
"play_turns": 1, "model": "fake:demo", "probe": False}, reg)
|
| 391 |
+
sid = created["session_id"]
|
| 392 |
+
server.handle_request("POST", f"/spectate/{sid}/next", None, reg) # exhausts budget
|
| 393 |
+
status, payload, _ = server.handle_request("POST", f"/spectate/{sid}/next", None, reg)
|
| 394 |
+
assert status == 409 and "error" in payload
|
| 395 |
+
```
|
| 396 |
+
|
| 397 |
+
- [ ] **Step 2: Run test to verify it fails**
|
| 398 |
+
|
| 399 |
+
Run: `.venv/bin/python -m pytest tests/web/test_spectate_routes.py -q`
|
| 400 |
+
Expected: FAIL — `/config` lacks `providers`; `/spectate` route returns 404.
|
| 401 |
+
|
| 402 |
+
- [ ] **Step 3: Modify `proteus/web/server.py`**
|
| 403 |
+
|
| 404 |
+
Add lazy imports + the spectate handlers (providers/agents imported INSIDE the create
|
| 405 |
+
handler to keep the offline import-safety invariant). Extend `_config_payload`, and add
|
| 406 |
+
routes to `handle_request`.
|
| 407 |
+
|
| 408 |
+
Add these handler functions (above `handle_request`):
|
| 409 |
+
|
| 410 |
+
```python
|
| 411 |
+
def _create_spectate(body: dict, registry: dict) -> Response:
|
| 412 |
+
# Lazy imports keep `import proteus.web.server` free of provider SDKs.
|
| 413 |
+
from proteus.agents.vanilla import VanillaAgent
|
| 414 |
+
from proteus.providers import make_provider
|
| 415 |
+
from proteus.runtime.spectate import SpectateSession
|
| 416 |
+
try:
|
| 417 |
+
scenario = body["scenario"]
|
| 418 |
+
difficulty = Difficulty(body.get("difficulty", "easy"))
|
| 419 |
+
seed = body.get("seed", _DEFAULT_SEED)
|
| 420 |
+
seed = None if seed in ("", None) else int(seed)
|
| 421 |
+
play_turns = int(body.get("play_turns", 10))
|
| 422 |
+
model = body.get("model", "fake:demo")
|
| 423 |
+
probe = bool(body.get("probe", False))
|
| 424 |
+
except (KeyError, ValueError) as exc:
|
| 425 |
+
return _err(400, f"bad spectate params: {exc}")
|
| 426 |
+
if scenario not in list_scenarios():
|
| 427 |
+
return _err(400, f"unknown scenario {scenario!r}")
|
| 428 |
+
try:
|
| 429 |
+
provider = make_provider(model)
|
| 430 |
+
except ValueError as exc:
|
| 431 |
+
return _err(400, str(exc))
|
| 432 |
+
agent = VanillaAgent(provider)
|
| 433 |
+
session = SpectateSession(
|
| 434 |
+
scenario, agent=agent, model_name=provider.model_name,
|
| 435 |
+
difficulty=difficulty, seed=seed, play_turns=play_turns, use_probe=probe,
|
| 436 |
+
)
|
| 437 |
+
sid = uuid.uuid4().hex
|
| 438 |
+
registry[sid] = session
|
| 439 |
+
return 200, {"session_id": sid, "state": session.state()}, _JSON
|
| 440 |
+
|
| 441 |
+
|
| 442 |
+
def _spectate_next(sid: str, registry: dict) -> Response:
|
| 443 |
+
session = registry.get(sid)
|
| 444 |
+
if session is None:
|
| 445 |
+
return _err(404, f"unknown session {sid!r}")
|
| 446 |
+
try:
|
| 447 |
+
state = session.advance()
|
| 448 |
+
except core.SessionFinishedError:
|
| 449 |
+
return _err(409, "session already finished")
|
| 450 |
+
except Exception as exc: # provider failure (missing key / network): no stack leak
|
| 451 |
+
return _err(500, f"agent step failed: {exc}")
|
| 452 |
+
return 200, {"state": state}, _JSON
|
| 453 |
+
|
| 454 |
+
|
| 455 |
+
def _spectate_finish(sid: str, body: dict, registry: dict) -> Response:
|
| 456 |
+
session = registry.get(sid)
|
| 457 |
+
if session is None:
|
| 458 |
+
return _err(404, f"unknown session {sid!r}")
|
| 459 |
+
try:
|
| 460 |
+
trace = session.finish()
|
| 461 |
+
except core.SessionNotFinishedError:
|
| 462 |
+
return _err(409, "session not finished")
|
| 463 |
+
out = (body or {}).get("out") or f"runs/web_spectate_{trace.scenario}_{trace.difficulty}.jsonl"
|
| 464 |
+
written = append_trace(trace, out)
|
| 465 |
+
return 200, {"trace_path": str(written), "metrics": trace.metrics}, _JSON
|
| 466 |
+
```
|
| 467 |
+
|
| 468 |
+
Extend `_config_payload` to add providers + default model:
|
| 469 |
+
|
| 470 |
+
```python
|
| 471 |
+
def _config_payload() -> dict:
|
| 472 |
+
from proteus.providers import available_providers # local: keep module import light
|
| 473 |
+
return {
|
| 474 |
+
"scenarios": list_scenarios(),
|
| 475 |
+
"difficulties": [d.value for d in Difficulty],
|
| 476 |
+
"color_map": {str(k): v for k, v in COLOR_MAP.items()},
|
| 477 |
+
"default_seed": _DEFAULT_SEED,
|
| 478 |
+
"providers": available_providers(),
|
| 479 |
+
"default_model": "fake:demo",
|
| 480 |
+
}
|
| 481 |
+
```
|
| 482 |
+
|
| 483 |
+
Add the routes in `handle_request` (after the `/session/` block, before the final 404):
|
| 484 |
+
|
| 485 |
+
```python
|
| 486 |
+
if method == "POST" and path == "/spectate":
|
| 487 |
+
return _create_spectate(body or {}, registry)
|
| 488 |
+
if path.startswith("/spectate/"):
|
| 489 |
+
rest = path[len("/spectate/"):]
|
| 490 |
+
if method == "POST" and rest.endswith("/next"):
|
| 491 |
+
return _spectate_next(rest[: -len("/next")], registry)
|
| 492 |
+
if method == "POST" and rest.endswith("/finish"):
|
| 493 |
+
return _spectate_finish(rest[: -len("/finish")], body or {}, registry)
|
| 494 |
+
if method == "GET":
|
| 495 |
+
session = registry.get(rest)
|
| 496 |
+
if session is None:
|
| 497 |
+
return _err(404, f"unknown session {rest!r}")
|
| 498 |
+
return 200, {"state": session.state()}, _JSON
|
| 499 |
+
```
|
| 500 |
+
|
| 501 |
+
(`available_providers` / `make_provider` are imported **locally** so the top-level
|
| 502 |
+
`import proteus.web.server` still pulls no provider SDK — Task 6 guards this.)
|
| 503 |
+
|
| 504 |
+
- [ ] **Step 4: Run test to verify it passes**
|
| 505 |
+
|
| 506 |
+
Run: `.venv/bin/python -m pytest tests/web/test_spectate_routes.py -q`
|
| 507 |
+
Expected: PASS (5 passed).
|
| 508 |
+
|
| 509 |
+
- [ ] **Step 5: Commit**
|
| 510 |
+
|
| 511 |
+
```bash
|
| 512 |
+
git add proteus/web/server.py tests/web/test_spectate_routes.py
|
| 513 |
+
git commit -m "feat(spectate): /spectate routes + provider config (lazy-imported)"
|
| 514 |
+
```
|
| 515 |
+
|
| 516 |
+
---
|
| 517 |
+
|
| 518 |
+
## Task 4: Frontend — spectate mode (toggle, analysis panel, auto-play/pause/step)
|
| 519 |
+
|
| 520 |
+
**Files:**
|
| 521 |
+
- Modify: `proteus/web/static/index.html`
|
| 522 |
+
|
| 523 |
+
- [ ] **Step 1: Add a mode toggle + model input to the start screen**
|
| 524 |
+
|
| 525 |
+
In the start-screen `.row` (after the `probe` checkbox `<div>`), add:
|
| 526 |
+
|
| 527 |
+
```html
|
| 528 |
+
<div><label>mode</label><select id="mode">
|
| 529 |
+
<option value="play">Play (human)</option>
|
| 530 |
+
<option value="spectate">Spectate (LLM)</option>
|
| 531 |
+
</select></div>
|
| 532 |
+
<div id="modelWrap" class="hidden"><label>model</label>
|
| 533 |
+
<input id="model" value="fake:demo" style="width:140px"></div>
|
| 534 |
+
```
|
| 535 |
+
|
| 536 |
+
- [ ] **Step 2: Add a spectate analysis panel to the play screen**
|
| 537 |
+
|
| 538 |
+
After the `#grid` div inside `#play`, add:
|
| 539 |
+
|
| 540 |
+
```html
|
| 541 |
+
<div id="analysis" class="hidden">
|
| 542 |
+
<div id="thinking" style="color:var(--muted)"></div>
|
| 543 |
+
<div id="reasoning" style="white-space:pre-wrap;margin:8px 0"></div>
|
| 544 |
+
<table id="turnlog"><tr><th>turn</th><th>action</th><th>optimal</th>
|
| 545 |
+
<th>habit</th><th>reward</th><th>=opt?</th></tr></table>
|
| 546 |
+
<p style="text-align:center;margin-top:10px">
|
| 547 |
+
<button id="pauseBtn">Pause</button>
|
| 548 |
+
<button id="stepBtn">Step</button></p>
|
| 549 |
+
</div>
|
| 550 |
+
```
|
| 551 |
+
|
| 552 |
+
- [ ] **Step 3: Add the spectate JS (append inside the existing `<script>`)**
|
| 553 |
+
|
| 554 |
+
```javascript
|
| 555 |
+
let SPECT = false, PAUSED = false;
|
| 556 |
+
|
| 557 |
+
function showModelInput() {
|
| 558 |
+
$("modelWrap").classList.toggle("hidden", $("mode").value !== "spectate");
|
| 559 |
+
}
|
| 560 |
+
|
| 561 |
+
function appendTurn(t) {
|
| 562 |
+
const tr = document.createElement("tr");
|
| 563 |
+
if (t.is_diagnostic) tr.className = "diag";
|
| 564 |
+
tr.innerHTML = `<td>${t.turn_idx}</td><td>${t.action}</td><td>${t.motive_action}</td>`
|
| 565 |
+
+ `<td>${t.habit_action}</td><td>${t.reward.toFixed(1)}</td>`
|
| 566 |
+
+ `<td>${t.was_congruent ? '✓':'✗'}</td>`;
|
| 567 |
+
$("turnlog").appendChild(tr);
|
| 568 |
+
}
|
| 569 |
+
|
| 570 |
+
function showSpectateState(st) {
|
| 571 |
+
$("turnInfo").textContent = `Turn ${st.turn_idx} / ${st.play_turns}`;
|
| 572 |
+
drawGrid(st.grid);
|
| 573 |
+
const last = st.turns_so_far[st.turns_so_far.length - 1];
|
| 574 |
+
if (last) { $("reasoning").textContent = last.reasoning || ""; appendTurn(last); }
|
| 575 |
+
}
|
| 576 |
+
|
| 577 |
+
async function spectateLoop() {
|
| 578 |
+
while (SPECT && !PAUSED) {
|
| 579 |
+
$("thinking").textContent = "thinking…";
|
| 580 |
+
const {status, data} = await jpost(`/spectate/${SID}/next`, {});
|
| 581 |
+
$("thinking").textContent = "";
|
| 582 |
+
if (status === 409) break;
|
| 583 |
+
if (status !== 200) { $("thinking").textContent = data.error || "error"; SPECT = false; break; }
|
| 584 |
+
showSpectateState(data.state);
|
| 585 |
+
if (data.state.outcome) {
|
| 586 |
+
const fin = await jpost(`/spectate/${SID}/finish`, {});
|
| 587 |
+
if (fin.status === 200) $("savedPath").textContent = "trace saved: " + fin.data.trace_path;
|
| 588 |
+
$("play").classList.add("hidden"); $("end").classList.remove("hidden");
|
| 589 |
+
const o = $("outcome");
|
| 590 |
+
o.textContent = data.state.outcome === "survived" ? "SURVIVED" : "ELIMINATED";
|
| 591 |
+
o.className = "outcome " + (data.state.outcome === "survived" ? "win" : "lose");
|
| 592 |
+
SPECT = false; break;
|
| 593 |
+
}
|
| 594 |
+
}
|
| 595 |
+
}
|
| 596 |
+
|
| 597 |
+
async function startSpectate(body) {
|
| 598 |
+
const {status, data} = await jpost("/spectate", body);
|
| 599 |
+
if (status !== 200) { alert(data.error || "failed to start"); return; }
|
| 600 |
+
SID = data.session_id; SPECT = true; PAUSED = false;
|
| 601 |
+
$("turnlog").querySelectorAll("tr:not(:first-child)").forEach(r => r.remove());
|
| 602 |
+
$("start").classList.add("hidden"); $("end").classList.add("hidden");
|
| 603 |
+
$("play").classList.remove("hidden"); $("analysis").classList.remove("hidden"); $("pad")?.classList?.add?.("hidden");
|
| 604 |
+
$("meta").textContent = `${body.scenario} · ${body.difficulty} · ${body.model}`;
|
| 605 |
+
const st = data.state;
|
| 606 |
+
if (st.cut_frames) await animateCut(st.cut_frames);
|
| 607 |
+
drawGrid(st.grid);
|
| 608 |
+
spectateLoop();
|
| 609 |
+
}
|
| 610 |
+
```
|
| 611 |
+
|
| 612 |
+
- [ ] **Step 4: Wire the controls in `startGame`/`init`**
|
| 613 |
+
|
| 614 |
+
In `init()`, add:
|
| 615 |
+
|
| 616 |
+
```javascript
|
| 617 |
+
$("mode").addEventListener("change", showModelInput);
|
| 618 |
+
$("pauseBtn").addEventListener("click", () => {
|
| 619 |
+
PAUSED = !PAUSED; $("pauseBtn").textContent = PAUSED ? "Resume" : "Pause";
|
| 620 |
+
if (!PAUSED) spectateLoop();
|
| 621 |
+
});
|
| 622 |
+
$("stepBtn").addEventListener("click", async () => {
|
| 623 |
+
if (!SPECT) return;
|
| 624 |
+
const {status, data} = await jpost(`/spectate/${SID}/next`, {});
|
| 625 |
+
if (status === 200) { showSpectateState(data.state);
|
| 626 |
+
if (data.state.outcome) { await jpost(`/spectate/${SID}/finish`, {}); SPECT = false; } }
|
| 627 |
+
});
|
| 628 |
+
```
|
| 629 |
+
|
| 630 |
+
In `startGame()`, branch on mode at the top:
|
| 631 |
+
|
| 632 |
+
```javascript
|
| 633 |
+
if ($("mode").value === "spectate") {
|
| 634 |
+
return startSpectate({
|
| 635 |
+
scenario: $("scenario").value, difficulty: $("difficulty").value,
|
| 636 |
+
seed: Number($("seed").value), play_turns: Number($("turns").value),
|
| 637 |
+
model: $("model").value, probe: $("probe").checked,
|
| 638 |
+
});
|
| 639 |
+
}
|
| 640 |
+
// ... existing human-play body unchanged ...
|
| 641 |
+
```
|
| 642 |
+
|
| 643 |
+
Also hide the human direction pad in spectate mode (give the `.pad` div `id="pad"` if it
|
| 644 |
+
has none, or guard the `.pad` listeners with `if (SPECT) return;` inside `act`).
|
| 645 |
+
|
| 646 |
+
- [ ] **Step 5: Verify the server byte test still passes + manual check**
|
| 647 |
+
|
| 648 |
+
Run: `.venv/bin/python -m pytest tests/web/test_server.py::test_root_serves_html_bytes -q`
|
| 649 |
+
Expected: PASS.
|
| 650 |
+
Manual: `.venv/bin/python -m proteus.web --port 8124`, open `http://127.0.0.1:8124/`,
|
| 651 |
+
pick **Spectate (LLM)**, model `fake:demo` → grid auto-advances with the analysis table
|
| 652 |
+
+ pause/step working → on end, SURVIVED/ELIMINATED + "trace saved". Stop with Ctrl-C.
|
| 653 |
+
|
| 654 |
+
- [ ] **Step 6: Commit**
|
| 655 |
+
|
| 656 |
+
```bash
|
| 657 |
+
git add proteus/web/static/index.html
|
| 658 |
+
git commit -m "feat(spectate): frontend — mode toggle + live analysis + auto-play/pause/step"
|
| 659 |
+
```
|
| 660 |
+
|
| 661 |
+
---
|
| 662 |
+
|
| 663 |
+
## Task 5: End-to-end + full suite
|
| 664 |
+
|
| 665 |
+
**Files:** none (verification only)
|
| 666 |
+
|
| 667 |
+
- [ ] **Step 1: Full suite**
|
| 668 |
+
|
| 669 |
+
Run: `.venv/bin/python -m pytest -q`
|
| 670 |
+
Expected: PASS — all existing + new spectate tests, zero failures.
|
| 671 |
+
|
| 672 |
+
- [ ] **Step 2: End-to-end — a spectated (fake) trace is replayable + comparable**
|
| 673 |
+
|
| 674 |
+
```bash
|
| 675 |
+
.venv/bin/python -m proteus.web --port 8124 & SRV=$!; sleep 1.5
|
| 676 |
+
SID=$(curl -s -X POST http://127.0.0.1:8124/spectate -H 'Content-Type: application/json' \
|
| 677 |
+
-d '{"scenario":"predator_evade","difficulty":"easy","seed":42,"play_turns":3,"model":"fake:demo","probe":false}' \
|
| 678 |
+
| .venv/bin/python -c "import sys,json;print(json.load(sys.stdin)['session_id'])")
|
| 679 |
+
for i in 1 2 3; do curl -s -X POST http://127.0.0.1:8124/spectate/$SID/next >/dev/null; done
|
| 680 |
+
curl -s -X POST http://127.0.0.1:8124/spectate/$SID/finish \
|
| 681 |
+
-H 'Content-Type: application/json' -d '{"out":"runs/spectate_e2e.jsonl"}'; echo
|
| 682 |
+
kill $SRV
|
| 683 |
+
.venv/bin/python -m proteus replay runs/spectate_e2e.jsonl
|
| 684 |
+
.venv/bin/python -m proteus compare runs/spectate_e2e.jsonl
|
| 685 |
+
rm -f runs/spectate_e2e.jsonl
|
| 686 |
+
```
|
| 687 |
+
Expected: `/finish` returns `{trace_path, metrics}`; `replay`/`compare` accept the
|
| 688 |
+
trace as a `model=fake` group.
|
| 689 |
+
|
| 690 |
+
---
|
| 691 |
+
|
| 692 |
+
## Task 6: Offline import-safety still holds
|
| 693 |
+
|
| 694 |
+
**Files:**
|
| 695 |
+
- Modify: `tests/web/test_import_safety.py`
|
| 696 |
+
|
| 697 |
+
- [ ] **Step 1: Extend the guard**
|
| 698 |
+
|
| 699 |
+
Append to the existing test body (after the current asserts) — re-clearing and
|
| 700 |
+
re-importing already happens; just confirm the spectate additions did not pull an SDK:
|
| 701 |
+
|
| 702 |
+
```python
|
| 703 |
+
# spectate routes import providers LAZILY (inside the handler), so a plain
|
| 704 |
+
# import of the web layer must still pull no provider SDK.
|
| 705 |
+
import proteus.web.server # noqa: F401 (re-exercised after the clear above)
|
| 706 |
+
for sdk in ("openai", "anthropic", "google", "ollama"):
|
| 707 |
+
assert sdk not in sys.modules
|
| 708 |
+
```
|
| 709 |
+
|
| 710 |
+
- [ ] **Step 2: Run it**
|
| 711 |
+
|
| 712 |
+
Run: `.venv/bin/python -m pytest tests/web/test_import_safety.py -q`
|
| 713 |
+
Expected: PASS (1 passed).
|
| 714 |
+
|
| 715 |
+
- [ ] **Step 3: Commit**
|
| 716 |
+
|
| 717 |
+
```bash
|
| 718 |
+
git add tests/web/test_import_safety.py
|
| 719 |
+
git commit -m "test(spectate): offline import-safety still holds (providers lazy)"
|
| 720 |
+
```
|
| 721 |
+
|
| 722 |
+
---
|
| 723 |
+
|
| 724 |
+
## Task 7: Update HANDOFF.md
|
| 725 |
+
|
| 726 |
+
**Files:**
|
| 727 |
+
- Modify: `HANDOFF.md`
|
| 728 |
+
|
| 729 |
+
- [ ] **Step 1: Add a "Done (Web LLM spectating)" section**
|
| 730 |
+
|
| 731 |
+
Summarize: watch an LLM play in the browser (color grid + live reasoning/analysis,
|
| 732 |
+
auto-play with pause/step); `SpectateSession` (agent-driven, `_session_core`-shared)
|
| 733 |
+
pinned to `SessionRunner(VanillaAgent)` by a golden test; real providers + `fake:demo`
|
| 734 |
+
fallback, **lazy-imported** (offline import-safety preserved); spectated runs save a
|
| 735 |
+
`runs/web_spectate_*.jsonl` LLM-baseline trace accepted by `proteus replay`/`compare`.
|
| 736 |
+
Update the full-suite count.
|
| 737 |
+
|
| 738 |
+
- [ ] **Step 2: Commit**
|
| 739 |
+
|
| 740 |
+
```bash
|
| 741 |
+
git add HANDOFF.md
|
| 742 |
+
git commit -m "docs(spectate): handoff — watch an LLM play in color shipped"
|
| 743 |
+
```
|
| 744 |
+
|
| 745 |
+
---
|
| 746 |
+
|
| 747 |
+
## Self-Review notes (already reconciled)
|
| 748 |
+
|
| 749 |
+
- **Spec coverage:** §3 components → Tasks 1,3,4; §4 SpectateSession → Task 1; §5 golden → Task 2; §6 API → Task 3; §7 frontend → Task 4; §8 trace → Tasks 3,5; §9 testing → Tasks 1,2,3,5,6; §10 acceptance → Task 5.
|
| 750 |
+
- **Parallel-safe vs pack_evade:** writable files here (`spectate.py`, `web/server.py`, `web/static/index.html`, spectate tests) are disjoint from pack_evade's (grid + runtime-core). `SpectateSession` reads `default_memory` via `getattr(..., None)` and passes `memory=` only if `build_observation` accepts it, so it builds and tests green whether pack_evade has merged or not.
|
| 751 |
+
- **Offline invariant:** providers/agents imported only inside `_create_spectate` (and `available_providers` locally in `_config_payload`); Task 6 guards that `import proteus.web.server` pulls no SDK.
|
| 752 |
+
- **Type consistency:** `SpectateSession(agent=, model_name=)`; `advance()`/`state()`/`finish()` mirror the `_session_core` calls used by `InteractiveSession`/`SessionRunner`; `state()["turns_so_far"]` keys match the frontend `appendTurn`/`showSpectateState` reads; route handler names (`_create_spectate`/`_spectate_next`/`_spectate_finish`) match their `handle_request` dispatch.
|
| 753 |
+
```
|