irregular6612 commited on
Commit
6dafaf5
·
1 Parent(s): c4fd26d

docs(plan): web LLM spectating — 7-task TDD implementation plan

Browse files
docs/superpowers/plans/2026-06-02-proteus-web-spectate.md ADDED
@@ -0,0 +1,753 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # PROTEUS Web LLM Spectating Implementation Plan
2
+
3
+ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
+
5
+ **Goal:** Let a person watch an LLM play the arena in the browser (color grid + live reasoning/analysis), auto-advancing turn by turn with pause/step, always saving a `proteus compare`-compatible LLM trace.
6
+
7
+ **Architecture:** A new threadless `SpectateSession` (agent-driven sibling of `InteractiveSession`) advances exactly one LLM turn per HTTP request via the shared `_session_core`, and is pinned to `SessionRunner(VanillaAgent(...))` by a golden equivalence test. The stdlib `http.server` gains `/spectate*` routes; providers are **lazy-imported inside the create handler** so the offline import-safety invariant holds. The browser auto-loops `POST /spectate/{id}/next` (pause/step), showing each turn's grid + reasoning + optimal/habit/reward (rich disclosure — a spectator is not the scored subject).
8
+
9
+ **Tech Stack:** Python 3.12 stdlib (`http.server`) + existing providers/agents + pydantic v2. Vanilla HTML/JS (no build, no CDN). **Zero new dependencies** (providers already exist; lazy-imported — `.venv` import surface unchanged).
10
+
11
+ **Reference spec:** `docs/superpowers/specs/2026-06-02-proteus-web-spectate-design.md`
12
+
13
+ **Run tests with:** `.venv/bin/python -m pytest -q` (`python` is NOT on PATH; use the venv).
14
+
15
+ **IMPORTANT — isolation & parallelism:** Execute in an **isolated git worktree** (superpowers:using-git-worktrees) branched from current `master`. This plan's writable files (`proteus/runtime/spectate.py`, `proteus/web/server.py`, `proteus/web/static/index.html`, new spectate tests) are **disjoint** from the `pack_evade` plan's writable files (grid + runtime-core), so the two can be implemented in parallel on separate branches and merged without cross-conflict. `SpectateSession` reads `default_memory` via `getattr(..., None)` so it works whether or not pack_evade has merged yet.
16
+
17
+ ---
18
+
19
+ ## File Structure
20
+
21
+ | File | Responsibility |
22
+ |---|---|
23
+ | `proteus/runtime/spectate.py` | **Create.** `SpectateSession`: agent-driven `state()`/`advance()`/`finish()` over `_session_core`; rich (non-fair) live disclosure. |
24
+ | `proteus/web/server.py` | **Modify.** `/spectate`, `/spectate/{id}/next`, `/spectate/{id}` (GET), `/spectate/{id}/finish`; `/config` += providers/default_model; providers lazy-imported in the create handler. |
25
+ | `proteus/web/static/index.html` | **Modify.** Mode toggle (Play / Spectate), model input, analysis panel, auto-play/pause/step. |
26
+ | `tests/runtime/test_spectate.py` | **Create.** advance / rich live disclosure / finish / advance-after-done (FakeProvider, offline). |
27
+ | `tests/runtime/test_spectate_equivalence.py` | **Create.** Golden: `SpectateSession` trace == `SessionRunner(VanillaAgent(same FakeProvider))`. |
28
+ | `tests/web/test_spectate_routes.py` | **Create.** `/spectate` + `/next` + `/finish` + errors (fake:demo, no socket). |
29
+ | `tests/web/test_import_safety.py` | **Modify.** Extend the existing guard: spectate additions still pull no provider SDK at import. |
30
+
31
+ ---
32
+
33
+ ## Task 1: `SpectateSession` — agent-driven stepwise driver
34
+
35
+ **Files:**
36
+ - Create: `proteus/runtime/spectate.py`
37
+ - Test: `tests/runtime/test_spectate.py`
38
+
39
+ - [ ] **Step 1: Write the failing test**
40
+
41
+ ```python
42
+ # tests/runtime/test_spectate.py
43
+ """Unit tests for SpectateSession (agent-driven, stepwise, rich disclosure)."""
44
+ from __future__ import annotations
45
+
46
+ import pytest
47
+
48
+ import proteus.grid # noqa: F401
49
+ from proteus.agents.vanilla import VanillaAgent
50
+ from proteus.grid.difficulty import Difficulty
51
+ from proteus.providers.fake import FakeProvider
52
+ from proteus.runtime._session_core import SessionFinishedError
53
+ from proteus.runtime.spectate import SpectateSession
54
+
55
+
56
+ def _agent(n=10):
57
+ return VanillaAgent(FakeProvider(responses=["ACTION: stay"] * n, model_name="demo"))
58
+
59
+
60
+ def _new(play_turns=5):
61
+ return SpectateSession(
62
+ "predator_evade", agent=_agent(), model_name="fake:demo",
63
+ difficulty=Difficulty.EASY, seed=42, play_turns=play_turns, use_probe=False,
64
+ )
65
+
66
+
67
+ def test_initial_state_is_cut_intro_int_grid():
68
+ s = _new()
69
+ st = s.state()
70
+ assert st["phase"] == "cut_intro"
71
+ assert st["turn_idx"] == 0
72
+ assert isinstance(st["grid"][0][0], int)
73
+ assert st["turns_so_far"] == []
74
+
75
+
76
+ def test_advance_runs_one_llm_turn_with_rich_disclosure():
77
+ s = _new()
78
+ st = s.advance()
79
+ assert st["turn_idx"] == 1
80
+ last = st["turns_so_far"][-1]
81
+ # rich: optimal/habit/reward/reasoning ARE exposed live (spectator, not scored).
82
+ for k in ("action", "motive_action", "habit_action", "reward",
83
+ "is_diagnostic", "was_congruent", "reasoning"):
84
+ assert k in last
85
+
86
+
87
+ def test_advance_to_budget_then_done_and_finish():
88
+ s = _new(play_turns=2)
89
+ while s.state()["outcome"] is None:
90
+ s.advance()
91
+ st = s.state()
92
+ assert st["phase"] == "done" and st["outcome"] in ("survived", "eliminated")
93
+ trace = s.finish()
94
+ assert trace.model == "fake:demo" and trace.scenario == "predator_evade"
95
+
96
+
97
+ def test_advance_after_done_raises():
98
+ s = _new(play_turns=1)
99
+ s.advance()
100
+ with pytest.raises(SessionFinishedError):
101
+ s.advance()
102
+ ```
103
+
104
+ - [ ] **Step 2: Run test to verify it fails**
105
+
106
+ Run: `.venv/bin/python -m pytest tests/runtime/test_spectate.py -q`
107
+ Expected: FAIL — `ModuleNotFoundError: proteus.runtime.spectate`.
108
+
109
+ - [ ] **Step 3: Write the implementation**
110
+
111
+ ```python
112
+ # proteus/runtime/spectate.py
113
+ """SpectateSession — a threadless, agent-driven driver for watching an LLM play.
114
+
115
+ Sibling of InteractiveSession, but the server (not a human) supplies each action
116
+ by calling the agent once per HTTP request. Built on the same ``_session_core``
117
+ helpers, so its trace matches SessionRunner for the same agent (pinned by
118
+ tests/runtime/test_spectate_equivalence.py).
119
+
120
+ Disclosure: a spectator is NOT the scored subject, so ``state()`` exposes the per
121
+ turn answer keys (optimal/habit), reward, and the model's reasoning LIVE. (The
122
+ model itself still only ever receives the observation passed to ``agent.act``.)
123
+ """
124
+ from __future__ import annotations
125
+
126
+ from proteus.agents.base import Agent
127
+ from proteus.grid.difficulty import Difficulty
128
+ from proteus.runtime import _session_core as core
129
+ from proteus.runtime.trace import SessionTrace, TurnTrace
130
+
131
+
132
+ class SpectateSession:
133
+ def __init__(
134
+ self,
135
+ scenario_name: str,
136
+ *,
137
+ agent: Agent,
138
+ model_name: str,
139
+ difficulty: Difficulty = Difficulty.EASY,
140
+ seed: int | None = None,
141
+ play_turns: int = 15,
142
+ use_probe: bool = False,
143
+ motive_category: str = "survival",
144
+ ) -> None:
145
+ self._scenario_name = scenario_name
146
+ self._agent = agent
147
+ self._model_name = model_name
148
+ self._difficulty = difficulty
149
+ self._seed = seed
150
+ self._play_turns = play_turns
151
+ self._use_probe = use_probe
152
+ self._motive_category = motive_category
153
+
154
+ built = core.build_session(scenario_name, seed, difficulty, play_turns)
155
+ self._scenario = built.scenario
156
+ self._game = built.game
157
+ self._cut_frames = built.cut_frames
158
+ self._cut_grids = built.cut_grids
159
+ # Forward-compatible with pack_evade's default_memory (absent -> None).
160
+ self._memory = getattr(built, "default_memory", None)
161
+ self._system_prompt = self._scenario.rules_text + core._HANDOVER_FRAMING
162
+ self._turns: list[TurnTrace] = []
163
+ self._trace: SessionTrace | None = None
164
+
165
+ def _is_done(self) -> bool:
166
+ return (
167
+ self._game.eliminated
168
+ or self._game.survived
169
+ or len(self._turns) >= self._play_turns
170
+ )
171
+
172
+ def state(self) -> dict:
173
+ done = self._is_done()
174
+ played = len(self._turns)
175
+ phase = "done" if done else ("cut_intro" if played == 0 else "play")
176
+ st: dict = {
177
+ "phase": phase,
178
+ "turn_idx": played,
179
+ "play_turns": self._play_turns,
180
+ "model": self._model_name,
181
+ "grid": core.grid_to_list(self._game.current_grid()),
182
+ "legend": {str(k): v for k, v in self._scenario.legend().items()},
183
+ "actions": list(core._ACTIONS),
184
+ "outcome": None,
185
+ "cut_frames": self._cut_grids if played == 0 else None,
186
+ # RICH disclosure (spectator): per-turn answer keys + reward + reasoning.
187
+ "turns_so_far": [
188
+ {
189
+ "turn_idx": t.turn_idx,
190
+ "action": t.action,
191
+ "motive_action": t.motive_action,
192
+ "habit_action": t.habit_action,
193
+ "reward": t.reward,
194
+ "is_diagnostic": t.is_diagnostic,
195
+ "was_congruent": t.was_congruent,
196
+ "reasoning": t.reasoning,
197
+ }
198
+ for t in self._turns
199
+ ],
200
+ "review": None,
201
+ }
202
+ if done:
203
+ trace = self.finish()
204
+ st["outcome"] = trace.outcome
205
+ st["review"] = {"outcome": trace.outcome, "metrics": trace.metrics}
206
+ return st
207
+
208
+ def advance(self) -> dict:
209
+ if self._is_done():
210
+ raise core.SessionFinishedError("session already finished")
211
+ turn_idx = len(self._turns) + 1
212
+ observation = core.build_observation(
213
+ self._scenario, self._game, self._cut_frames, turn_idx,
214
+ memory=self._memory,
215
+ )
216
+ result = self._agent.act(observation, list(core._ACTIONS), self._system_prompt)
217
+ self._turns.append(core.make_turn_trace(
218
+ self._scenario, self._game,
219
+ turn_idx=turn_idx, observation=observation,
220
+ action=result.action, reasoning=result.reasoning, raw_text=result.raw_text,
221
+ input_tokens=result.input_tokens, output_tokens=result.output_tokens,
222
+ thinking_tokens=result.thinking_tokens,
223
+ ))
224
+ return self.state()
225
+
226
+ def finish(self) -> SessionTrace:
227
+ if self._trace is not None:
228
+ return self._trace
229
+ self._trace = core.finalize(
230
+ self._scenario_name, self._scenario, self._game,
231
+ seed=self._seed, difficulty=self._difficulty,
232
+ play_turns=self._play_turns, turns=self._turns,
233
+ cut_frames=self._cut_frames, motive_category=self._motive_category,
234
+ model=self._model_name,
235
+ )
236
+ return self._trace
237
+ ```
238
+
239
+ NOTE: `core.build_observation(..., memory=...)` is the CP7-merged signature already on
240
+ `master`. If `build_observation` in this tree does not yet accept `memory`, drop the
241
+ `memory=self._memory` kwarg (predator_evade does not use it); the pack_evade plan
242
+ adds/relies on it.
243
+
244
+ - [ ] **Step 4: Run test to verify it passes**
245
+
246
+ Run: `.venv/bin/python -m pytest tests/runtime/test_spectate.py -q`
247
+ Expected: PASS (4 passed).
248
+
249
+ - [ ] **Step 5: Commit**
250
+
251
+ ```bash
252
+ git add proteus/runtime/spectate.py tests/runtime/test_spectate.py
253
+ git commit -m "feat(spectate): SpectateSession — agent-driven stepwise LLM driver"
254
+ ```
255
+
256
+ ---
257
+
258
+ ## Task 2: Golden equivalence — SpectateSession == SessionRunner(VanillaAgent)
259
+
260
+ **Files:**
261
+ - Test: `tests/runtime/test_spectate_equivalence.py`
262
+
263
+ - [ ] **Step 1: Write the test**
264
+
265
+ ```python
266
+ # tests/runtime/test_spectate_equivalence.py
267
+ """Golden: the same scripted FakeProvider produces an identical SessionTrace
268
+ whether driven through SpectateSession (one advance per turn) or SessionRunner +
269
+ VanillaAgent (batch). Pins the spectate path to the runner path."""
270
+ from __future__ import annotations
271
+
272
+ import proteus.grid # noqa: F401
273
+ from proteus.agents.vanilla import VanillaAgent
274
+ from proteus.grid.difficulty import Difficulty
275
+ from proteus.providers.fake import FakeProvider
276
+ from proteus.runtime.session import SessionRunner
277
+ from proteus.runtime.spectate import SpectateSession
278
+
279
+ RESPONSES = ["ACTION: up", "ACTION: up", "ACTION: left",
280
+ "ACTION: stay", "ACTION: right", "ACTION: up"]
281
+
282
+
283
+ def _agent():
284
+ return VanillaAgent(FakeProvider(responses=list(RESPONSES), model_name="demo"))
285
+
286
+
287
+ def test_spectate_matches_session_runner():
288
+ runner = SessionRunner(
289
+ "predator_evade", _agent(), difficulty=Difficulty.EASY, seed=42,
290
+ play_turns=len(RESPONSES), use_probe=False,
291
+ )
292
+ batch_trace = runner.run()
293
+
294
+ sess = SpectateSession(
295
+ "predator_evade", agent=_agent(), model_name="demo",
296
+ difficulty=Difficulty.EASY, seed=42, play_turns=len(RESPONSES), use_probe=False,
297
+ )
298
+ while sess.state()["outcome"] is None:
299
+ sess.advance()
300
+ watch_trace = sess.finish()
301
+
302
+ assert watch_trace.model == batch_trace.model == "demo"
303
+ assert watch_trace.model_dump() == batch_trace.model_dump()
304
+ ```
305
+
306
+ - [ ] **Step 2: Run the test**
307
+
308
+ Run: `.venv/bin/python -m pytest tests/runtime/test_spectate_equivalence.py -q`
309
+ Expected: PASS (1 passed). If a field differs, the spectate path diverges from the
310
+ runner — fix `spectate.py` (system_prompt, memory resolution, finalize args) until the
311
+ `model_dump()` is byte-identical. Do not weaken the assert.
312
+
313
+ - [ ] **Step 3: Commit**
314
+
315
+ ```bash
316
+ git add tests/runtime/test_spectate_equivalence.py
317
+ git commit -m "test(spectate): golden — SpectateSession trace == SessionRunner(VanillaAgent)"
318
+ ```
319
+
320
+ ---
321
+
322
+ ## Task 3: Server routes (`/spectate*`) + provider config
323
+
324
+ **Files:**
325
+ - Modify: `proteus/web/server.py`
326
+ - Test: `tests/web/test_spectate_routes.py`
327
+
328
+ - [ ] **Step 1: Write the failing test**
329
+
330
+ ```python
331
+ # tests/web/test_spectate_routes.py
332
+ """Route-level tests for spectating (offline via fake:demo, no socket)."""
333
+ from __future__ import annotations
334
+
335
+ import proteus.grid # noqa: F401
336
+ from proteus.web import server
337
+
338
+
339
+ def _reg():
340
+ return {}
341
+
342
+
343
+ def test_config_lists_providers_and_default_model():
344
+ _, payload, _ = server.handle_request("GET", "/config", None, _reg())
345
+ assert "fake" in payload["providers"]
346
+ assert payload["default_model"] == "fake:demo"
347
+
348
+
349
+ def test_spectate_create_next_finish(tmp_path):
350
+ reg = _reg()
351
+ status, created, _ = server.handle_request(
352
+ "POST", "/spectate",
353
+ {"scenario": "predator_evade", "difficulty": "easy", "seed": 42,
354
+ "play_turns": 3, "model": "fake:demo", "probe": False}, reg)
355
+ assert status == 200
356
+ sid = created["session_id"]
357
+ assert created["state"]["phase"] == "cut_intro"
358
+
359
+ # advance to terminal
360
+ last = None
361
+ for _ in range(3):
362
+ status, payload, _ = server.handle_request(
363
+ "POST", f"/spectate/{sid}/next", None, reg)
364
+ assert status == 200
365
+ last = payload["state"]
366
+ if last["outcome"] is not None:
367
+ break
368
+ assert last["turns_so_far"] # rich disclosure present
369
+
370
+ out = tmp_path / "spectate.jsonl"
371
+ status, payload, _ = server.handle_request(
372
+ "POST", f"/spectate/{sid}/finish", {"out": str(out)}, reg)
373
+ assert status == 200 and out.exists()
374
+ assert payload["trace_path"] == str(out) and "survival_fraction" in payload["metrics"]
375
+
376
+
377
+ def test_unknown_model_is_400():
378
+ status, payload, _ = server.handle_request(
379
+ "POST", "/spectate",
380
+ {"scenario": "predator_evade", "difficulty": "easy", "seed": 1,
381
+ "play_turns": 3, "model": "nope:x", "probe": False}, _reg())
382
+ assert status == 400 and "error" in payload
383
+
384
+
385
+ def test_next_after_done_is_409():
386
+ reg = _reg()
387
+ _, created, _ = server.handle_request(
388
+ "POST", "/spectate",
389
+ {"scenario": "predator_evade", "difficulty": "easy", "seed": 42,
390
+ "play_turns": 1, "model": "fake:demo", "probe": False}, reg)
391
+ sid = created["session_id"]
392
+ server.handle_request("POST", f"/spectate/{sid}/next", None, reg) # exhausts budget
393
+ status, payload, _ = server.handle_request("POST", f"/spectate/{sid}/next", None, reg)
394
+ assert status == 409 and "error" in payload
395
+ ```
396
+
397
+ - [ ] **Step 2: Run test to verify it fails**
398
+
399
+ Run: `.venv/bin/python -m pytest tests/web/test_spectate_routes.py -q`
400
+ Expected: FAIL — `/config` lacks `providers`; `/spectate` route returns 404.
401
+
402
+ - [ ] **Step 3: Modify `proteus/web/server.py`**
403
+
404
+ Add lazy imports + the spectate handlers (providers/agents imported INSIDE the create
405
+ handler to keep the offline import-safety invariant). Extend `_config_payload`, and add
406
+ routes to `handle_request`.
407
+
408
+ Add these handler functions (above `handle_request`):
409
+
410
+ ```python
411
+ def _create_spectate(body: dict, registry: dict) -> Response:
412
+ # Lazy imports keep `import proteus.web.server` free of provider SDKs.
413
+ from proteus.agents.vanilla import VanillaAgent
414
+ from proteus.providers import make_provider
415
+ from proteus.runtime.spectate import SpectateSession
416
+ try:
417
+ scenario = body["scenario"]
418
+ difficulty = Difficulty(body.get("difficulty", "easy"))
419
+ seed = body.get("seed", _DEFAULT_SEED)
420
+ seed = None if seed in ("", None) else int(seed)
421
+ play_turns = int(body.get("play_turns", 10))
422
+ model = body.get("model", "fake:demo")
423
+ probe = bool(body.get("probe", False))
424
+ except (KeyError, ValueError) as exc:
425
+ return _err(400, f"bad spectate params: {exc}")
426
+ if scenario not in list_scenarios():
427
+ return _err(400, f"unknown scenario {scenario!r}")
428
+ try:
429
+ provider = make_provider(model)
430
+ except ValueError as exc:
431
+ return _err(400, str(exc))
432
+ agent = VanillaAgent(provider)
433
+ session = SpectateSession(
434
+ scenario, agent=agent, model_name=provider.model_name,
435
+ difficulty=difficulty, seed=seed, play_turns=play_turns, use_probe=probe,
436
+ )
437
+ sid = uuid.uuid4().hex
438
+ registry[sid] = session
439
+ return 200, {"session_id": sid, "state": session.state()}, _JSON
440
+
441
+
442
+ def _spectate_next(sid: str, registry: dict) -> Response:
443
+ session = registry.get(sid)
444
+ if session is None:
445
+ return _err(404, f"unknown session {sid!r}")
446
+ try:
447
+ state = session.advance()
448
+ except core.SessionFinishedError:
449
+ return _err(409, "session already finished")
450
+ except Exception as exc: # provider failure (missing key / network): no stack leak
451
+ return _err(500, f"agent step failed: {exc}")
452
+ return 200, {"state": state}, _JSON
453
+
454
+
455
+ def _spectate_finish(sid: str, body: dict, registry: dict) -> Response:
456
+ session = registry.get(sid)
457
+ if session is None:
458
+ return _err(404, f"unknown session {sid!r}")
459
+ try:
460
+ trace = session.finish()
461
+ except core.SessionNotFinishedError:
462
+ return _err(409, "session not finished")
463
+ out = (body or {}).get("out") or f"runs/web_spectate_{trace.scenario}_{trace.difficulty}.jsonl"
464
+ written = append_trace(trace, out)
465
+ return 200, {"trace_path": str(written), "metrics": trace.metrics}, _JSON
466
+ ```
467
+
468
+ Extend `_config_payload` to add providers + default model:
469
+
470
+ ```python
471
+ def _config_payload() -> dict:
472
+ from proteus.providers import available_providers # local: keep module import light
473
+ return {
474
+ "scenarios": list_scenarios(),
475
+ "difficulties": [d.value for d in Difficulty],
476
+ "color_map": {str(k): v for k, v in COLOR_MAP.items()},
477
+ "default_seed": _DEFAULT_SEED,
478
+ "providers": available_providers(),
479
+ "default_model": "fake:demo",
480
+ }
481
+ ```
482
+
483
+ Add the routes in `handle_request` (after the `/session/` block, before the final 404):
484
+
485
+ ```python
486
+ if method == "POST" and path == "/spectate":
487
+ return _create_spectate(body or {}, registry)
488
+ if path.startswith("/spectate/"):
489
+ rest = path[len("/spectate/"):]
490
+ if method == "POST" and rest.endswith("/next"):
491
+ return _spectate_next(rest[: -len("/next")], registry)
492
+ if method == "POST" and rest.endswith("/finish"):
493
+ return _spectate_finish(rest[: -len("/finish")], body or {}, registry)
494
+ if method == "GET":
495
+ session = registry.get(rest)
496
+ if session is None:
497
+ return _err(404, f"unknown session {rest!r}")
498
+ return 200, {"state": session.state()}, _JSON
499
+ ```
500
+
501
+ (`available_providers` / `make_provider` are imported **locally** so the top-level
502
+ `import proteus.web.server` still pulls no provider SDK — Task 6 guards this.)
503
+
504
+ - [ ] **Step 4: Run test to verify it passes**
505
+
506
+ Run: `.venv/bin/python -m pytest tests/web/test_spectate_routes.py -q`
507
+ Expected: PASS (5 passed).
508
+
509
+ - [ ] **Step 5: Commit**
510
+
511
+ ```bash
512
+ git add proteus/web/server.py tests/web/test_spectate_routes.py
513
+ git commit -m "feat(spectate): /spectate routes + provider config (lazy-imported)"
514
+ ```
515
+
516
+ ---
517
+
518
+ ## Task 4: Frontend — spectate mode (toggle, analysis panel, auto-play/pause/step)
519
+
520
+ **Files:**
521
+ - Modify: `proteus/web/static/index.html`
522
+
523
+ - [ ] **Step 1: Add a mode toggle + model input to the start screen**
524
+
525
+ In the start-screen `.row` (after the `probe` checkbox `<div>`), add:
526
+
527
+ ```html
528
+ <div><label>mode</label><select id="mode">
529
+ <option value="play">Play (human)</option>
530
+ <option value="spectate">Spectate (LLM)</option>
531
+ </select></div>
532
+ <div id="modelWrap" class="hidden"><label>model</label>
533
+ <input id="model" value="fake:demo" style="width:140px"></div>
534
+ ```
535
+
536
+ - [ ] **Step 2: Add a spectate analysis panel to the play screen**
537
+
538
+ After the `#grid` div inside `#play`, add:
539
+
540
+ ```html
541
+ <div id="analysis" class="hidden">
542
+ <div id="thinking" style="color:var(--muted)"></div>
543
+ <div id="reasoning" style="white-space:pre-wrap;margin:8px 0"></div>
544
+ <table id="turnlog"><tr><th>turn</th><th>action</th><th>optimal</th>
545
+ <th>habit</th><th>reward</th><th>=opt?</th></tr></table>
546
+ <p style="text-align:center;margin-top:10px">
547
+ <button id="pauseBtn">Pause</button>
548
+ <button id="stepBtn">Step</button></p>
549
+ </div>
550
+ ```
551
+
552
+ - [ ] **Step 3: Add the spectate JS (append inside the existing `<script>`)**
553
+
554
+ ```javascript
555
+ let SPECT = false, PAUSED = false;
556
+
557
+ function showModelInput() {
558
+ $("modelWrap").classList.toggle("hidden", $("mode").value !== "spectate");
559
+ }
560
+
561
+ function appendTurn(t) {
562
+ const tr = document.createElement("tr");
563
+ if (t.is_diagnostic) tr.className = "diag";
564
+ tr.innerHTML = `<td>${t.turn_idx}</td><td>${t.action}</td><td>${t.motive_action}</td>`
565
+ + `<td>${t.habit_action}</td><td>${t.reward.toFixed(1)}</td>`
566
+ + `<td>${t.was_congruent ? '✓':'✗'}</td>`;
567
+ $("turnlog").appendChild(tr);
568
+ }
569
+
570
+ function showSpectateState(st) {
571
+ $("turnInfo").textContent = `Turn ${st.turn_idx} / ${st.play_turns}`;
572
+ drawGrid(st.grid);
573
+ const last = st.turns_so_far[st.turns_so_far.length - 1];
574
+ if (last) { $("reasoning").textContent = last.reasoning || ""; appendTurn(last); }
575
+ }
576
+
577
+ async function spectateLoop() {
578
+ while (SPECT && !PAUSED) {
579
+ $("thinking").textContent = "thinking…";
580
+ const {status, data} = await jpost(`/spectate/${SID}/next`, {});
581
+ $("thinking").textContent = "";
582
+ if (status === 409) break;
583
+ if (status !== 200) { $("thinking").textContent = data.error || "error"; SPECT = false; break; }
584
+ showSpectateState(data.state);
585
+ if (data.state.outcome) {
586
+ const fin = await jpost(`/spectate/${SID}/finish`, {});
587
+ if (fin.status === 200) $("savedPath").textContent = "trace saved: " + fin.data.trace_path;
588
+ $("play").classList.add("hidden"); $("end").classList.remove("hidden");
589
+ const o = $("outcome");
590
+ o.textContent = data.state.outcome === "survived" ? "SURVIVED" : "ELIMINATED";
591
+ o.className = "outcome " + (data.state.outcome === "survived" ? "win" : "lose");
592
+ SPECT = false; break;
593
+ }
594
+ }
595
+ }
596
+
597
+ async function startSpectate(body) {
598
+ const {status, data} = await jpost("/spectate", body);
599
+ if (status !== 200) { alert(data.error || "failed to start"); return; }
600
+ SID = data.session_id; SPECT = true; PAUSED = false;
601
+ $("turnlog").querySelectorAll("tr:not(:first-child)").forEach(r => r.remove());
602
+ $("start").classList.add("hidden"); $("end").classList.add("hidden");
603
+ $("play").classList.remove("hidden"); $("analysis").classList.remove("hidden"); $("pad")?.classList?.add?.("hidden");
604
+ $("meta").textContent = `${body.scenario} · ${body.difficulty} · ${body.model}`;
605
+ const st = data.state;
606
+ if (st.cut_frames) await animateCut(st.cut_frames);
607
+ drawGrid(st.grid);
608
+ spectateLoop();
609
+ }
610
+ ```
611
+
612
+ - [ ] **Step 4: Wire the controls in `startGame`/`init`**
613
+
614
+ In `init()`, add:
615
+
616
+ ```javascript
617
+ $("mode").addEventListener("change", showModelInput);
618
+ $("pauseBtn").addEventListener("click", () => {
619
+ PAUSED = !PAUSED; $("pauseBtn").textContent = PAUSED ? "Resume" : "Pause";
620
+ if (!PAUSED) spectateLoop();
621
+ });
622
+ $("stepBtn").addEventListener("click", async () => {
623
+ if (!SPECT) return;
624
+ const {status, data} = await jpost(`/spectate/${SID}/next`, {});
625
+ if (status === 200) { showSpectateState(data.state);
626
+ if (data.state.outcome) { await jpost(`/spectate/${SID}/finish`, {}); SPECT = false; } }
627
+ });
628
+ ```
629
+
630
+ In `startGame()`, branch on mode at the top:
631
+
632
+ ```javascript
633
+ if ($("mode").value === "spectate") {
634
+ return startSpectate({
635
+ scenario: $("scenario").value, difficulty: $("difficulty").value,
636
+ seed: Number($("seed").value), play_turns: Number($("turns").value),
637
+ model: $("model").value, probe: $("probe").checked,
638
+ });
639
+ }
640
+ // ... existing human-play body unchanged ...
641
+ ```
642
+
643
+ Also hide the human direction pad in spectate mode (give the `.pad` div `id="pad"` if it
644
+ has none, or guard the `.pad` listeners with `if (SPECT) return;` inside `act`).
645
+
646
+ - [ ] **Step 5: Verify the server byte test still passes + manual check**
647
+
648
+ Run: `.venv/bin/python -m pytest tests/web/test_server.py::test_root_serves_html_bytes -q`
649
+ Expected: PASS.
650
+ Manual: `.venv/bin/python -m proteus.web --port 8124`, open `http://127.0.0.1:8124/`,
651
+ pick **Spectate (LLM)**, model `fake:demo` → grid auto-advances with the analysis table
652
+ + pause/step working → on end, SURVIVED/ELIMINATED + "trace saved". Stop with Ctrl-C.
653
+
654
+ - [ ] **Step 6: Commit**
655
+
656
+ ```bash
657
+ git add proteus/web/static/index.html
658
+ git commit -m "feat(spectate): frontend — mode toggle + live analysis + auto-play/pause/step"
659
+ ```
660
+
661
+ ---
662
+
663
+ ## Task 5: End-to-end + full suite
664
+
665
+ **Files:** none (verification only)
666
+
667
+ - [ ] **Step 1: Full suite**
668
+
669
+ Run: `.venv/bin/python -m pytest -q`
670
+ Expected: PASS — all existing + new spectate tests, zero failures.
671
+
672
+ - [ ] **Step 2: End-to-end — a spectated (fake) trace is replayable + comparable**
673
+
674
+ ```bash
675
+ .venv/bin/python -m proteus.web --port 8124 & SRV=$!; sleep 1.5
676
+ SID=$(curl -s -X POST http://127.0.0.1:8124/spectate -H 'Content-Type: application/json' \
677
+ -d '{"scenario":"predator_evade","difficulty":"easy","seed":42,"play_turns":3,"model":"fake:demo","probe":false}' \
678
+ | .venv/bin/python -c "import sys,json;print(json.load(sys.stdin)['session_id'])")
679
+ for i in 1 2 3; do curl -s -X POST http://127.0.0.1:8124/spectate/$SID/next >/dev/null; done
680
+ curl -s -X POST http://127.0.0.1:8124/spectate/$SID/finish \
681
+ -H 'Content-Type: application/json' -d '{"out":"runs/spectate_e2e.jsonl"}'; echo
682
+ kill $SRV
683
+ .venv/bin/python -m proteus replay runs/spectate_e2e.jsonl
684
+ .venv/bin/python -m proteus compare runs/spectate_e2e.jsonl
685
+ rm -f runs/spectate_e2e.jsonl
686
+ ```
687
+ Expected: `/finish` returns `{trace_path, metrics}`; `replay`/`compare` accept the
688
+ trace as a `model=fake` group.
689
+
690
+ ---
691
+
692
+ ## Task 6: Offline import-safety still holds
693
+
694
+ **Files:**
695
+ - Modify: `tests/web/test_import_safety.py`
696
+
697
+ - [ ] **Step 1: Extend the guard**
698
+
699
+ Append to the existing test body (after the current asserts) — re-clearing and
700
+ re-importing already happens; just confirm the spectate additions did not pull an SDK:
701
+
702
+ ```python
703
+ # spectate routes import providers LAZILY (inside the handler), so a plain
704
+ # import of the web layer must still pull no provider SDK.
705
+ import proteus.web.server # noqa: F401 (re-exercised after the clear above)
706
+ for sdk in ("openai", "anthropic", "google", "ollama"):
707
+ assert sdk not in sys.modules
708
+ ```
709
+
710
+ - [ ] **Step 2: Run it**
711
+
712
+ Run: `.venv/bin/python -m pytest tests/web/test_import_safety.py -q`
713
+ Expected: PASS (1 passed).
714
+
715
+ - [ ] **Step 3: Commit**
716
+
717
+ ```bash
718
+ git add tests/web/test_import_safety.py
719
+ git commit -m "test(spectate): offline import-safety still holds (providers lazy)"
720
+ ```
721
+
722
+ ---
723
+
724
+ ## Task 7: Update HANDOFF.md
725
+
726
+ **Files:**
727
+ - Modify: `HANDOFF.md`
728
+
729
+ - [ ] **Step 1: Add a "Done (Web LLM spectating)" section**
730
+
731
+ Summarize: watch an LLM play in the browser (color grid + live reasoning/analysis,
732
+ auto-play with pause/step); `SpectateSession` (agent-driven, `_session_core`-shared)
733
+ pinned to `SessionRunner(VanillaAgent)` by a golden test; real providers + `fake:demo`
734
+ fallback, **lazy-imported** (offline import-safety preserved); spectated runs save a
735
+ `runs/web_spectate_*.jsonl` LLM-baseline trace accepted by `proteus replay`/`compare`.
736
+ Update the full-suite count.
737
+
738
+ - [ ] **Step 2: Commit**
739
+
740
+ ```bash
741
+ git add HANDOFF.md
742
+ git commit -m "docs(spectate): handoff — watch an LLM play in color shipped"
743
+ ```
744
+
745
+ ---
746
+
747
+ ## Self-Review notes (already reconciled)
748
+
749
+ - **Spec coverage:** §3 components → Tasks 1,3,4; §4 SpectateSession → Task 1; §5 golden → Task 2; §6 API → Task 3; §7 frontend → Task 4; §8 trace → Tasks 3,5; §9 testing → Tasks 1,2,3,5,6; §10 acceptance → Task 5.
750
+ - **Parallel-safe vs pack_evade:** writable files here (`spectate.py`, `web/server.py`, `web/static/index.html`, spectate tests) are disjoint from pack_evade's (grid + runtime-core). `SpectateSession` reads `default_memory` via `getattr(..., None)` and passes `memory=` only if `build_observation` accepts it, so it builds and tests green whether pack_evade has merged or not.
751
+ - **Offline invariant:** providers/agents imported only inside `_create_spectate` (and `available_providers` locally in `_config_payload`); Task 6 guards that `import proteus.web.server` pulls no SDK.
752
+ - **Type consistency:** `SpectateSession(agent=, model_name=)`; `advance()`/`state()`/`finish()` mirror the `_session_core` calls used by `InteractiveSession`/`SessionRunner`; `state()["turns_so_far"]` keys match the frontend `appendTurn`/`showSpectateState` reads; route handler names (`_create_spectate`/`_spectate_next`/`_spectate_finish`) match their `handle_request` dispatch.
753
+ ```