docs(cp7): design spec — LLM-generated memory pre-roll (hybrid handover)

Memory generation stage: model self-plays a full-info episode of the same
scenario; trajectory persists as a per-model timestamped checkpoint and is
shown at the handover as prior experience, lengthening the believable history.
Hybrid keeps the scripted diagnostic Cut + step_reward + metrics unchanged
(127-preserving, optimal!=habit invariant intact). Offline by FakeProvider;
Ollama acceptance smoke. Curiosity/new-category deferred to CP8.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

docs/superpowers/specs/2026-06-02-proteus-cp7-memory-preroll-design.md

@@ -0,0 +1,297 @@
+# PROTEUS CP7 — LLM-Generated Memory Pre-roll
+
+> CP7 sub-design under the immutable parent SSOT
+> `docs/superpowers/specs/2026-06-01-proteus-arena-slice-design.md`
+> (§4 Scenario contract, §6 SessionRunner, §7 trace/metrics, §11 CP table, §12 deferred).
+> CP0–CP6 are complete (see `HANDOFF.md`; `.venv/bin/python -m pytest -q` → 127 passed).
+> This document is the design SSOT for CP7.
+
+**Date:** 2026-06-02
+
+## 0. One-line summary
+
+CP7 adds an **LLM-generated memory pre-roll**: before the scored game, the playing model runs a
+full-information self-play episode of the same scenario (the "memory generation stage"); that trajectory
+is persisted as a per-model, timestamped **checkpoint** and shown at the handover as the model's *prior
+experience*, lengthening the believable "this is your movement, continue" history. The scored game keeps
+its hand-authored scripted Cut and diagnostic invariant **unchanged** (hybrid design), so all 127
+existing tests and the `optimal_action != habit_action` measurement are preserved. Offline by default
+(FakeProvider, deterministic); the acceptance smoke uses the Ollama API.
+
+## 1. Motivation / context
+
+The handover framing is "what you are seeing is *your* movement — now continue." Today that history is
+just the scripted Cut pre-roll (`cut_focal_policy` = "left" for `cut_length` 2–3 steps), so the model is
+asked to "continue" a trajectory only 2–3 frames long. That is too thin to read as the model's own
+established behaviour.
+
+CP7 lengthens the believable history by having the LLM **generate its own pre-roll**: a transparent
+full-information self-play run of the same scenario, recorded and replayed at the handover as "your
+earlier play." Crucially, the **measurement** (does the model read the predator's motive at the
+diagnostic Cut?) must not be weakened. A naive "let the LLM play the lead-in itself" would let a
+competent model dodge the hand-authored dead-end, destroying the `optimal != habit` invariant. The
+**hybrid** design resolves this: the self-generated memory is shown *in addition to* the unchanged
+scripted Cut, not in place of it.
+
+## 2. Confirmed decisions (do not silently change)
+
+- **Hybrid handover.** The turn-1 observation = `[MEMORY: LLM full-info self-play trajectory]` +
+  `[NOW: existing scripted diagnostic Cut frames]` → handover. The scripted Cut, `step_reward`,
+  `optimal_action`/`habit_action`, and all metrics are **unmodified** → 127 regression-free and the
+  diagnostic invariant preserved.
+- **Memory generation = real LLM self-play.** The agent is given a transparent brief (full rules +
+  predator chase rule + map + goal) and **chooses each action itself**; the optimal action is never
+  revealed. The episode runs through the real engine (predator chases), capped at `memory_turns`,
+  stopping early on terminal.
+- **Per-model, timestamped checkpoints.** Each generated memory is persisted as a single-file JSON
+  checkpoint at `runs/memory/<model>/<timestamp>.json`. `created_at` is produced by an **injectable
+  clock** (real UTC by default) so tests stay deterministic.
+- **Reuse is flag-selected.** `proteus run --memory MODE` with `MODE ∈ {none(default), generate,
+  latest, <path>}` chooses whether to generate fresh, reuse the model's latest checkpoint, load a
+  specific file, or keep current behaviour.
+- **Memory world = same scenario + difficulty + seed** as the scored game, played with full info and
+  free choice (the same map seen as "prior experience"). `memory_turns` default **10**.
+- **Offline invariant holds.** All automated tests use FakeProvider (deterministic). The Ollama smoke
+  is a manual acceptance step; no provider SDK is added to `.venv`.
+- **This is CP7;** the curiosity / new-motive-category work (spec §12) moves to **CP8**.
+
+## 3. Scope / non-goals
+
+In scope: a memory data model + single-file persistence, a memory-generation self-play stage, hybrid
+handover injection into `SessionRunner`, a scenario-sourced transparent brief, a `proteus memory`
+subcommand + `proteus run --memory`, and an Ollama acceptance smoke.
+
+Deferred (YAGNI, parent spec §12):
+- **New motive categories** (curiosity, sociality) → CP8. CP7 touches only the survival scenario.
+- **Generalizing `_PROBE_QUESTION` / `_ACTION_DIRECTIVE`** out of `SessionRunner`/`VanillaAgent`. CP7
+  adds a scenario-sourced `memory_brief` only; the probe/act-directive predator-framing stays as-is
+  (carried to CP8, where the second scenario forces it). This avoids scope creep.
+- **Memory accumulation / multi-episode memory** (a checkpoint holds exactly one self-play episode),
+  LLM-as-judge scoring of memory reasoning, web UI.
+
+## 4. Section A — Memory data model + persistence
+
+New module **`proteus/runtime/memory.py`** — pure pydantic models + single-file JSON I/O, importing
+only pydantic + stdlib (`json`, `pathlib`, `datetime`). Like `trace.py`, it imports **no other runtime
+module** (serialization boundary).
+
+```python
+class MemoryTurn(BaseModel):       # one self-play turn of the memory episode
+    turn_idx: int
+    frame_ascii: str               # pre-move grid (what the model saw)
+    action: str
+    reasoning: str                 # excerpt (may be truncated for size)
+    focal_pos: tuple[int, int]     # pre-move
+    predator_pos: tuple[int, int]  # pre-move
+
+class MemoryCheckpoint(BaseModel):
+    model: str
+    scenario: str
+    motive_category: str = "survival"
+    difficulty: str
+    seed: int | None
+    created_at: str                # ISO-8601, from an injectable clock
+    memory_turns: list[MemoryTurn]
+    outcome: str                   # "survived" | "eliminated" (engine: budget done = survived)
+    transparent_prompt: str        # the brief used (audit / reproducibility)
+```
+
+Persistence helpers in the same module:
+
+- `save_checkpoint(ckpt, root="runs/memory") -> Path` — writes
+  `<root>/<safe(model)>/<created_at-derived-stamp>.json` (creates parent dirs; model string sanitised
+  for the filesystem). Returns the path written.
+- `load_checkpoint(path) -> MemoryCheckpoint` — reads one file (raises `FileNotFoundError` on miss).
+- `latest_for_model(model, root="runs/memory") -> MemoryCheckpoint | None` — returns the newest
+  checkpoint for `model` (by filename stamp / mtime), or `None` if none exist.
+
+The filename stamp is derived from `created_at` (e.g. `2026-06-02T10-40-56Z`) so the on-disk ordering
+matches the logical creation order; ties break deterministically.
+
+## 5. Section B — Memory generation stage
+
+New module **`proteus/runtime/memory_gen.py`** (engine-coupled, mirrors `rollout.py`'s replay pattern;
+imports `grid`, `agents`, `ascii_view`, and the `memory` models).
+
+```python
+def generate_memory(
+    scenario_name: str,
+    agent: Agent,
+    *,
+    difficulty: Difficulty,
+    seed: int | None,
+    memory_turns: int,
+    model_name: str,
+    clock: Callable[[], str] = _utc_now_iso,
+) -> MemoryCheckpoint: ...
+```
+
+Flow:
+1. Build a fresh `scenario` + `MotiveGridGame` from `(scenario_name, seed, difficulty)` with
+   `max_steps = memory_turns` (**no scripted Cut** — the memory is free play from the initial layout).
+2. `system_prompt = scenario.memory_brief or scenario.rules_text` (the transparent brief, Section D.1).
+3. For `turn_idx` in `1..memory_turns`:
+   - render the pre-move observation (frame + legend + available actions);
+   - capture pre-move `focal_pos` / `predator_pos`;
+   - `result = agent.act(observation, actions, system_prompt)`;
+   - append a `MemoryTurn` (frame, `result.action`, `result.reasoning` excerpt, positions);
+   - `game.apply_motive_action(result.action)` + `scenario.record_focal_move(result.action)`;
+   - break if `game.eliminated or game.survived`.
+4. `outcome` = `"eliminated"` if captured, else `"survived"` (with `max_steps = memory_turns`, the
+   engine marks `survived` once the budget is spent — a completed practice episode).
+5. Return `MemoryCheckpoint(model=model_name, …, created_at=clock(), memory_turns=…, outcome=…,
+   transparent_prompt=system_prompt)`.
+
+Determinism: with a FakeProvider (scripted actions) the checkpoint is a pure function of
+`(scenario, seed, difficulty, scripted-actions, clock)`. The real engine drives the predator, so the
+memory is a genuine chase episode (it may end early in capture).
+
+## 6. Section C — Hybrid handover injection
+
+`SessionRunner.__init__` gains an optional `memory: MemoryCheckpoint | None = None` (default `None` →
+**exactly current behaviour**, 127-preserving). The only behavioural change is in `_observation`:
+
+- On **turn 1**, when `self._memory is not None`, prepend a labelled memory block built from
+  `memory.memory_turns` **before** the existing Cut block:
+
+  ```
+  MEMORY — your earlier play on this scenario:
+  Memory 1:
+  <frame_ascii>
+    you chose: up
+  Memory 2:
+  ...
+  NOW — this run so far:
+  Cut 1:
+  <existing scripted cut frame>
+  ...
+  Now:
+  <handover frame>
+  ```
+- Turns 2+ are unchanged. The scripted Cut replay, the diagnostic snapshot
+  (`optimal_action`/`habit_action`), `step_reward`, `optimal_rollout`, and every metric are untouched.
+
+The memory block is observation text only; it does **not** alter world state, the play budget, or any
+answer key. Result: the handover history is materially longer while the measurement is identical.
+
+The trace records that a memory was used via a new additive, optional field on `SessionTrace`:
+`memory_ref: str | None = None` (the checkpoint path or stamp; `None` when no memory). This keeps
+`compare`/`aggregate` and the existing schema backward-compatible (additive, defaulted).
+
+## 7. Section D — Scenario brief + CLI + smoke
+
+### 7.1 Scenario-sourced transparent brief
+
+`Scenario` ABC gains a concrete class attribute `memory_brief: str = ""` (no new abstract method).
+`predator_evade` sets it to a transparent description: the existing `rules_text` plus an explicit
+statement that the predator takes one BFS shortest-path step toward the focal each turn, a note that
+walls form dead-ends, and that this is a practice run to build experience (the optimal move is **not**
+revealed). `generate_memory` uses `scenario.memory_brief or scenario.rules_text`, so scenarios that
+omit a brief fall back gracefully.
+
+### 7.2 CLI
+
+- **`proteus memory`** (new subcommand) — generate + save a checkpoint:
+  ```
+  proteus memory --scenario predator_evade --model fake:demo \
+                 --difficulty easy --seed 42 --memory-turns 10 [--out PATH]
+  ```
+  Builds the agent from `--model` (same factory as `run`), calls `generate_memory`, saves via
+  `save_checkpoint` (auto-path `runs/memory/<model>/<stamp>.json` unless `--out`), prints the path +
+  the episode outcome. Exit codes mirror `run` (unknown model/scenario → 2).
+- **`proteus run --memory MODE`** (new option, default `none`):
+  - `none` → current behaviour (no memory).
+  - `generate` → call `generate_memory` inline (same model/scenario/difficulty/seed), save it, inject.
+  - `latest` → `latest_for_model(model)`; clean stderr + exit 2 if none exists.
+  - `<path>` → `load_checkpoint(<path>)`.
+  The loaded/!-generated checkpoint is passed to `SessionRunner(memory=…)` and recorded as `memory_ref`.
+
+### 7.3 Offline tests (FakeProvider)
+
+1. **Persistence round-trip**: `save_checkpoint` → `load_checkpoint` reproduces the model; bytes are
+   stable; `latest_for_model` picks the newest of several.
+2. **Generation determinism**: a FakeProvider with scripted actions yields a fixed `MemoryCheckpoint`
+   (positions, actions, outcome) under a fixed clock.
+3. **Hybrid injection**: with a memory, turn-1 observation contains the memory frames and is strictly
+   longer than without; turns 2+ identical; the diagnostic snapshot and `metrics` are unchanged from
+   the no-memory run (same seed) → invariant + 127 preserved.
+4. **Comparability**: `memory_ref` is additive; `set(trace.metrics)` unchanged; human/LLM comparability
+   test still holds.
+5. **CLI**: `proteus memory` writes a loadable checkpoint headlessly; `run --memory generate` and
+   `--memory <path>` produce a trace with a populated `memory_ref`; `--memory latest` with no
+   checkpoint exits 2.
+
+### 7.4 Ollama acceptance smoke (manual, networked)
+
+Using the CP4 recipe (throwaway venv with `httpx`; `.venv` stays SDK-free; the `OLLAMA_API_KEY` is the
+first whitespace token of the `.env` value):
+
+```
+proteus memory --scenario predator_evade --model ollama:gpt-oss:120b-cloud \
+               --difficulty easy --seed 42 --memory-turns 8
+proteus run    --scenario predator_evade --model ollama:gpt-oss:120b-cloud \
+               --difficulty easy --seed 42 --play-turns 10 --memory latest --out runs/cp7_real.jsonl
+```
+
+Acceptance: the memory checkpoint is written under `runs/memory/gpt-oss:120b-cloud/…`, the run's turn-1
+observation shows the memory block (verify via `replay`/trace), and the trace's `memory_ref` is set.
+
+## 8. Cross-cutting
+
+### 8.1 ABC / schema changes (summary)
+
+- `Scenario.memory_brief: str = ""` — concrete class attribute (no new abstract method; all existing
+  scenarios keep working).
+- `SessionRunner(..., memory: MemoryCheckpoint | None = None)` — optional, defaults to current
+  behaviour.
+- `SessionTrace.memory_ref: str | None = None` — additive optional field.
+
+No change to `step_reward`, `optimal_action`, `habit_action`, `advance_threat`, `build_level`,
+`metrics`, `aggregate`, or the scripted Cut.
+
+### 8.2 Determinism
+
+Memory generation is deterministic given `(scenario, seed, difficulty, agent-actions, clock)`. Tests
+inject a fixed clock and a scripted FakeProvider. The scored game remains a pure function of
+`(scenario, seed, difficulty)` + recorded actions; the memory block is observation-only and changes no
+world state.
+
+### 8.3 Module layout
+
+| File | Change |
+|------|--------|
+| `proteus/runtime/memory.py` (new) | `MemoryTurn`/`MemoryCheckpoint` pydantic + `save_checkpoint`/`load_checkpoint`/`latest_for_model` (no intra-runtime imports) |
+| `proteus/runtime/memory_gen.py` (new) | `generate_memory` full-info self-play (engine-coupled, like `rollout.py`) |
+| `proteus/grid/scenario.py` | add `memory_brief: str = ""` class attribute |
+| `proteus/grid/scenarios/predator_evade.py` | set transparent `memory_brief` |
+| `proteus/runtime/session.py` | accept `memory=`; prepend the memory block in `_observation` turn 1; record `memory_ref` |
+| `proteus/runtime/trace.py` | add `SessionTrace.memory_ref: str | None = None` |
+| `proteus/runtime/__init__.py` | export `MemoryCheckpoint`, `generate_memory`, persistence helpers |
+| `proteus/cli.py` | `memory` subcommand + `run --memory MODE` |
+
+Dependency direction (parent spec §3) preserved: `runtime` depends on `grid`; `grid`/`arc_grid` never
+import `runtime`. `runtime/memory.py` (like `trace.py`) imports no other runtime module.
+
+### 8.4 Completion criteria (parent spec §11 analogue)
+
+1. Full offline suite passes (`.venv/bin/python -m pytest -q`) — 127 prior + new, no network/display/SDK.
+2. Persistence round-trip + `latest_for_model` selection tests pass.
+3. Deterministic memory generation under FakeProvider + fixed clock.
+4. Hybrid injection lengthens turn-1 observation; diagnostic invariant + all metrics unchanged at the
+   same seed; comparability holds.
+5. `proteus memory` + `run --memory {generate,latest,<path>,none}` headless tests pass.
+6. Ollama smoke: memory checkpoint written, run's handover shows the memory block, `memory_ref` set.
+
+## 9. Open questions (carry forward)
+
+- **Reasoning size in checkpoints.** `MemoryTurn.reasoning` may be large for real models; CP7 stores an
+  excerpt (truncation length pinned in implementation). Revisit if full reasoning is needed for analysis.
+- **Memory block rendering at scale.** With `memory_turns` = 10 and larger grids the turn-1 observation
+  grows; acceptable for the slice. A compact textual summary (positions only, no full frames) is a
+  possible future toggle.
+- **Single-episode memory.** A checkpoint holds one episode; multi-episode / accumulated memory is
+  deferred (CP8+).
+- **`memory generate` vs `latest` divergence.** `--memory generate` always uses the current model; a
+  reused `latest` checkpoint could be from an older model build — `memory_ref` records which was used so
+  analysis can tell them apart.
+```