irregular6612 Claude Opus 4.8 (1M context) commited on
Commit
8431307
·
1 Parent(s): 51ee471

docs(cp7): HANDOFF — CP7 memory pre-roll complete + Ollama smoke; Next=CP8

Browse files

152 passed offline; Ollama acceptance (gpt-oss:120b-cloud) survived with a
6-turn self-played memory shown at the handover, memory_ref recorded, .venv
SDK-free. Curiosity/new-category renumbered CP8.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. HANDOFF.md +59 -10
HANDOFF.md CHANGED
@@ -4,8 +4,9 @@
4
  > `docs/superpowers/specs/2026-06-01-proteus-arena-slice-design.md` (immutable design) +
5
  > the per-CP plan in `docs/superpowers/plans/` + this file.
6
 
7
- **Last updated:** 2026-06-02 — CP6 complete (difficulty layouts + category-specific eval +
8
- human-baseline `compare` harness; offline, headless).
 
9
 
10
  ## Done (CP0–CP3)
11
 
@@ -145,6 +146,11 @@ Executed task-by-task via subagent-driven-development (3-gate: implement → spe
145
  (`run --out`) traces, then `.venv/bin/python -m proteus compare h.jsonl llm.jsonl --out summary.json`
146
  → per-`(model, difficulty)` metric means + n (the `model` token is the part AFTER the colon, e.g.
147
  `fake:demo` → `demo`).
 
 
 
 
 
148
  - Real-model smoke: see the throwaway-venv recipe above (needs a provider SDK + key; never in `.venv`).
149
 
150
  ## Current state
@@ -164,16 +170,59 @@ human-baseline aggregation CLI. Still a single scenario (`predator_evade`, survi
164
  `step_reward`/`safety_distance` ABC surface is now **open** for CP7 to plug new motive categories in
165
  without touching `SessionRunner`.
166
 
167
- ## Next: CP7 (new motive categories / scenarios) needs its own plan
168
-
169
- Per spec §12 (deferred from CP6). CP6 opened the `step_reward`/`safety_distance` ABC interface so a new
170
- motive **category** (e.g. curiosity = reward for visiting novel cells; sociality) can be added as a new
171
- `Scenario` subclass that defines its own per-turn reward **without touching `SessionRunner`** (reward is
172
- now scenario-owned) or the metrics/compare harness (additive, category-agnostic). Likely scope: a second
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
173
  scenario + its category reward + its own diagnostic invariant golden, reusing the difficulty-layout and
174
  rollout machinery. The `VanillaAgent._ACTION_DIRECTIVE` / `SessionRunner._PROBE_QUESTION` predator-framing
175
- (carried below) should be sourced from the `Scenario` when the second scenario lands. LLM-as-judge scoring
176
- of reasoning and the web UI / leaderboard remain deferred (spec §12). Brainstorm write the CP7 plan → execute.
 
 
177
 
178
  ## Deferred items (carry forward)
179
 
 
4
  > `docs/superpowers/specs/2026-06-01-proteus-arena-slice-design.md` (immutable design) +
5
  > the per-CP plan in `docs/superpowers/plans/` + this file.
6
 
7
+ **Last updated:** 2026-06-02 — CP7 complete (LLM-generated memory pre-roll; hybrid handover; offline
8
+ suite 152 passed + Ollama acceptance smoke passed). CP6 before it (difficulty layouts + category eval +
9
+ `compare`).
10
 
11
  ## Done (CP0–CP3)
12
 
 
146
  (`run --out`) traces, then `.venv/bin/python -m proteus compare h.jsonl llm.jsonl --out summary.json`
147
  → per-`(model, difficulty)` metric means + n (the `model` token is the part AFTER the colon, e.g.
148
  `fake:demo` → `demo`).
149
+ - Memory pre-roll (CP7): generate a checkpoint then reuse it —
150
+ `.venv/bin/python -m proteus memory --scenario predator_evade --model fake:demo --seed 42 --memory-turns 8`
151
+ writes `runs/memory/<model>/<stamp>.json`; `proteus run … --memory latest` (or `--memory generate` to do
152
+ both in one shot, `--memory <path>` for a specific file, default `none`) injects it at the handover.
153
+ `runs/memory/` is gitignored. Korean usage guide: `docs/USAGE-ko.md`.
154
  - Real-model smoke: see the throwaway-venv recipe above (needs a provider SDK + key; never in `.venv`).
155
 
156
  ## Current state
 
170
  `step_reward`/`safety_distance` ABC surface is now **open** for CP7 to plug new motive categories in
171
  without touching `SessionRunner`.
172
 
173
+ ## Done (CP7 LLM-generated memory pre-roll; design: `docs/superpowers/specs/2026-06-02-proteus-cp7-memory-preroll-design.md`; plan: `docs/superpowers/plans/2026-06-02-proteus-cp7-memory-preroll.md`)
174
+
175
+ Executed inline (executing-plans, TDD, commit-per-task). **Full suite: 152 passed** (127 baseline + 25
176
+ CP7), offline/headless; no provider SDK added to `.venv`. The original "CP7 = new motive category" idea
177
+ was **renamed to CP8** when the user pivoted to this memory feature; curiosity/sociality stay deferred.
178
+
179
+ - **Stage 1 — `runtime/memory.py`** (pure pydantic + stdlib, no intra-runtime imports, like `trace.py`):
180
+ `MemoryTurn`/`MemoryCheckpoint` models; `save_checkpoint`/`load_checkpoint` (single-file JSON at
181
+ `runs/memory/<safe(model)>/<stamp>.json`); `latest_for_model` (lexical stamp = chronological);
182
+ `render_memory_block` (pure observation renderer). Exported from `runtime/__init__`.
183
+ - **Stage 2 — `runtime/memory_gen.py`** (engine-coupled, like `rollout.py`): `generate_memory(scenario,
184
+ agent, *, difficulty, seed, memory_turns, model_name, clock)` runs a **full-info self-play** episode
185
+ (`max_steps=memory_turns`, no scripted Cut, no answer-key leakage), capturing frames/actions/reasoning
186
+ → `MemoryCheckpoint`. Deterministic under an injected `clock`. New scenario-sourced `Scenario.memory_brief`
187
+ (concrete attr default `""`); `predator_evade` sets a transparent brief (discloses the BFS chase). This
188
+ resolves the **memory-prompt** half of the old "source prompts from Scenario" item; the
189
+ `_PROBE_QUESTION`/`_ACTION_DIRECTIVE` predator-framing is still hard-coded (carried to CP8).
190
+ - **Stage 3 — hybrid handover** (`trace.py` + `session.py`): additive `SessionTrace.memory_ref: str | None`;
191
+ `SessionRunner(..., memory=None, memory_ref=None)` prepends the memory block (+ `NOW — this run so far:`)
192
+ to the **turn-1** observation only, **before** the unchanged scripted Cut. `step_reward`, answer keys,
193
+ metrics, and `_session_core.py` untouched → diagnostic invariant + 127 preserved (pinned by
194
+ `test_session_memory.py`: with/without memory the per-turn `motive_action`/`habit_action`/`is_diagnostic`
195
+ and `metrics` are identical; only the turn-1 observation grows).
196
+ - **Stage 4 — CLI** (`cli.py`): `proteus memory` (generate+save a checkpoint) and `proteus run --memory
197
+ MODE` where `MODE ∈ {none(default), generate, latest, <path>}` via a `_resolve_memory` helper. `--memory-turns`
198
+ (default 10) and `--memory-root` (default `runs/memory`) on both. Exit codes mirror `run` (unknown
199
+ model/scenario → 2; `latest` with no checkpoint → 2).
200
+
201
+ ### CP7 acceptance smoke (passed)
202
+
203
+ - **Offline end-to-end:** `proteus memory … fake:demo` → `proteus run … --memory latest` → the turn-1
204
+ observation carries the `MEMORY` block + `NOW` separator + scripted `Cut 1/2:` in order; `memory_ref` set.
205
+ - **Ollama (networked, manual):** `ollama:gpt-oss:120b-cloud`, seed 42 / easy. `proteus memory
206
+ --memory-turns 6` → the **real model self-played 6 turns and survived** (vs the fake `stay` agent which
207
+ is eliminated in 2), checkpoint written to `runs/memory/gpt-oss_120b-cloud/<stamp>.json`. Then `run
208
+ --play-turns 8 --memory latest` → **survived | motive_reading_accuracy=50% | reactivity_index=40%**;
209
+ turn-1 observation = 962 chars containing the 6-turn memory block + the scripted Cut;
210
+ `memory_ref="gpt-oss:120b-cloud@<stamp>"`. `.venv` stayed SDK-free (throwaway `/tmp/proteus-smoke-venv`
211
+ with `httpx`; key = first whitespace token of the `.env` value, CP4 gotcha). `runs/` is gitignored, so
212
+ the real trace/checkpoint/key were never committed.
213
+
214
+ ## Next: CP8 (new motive categories / scenarios) — needs its own plan
215
+
216
+ Per spec §12 (the work formerly numbered CP7). CP6 opened the `step_reward`/`safety_distance` ABC interface
217
+ so a new motive **category** (e.g. curiosity = reward for visiting novel cells; sociality) can be added as a
218
+ new `Scenario` subclass with its own per-turn reward — **without touching `SessionRunner`** (reward is
219
+ scenario-owned) or the metrics/compare harness (additive, category-agnostic). Likely scope: a second
220
  scenario + its category reward + its own diagnostic invariant golden, reusing the difficulty-layout and
221
  rollout machinery. The `VanillaAgent._ACTION_DIRECTIVE` / `SessionRunner._PROBE_QUESTION` predator-framing
222
+ (carried below) should be sourced from the `Scenario` when the second scenario lands (CP7 already added
223
+ `Scenario.memory_brief` as the pattern to follow). The curiosity brainstorming explored two diagnostic
224
+ geometries (A: blocked-dead-end analog with novelty reward; B: explored-pocket) — pick one in CP8. LLM-as-judge
225
+ scoring of reasoning and the web UI / leaderboard remain deferred (spec §12). Brainstorm → write the CP8 plan → execute.
226
 
227
  ## Deferred items (carry forward)
228