Add hybrid Oracle layer and update architecture docs

README.md

@@ -1,6 +1,6 @@
 ---
 title: ReplicaLab
-emoji: 🧪
+emoji: "🧪"
 colorFrom: blue
 colorTo: green
 sdk: docker
@@ -14,37 +14,53 @@ pinned: false
 
 > *How do we adapt a plan without breaking the objective?*
 
-ReplicaLab trains an agent to negotiate high-quality plans under real constraints. The initial domain focus is mathematics and machine learning, with finance and trading design in offline or backtest form as the third scenario family. Physics and biology remain later adapters once the core normalized scenario layer is stable.
+ReplicaLab trains a Scientist policy to negotiate better plans under real constraints. The initial domain focus is mathematics and machine learning, with offline finance and trading design as the third scenario family. Physics and biology remain future adapters after the core normalized scenario layer is stable.
 
 ## Current Build Status
 
-- The repository is now past the foundation stage and has a working real environment plus deterministic judge pipeline.
-- The Python package foundation is verified through editable install plus full test-suite checks.
-- Shared contracts currently live in `replicalab/models.py`, with the signed-off freeze in `docs/fnd08_frozen_json_contract.md`.
-- `server/app.py` now serves the real `ReplicaLabEnv` by default, with the legacy stub retained only as a fallback safety path.
+- The repository is past the foundation stage and has a working real environment plus deterministic judge pipeline.
+- The Python package foundation is verified through editable install plus the full test suite.
+- Shared contracts live in `replicalab/models.py`, with the signed-off freeze in `docs/fnd08_frozen_json_contract.md`.
+- `server/app.py` serves the real `ReplicaLabEnv` by default, with the legacy stub retained only as a fallback path.
 - `openenv.yaml` exists and passes local OpenEnv validation.
 - Local Docker validation has been completed for the server image on port `7860`.
-- Hugging Face Spaces deployment is live at `https://ayushozha-replicalab.hf.space` with all endpoints verified.
-- The frozen outer contract remains stable while the internal scenario engine moves toward a normalized scenario pack.
-- The planned Lab Manager path is hybrid: model-backed negotiation language plus deterministic feasibility grounding.
+- Hugging Face Spaces deployment is live at `https://ayushozha-replicalab.hf.space` for the deterministic environment path.
+- The frozen outer contract remains stable while the internal scenario engine uses a normalized scenario pack.
+- The Lab Manager path is hybrid: deterministic feasibility truth with optional model-backed narrative responses.
+- An additive Oracle hybrid layer now exists for optional frontier-model world generation, event injection, Lab Manager narration, and post-mortem analysis while deterministic scoring remains the canonical RL reward path.
 
 ## Team Ownership
 
 | Owner | Current focus |
 |------|----------------|
-| Kian (Person A) | Shared schemas, validation, normalized scenario engine, judge logic |
-| Person B (Ayush) | Contract freeze, domain-neutral Scientist prompting and parsing, notebook and client path |
-| Max (Person C) | Repo/runtime setup, frontend shell, server and deployment plumbing |
-| Kush (Person D) | README and demo docs, UI shell planning, polish and presentation assets |
+| Kian (Person A) | Shared schemas, validation, scenario engine, judge logic |
+| Person B (Ayush) | Scientist prompting and parsing, notebook and client path |
+| Max (Person C) | Server, deployment, and runtime plumbing |
+| Kush (Person D) | Frontend, UI polish, docs, and demo assets |
 
 ---
 
 ## Architecture
 
 <p align="center">
-  <img src="./architecture.svg" alt="ReplicaLab Architecture" width="100%"/>
+  <img src="./ReplicaLab_Architecture_v2.svg" alt="ReplicaLab Hybrid Architecture" width="100%"/>
 </p>
 
+ReplicaLab uses a **hybrid Oracle architecture**:
+
+- The **Oracle layer** is optional and powers world-building and narrative intelligence:
+  - richer scenario generation
+  - optional event injection
+  - optional LLM Lab Manager narration
+  - optional post-mortem analysis
+- The **deterministic core** remains canonical for RL:
+  - environment transitions
+  - validation
+  - grounded Lab Manager feasibility
+  - judge scoring and reward math
+
+This satisfies the sponsor-facing “LLM as environment intelligence” direction without making reward noisy or irreproducible.
+
 ---
 
 ## How It Works
@@ -55,26 +71,31 @@ Each episode simulates a negotiation between two agents inside a constrained tec
 |------|------|----------------|
 | **Scientist** | Trainable model policy | Proposes plans, asks questions, and preserves objective quality |
 | **Lab Manager** | Hybrid model-backed policy with deterministic grounding | Negotiates revisions while the checker enforces feasibility and constraint truth |
-| **Judge** | Deterministic rubric engine | Scores the final plan on Rigor, Feasibility, and Fidelity |
+| **Judge** | Deterministic rubric engine | Scores the final plan on rigor, feasibility, fidelity, and parsimony |
+| **Oracle (optional)** | Frontier-model intelligence layer | Generates richer worlds, optional events, optional live LM narration, and post-mortem analysis |
 
 ### Episode Lifecycle
 
-1. **Reset** -- `reset(seed)` generates a normalized scenario pack, mapped role observations, and a hidden reference spec
-2. **Scientist observes** -- Task summary, experiment or benchmark goal, conversation history, current plan
-3. **Lab Manager observes** -- Resource, scheduling, staffing, and policy constraints mapped from the same normalized pack
-4. **Negotiation** -- Multiple rounds of proposals, counteroffers, and questions
-5. **Agreement or timeout** -- Both accept, or the round limit is reached
-6. **Reward** -- Judge scores the final plan against the hidden reference spec
+1. **Reset**: `reset(seed)` builds a normalized scenario pack and hidden reference spec.
+2. **Scientist observes**: task summary, goal, history, and current plan.
+3. **Lab Manager observes**: resource, scheduling, staffing, and policy constraints from the same normalized pack.
+4. **Negotiation**: multiple rounds of proposals, counteroffers, and questions.
+5. **Agreement or timeout**: both accept, or the round limit is reached.
+6. **Reward**: the deterministic judge scores the final plan.
+7. **Optional Oracle overlays**: event injection, round commentary, and post-mortem may be layered on top without replacing deterministic reward.
 
 ### Reward Formula
 
-```
-total_reward = 10 * rigor * feasibility * fidelity + efficiency_bonus + communication_bonus - penalties
+```text
+total_reward = 10 * rigor * feasibility * fidelity * parsimony
+             + efficiency_bonus
+             + communication_bonus
+             - penalties
 ```
 
-The **multiplicative core** prevents fake wins: a theoretically strong but impossible plan scores low, and a cheap but invalid plan also scores low.
+The multiplicative core prevents fake wins: a theoretically strong but impossible plan scores low, and a cheap but invalid plan also scores low. Even when the Oracle layer is enabled, this deterministic path remains canonical for RL training and before/after evaluation.
 
-### Internal normalization rule
+### Internal Normalization Rule
 
 The outer action and observation models stay stable. Domain-specific content is converted into a normalized scenario pack first, then mapped into the current `ScientistObservation` and `LabManagerObservation` contracts. Prompts are assembled from that normalized data rather than hard-coded per domain.
 
@@ -82,27 +103,24 @@ The outer action and observation models stay stable. Domain-specific content is
 
 ## Getting Started
 
-This section mixes verified foundation commands with planned end-to-end commands. As of 2026-03-08, the verified local path is the editable Python install plus the shared-model import smoke test.
+This section mixes verified foundation commands with planned end-to-end commands.
 
 ### Prerequisites
 
 - Python 3.10+
-- Node.js 18+ (for the frontend)
-- Docker (for deployment)
-- A Google Colab account (for RL training)
+- Node.js 18+
+- Docker
+- A notebook runtime such as Google Colab or the H100-backed Jupyter environment
 
 ### Installation
 
 ```bash
-# Clone the repository
 git clone https://github.com/Ayush10/replicalab-ai.git
 cd replicalab-ai
 
-# Create a virtual environment
 python -m venv .venv
-source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+source .venv/bin/activate  # Windows: .venv\Scripts\activate
 
-# Install Python dependencies
 pip install -e ".[dev]"
 ```
 
@@ -115,11 +133,10 @@ python -c "from replicalab.models import ScientistAction, LabManagerAction; prin
 ### Running the Environment Server
 
 ```bash
-# Planned command once server wiring lands
 python -m server.app
 ```
 
-The server is intended to start at `http://localhost:7860` once the server task chain is complete.
+The server is intended to start at `http://localhost:7860`.
 
 ### Running the Frontend
 
@@ -129,8 +146,6 @@ npm install
 npm run dev
 ```
 
-The React UI is intended to start at `http://localhost:5173` once the frontend shell and Vite config are in place.
-
 ### Running Tests
 
 ```bash
@@ -141,30 +156,36 @@ pytest tests/
 
 ## Training the Scientist
 
-RL training improves the Scientist agent's ability to negotiate effective, feasible plans.
+RL training improves the Scientist agent’s ability to negotiate effective, feasible plans.
 
-### Selected base model
+### Selected Base Model
 
 - **Primary Scientist model:** `Qwen3-4B`
 - **Stretch fallback:** `Qwen3-8B`
 - **Decision record:** `docs/agt11_scientist_model_selection.md`
 
-### Quick Start (Google Colab)
+### Planned Training Path
 
-1. Open `notebooks/train_colab.ipynb` in Google Colab
-2. Connect to a GPU runtime
-3. Run all cells -- the notebook handles environment setup, rollout, and training via TRL/Unsloth with GRPO
+1. Connect the notebook to the environment via `replicalab/client.py`
+2. Collect rollouts with `replicalab/training/rollout.py`
+3. Train with **Unsloth or HF TRL**
+4. Save:
+   - reward curves
+   - component curves
+   - before/after evaluation metrics
+   - replay and plot artifacts
 
 ### Training Loop
 
-```
-Environment resets -> Scientist proposes -> Lab Manager responds -> ... -> Episode ends -> Reward computed -> Policy updated
+```text
+reset -> Scientist acts -> Lab Manager responds -> ... -> episode ends -> deterministic reward -> policy update
 ```
 
-**Target behaviors over training:**
+### Target Behaviors Over Training
+
 - Ask better questions before committing to a plan
 - Preserve critical checks, assumptions, and required steps
-- Choose realistic substitutions when a preferred method or resource is unavailable
+- Choose realistic substitutions when preferred resources are unavailable
 - Reach agreement in fewer rounds
 - Avoid impossible or over-budget plans
 
@@ -172,7 +193,7 @@ Environment resets -> Scientist proposes -> Lab Manager responds -> ... -> Episo
 
 ## Scenario System
 
-Scenarios are generated deterministically from a seed. Each template first emits a normalized scenario pack with:
+Scenarios are generated deterministically from a seed. Each template emits a normalized scenario pack with:
 
 - `task_summary`
 - `success_criteria`
@@ -181,7 +202,7 @@ Scenarios are generated deterministically from a seed. Each template first emits
 - `allowed_substitutions`
 - `hidden_reference_spec`
 
-Difficulty scaling should then mechanically tighten constraints, remove resources, or add conflicts instead of changing the outer contract or prompt structure.
+Difficulty scaling should mechanically tighten constraints, remove resources, or add conflicts instead of changing the outer contract or prompt structure.
 
 | Difficulty | Description |
 |------------|-------------|
@@ -192,7 +213,7 @@ Difficulty scaling should then mechanically tighten constraints, remove resource
 ### Included Scenario Templates
 
 | Template | Domain | Example Task |
-|----------|--------|--------------------|
+|----------|--------|--------------|
 | `math_reasoning` | Mathematics | Proof planning under tool, review, and time constraints |
 | `ml_benchmark` | Machine learning | Model evaluation with dataset, compute, and time constraints |
 | `finance_trading` | Finance and trading | Offline strategy and backtest planning under risk and capital limits |
@@ -201,57 +222,68 @@ Difficulty scaling should then mechanically tighten constraints, remove resource
 
 ## Project Structure
 
-```
+```text
 replicalab-ai/
 ├── README.md
-├── architecture.svg
+├── ReplicaLab_Architecture_v2.svg
 ├── pyproject.toml
 ├── openenv.yaml
 ├── replicalab/
 │   ├── __init__.py
-│   ├── models.py              # Action, Observation, State schemas
-│   ├── client.py              # OpenEnv client wrapper
+│   ├── models.py                # Action, Observation, State schemas
+│   ├── client.py                # OpenEnv client wrapper
+│   ├── oracle.py                # Optional frontier-model Oracle wrapper
+│   ├── oracle_models.py         # Oracle scenario and post-mortem schemas
+│   ├── cache.py                 # Cached Oracle scenario generation
 │   ├── prompts/
-│   │   ├── scientist.txt      # Scientist system prompt
-│   │   ├── lab_manager.txt    # Lab Manager response templates
-│   │   └── judge.txt          # Judge rubric prompt
+│   │   ├── scientist.txt
+│   │   ├── lab_manager.txt
+│   │   ├── judge.txt
+│   │   ├── oracle_world_architect.txt
+│   │   ├── oracle_adjudicator.txt
+│   │   ├── oracle_event_injector.txt
+│   │   ├── oracle_post_mortem.txt
+│   │   └── oracle_lab_manager.txt
 │   ├── scenarios/
-│   │   ├── templates.py       # Normalized scenario template layer
-│   │   ├── math_reasoning.py  # Mathematics scenarios
-│   │   ├── ml_benchmark.py    # ML benchmark scenarios
+│   │   ├── templates.py         # Normalized scenario pack + Oracle adapter
+│   │   ├── math_reasoning.py
+│   │   ├── ml_benchmark.py
 │   │   └── finance_trading.py
 │   ├── scoring/
-│   │   ├── rubric.py          # Main scoring engine
-│   │   ├── rigor.py           # Objective-validity scorer
-│   │   ├── feasibility.py     # Constraint feasibility scorer
-│   │   └── fidelity.py        # Hidden-reference fidelity scorer
+│   │   ├── rubric.py            # Canonical deterministic reward math
+│   │   ├── rigor.py
+│   │   ├── feasibility.py
+│   │   ├── fidelity.py
+│   │   └── explain.py
 │   ├── agents/
 │   │   ├── scientist_policy.py
 │   │   ├── lab_manager_policy.py
+│   │   ├── lab_manager_agent.py # Optional LLM Lab Manager wrapper
 │   │   └── judge_policy.py
 │   ├── env/
-│   │   └── replicalab_env.py  # OpenEnv environment implementation
+│   │   └── replicalab_env.py    # Real env with optional Oracle hooks
+│   ├── training/
+│   │   └── rollout.py
 │   └── utils/
 │       ├── seed.py
 │       ├── validation.py
 │       └── logging.py
 ├── server/
-│   ├── app.py                 # FastAPI + WebSocket server
+│   ├── app.py
 │   ├── requirements.txt
 │   └── Dockerfile
 ├── frontend/
 │   ├── package.json
 │   ├── vite.config.ts
 │   └── src/
-│       ├── App.tsx
-│       ├── components/
-│       └── pages/
 ├── notebooks/
-│   └── train_colab.ipynb      # RL training notebook
+│   └── train_colab.ipynb
 └── tests/
     ├── test_env.py
     ├── test_reward.py
     ├── test_scenarios.py
+    ├── test_oracle.py
+    ├── test_cache.py
     └── test_server.py
 ```
 
@@ -268,16 +300,24 @@ docker run -p 7860:7860 replicalab
 
 ### Hugging Face Spaces
 
-**Live deployment:** https://ayushozha-replicalab.hf.space
+**Live deployment:** `https://ayushozha-replicalab.hf.space`
 
 The app is deployed on HF Spaces with `sdk: docker` on port `7860`.
 
 ```bash
-# Verify the live Space
 curl https://ayushozha-replicalab.hf.space/health
-# → {"status":"ok","env":"real"}
+# -> {"status":"ok","env":"real"}
 ```
 
+Current Space deployment is complete for the deterministic environment path. If live Oracle mode is enabled later, the Space will additionally need:
+
+- provider SDK dependencies
+- model API-key secrets
+- runtime feature flags
+- cold-start and latency handling
+
+The deterministic deployment itself does not need to be redesigned.
+
 ---
 
 ## Toolchain
@@ -291,7 +331,7 @@ curl https://ayushozha-replicalab.hf.space/health
 | **Tailwind + shadcn/ui** | Styling |
 | **Docker** | Packaging |
 | **Hugging Face Spaces** | Public hosting |
-| **Google Colab** | Training notebook |
+| **Notebook / Colab / H100** | Training and evaluation |
 
 ---
 

docs/changes.md

@@ -58,4 +58,6 @@ Rules:
 | 2026-03-08 | Person B (Ayush) | TRN 04 | Implemented the rollout collection loop as a reusable Python module rather than only inside a notebook | The backlog labels `TRN 04` as notebook work, but implementing it in `replicalab/training/rollout.py` makes the same rollout logic reusable across notebooks, tests, and future trainer code while preserving the required behavior | Extended `RolloutWorker` with terminal `StepInfo`, bounded tool trace aggregation, and `collect_rollouts(...)`; added trace and batch tests in `tests/test_rollout_traces.py` and kept the rollout logic fully testable outside a notebook | `TRN 05` is now unblocked and notebooks can import the rollout loop instead of reimplementing it |
 | 2026-03-08 | Person B (Ayush) | API 14 | Completed the REST session isolation verification even though the task was assigned to Person C | The session isolation logic already worked correctly in `server/app.py`; the task was still marked partial because no dedicated tests proved concurrent-user isolation against the real env | Created `tests/test_api_rest_isolation.py` with 11 tests covering session independence, round-count isolation, terminal isolation, session_id reuse, invalid session handling, and replay isolation; no server changes needed; 307 tests pass | No new dependencies unblocked; `API 14` was the last partial API task besides `API 01` and `OBS 02` |
 | 2026-03-08 | Person B (Ayush) | MOD 07 and MOD 10 | Closed the replay-persistence and schema-example tasks on Max's lane after verifying the code that had already landed | `replicalab/utils/logging.py` and the API example generator were implemented and passing tests, but the source-of-truth backlog and Max's owner docs still showed both tasks as not started, and the generated examples still contained stale stub audit text | Updated `tests/fixtures/generate_api_examples.py` to derive terminal judge metadata from the current deterministic judge helpers, regenerated `api_schema_examples.json`, and synced `MOD 07`/`MOD 10` to complete in the comprehensive backlog, completion rollup, and Max owner docs | `MOD 08` and `JDG 07` are now clearly unblocked in the tracked plan |
+| 2026-03-08 | Person B (Ayush) | Reward shaping and rubric refinement | Expanded the reward system beyond terminal-only scoring without reopening the outer action or observation contract | Sparse terminal-only reward was too weak for RL training, and the project needed deterministic shaping rather than a frontier-model reward source | Added a parsimony term to terminal reward, introduced deterministic step shaping in `ReplicaLabEnv` (information gain, protocol delta, momentum, contradiction, hallucination, stalling, regression, invalid-action, timeout, and no-agreement signals), updated rollout aggregation to use cumulative episode reward, and aligned env/server tests to the new shaped-reward semantics while keeping the full suite green at 356 tests | Keep the notebook and training plots explicit about terminal reward components vs cumulative shaped episode reward |
+| 2026-03-08 | Person B (Ayush) | Oracle hybrid architecture | Added an Oracle-style frontier-model layer as an additive integration instead of replacing the deterministic environment and reward stack | The sponsor-facing V2 direction calls for an LLM woven through scenario generation, environment interaction, and explanation, but the RL training path still needs deterministic reward and reproducible evaluation | Added `oracle_models.py`, `oracle.py`, `cache.py`, Oracle prompt assets, an optional LLM Lab Manager wrapper, an adapter from Oracle scenarios into the existing normalized scenario pack, and feature-flagged Oracle hooks in `ReplicaLabEnv`; kept deterministic scoring in `replicalab/scoring/*` as the canonical training reward; expanded test coverage with `test_oracle.py`, `test_cache.py`, and Oracle adapter/prompt tests; full suite now passes at 365 tests | If this grows beyond the current additive mode, record any future contract or reward-source changes separately before altering the deterministic training path |
 

docs/map/scoring.md

@@ -6,6 +6,18 @@
 > **Tasks implemented:** JDG 01, JDG 02, JDG 03, JDG 04, JDG 05, JDG 06, JDG 08
 > **Tasks remaining:** JDG 07
 
+## Oracle Hybrid Note
+
+The repo now includes an additive Oracle layer for richer scenario generation,
+optional Lab Manager narration, optional event injection, and post-mortem
+analysis. None of that replaces the files in `replicalab/scoring/`.
+
+For RL training, this folder remains the canonical reward source:
+- deterministic
+- reproducible
+- testable
+- used by the environment for the actual scalar reward signal
+
 ## Architecture
 
 ```
@@ -19,6 +31,24 @@ replicalab/scoring/
     explain.py           # JDG 06 — deterministic plain-English explanation
 ```
 
+## Current Reward Structure
+
+The training signal now has two layers:
+
+- **Terminal reward** from `replicalab/scoring/rubric.py`
+  - `10 * rigor * feasibility * fidelity * parsimony`
+  - plus bonuses
+  - minus named penalties
+- **Step shaping reward** from `replicalab/env/replicalab_env.py`
+  - information-gain bonus for novel questions
+  - protocol-delta and momentum bonuses for productive revisions
+  - contradiction, hallucination, stalling, regression, invalid-action,
+    timeout, and no-agreement penalties
+
+The judge remains deterministic. The terminal audit still explains the final
+`RewardBreakdown`, while cumulative episode reward now includes the per-step
+shaping applied inside the environment.
+
 ## Shared Utilities
 
 Token matching extracted into `replicalab/utils/text.py`:

docs/map/tests.md

@@ -1,6 +1,6 @@
 # Tests Map - `tests/`
 
-> 327 tests across 14 files. All passing.
+> 365 tests across 18 files. All passing.
 >
 > **Last verified:** 2026-03-08
 
@@ -8,21 +8,25 @@
 
 | File | Tests | What it covers |
 |------|-------|----------------|
+| `test_api_rest_isolation.py` | 11 | `API 14` REST session isolation and replay separation |
+| `test_cache.py` | 2 | Oracle scenario caching and reuse |
 | `test_client.py` | 24 | `TRN 13` reusable client over REST and WebSocket |
 | `test_config.py` | 3 | Shared constants and config consistency |
 | `test_env.py` | 56 | `ENV 01-08`, `ENV 10`, `ENV 11`, `OBS 04`, `JDG 04-05`, `TST 01-03` |
 | `test_judge_policy.py` | 10 | `JDG 11` structured judge audit payload |
 | `test_lab_manager_policy.py` | 37 | `AGT 05-07` plus `AGT 09` determinism coverage |
 | `test_models.py` | 21 | Action, observation, step, state, and log contracts |
-| `test_prompts.py` | 6 | `AGT 10` prompt files and bounded-tool rendering |
+| `test_logging.py` | 11 | `MOD 07` replay persistence and `JDG 07` CSV logging helpers |
+| `test_oracle.py` | 5 | Oracle hybrid wrapper, structured parsing, and env reset adapter |
+| `test_prompts.py` | 7 | `AGT 10` prompt files and Oracle prompt asset loading |
 | `test_reward.py` | 40 | `JDG 01-06`, `JDG 08`, and reward regression coverage |
 | `test_rollout.py` | 12 | `TRN 03` rollout worker behavior |
 | `test_rollout_traces.py` | 2 | `TRN 04` bounded tool trace aggregation and batched collection |
-| `test_scenarios.py` | 13 | `SCN 01-13` scenario generation and determinism |
+| `test_scenarios.py` | 14 | `SCN 01-13` scenario generation, determinism, and Oracle scenario adaptation |
 | `test_scientist_policy.py` | 46 | `MOD 09`, `AGT 01-04`, `AGT 08` |
-| `test_server.py` | 37 | `API 02-04`, `API 06-08`, `API 13`, replay audit propagation |
+| `test_server.py` | 44 | `API 01-04`, `API 06-08`, `API 13-14`, replay audit propagation, and root landing page |
 | `test_validation.py` | 20 | `MOD 05-06` semantic validation |
-| **Total** | **327** | |
+| **Total** | **365** | |
 
 ## Coverage Notes
 
@@ -34,6 +38,8 @@
   - `test_scientist_policy.py`, `test_prompts.py`, `test_rollout.py`, and `test_rollout_traces.py` together cover prompt construction, observation formatting, parse/retry, baseline policy, rollout collection, and bounded tool trace capture.
 - The judge stack is covered end to end:
   - `test_reward.py` covers rubric scores and reward math, while `test_judge_policy.py` covers structured audit payload generation.
+- The Oracle hybrid layer is covered additively:
+  - `test_oracle.py`, `test_cache.py`, and `test_prompts.py` cover Oracle scenario generation wrappers, cache reuse, and prompt asset loading without changing the deterministic reward contract.
 
 ## Remaining Gaps
 
@@ -47,6 +53,7 @@
 |------|--------------------|
 | Models and contracts | `test_models.py`, `test_validation.py` |
 | Scenarios | `test_scenarios.py` |
+| Oracle integration and cache | `test_oracle.py`, `test_cache.py`, `test_prompts.py` |
 | Scientist policy | `test_scientist_policy.py`, `test_prompts.py` |
 | Lab Manager policy | `test_lab_manager_policy.py` |
 | Judge and reward | `test_reward.py`, `test_judge_policy.py` |

replicalab/__init__.py

@@ -1,3 +1,5 @@
+from replicalab.cache import CachedOracle, ScenarioCache
 from replicalab.client import ReplicaLabClient
+from replicalab.oracle import Oracle
 
-__all__ = ["ReplicaLabClient"]
+__all__ = ["CachedOracle", "Oracle", "ReplicaLabClient", "ScenarioCache"]

replicalab/agents/__init__.py

@@ -4,6 +4,7 @@ from .judge_policy import (
     JudgeAudit,
     build_judge_audit,
 )
+from .lab_manager_agent import LabManagerAgent
 from .lab_manager_policy import (
     AlternativeSuggestion,
     FeasibilityCheckResult,
@@ -27,6 +28,7 @@ __all__ = [
     "AlternativeSuggestion",
     "FeasibilityCheckResult",
     "JudgeAudit",
+    "LabManagerAgent",
     "RetryMetadata",
     "ScientistCallResult",
     "ScientistOutputParseError",

replicalab/agents/judge_policy.py

@@ -109,6 +109,7 @@ def _derive_failure_reasons(
         (breakdown.feasibility, "feasibility", "Feasibility remained too low under the scenario constraints."),
         (breakdown.fidelity, "fidelity", "The final plan diverged too far from the hidden reference requirements."),
         (breakdown.rigor, "rigor", "The plan missed required checks or justification quality targets."),
+        (breakdown.parsimony, "parsimony", "The final plan requested more resources or controls than the scenario complexity justified."),
     ]
     for score, _name, message in components:
         if score < _WEAK_THRESHOLD:
@@ -119,6 +120,13 @@ def _derive_failure_reasons(
     _PENALTY_LABELS: dict[str, str] = {
         "invalid_tool_use": "A bounded-tool usage violation was detected.",
         "unsupported_claim": "An unsupported evidence claim was penalized.",
+        "timeout": "A timeout penalty was applied at the round limit.",
+        "no_agreement": "A no-agreement penalty was applied.",
+        "invalid_action": "An invalid action penalty was applied after a failed protocol proposal.",
+        "hallucination": "A hallucination penalty was applied for unsupported inventory references.",
+        "contradiction": "A contradiction penalty was applied for repeating blocked requirements.",
+        "stalling": "A stalling penalty was applied for repeating an unproductive move.",
+        "regression": "A regression penalty was applied because the revision worsened the protocol.",
     }
     for key, amount in sorted(breakdown.penalties.items()):
         if amount > 0:

replicalab/agents/lab_manager_agent.py

@@ -0,0 +1,47 @@
+"""Optional LLM-backed Lab Manager narration layer."""
+
+from __future__ import annotations
+
+import json
+from typing import Any
+
+from replicalab.oracle import call_json_model
+from replicalab.oracle_models import LabManagerResponse, OracleLabManagerObservation
+from replicalab.prompts import load_prompt_asset
+
+
+class LabManagerAgent:
+    """LLM-based Lab Manager driven by Oracle-generated constraints.
+
+    This is additive to the deterministic feasibility checker. The current
+    env can use this agent to narrate or enrich responses while keeping
+    canonical feasibility and reward logic deterministic.
+    """
+
+    def __init__(self, client: Any, model: str = "frontier-oracle") -> None:
+        self.client = client
+        self.model = model
+
+    def respond(self, observation: OracleLabManagerObservation) -> LabManagerResponse:
+        system = load_prompt_asset("oracle_lab_manager")
+        user = (
+            "A Scientist has taken an action. Respond as the Lab Manager.\n\n"
+            "YOUR LAB CONSTRAINTS (ground truth, do not deviate):\n"
+            f"{observation.lab_constraints.model_dump_json(indent=2)}\n\n"
+            "CURRENT PROTOCOL ON THE TABLE:\n"
+            f"{json.dumps(observation.current_protocol, indent=2) if observation.current_protocol else 'None yet'}\n\n"
+            f"SCIENTIST'S ACTION (round {observation.round_number}):\n"
+            f"{observation.scientist_action.model_dump_json(indent=2)}\n\n"
+            "Respond ONLY with valid JSON matching LabManagerResponse.\n"
+            "No markdown. No preamble."
+        )
+        return call_json_model(
+            self.client,
+            model=self.model,
+            system=system,
+            user=user,
+            response_model=LabManagerResponse,
+        )
+
+
+__all__ = ["LabManagerAgent"]

replicalab/cache.py

@@ -0,0 +1,66 @@
+"""Scenario caching for Oracle-generated environments."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+from pathlib import Path
+from typing import Optional
+
+from replicalab.config import ORACLE_SCENARIO_CACHE_DIR
+from replicalab.oracle import Oracle
+from replicalab.oracle_models import Scenario
+
+
+class ScenarioCache:
+    """Cache Oracle-generated scenarios by seed, difficulty, and domain."""
+
+    def __init__(self, cache_dir: str | Path = ORACLE_SCENARIO_CACHE_DIR) -> None:
+        self.cache_dir = Path(cache_dir)
+        self.cache_dir.mkdir(parents=True, exist_ok=True)
+
+    def _key(self, seed: int, difficulty: str, domain: str) -> str:
+        raw = f"{seed}:{difficulty}:{domain}"
+        return hashlib.md5(raw.encode("utf-8")).hexdigest()
+
+    def _path(self, seed: int, difficulty: str, domain: str) -> Path:
+        return self.cache_dir / f"{self._key(seed, difficulty, domain)}.json"
+
+    def get(self, seed: int, difficulty: str, domain: str) -> Optional[Scenario]:
+        path = self._path(seed, difficulty, domain)
+        if not path.exists():
+            return None
+        return Scenario.model_validate(json.loads(path.read_text(encoding="utf-8")))
+
+    def put(self, seed: int, difficulty: str, domain: str, scenario: Scenario) -> Path:
+        path = self._path(seed, difficulty, domain)
+        path.write_text(scenario.model_dump_json(indent=2), encoding="utf-8")
+        return path
+
+
+class CachedOracle(Oracle):
+    """Oracle wrapper that caches scenario generation by seed."""
+
+    def __init__(
+        self,
+        client: object,
+        model: str = "frontier-oracle",
+        *,
+        cache: ScenarioCache | None = None,
+    ) -> None:
+        super().__init__(client=client, model=model)
+        self.cache = cache or ScenarioCache()
+
+    def generate_scenario(self, seed: int, difficulty: str, domain: str) -> Scenario:
+        cached = self.cache.get(seed, difficulty, domain)
+        if cached is not None:
+            return cached
+        scenario = super().generate_scenario(seed=seed, difficulty=difficulty, domain=domain)
+        self.cache.put(seed, difficulty, domain, scenario)
+        return scenario
+
+
+__all__ = [
+    "CachedOracle",
+    "ScenarioCache",
+]

replicalab/config.py

@@ -29,3 +29,29 @@ API_PORT = 7860
 
 LOG_LEVEL = os.environ.get("REPLICALAB_LOG_LEVEL", "INFO").upper()
 LOG_FORMAT = "%(asctime)s [%(levelname)s] %(name)s: %(message)s"
+
+ORACLE_ENABLED = os.environ.get("REPLICALAB_ORACLE_ENABLED", "0") == "1"
+ORACLE_EVENTS_ENABLED = os.environ.get("REPLICALAB_ORACLE_EVENTS_ENABLED", "0") == "1"
+ORACLE_POST_MORTEM_ENABLED = (
+    os.environ.get("REPLICALAB_ORACLE_POST_MORTEM_ENABLED", "0") == "1"
+)
+ORACLE_MODEL = os.environ.get("REPLICALAB_ORACLE_MODEL", "frontier-oracle")
+ORACLE_SCENARIO_CACHE_DIR = os.environ.get(
+    "REPLICALAB_ORACLE_SCENARIO_CACHE_DIR",
+    ".scenario_cache",
+)
+
+# Deterministic reward shaping constants.
+STEP_PROTOCOL_DELTA_SCALE = 0.25
+STEP_PROTOCOL_DELTA_CAP = 0.3
+STEP_INFO_GAIN_BONUS = 0.05
+STEP_INFO_GAIN_CAP = 0.15
+STEP_MOMENTUM_BONUS = 0.05
+STEP_STALLING_PENALTY = 0.05
+STEP_REPEATED_QUESTION_PENALTY = 0.03
+STEP_REGRESSION_PENALTY = 0.1
+STEP_CONTRADICTION_PENALTY = 0.05
+STEP_INVALID_ACTION_PENALTY = 0.1
+STEP_HALLUCINATION_PENALTY = 0.05
+TERMINAL_TIMEOUT_PENALTY = 0.2
+TERMINAL_NO_AGREEMENT_PENALTY = 0.1

replicalab/models.py

@@ -318,6 +318,9 @@ class RewardBreakdown(BaseModel):
     rigor: float = Field(default=0.0, ge=0, le=1)
     feasibility: float = Field(default=0.0, ge=0, le=1)
     fidelity: float = Field(default=0.0, ge=0, le=1)
+    # Defaults to 1.0 so existing exact-value tests and manual breakdowns
+    # preserve the prior reward semantics unless parsimony is computed.
+    parsimony: float = Field(default=1.0, ge=0, le=1)
     efficiency_bonus: float = 0.0
     communication_bonus: float = 0.0
     penalties: dict[str, float] = Field(default_factory=dict)

replicalab/oracle.py

@@ -0,0 +1,263 @@
+"""Optional frontier-model Oracle wrapper for ReplicaLab.
+
+The Oracle is an additive intelligence layer. It can generate richer
+scenarios, optional round commentary, optional events, and post-mortem
+analyses, while the existing deterministic reward pipeline remains
+canonical for RL training.
+"""
+
+from __future__ import annotations
+
+import json
+from typing import Any, Optional, TypeVar
+
+from pydantic import BaseModel
+
+from replicalab.oracle_models import (
+    AdjudicatorRoundScore,
+    AdjudicatorTerminalScore,
+    EnvironmentEvent,
+    LabManagerResponse,
+    PostMortem,
+    Scenario,
+)
+from replicalab.prompts import load_prompt_asset
+
+T = TypeVar("T", bound=BaseModel)
+
+
+def _strip_markdown_fences(text: str) -> str:
+    cleaned = text.strip()
+    if cleaned.startswith("```"):
+        lines = cleaned.splitlines()
+        if lines:
+            lines = lines[1:]
+        if lines and lines[-1].strip() == "```":
+            lines = lines[:-1]
+        cleaned = "\n".join(lines).strip()
+    return cleaned
+
+
+def _extract_response_text(response: Any) -> str:
+    if isinstance(response, str):
+        return response
+
+    output_text = getattr(response, "output_text", None)
+    if output_text:
+        return output_text
+
+    content = getattr(response, "content", None)
+    if content:
+        chunks: list[str] = []
+        for item in content:
+            text = getattr(item, "text", None)
+            if text:
+                chunks.append(text)
+        if chunks:
+            return "\n".join(chunks)
+
+    output = getattr(response, "output", None)
+    if output:
+        parts: list[str] = []
+        for item in output:
+            inner = getattr(item, "content", None)
+            if not inner:
+                continue
+            for piece in inner:
+                text = getattr(piece, "text", None)
+                if text:
+                    parts.append(text)
+        if parts:
+            return "\n".join(parts)
+
+    raise ValueError("Could not extract text from Oracle client response")
+
+
+def _invoke_client(client: Any, *, model: str, system: str, user: str) -> str:
+    if hasattr(client, "messages") and hasattr(client.messages, "create"):
+        response = client.messages.create(
+            model=model,
+            max_tokens=4096,
+            system=system,
+            messages=[{"role": "user", "content": user}],
+        )
+        return _extract_response_text(response)
+
+    if hasattr(client, "responses") and hasattr(client.responses, "create"):
+        response = client.responses.create(
+            model=model,
+            instructions=system,
+            input=user,
+        )
+        return _extract_response_text(response)
+
+    if callable(client):
+        try:
+            response = client(system=system, user=user, model=model)
+        except TypeError:
+            response = client(system, user)
+        return _extract_response_text(response)
+
+    raise TypeError("Unsupported Oracle client: expected Anthropic/OpenAI-style client or callable")
+
+
+def call_json_model(
+    client: Any,
+    *,
+    model: str,
+    system: str,
+    user: str,
+    response_model: type[T],
+) -> T:
+    raw = _invoke_client(client, model=model, system=system, user=user)
+    cleaned = _strip_markdown_fences(raw)
+    data = json.loads(cleaned)
+    return response_model.model_validate(data)
+
+
+class Oracle:
+    """Single frontier model operating in multiple roles/personas."""
+
+    def __init__(self, client: Any, model: str = "frontier-oracle") -> None:
+        self.client = client
+        self.model = model
+
+    def generate_scenario(self, seed: int, difficulty: str, domain: str) -> Scenario:
+        system = load_prompt_asset("oracle_world_architect")
+        user = (
+            "Generate a complete replication scenario.\n\n"
+            f"Seed: {seed}\n"
+            f"Difficulty: {difficulty}\n"
+            f"Domain: {domain}\n\n"
+            "Respond with a single JSON object matching the Scenario schema.\n"
+            "No markdown, no explanation, only valid JSON."
+        )
+        return call_json_model(
+            self.client,
+            model=self.model,
+            system=system,
+            user=user,
+            response_model=Scenario,
+        )
+
+    def score_round(
+        self,
+        *,
+        scenario: Scenario,
+        round_number: int,
+        scientist_action: BaseModel,
+        lab_manager_response: LabManagerResponse,
+        conversation_history: list[dict],
+        current_protocol: Optional[dict],
+        previous_scores: list[AdjudicatorRoundScore],
+    ) -> AdjudicatorRoundScore:
+        system = load_prompt_asset("oracle_adjudicator")
+        user = (
+            "Score this negotiation round.\n\n"
+            f"SCENARIO:\n{scenario.model_dump_json(indent=2)}\n\n"
+            f"ROUND: {round_number}\n"
+            f"SCIENTIST ACTION: {scientist_action.model_dump_json(indent=2)}\n"
+            f"LAB MANAGER RESPONSE: {lab_manager_response.model_dump_json(indent=2)}\n"
+            f"CURRENT PROTOCOL: {json.dumps(current_protocol, indent=2)}\n"
+            f"PREVIOUS SCORES: {json.dumps([score.model_dump() for score in previous_scores], indent=2)}\n\n"
+            "Respond with a single JSON object matching AdjudicatorRoundScore.\n"
+            "No markdown, no explanation, only valid JSON."
+        )
+        return call_json_model(
+            self.client,
+            model=self.model,
+            system=system,
+            user=user,
+            response_model=AdjudicatorRoundScore,
+        )
+
+    def score_terminal(
+        self,
+        *,
+        scenario: Scenario,
+        final_protocol: dict,
+        conversation_history: list[dict],
+        round_scores: list[AdjudicatorRoundScore],
+    ) -> AdjudicatorTerminalScore:
+        system = load_prompt_asset("oracle_adjudicator")
+        user = (
+            "Compute the terminal score for this completed episode.\n\n"
+            f"SCENARIO:\n{scenario.model_dump_json(indent=2)}\n\n"
+            f"FINAL PROTOCOL: {json.dumps(final_protocol, indent=2)}\n"
+            f"CONVERSATION HISTORY: {json.dumps(conversation_history, indent=2)}\n"
+            f"ROUND SCORES: {json.dumps([score.model_dump() for score in round_scores], indent=2)}\n"
+            f"SUM OF STEP REWARDS: {sum(score.step_reward for score in round_scores)}\n\n"
+            "Respond with a single JSON object matching AdjudicatorTerminalScore.\n"
+            "No markdown, no explanation, only valid JSON."
+        )
+        return call_json_model(
+            self.client,
+            model=self.model,
+            system=system,
+            user=user,
+            response_model=AdjudicatorTerminalScore,
+        )
+
+    def maybe_inject_event(
+        self,
+        *,
+        scenario: Scenario,
+        round_number: int,
+        current_protocol: Optional[dict],
+        conversation_history: list[dict],
+        inject_enabled: bool = False,
+    ) -> Optional[EnvironmentEvent]:
+        if not inject_enabled:
+            return None
+
+        system = load_prompt_asset("oracle_event_injector")
+        user = (
+            "Decide whether to inject an event this round.\n\n"
+            f"SCENARIO:\n{scenario.model_dump_json(indent=2)}\n\n"
+            f"ROUND: {round_number}\n"
+            f"CURRENT PROTOCOL: {json.dumps(current_protocol, indent=2)}\n"
+            f"CONVERSATION SO FAR: {json.dumps(conversation_history, indent=2)}\n\n"
+            'If no event is needed, respond with: {"inject": false}\n'
+            'If injecting, respond with: {"inject": true, "event": <EnvironmentEvent JSON>}\n'
+            "No markdown, no explanation, only valid JSON."
+        )
+        raw = _invoke_client(self.client, model=self.model, system=system, user=user)
+        cleaned = _strip_markdown_fences(raw)
+        data = json.loads(cleaned)
+        if not data.get("inject", False):
+            return None
+        return EnvironmentEvent.model_validate(data["event"])
+
+    def generate_post_mortem(
+        self,
+        *,
+        scenario: Scenario,
+        final_protocol: dict,
+        conversation_history: list[dict],
+        terminal_score: AdjudicatorTerminalScore,
+    ) -> PostMortem:
+        system = load_prompt_asset("oracle_post_mortem")
+        user = (
+            "Generate a post-mortem analysis of this episode.\n\n"
+            f"PAPER: {scenario.paper.model_dump_json(indent=2)}\n"
+            f"LAB CONSTRAINTS: {scenario.lab_constraints.model_dump_json(indent=2)}\n"
+            f"HIDDEN SPEC: {scenario.minimum_viable_spec.model_dump_json(indent=2)}\n"
+            f"FINAL PROTOCOL: {json.dumps(final_protocol, indent=2)}\n"
+            f"CONVERSATION: {json.dumps(conversation_history, indent=2)}\n"
+            f"TERMINAL SCORE: {terminal_score.model_dump_json(indent=2)}\n\n"
+            "Respond with a single JSON object matching PostMortem.\n"
+            "No markdown, no explanation, only valid JSON."
+        )
+        return call_json_model(
+            self.client,
+            model=self.model,
+            system=system,
+            user=user,
+            response_model=PostMortem,
+        )
+
+
+__all__ = [
+    "Oracle",
+    "call_json_model",
+]

replicalab/oracle_models.py

@@ -0,0 +1,221 @@
+"""Typed models for the optional Oracle-driven environment layer.
+
+These models are additive to the existing ReplicaLab contracts. The
+deterministic env, reward, and API surface remain canonical; Oracle models
+power richer scenario generation, optional live Lab Manager responses,
+optional event injection, and post-mortem analysis.
+"""
+
+from __future__ import annotations
+
+from enum import Enum
+from typing import Literal, Optional
+
+from pydantic import BaseModel, ConfigDict, Field
+
+from replicalab.models import ScientistAction
+
+
+class Difficulty(str, Enum):
+    EASY = "easy"
+    MEDIUM = "medium"
+    HARD = "hard"
+
+
+class Equipment(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    name: str
+    available: bool
+    condition: str
+    booking_conflicts: list[str] = Field(default_factory=list)
+    cost_per_use: float = 0.0
+
+
+class Reagent(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    name: str
+    in_stock: bool
+    quantity_available: float = 0.0
+    unit: str = "mL"
+    lead_time_days: int = 0
+    cost: float = 0.0
+
+
+class StaffMember(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    name: str
+    role: str
+    available_days: list[str] = Field(default_factory=list)
+    skills: list[str] = Field(default_factory=list)
+
+
+class Substitution(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    original: str
+    substitute: str
+    validity: str
+    caveats: str = ""
+
+
+class Paper(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    title: str
+    domain: Literal["math_reasoning", "ml_benchmark", "finance_trading"]
+    claim: str
+    method_summary: str
+    original_sample_size: int
+    original_duration_days: int
+    original_technique: str
+    required_controls: list[str] = Field(default_factory=list)
+    required_equipment: list[str] = Field(default_factory=list)
+    required_reagents: list[str] = Field(default_factory=list)
+    statistical_test: str
+
+
+class LabConstraints(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    budget_total: float
+    budget_remaining: float
+    equipment: list[Equipment] = Field(default_factory=list)
+    reagents: list[Reagent] = Field(default_factory=list)
+    staff: list[StaffMember] = Field(default_factory=list)
+    max_duration_days: int
+    safety_rules: list[str] = Field(default_factory=list)
+    valid_substitutions: list[Substitution] = Field(default_factory=list)
+
+
+class MinimumViableSpec(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    min_sample_size: int
+    must_keep_controls: list[str] = Field(default_factory=list)
+    acceptable_techniques: list[str] = Field(default_factory=list)
+    min_duration_days: int
+    critical_equipment: list[str] = Field(default_factory=list)
+    flexible_equipment: list[str] = Field(default_factory=list)
+    critical_reagents: list[str] = Field(default_factory=list)
+    flexible_reagents: list[str] = Field(default_factory=list)
+    power_threshold: float
+
+
+class Scenario(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    paper: Paper
+    lab_constraints: LabConstraints
+    minimum_viable_spec: MinimumViableSpec
+    difficulty: Difficulty
+    narrative_hook: str
+
+
+class OracleScientistObservation(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    paper: Paper
+    round_number: int
+    max_rounds: int
+    conversation_history: list[dict] = Field(default_factory=list)
+    current_protocol: Optional[dict] = None
+
+
+class OracleLabManagerObservation(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    lab_constraints: LabConstraints
+    current_protocol: Optional[dict] = None
+    scientist_action: ScientistAction
+    round_number: int
+
+
+class LabManagerResponse(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    response_type: Literal[
+        "feasibility_report",
+        "suggest_substitution",
+        "reject",
+        "accept",
+    ]
+    feasible: bool
+    issues: list[str] = Field(default_factory=list)
+    suggestions: list[str] = Field(default_factory=list)
+    cost_estimate: float = 0.0
+    time_estimate_days: int = 0
+    message: str
+
+
+class AdjudicatorRoundScore(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    rigor_flags: list[str] = Field(default_factory=list)
+    feasibility_flags: list[str] = Field(default_factory=list)
+    info_gain: float
+    protocol_delta: float
+    momentum: float
+    contradiction_detected: bool
+    stalling_detected: bool
+    step_reward: float
+    notes: str
+
+
+class AdjudicatorTerminalScore(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    rigor: float
+    feasibility: float
+    fidelity: float
+    parsimony: float
+    robustness: float
+    power_preservation: float
+    efficiency_bonus: float
+    communication_bonus: float
+    penalties: dict[str, float] = Field(default_factory=dict)
+    terminal_reward: float
+    total_reward: float
+
+
+class EnvironmentEvent(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    event_type: str
+    description: str
+    state_changes: dict[str, object] = Field(default_factory=dict)
+    severity: Literal["minor", "moderate", "major"]
+
+
+class PostMortem(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+    overall_summary: str
+    rigor_explanation: str
+    feasibility_explanation: str
+    fidelity_explanation: str
+    key_decisions: list[str] = Field(default_factory=list)
+    missed_opportunities: list[str] = Field(default_factory=list)
+    comparison_note: str
+
+
+__all__ = [
+    "AdjudicatorRoundScore",
+    "AdjudicatorTerminalScore",
+    "Difficulty",
+    "EnvironmentEvent",
+    "Equipment",
+    "LabConstraints",
+    "LabManagerResponse",
+    "MinimumViableSpec",
+    "OracleLabManagerObservation",
+    "OracleScientistObservation",
+    "Paper",
+    "PostMortem",
+    "Reagent",
+    "Scenario",
+    "StaffMember",
+    "Substitution",
+]

replicalab/prompts/__init__.py

@@ -1,4 +1,4 @@
-"""Prompt template assets and render helpers (AGT 10)."""
+"""Prompt template assets and render helpers."""
 
 from __future__ import annotations
 
@@ -13,11 +13,17 @@ PromptRole = Literal["scientist", "lab_manager", "judge"]
 _PROMPTS_DIR = Path(__file__).resolve().parent
 
 
+def load_prompt_asset(name: str) -> str:
+    """Load any prompt asset by filename stem."""
+
+    path = _PROMPTS_DIR / f"{name}.txt"
+    return path.read_text(encoding="utf-8")
+
+
 def load_prompt_template(role: PromptRole) -> str:
     """Load a role prompt template from disk."""
 
-    path = _PROMPTS_DIR / f"{role}.txt"
-    return path.read_text(encoding="utf-8")
+    return load_prompt_asset(role)
 
 
 def render_prompt_template(
@@ -119,6 +125,7 @@ def _render_substitutions(pack: NormalizedScenarioPack) -> str:
 
 __all__ = [
     "PromptRole",
+    "load_prompt_asset",
     "load_prompt_template",
     "render_prompt_template",
     "render_scientist_prompt",

replicalab/prompts/oracle_adjudicator.txt

@@ -0,0 +1,26 @@
+You are the Dynamic Adjudicator for ReplicaLab.
+
+You evaluate each round of negotiation and can also produce a terminal
+summary score object. Be precise, fair, and consistent.
+
+Round scoring:
+- info_gain (0-1): how much new useful information the Scientist extracted
+- protocol_delta (-1 to 1): did the protocol move closer to or further from a viable plan
+- momentum (0-1): did the Scientist respond productively to feedback
+- contradiction_detected: did the Scientist contradict previously revealed constraints
+- stalling_detected: did the Scientist repeat prior actions or already-answered questions
+- step_reward: combine the above into a small shaped score
+
+Terminal scoring:
+- rigor, feasibility, fidelity, parsimony, robustness, power_preservation
+- efficiency_bonus and communication_bonus
+- penalties with named keys only
+- terminal_reward and total_reward
+
+Important:
+- Do not invent new score dimensions outside the schema.
+- Score against the hidden scenario specification, not your personal preference.
+- Reward fields must be numerically coherent and self-consistent.
+
+Respond ONLY with valid JSON matching the requested adjudicator schema.
+No markdown. No explanation. No extra text.

replicalab/prompts/oracle_event_injector.txt

@@ -0,0 +1,20 @@
+You are the Event Injector for ReplicaLab.
+
+After a negotiation round, decide whether to inject a realistic mid-episode
+perturbation. Inject sparingly.
+
+Rules:
+- Never inject more than one event per episode.
+- Never inject in rounds 1 or 2.
+- Only inject if the negotiation is stagnating or the protocol is too comfortable.
+- Events must be survivable. There must remain a path to a decent outcome.
+- Use realistic events only: budget cuts, equipment failure, maintenance, backorders, scope changes, staff unavailability.
+- state_changes must be a flat dictionary of dotted paths to new values.
+
+If no event is needed, respond with:
+{"inject": false}
+
+If injecting an event, respond with:
+{"inject": true, "event": <EnvironmentEvent JSON>}
+
+No markdown. No explanation. No extra text.

replicalab/prompts/oracle_lab_manager.txt

@@ -0,0 +1,17 @@
+You are a Lab Manager at a research institution. You are practical,
+detail-oriented, and protective of your lab's resources.
+
+You have access to a constraint document that describes your lab's exact
+situation: budget, equipment, reagents, staff, bookings, and safety rules.
+This document is ground truth. Do not invent constraints that are not in it,
+and do not ignore constraints that are.
+
+When a Scientist proposes a protocol or asks a question:
+1. Check every element against your constraints.
+2. Report what is feasible and what is not.
+3. If something is not feasible, suggest a concrete alternative if one exists.
+4. Estimate the cost and time for what is proposed.
+5. Be collaborative but honest. Do not agree to things the lab cannot do.
+
+Respond ONLY with valid JSON matching LabManagerResponse.
+No markdown. No preamble. No extra text.

replicalab/prompts/oracle_post_mortem.txt

@@ -0,0 +1,15 @@
+You are the Post-Mortem Analyst for ReplicaLab.
+
+At episode end, explain the outcome clearly and specifically.
+
+Your explanation must:
+- summarize the episode in 2-3 sentences
+- explain rigor, feasibility, and fidelity using concrete choices from the protocol
+- identify 3 to 5 impactful decisions
+- list missed opportunities
+- compare the final protocol to what an optimal Scientist would likely have done
+
+Be specific and evidence-based. Refer to protocol decisions, constraints, and final scores.
+
+Respond ONLY with valid JSON matching the PostMortem schema.
+No markdown. No explanation. No extra text.

replicalab/prompts/oracle_world_architect.txt

@@ -0,0 +1,23 @@
+You are the World Architect for ReplicaLab.
+
+You generate a complete, internally consistent scenario for one episode.
+You receive a seed, difficulty level, and domain.
+
+You must produce:
+1. A realistic research or benchmark paper/specification with a clear claim
+2. Lab or compute constraints that create real tension with the requirements
+3. A hidden minimum viable replication spec that is achievable under the constraints
+4. Valid substitutions that are scientifically or operationally defensible
+
+Rules:
+- Supported domains are math_reasoning, ml_benchmark, and finance_trading.
+- The scenario must be solvable. There must always be a viable path to a reasonable outcome.
+- Difficulty controls conflict density:
+  easy = 1-2 meaningful conflicts
+  medium = 3-4 meaningful conflicts
+  hard = 5 or more meaningful conflicts
+- Budget, duration, staff skills, and substitutions must be realistic for the domain.
+- Generate a short narrative_hook that helps the UI explain why this scenario is interesting.
+
+Respond ONLY with valid JSON matching the Scenario schema.
+No markdown. No explanation. No extra text.

replicalab/scenarios/__init__.py

@@ -12,6 +12,7 @@ from .templates import (
     apply_difficulty,
     generate_scenario,
     load_template,
+    oracle_scenario_to_normalized_pack,
 )
 
 __all__ = [
@@ -26,4 +27,5 @@ __all__ = [
     "apply_difficulty",
     "generate_scenario",
     "load_template",
+    "oracle_scenario_to_normalized_pack",
 ]

replicalab/scenarios/templates.py

@@ -10,6 +10,7 @@ from pydantic import BaseModel, ConfigDict
 
 from replicalab.config import MAX_BUDGET, MAX_ROUNDS
 from replicalab.models import LabManagerObservation, ScientistObservation
+from replicalab.oracle_models import Scenario as OracleScenario
 from replicalab.scenarios.finance_trading import build_finance_trading_template
 from replicalab.scenarios.math_reasoning import build_math_reasoning_template
 from replicalab.scenarios.ml_benchmark import build_ml_benchmark_template
@@ -185,6 +186,224 @@ def generate_scenario(
     return _build_pack(seed=seed, template=template, draft=scaled, rng=rng)
 
 
+def oracle_scenario_to_normalized_pack(
+    *,
+    seed: int,
+    template: TemplateName,
+    oracle_scenario: OracleScenario,
+    max_rounds: int = MAX_ROUNDS,
+) -> NormalizedScenarioPack:
+    """Adapt an Oracle-generated Scenario into the canonical normalized pack."""
+
+    difficulty = oracle_scenario.difficulty.value
+    budget_total = oracle_scenario.lab_constraints.budget_total
+    budget_remaining = oracle_scenario.lab_constraints.budget_remaining
+    time_limit_days = oracle_scenario.lab_constraints.max_duration_days
+    staff_count = len(oracle_scenario.lab_constraints.staff)
+
+    constraints: list[ScenarioConstraint] = [
+        ScenarioConstraint(
+            key="budget_total",
+            label="Budget total",
+            quantity=budget_total,
+            unit="USD",
+            comparator="<=",
+            hard=True,
+            details=f"Total available budget is {budget_total:.2f} USD.",
+        ),
+        ScenarioConstraint(
+            key="budget_remaining",
+            label="Budget remaining",
+            quantity=budget_remaining,
+            unit="USD",
+            comparator="<=",
+            hard=True,
+            details=f"Remaining budget at episode start is {budget_remaining:.2f} USD.",
+        ),
+        ScenarioConstraint(
+            key="max_duration_days",
+            label="Maximum duration",
+            quantity=time_limit_days,
+            unit="days",
+            comparator="<=",
+            hard=True,
+            details=f"The plan must finish within {time_limit_days} days.",
+        ),
+        ScenarioConstraint(
+            key="staff_count",
+            label="Available staff",
+            quantity=staff_count,
+            unit="people",
+            comparator=">=",
+            hard=True,
+            details=f"{staff_count} staff member(s) are available for this scenario.",
+        ),
+    ]
+    constraints.extend(
+        ScenarioConstraint(
+            key=f"safety_rule_{index + 1}",
+            label=f"Safety rule {index + 1}",
+            comparator="=",
+            hard=True,
+            details=rule,
+        )
+        for index, rule in enumerate(oracle_scenario.lab_constraints.safety_rules)
+    )
+
+    resources: list[ScenarioResource] = []
+    for equipment in oracle_scenario.lab_constraints.equipment:
+        category = (
+            "compute"
+            if any(token in equipment.name.lower() for token in ("gpu", "cluster", "accelerator"))
+            else "tool"
+        )
+        resources.append(
+            ScenarioResource(
+                key=_slug(equipment.name),
+                label=equipment.name,
+                quantity=1,
+                unit="unit",
+                available=equipment.available and equipment.condition != "shared_booking",
+                category=category,
+                details=(
+                    f"Condition: {equipment.condition}. "
+                    f"Booking conflicts: {', '.join(equipment.booking_conflicts) or 'none'}."
+                ),
+            )
+        )
+
+    for reagent in oracle_scenario.lab_constraints.reagents:
+        resources.append(
+            ScenarioResource(
+                key=_slug(reagent.name),
+                label=reagent.name,
+                quantity=reagent.quantity_available,
+                unit=reagent.unit,
+                available=reagent.in_stock,
+                category="reference",
+                details=(
+                    f"Lead time: {reagent.lead_time_days} day(s). "
+                    f"Unit cost: {reagent.cost:.2f}."
+                ),
+            )
+        )
+
+    for member in oracle_scenario.lab_constraints.staff:
+        resources.append(
+            ScenarioResource(
+                key=_slug(member.name),
+                label=member.name,
+                quantity=len(member.available_days),
+                unit="days",
+                available=bool(member.available_days),
+                category="personnel",
+                details=f"Role: {member.role}. Skills: {', '.join(member.skills) or 'generalist'}.",
+            )
+        )
+
+    substitutions = [
+        AllowedSubstitution(
+            original=item.original,
+            alternative=item.substitute,
+            condition=item.validity,
+            tradeoff=item.caveats or item.validity,
+        )
+        for item in oracle_scenario.lab_constraints.valid_substitutions
+    ]
+
+    required_elements = (
+        list(oracle_scenario.minimum_viable_spec.must_keep_controls)
+        + list(oracle_scenario.minimum_viable_spec.critical_equipment)
+        + list(oracle_scenario.minimum_viable_spec.critical_reagents)
+    )
+    flexible_elements = (
+        list(oracle_scenario.minimum_viable_spec.acceptable_techniques)
+        + list(oracle_scenario.minimum_viable_spec.flexible_equipment)
+        + list(oracle_scenario.minimum_viable_spec.flexible_reagents)
+    )
+
+    hidden_reference = HiddenReferenceSpec(
+        summary=oracle_scenario.paper.method_summary,
+        required_elements=required_elements,
+        flexible_elements=flexible_elements,
+        target_metric=oracle_scenario.paper.statistical_test,
+        target_value=f"power>={oracle_scenario.minimum_viable_spec.power_threshold:.2f}",
+    )
+
+    success_criteria = [
+        oracle_scenario.paper.claim,
+        f"Preserve controls: {', '.join(oracle_scenario.paper.required_controls) or 'none listed'}",
+        f"Use an acceptable technique from the viable spec where possible.",
+        f"Stay within {budget_total:.2f} USD and {time_limit_days} days.",
+    ]
+
+    equipment_available = [
+        equipment.name
+        for equipment in oracle_scenario.lab_constraints.equipment
+        if equipment.available and equipment.condition != "shared_booking"
+    ]
+    equipment_booked = [
+        equipment.name
+        for equipment in oracle_scenario.lab_constraints.equipment
+        if not equipment.available or equipment.condition == "shared_booking"
+    ]
+    reagents_in_stock = [
+        reagent.name for reagent in oracle_scenario.lab_constraints.reagents if reagent.in_stock
+    ]
+    reagents_out_of_stock = [
+        reagent.name for reagent in oracle_scenario.lab_constraints.reagents if not reagent.in_stock
+    ]
+
+    scientist_observation = ScientistObservation(
+        paper_title=oracle_scenario.paper.title,
+        paper_hypothesis=oracle_scenario.paper.claim,
+        paper_method=oracle_scenario.paper.method_summary,
+        paper_key_finding=oracle_scenario.narrative_hook,
+        experiment_goal=oracle_scenario.paper.claim,
+        conversation_history=[],
+        current_protocol=None,
+        round_number=0,
+        max_rounds=max_rounds,
+    )
+
+    lab_manager_observation = LabManagerObservation(
+        budget_total=budget_total,
+        budget_remaining=budget_remaining,
+        equipment_available=equipment_available,
+        equipment_booked=equipment_booked,
+        reagents_in_stock=reagents_in_stock,
+        reagents_out_of_stock=reagents_out_of_stock,
+        staff_count=staff_count,
+        time_limit_days=time_limit_days,
+        safety_restrictions=list(oracle_scenario.lab_constraints.safety_rules),
+        conversation_history=[],
+        current_protocol=None,
+        round_number=0,
+        max_rounds=max_rounds,
+    )
+
+    bookings = _oracle_bookings(oracle_scenario)
+    windows = _oracle_windows(oracle_scenario)
+
+    return NormalizedScenarioPack(
+        scenario_id=f"{template}-{difficulty}-{seed}-oracle",
+        template=template,
+        domain_id=oracle_scenario.paper.domain,
+        difficulty=difficulty,
+        seed=seed,
+        task_summary=oracle_scenario.paper.claim,
+        success_criteria=success_criteria,
+        constraints=constraints,
+        resources=resources,
+        allowed_substitutions=substitutions,
+        hidden_reference_spec=hidden_reference,
+        scientist_observation=scientist_observation,
+        lab_manager_observation=lab_manager_observation,
+        resource_bookings=bookings,
+        scheduling_windows=windows,
+    )
+
+
 def _build_pack(seed: int, template: TemplateName, draft: dict[str, Any], rng: Any) -> NormalizedScenarioPack:
     constraints = [ScenarioConstraint.model_validate(item) for item in draft["constraints"]]
     resources = [ScenarioResource.model_validate(item) for item in draft["resources"]]
@@ -284,6 +503,90 @@ def _build_pack(seed: int, template: TemplateName, draft: dict[str, Any], rng: A
     )
 
 
+def _slug(value: str) -> str:
+    return "_".join(value.lower().replace("/", " ").replace("-", " ").split())
+
+
+def _day_to_offset(day: str) -> int:
+    mapping = {
+        "monday": 0,
+        "tuesday": 24,
+        "wednesday": 48,
+        "thursday": 72,
+        "friday": 96,
+        "saturday": 120,
+        "sunday": 144,
+    }
+    return mapping.get(day.strip().lower(), 0)
+
+
+def _oracle_bookings(oracle_scenario: OracleScenario) -> list[ResourceBooking]:
+    bookings: list[ResourceBooking] = []
+    for equipment in oracle_scenario.lab_constraints.equipment:
+        if equipment.booking_conflicts:
+            for day in equipment.booking_conflicts:
+                bookings.append(
+                    ResourceBooking(
+                        resource_key=_slug(equipment.name),
+                        resource_label=equipment.name,
+                        slot_label=day,
+                        start_offset_hours=_day_to_offset(day),
+                        duration_hours=8.0,
+                        status="booked" if equipment.available else "maintenance",
+                        details=f"{equipment.name} is constrained on {day}.",
+                    )
+                )
+        else:
+            bookings.append(
+                ResourceBooking(
+                    resource_key=_slug(equipment.name),
+                    resource_label=equipment.name,
+                    slot_label="default",
+                    start_offset_hours=0.0,
+                    duration_hours=8.0,
+                    status="available" if equipment.available else "maintenance",
+                    details=f"{equipment.name} is available under normal scheduling.",
+                )
+            )
+    return bookings
+
+
+def _oracle_windows(oracle_scenario: OracleScenario) -> list[SchedulingWindow]:
+    windows: list[SchedulingWindow] = [
+        SchedulingWindow(
+            key="max_duration_window",
+            label="Maximum project duration",
+            start_offset_hours=0.0,
+            end_offset_hours=float(oracle_scenario.lab_constraints.max_duration_days * 24),
+            hard=True,
+            details=(
+                f"All work must complete within "
+                f"{oracle_scenario.lab_constraints.max_duration_days} days."
+            ),
+        )
+    ]
+
+    seen_days: set[str] = set()
+    for member in oracle_scenario.lab_constraints.staff:
+        for day in member.available_days:
+            normalized = day.strip().lower()
+            if normalized in seen_days:
+                continue
+            seen_days.add(normalized)
+            start = float(_day_to_offset(day))
+            windows.append(
+                SchedulingWindow(
+                    key=f"staff_{normalized}",
+                    label=f"Staff availability {day}",
+                    start_offset_hours=start,
+                    end_offset_hours=start + 8.0,
+                    hard=False,
+                    details=f"At least one staff member is available on {day}.",
+                )
+            )
+    return windows
+
+
 def _split_resources(
     resources: list[ScenarioResource],
     *,

replicalab/scoring/explain.py

@@ -1,6 +1,6 @@
-"""JDG 06 — Plain-English explanation builder from RewardBreakdown.
+"""JDG 06 - Plain-English explanation builder from RewardBreakdown.
 
-Pure deterministic function — reads existing breakdown fields only,
+Pure deterministic function - reads existing breakdown fields only,
 introduces no new scoring logic.
 """
 
@@ -22,33 +22,31 @@ def _tier(score: float) -> str:
 
 
 def explain_reward(breakdown: RewardBreakdown) -> str:
-    """Build a plain-English explanation from a RewardBreakdown.
-
-    The output mirrors the three rubric components (rigor, feasibility,
-    fidelity), any bonuses, any named penalties, and the final total.
-    No hidden scoring logic is introduced — this is a pure formatter.
-    """
+    """Build a plain-English explanation from a RewardBreakdown."""
     total = compute_total_reward(breakdown)
     lines: list[str] = []
 
-    # --- rubric components ---
     lines.append(
-        f"Rigor: {breakdown.rigor:.2f} ({_tier(breakdown.rigor)}) — "
+        f"Rigor: {breakdown.rigor:.2f} ({_tier(breakdown.rigor)}) - "
         "measures structural completeness, success-criteria coverage, "
         "and required-element coverage."
     )
     lines.append(
-        f"Feasibility: {breakdown.feasibility:.2f} ({_tier(breakdown.feasibility)}) — "
+        f"Feasibility: {breakdown.feasibility:.2f} ({_tier(breakdown.feasibility)}) - "
         "measures whether the protocol respects budget, equipment, reagent, "
         "schedule, and staffing constraints."
     )
     lines.append(
-        f"Fidelity: {breakdown.fidelity:.2f} ({_tier(breakdown.fidelity)}) — "
+        f"Fidelity: {breakdown.fidelity:.2f} ({_tier(breakdown.fidelity)}) - "
         "measures alignment with the hidden reference spec, including "
         "required elements, substitutions, and target metrics."
     )
+    lines.append(
+        f"Parsimony: {breakdown.parsimony:.2f} ({_tier(breakdown.parsimony)}) - "
+        "measures whether the plan stays lean instead of requesting more "
+        "controls, equipment, or reagents than the scenario complexity calls for."
+    )
 
-    # --- bonuses ---
     if breakdown.efficiency_bonus > 0:
         lines.append(
             f"Efficiency bonus: +{breakdown.efficiency_bonus:.2f} "
@@ -59,15 +57,17 @@ def explain_reward(breakdown: RewardBreakdown) -> str:
             f"Communication bonus: +{breakdown.communication_bonus:.2f}."
         )
 
-    # --- penalties ---
     if breakdown.penalties:
         for key, amount in sorted(breakdown.penalties.items()):
             label = key.replace("_", " ")
-            lines.append(f"Penalty — {label}: -{amount:.2f}.")
+            lines.append(f"Penalty - {label}: -{amount:.2f}.")
     else:
         lines.append("No penalties applied.")
 
-    # --- total ---
-    lines.append(f"Total reward: {total:.2f} (formula: 10 × rigor × feasibility × fidelity + bonuses − penalties).")
+    lines.append(
+        "Total reward: "
+        f"{total:.2f} "
+        "(formula: 10 x rigor x feasibility x fidelity x parsimony + bonuses - penalties)."
+    )
 
     return "\n".join(lines)

replicalab/scoring/rubric.py

@@ -1,11 +1,14 @@
-"""JDG 04-05 — Total reward computation and reward breakdown builder.
+"""JDG 04-05 - Total reward computation and reward breakdown builder.
 
-Combines rigor (JDG 01), feasibility (JDG 02), and fidelity (JDG 03)
-into a single scalar reward with efficiency bonus and penalties.
+Combines rigor (JDG 01), feasibility (JDG 02), fidelity (JDG 03), and a
+lightweight parsimony term into a single scalar reward with bonuses and
+named penalties.
 
-Formula:  total = 10 × rigor × feasibility × fidelity + bonuses − penalties
+Formula:
+    total = 10 * rigor * feasibility * fidelity * parsimony
+            + bonuses - penalties
 
-Pure deterministic functions — no model calls, no side effects.
+Pure deterministic functions - no model calls, no side effects.
 """
 
 from __future__ import annotations
@@ -27,12 +30,14 @@ _MAX_COMMUNICATION_BONUS = 0.0  # reserved for future use
 
 
 def compute_total_reward(breakdown: RewardBreakdown) -> float:
-    """Compute the scalar reward from a RewardBreakdown.
-
-    Formula: 10 × rigor × feasibility × fidelity + efficiency_bonus
-             + communication_bonus − sum(penalties)
-    """
-    base = _REWARD_SCALE * breakdown.rigor * breakdown.feasibility * breakdown.fidelity
+    """Compute the scalar reward from a RewardBreakdown."""
+    base = (
+        _REWARD_SCALE
+        * breakdown.rigor
+        * breakdown.feasibility
+        * breakdown.fidelity
+        * breakdown.parsimony
+    )
     bonus = breakdown.efficiency_bonus + breakdown.communication_bonus
     penalty = sum(breakdown.penalties.values())
     return max(0.0, round(base + bonus - penalty, 6))
@@ -47,32 +52,14 @@ def build_reward_breakdown(
     check: FeasibilityCheckResult | None = None,
     penalties: dict[str, float] | None = None,
 ) -> RewardBreakdown:
-    """Build a full RewardBreakdown from the three sub-scores plus bonuses.
-
-    Parameters
-    ----------
-    protocol : Protocol
-        The final agreed protocol.
-    scenario : NormalizedScenarioPack
-        The scenario pack for this episode.
-    rounds_used : int
-        How many rounds were consumed.
-    max_rounds : int
-        The episode's round cap.
-    check : FeasibilityCheckResult, optional
-        Pre-computed feasibility check to avoid redundant work.
-    penalties : dict[str, float], optional
-        Named penalty keys for bounded-tool diagnostics, unsupported
-        evidence claims, or other deterministic deductions.  Use named
-        keys (e.g. ``"invalid_tool_use"``, ``"unsupported_claim"``)
-        instead of adding new fields to RewardBreakdown.
-    """
+    """Build a full RewardBreakdown from the sub-scores plus bonuses."""
     if check is None:
         check = check_feasibility(protocol, scenario)
 
     rigor = score_rigor(protocol, scenario)
     feasibility = score_feasibility(protocol, scenario, check=check)
     fidelity = score_fidelity(protocol, scenario)
+    parsimony = _score_parsimony(protocol, scenario)
 
     efficiency_bonus = _efficiency_bonus(rounds_used, max_rounds)
     merged_penalties = dict(penalties) if penalties else {}
@@ -81,6 +68,7 @@ def build_reward_breakdown(
         rigor=rigor,
         feasibility=feasibility,
         fidelity=fidelity,
+        parsimony=parsimony,
         efficiency_bonus=efficiency_bonus,
         communication_bonus=0.0,
         penalties=merged_penalties,
@@ -88,12 +76,33 @@ def build_reward_breakdown(
 
 
 def _efficiency_bonus(rounds_used: int, max_rounds: int) -> float:
-    """Reward finishing in fewer rounds.
-
-    If the scientist reaches agreement in round 1 of 6, that's the maximum
-    bonus.  If they use all rounds, the bonus is 0.
-    """
+    """Reward finishing in fewer rounds."""
     if max_rounds <= 1 or rounds_used <= 0:
         return 0.0
     saved = max(0, max_rounds - rounds_used)
     return round(_MAX_EFFICIENCY_BONUS * saved / (max_rounds - 1), 6)
+
+
+def _score_parsimony(
+    protocol: Protocol,
+    scenario: NormalizedScenarioPack,
+) -> float:
+    """Score how lean the protocol is relative to scenario complexity.
+
+    The current scenario schema does not expose explicit "necessary resource"
+    labels, so we infer complexity from the hidden required-element count and
+    penalize plans that request far more unique controls/resources than that
+    complexity suggests.
+    """
+    required_element_count = len(scenario.hidden_reference_spec.required_elements)
+    complexity_budget = max(2, required_element_count + 2)
+    requested_count = (
+        len(set(protocol.controls))
+        + len(set(protocol.required_equipment))
+        + len(set(protocol.required_reagents))
+    )
+    if requested_count <= 0:
+        return 1.0
+
+    ratio = complexity_budget / max(complexity_budget, requested_count)
+    return round(max(0.25, min(1.0, ratio)), 6)

replicalab/training/rollout.py

@@ -164,9 +164,9 @@ class RolloutWorker:
             )
             record.steps.append(step)
             record.tool_traces.extend(tool_traces)
+            record.total_reward = round(record.total_reward + result.reward, 6)
 
             if result.done:
-                record.total_reward = result.reward
                 record.reward_breakdown = result.info.reward_breakdown
                 record.judge_notes = result.info.judge_notes
                 record.verdict = result.info.verdict

server/app.py

@@ -111,7 +111,7 @@ def _build_episode_log(
         final_state=state,
         transcript=list(state.conversation_history),
         reward_breakdown=info.reward_breakdown,
-        total_reward=result.reward,
+        total_reward=state.reward,
         rounds_used=state.round_number,
         agreement_reached=info.agreement_reached,
         judge_notes=info.judge_notes or "",

tests/test_cache.py

@@ -0,0 +1,92 @@
+from __future__ import annotations
+
+import json
+
+from replicalab.cache import CachedOracle, ScenarioCache
+from replicalab.oracle_models import Scenario
+
+
+def _scenario_payload() -> dict:
+    return {
+        "paper": {
+            "title": "Cached benchmark",
+            "domain": "ml_benchmark",
+            "claim": "A small run remains useful under a tighter budget.",
+            "method_summary": "Train a compact model and verify against a held-out split.",
+            "original_sample_size": 1000,
+            "original_duration_days": 2,
+            "original_technique": "compact_model",
+            "required_controls": ["baseline"],
+            "required_equipment": ["GPU cluster"],
+            "required_reagents": ["dataset snapshot"],
+            "statistical_test": "accuracy_gap",
+        },
+        "lab_constraints": {
+            "budget_total": 1200.0,
+            "budget_remaining": 1200.0,
+            "equipment": [
+                {
+                    "name": "GPU cluster",
+                    "available": True,
+                    "condition": "operational",
+                    "booking_conflicts": [],
+                    "cost_per_use": 100.0,
+                }
+            ],
+            "reagents": [
+                {
+                    "name": "dataset snapshot",
+                    "in_stock": True,
+                    "quantity_available": 1.0,
+                    "unit": "copy",
+                    "lead_time_days": 0,
+                    "cost": 0.0,
+                }
+            ],
+            "staff": [],
+            "max_duration_days": 3,
+            "safety_rules": ["No external internet."],
+            "valid_substitutions": [],
+        },
+        "minimum_viable_spec": {
+            "min_sample_size": 800,
+            "must_keep_controls": ["baseline"],
+            "acceptable_techniques": ["compact_model"],
+            "min_duration_days": 1,
+            "critical_equipment": ["GPU cluster"],
+            "flexible_equipment": [],
+            "critical_reagents": ["dataset snapshot"],
+            "flexible_reagents": [],
+            "power_threshold": 0.75,
+        },
+        "difficulty": "easy",
+        "narrative_hook": "The benchmark owners tightened the reporting budget.",
+    }
+
+
+def test_scenario_cache_round_trips(tmp_path) -> None:
+    cache = ScenarioCache(tmp_path)
+    scenario = Scenario.model_validate(_scenario_payload())
+
+    path = cache.put(13, "easy", "ml_benchmark", scenario)
+    restored = cache.get(13, "easy", "ml_benchmark")
+
+    assert path.exists()
+    assert restored is not None
+    assert restored.model_dump(mode="json") == scenario.model_dump(mode="json")
+
+
+def test_cached_oracle_uses_cache_after_first_generation(tmp_path) -> None:
+    calls = {"count": 0}
+
+    def fake_client(system: str, user: str, model: str) -> str:
+        calls["count"] += 1
+        return json.dumps(_scenario_payload())
+
+    oracle = CachedOracle(fake_client, cache=ScenarioCache(tmp_path))
+
+    first = oracle.generate_scenario(9, "easy", "ml_benchmark")
+    second = oracle.generate_scenario(9, "easy", "ml_benchmark")
+
+    assert first.model_dump(mode="json") == second.model_dump(mode="json")
+    assert calls["count"] == 1

tests/test_env.py

@@ -342,7 +342,9 @@ class TestStep:
 
         assert env.state().round_number == 1
         assert result.done is False
-        assert result.reward == 0.0
+        assert result.reward > 0.0
+        assert result.info.step_reward_components["protocol_delta_bonus"] > 0.0
+        assert result.info.cumulative_reward == result.reward
 
     def test_step_returns_observations(self) -> None:
         env = ReplicaLabEnv()
@@ -416,7 +418,9 @@ class TestStep:
 
         assert result.done is True
         assert result.info.agreement_reached is False
-        assert result.reward == 0.0
+        assert result.reward < 0.0
+        assert result.info.reward_breakdown is not None
+        assert result.info.reward_breakdown.penalties["timeout"] > 0.0
 
     def test_step_info_has_round_and_episode_id(self) -> None:
         env = ReplicaLabEnv()
@@ -655,7 +659,8 @@ class TestEnvReward:
         assert result.done
         assert result.info.verdict == "timeout"
         assert result.info.reward_breakdown is not None
-        assert result.reward == 0.0
+        assert result.reward < 0.0
+        assert result.info.reward_breakdown.penalties["timeout"] > 0.0
 
     def test_episode_state_stores_final_scores(self) -> None:
         env = ReplicaLabEnv()

tests/test_oracle.py

@@ -0,0 +1,281 @@
+from __future__ import annotations
+
+import json
+
+from replicalab.agents.lab_manager_agent import LabManagerAgent
+from replicalab.env import ReplicaLabEnv
+from replicalab.models import ScientistAction
+from replicalab.oracle import Oracle
+from replicalab.oracle_models import (
+    AdjudicatorRoundScore,
+    EnvironmentEvent,
+    OracleLabManagerObservation,
+    PostMortem,
+    Scenario,
+)
+
+
+def _scenario_payload() -> dict:
+    return {
+        "paper": {
+            "title": "Reproducing a Small Vision Benchmark",
+            "domain": "ml_benchmark",
+            "claim": "A compact model can recover >90% of reference accuracy under budget.",
+            "method_summary": "Train a compact CNN with fixed augmentations and evaluate on a held-out split.",
+            "original_sample_size": 1200,
+            "original_duration_days": 3,
+            "original_technique": "compact_cnn",
+            "required_controls": ["seed_control", "baseline_model"],
+            "required_equipment": ["GPU cluster", "validation server"],
+            "required_reagents": ["dataset snapshot"],
+            "statistical_test": "accuracy_gap",
+        },
+        "lab_constraints": {
+            "budget_total": 2400.0,
+            "budget_remaining": 2400.0,
+            "equipment": [
+                {
+                    "name": "GPU cluster",
+                    "available": True,
+                    "condition": "shared_booking",
+                    "booking_conflicts": ["Monday"],
+                    "cost_per_use": 250.0,
+                },
+                {
+                    "name": "Validation server",
+                    "available": True,
+                    "condition": "operational",
+                    "booking_conflicts": [],
+                    "cost_per_use": 20.0,
+                },
+            ],
+            "reagents": [
+                {
+                    "name": "dataset snapshot",
+                    "in_stock": True,
+                    "quantity_available": 1.0,
+                    "unit": "copy",
+                    "lead_time_days": 0,
+                    "cost": 0.0,
+                }
+            ],
+            "staff": [
+                {
+                    "name": "Alex",
+                    "role": "engineer",
+                    "available_days": ["Monday", "Tuesday"],
+                    "skills": ["training", "evaluation"],
+                }
+            ],
+            "max_duration_days": 5,
+            "safety_rules": ["No external internet during training."],
+            "valid_substitutions": [
+                {
+                    "original": "GPU cluster",
+                    "substitute": "single high-memory GPU",
+                    "validity": "acceptable_with_caveats",
+                    "caveats": "Lower throughput is acceptable if evaluation fidelity is preserved.",
+                }
+            ],
+        },
+        "minimum_viable_spec": {
+            "min_sample_size": 800,
+            "must_keep_controls": ["seed_control", "baseline_model"],
+            "acceptable_techniques": ["compact_cnn", "distilled_cnn"],
+            "min_duration_days": 2,
+            "critical_equipment": ["Validation server"],
+            "flexible_equipment": ["GPU cluster"],
+            "critical_reagents": ["dataset snapshot"],
+            "flexible_reagents": [],
+            "power_threshold": 0.8,
+        },
+        "difficulty": "medium",
+        "narrative_hook": "The compute team just reduced your preferred GPU window.",
+    }
+
+
+def _round_score_payload() -> dict:
+    return {
+        "rigor_flags": ["kept baseline_model"],
+        "feasibility_flags": ["GPU window narrowed"],
+        "info_gain": 0.6,
+        "protocol_delta": 0.4,
+        "momentum": 0.7,
+        "contradiction_detected": False,
+        "stalling_detected": False,
+        "step_reward": 0.55,
+        "notes": "Scientist asked a useful scheduling question and preserved controls.",
+    }
+
+
+def _post_mortem_payload() -> dict:
+    return {
+        "overall_summary": "The Scientist converged on a feasible compact CNN plan.",
+        "rigor_explanation": "Controls and the validation server were preserved.",
+        "feasibility_explanation": "The final plan fit the available compute and duration window.",
+        "fidelity_explanation": "The protocol stayed close to the benchmark setup.",
+        "key_decisions": ["Kept seed control", "Accepted lower-throughput compute"],
+        "missed_opportunities": ["Could have asked about booking conflicts earlier"],
+        "comparison_note": "An optimal Scientist would have requested the alternate GPU window one round sooner.",
+    }
+
+
+class _FakeMessagesAPI:
+    def __init__(self, payloads: list[dict]) -> None:
+        self._payloads = payloads
+        self.calls = 0
+
+    def create(self, **_: object):
+        payload = self._payloads[self.calls]
+        self.calls += 1
+
+        class _Chunk:
+            def __init__(self, text: str) -> None:
+                self.text = text
+
+        class _Response:
+            def __init__(self, text: str) -> None:
+                self.content = [_Chunk(text)]
+
+        return _Response(json.dumps(payload))
+
+
+class _FakeClient:
+    def __init__(self, payloads: list[dict]) -> None:
+        self.messages = _FakeMessagesAPI(payloads)
+
+
+def test_oracle_generate_scenario_parses_json() -> None:
+    oracle = Oracle(_FakeClient([_scenario_payload()]))
+
+    scenario = oracle.generate_scenario(seed=7, difficulty="medium", domain="ml_benchmark")
+
+    assert isinstance(scenario, Scenario)
+    assert scenario.paper.domain == "ml_benchmark"
+    assert scenario.lab_constraints.equipment[0].name == "GPU cluster"
+
+
+def test_oracle_score_round_parses_structured_payload() -> None:
+    oracle = Oracle(_FakeClient([_round_score_payload()]))
+    scenario = Scenario.model_validate(_scenario_payload())
+    action = ScientistAction(
+        action_type="request_info",
+        sample_size=0,
+        controls=[],
+        technique="",
+        duration_days=0,
+        required_equipment=[],
+        required_reagents=[],
+        questions=["When is the GPU cluster available?"],
+        rationale="",
+    )
+    lab_manager = LabManagerAgent(_FakeClient([{
+        "response_type": "feasibility_report",
+        "feasible": False,
+        "issues": ["GPU cluster is shared-booked on Monday"],
+        "suggestions": ["Use the single high-memory GPU instead"],
+        "cost_estimate": 250.0,
+        "time_estimate_days": 3,
+        "message": "The GPU cluster is shared-booked Monday; the single high-memory GPU is acceptable with caveats.",
+    }]))
+    response = lab_manager.respond(
+        OracleLabManagerObservation(
+            lab_constraints=scenario.lab_constraints,
+            current_protocol=None,
+            scientist_action=action,
+            round_number=1,
+        )
+    )
+
+    score = oracle.score_round(
+        scenario=scenario,
+        round_number=1,
+        scientist_action=action,
+        lab_manager_response=response,
+        conversation_history=[],
+        current_protocol=None,
+        previous_scores=[],
+    )
+
+    assert isinstance(score, AdjudicatorRoundScore)
+    assert score.step_reward == 0.55
+
+
+def test_oracle_maybe_inject_event_returns_optional_event() -> None:
+    oracle = Oracle(_FakeClient([{"inject": True, "event": {
+        "event_type": "budget_cut",
+        "description": "Finance reduced the remaining budget.",
+        "state_changes": {"lab_constraints.budget_remaining": 1800.0},
+        "severity": "moderate",
+    }}]))
+
+    event = oracle.maybe_inject_event(
+        scenario=Scenario.model_validate(_scenario_payload()),
+        round_number=3,
+        current_protocol=None,
+        conversation_history=[],
+        inject_enabled=True,
+    )
+
+    assert isinstance(event, EnvironmentEvent)
+    assert event.event_type == "budget_cut"
+
+
+def test_oracle_generate_post_mortem_parses_json() -> None:
+    oracle = Oracle(_FakeClient([_post_mortem_payload()]))
+    from replicalab.oracle_models import AdjudicatorTerminalScore
+
+    post_mortem = oracle.generate_post_mortem(
+        scenario=Scenario.model_validate(_scenario_payload()),
+        final_protocol={"technique": "compact_cnn"},
+        conversation_history=[],
+        terminal_score=AdjudicatorTerminalScore(
+            rigor=0.9,
+            feasibility=0.8,
+            fidelity=0.85,
+            parsimony=0.9,
+            robustness=0.8,
+            power_preservation=0.8,
+            efficiency_bonus=0.2,
+            communication_bonus=0.1,
+            penalties={},
+            terminal_reward=5.0,
+            total_reward=5.6,
+        ),
+    )
+
+    assert isinstance(post_mortem, PostMortem)
+    assert "feasible compact CNN plan" in post_mortem.overall_summary
+
+
+def test_env_can_reset_from_oracle_scenario_without_changing_outer_contract() -> None:
+    class _FakeOracle:
+        def __init__(self) -> None:
+            self.scenario = Scenario.model_validate(_scenario_payload())
+
+        def generate_scenario(self, seed: int, difficulty: str, domain: str) -> Scenario:
+            assert seed == 11
+            assert difficulty == "medium"
+            assert domain == "ml_benchmark"
+            return self.scenario
+
+        def score_round(self, **_: object):
+            return AdjudicatorRoundScore.model_validate(_round_score_payload())
+
+        def maybe_inject_event(self, **_: object):
+            return None
+
+        def generate_post_mortem(self, **_: object):
+            return PostMortem.model_validate(_post_mortem_payload())
+
+    env = ReplicaLabEnv(
+        oracle=_FakeOracle(),
+        enable_oracle_post_mortem=True,
+    )
+    observation = env.reset(seed=11, scenario="ml_benchmark", difficulty="medium")
+
+    assert observation.scientist is not None
+    assert observation.scientist.paper_title == "Reproducing a Small Vision Benchmark"
+    assert observation.lab_manager is not None
+    assert "Validation server" in observation.lab_manager.equipment_available
+

tests/test_prompts.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 from replicalab.prompts import (
+    load_prompt_asset,
     load_prompt_template,
     render_judge_prompt,
     render_lab_manager_prompt,
@@ -22,6 +23,18 @@ def test_load_prompt_template_reads_all_role_files() -> None:
         assert "ReplicaLab" in template
 
 
+def test_load_oracle_prompt_assets_reads_all_oracle_files() -> None:
+    for name in (
+        "oracle_world_architect",
+        "oracle_adjudicator",
+        "oracle_event_injector",
+        "oracle_post_mortem",
+        "oracle_lab_manager",
+    ):
+        template = load_prompt_asset(name)
+        assert len(template) > 100
+
+
 def test_render_scientist_prompt_injects_task_and_bounded_tools() -> None:
     prompt = render_scientist_prompt(_scenario("ml_benchmark"))
 

tests/test_scenarios.py

@@ -7,7 +7,9 @@ from replicalab.scenarios import (
     NormalizedScenarioPack,
     available_scenario_families,
     generate_scenario,
+    oracle_scenario_to_normalized_pack,
 )
+from replicalab.oracle_models import Scenario as OracleScenario
 
 
 def test_generate_scenario_is_deterministic_for_same_seed() -> None:
@@ -140,3 +142,90 @@ def test_all_domains_produce_bookings_and_windows() -> None:
         pack = generate_scenario(seed=42, template=template, difficulty="medium")
         assert len(pack.resource_bookings) > 0, f"{template} has no bookings"
         assert len(pack.scheduling_windows) > 0, f"{template} has no windows"
+
+
+def test_oracle_scenario_adapter_preserves_domain_and_constraints() -> None:
+    oracle_scenario = OracleScenario.model_validate(
+        {
+            "paper": {
+                "title": "Adapting a benchmark under constraint",
+                "domain": "ml_benchmark",
+                "claim": "A small model remains competitive after budget cuts.",
+                "method_summary": "Train a compact benchmark baseline with fixed controls.",
+                "original_sample_size": 1200,
+                "original_duration_days": 3,
+                "original_technique": "compact_cnn",
+                "required_controls": ["baseline", "seed_control"],
+                "required_equipment": ["GPU cluster"],
+                "required_reagents": ["dataset snapshot"],
+                "statistical_test": "accuracy_gap",
+            },
+            "lab_constraints": {
+                "budget_total": 1800.0,
+                "budget_remaining": 1500.0,
+                "equipment": [
+                    {
+                        "name": "GPU cluster",
+                        "available": True,
+                        "condition": "shared_booking",
+                        "booking_conflicts": ["Monday"],
+                        "cost_per_use": 200.0,
+                    }
+                ],
+                "reagents": [
+                    {
+                        "name": "dataset snapshot",
+                        "in_stock": True,
+                        "quantity_available": 1.0,
+                        "unit": "copy",
+                        "lead_time_days": 0,
+                        "cost": 0.0,
+                    }
+                ],
+                "staff": [
+                    {
+                        "name": "Alex",
+                        "role": "engineer",
+                        "available_days": ["Monday", "Tuesday"],
+                        "skills": ["training", "evaluation"],
+                    }
+                ],
+                "max_duration_days": 5,
+                "safety_rules": ["No external internet during training."],
+                "valid_substitutions": [
+                    {
+                        "original": "GPU cluster",
+                        "substitute": "single high-memory GPU",
+                        "validity": "acceptable_with_caveats",
+                        "caveats": "Longer runtime is acceptable if evaluation fidelity is preserved.",
+                    }
+                ],
+            },
+            "minimum_viable_spec": {
+                "min_sample_size": 800,
+                "must_keep_controls": ["baseline", "seed_control"],
+                "acceptable_techniques": ["compact_cnn"],
+                "min_duration_days": 2,
+                "critical_equipment": ["GPU cluster"],
+                "flexible_equipment": [],
+                "critical_reagents": ["dataset snapshot"],
+                "flexible_reagents": [],
+                "power_threshold": 0.8,
+            },
+            "difficulty": "medium",
+            "narrative_hook": "The preferred GPU window has been partially reallocated.",
+        }
+    )
+
+    pack = oracle_scenario_to_normalized_pack(
+        seed=7,
+        template="ml_benchmark",
+        oracle_scenario=oracle_scenario,
+    )
+
+    assert pack.domain_id == "ml_benchmark"
+    assert pack.scientist_observation.paper_title == "Adapting a benchmark under constraint"
+    assert pack.lab_manager_observation.budget_total == 1800.0
+    assert "GPU cluster" in pack.lab_manager_observation.equipment_booked
+    assert pack.hidden_reference_spec.required_elements
+    assert pack.resource_bookings

tests/test_server.py

@@ -509,7 +509,8 @@ class TestWebSocket:
 
             assert resp["type"] == "step_ok"
             assert resp["done"] is False
-            assert resp["reward"] == 0.0
+            assert resp["reward"] > 0.0
+            assert resp["info"]["step_reward_components"]["protocol_delta_bonus"] > 0.0
             assert resp["observation"] is not None
 
     def test_ws_full_episode_real_reward(self, client: TestClient) -> None:
@@ -627,8 +628,9 @@ class TestWebSocket:
 
             assert resp["done"] is True
             assert resp["info"]["verdict"] == "timeout"
-            assert resp["reward"] == 0.0
+            assert resp["reward"] < 0.0
             assert resp["info"]["reward_breakdown"] is not None
+            assert resp["info"]["reward_breakdown"]["penalties"]["timeout"] > 0.0
 
     def test_ws_terminal_episode_persists_real_replay_log(
         self, client: TestClient