Upload folder using huggingface_hub

README.md

@@ -1,300 +1,205 @@
 ---
-title: ACDE OpenEnv
-emoji: "🚑"
+title: Acde-Openenv Environment Server
+emoji: 🚑
 colorFrom: blue
 colorTo: green
 sdk: docker
 pinned: false
+app_port: 8000
 base_path: /web
+tags:
+  - openenv
 ---
 
-# Emergency Routing Simulation (ACDE)
+# Adaptive Crisis Decision Environment (ACDE)
 
-This project is a simulation environment for emergency ambulance routing.
+ACDE is an OpenEnv-compliant, real-world emergency dispatch environment for evaluating decision-making under uncertainty. The task models ambulance routing to hospitals under changing traffic, hidden ICU truth, and patient deterioration.
 
-In simple terms:
-- A patient needs urgent care.
-- Several hospitals are available.
-- Each hospital has trade-offs (distance, traffic, ICU certainty, specialization).
-- Conditions can change while the ambulance is moving.
-- The agent must decide where to go, step by step.
+## Why This Is Real-World
 
-The goal is not to be perfect every time. The goal is to make realistic decisions under uncertainty.
+This environment simulates a practical operation used by emergency coordinators:
 
-## What This Project Does
+- choosing destination hospitals under incomplete information
+- trading off travel time vs ICU availability vs specialization
+- handling a non-stationary world where traffic and capacity shift
 
-This environment helps you test decision logic in situations where information is incomplete and time is limited.
+## OpenEnv API
 
-It supports three difficulty levels:
-- `acde_easy`
-- `acde_medium`
-- `acde_hard`
+Implemented in [app/environment/core.py](app/environment/core.py) and exposed via [app/server/app.py](app/server/app.py).
 
-As difficulty increases, uncertainty and penalties increase too.
+- reset(seed, task_id) -> observation
+- step(action) -> observation, reward, done, info
+- state() -> state
 
-## How It Works (Simple Flow)
+HTTP endpoints:
 
-Every episode follows this loop:
-
-1. The environment is reset with a seed and task.
-2. You get an observation:
-- Patient condition
-- Required specialization
-- Hospital list with visible signals
-3. The policy scores hospitals.
-4. One hospital is selected.
-5. The environment validates arrival using hidden checks.
-6. You receive outcome + reward.
-7. If not done, repeat until success or failure.
-
-## What Makes It Realistic
-
-This is not a static lookup problem. It includes realistic uncertainty:
-
-- Displayed ICU status can differ from actual ICU status.
-- Traffic can change between steps.
-- Hospital overload can change outcomes.
-- Specialist availability can fail at arrival.
-- A hospital that failed once may become usable later.
-
-The policy includes safety rules such as:
-- Immediate retry protection after rejection.
-- Cooldown handling for recently failed hospitals.
-- Exploration among top options (not blind random picks).
-
-## Project Layout
-
-Key files:
-
-- `app/environment/core.py`
-  - Main environment loop (`reset`, `step`, transition logic)
-- `app/environment/validation.py`
-  - Hidden validation checks (ICU, specialist, overload, outcome)
-- `app/environment/graders.py`
-  - Final scoring and pass/fail grading
-- `app/models/`
-  - Pydantic models for state, observation, reward, action
-- `app/server/app.py`
-  - FastAPI server endpoints
-- `inference.py`
-  - Local policy runner (CLI episodes)
-- `data/learning_memory.json`
-  - Rolling policy memory
-- `data/trajectory_history.jsonl`
-  - Per-step trajectory logs
-
-## API Endpoints
-
-When server mode is running:
-
-- `GET /health`
-- `POST /reset`
-- `POST /step`
-- `GET /state`
+- GET /health
+- POST /reset
+- POST /step
+- GET /state
 
 ## Action Space
 
-The agent sends one action per step as JSON:
+Action model: [app/models/action.py](app/models/action.py)
 
-```json
-{
-  "step": 1,
-  "hospital_id": "H3",
-  "rationale": "short decision reason"
-}
-```
-
-Action fields:
-- `step` (int, >=1): must match current environment step
-- `hospital_id` (str): target hospital identifier
-- `rationale` (str, optional): policy explanation
+- step: int (1..3)
+- hospital_id: string (H1/H2/H3)
+- rationale: optional string
 
 ## Observation Space
 
-Each `reset()` and `step()` returns an observation with:
-- episode metadata: `episode_id`, `seed`, `task_id`, `scenario_name`, `scenario_difficulty`
-- patient state: `patient_condition`, `required_specialization`, remaining time fields
-- hospital list: `hospital_id`, `distance_km`, `icu`, `specialization`, `traffic`
-- routing history: visited/failed hospitals and failure reasons
-- hidden-state feedback: `last_arrival_outcome` summary (status/reason/suitability)
-- memory snapshot used by the baseline policy
-
-Core schema is defined by Pydantic models in:
-- `app/models/action.py`
-- `app/models/observation.py`
-- `app/models/state.py`
-- `app/models/reward.py`
-
-## Required Environment Variables
-
-Before running `inference.py`, define:
-- `API_BASE_URL`: API base URL for the OpenAI-compatible endpoint
-- `MODEL_NAME`: model name used for rationale generation
-- `HF_TOKEN`: API key/token
-
-Windows PowerShell example:
-
-```powershell
-$env:API_BASE_URL = "https://api-inference.huggingface.co/v1"
-$env:MODEL_NAME = "your-model-id"
-$env:HF_TOKEN = "your-token"
-```
+Observation model: [app/models/observation.py](app/models/observation.py)
 
-## Installation
+- task_id, task_objective
+- scenario metadata (name, difficulty, condition)
+- required_specialization
+- critical_time_limit_minutes
+- current step and max_steps
+- hospital list:
+  - hospital_id
+  - distance_km
+  - icu (available/unknown)
+  - specialization
+  - traffic (low/medium/high)
 
-## 1) Prerequisites
+## Reward and Info Models
 
-- Python 3.10+ (3.12 works)
-- `pip`
+Reward and grader metadata: [app/models/reward.py](app/models/reward.py)
 
-## 2) Open a terminal in this folder
+- RewardModel.value in [0, 1]
+- RewardBreakdown:
+  - survival_component
+  - time_efficiency_component
+  - specialization_component
+  - delay_penalty
+- StepInfo includes:
+  - last_action_error
+  - task_id, difficulty, objective
+  - progress_score
+  - reward_model
+  - grader (final step)
 
-Folder should be:
-- `my_env`
+## Tasks and Agent Graders
 
-## 3) Create and activate a virtual environment (recommended)
+Three deterministic tasks are supported:
 
-Windows PowerShell:
+1. acde_easy
+2. acde_medium
+3. acde_hard
 
-```powershell
-python -m venv .venv
-.\.venv\Scripts\Activate.ps1
-```
+Task objective and difficulty are fixed by task_id and drive scenario selection.
 
-macOS/Linux:
+Deterministic grader implementation: [app/environment/graders.py](app/environment/graders.py)
 
-```bash
-python -m venv .venv
-source .venv/bin/activate
-```
+Final grader score in [0, 1] is based on:
 
-## 4) Install dependencies
+- survival_rate
+- margin_rate (time margin vs critical limit)
+- specialization_rate
+- repeat_failure_penalty
 
-```bash
-pip install -e .
-```
+Difficulty-specific pass thresholds:
 
-If editable install is not needed:
+- easy: 0.75
+- medium: 0.65
+- hard: 0.50
 
-```bash
-pip install .
-```
+## Physics and Dynamics
 
-## Running the Project
+Travel model:
 
-## Option A: Run policy episodes directly (most common)
+- speed = base_speed * traffic_factor
+- traffic_factor: low=1.0, medium=0.6, high=0.3
+- travel_time_minutes = (distance_km / speed_kmh) * 60
 
-Run one medium episode:
+Hidden truth:
 
-```bash
-python inference.py --mode single --task acde_medium --episodes 1 --seed 555
-```
+- displayed ICU can be available/unknown
+- actual ICU true/false is hidden and used for survival logic
 
-Run 10 hard episodes:
+Environment changes between steps:
 
-```bash
-python inference.py --mode single --task acde_hard --episodes 10 --seed 555
-```
+- traffic can change
+- ICU truth can flip
+- displayed ICU can drift from truth
+- patient critical time window shrinks
 
-Run all levels in sequence:
+## Reward Function (Partial Signals)
 
-```bash
-python inference.py --mode full --episodes 3 --seed 555
-```
+Per-step reward in [0, 1] combines:
 
-If you run without `--task`, the script asks for level interactively.
+- survival (primary)
+- time efficiency
+- specialization match
+- delay penalty
 
-## Option B: Run as HTTP service
+This gives dense trajectory feedback and penalizes poor choices.
 
-Start API server:
+## Reproducibility
 
-```bash
-uvicorn app.server.app:app --host 0.0.0.0 --port 7860
-```
+Seed controls scenario generation, hospital parameters, traffic, and hidden ICU state. Same seed + task_id yields deterministic episode construction.
 
-Health check:
-
-```bash
-curl http://127.0.0.1:7860/health
-```
+## Inference Baseline (Required Format)
 
-## Understanding Output
+Script: [inference.py](inference.py)
 
-During `inference.py` runs, you will see:
+- uses OpenAI client for model calls
+- reads API_BASE_URL, MODEL_NAME, HF_TOKEN (or OPENAI_API_KEY)
+- emits strict structured logs:
+  - [START]
+  - [STEP]
+  - [END]
 
-- Scenario details
-- Hospital options and scores
-- Decision strategy text
-- Outcome per step (`ACCEPTED`, `PARTIAL`, `REJECTED`)
-- Final episode summary
-- Batch summary (success rate, average score, average steps)
-
-Example summary:
+Log format example:
 
 ```text
-Batch summary:
-  Success rate: 20.0%
-  Average score: 0.39
-  Average steps: 3.6
+[START] task=acde_easy env=acde-openenv model=Qwen/Qwen2.5-72B-Instruct
+[STEP] step=1 action=route('H1') reward=0.83 done=false error=null
+[STEP] step=2 action=route('H2') reward=0.54 done=false error=null
+[STEP] step=3 action=route('H1') reward=0.92 done=true error=null
+[END] success=true steps=3 score=0.84 rewards=0.83,0.54,0.92
 ```
 
-## Data Files
-
-The simulation writes data to `data/`:
-
-- `learning_memory.json`
-  - Long-term policy memory
-- `trajectory_history.jsonl`
-  - One JSON object per step
-- `learning_archive.json`
-  - Aggregate run history and profiles
-
-If you want a clean run baseline, back up and clear these files.
-
-## Typical Targets (Guideline)
-
-These are practical targets, not strict rules:
+## Local Setup
 
-- Easy: usually high success, often fewer steps
-- Medium: mixed outcomes with meaningful rerouting
-- Hard: lower success, more failures, more steps
-
-If hard success is too high, increase uncertainty or rejection pressure.
-If hard success is too low, ease one or two hard-only probabilities.
-
-## Troubleshooting
-
-## "NameError" or model field errors
-
-Make sure model fields and observation fields match after logic changes.
-If you added new state keys, also add them in observation models.
-
-## Script asks for seed/level unexpectedly
+```bash
+pip install -e .
+uvicorn app.server.app:app --host 0.0.0.0 --port 7860
+```
 
-Pass flags explicitly:
+Run inference:
 
 ```bash
-python inference.py --mode single --task acde_hard --episodes 10 --seed 555
+set API_BASE_URL=https://router.huggingface.co/v1
+set MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
+set HF_TOKEN=***
+set ENV_BASE_URL=http://127.0.0.1:7860
+python inference.py
 ```
 
-## No module named app
-
-Run commands from inside `my_env` folder, and ensure install succeeded:
+## Docker
 
 ```bash
-pip install -e .
+docker build -t acde-openenv .
+docker run --rm -p 7860:7860 acde-openenv
 ```
 
-## Uvicorn command not found
+Target runtime envelope: 2 vCPU, 8 GB RAM.
 
-Install server deps in your active environment:
+## Hugging Face Spaces Deployment
 
-```bash
-pip install uvicorn fastapi
-```
+1. Create a new Docker Space.
+2. Push this repository as the Space source.
+3. Set Space metadata tag to openenv.
+4. Add Secrets/Variables:
+   - API_BASE_URL
+   - MODEL_NAME
+   - HF_TOKEN
+5. Space should expose port 7860 and respond to /health and /reset.
 
-## Notes
+## Validation Checklist
 
-- This project is designed for iterative policy tuning.
-- Small changes in hard-mode probabilities can noticeably shift success rates.
-- Always test with at least 10-30 episodes before concluding behavior changes.
+- openenv validate passes
+- docker build succeeds
+- /health, /reset, /step, /state respond
+- inference.py runs end-to-end and prints required structured logs

app/environment/core.py

@@ -676,7 +676,7 @@ class EmergencyEnv:
             status="rejected",
             reason="Hidden mismatch at arrival (wrong risky guess). Rerouting required.",
             validation_details=arrival_outcome.validation_details,
-            reward_modifier=0.0,
+            reward_modifier=0.001,
         )
         return (
             forced_reject,
@@ -754,7 +754,7 @@ class EmergencyEnv:
                 status="rejected",
                 reason=reason,
                 validation_details=new_validation,
-                reward_modifier=0.0,
+                reward_modifier=0.001,
             ),
             0.12,
             f"Hidden case: {reason}. Rerouting required.",
@@ -805,7 +805,7 @@ class EmergencyEnv:
                 status="rejected",
                 reason="Condition became irreversible after delays",
                 validation_details=arrival_outcome.validation_details,
-                reward_modifier=0.0,
+                reward_modifier=0.001,
             ),
             "Partial chain cap: condition became irreversible.",
         )
@@ -883,7 +883,7 @@ class EmergencyEnv:
                     status="rejected",
                     reason="Condition relapsed after temporary stabilization",
                     validation_details=arrival_outcome.validation_details,
-                    reward_modifier=0.0,
+                    reward_modifier=0.001,
                     terminal=False,
                 ),
                 "Recovery guard: partial relapsed to rejected.",

app/environment/graders.py

@@ -1,3 +1,4 @@
+from typing import Literal, cast
 from app.models.reward import GraderResult
 
 
@@ -39,7 +40,7 @@ def grade_task(
     margin_rate = sum(
         _norm_margin(t.get("travel_time", 0.0), t.get("critical_limit", 1.0)) 
         for t in trajectory
-    ) / steps if trajectory else 0.0
+    ) / steps if trajectory else MIN_SCORE
 
     # Penalty for repeated failures at same hospital
     repeat_failures = 0
@@ -90,8 +91,8 @@ def grade_task(
     score = max(MIN_SCORE, min(MAX_SCORE, score))
 
     return GraderResult(
-        task_id=task_id,
-        difficulty=difficulty,
+        task_id=cast(Literal["acde_easy", "acde_hard", "acde_medium"], task_id),
+        difficulty=cast(Literal["easy", "hard", "medium"], difficulty),
         objective=objective,
         score=score,
         passed=score >= threshold,

app/environment/validation.py

@@ -4,6 +4,7 @@ Simulates hidden validation checks performed when an ambulance arrives at a hosp
 Outcomes are based on difficulty level, hospital capacity, patient suitability, and randomness.
 """
 
+from typing import Literal, cast
 from app.models.state import ArrivalOutcome, HospitalValidationDetails, HospitalState
 from app.utils.randomizer import SeededRandomizer
 
@@ -64,7 +65,7 @@ class HospitalValidator:
             icu_available=icu_available,
             doctor_available=doctor_available,
             equipment_functional=equipment_functional,
-            overload_status=overload_status,
+            overload_status=cast(Literal["clear", "moderate", "severe"], overload_status),
             patient_suitability=patient_suitability,
         )
         
@@ -79,7 +80,7 @@ class HospitalValidator:
         )
         
         return ArrivalOutcome(
-            status=status,
+            status=cast(Literal["accepted", "partial", "rejected"], status),
             reason=reason,
             validation_details=validation_details,
             reward_modifier=reward_modifier,
@@ -190,7 +191,7 @@ class HospitalValidator:
         # Add difficulty-based noise
         if difficulty == "hard":
             noise = self.rng.uniform(-0.15, 0.15)
-            suitability = max(0.0, min(1.0, suitability + noise))
+            suitability = max(0.001, min(0.999, suitability + noise))
         
         return suitability
 
@@ -269,7 +270,7 @@ class HospitalValidator:
             return (
                 "rejected",
                 f"Hospital cannot admit: {', '.join(rejection_reasons[:2])}",
-                0.0,
+                0.001,
                 False,
             )
         
@@ -345,7 +346,7 @@ class HospitalValidator:
                 return (
                     "rejected",
                     "Condition became non-transferable during delay; immediate critical care failed",
-                    0.0,
+                    0.001,
                     True,
                 )
 
@@ -376,7 +377,7 @@ class HospitalValidator:
             return (
                 "rejected",
                 "Unexpected complication at arrival",
-                0.0,
+                0.001,
                 False,
             )
 

app/models/observation.py

@@ -15,7 +15,7 @@ class ArrivalOutcomeObservation(BaseModel):
     """What happened when ambulance arrived at hospital"""
     status: Literal["accepted", "partial", "rejected"]
     reason: str
-    suitability_score: float = Field(ge=0.0, le=1.0)
+    suitability_score: float = Field(ge=0.001, le=0.999)
 
 
 class Observation(BaseModel):

app/models/state.py

@@ -6,7 +6,7 @@ from pydantic import BaseModel, Field
 class LearningEntry(BaseModel):
     success: int = Field(default=0, ge=0)
     fail: int = Field(default=0, ge=0)
-    avg: float = Field(default=0.0, ge=0.0, le=1.0)
+    avg: float = Field(default=0.001, ge=0.001, le=0.999)
     accepted: int = Field(default=0, ge=0)
     rejected: int = Field(default=0, ge=0)
 
@@ -17,7 +17,7 @@ class HospitalValidationDetails(BaseModel):
     doctor_available: bool
     equipment_functional: bool
     overload_status: Literal["clear", "moderate", "severe"]
-    patient_suitability: float = Field(ge=0.0, le=1.0)  # 0=unsuitable, 1=ideal
+    patient_suitability: float = Field(ge=0.001, le=0.999)  # near-0=unsuitable, near-1=ideal
 
 
 class ArrivalOutcome(BaseModel):
@@ -25,7 +25,7 @@ class ArrivalOutcome(BaseModel):
     status: Literal["accepted", "partial", "rejected"]
     reason: str
     validation_details: HospitalValidationDetails | None = None
-    reward_modifier: float = Field(default=1.0, ge=0.0, le=1.5)
+    reward_modifier: float = Field(default=1.0, ge=0.001, le=1.5)
     terminal: bool = False
 
 

data/learning_archive.json

@@ -8086,6 +8086,643 @@
       "best_scenario_name": "Wildfire Front (Evacuation Gridlock)",
       "best_difficulty": "hard",
       "best_required_specialization": "general"
+    },
+    "509813872|acde_easy": {
+      "attempts": 1,
+      "best_score": 0.5568666666666667,
+      "best_actions": [
+        "H3",
+        "H1",
+        "H3"
+      ],
+      "best_steps": 3,
+      "step_stats": {
+        "1": {
+          "H3": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 1,
+            "rejected": 0,
+            "total_reward": 0.15500000000000003,
+            "avg_reward": 0.15500000000000003,
+            "last_status": "PARTIAL",
+            "last_reason": "Early rejection mitigated by emergency field stabilization",
+            "success_rate": 0.0
+          }
+        },
+        "2": {
+          "H1": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.139,
+            "avg_reward": 0.139,
+            "last_status": "REJECTED",
+            "last_reason": "Hospital cannot admit: ICU unavailable, Hospital overloaded",
+            "success_rate": 0.0
+          }
+        },
+        "3": {
+          "H3": {
+            "count": 1,
+            "success": 1,
+            "accepted": 1,
+            "partial": 0,
+            "rejected": 0,
+            "total_reward": 0.7160000000000001,
+            "avg_reward": 0.7160000000000001,
+            "last_status": "ACCEPTED",
+            "last_reason": "Patient stabilized after delayed admission",
+            "success_rate": 1.0
+          }
+        }
+      },
+      "last_score": 0.5568666666666667,
+      "last_success": true,
+      "last_run_at": "2026-04-09T08:39:45.824383+00:00",
+      "last_actions": [
+        "H3",
+        "H1",
+        "H3"
+      ],
+      "last_required_specialization": "trauma",
+      "last_scenario_type": "accident",
+      "last_scenario_name": "Highway Collision (Severe)",
+      "best_success": true,
+      "best_scenario_name": "Highway Collision (Severe)",
+      "best_difficulty": "easy",
+      "best_required_specialization": "trauma"
+    },
+    "509813873|acde_medium": {
+      "attempts": 1,
+      "best_score": 0.15059999999999998,
+      "best_actions": [
+        "H3"
+      ],
+      "best_steps": 1,
+      "step_stats": {
+        "1": {
+          "H3": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Condition became non-transferable during delay; immediate critical care failed",
+            "success_rate": 0.0
+          }
+        }
+      },
+      "last_score": 0.15059999999999998,
+      "last_success": false,
+      "last_run_at": "2026-04-09T08:39:46.224541+00:00",
+      "last_actions": [
+        "H3"
+      ],
+      "last_required_specialization": "general",
+      "last_scenario_type": "fire",
+      "last_scenario_name": "Factory Fire (Chemical Exposure)",
+      "best_success": false,
+      "best_scenario_name": "Factory Fire (Chemical Exposure)",
+      "best_difficulty": "medium",
+      "best_required_specialization": "general"
+    },
+    "509813874|acde_hard": {
+      "attempts": 1,
+      "best_score": 0.15297499999999997,
+      "best_actions": [
+        "H1",
+        "H6",
+        "H4",
+        "H6"
+      ],
+      "best_steps": 4,
+      "step_stats": {
+        "1": {
+          "H1": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hospital cannot admit: ICU unavailable, Hospital overloaded",
+            "success_rate": 0.0
+          }
+        },
+        "2": {
+          "H6": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.039,
+            "avg_reward": 0.039,
+            "last_status": "REJECTED",
+            "last_reason": "Hospital cannot admit: ICU unavailable",
+            "success_rate": 0.0
+          }
+        },
+        "3": {
+          "H4": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hospital cannot admit: ICU unavailable",
+            "success_rate": 0.0
+          }
+        },
+        "4": {
+          "H6": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hidden mismatch at arrival (wrong risky guess). Rerouting required.",
+            "success_rate": 0.0
+          }
+        }
+      },
+      "last_score": 0.15297499999999997,
+      "last_success": false,
+      "last_run_at": "2026-04-09T08:39:47.915419+00:00",
+      "last_actions": [
+        "H1",
+        "H6",
+        "H4",
+        "H6"
+      ],
+      "last_required_specialization": "general",
+      "last_scenario_type": "fire",
+      "last_scenario_name": "Wildfire Front (Evacuation Gridlock)",
+      "best_success": false,
+      "best_scenario_name": "Wildfire Front (Evacuation Gridlock)",
+      "best_difficulty": "hard",
+      "best_required_specialization": "general"
+    },
+    "579018919|acde_easy": {
+      "attempts": 1,
+      "best_score": 0.4829,
+      "best_actions": [
+        "H4",
+        "H1",
+        "H4"
+      ],
+      "best_steps": 3,
+      "step_stats": {
+        "1": {
+          "H4": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 1,
+            "rejected": 0,
+            "total_reward": 0.30400000000000005,
+            "avg_reward": 0.30400000000000005,
+            "last_status": "PARTIAL",
+            "last_reason": "Early rejection mitigated by emergency field stabilization",
+            "success_rate": 0.0
+          }
+        },
+        "2": {
+          "H1": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.109,
+            "avg_reward": 0.109,
+            "last_status": "REJECTED",
+            "last_reason": "Condition became irreversible after delays",
+            "success_rate": 0.0
+          }
+        },
+        "3": {
+          "H4": {
+            "count": 1,
+            "success": 1,
+            "accepted": 1,
+            "partial": 0,
+            "rejected": 0,
+            "total_reward": 0.562,
+            "avg_reward": 0.562,
+            "last_status": "ACCEPTED",
+            "last_reason": "Condition stabilized after progressive treatment",
+            "success_rate": 1.0
+          }
+        }
+      },
+      "last_score": 0.4829,
+      "last_success": true,
+      "last_run_at": "2026-04-09T08:42:06.312270+00:00",
+      "last_actions": [
+        "H4",
+        "H1",
+        "H4"
+      ],
+      "last_required_specialization": "general",
+      "last_scenario_type": "fire",
+      "last_scenario_name": "Apartment Fire (Smoke Inhalation)",
+      "best_success": true,
+      "best_scenario_name": "Apartment Fire (Smoke Inhalation)",
+      "best_difficulty": "easy",
+      "best_required_specialization": "general"
+    },
+    "579018920|acde_medium": {
+      "attempts": 1,
+      "best_score": 0.17291249999999997,
+      "best_actions": [
+        "H3",
+        "H5",
+        "H1",
+        "H4"
+      ],
+      "best_steps": 4,
+      "step_stats": {
+        "1": {
+          "H3": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hospital cannot admit: ICU unavailable",
+            "success_rate": 0.0
+          }
+        },
+        "2": {
+          "H5": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 1,
+            "rejected": 0,
+            "total_reward": 0.358,
+            "avg_reward": 0.358,
+            "last_status": "PARTIAL",
+            "last_reason": "Admitted with significant risk: No specialist available",
+            "success_rate": 0.0
+          }
+        },
+        "3": {
+          "H1": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hospital cannot admit: ICU unavailable",
+            "success_rate": 0.0
+          }
+        },
+        "4": {
+          "H4": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hidden mismatch at arrival (wrong risky guess). Rerouting required.",
+            "success_rate": 0.0
+          }
+        }
+      },
+      "last_score": 0.17291249999999997,
+      "last_success": false,
+      "last_run_at": "2026-04-09T08:42:08.231621+00:00",
+      "last_actions": [
+        "H3",
+        "H5",
+        "H1",
+        "H4"
+      ],
+      "last_required_specialization": "general",
+      "last_scenario_type": "fire",
+      "last_scenario_name": "Factory Fire (Chemical Exposure)",
+      "best_success": false,
+      "best_scenario_name": "Factory Fire (Chemical Exposure)",
+      "best_difficulty": "medium",
+      "best_required_specialization": "general"
+    },
+    "579018921|acde_hard": {
+      "attempts": 1,
+      "best_score": 0.3184875000000001,
+      "best_actions": [
+        "H1",
+        "H6",
+        "H3",
+        "H5"
+      ],
+      "best_steps": 4,
+      "step_stats": {
+        "1": {
+          "H1": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hospital cannot admit: ICU unavailable",
+            "success_rate": 0.0
+          }
+        },
+        "2": {
+          "H6": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hospital cannot admit: Hospital overloaded",
+            "success_rate": 0.0
+          }
+        },
+        "3": {
+          "H3": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hospital cannot admit: Hospital overloaded",
+            "success_rate": 0.0
+          }
+        },
+        "4": {
+          "H5": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 1,
+            "rejected": 0,
+            "total_reward": 0.40800000000000014,
+            "avg_reward": 0.40800000000000014,
+            "last_status": "PARTIAL",
+            "last_reason": "Admitted with delays: prolonged transfer strain",
+            "success_rate": 0.0
+          }
+        }
+      },
+      "last_score": 0.3184875000000001,
+      "last_success": false,
+      "last_run_at": "2026-04-09T08:42:10.003857+00:00",
+      "last_actions": [
+        "H1",
+        "H6",
+        "H3",
+        "H5"
+      ],
+      "last_required_specialization": "general",
+      "last_scenario_type": "fire",
+      "last_scenario_name": "Wildfire Front (Evacuation Gridlock)",
+      "best_success": false,
+      "best_scenario_name": "Wildfire Front (Evacuation Gridlock)",
+      "best_difficulty": "hard",
+      "best_required_specialization": "general"
+    },
+    "468098215|acde_easy": {
+      "attempts": 1,
+      "best_score": 0.23815,
+      "best_actions": [
+        "H3",
+        "H4",
+        "H1"
+      ],
+      "best_steps": 3,
+      "step_stats": {
+        "1": {
+          "H3": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 1,
+            "rejected": 0,
+            "total_reward": 0.334,
+            "avg_reward": 0.334,
+            "last_status": "PARTIAL",
+            "last_reason": "Critical deterioration managed temporarily; reroute still needed",
+            "success_rate": 0.0
+          }
+        },
+        "2": {
+          "H4": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Emergency beds became unavailable during arrival",
+            "success_rate": 0.0
+          }
+        },
+        "3": {
+          "H1": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.139,
+            "avg_reward": 0.139,
+            "last_status": "REJECTED",
+            "last_reason": "Condition became non-transferable during delay; immediate critical care failed",
+            "success_rate": 0.0
+          }
+        }
+      },
+      "last_score": 0.23815,
+      "last_success": false,
+      "last_run_at": "2026-04-09T08:48:34.391142+00:00",
+      "last_actions": [
+        "H3",
+        "H4",
+        "H1"
+      ],
+      "last_required_specialization": "general",
+      "last_scenario_type": "fire",
+      "last_scenario_name": "Apartment Fire (Smoke Inhalation)",
+      "best_success": false,
+      "best_scenario_name": "Apartment Fire (Smoke Inhalation)",
+      "best_difficulty": "easy",
+      "best_required_specialization": "general"
+    },
+    "468098216|acde_medium": {
+      "attempts": 1,
+      "best_score": 0.7869000000000002,
+      "best_actions": [
+        "H2",
+        "H3"
+      ],
+      "best_steps": 2,
+      "step_stats": {
+        "1": {
+          "H2": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 1,
+            "rejected": 0,
+            "total_reward": 0.37,
+            "avg_reward": 0.37,
+            "last_status": "PARTIAL",
+            "last_reason": "Initial triage completed; transfer monitoring still required",
+            "success_rate": 0.0
+          }
+        },
+        "2": {
+          "H3": {
+            "count": 1,
+            "success": 1,
+            "accepted": 1,
+            "partial": 0,
+            "rejected": 0,
+            "total_reward": 0.8660000000000001,
+            "avg_reward": 0.8660000000000001,
+            "last_status": "ACCEPTED",
+            "last_reason": "Patient stabilized after delayed admission",
+            "success_rate": 1.0
+          }
+        }
+      },
+      "last_score": 0.7869000000000002,
+      "last_success": true,
+      "last_run_at": "2026-04-09T08:48:35.228474+00:00",
+      "last_actions": [
+        "H2",
+        "H3"
+      ],
+      "last_required_specialization": "trauma",
+      "last_scenario_type": "accident",
+      "last_scenario_name": "Urban Pile-up (Rush Hour)",
+      "best_success": true,
+      "best_scenario_name": "Urban Pile-up (Rush Hour)",
+      "best_difficulty": "medium",
+      "best_required_specialization": "trauma"
+    },
+    "468098217|acde_hard": {
+      "attempts": 1,
+      "best_score": 0.15059999999999998,
+      "best_actions": [
+        "H2",
+        "H3",
+        "H5",
+        "H2"
+      ],
+      "best_steps": 4,
+      "step_stats": {
+        "1": {
+          "H2": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hospital cannot admit: Hospital overloaded",
+            "success_rate": 0.0
+          }
+        },
+        "2": {
+          "H3": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hidden mismatch at arrival (wrong risky guess). Rerouting required.",
+            "success_rate": 0.0
+          }
+        },
+        "3": {
+          "H5": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hidden mismatch at arrival (wrong risky guess). Rerouting required.",
+            "success_rate": 0.0
+          }
+        },
+        "4": {
+          "H2": {
+            "count": 1,
+            "success": 0,
+            "accepted": 0,
+            "partial": 0,
+            "rejected": 1,
+            "total_reward": 0.001,
+            "avg_reward": 0.001,
+            "last_status": "REJECTED",
+            "last_reason": "Hospital cannot admit: ICU unavailable, No specialist available",
+            "success_rate": 0.0
+          }
+        }
+      },
+      "last_score": 0.15059999999999998,
+      "last_success": false,
+      "last_run_at": "2026-04-09T08:48:36.666940+00:00",
+      "last_actions": [
+        "H2",
+        "H3",
+        "H5",
+        "H2"
+      ],
+      "last_required_specialization": "cardiac",
+      "last_scenario_type": "medical",
+      "last_scenario_name": "Mass Cardiac Event (Overload)",
+      "best_success": false,
+      "best_scenario_name": "Mass Cardiac Event (Overload)",
+      "best_difficulty": "hard",
+      "best_required_specialization": "cardiac"
     }
   },
   "episodes": [
@@ -10137,6 +10774,142 @@
         "H5"
       ],
       "timestamp": "2026-04-09T05:19:13.261267+00:00"
+    },
+    {
+      "seed": 509813872,
+      "task_id": "acde_easy",
+      "difficulty": "easy",
+      "required_specialization": "trauma",
+      "scenario_name": "Highway Collision (Severe)",
+      "score": 0.5568666666666667,
+      "success": true,
+      "actions": [
+        "H3",
+        "H1",
+        "H3"
+      ],
+      "timestamp": "2026-04-09T08:39:45.824383+00:00"
+    },
+    {
+      "seed": 509813873,
+      "task_id": "acde_medium",
+      "difficulty": "medium",
+      "required_specialization": "general",
+      "scenario_name": "Factory Fire (Chemical Exposure)",
+      "score": 0.15059999999999998,
+      "success": false,
+      "actions": [
+        "H3"
+      ],
+      "timestamp": "2026-04-09T08:39:46.224541+00:00"
+    },
+    {
+      "seed": 509813874,
+      "task_id": "acde_hard",
+      "difficulty": "hard",
+      "required_specialization": "general",
+      "scenario_name": "Wildfire Front (Evacuation Gridlock)",
+      "score": 0.15297499999999997,
+      "success": false,
+      "actions": [
+        "H1",
+        "H6",
+        "H4",
+        "H6"
+      ],
+      "timestamp": "2026-04-09T08:39:47.915419+00:00"
+    },
+    {
+      "seed": 579018919,
+      "task_id": "acde_easy",
+      "difficulty": "easy",
+      "required_specialization": "general",
+      "scenario_name": "Apartment Fire (Smoke Inhalation)",
+      "score": 0.4829,
+      "success": true,
+      "actions": [
+        "H4",
+        "H1",
+        "H4"
+      ],
+      "timestamp": "2026-04-09T08:42:06.312270+00:00"
+    },
+    {
+      "seed": 579018920,
+      "task_id": "acde_medium",
+      "difficulty": "medium",
+      "required_specialization": "general",
+      "scenario_name": "Factory Fire (Chemical Exposure)",
+      "score": 0.17291249999999997,
+      "success": false,
+      "actions": [
+        "H3",
+        "H5",
+        "H1",
+        "H4"
+      ],
+      "timestamp": "2026-04-09T08:42:08.231621+00:00"
+    },
+    {
+      "seed": 579018921,
+      "task_id": "acde_hard",
+      "difficulty": "hard",
+      "required_specialization": "general",
+      "scenario_name": "Wildfire Front (Evacuation Gridlock)",
+      "score": 0.3184875000000001,
+      "success": false,
+      "actions": [
+        "H1",
+        "H6",
+        "H3",
+        "H5"
+      ],
+      "timestamp": "2026-04-09T08:42:10.003857+00:00"
+    },
+    {
+      "seed": 468098215,
+      "task_id": "acde_easy",
+      "difficulty": "easy",
+      "required_specialization": "general",
+      "scenario_name": "Apartment Fire (Smoke Inhalation)",
+      "score": 0.23815,
+      "success": false,
+      "actions": [
+        "H3",
+        "H4",
+        "H1"
+      ],
+      "timestamp": "2026-04-09T08:48:34.391142+00:00"
+    },
+    {
+      "seed": 468098216,
+      "task_id": "acde_medium",
+      "difficulty": "medium",
+      "required_specialization": "trauma",
+      "scenario_name": "Urban Pile-up (Rush Hour)",
+      "score": 0.7869000000000002,
+      "success": true,
+      "actions": [
+        "H2",
+        "H3"
+      ],
+      "timestamp": "2026-04-09T08:48:35.228474+00:00"
+    },
+    {
+      "seed": 468098217,
+      "task_id": "acde_hard",
+      "difficulty": "hard",
+      "required_specialization": "cardiac",
+      "scenario_name": "Mass Cardiac Event (Overload)",
+      "score": 0.15059999999999998,
+      "success": false,
+      "actions": [
+        "H2",
+        "H3",
+        "H5",
+        "H2"
+      ],
+      "timestamp": "2026-04-09T08:48:36.666940+00:00"
     }
   ]
 }

data/learning_memory.json

@@ -1,44 +1,44 @@
 {
   "H2": {
-    "success": 110,
-    "fail": 212,
-    "avg": 0.337900621118013,
-    "accepted": 110,
-    "rejected": 212
+    "success": 111,
+    "fail": 214,
+    "avg": 0.3359261538461545,
+    "accepted": 111,
+    "rejected": 214
   },
   "H6": {
     "success": 50,
-    "fail": 143,
-    "avg": 0.2940440414507777,
+    "fail": 145,
+    "avg": 0.291038461538462,
     "accepted": 50,
-    "rejected": 143
+    "rejected": 145
   },
   "H5": {
-    "success": 114,
-    "fail": 178,
-    "avg": 0.38379143835616414,
-    "accepted": 114,
-    "rejected": 178
+    "success": 116,
+    "fail": 179,
+    "avg": 0.38248847457627094,
+    "accepted": 116,
+    "rejected": 179
   },
   "H1": {
     "success": 111,
-    "fail": 110,
-    "avg": 0.41240090497737525,
+    "fail": 117,
+    "avg": 0.401454385964912,
     "accepted": 111,
-    "rejected": 110
+    "rejected": 117
   },
   "H3": {
-    "success": 102,
-    "fail": 156,
-    "avg": 0.3559833333333331,
-    "accepted": 102,
-    "rejected": 156
+    "success": 106,
+    "fail": 160,
+    "avg": 0.35322067669172913,
+    "accepted": 106,
+    "rejected": 160
   },
   "H4": {
-    "success": 41,
-    "fail": 105,
-    "avg": 0.34951301369862975,
-    "accepted": 41,
-    "rejected": 105
+    "success": 43,
+    "fail": 108,
+    "avg": 0.34369470198675456,
+    "accepted": 43,
+    "rejected": 108
   }
 }

data/trajectory_history.jsonl

@@ -355,3 +355,31 @@
 {"seed": 600335718, "task": "acde_medium", "difficulty": "medium", "step": 2, "state": {"patient_condition": "serious", "remaining_time_minutes": 17.0, "failed_hospitals": ["H4"], "visited_hospitals": ["H4"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H4", "policy_score": 0.07313118251968914, "strategy": "risk-aware policy + anti-stupidity guard + immediate-retry override"}, "outcome": {"status": "ACCEPTED", "reason": "Patient stabilized after delayed admission"}, "reward": 0.516}
 {"seed": 600335719, "task": "acde_hard", "difficulty": "hard", "step": 1, "state": {"patient_condition": "critical", "remaining_time_minutes": 11.0, "failed_hospitals": [], "visited_hospitals": [], "ambulance_status": "en_route"}, "action": {"hospital_id": "H6", "policy_score": 0.3244976864117649, "strategy": "safe policy + critical triage + anti-stupidity guard"}, "outcome": {"status": "REJECTED", "reason": "Hidden mismatch at arrival (wrong risky guess). Rerouting required."}, "reward": 0.001}
 {"seed": 600335719, "task": "acde_hard", "difficulty": "hard", "step": 2, "state": {"patient_condition": "critical", "remaining_time_minutes": 11.0, "failed_hospitals": ["H6"], "visited_hospitals": ["H6"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H5", "policy_score": 0.2976383394515107, "strategy": "risk-aware policy"}, "outcome": {"status": "REJECTED", "reason": "Condition became non-transferable during delay; immediate critical care failed"}, "reward": 0.001}
+{"seed": 509813872, "task": "acde_easy", "difficulty": "easy", "step": 1, "state": {"patient_condition": "serious", "remaining_time_minutes": 20.0, "failed_hospitals": [], "visited_hospitals": [], "ambulance_status": "en_route"}, "action": {"hospital_id": "H3", "policy_score": 0.3729382136813107, "strategy": "safe policy"}, "outcome": {"status": "PARTIAL", "reason": "Early rejection mitigated by emergency field stabilization"}, "reward": 0.15500000000000003}
+{"seed": 509813872, "task": "acde_easy", "difficulty": "easy", "step": 2, "state": {"patient_condition": "serious", "remaining_time_minutes": 20.0, "failed_hospitals": [], "visited_hospitals": ["H3"], "ambulance_status": "in_transit"}, "action": {"hospital_id": "H1", "policy_score": 0.4212703202476848, "strategy": "balanced policy"}, "outcome": {"status": "REJECTED", "reason": "Hospital cannot admit: ICU unavailable, Hospital overloaded"}, "reward": 0.139}
+{"seed": 509813872, "task": "acde_easy", "difficulty": "easy", "step": 3, "state": {"patient_condition": "serious", "remaining_time_minutes": 20.0, "failed_hospitals": ["H1"], "visited_hospitals": ["H3", "H1"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H3", "policy_score": 0.4992501253122812, "strategy": "risk-aware policy + anti-stupidity guard"}, "outcome": {"status": "ACCEPTED", "reason": "Patient stabilized after delayed admission"}, "reward": 0.7160000000000001}
+{"seed": 509813873, "task": "acde_medium", "difficulty": "medium", "step": 1, "state": {"patient_condition": "critical", "remaining_time_minutes": 15.0, "failed_hospitals": [], "visited_hospitals": [], "ambulance_status": "en_route"}, "action": {"hospital_id": "H3", "policy_score": 0.4050067561693999, "strategy": "safe policy + critical triage"}, "outcome": {"status": "REJECTED", "reason": "Condition became non-transferable during delay; immediate critical care failed"}, "reward": 0.001}
+{"seed": 509813874, "task": "acde_hard", "difficulty": "hard", "step": 1, "state": {"patient_condition": "critical", "remaining_time_minutes": 11.0, "failed_hospitals": [], "visited_hospitals": [], "ambulance_status": "en_route"}, "action": {"hospital_id": "H1", "policy_score": 0.49875162318940436, "strategy": "safe policy + critical triage"}, "outcome": {"status": "REJECTED", "reason": "Hospital cannot admit: ICU unavailable, Hospital overloaded"}, "reward": 0.001}
+{"seed": 509813874, "task": "acde_hard", "difficulty": "hard", "step": 2, "state": {"patient_condition": "critical", "remaining_time_minutes": 11.0, "failed_hospitals": ["H1"], "visited_hospitals": ["H1"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H6", "policy_score": 0.49875162318940436, "strategy": "risk-aware policy + guided-exploration + anti-stupidity guard"}, "outcome": {"status": "REJECTED", "reason": "Hospital cannot admit: ICU unavailable"}, "reward": 0.039}
+{"seed": 509813874, "task": "acde_hard", "difficulty": "hard", "step": 3, "state": {"patient_condition": "critical", "remaining_time_minutes": 11.0, "failed_hospitals": ["H1", "H3"], "visited_hospitals": ["H1", "H3"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H4", "policy_score": 0.14285712242855392, "strategy": "risk-aware policy"}, "outcome": {"status": "REJECTED", "reason": "Hospital cannot admit: ICU unavailable"}, "reward": 0.001}
+{"seed": 509813874, "task": "acde_hard", "difficulty": "hard", "step": 4, "state": {"patient_condition": "critical", "remaining_time_minutes": 11.0, "failed_hospitals": ["H1", "H3", "H4"], "visited_hospitals": ["H1", "H3", "H4"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H6", "policy_score": 0.49875162318940436, "strategy": "risk-aware policy + anti-stupidity guard"}, "outcome": {"status": "REJECTED", "reason": "Hidden mismatch at arrival (wrong risky guess). Rerouting required."}, "reward": 0.001}
+{"seed": 579018919, "task": "acde_easy", "difficulty": "easy", "step": 1, "state": {"patient_condition": "serious", "remaining_time_minutes": 18.0, "failed_hospitals": [], "visited_hospitals": [], "ambulance_status": "en_route"}, "action": {"hospital_id": "H4", "policy_score": 0.32949151496142953, "strategy": "safe policy"}, "outcome": {"status": "PARTIAL", "reason": "Early rejection mitigated by emergency field stabilization"}, "reward": 0.30400000000000005}
+{"seed": 579018919, "task": "acde_easy", "difficulty": "easy", "step": 2, "state": {"patient_condition": "serious", "remaining_time_minutes": 18.0, "failed_hospitals": [], "visited_hospitals": ["H4"], "ambulance_status": "in_transit"}, "action": {"hospital_id": "H1", "policy_score": 0.34824268005046066, "strategy": "balanced policy + anti-stupidity guard"}, "outcome": {"status": "REJECTED", "reason": "Condition became irreversible after delays"}, "reward": 0.109}
+{"seed": 579018919, "task": "acde_easy", "difficulty": "easy", "step": 3, "state": {"patient_condition": "unstable", "remaining_time_minutes": 18.0, "failed_hospitals": ["H1"], "visited_hospitals": ["H4", "H1"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H4", "policy_score": 0.4992501253122812, "strategy": "risk-aware policy + anti-stupidity guard"}, "outcome": {"status": "ACCEPTED", "reason": "Condition stabilized after progressive treatment"}, "reward": 0.562}
+{"seed": 579018920, "task": "acde_medium", "difficulty": "medium", "step": 1, "state": {"patient_condition": "critical", "remaining_time_minutes": 15.0, "failed_hospitals": [], "visited_hospitals": [], "ambulance_status": "en_route"}, "action": {"hospital_id": "H3", "policy_score": 0.4843607591902248, "strategy": "safe policy + critical triage + anti-stupidity guard"}, "outcome": {"status": "REJECTED", "reason": "Hospital cannot admit: ICU unavailable"}, "reward": 0.001}
+{"seed": 579018920, "task": "acde_medium", "difficulty": "medium", "step": 2, "state": {"patient_condition": "critical", "remaining_time_minutes": 15.0, "failed_hospitals": ["H3"], "visited_hospitals": ["H3"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H5", "policy_score": 0.48373386963831955, "strategy": "risk-aware policy + anti-stupidity guard"}, "outcome": {"status": "PARTIAL", "reason": "Admitted with significant risk: No specialist available"}, "reward": 0.358}
+{"seed": 579018920, "task": "acde_medium", "difficulty": "medium", "step": 3, "state": {"patient_condition": "critical", "remaining_time_minutes": 15.0, "failed_hospitals": ["H3"], "visited_hospitals": ["H3", "H5"], "ambulance_status": "in_transit"}, "action": {"hospital_id": "H1", "policy_score": 0.16666663886108793, "strategy": "balanced policy"}, "outcome": {"status": "REJECTED", "reason": "Hospital cannot admit: ICU unavailable"}, "reward": 0.001}
+{"seed": 579018920, "task": "acde_medium", "difficulty": "medium", "step": 4, "state": {"patient_condition": "critical", "remaining_time_minutes": 15.0, "failed_hospitals": ["H3", "H1"], "visited_hospitals": ["H3", "H5", "H1"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H4", "policy_score": 0.16666663886108793, "strategy": "risk-aware policy"}, "outcome": {"status": "REJECTED", "reason": "Hidden mismatch at arrival (wrong risky guess). Rerouting required."}, "reward": 0.001}
+{"seed": 579018921, "task": "acde_hard", "difficulty": "hard", "step": 1, "state": {"patient_condition": "critical", "remaining_time_minutes": 11.0, "failed_hospitals": [], "visited_hospitals": [], "ambulance_status": "en_route"}, "action": {"hospital_id": "H1", "policy_score": 0.35191055672786264, "strategy": "safe policy + critical triage"}, "outcome": {"status": "REJECTED", "reason": "Hospital cannot admit: ICU unavailable"}, "reward": 0.001}
+{"seed": 579018921, "task": "acde_hard", "difficulty": "hard", "step": 2, "state": {"patient_condition": "critical", "remaining_time_minutes": 11.0, "failed_hospitals": ["H1"], "visited_hospitals": ["H1"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H6", "policy_score": 0.49875162318940436, "strategy": "risk-aware policy"}, "outcome": {"status": "REJECTED", "reason": "Hospital cannot admit: Hospital overloaded"}, "reward": 0.001}
+{"seed": 579018921, "task": "acde_hard", "difficulty": "hard", "step": 3, "state": {"patient_condition": "critical", "remaining_time_minutes": 11.0, "failed_hospitals": ["H1", "H6"], "visited_hospitals": ["H1", "H6"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H3", "policy_score": 0.14285712242855392, "strategy": "risk-aware policy"}, "outcome": {"status": "REJECTED", "reason": "Hospital cannot admit: Hospital overloaded"}, "reward": 0.001}
+{"seed": 579018921, "task": "acde_hard", "difficulty": "hard", "step": 4, "state": {"patient_condition": "critical", "remaining_time_minutes": 11.0, "failed_hospitals": ["H1", "H6", "H4"], "visited_hospitals": ["H1", "H6", "H4"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H5", "policy_score": 0.49875162318940436, "strategy": "risk-aware policy + anti-stupidity guard"}, "outcome": {"status": "PARTIAL", "reason": "Admitted with delays: prolonged transfer strain"}, "reward": 0.40800000000000014}
+{"seed": 468098215, "task": "acde_easy", "difficulty": "easy", "step": 1, "state": {"patient_condition": "serious", "remaining_time_minutes": 18.0, "failed_hospitals": [], "visited_hospitals": [], "ambulance_status": "en_route"}, "action": {"hospital_id": "H3", "policy_score": 0.3443315337257757, "strategy": "safe policy + anti-stupidity guard"}, "outcome": {"status": "PARTIAL", "reason": "Critical deterioration managed temporarily; reroute still needed"}, "reward": 0.334}
+{"seed": 468098215, "task": "acde_easy", "difficulty": "easy", "step": 2, "state": {"patient_condition": "stable", "remaining_time_minutes": 18.0, "failed_hospitals": [], "visited_hospitals": ["H3"], "ambulance_status": "in_transit"}, "action": {"hospital_id": "H4", "policy_score": 0.24369633883371342, "strategy": "balanced policy"}, "outcome": {"status": "REJECTED", "reason": "Emergency beds became unavailable during arrival"}, "reward": 0.001}
+{"seed": 468098215, "task": "acde_easy", "difficulty": "easy", "step": 3, "state": {"patient_condition": "stable", "remaining_time_minutes": 18.0, "failed_hospitals": ["H4"], "visited_hospitals": ["H3", "H4"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H1", "policy_score": 0.27278944968751606, "strategy": "risk-aware policy"}, "outcome": {"status": "REJECTED", "reason": "Condition became non-transferable during delay; immediate critical care failed"}, "reward": 0.139}
+{"seed": 468098216, "task": "acde_medium", "difficulty": "medium", "step": 1, "state": {"patient_condition": "serious", "remaining_time_minutes": 17.0, "failed_hospitals": [], "visited_hospitals": [], "ambulance_status": "en_route"}, "action": {"hospital_id": "H2", "policy_score": 0.4171322328020302, "strategy": "safe policy + guided-exploration"}, "outcome": {"status": "PARTIAL", "reason": "Initial triage completed; transfer monitoring still required"}, "reward": 0.37}
+{"seed": 468098216, "task": "acde_medium", "difficulty": "medium", "step": 2, "state": {"patient_condition": "serious", "remaining_time_minutes": 17.0, "failed_hospitals": [], "visited_hospitals": ["H2"], "ambulance_status": "in_transit"}, "action": {"hospital_id": "H3", "policy_score": 0.43825439745695755, "strategy": "balanced policy + anti-stupidity guard"}, "outcome": {"status": "ACCEPTED", "reason": "Patient stabilized after delayed admission"}, "reward": 0.8660000000000001}
+{"seed": 468098217, "task": "acde_hard", "difficulty": "hard", "step": 1, "state": {"patient_condition": "critical", "remaining_time_minutes": 12.0, "failed_hospitals": [], "visited_hospitals": [], "ambulance_status": "en_route"}, "action": {"hospital_id": "H2", "policy_score": 0.49964989496849055, "strategy": "safe policy + critical triage"}, "outcome": {"status": "REJECTED", "reason": "Hospital cannot admit: Hospital overloaded"}, "reward": 0.001}
+{"seed": 468098217, "task": "acde_hard", "difficulty": "hard", "step": 2, "state": {"patient_condition": "critical", "remaining_time_minutes": 12.0, "failed_hospitals": ["H2"], "visited_hospitals": ["H2"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H3", "policy_score": 0.29411764705882354, "strategy": "risk-aware policy"}, "outcome": {"status": "REJECTED", "reason": "Hidden mismatch at arrival (wrong risky guess). Rerouting required."}, "reward": 0.001}
+{"seed": 468098217, "task": "acde_hard", "difficulty": "hard", "step": 3, "state": {"patient_condition": "critical", "remaining_time_minutes": 12.0, "failed_hospitals": ["H2", "H3"], "visited_hospitals": ["H2", "H3"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H5", "policy_score": 0.039999999999999994, "strategy": "risk-aware policy"}, "outcome": {"status": "REJECTED", "reason": "Hidden mismatch at arrival (wrong risky guess). Rerouting required."}, "reward": 0.001}
+{"seed": 468098217, "task": "acde_hard", "difficulty": "hard", "step": 4, "state": {"patient_condition": "critical", "remaining_time_minutes": 12.0, "failed_hospitals": ["H2", "H3", "H1"], "visited_hospitals": ["H2", "H3", "H1"], "ambulance_status": "rerouting"}, "action": {"hospital_id": "H2", "policy_score": 0.29411764705882354, "strategy": "risk-aware policy + immediate-retry override"}, "outcome": {"status": "REJECTED", "reason": "Hospital cannot admit: ICU unavailable, No specialist available"}, "reward": 0.001}

inference.py

@@ -18,13 +18,17 @@ import random
 from pathlib import Path
 from datetime import datetime, timezone
 
+from typing import Any, Literal, cast, TYPE_CHECKING
+if TYPE_CHECKING:
+    from openai import OpenAI
+
 from app.environment.core import EmergencyEnv
 from app.models.action import Action
 
 try:
     from openai import OpenAI
 except Exception:  # pragma: no cover - fallback for missing optional dependency
-    OpenAI = None
+    OpenAI = None  # type: ignore
 
 TASK_ORDER = ["acde_easy", "acde_medium", "acde_hard"]
 LEVEL_TO_TASK = {
@@ -71,7 +75,7 @@ def runtime_llm_config() -> dict[str, str]:
     }
 
 
-def require_llm_config() -> tuple[object, str]:
+def require_llm_config() -> tuple[Any, str]:
     config = runtime_llm_config()
     missing = [name for name, value in config.items() if not value]
     if missing:
@@ -88,7 +92,7 @@ def require_llm_config() -> tuple[object, str]:
 
 
 def llm_rationale(
-    client: object,
+    client: Any,
     model_name: str,
     observation: dict,
     chosen: dict,
@@ -315,7 +319,7 @@ def memory_score_for_hospital(
     if recent_failed:
         value -= 0.3
 
-    return max(0.0, min(1.0, value))
+    return max(0.001, min(0.999, value))
 
 
 def score_hospitals(observation: dict, learning_profile: dict | None = None) -> list[dict]:
@@ -328,8 +332,8 @@ def score_hospitals(observation: dict, learning_profile: dict | None = None) ->
     last_status = str(last_arrival.get("status", "")).lower()
 
     scored: list[dict] = []
-    initial_limit = float(observation.get("initial_critical_time_limit_minutes", observation["critical_time_limit_minutes"]))
-    remaining_time = float(observation.get("remaining_time_minutes", observation["critical_time_limit_minutes"]))
+    initial_limit = float(observation.get("initial_critical_time_limit_minutes") or observation.get("critical_time_limit_minutes") or 15.0)
+    remaining_time = float(observation.get("remaining_time_minutes") or observation.get("critical_time_limit_minutes") or 15.0)
     urgency = 1.0 - min(1.0, max(0.0, remaining_time / max(initial_limit, 1e-6)))
 
     patient_condition = observation.get("patient_condition", "").lower()
@@ -473,7 +477,7 @@ def score_hospitals(observation: dict, learning_profile: dict | None = None) ->
                 "specialization": hospital["specialization"],
                 "travel_time": travel_time,
                 "memory_score": mem_score,
-                "policy_score": max(0.0, min(1.0, score)),
+                "policy_score": max(0.001, min(0.999, score)),
                 "specialization_match": spec_match,
                 "tie_break_score": (
                     (distance_score * 0.35)
@@ -500,7 +504,7 @@ def score_hospitals(observation: dict, learning_profile: dict | None = None) ->
                     )
                     jitter_rng = random.Random(jitter_seed)
                     normalized *= jitter_rng.uniform(0.3, 0.7)
-                item["policy_score"] = max(0.0, min(1.0, normalized))
+                item["policy_score"] = max(0.001, min(0.999, normalized))
         elif max_score > 0:
             for item in scored:
                 normalized = item["policy_score"] / max_score
@@ -512,7 +516,7 @@ def score_hospitals(observation: dict, learning_profile: dict | None = None) ->
                     )
                     jitter_rng = random.Random(jitter_seed)
                     normalized *= jitter_rng.uniform(0.3, 0.7)
-                item["policy_score"] = max(0.0, min(1.0, normalized))
+                item["policy_score"] = max(0.001, min(0.999, normalized))
         else:
             tie_min = min(item.get("tie_break_score", 0.0) for item in scored)
             tie_max = max(item.get("tie_break_score", 0.0) for item in scored)
@@ -528,10 +532,10 @@ def score_hospitals(observation: dict, learning_profile: dict | None = None) ->
                         )
                         jitter_rng = random.Random(jitter_seed)
                         normalized *= jitter_rng.uniform(0.3, 0.7)
-                    item["policy_score"] = max(0.0, min(1.0, normalized))
+                    item["policy_score"] = max(0.001, min(0.999, normalized))
             else:
                 for item in scored:
-                    item["policy_score"] = 0.0
+                    item["policy_score"] = 0.001
 
         # Remove hard-zero scores and normalize to probability-like values.
         for item in scored:
@@ -585,7 +589,7 @@ def score_hospitals(observation: dict, learning_profile: dict | None = None) ->
                 )
                 jitter_rng = random.Random(jitter_seed)
                 normalized_score = jitter_rng.uniform(0.01, 0.03)
-            item["policy_score"] = normalized_score
+            item["policy_score"] = max(0.001, min(0.999, normalized_score))
 
         scored.sort(key=lambda item: item["policy_score"], reverse=True)
 
@@ -1060,9 +1064,9 @@ def run_episode(
             "seed": seed,
             "result": final_result,
             "success": final_result == "SUCCESS",
-            "score": round(final_score, 4),
+            "score": max(0.001, min(0.999, round(final_score, 4))),
             "steps": steps,
-            "average_reward": round(total_reward / max(1, steps), 4),
+            "average_reward": max(0.001, min(0.999, round(total_reward / max(1, steps), 4))),
         },
     )