Phase 1: repo setup and environment port

Dockerfile

@@ -0,0 +1,16 @@
+FROM python:3.11-slim
+
+WORKDIR /app
+
+# Copy requirements first (layer caching)
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy all source
+COPY . .
+
+# Expose port (HF Spaces uses 7860)
+EXPOSE 7860
+
+# Start server
+CMD ["uvicorn", "server.app:app", "--host", "0.0.0.0", "--port", "7860"]

PHASE_1.md

@@ -0,0 +1,429 @@
+# PHASE 1 — Repo Setup + Environment Port + AMD LLM Swap
+
+> **Goal:** New repo created, old codebase copied cleanly, AMD Developer Cloud LLM working as a drop-in replacement for Groq. Environment boots, `/health` responds, and a test LLM call returns a valid response.
+>
+> **Time budget:** Day 1 (full day)
+>
+> **Success condition:** `curl http://localhost:7860/health` returns ok AND `python amd_client.py` returns a valid LLM response from AMD.
+
+---
+
+## STEP 1 — Create the New GitHub Repo
+
+### 1.1 Create repo on GitHub
+- Go to https://github.com/new
+- Name: `agentic-triage-amd`
+- Visibility: **Public**
+- Initialize with: **nothing** (no README, no .gitignore)
+- Click **Create repository**
+
+### 1.2 Clone it locally
+```bash
+cd ~/projects   # or wherever you keep your code
+git clone https://github.com/YOUR_USERNAME/agentic-triage-amd.git
+cd agentic-triage-amd
+```
+
+### 1.3 Set up base folder structure
+```bash
+mkdir -p server/scenarios server/graders agents
+touch amd_client.py run_agent.py .env.example
+```
+
+**Checklist:**
+- [ ] Repo created on GitHub as Public
+- [ ] Cloned locally
+- [ ] Folder structure created
+
+---
+
+## STEP 2 — Copy Core Files from Old Repo
+
+### 2.1 Copy the server directory
+From inside `agentic-triage-amd/`, run:
+
+```bash
+cp ../logtriage-env/server/app.py ./server/
+cp ../logtriage-env/server/environment.py ./server/
+cp ../logtriage-env/server/models.py ./server/
+cp ../logtriage-env/server/log_generator.py ./server/
+cp ../logtriage-env/server/scenarios/single_crash.py ./server/scenarios/
+cp ../logtriage-env/server/scenarios/cascading.py ./server/scenarios/
+cp ../logtriage-env/server/scenarios/silent_degrade.py ./server/scenarios/
+cp ../logtriage-env/server/graders/base_grader.py ./server/graders/
+cp ../logtriage-env/server/graders/crash_grader.py ./server/graders/
+cp ../logtriage-env/server/graders/cascade_grader.py ./server/graders/
+cp ../logtriage-env/Dockerfile ./
+cp ../logtriage-env/requirements.txt ./
+```
+
+> Adjust the `../logtriage-env/` path  to "C:\Users\Rohit\Desktop\logtriage-env"  because we need to copy files from here 
+### 2.2 Create `__init__.py` files
+```bash
+touch server/__init__.py
+touch server/scenarios/__init__.py
+touch server/graders/__init__.py
+touch agents/__init__.py
+```
+
+### 2.3 Verify structure
+```bash
+find . -type f -name "*.py" | sort
+```
+
+Expected output:
+```
+./agents/__init__.py
+./amd_client.py
+./run_agent.py
+./server/__init__.py
+./server/app.py
+./server/environment.py
+./server/graders/__init__.py
+./server/graders/base_grader.py
+./server/graders/cascade_grader.py
+./server/graders/crash_grader.py
+./server/log_generator.py
+./server/models.py
+./server/scenarios/__init__.py
+./server/scenarios/cascading.py
+./server/scenarios/single_crash.py
+./server/scenarios/silent_degrade.py
+```
+
+**Checklist:**
+- [ ] All server files copied
+- [ ] All grader files copied
+- [ ] All scenario files copied
+- [ ] `__init__.py` files created in all 4 directories
+
+---
+
+## STEP 3 — Update requirements.txt
+
+Replace the entire content of `requirements.txt` with:
+
+```txt
+# Environment
+fastapi>=0.104.0
+uvicorn>=0.24.0
+pydantic>=2.0.0
+openenv-core==0.2.3
+
+# HTTP
+requests>=2.25.0
+httpx>=0.24.0
+
+# LLM + Agents
+openai>=1.0.0
+langgraph>=0.1.0
+langchain>=0.2.0
+langchain-openai>=0.1.0
+
+# Utilities
+python-dotenv>=1.0.0
+```
+
+Install:
+```bash
+pip install -r requirements.txt
+```
+
+Verify:
+```bash
+python -c "import fastapi; import langgraph; import openai; print('All packages OK')"
+```
+
+**Checklist:**
+- [ ] `requirements.txt` updated (Groq removed, LangGraph added)
+- [ ] `pip install` completes without errors
+- [ ] Verification print shows `All packages OK`
+
+---
+
+## STEP 4 — AMD Developer Cloud Setup
+
+### 4.1 Get your AMD API key
+- Go to https://www.amd.com/en/developer/resources/ml-software/developer-cloud.html
+- Sign up / log in to AMD Developer Cloud
+- Navigate to API Keys section in the dashboard
+- Generate a new key — copy it immediately
+
+### 4.2 Note your model and base URL
+- In the dashboard, find the Inference / API section
+- Note down exactly:
+  - The base URL (e.g. `https://api.amd.com/v1`)
+  - Available model names (e.g. `mistral-7b-instruct`, `llama-3-8b-instruct`)
+
+### 4.3 Create `.env`
+```bash
+cat > .env << EOF
+AMD_API_KEY=your_actual_key_here
+AMD_BASE_URL=https://api.amd.com/v1
+AMD_MODEL=mistral-7b-instruct
+ENV_HOST=0.0.0.0
+ENV_PORT=7860
+EOF
+```
+
+### 4.4 Create `.env.example` (safe to commit)
+```bash
+cat > .env.example << EOF
+AMD_API_KEY=your_amd_developer_cloud_api_key
+AMD_BASE_URL=https://api.amd.com/v1
+AMD_MODEL=mistral-7b-instruct
+ENV_HOST=0.0.0.0
+ENV_PORT=7860
+EOF
+```
+
+### 4.5 Create `.gitignore`
+```bash
+cat > .gitignore << EOF
+.env
+__pycache__/
+*.pyc
+*.pyo
+.DS_Store
+*.egg-info/
+dist/
+build/
+.venv/
+venv/
+EOF
+```
+
+**Checklist:**
+- [ ] AMD Developer Cloud account active
+- [ ] API key obtained and saved
+- [ ] Base URL and model name confirmed from dashboard
+- [ ] `.env` created with real values
+- [ ] `.env.example` created
+- [ ] `.gitignore` created
+
+---
+
+## STEP 5 — Write amd_client.py
+
+Open `amd_client.py` and paste this:
+
+```python
+import os
+from openai import OpenAI
+from dotenv import load_dotenv
+
+load_dotenv()
+
+
+def get_amd_client() -> OpenAI:
+    """Returns an OpenAI-compatible client pointed at AMD Developer Cloud."""
+    return OpenAI(
+        api_key=os.environ["AMD_API_KEY"],
+        base_url=os.environ["AMD_BASE_URL"],
+    )
+
+
+def call_amd_llm(
+    prompt: str,
+    system_prompt: str = None,
+    temperature: float = 0.2
+) -> str:
+    """
+    Single LLM call to AMD Developer Cloud.
+    Returns the response text as a plain string.
+    """
+    client = get_amd_client()
+    model = os.environ.get("AMD_MODEL", "mistral-7b-instruct")
+
+    messages = []
+    if system_prompt:
+        messages.append({"role": "system", "content": system_prompt})
+    messages.append({"role": "user", "content": prompt})
+
+    response = client.chat.completions.create(
+        model=model,
+        messages=messages,
+        temperature=temperature,
+        max_tokens=1024,
+    )
+
+    return response.choices[0].message.content.strip()
+
+
+if __name__ == "__main__":
+    # Quick connection test
+    print("Testing AMD Developer Cloud connection...")
+    result = call_amd_llm(
+        prompt=(
+            "A payment-service is throwing NullPointerException. "
+            "Error rate is 100%. All downstream services are timing out. "
+            "What severity is this? Answer with P1, P2, or P3 only."
+        ),
+        system_prompt="You are an expert Site Reliability Engineer. Be concise."
+    )
+    print(f"\nAMD LLM Response: {result}")
+    print("\nConnection test PASSED." if result else "Connection test FAILED.")
+```
+
+### Run the test:
+```bash
+python amd_client.py
+```
+
+Expected output:
+```
+Testing AMD Developer Cloud connection...
+
+AMD LLM Response: P1
+
+Connection test PASSED.
+```
+
+Any coherent text response means the AMD connection works.
+
+**Checklist:**
+- [ ] `amd_client.py` written
+- [ ] `python amd_client.py` runs without errors
+- [ ] AMD LLM returns a valid response
+- [ ] No 401/403/404 errors
+
+---
+
+## STEP 6 — Verify the Environment Boots
+
+### 6.1 Start the FastAPI server
+```bash
+uvicorn server.app:app --host 0.0.0.0 --port 7860 --reload
+```
+
+Watch the terminal — it should say `Application startup complete.`
+
+### 6.2 Test all endpoints (new terminal)
+```bash
+# Health check
+curl http://localhost:7860/health
+
+# List tasks
+curl http://localhost:7860/tasks
+
+# Reset to single_crash task
+curl -X POST http://localhost:7860/reset \
+  -H "Content-Type: application/json" \
+  -d '{"task_id": "single_crash", "seed": 42}'
+```
+
+Expected:
+- `/health` → `{"status": "ok"}`
+- `/tasks` → JSON array with 3 task objects
+- `/reset` → Observation JSON with `logs`, `service_state`, `reward`, `done` fields
+
+If `/reset` returns a proper observation with logs — the environment is fully working.
+
+**Checklist:**
+- [ ] Server starts with no import errors
+- [ ] `/health` returns ok
+- [ ] `/tasks` returns 3 tasks
+- [ ] `/reset` with `single_crash` returns an observation with logs
+
+---
+
+## STEP 7 — Add README and First Git Push
+
+### 7.1 Add the README
+Copy the `README.md` from the files provided into the repo root. Update:
+- Replace `YOUR_USERNAME` with your actual GitHub username
+- Fill in your teammate's name in the Team table
+
+### 7.2 Commit and push everything
+```bash
+# Stage all files
+git add .
+
+# Verify .env is NOT being staged
+git status
+# You should NOT see .env in the list — if you do, check .gitignore
+
+# Initial commit
+git commit -m "Phase 1: Repo setup, environment port, AMD LLM client"
+
+# Push to GitHub
+git push origin main
+```
+
+### 7.3 Verify on GitHub
+- Open the repo on GitHub
+- Confirm all files are there
+- Confirm `.env` is NOT visible (only `.env.example` should be there)
+
+**Checklist:**
+- [ ] README added with correct username
+- [ ] `git status` does NOT show `.env`
+- [ ] Committed with descriptive message
+- [ ] `git push` successful
+- [ ] Repo looks clean on GitHub
+
+---
+
+## PHASE 1 COMPLETE — Final Verification
+
+Run through every item before calling Phase 1 done:
+
+- [ ] `agentic-triage-amd` repo exists on GitHub (Public)
+- [ ] All server/ files present and importable
+- [ ] LangGraph and openai installed without errors
+- [ ] `.env` has real AMD credentials
+- [ ] `python amd_client.py` → valid LLM response from AMD
+- [ ] `uvicorn server.app:app` starts cleanly
+- [ ] `/health` → ok
+- [ ] `/tasks` → 3 tasks returned
+- [ ] `/reset` → observation with logs returned
+- [ ] `.env` NOT committed to GitHub
+- [ ] First commit pushed successfully
+
+---
+
+## Troubleshooting
+
+**`ModuleNotFoundError: No module named 'server'`**
+```bash
+# Always run from the project root directory
+cd agentic-triage-amd
+uvicorn server.app:app --host 0.0.0.0 --port 7860
+```
+
+**`AMD API returns 401 Unauthorized`**
+- Double check `AMD_API_KEY` in `.env` — no extra spaces or quotes
+- Confirm `python-dotenv` is installed and `load_dotenv()` is called
+- Verify the key is active in the AMD Developer Cloud dashboard
+
+**`AMD API returns 404 Not Found`**
+- The model name might be wrong — check exact names in AMD dashboard
+- Update `AMD_MODEL` in `.env` to match the exact available model string
+
+**`AMD API returns connection error`**
+- Check `AMD_BASE_URL` — get the exact URL from the dashboard, don't guess
+
+**`ImportError` on any scenario or grader file**
+- Check `__init__.py` exists in `server/`, `server/scenarios/`, `server/graders/`, `agents/`
+```bash
+find . -name "__init__.py"
+```
+
+**Port 7860 already in use**
+```bash
+# Kill whatever is on that port
+lsof -i :7860
+kill -9 <PID>
+```
+
+---
+
+## What's Next — Phase 2 Preview
+
+Phase 2 builds the three agents:
+- `agents/planner.py` — reads the initial observation, outputs a triage strategy
+- `agents/executor.py` — loops through steps, calls the environment, takes actions
+- `agents/summarizer.py` — takes the completed episode, generates an incident report
+- `agents/pipeline.py` — LangGraph graph wiring all three together
+
+**Once all Phase 1 checkboxes are ticked, share your summary and we build Phase 2.**

README.md

@@ -0,0 +1,249 @@
+# 🔴 AgentTriage — Agentic SRE Incident Response on AMD Developer Cloud
+
+> Multi-agent log triage system that autonomously diagnoses production incidents using AMD-hosted LLMs and a LangGraph-powered agent pipeline.
+
+Built for the **AMD Developer Cloud Hackathon — Track 1: AI Agents & Agentic Workflows**
+
+---
+
+## 📌 What Is This?
+
+AgentTriage is a production-grade agentic system where an AI agent pipeline automatically triages software incidents — the same work a human Site Reliability Engineer (SRE) does when production goes down.
+
+When something breaks in production (a server crashes, a database causes a cascade failure, or a service silently degrades), engineers need to:
+1. Diagnose the severity (P1/P2/P3)
+2. Identify the root cause (which service/component)
+3. Decide on remediation (restart, kill-query, flush-cache)
+4. Escalate to the right team
+
+AgentTriage automates this entire workflow using a **multi-agent pipeline** running on **AMD Developer Cloud**.
+
+---
+
+## 🧠 How It Works
+
+### The Environment (LogTriageEnv)
+A simulated microservice incident environment with a REST API interface (OpenEnv-compatible). The agent interacts via a reset → step loop, reads logs and service states, takes actions, and gets scored.
+
+**Three incident scenarios:**
+
+| Task | Difficulty | Noise | Incident Type |
+|---|---|---|---|
+| `single_crash` | Easy | 20% | Payment service NullPointerException |
+| `cascading_failure` | Medium | 30% | user-db slow query → auth → gateway cascade |
+| `silent_degradation` | Hard | 60% | Gradual payment-db latency increase (no crash) |
+
+### The Agent Pipeline (New — AMD-Powered)
+
+```
+Incoming Logs + Service State
+         ↓
+   [PLANNER AGENT]
+   Reads logs, decides strategy
+         ↓
+   [EXECUTOR AGENT]
+   Takes triage actions step-by-step
+   (classify_severity → identify_root_cause → remediate → resolve)
+         ↓
+   [SUMMARIZER AGENT]
+   Produces structured incident report
+         ↓
+   Episode Score (0.0 → 1.0)
+```
+
+All agents run on **AMD Developer Cloud hosted LLMs** (Mistral/Llama) via their OpenAI-compatible API endpoint.
+
+---
+
+## 🏗️ Architecture
+
+```
+┌─────────────────────────────────────────────────┐
+│              AgentTriage System                  │
+│                                                  │
+│  ┌──────────────┐     ┌──────────────────────┐  │
+│  │  LangGraph   │────▶│   AMD Developer      │  │
+│  │  Agent Loop  │     │   Cloud LLM API      │  │
+│  │              │◀────│   (Mistral/Llama)    │  │
+│  └──────┬───────┘     └──────────────────────┘  │
+│         │                                        │
+│  ┌──────▼───────┐                               │
+│  │  LogTriage   │                               │
+│  │  Environment │                               │
+│  │  (FastAPI)   │                               │
+│  └──────┬───────┘                               │
+│         │                                        │
+│  ┌──────▼───────────────────────────────────┐   │
+│  │          Scenario Engine                  │   │
+│  │  single_crash | cascading | silent_degrade│   │
+│  └──────┬───────────────────────────────────┘   │
+│         │                                        │
+│  ┌──────▼───────┐                               │
+│  │    Grader    │  → Episode Score (0.0–1.0)    │
+│  └──────────────┘                               │
+└─────────────────────────────────────────────────┘
+```
+
+---
+
+## 🛠️ Tech Stack
+
+| Layer | Technology |
+|---|---|
+| Agent Framework | LangGraph |
+| LLM Backend | AMD Developer Cloud (Mistral-7B / Llama-3) |
+| Environment API | FastAPI + Uvicorn |
+| Data Validation | Pydantic v2 |
+| Containerization | Docker |
+| Environment Interface | OpenEnv-compatible (reset/step) |
+| Language | Python 3.11 |
+
+---
+
+## 📁 Project Structure
+
+```
+agentic-triage-amd/
+│
+├── server/                        # LogTriageEnv (environment core)
+│   ├── app.py                     # FastAPI endpoints (/reset, /step, /state, /tasks)
+│   ├── environment.py             # Core simulator (reset/step/state)
+│   ├── models.py                  # Pydantic schemas (LogLine, TriageAction, etc.)
+│   ├── log_generator.py           # Log + service state generation
+│   ├── scenarios/
+│   │   ├── single_crash.py        # Task 1: Payment service crash
+│   │   ├── cascading.py           # Task 2: user-db → auth → gateway cascade
+│   │   └── silent_degrade.py      # Task 3: Gradual latency degradation
+│   └── graders/
+│       ├── base_grader.py         # Abstract grader interface
+│       ├── crash_grader.py        # Task 1 scoring
+│       └── cascade_grader.py      # Task 2 scoring
+│
+├── agents/                        # Multi-agent pipeline (NEW)
+│   ├── planner.py                 # Planner agent — reads logs, sets strategy
+│   ├── executor.py                # Executor agent — takes step-by-step actions
+│   ├── summarizer.py              # Summarizer agent — generates incident report
+│   └── pipeline.py                # LangGraph graph definition
+│
+├── amd_client.py                  # AMD Developer Cloud LLM client
+├── run_agent.py                   # Entry point — runs agent on all 3 tasks
+├── Dockerfile                     # Container definition
+├── requirements.txt               # Python dependencies
+├── .env.example                   # Environment variable template
+└── README.md                      # This file
+```
+
+---
+
+## ⚙️ Setup & Installation
+
+### Prerequisites
+- Python 3.11+
+- Docker
+- AMD Developer Cloud account + API key
+
+### 1. Clone the repo
+```bash
+git clone https://github.com/YOUR_USERNAME/agentic-triage-amd.git
+cd agentic-triage-amd
+```
+
+### 2. Set environment variables
+```bash
+cp .env.example .env
+# Edit .env — add your AMD_API_KEY
+```
+
+### 3. Run with Docker
+```bash
+docker build -t agentic-triage-amd .
+docker run -p 7860:7860 --env-file .env agentic-triage-amd
+```
+
+### 4. Run locally
+```bash
+pip install -r requirements.txt
+# Terminal 1 — start the environment
+uvicorn server.app:app --host 0.0.0.0 --port 7860
+# Terminal 2 — run the agent
+python run_agent.py
+```
+
+---
+
+## 🧪 Scoring System
+
+Each task is scored 0.0 to 1.0:
+
+| Action | Points |
+|---|---|
+| Correct severity classification | +0.30 |
+| Correct root cause identification | +0.35 |
+| Correct remediation command | +0.25 |
+| Speed bonus (within step threshold) | +0.10 |
+| Wrong escalation | -0.10 |
+| Ignoring a P1 incident | -0.50 |
+
+**Task 1:** severity=P1, root_cause=payment-service, remediation=restart:payment-service
+**Task 2:** severity=P1, root_cause=user-db, remediation=kill-query:user-db
+**Task 3:** severity=P2, root_cause=payment-db, remediation=flush-cache:payment-db
+
+---
+
+## 📊 Results
+
+| Task | Score |
+|---|---|
+| single_crash | — |
+| cascading_failure | — |
+| silent_degradation | — |
+| **Average** | **—** |
+
+*(Updated after final runs)*
+
+---
+
+## 🔑 Environment Variables
+
+```env
+AMD_API_KEY=your_amd_developer_cloud_api_key
+AMD_BASE_URL=https://api.amd.com/v1
+AMD_MODEL=mistral-7b-instruct
+ENV_HOST=0.0.0.0
+ENV_PORT=7860
+```
+
+---
+
+## ✅ Hackathon Checklist
+
+- [ ] AMD Developer Cloud account activated
+- [ ] LLM swap: Groq → AMD hosted model working
+- [ ] Planner agent implemented
+- [ ] Executor agent implemented
+- [ ] Summarizer agent implemented
+- [ ] LangGraph pipeline connecting all three agents
+- [ ] All 3 tasks running end-to-end
+- [ ] Scores recorded and documented
+- [ ] Demo video recorded
+- [ ] Devpost submission written
+- [ ] Repo open-sourced and clean
+
+---
+
+## 🙋 Team
+
+| Name | Role |
+|---|---|
+| Rohit Patil (Sonic) | Environment + Agent Pipeline |
+| [Teammate] | [Role TBD] |
+
+---
+
+## 📄 License
+
+MIT License — open source, built for AMD Developer Cloud Hackathon.
+
+---
+
+> **Note:** This project builds on prior research work in agentic log triage environments, redesigned and upgraded with a multi-agent architecture specifically for AMD Developer Cloud infrastructure.

requirements.txt

@@ -0,0 +1,18 @@
+# Environment
+fastapi>=0.104.0
+uvicorn>=0.24.0
+pydantic>=2.0.0
+openenv-core==0.2.3
+
+# HTTP
+requests>=2.25.0
+httpx>=0.24.0
+
+# LLM + Agents
+openai>=1.0.0
+langgraph>=0.1.0
+langchain>=0.2.0
+langchain-openai>=0.1.0
+
+# Utilities
+python-dotenv>=1.0.0

server/app.py

@@ -0,0 +1,170 @@
+from fastapi import FastAPI, Query
+from fastapi.responses import JSONResponse
+import uvicorn
+import os
+
+from server.models import TriageAction
+from server.environment import LogTriageEnvironment
+
+app = FastAPI(
+    title="LogTriageEnv",
+    description="OpenEnv environment for SRE incident triage",
+    version="1.0.0",
+)
+
+# One environment instance per server process
+env = LogTriageEnvironment()
+
+
+@app.get("/health")
+def health():
+    return {"status": "ok", "environment": "logtriage-env", "version": "1.0.0"}
+
+
+@app.post("/reset")
+def reset(
+    task: str = Query(default="single_crash", description="Task ID to run"),
+    seed: int = Query(default=None, description="Random seed for reproducibility"),
+):
+    try:
+        obs = env.reset(task_id=task, seed=seed)
+        return obs.model_dump()
+    except ValueError as e:
+        return JSONResponse(status_code=400, content={"error": str(e)})
+
+
+@app.post("/step")
+def step(action: TriageAction):
+    valid, err = action.is_valid()
+    if not valid:
+        return JSONResponse(status_code=422, content={"error": err})
+    try:
+        obs = env.step(action)
+        return obs.model_dump()
+    except RuntimeError as e:
+        return JSONResponse(status_code=400, content={"error": str(e)})
+
+
+@app.get("/state")
+def state():
+    try:
+        return env.state.model_dump()
+    except RuntimeError as e:
+        return JSONResponse(status_code=400, content={"error": str(e)})
+
+
+@app.get("/tasks")
+def get_tasks():
+    return {
+        "tasks": [
+            {
+                "id": "single_crash",
+                "name": "Single Service Crash",
+                "difficulty": "easy",
+                "max_steps": 8,
+                "description": "One service crashes. Classify severity, find root cause, remediate.",
+                "action_schema": {
+                    "action_type": "classify_severity | identify_root_cause | escalate | remediate | request_more_logs | resolve | ignore",
+                    "value": "string (depends on action_type — see README)",
+                    "confidence": "float [0.0, 1.0]",
+                    "reasoning": "string (optional)",
+                },
+            },
+            {
+                "id": "cascading_failure",
+                "name": "Cascading Failure",
+                "difficulty": "medium",
+                "max_steps": 12,
+                "description": "DB slowdown cascades upstream. Find the true root cause, not symptoms.",
+                "action_schema": {
+                    "action_type": "classify_severity | identify_root_cause | escalate | remediate | request_more_logs | resolve | ignore",
+                    "value": "string (depends on action_type — see README)",
+                    "confidence": "float [0.0, 1.0]",
+                    "reasoning": "string (optional)",
+                },
+            },
+            {
+                "id": "silent_degradation",
+                "name": "Silent Degradation with Noise",
+                "difficulty": "hard",
+                "max_steps": 15,
+                "description": "Slow degradation hidden in 60% noise. Nuanced P2 severity judgment.",
+                "action_schema": {
+                    "action_type": "classify_severity | identify_root_cause | escalate | remediate | request_more_logs | resolve | ignore",
+                    "value": "string (depends on action_type — see README)",
+                    "confidence": "float [0.0, 1.0]",
+                    "reasoning": "string (optional)",
+                },
+            },
+        ]
+    }
+
+
+@app.post("/grader")
+def grader():
+    try:
+        from server.graders import score_episode
+        state = env.state
+        result = score_episode(state.task_id, state)
+        return result
+    except RuntimeError as e:
+        return JSONResponse(status_code=400, content={"error": str(e)})
+    except ValueError as e:
+        return JSONResponse(status_code=400, content={"error": str(e)})
+
+
+@app.post("/baseline")
+def baseline():
+    """
+    Run the baseline inference script against all 3 tasks.
+    Returns scores for each task produced by the LLM agent.
+    Note: Requires HF_TOKEN (or GROQ_API_KEY) to be set.
+    """
+    import subprocess
+    import sys
+    import json as json_lib
+
+    try:
+        result = subprocess.run(
+            [sys.executable, "inference.py"],
+            capture_output=True,
+            text=True,
+            timeout=1200,  # 20 minute timeout (matches spec)
+            cwd=os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
+        )
+
+        if result.returncode != 0:
+            return JSONResponse(
+                status_code=500,
+                content={
+                    "error": "Inference script failed",
+                    "stderr": result.stderr[-500:] if result.stderr else "",
+                }
+            )
+
+        # Extract JSON from output
+        output_lines = result.stdout.strip().split("\n")
+        json_start = None
+        for i, line in enumerate(output_lines):
+            if line.strip() == "JSON Output:":
+                json_start = i + 1
+                break
+
+        if json_start and json_start < len(output_lines):
+            json_str = "\n".join(output_lines[json_start:])
+            return json_lib.loads(json_str)
+        else:
+            return {"message": "Baseline completed", "output": result.stdout[-1000:]}
+
+    except subprocess.TimeoutExpired:
+        return JSONResponse(status_code=504, content={"error": "Inference timed out (20min limit)"})
+    except Exception as e:
+        return JSONResponse(status_code=500, content={"error": str(e)})
+
+
+def main():
+    uvicorn.run("server.app:app", host="0.0.0.0", port=7860, reload=False)
+
+
+if __name__ == "__main__":
+    main()

server/environment.py

@@ -0,0 +1,341 @@
+"""
+Core LogTriageEnvironment class.
+Implements OpenEnv interface: reset(), step(), state property.
+"""
+from __future__ import annotations
+import random
+from datetime import datetime
+from uuid import uuid4
+
+from server.models import (
+    TriageAction,
+    TriageObservation,
+    EpisodeState,
+    LogLine,
+    ServiceStatus,
+)
+from server.scenarios import single_crash
+from server.scenarios import cascading
+from server.scenarios import silent_degrade
+from server.log_generator import generate_healthy_system_state, _make_timestamp
+
+# ─── TASK REGISTRY ─────────────────────────────────────────────────────────────
+
+TASK_MAX_STEPS = {
+    "single_crash":      8,
+    "cascading_failure": 12,
+    "silent_degradation": 15,
+}
+
+# ─── REWARD CONSTANTS ──────────────────────────────────────────────────────────
+
+R_CORRECT_SEVERITY     =  0.30
+R_CORRECT_ROOT_CAUSE   =  0.35
+R_CORRECT_REMEDIATION  =  0.25
+R_CORRECT_ESCALATION   =  0.10
+R_SPEED_BONUS          =  0.10
+R_PARTIAL_SERVICE_FAM  =  0.10
+R_PARTIAL_SEVERITY_ADJ =  0.10
+
+P_WRONG_ESCALATION     = -0.10
+P_IGNORE_P1            = -0.50
+P_REDUNDANT_ACTION     = -0.05
+P_EXCEEDED_BUDGET      = -0.20
+P_OVERESCALATE_P3_P1   = -0.15
+
+
+class LogTriageEnvironment:
+    """
+    OpenEnv-compatible environment for SRE incident triage.
+
+    Usage:
+        env = LogTriageEnvironment()
+        obs = env.reset(task_id="single_crash", seed=42)
+        while not obs.done:
+            action = agent.act(obs)
+            obs = env.step(action)
+        score = env.get_grader_score()
+    """
+
+    def __init__(self):
+        self._state: EpisodeState | None = None
+        self._rng: random.Random = random.Random()
+        self._base_time: datetime = datetime.utcnow()
+        self._task_id: str = "single_crash"
+        self._ground_truth: dict = {}
+        self._current_obs: TriageObservation | None = None
+
+    # ─── OPENENV INTERFACE ─────────────────────────────────────────────────────
+
+    def reset(self, task_id: str = "single_crash", seed: int | None = None) -> TriageObservation:
+        """Start a fresh episode. Returns initial observation."""
+        if task_id not in TASK_MAX_STEPS:
+            raise ValueError(f"Unknown task_id '{task_id}'. Valid: {list(TASK_MAX_STEPS.keys())}")
+
+        self._task_id = task_id
+        self._rng = random.Random(seed)
+        self._base_time = datetime.utcnow()
+
+        # Load ground truth for this task
+        if task_id == "single_crash":
+            self._ground_truth = single_crash.GROUND_TRUTH
+        elif task_id == "cascading_failure":
+            self._ground_truth = cascading.GROUND_TRUTH
+        elif task_id == "silent_degradation":
+            self._ground_truth = silent_degrade.GROUND_TRUTH
+
+        # Initialize episode state
+        self._state = EpisodeState(
+            episode_id=str(uuid4()),
+            task_id=task_id,
+            step_count=0,
+            max_steps=TASK_MAX_STEPS[task_id],
+            done=False,
+            cumulative_score=0.0,
+            actions_taken=[],
+            correct_severity=None,
+            correct_root_cause=None,
+            correct_remediation=False,
+        )
+
+        # Get initial observation (step 0)
+        logs, system_state = self._get_step_data(0)
+        alerts = self._get_alerts(0)
+
+        obs = TriageObservation(
+            logs=logs,
+            system_state=system_state,
+            incident_id=self._state.episode_id,
+            task_id=task_id,
+            step_count=0,
+            time_elapsed_seconds=0,
+            active_alerts=alerts,
+            reward=0.0,
+            cumulative_score=0.0,
+            done=False,
+            last_action_feedback="Incident detected. Analyze the logs and take action.",
+            invalid_action_error=None,
+        )
+        self._current_obs = obs
+        return obs
+
+    def step(self, action: TriageAction) -> TriageObservation:
+        """Take one action. Returns next observation + reward."""
+        if self._state is None:
+            raise RuntimeError("Call reset() before step()")
+        if self._state.done:
+            raise RuntimeError("Episode is done. Call reset() to start a new episode.")
+
+        # Validate action
+        valid, err = action.is_valid()
+        if not valid:
+            return self._make_obs(
+                reward=0.0,
+                feedback=f"Invalid action: {err}",
+                invalid_action_error=err,
+                advance_step=False,
+            )
+
+        # Calculate reward for this action
+        reward, feedback = self._evaluate_action(action)
+
+        # Update state
+        self._state.cumulative_score = round(
+            self._state.cumulative_score + reward, 4
+        )
+        self._state.actions_taken.append(action.action_type)
+        self._state.action_history.append(action.model_dump())
+        self._state.step_count += 1
+
+        # Check if episode should end
+        done = self._check_done(action)
+        self._state.done = done
+
+        # If done due to budget exceeded, apply penalty
+        if self._state.step_count >= self._state.max_steps and not done:
+            self._state.cumulative_score = round(
+                self._state.cumulative_score + P_EXCEEDED_BUDGET, 4
+            )
+            self._state.done = True
+            feedback += f" Step budget exceeded ({self._state.max_steps} steps). Penalty applied."
+
+        return self._make_obs(reward=reward, feedback=feedback, advance_step=True)
+
+    @property
+    def state(self) -> EpisodeState:
+        """Return current episode state."""
+        if self._state is None:
+            raise RuntimeError("Call reset() first.")
+        return self._state
+
+    def get_grader_score(self) -> float:
+        """
+        Return final grader score for the completed episode.
+        Score is normalized to [0.0, 1.0].
+        """
+        if self._state is None:
+            return 0.0
+        # Clamp score to [0.0, 1.0]
+        raw = self._state.cumulative_score
+        return round(max(0.0, min(1.0, raw)), 4)
+
+    # ─── INTERNAL HELPERS ──────────────────────────────────────────────────────
+
+    def _evaluate_action(self, action: TriageAction) -> tuple[float, str]:
+        """
+        Evaluate the action against ground truth.
+        Returns (reward: float, feedback: str).
+        """
+        gt = self._ground_truth
+        reward = 0.0
+        feedback_parts = []
+
+        # Penalize redundant actions
+        if action.action_type in self._state.actions_taken:
+            reward += P_REDUNDANT_ACTION
+            feedback_parts.append("Redundant action — you've already done this.")
+
+        # ── classify_severity ──────────────────────────────────────────────────
+        if action.action_type == "classify_severity":
+            correct_sev = gt.get("severity", "")
+            if action.value == correct_sev:
+                if self._state.correct_severity is None:  # only reward first time
+                    reward += R_CORRECT_SEVERITY
+                    feedback_parts.append(f"Correct severity: {action.value}. +{R_CORRECT_SEVERITY}")
+                    self._state.correct_severity = action.value
+            else:
+                # Partial credit: P1 vs P2 is close, P1 vs P3 is not
+                if correct_sev == "P1" and action.value == "P3":
+                    reward += P_OVERESCALATE_P3_P1  # wrong direction
+                    feedback_parts.append(f"Incorrect severity: {action.value}. P1 expected. This is a customer-impacting incident.")
+                elif correct_sev == "P1" and action.value == "P2":
+                    reward += R_PARTIAL_SEVERITY_ADJ
+                    feedback_parts.append(f"Close — {action.value} given, P1 expected. Partial credit.")
+                else:
+                    feedback_parts.append(f"Incorrect severity: {action.value}. Reassess.")
+
+        # ── identify_root_cause ────────────────────────────────────────────────
+        elif action.action_type == "identify_root_cause":
+            correct_rc = gt.get("root_cause", "")
+            if action.value == correct_rc:
+                if self._state.correct_root_cause is None:
+                    reward += R_CORRECT_ROOT_CAUSE
+                    feedback_parts.append(f"Correct root cause: {action.value}. +{R_CORRECT_ROOT_CAUSE}")
+                    self._state.correct_root_cause = action.value
+            else:
+                # Partial credit: same tier (e.g. payment-db instead of payment-service)
+                if correct_rc.split("-")[0] == action.value.split("-")[0]:
+                    reward += R_PARTIAL_SERVICE_FAM
+                    feedback_parts.append(f"Close — {action.value} is in the right service family. Check more carefully.")
+                else:
+                    feedback_parts.append(f"Incorrect root cause: {action.value}. Look at which service is actually failing.")
+
+        # ── escalate ──────────────────────────────────────────────────────────
+        elif action.action_type == "escalate":
+            correct_teams = gt.get("correct_teams", set())
+            if action.value in correct_teams:
+                reward += R_CORRECT_ESCALATION
+                feedback_parts.append(f"Correct escalation to {action.value}. +{R_CORRECT_ESCALATION}")
+            else:
+                reward += P_WRONG_ESCALATION
+                feedback_parts.append(f"Wrong team escalated: {action.value}. Penalty applied.")
+
+        # ── remediate ─────────────────────────────────────────────────────────
+        elif action.action_type == "remediate":
+            prefix = action.value.split(":")[0]
+            service = action.value.split(":")[1] if ":" in action.value else ""
+            correct_prefixes = gt.get("remediation_prefixes", set())
+            correct_service = gt.get("remediation_service", "")
+
+            if prefix in correct_prefixes and service == correct_service:
+                if not self._state.correct_remediation:
+                    reward += R_CORRECT_REMEDIATION
+                    feedback_parts.append(f"Correct remediation: {action.value}. +{R_CORRECT_REMEDIATION}")
+                    self._state.correct_remediation = True
+            elif service == correct_service and prefix not in correct_prefixes:
+                reward += 0.05  # right service, wrong action
+                feedback_parts.append(f"Right service, but '{prefix}' may not fix this. Try another remediation type.")
+            else:
+                feedback_parts.append(f"Incorrect remediation: {action.value}. Reconsider which service needs fixing.")
+
+        # ── ignore ────────────────────────────────────────────────────────────
+        elif action.action_type == "ignore":
+            correct_sev = gt.get("severity", "")
+            if correct_sev == "P1":
+                reward += P_IGNORE_P1
+                feedback_parts.append(f"CRITICAL ERROR: Ignored a P1 incident! Major penalty applied.")
+            else:
+                feedback_parts.append("Marked as noise.")
+
+        # ── request_more_logs ─────────────────────────────────────────────────
+        elif action.action_type == "request_more_logs":
+            feedback_parts.append(f"Fetching more logs for {action.value}...")
+
+        # ── resolve ───────────────────────────────────────────────────────────
+        elif action.action_type == "resolve":
+            # Speed bonus if resolved within 60% of step budget
+            step_budget = self._state.max_steps
+            if self._state.step_count <= int(step_budget * 0.6):
+                reward += R_SPEED_BONUS
+                feedback_parts.append(f"Incident resolved efficiently. Speed bonus: +{R_SPEED_BONUS}")
+            else:
+                feedback_parts.append("Incident resolved.")
+
+        return round(reward, 4), " | ".join(feedback_parts) or "Action processed."
+
+    def _check_done(self, action: TriageAction) -> bool:
+        """Episode ends on resolve, ignore (with P1), or step budget exhausted."""
+        if action.action_type == "resolve":
+            return True
+        if action.action_type == "ignore" and self._ground_truth.get("severity") == "P1":
+            return True  # Catastrophic — episode ends immediately
+        if self._state.step_count >= self._state.max_steps:
+            return True
+        return False
+
+    def _get_step_data(self, step: int):
+        """Get logs and system state for the current step."""
+        if self._task_id == "single_crash":
+            return single_crash.get_step_data(step, self._base_time, self._rng)
+        elif self._task_id == "cascading_failure":
+            return cascading.get_step_data(step, self._base_time, self._rng)
+        elif self._task_id == "silent_degradation":
+            return silent_degrade.get_step_data(step, self._base_time, self._rng)
+        return [], generate_healthy_system_state(self._base_time)
+
+    def _get_alerts(self, step: int) -> list[str]:
+        """Get active alerts for the current step."""
+        if self._task_id == "single_crash":
+            return single_crash.get_active_alerts(step)
+        elif self._task_id == "cascading_failure":
+            return cascading.get_active_alerts(step)
+        elif self._task_id == "silent_degradation":
+            return silent_degrade.get_active_alerts(step)
+        return []
+
+    def _make_obs(
+        self,
+        reward: float,
+        feedback: str,
+        invalid_action_error: str | None = None,
+        advance_step: bool = True,
+    ) -> TriageObservation:
+        """Build a TriageObservation for the current state."""
+        step = self._state.step_count
+        logs, system_state = self._get_step_data(step)
+        alerts = self._get_alerts(step)
+
+        return TriageObservation(
+            logs=logs,
+            system_state=system_state,
+            incident_id=self._state.episode_id,
+            task_id=self._state.task_id,
+            step_count=step,
+            time_elapsed_seconds=step * 30,
+            active_alerts=alerts,
+            reward=reward,
+            cumulative_score=self._state.cumulative_score,
+            done=self._state.done,
+            last_action_feedback=feedback,
+            invalid_action_error=invalid_action_error,
+        )

server/graders/base_grader.py

@@ -0,0 +1,68 @@
+"""
+Abstract base grader interface.
+All task graders must inherit from this and implement score().
+"""
+from __future__ import annotations
+from abc import ABC, abstractmethod
+from server.models import EpisodeState
+
+
+class BaseGrader(ABC):
+    """
+    Abstract grader base class.
+
+    A grader evaluates the complete episode history and produces
+    a final score in [0.0, 1.0].
+
+    Unlike the reward function (which fires after every step),
+    the grader fires once at episode end and produces the
+    official score used by judges.
+    """
+
+    @abstractmethod
+    def score(self, state: EpisodeState) -> float:
+        """
+        Score the completed episode.
+
+        Args:
+            state: Final EpisodeState including full action_history
+
+        Returns:
+            float in [0.0, 1.0] — the official episode score
+        """
+        raise NotImplementedError
+
+    def _clamp(self, value: float) -> float:
+        """Clamp score to valid range (0.0, 1.0) — strictly between 0 and 1."""
+        return round(max(0.0001, min(0.9999, value)), 4)
+
+    def _get_actions_of_type(
+        self, state: EpisodeState, action_type: str
+    ) -> list[dict]:
+        """Return all actions of a given type from episode history."""
+        return [
+            a for a in state.action_history
+            if a.get("action_type") == action_type
+        ]
+
+    def _was_action_taken(self, state: EpisodeState, action_type: str) -> bool:
+        """Check if an action type was taken at any point in the episode."""
+        return any(
+            a.get("action_type") == action_type
+            for a in state.action_history
+        )
+
+    def _get_first_value(
+        self, state: EpisodeState, action_type: str
+    ) -> str | None:
+        """Get the value of the first action of a given type."""
+        actions = self._get_actions_of_type(state, action_type)
+        return actions[0].get("value") if actions else None
+
+    def _episode_resolved(self, state: EpisodeState) -> bool:
+        """Check if agent explicitly resolved the episode."""
+        return self._was_action_taken(state, "resolve")
+
+    def _steps_used(self, state: EpisodeState) -> int:
+        """Return number of steps taken."""
+        return state.step_count

server/graders/cascade_grader.py

@@ -0,0 +1,123 @@
+"""
+Grader for Task 2 — Cascading Failure (Medium)
+
+Scoring breakdown:
+  Correct severity (P1)                    → +0.20
+  Correct root cause (user-db)             → +0.35
+  Correct remediation (kill-query/restart) → +0.25
+  Ordering bonus (no symptom fix first)    → +0.10
+  Speed bonus (resolved ≤ 8 steps)         → +0.10
+  ─────────────────────────────────────────────────
+  Maximum possible score                   →  1.00
+
+Penalties:
+  Identified symptom as root cause         →  0.00 (no credit)
+  Remediated symptom service first         → -0.10 (ordering penalty)
+  Never resolved                           → -0.10
+"""
+from __future__ import annotations
+from server.models import EpisodeState
+from server.graders.base_grader import BaseGrader
+
+
+class CascadeGrader(BaseGrader):
+    """Official grader for Task 2 — Cascading Failure."""
+
+    CORRECT_SEVERITY = "P1"
+    CORRECT_ROOT_CAUSE = "user-db"
+    CORRECT_REMEDIATION_PREFIXES = {"kill-query", "restart"}
+    CORRECT_REMEDIATION_SERVICE = "user-db"
+    SYMPTOM_SERVICES = {"api-gateway", "auth-service"}  # wrong answers
+    MAX_STEPS = 12
+    SPEED_THRESHOLD = 8
+
+    def score(self, state: EpisodeState) -> float:
+        """
+        Score the completed Task 2 episode.
+        Penalizes agents that treat symptoms instead of root cause.
+        """
+        total = 0.0
+        breakdown = {}
+
+        # ── 1. Severity classification ─────────────────────────────────────────
+        severity_value = self._get_first_value(state, "classify_severity")
+        if severity_value == self.CORRECT_SEVERITY:
+            total += 0.20
+            breakdown["severity"] = "+0.20 (correct: P1)"
+        elif severity_value == "P2":
+            total += 0.08
+            breakdown["severity"] = "+0.08 (partial: P2 given, P1 expected)"
+        elif severity_value is None:
+            breakdown["severity"] = "+0.00 (never classified)"
+        else:
+            breakdown["severity"] = f"+0.00 (wrong: {severity_value})"
+
+        # ── 2. Root cause identification ───────────────────────────────────────
+        root_cause_value = self._get_first_value(state, "identify_root_cause")
+        if root_cause_value == self.CORRECT_ROOT_CAUSE:
+            total += 0.35
+            breakdown["root_cause"] = "+0.35 (correct: user-db)"
+        elif root_cause_value in self.SYMPTOM_SERVICES:
+            # Identified a symptom, not root cause — no credit
+            breakdown["root_cause"] = f"+0.00 (wrong: {root_cause_value} is a symptom, not root cause)"
+        elif root_cause_value and "db" in root_cause_value:
+            total += 0.10  # right tier (database), wrong specific service
+            breakdown["root_cause"] = f"+0.10 (partial: {root_cause_value}, right tier)"
+        elif root_cause_value is None:
+            breakdown["root_cause"] = "+0.00 (never identified)"
+        else:
+            breakdown["root_cause"] = f"+0.00 (wrong: {root_cause_value})"
+
+        # ── 3. Remediation + Ordering ──────────────────────────────────────────
+        remediation_actions = self._get_actions_of_type(state, "remediate")
+        remediation_scored = False
+        symptom_remediated_first = False
+
+        for i, action in enumerate(remediation_actions):
+            value = action.get("value", "")
+            parts = value.split(":")
+            if len(parts) != 2:
+                continue
+            prefix, service = parts
+
+            # Check if agent remediated a symptom service before root cause
+            if service in self.SYMPTOM_SERVICES and not remediation_scored:
+                symptom_remediated_first = True
+
+            # Check for correct remediation
+            if (
+                prefix in self.CORRECT_REMEDIATION_PREFIXES
+                and service == self.CORRECT_REMEDIATION_SERVICE
+                and not remediation_scored
+            ):
+                total += 0.25
+                breakdown["remediation"] = f"+0.25 (correct: {value})"
+                remediation_scored = True
+
+        if not remediation_scored:
+            breakdown["remediation"] = "+0.00 (no correct remediation)"
+
+        # ── 4. Ordering bonus ──────────────────────────────────────────────────
+        if not symptom_remediated_first and remediation_scored:
+            total += 0.10
+            breakdown["ordering"] = "+0.10 (correctly targeted root cause, not symptoms)"
+        elif symptom_remediated_first:
+            total -= 0.10
+            breakdown["ordering"] = "-0.10 (remediated symptom service before root cause)"
+
+        # ── 5. Speed bonus ─────────────────────────────────────────────────────
+        if self._episode_resolved(state):
+            if self._steps_used(state) <= self.SPEED_THRESHOLD:
+                total += 0.10
+                breakdown["speed"] = f"+0.10 (resolved in {self._steps_used(state)} steps)"
+            else:
+                breakdown["speed"] = f"+0.00 (resolved but used {self._steps_used(state)} steps)"
+        else:
+            total -= 0.10
+            breakdown["resolution"] = "-0.10 (never resolved)"
+
+        self._breakdown = breakdown
+        return self._clamp(total)
+
+    def get_breakdown(self) -> dict:
+        return getattr(self, "_breakdown", {})

server/graders/crash_grader.py

@@ -0,0 +1,110 @@
+"""
+Grader for Task 1 — Single Service Crash (Easy)
+
+Scoring breakdown:
+  Correct severity (P1)                    → +0.30
+  Correct root cause (payment-service)     → +0.35
+  Correct remediation (restart:payment-*) → +0.25
+  Speed bonus (resolved ≤ 5 steps)         → +0.10
+  ─────────────────────────────────────────────────
+  Maximum possible score                   →  1.00
+
+Penalties:
+  Ignored P1 incident                      → -0.30 (from base)
+  Wrong root cause identified              →  0.00 (no credit)
+  Never resolved                           → -0.10
+"""
+from __future__ import annotations
+from server.models import EpisodeState
+from server.graders.base_grader import BaseGrader
+
+
+class CrashGrader(BaseGrader):
+    """Official grader for Task 1 — Single Service Crash."""
+
+    # Ground truth constants
+    CORRECT_SEVERITY = "P1"
+    CORRECT_ROOT_CAUSE = "payment-service"
+    CORRECT_REMEDIATION_PREFIX = "restart"
+    CORRECT_REMEDIATION_SERVICE = "payment-service"
+    MAX_STEPS = 8
+    SPEED_THRESHOLD = 5  # must resolve within this many steps for speed bonus
+
+    def score(self, state: EpisodeState) -> float:
+        """
+        Score the completed Task 1 episode.
+        Deterministic — same action history always produces same score.
+        """
+        total = 0.0
+        breakdown = {}
+
+        # ── 1. Severity classification ─────────────────────────────────────────
+        severity_value = self._get_first_value(state, "classify_severity")
+        if severity_value == self.CORRECT_SEVERITY:
+            total += 0.30
+            breakdown["severity"] = "+0.30 (correct: P1)"
+        elif severity_value == "P2":
+            total += 0.10  # partial credit — close but not right
+            breakdown["severity"] = "+0.10 (partial: P2 given, P1 expected)"
+        elif severity_value is None:
+            breakdown["severity"] = "+0.00 (never classified)"
+        else:
+            breakdown["severity"] = f"+0.00 (wrong: {severity_value})"
+
+        # ── 2. Root cause identification ───────────────────────────────────────
+        root_cause_value = self._get_first_value(state, "identify_root_cause")
+        if root_cause_value == self.CORRECT_ROOT_CAUSE:
+            total += 0.35
+            breakdown["root_cause"] = "+0.35 (correct: payment-service)"
+        elif root_cause_value and root_cause_value.startswith("payment"):
+            total += 0.10  # partial — right service family
+            breakdown["root_cause"] = f"+0.10 (partial: {root_cause_value}, right family)"
+        elif root_cause_value is None:
+            breakdown["root_cause"] = "+0.00 (never identified)"
+        else:
+            breakdown["root_cause"] = f"+0.00 (wrong: {root_cause_value})"
+
+        # ── 3. Remediation ─────────────────────────────────────────────────────
+        remediation_actions = self._get_actions_of_type(state, "remediate")
+        remediation_scored = False
+        for action in remediation_actions:
+            value = action.get("value", "")
+            parts = value.split(":")
+            if len(parts) == 2:
+                prefix, service = parts
+                if prefix == self.CORRECT_REMEDIATION_PREFIX and service == self.CORRECT_REMEDIATION_SERVICE:
+                    total += 0.25
+                    breakdown["remediation"] = f"+0.25 (correct: {value})"
+                    remediation_scored = True
+                    break
+                elif service == self.CORRECT_REMEDIATION_SERVICE:
+                    total += 0.08  # right service, wrong action type
+                    breakdown["remediation"] = f"+0.08 (partial: right service, wrong action)"
+                    remediation_scored = True
+                    break
+
+        if not remediation_scored:
+            breakdown["remediation"] = "+0.00 (no correct remediation)"
+
+        # ── 4. Speed bonus ─────────────────────────────────────────────────────
+        if self._episode_resolved(state):
+            if self._steps_used(state) <= self.SPEED_THRESHOLD:
+                total += 0.10
+                breakdown["speed"] = f"+0.10 (resolved in {self._steps_used(state)} steps)"
+            else:
+                breakdown["speed"] = f"+0.00 (resolved but slow: {self._steps_used(state)} steps)"
+        else:
+            total -= 0.10  # penalty for not resolving
+            breakdown["resolution"] = "-0.10 (never resolved)"
+
+        # ── 5. Ignore penalty ─────────────────────────────��────────────────────
+        if self._was_action_taken(state, "ignore"):
+            total -= 0.30
+            breakdown["ignore_penalty"] = "-0.30 (ignored P1 incident)"
+
+        self._breakdown = breakdown
+        return self._clamp(total)
+
+    def get_breakdown(self) -> dict:
+        """Return scoring breakdown from last score() call."""
+        return getattr(self, "_breakdown", {})

server/log_generator.py

@@ -0,0 +1,188 @@
+"""
+Log generator for LogTriageEnv.
+Produces realistic-looking log lines for the simulated microservice cluster.
+"""
+from __future__ import annotations
+import random
+from datetime import datetime, timedelta
+from server.models import LogLine, ServiceStatus
+
+# ─── SERVICES ─────────────────────────────────────────────────────────────────
+
+SERVICES = [
+    "api-gateway",
+    "auth-service",
+    "user-db",
+    "payment-service",
+    "payment-db",
+    "notification-service",
+    "email-queue",
+]
+
+# ─── LOG TEMPLATES ────────────────────────────────────────────────────────────
+
+# Noise logs — realistic but irrelevant to the incident
+NOISE_TEMPLATES = {
+    "api-gateway": [
+        ("INFO",  "health check passed — all upstream services reachable"),
+        ("INFO",  "request completed: GET /api/v1/users/profile [200] 45ms"),
+        ("INFO",  "rate limiter: 1240/5000 requests this minute"),
+        ("DEBUG", "connection pool: 12/100 active connections"),
+        ("INFO",  "TLS certificate valid for 87 more days"),
+    ],
+    "auth-service": [
+        ("INFO",  "JWT token issued for user_id=88142 [expires: 3600s]"),
+        ("INFO",  "OAuth2 flow completed successfully"),
+        ("DEBUG", "session cache hit ratio: 94.2%"),
+        ("INFO",  "password reset email queued for user_id=23019"),
+    ],
+    "user-db": [
+        ("INFO",  "daily vacuum completed: 0 dead tuples removed"),
+        ("INFO",  "checkpoint complete: wrote 142 buffers"),
+        ("DEBUG", "autovacuum: processing table 'sessions'"),
+        ("INFO",  "replication lag: 12ms (within threshold)"),
+    ],
+    "payment-service": [
+        ("INFO",  "payment processed: txn_id=TXN-8812 amount=299.00 INR [success]"),
+        ("INFO",  "webhook delivered: stripe event=payment.succeeded"),
+        ("DEBUG", "idempotency key cache: 2341 keys active"),
+    ],
+    "payment-db": [
+        ("INFO",  "connection pool: 8/50 active"),
+        ("DEBUG", "query plan cache: 88% hit ratio"),
+        ("INFO",  "index usage: 99.1% queries using indexed scans"),
+    ],
+    "notification-service": [
+        ("INFO",  "email dispatched: template=welcome_email to=user@example.com"),
+        ("INFO",  "SMS delivered: +91XXXXXXXXXX [provider=twilio]"),
+        ("WARN",  "email bounce rate: 1.2% (threshold: 5%)"),
+        ("INFO",  "push notification sent: device_tokens=1240"),
+    ],
+    "email-queue": [
+        ("INFO",  "queue depth: 42 messages pending"),
+        ("INFO",  "consumer lag: 0.3s (healthy)"),
+        ("DEBUG", "partition rebalance completed in 120ms"),
+    ],
+}
+
+# Signal logs — actual incident indicators
+SIGNAL_TEMPLATES = {
+    # Single service crash signals (Task 1 — payment-service crash)
+    "single_crash_payment": [
+        ("ERROR", "NullPointerException: Cannot invoke method processPayment() on null object — PaymentProcessor.java:142"),
+        ("ERROR", "HTTP 500 Internal Server Error: payment gateway returned null response"),
+        ("ERROR", "NullPointerException in PaymentService.execute() — retrying (attempt 1/3)"),
+        ("ERROR", "NullPointerException in PaymentService.execute() — retrying (attempt 2/3)"),
+        ("FATAL", "NullPointerException in PaymentService.execute() — all retries exhausted, request failed"),
+        ("ERROR", "health check FAILED: payment-service returned 500 (was 200)"),
+        ("ERROR", "circuit breaker OPEN: payment-service error rate 98.2% (threshold: 10%)"),
+    ],
+    # Cascading failure signals (Task 2 — user-db → auth-service → api-gateway)
+    "cascading_userdb": [
+        ("WARN",  "slow query detected: SELECT * FROM sessions WHERE user_id=? [latency: 2847ms, threshold: 200ms]"),
+        ("ERROR", "slow query detected: SELECT * FROM sessions WHERE user_id=? [latency: 4120ms]"),
+        ("ERROR", "query timeout: SELECT * FROM active_sessions [timeout after 5000ms]"),
+    ],
+    "cascading_auth": [
+        ("WARN",  "db connection pool: 42/50 active connections (84% utilization)"),
+        ("ERROR", "db connection pool exhausted: 50/50 connections in use — requests queuing"),
+        ("ERROR", "authentication request timed out waiting for db connection [5200ms]"),
+    ],
+    "cascading_gateway": [
+        ("ERROR", "upstream timeout: auth-service failed to respond within 5000ms [req-id: {req_id}]"),
+        ("ERROR", "upstream timeout: auth-service [req-id: {req_id}] — returning 504 to client"),
+        ("WARN",  "error rate spike: 34.2% of requests failing (threshold: 5%)"),
+    ],
+    # Silent degradation signals (Task 3 — payment-db slow)
+    "silent_paymentdb": [
+        ("WARN",  "query latency elevated: avg=450ms (normal: 80ms) — monitoring"),
+        ("WARN",  "query latency elevated: avg=620ms — possible memory pressure"),
+        ("WARN",  "query latency elevated: avg=890ms — recommend investigation"),
+        ("WARN",  "query latency elevated: avg=1200ms — approaching timeout threshold"),
+        ("WARN",  "buffer cache hit ratio degraded: 87% (normal: 98%) — possible memory issue"),
+    ],
+}
+
+
+def _make_timestamp(base_time: datetime, offset_seconds: int = 0) -> str:
+    t = base_time + timedelta(seconds=offset_seconds)
+    return t.strftime("%Y-%m-%dT%H:%M:%SZ")
+
+
+def _noise_log(service: str, base_time: datetime, offset: int) -> LogLine:
+    templates = NOISE_TEMPLATES.get(service, [("INFO", "routine operation completed")])
+    level, message = random.choice(templates)
+    return LogLine(
+        timestamp=_make_timestamp(base_time, offset),
+        level=level,
+        service=service,
+        request_id=None,
+        message=message,
+        latency_ms=None,
+    )
+
+
+def generate_log_batch(
+    scenario_signals: list[tuple[str, str, str]],  # [(service, level, message), ...]
+    step: int,
+    base_time: datetime,
+    noise_ratio: float = 0.3,
+    batch_size: int = 8,
+    rng: random.Random = None,
+) -> list[LogLine]:
+    """
+    Generate a mixed batch of signal + noise log lines.
+
+    Args:
+        scenario_signals: List of (service, level, message) tuples — the actual signals for this step
+        step: Current step number (used for timestamp offset)
+        base_time: Episode start time (used for timestamps)
+        noise_ratio: Fraction of logs that are noise (0.0 = all signal, 1.0 = all noise)
+        batch_size: Total number of log lines to return
+        rng: Optional seeded Random for reproducibility
+
+    Returns:
+        List of LogLine objects, shuffled (signal mixed into noise)
+    """
+    if rng is None:
+        rng = random.Random()
+
+    logs = []
+    base_offset = step * 30  # 30 simulated seconds per step
+
+    # Add signal logs
+    for i, (service, level, message) in enumerate(scenario_signals):
+        req_id = f"req-{rng.randint(1000, 9999)}" if level in ("ERROR", "WARN") else None
+        logs.append(LogLine(
+            timestamp=_make_timestamp(base_time, base_offset + i),
+            level=level,
+            service=service,
+            request_id=req_id,
+            message=message,
+            latency_ms=rng.randint(200, 5000) if "timeout" in message.lower() or "latency" in message.lower() else None,
+        ))
+
+    # Fill remaining slots with noise logs
+    noise_count = max(0, batch_size - len(logs))
+    noise_services = rng.choices(SERVICES, k=noise_count)
+    for i, svc in enumerate(noise_services):
+        logs.append(_noise_log(svc, base_time, base_offset + len(scenario_signals) + i))
+
+    # Shuffle — signal should not always be first
+    rng.shuffle(logs)
+    return logs[:batch_size]
+
+
+def generate_healthy_system_state(base_time: datetime) -> dict[str, ServiceStatus]:
+    """Generate a fully healthy system state snapshot."""
+    now = _make_timestamp(base_time)
+    return {
+        svc: ServiceStatus(
+            name=svc,
+            status="up",
+            error_rate=round(random.uniform(0.001, 0.01), 4),
+            latency_p99_ms=random.randint(20, 80),
+            last_updated=now,
+        )
+        for svc in SERVICES
+    }

server/models.py

@@ -0,0 +1,221 @@
+from __future__ import annotations
+from typing import Literal, Optional, ClassVar
+from pydantic import BaseModel, Field
+
+
+# ─── LOG LINE ─────────────────────────────────────────────────────────────────
+
+class LogLine(BaseModel):
+    """A single log line from the simulated microservice cluster."""
+    timestamp: str = Field(..., description="ISO 8601 timestamp")
+    level: Literal["DEBUG", "INFO", "WARN", "ERROR", "FATAL"]
+    service: str = Field(..., description="Service that emitted the log")
+    request_id: Optional[str] = Field(None, description="Request trace ID if present")
+    message: str = Field(..., description="Log message content")
+    latency_ms: Optional[int] = Field(None, description="Latency if relevant")
+
+
+# ─── SERVICE STATUS ────────────────────────────────────────────────────────────
+
+class ServiceStatus(BaseModel):
+    """Current health snapshot of one microservice."""
+    name: str
+    status: Literal["up", "degraded", "down"]
+    error_rate: float = Field(..., ge=0.0, le=1.0, description="Error rate 0.0-1.0")
+    latency_p99_ms: int = Field(..., description="99th percentile latency in ms")
+    last_updated: str = Field(..., description="ISO 8601 timestamp of last update")
+
+
+# ─── ACTION ───────────────────────────────────────────────────────────────────
+
+class TriageAction(BaseModel):
+    """
+    Action taken by the agent in one step.
+
+    action_type options:
+      - classify_severity  : value must be "P1", "P2", or "P3"
+      - identify_root_cause: value must be a valid service name
+      - escalate           : value must be a valid team name
+      - remediate          : value must be "restart:<svc>", "rollback:<svc>",
+                             "scale:<svc>", "flush-cache:<svc>", "kill-query:<svc>"
+      - request_more_logs  : value must be a service name or "all"
+      - resolve            : value must be "resolved"
+      - ignore             : value must be "noise"
+    """
+    action_type: Literal[
+        "classify_severity",
+        "identify_root_cause",
+        "escalate",
+        "remediate",
+        "request_more_logs",
+        "resolve",
+        "ignore",
+    ] = Field(..., description="Type of triage action to perform")
+
+    value: str = Field(
+        ...,
+        description="Action value — depends on action_type (see docstring)"
+    )
+
+    confidence: float = Field(
+        default=1.0,
+        ge=0.0,
+        le=1.0,
+        description="Agent self-reported confidence in this action (0.0-1.0)"
+    )
+
+    reasoning: str = Field(
+        default="",
+        description="Optional free-text reasoning (used for interpretability)"
+    )
+
+    # ── Valid value constants ──────────────────────────────────────────────────
+    VALID_SEVERITIES: ClassVar = {"P1", "P2", "P3"}
+    VALID_SERVICES: ClassVar = {
+        "api-gateway",
+        "auth-service",
+        "user-db",
+        "payment-service",
+        "payment-db",
+        "notification-service",
+        "email-queue",
+    }
+    VALID_TEAMS: ClassVar = {
+        "sre-team",
+        "backend-team",
+        "dba-team",
+        "security-team",
+    }
+    VALID_REMEDIATION_PREFIXES: ClassVar = {
+        "restart",
+        "rollback",
+        "scale",
+        "flush-cache",
+        "kill-query",
+    }
+
+    def is_valid(self) -> tuple[bool, str]:
+        """
+        Validate the action value against its action_type.
+        Returns (is_valid: bool, error_message: str).
+        """
+        if self.action_type == "classify_severity":
+            if self.value not in self.VALID_SEVERITIES:
+                return False, f"classify_severity value must be one of {self.VALID_SEVERITIES}"
+
+        elif self.action_type == "identify_root_cause":
+            if self.value not in self.VALID_SERVICES:
+                return False, f"identify_root_cause value must be one of {self.VALID_SERVICES}"
+
+        elif self.action_type == "escalate":
+            if self.value not in self.VALID_TEAMS:
+                return False, f"escalate value must be one of {self.VALID_TEAMS}"
+
+        elif self.action_type == "remediate":
+            prefix = self.value.split(":")[0]
+            if prefix not in self.VALID_REMEDIATION_PREFIXES:
+                return False, f"remediate prefix must be one of {self.VALID_REMEDIATION_PREFIXES}"
+            parts = self.value.split(":")
+            if len(parts) != 2 or parts[1] not in self.VALID_SERVICES:
+                return False, f"remediate format must be '<action>:<service>'"
+
+        elif self.action_type == "request_more_logs":
+            if self.value != "all" and self.value not in self.VALID_SERVICES:
+                return False, f"request_more_logs value must be 'all' or a valid service name"
+
+        elif self.action_type == "resolve":
+            if self.value != "resolved":
+                return False, "resolve value must be 'resolved'"
+
+        elif self.action_type == "ignore":
+            if self.value != "noise":
+                return False, "ignore value must be 'noise'"
+
+        return True, ""
+
+
+# ─── OBSERVATION ──────────────────────────────────────────────────────────────
+
+class TriageObservation(BaseModel):
+    """
+    Observation returned to the agent after each step (and after reset).
+    Contains the current log batch, system state, incident metadata,
+    and reward signals.
+    """
+    # Log batch for this step
+    logs: list[LogLine] = Field(
+        ...,
+        description="Current batch of log lines (5-15 lines)"
+    )
+
+    # System state snapshot
+    system_state: dict[str, ServiceStatus] = Field(
+        ...,
+        description="Per-service health snapshot keyed by service name"
+    )
+
+    # Incident metadata
+    incident_id: str = Field(..., description="Unique ID for this episode")
+    task_id: str = Field(..., description="Which task is being run")
+    step_count: int = Field(..., description="Current step number (0-indexed)")
+    time_elapsed_seconds: int = Field(
+        ...,
+        description="Simulated incident time elapsed in seconds"
+    )
+    active_alerts: list[str] = Field(
+        default_factory=list,
+        description="Currently firing alert names"
+    )
+
+    # Reward signals
+    reward: float = Field(
+        default=0.0,
+        description="Reward received for the last action"
+    )
+    cumulative_score: float = Field(
+        default=0.0,
+        description="Running total score for this episode"
+    )
+    done: bool = Field(
+        default=False,
+        description="Whether the episode has ended"
+    )
+
+    # Feedback
+    last_action_feedback: str = Field(
+        default="",
+        description="Natural language feedback on the previous action"
+    )
+    invalid_action_error: Optional[str] = Field(
+        default=None,
+        description="Set if the last action was invalid (wrong format/value)"
+    )
+
+
+# ─── EPISODE STATE ────────────────────────────────────────────────────────────
+
+class EpisodeState(BaseModel):
+    """Internal state of the current episode (returned by state() endpoint)."""
+    episode_id: str
+    task_id: str
+    step_count: int
+    max_steps: int
+    done: bool
+    cumulative_score: float
+    actions_taken: list[str] = Field(
+        default_factory=list,
+        description="List of action_type values taken so far this episode"
+    )
+    action_history: list[dict] = Field(
+        default_factory=list,
+        description="Full action objects taken this episode (for grader evaluation)"
+    )
+    correct_severity: Optional[str] = Field(
+        None,
+        description="Whether agent has correctly classified severity yet"
+    )
+    correct_root_cause: Optional[str] = Field(
+        None,
+        description="Whether agent has correctly identified root cause yet"
+    )
+    correct_remediation: bool = False

server/scenarios/cascading.py

@@ -0,0 +1,211 @@
+"""
+Task 2 — Cascading Failure (Medium)
+
+Scenario: user-db develops a slow query that exhausts the auth-service connection pool,
+which then causes the api-gateway to return timeouts to all users.
+
+Surface logs show gateway errors most loudly (symptom), but root cause is hidden (user-db).
+Agent must trace backward through the cascade chain — NOT treat symptoms as root cause.
+
+Ground truth:
+  - severity: P1
+  - root_cause: user-db
+  - remediation: kill-query:user-db OR restart:user-db
+  - correct_teams: dba-team, sre-team
+  - noise_ratio: 30%
+"""
+from __future__ import annotations
+import random
+from datetime import datetime
+from server.models import LogLine, ServiceStatus
+from server.log_generator import (
+    generate_log_batch,
+    generate_healthy_system_state,
+    _make_timestamp,
+)
+
+# ─── GROUND TRUTH ─────────────────────────────────────────────────────────────
+
+GROUND_TRUTH = {
+    "severity": "P1",
+    "root_cause": "user-db",
+    "remediation_prefixes": {"kill-query", "restart"},
+    "remediation_service": "user-db",
+    "correct_teams": {"dba-team", "sre-team"},
+    "max_steps": 12,
+    "noise_ratio": 0.30,
+}
+
+# ─── STEP-BY-STEP SIGNAL PLAN ─────────────────────────────────────────────────
+# Cascade chain: user-db slow query → auth-service pool exhausted → api-gateway timeouts
+# Steps 0-1: Gateway errors surface (symptoms only — most visible)
+# Steps 2-3: Auth-service DB pressure becomes visible
+# Steps 4-5: user-db slow queries exposed; circuit breaker opens
+# Steps 6-7: Full cascade — all 3 services degraded/down
+# Steps 8-11: Escalating alerts; root cause becomes unmistakable
+
+STEP_SIGNALS = [
+    # Step 0: Gateway errors first to appear (surface symptom)
+    [
+        ("api-gateway", "ERROR", "upstream timeout from auth-service: 5002ms"),
+        ("api-gateway", "WARN",  "error rate: 8.3% on /auth/* routes"),
+    ],
+    # Step 1: More gateway errors; first hints of auth-service pressure
+    [
+        ("api-gateway", "ERROR", "upstream timeout from auth-service: 30007ms"),
+        ("api-gateway", "WARN",  "error rate: 15.7% — auth-service latency climbing"),
+    ],
+    # Step 2: Auth-service connection pool pressure visible
+    [
+        ("auth-service", "WARN",  "db connection pool at 42/50 — pressure building"),
+        ("api-gateway",  "ERROR", "upstream timeout from auth-service: 30005ms"),
+        ("auth-service", "ERROR", "db query timeout: SELECT session WHERE user_id=? [5001ms]"),
+    ],
+    # Step 3: Auth-service pool nearly exhausted
+    [
+        ("auth-service", "ERROR", "db connection pool EXHAUSTED (50/50) — blocking new requests"),
+        ("api-gateway",  "ERROR", "auth-service unavailable: connection pool full"),
+        ("auth-service", "WARN",  "request queue depth: 127 — approaching overflow"),
+    ],
+    # Step 4: user-db slow query finally exposed
+    [
+        ("user-db",      "WARN",  "slow query detected: SELECT * FROM sessions WHERE user_id=? [2847ms]"),
+        ("auth-service", "ERROR", "db connection timeout after 5000ms — query hanging"),
+        ("user-db",      "ERROR", "lock wait timeout: session table — blocking reads"),
+    ],
+    # Step 5: user-db circuit breaker opens; auth-service starts failing fast
+    [
+        ("user-db",      "WARN",  "slow query: 4500ms — circuit breaker approaching threshold"),
+        ("auth-service", "ERROR", "circuit breaker OPEN for user-db: latency exceeded 5000ms"),
+        ("api-gateway",  "ERROR", "all /auth/* requests failing — upstream unavailable"),
+    ],
+    # Step 6: Full cascade — all 3 services degraded
+    [
+        ("api-gateway",  "ERROR", "error rate: 67.4% — multiple upstreams timing out"),
+        ("auth-service", "ERROR", "health check FAILED: cannot reach user-db"),
+        ("user-db",      "ERROR", "connection pool saturated: 95/100 connections in use"),
+    ],
+    # Step 7: api-gateway now fully symptomatic
+    [
+        ("api-gateway",  "FATAL", "SLA breach: /auth endpoint availability < 95%"),
+        ("auth-service", "ERROR", "auth-service DOWN: 3/3 health checks failed"),
+        ("user-db",      "WARN",  "slow query count: 847 in last 60s — severe degradation"),
+    ],
+    # Step 8: Database fully exposed as root cause
+    [
+        ("user-db",      "ERROR", "CRITICAL: user-db query latency 8000ms+ — active sessions timing out"),
+        ("auth-service", "ERROR", "rejected: user-db connection pool exhausted"),
+        ("api-gateway",  "ERROR", "user-auth endpoint returning 503 — cascade failure"),
+    ],
+    # Step 9: Escalating
+    [
+        ("user-db",      "FATAL", "user-db DOWN: connection pool 100/100 — no connections available"),
+        ("api-gateway",  "ERROR", "error rate: 89.2% — auth-service and user-db both unreachable"),
+    ],
+    # Step 10: Critical
+    [
+        ("api-gateway",  "FATAL", "CRITICAL: auth-service DOWN for 90s — 100% of login attempts failing"),
+        ("user-db",      "ERROR", "lock contention: session table fully locked — queries timing out"),
+    ],
+    # Step 11: Maximum severity
+    [
+        ("user-db",      "FATAL", "user-db unresponsive for 180s — database crisis"),
+        ("api-gateway",  "FATAL", "SLA_BREACH: auth availability 0% — complete user-auth outage"),
+    ],
+]
+
+
+def get_system_state(step: int, base_time: datetime) -> dict[str, ServiceStatus]:
+    """Return system state for this step. Cascade: user-db → auth-service → api-gateway."""
+    now = _make_timestamp(base_time, step * 30)
+    state = generate_healthy_system_state(base_time)
+
+    # Escalating degradation based on step
+    if step <= 1:
+        # Gateway just starting to see issues
+        state["api-gateway"] = ServiceStatus(
+            name="api-gateway", status="degraded", error_rate=0.083, latency_p99_ms=2500, last_updated=now
+        )
+    elif step <= 3:
+        # Auth-service pool pressure
+        state["api-gateway"] = ServiceStatus(
+            name="api-gateway", status="degraded", error_rate=0.157, latency_p99_ms=5000, last_updated=now
+        )
+        state["auth-service"] = ServiceStatus(
+            name="auth-service", status="degraded", error_rate=0.15, latency_p99_ms=5000, last_updated=now
+        )
+    elif step <= 5:
+        # user-db slow queries exposed
+        state["api-gateway"] = ServiceStatus(
+            name="api-gateway", status="degraded", error_rate=0.45, latency_p99_ms=8000, last_updated=now
+        )
+        state["auth-service"] = ServiceStatus(
+            name="auth-service", status="down", error_rate=0.85, latency_p99_ms=10000, last_updated=now
+        )
+        state["user-db"] = ServiceStatus(
+            name="user-db", status="degraded", error_rate=0.30, latency_p99_ms=4500, last_updated=now
+        )
+    elif step <= 7:
+        # Full cascade
+        state["api-gateway"] = ServiceStatus(
+            name="api-gateway", status="down", error_rate=0.89, latency_p99_ms=10000, last_updated=now
+        )
+        state["auth-service"] = ServiceStatus(
+            name="auth-service", status="down", error_rate=0.95, latency_p99_ms=10000, last_updated=now
+        )
+        state["user-db"] = ServiceStatus(
+            name="user-db", status="down", error_rate=0.50, latency_p99_ms=8000, last_updated=now
+        )
+    else:
+        # Maximum severity
+        state["api-gateway"] = ServiceStatus(
+            name="api-gateway", status="down", error_rate=0.99, latency_p99_ms=10000, last_updated=now
+        )
+        state["auth-service"] = ServiceStatus(
+            name="auth-service", status="down", error_rate=1.0, latency_p99_ms=10000, last_updated=now
+        )
+        state["user-db"] = ServiceStatus(
+            name="user-db", status="down", error_rate=0.75, latency_p99_ms=10000, last_updated=now
+        )
+
+    return state
+
+
+def get_step_data(step: int, base_time: datetime, rng: random.Random) -> tuple[list[LogLine], dict[str, ServiceStatus]]:
+    """
+    Returns (logs, system_state) for the given step.
+    Signal gets louder over time if agent hasn't acted.
+    """
+    signal_idx = min(step, len(STEP_SIGNALS) - 1)
+    signals = STEP_SIGNALS[signal_idx]
+
+    logs = generate_log_batch(
+        scenario_signals=signals,
+        step=step,
+        base_time=base_time,
+        noise_ratio=GROUND_TRUTH["noise_ratio"],
+        batch_size=10,
+        rng=rng,
+    )
+    system_state = get_system_state(step, base_time)
+    return logs, system_state
+
+
+def get_active_alerts(step: int) -> list[str]:
+    """Return active alerts for this step."""
+    alerts = []
+    if step >= 0:
+        alerts.append("api-gateway: elevated error rate on /auth/* routes")
+    if step >= 2:
+        alerts.append("auth-service: db connection pool pressure")
+    if step >= 4:
+        alerts.append("user-db: slow queries detected — latency 2000ms+")
+    if step >= 5:
+        alerts.append("auth-service: circuit breaker OPEN for user-db")
+    if step >= 6:
+        alerts.append("SLA_BREACH: /auth availability < 90%")
+    if step >= 8:
+        alerts.append("CRITICAL: user-db connection pool saturated")
+    if step >= 10:
+        alerts.append("CRITICAL: full auth cascade failure — P1 incident")
+    return alerts

server/scenarios/silent_degrade.py

@@ -0,0 +1,132 @@
+"""
+Task 3 — Silent Degradation with Noise (Hard)
+
+Scenario: payment-db query latency slowly increases over time.
+No service crashes. Error rate stays below P1 threshold (5%).
+60% of logs are irrelevant noise from unrelated services.
+Agent must filter noise, identify subtle signal, classify as P2 (NOT P1, NOT P3).
+
+Ground truth:
+  - severity: P2 (nuanced — trending toward breach but no hard outage yet)
+  - root_cause: payment-db
+  - remediation: flush-cache:payment-db OR kill-query:payment-db
+  - correct_teams: dba-team
+  - noise_ratio: 60% (hardest noise ratio of all tasks)
+"""
+from __future__ import annotations
+import random
+from datetime import datetime
+from server.models import LogLine, ServiceStatus
+from server.log_generator import (
+    generate_log_batch,
+    generate_healthy_system_state,
+    _make_timestamp,
+)
+
+# Ground Truth
+
+GROUND_TRUTH = {
+    "severity": "P2",
+    "root_cause": "payment-db",
+    "remediation_prefixes": {"flush-cache", "kill-query"},
+    "remediation_service": "payment-db",
+    "correct_teams": {"dba-team"},
+    "max_steps": 15,
+    "noise_ratio": 0.60,
+}
+
+# Step signals: (service, level, message)
+STEP_SIGNALS = [
+    # Step 0: Very subtle
+    [("payment-db", "WARN", "payment-db: query latency elevated 450ms (baseline: 12ms)")],
+    # Step 1
+    [("payment-db", "WARN", "payment-db: query latency 620ms")],
+    # Step 2
+    [("payment-db", "WARN", "payment-db: slow query: SELECT * FROM transactions WHERE user_id=? [890ms]")],
+    # Step 3
+    [("payment-db", "WARN", "payment-db: buffer cache hit ratio dropping: 89% to 71%")],
+    # Step 4
+    [("payment-db", "WARN", "payment-db: query latency 1200ms"), ("payment-service", "WARN", "payment-service: error rate 2.1%")],
+    # Step 5
+    [("payment-db", "WARN", "payment-db: buffer cache hit ratio 54% — cache thrashing")],
+    # Step 6
+    [("payment-db", "WARN", "payment-db: slow query: SELECT * FROM transactions [2200ms]")],
+    # Step 7
+    [("payment-db", "WARN", "payment-db: query latency 2800ms"), ("payment-service", "ERROR", "payment-service: 3.4% error rate")],
+    # Step 8
+    [("payment-db", "ERROR", "payment-db: slow query timeout: [3100ms] — query cancelled")],
+    # Step 9
+    [("payment-db", "WARN", "payment-db: query latency 4200ms — cache fully cold")],
+    # Step 10
+    [("payment-db", "ERROR", "payment-db: query latency 4500ms")],
+    # Step 11
+    [("payment-db", "WARN", "payment-db: buffer pool pages: 94% dirty")],
+    # Step 12
+    [("payment-db", "ERROR", "payment-db: query latency 4600ms — timeouts beginning"), ("payment-service", "ERROR", "payment-service: error rate 4.9%")],
+    # Step 13: P1 breached
+    [("payment-db", "ERROR", "payment-db: CRITICAL query latency 4950ms — P1 breached"), ("payment-service", "ERROR", "payment-service: error rate 5.1% — P1 exceeded")],
+    # Step 14: Worst case
+    [("payment-db", "FATAL", "payment-db: query latency 5000ms+ — connection pool exhausted"), ("payment-service", "FATAL", "payment-service: P1 CRITICAL — 6.2% error rate")],
+]
+
+
+def get_system_state(step: int, base_time: datetime) -> dict[str, ServiceStatus]:
+    now = _make_timestamp(base_time, step * 30)
+    state = generate_healthy_system_state(base_time)
+
+    latencies = [450, 620, 890, 1200, 1400, 1800, 2200, 2800, 3100, 4200, 4500, 4600, 4600, 4950, 5000]
+    error_rates = [0.0, 0.005, 0.01, 0.021, 0.021, 0.025, 0.028, 0.034, 0.038, 0.042, 0.047, 0.049, 0.049, 0.051, 0.062]
+
+    step_idx = min(step, len(latencies) - 1)
+    db_latency = latencies[step_idx]
+    db_error = error_rates[step_idx]
+
+    psvc_latency = min(5000, 340 + db_latency // 2)
+    psvc_error = min(0.10, db_error * 0.8)
+
+    state["payment-db"] = ServiceStatus(
+        name="payment-db",
+        status="up" if step < 3 else "degraded",
+        error_rate=db_error,
+        latency_p99_ms=db_latency,
+        last_updated=now,
+    )
+    state["payment-service"] = ServiceStatus(
+        name="payment-service",
+        status="degraded" if step >= 4 else "up",
+        error_rate=psvc_error,
+        latency_p99_ms=psvc_latency,
+        last_updated=now,
+    )
+    return state
+
+
+def get_step_data(step: int, base_time: datetime, rng: random.Random) -> tuple[list[LogLine], dict[str, ServiceStatus]]:
+    signal_idx = min(step, len(STEP_SIGNALS) - 1)
+    signals = STEP_SIGNALS[signal_idx]
+
+    logs = generate_log_batch(
+        scenario_signals=signals,
+        step=step,
+        base_time=base_time,
+        noise_ratio=GROUND_TRUTH["noise_ratio"],
+        batch_size=12,
+        rng=rng,
+    )
+    system_state = get_system_state(step, base_time)
+    return logs, system_state
+
+
+def get_active_alerts(step: int) -> list[str]:
+    alerts = []
+    if step >= 4:
+        alerts.append("payment-service: error rate 2%+ — watching")
+    if step >= 6:
+        alerts.append("payment-service: p99 latency above threshold")
+    if step >= 9:
+        alerts.append("payment-db: query latency 4000ms+ — approaching P1 threshold")
+    if step >= 12:
+        alerts.append("WARNING: payment error rate approaching 5% P1 threshold")
+    if step >= 13:
+        alerts.append("ALERT: P1 threshold BREACHED for payment-service")
+    return alerts

server/scenarios/single_crash.py

@@ -0,0 +1,124 @@
+"""
+Task 1 — Single Service Crash (Easy)
+
+Scenario: payment-service crashes with NullPointerException on every request.
+All other services are healthy. Logs are mostly unambiguous.
+Noise ratio: ~20%.
+
+Ground truth:
+  - severity: P1
+  - root_cause: payment-service
+  - remediation: restart:payment-service
+  - correct_team: backend-team
+"""
+from __future__ import annotations
+import random
+from datetime import datetime
+from server.models import LogLine, ServiceStatus
+from server.log_generator import (
+    generate_log_batch,
+    generate_healthy_system_state,
+    SIGNAL_TEMPLATES,
+    _make_timestamp,
+)
+
+# ─── GROUND TRUTH ─────────────────────────────────────────────────────────────
+
+GROUND_TRUTH = {
+    "severity": "P1",
+    "root_cause": "payment-service",
+    "remediation_prefixes": {"restart"},          # restart:payment-service is correct
+    "remediation_service": "payment-service",
+    "correct_teams": {"backend-team", "sre-team"},
+    "max_steps": 8,
+    "noise_ratio": 0.20,
+}
+
+# ─── STEP-BY-STEP SIGNAL PLAN ─────────────────────────────────────────────────
+# Each list = signals injected at that step index.
+# Step 0 = after reset (first observation), Step 7 = last possible step.
+
+STEP_SIGNALS = [
+    # Step 0: first signs — circuit breaker opens, error rate spike
+    [
+        ("payment-service", "ERROR", "NullPointerException: Cannot invoke processPayment() on null — PaymentProcessor.java:142"),
+        ("api-gateway",     "WARN",  "error rate spike: 28.4% of /payment requests failing"),
+    ],
+    # Step 1: escalating — more errors, health check fails
+    [
+        ("payment-service", "FATAL", "NullPointerException in PaymentService.execute() — all retries (3/3) exhausted"),
+        ("payment-service", "ERROR", "health check FAILED: payment-service returned HTTP 500"),
+    ],
+    # Step 2: circuit breaker fully open
+    [
+        ("api-gateway",     "ERROR", "circuit breaker OPEN: payment-service error rate 98.2% (threshold: 10%)"),
+        ("payment-service", "ERROR", "NullPointerException: Cannot invoke processPayment() on null — PaymentProcessor.java:142"),
+    ],
+    # Step 3+: same signals repeat — incident ongoing until agent acts
+    [
+        ("payment-service", "ERROR", "NullPointerException in PaymentService.execute() — retrying (1/3)"),
+        ("api-gateway",     "ERROR", "upstream failure: payment-service unavailable [circuit breaker: OPEN]"),
+    ],
+    [
+        ("payment-service", "FATAL", "payment-service health check FAILED for 90s — marking as DOWN"),
+        ("api-gateway",     "WARN",  "payment endpoint degraded — all requests returning 503"),
+    ],
+    [
+        ("payment-service", "ERROR", "NullPointerException: Cannot invoke processPayment() on null — PaymentProcessor.java:142"),
+        ("api-gateway",     "ERROR", "error rate: 99.1% on /payment/* routes"),
+    ],
+    [
+        ("payment-service", "FATAL", "NullPointerException — service unresponsive for 180s"),
+        ("api-gateway",     "ERROR", "SLA breach: payment service uptime < 99.9%"),
+    ],
+    [
+        ("payment-service", "FATAL", "CRITICAL: payment-service has been DOWN for 210s — immediate action required"),
+        ("api-gateway",     "ERROR", "all payment transactions failing — revenue impact ongoing"),
+    ],
+]
+
+
+def get_system_state(step: int, base_time: datetime) -> dict[str, ServiceStatus]:
+    """Return system state for this step. payment-service is down; others are healthy."""
+    now = _make_timestamp(base_time, step * 30)
+    state = generate_healthy_system_state(base_time)
+
+    # Override payment-service to be DOWN
+    state["payment-service"] = ServiceStatus(
+        name="payment-service",
+        status="down",
+        error_rate=0.982,
+        latency_p99_ms=5000,
+        last_updated=now,
+    )
+    return state
+
+
+def get_step_data(step: int, base_time: datetime, rng: random.Random) -> tuple[list[LogLine], dict[str, ServiceStatus]]:
+    """
+    Returns (logs, system_state) for the given step.
+    Signals get louder over time if agent hasn't acted.
+    """
+    signal_idx = min(step, len(STEP_SIGNALS) - 1)
+    signals = STEP_SIGNALS[signal_idx]
+
+    logs = generate_log_batch(
+        scenario_signals=signals,
+        step=step,
+        base_time=base_time,
+        noise_ratio=GROUND_TRUTH["noise_ratio"],
+        batch_size=8,
+        rng=rng,
+    )
+    system_state = get_system_state(step, base_time)
+    return logs, system_state
+
+
+def get_active_alerts(step: int) -> list[str]:
+    """Return active alerts for this step."""
+    alerts = ["payment-service: circuit breaker OPEN", "payment-service: health check FAILING"]
+    if step >= 2:
+        alerts.append("SLA_BREACH: payment availability < 99.9%")
+    if step >= 5:
+        alerts.append("CRITICAL: payment-service DOWN > 150s")
+    return alerts