v2: physically correct SRF-PLL, realistic attacks, exploit-proof graders

Physics
- Fix Clarke/Park transforms (vq was cos(θ+θ̂), so the PLL never locked);
cosine-referenced amplitude-invariant transforms, all three phases,
5 kHz simulation, ωn=100 rad/s, ζ=0.707, amplitude-normalised PI loop
- Natural grid events (phase jumps, sags, single-phase faults, harmonic
steps, frequency events) as confounders; sensor noise + 12-bit ADC
- Coordinated phase-spoofing attack for the hard task; trusted, delayed
PMU stream; PMU-disciplined holdover and latched disconnect actions
- Desync measured against the true grid angle; shadow PLL gives the
counterfactual time-to-desync used by the hard grader

Environment / graders / reward
- 40 decisions x 50 ms, attack-free episodes, single seeded RNG tree
- Ground truth hidden until done; env detector no longer leaked in info
- Graders penalise false alarms; always/never/random policies score low,
oracle scores >= 0.9 on every seed
- Dense reward with Brier calibration, mitigation costs and harm penalty

OpenEnv / infra
- openenv-core 0.3.0 create_app (WS, schema, metadata, MCP, health) plus
session-aware HTTP reset/step/state; typed EnvClient; landing page demo
- inference.py: LLM every step, fixed seeds, in-process fallback, fast
fallback on permanent API errors, time budget, atomic log blocks
- Python 3.12 non-root Docker image, .dockerignore, CI workflow,
submission validator, 65 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.dockerignore

@@ -0,0 +1,16 @@
+# Never bake secrets or local state into the image
+.env
+.env.*
+!.env.example
+.git
+.github
+.venv
+venv
+**/__pycache__
+**/*.pyc
+.pytest_cache
+tests
+scripts
+uv.lock
+*.patch
+*.log

.env.example

@@ -1,10 +1,13 @@
-# LLM Configuration
-USE_LLM=1
+# --- LLM used by inference.py (OpenAI-compatible endpoint) ---
 API_BASE_URL=https://router.huggingface.co/v1
 MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
-# Get your token from https://huggingface.co/settings/tokens
-HF_TOKEN=your_huggingface_token_here
+# Hugging Face token (https://huggingface.co/settings/tokens) or any key for API_BASE_URL
+HF_TOKEN=your_token_here
 
-# Environment Configuration
-# Use http://0.0.0.0:7860 or http://localhost:7860 locally
-ENV_URL=http://localhost:7860
+# --- Optional ---
+# Environment server; leave empty to run the environment in-process
+ENV_URL=
+# "llm" (default when a token is set) or "heuristic"
+AGENT=llm
+EPISODES_PER_TASK=3
+MAX_WORKERS=3

.gitattributes

@@ -0,0 +1,2 @@
+*.sh text eol=lf
+Dockerfile text eol=lf

.github/workflows/ci.yml

@@ -0,0 +1,40 @@
+name: CI
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+      - name: Install
+        run: pip install -r requirements-dev.txt
+      - name: Unit tests
+        run: pytest -q
+      - name: OpenEnv structure
+        run: openenv validate .
+      - name: OpenEnv runtime contract
+        run: |
+          uvicorn server.app:app --port 7860 &
+          for i in $(seq 1 30); do curl -sf localhost:7860/health && break; sleep 1; done
+          openenv validate --url http://localhost:7860 | tee runtime.json
+          python -c "import json,sys; sys.exit(0 if json.load(open('runtime.json'))['passed'] else 1)"
+      - name: Baseline (heuristic, through the server)
+        run: AGENT=heuristic ENV_URL=http://localhost:7860 python inference.py
+
+  docker:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Build and smoke-test image
+        run: |
+          docker build -t pll-cyberguard .
+          docker run -d -p 7860:7860 --name env pll-cyberguard
+          for i in $(seq 1 30); do curl -sf localhost:7860/health && break; sleep 1; done
+          curl -sf -X POST localhost:7860/reset -H 'content-type: application/json' -d '{"task_id":2,"seed":1}' > /dev/null

.github/workflows/deploy.yml

@@ -2,22 +2,18 @@ name: Deploy to Hugging Face Space
 
 on:
   push:
-    branches:
-      - master
+    branches: [master, main]
+  workflow_dispatch:
 
 jobs:
   deploy:
     runs-on: ubuntu-latest
-
     steps:
-      - name: Checkout repository
-        uses: actions/checkout@v3
+      - uses: actions/checkout@v4
         with:
-          lfs: true
           fetch-depth: 0
-
+          lfs: true
       - name: Push to Hugging Face Space
         env:
           HF_TOKEN: ${{ secrets.HF_TOKEN }}
-        run: |
-          git push https://krishuggingface:${HF_TOKEN}@huggingface.co/spaces/krishuggingface/CyberAttack-PLL HEAD:main
+        run: git push https://krishuggingface:${HF_TOKEN}@huggingface.co/spaces/krishuggingface/CyberAttack-PLL HEAD:main

.gitignore

@@ -1,24 +1,26 @@
 # Python
 __pycache__/
-*.pyc
-*.pyo
+*.py[cod]
 *.egg-info/
 dist/
 build/
 
-# Environment
+# Environments / secrets
 .env
 .venv/
 venv/
 
-# Testing
+# Tooling
 .pytest_cache/
+.ruff_cache/
+.coverage
+htmlcov/
 
-# Data
-sample_data/
+# Local outputs
+results/
+*.log
 
 # IDE
 .vscode/
 .idea/
 *.swp
-*.swo

Dockerfile

@@ -1,17 +1,25 @@
-FROM python:3.10-slim
+FROM python:3.12-slim
 
-WORKDIR /app
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1 \
+    PIP_NO_CACHE_DIR=1 \
+    PORT=7860
 
-RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*
+# Hugging Face Spaces run containers as uid 1000
+RUN useradd --create-home --uid 1000 app
+WORKDIR /home/app/env
 
 COPY requirements.txt .
-RUN pip install --no-cache-dir -r requirements.txt
+RUN pip install --upgrade pip && pip install -r requirements.txt
 
-COPY . .
-
-HEALTHCHECK --interval=30s --timeout=10s --start-period=5s \
-    CMD curl -f http://localhost:7860/health || exit 1
+COPY --chown=app:app pll_cyber ./pll_cyber
+COPY --chown=app:app server ./server
+COPY --chown=app:app inference.py openenv.yaml pyproject.toml README.md ./
 
+USER app
 EXPOSE 7860
 
-CMD ["uvicorn", "src.api:app", "--host", "0.0.0.0", "--port", "7860"]
+HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
+    CMD python -c "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:7860/health', timeout=4).status == 200 else 1)"
+
+CMD ["sh", "-c", "uvicorn server.app:app --host 0.0.0.0 --port ${PORT}"]

README.md

@@ -1,162 +1,265 @@
 ---
-title: PLL Cyberattack Detection
+title: PLL Cyber-Guard
 emoji: ⚡
 colorFrom: blue
 colorTo: purple
 sdk: docker
-app_file: Dockerfile
+app_port: 7860
 pinned: false
+tags:
+  - openenv
+  - reinforcement-learning
+  - power-systems
+  - cybersecurity
 ---
-# PLL Cyberattack Detection — OpenEnv
-
-[![HF Space](https://img.shields.io/badge/🤗%20HuggingFace-Space%20Live-blue)](https://huggingface.co/spaces/krishuggingface/CyberAttack-PLL)
-[![Docker](https://img.shields.io/badge/Docker-Ready-2496ED?logo=docker&logoColor=white)](Dockerfile)
-[![OpenEnv](https://img.shields.io/badge/OpenEnv-Compliant-green)](openenv.yaml)
-[![Python](https://img.shields.io/badge/Python-3.10%2B-3776AB?logo=python&logoColor=white)](https://python.org)
-
-> AI-driven cyberattack detection on SRF Phase-Locked Loops (PLLs) in grid-connected inverters.
-
-## Overview
-
-Phase-Locked Loops (PLLs) are critical components in grid-connected power converters, responsible for synchronizing the inverter's output with the utility grid. The Synchronous Reference Frame PLL (SRF-PLL) estimates grid frequency and phase angle by tracking the q-axis voltage component. Because of its critical role and reliance on sensor data, the SRF-PLL is a high-value target for **False Data Injection (FDI)** cyberattacks.
-
-This OpenEnv environment simulates an SRF-PLL subjected to varied FDI attack scenarios. An AI agent acts as a cyber-guard: it monitors arriving time-windowed sensor observations—such as voltages and frequency deviations—and must accurately detect, classify, and mitigate attacks in real-time before grid synchronization is lost.
-
-## Architecture
-
-The environment relies on a discrete-time SRF-PLL simulation running at a 1 ms step size. A streamlined view of the signal flow is below:
-
-```text
-Grid Voltage (50Hz)
-     │
-     ▼
-[FDI Attack Injection]  ◄── Attacker injects a malicious signal on phase `va`
-     │
-     ▼
-Clarke Transform (αβ)
-     │
-     ▼
-Park Transform (dq)     ◄── Uses the currently estimated angle θ̂
-     │
-     ▼
-PI Controller           ──► ω̂, θ̂ are updated continuously
-     │
-     ▼
-Agent Observation       ──► Agent receives: `vq_window`, `omega_deviation_window`, `raw_voltages`
-     │
-     ▼
-Agent Action            ──► Agent outputs: `attack_detected`, `attack_type`, `confidence`
-```
-
-## Inference Flow & Detector Walkthrough
-
-To balance speed and accuracy across thousands of steps, the standard inference client (`inference.py`) deploys a **Smart Blending Strategy**:
-
-1. **Environment Simulation (`env.py`)**: 
-   Every step, the PLL updates its internal math based on potential attack injections. It yields a rich observation window of the last 20 frames for variables like $V_q$ and $\omega_{dev}$.
-2. **Adaptive Physics-Informed Detector (`src/detector.py`)**: 
-   Before returning the observation to the client, the environment evaluates the data using an intrinsic physics-based detector. This detector calibrates anomaly residuals during the first 20 "healthy" warm-up steps. It tracks variances and symmetry to identify stealthy voltage anomalies, providing a baseline `confidence` score.
-3. **Smart Blending Client (`inference.py`)**: 
-   The client receives the observation and the detector's baseline prediction. 
-   * If the intrinsic detector has high confidence (> 50%), the client adopts its recommendation.
-   * If the anomaly is ambiguous (confidence < 50%), the client queries its own **Rule-Based Heuristic Agent**, which monitors historical $V_q$ growth, monotonicity, and zero-crossing density.
-   * *Optional*: If `USE_LLM=1` is set, the client uses an LLM (e.g., `Qwen2.5-72B`) for advanced reasoning. A resilient "circuit breaker" automatically transitions to the heuristic model if network or authentication failures occur.
-
-## Tasks
-
-The environment supports three sequentially evaluated difficulty levels:
 
-| Task | ID | Difficulty | Attack Type | Objective | Score Metric |
-|------|----|-----------|-------------|-----------|-------|
-| Sinusoidal FDI | 0 | Easy | Sinusoidal Injection | Detect attack within 100 steps of initiation. | Time-decaying detection reward. |
-| Multi-Attack Class. | 1 | Medium | Sinusoidal, Ramp, Pulse | Safely and correctly classify the specific attack type. | Accuracy and speed aggregate. |
-| Stealthy Detection | 2 | Hard | Low-amplitude phase drift | Detect slow deviations before the PLL loses lock (θ_error > 5°). | Preventative lock-loss metric. |
+# PLL Cyber-Guard — an OpenEnv environment for cyber-attacks on grid inverters
 
-## Observation Space
+[![HF Space](https://img.shields.io/badge/🤗%20Space-live-blue)](https://huggingface.co/spaces/krishuggingface/CyberAttack-PLL)
+[![OpenEnv](https://img.shields.io/badge/OpenEnv-validated-green)](openenv.yaml)
+[![CI](https://github.com/KrisKeshav/PLL-Cyberattack-Detection-OpenEnv/actions/workflows/ci.yml/badge.svg)](.github/workflows/ci.yml)
 
-At each step, the environment provides a JSON observation containing:
+An agent plays the **cyber-security guard of a grid-connected solar/battery inverter**. Every 50 ms it reads
+the signals of the inverter's phase-locked loop (PLL) and a trusted but delayed phasor-measurement (PMU)
+stream. It must **detect, classify and mitigate false-data-injection (FDI) attacks** on the voltage sensors,
+without raising alarms on the natural disturbances a real grid produces all the time.
 
-| Field | Shape | Description |
-|-------|-------|-------------|
-| `vq_window` | `[20]` | q-axis voltage error signal (pu). |
-| `vd_window` | `[20]` | d-axis voltage (pu). |
-| `omega_window` | `[20]` | Normalized frequency deviation from nominal. |
-| `omega_deviation_window` | `[20]` | Frequency deviation from nominal (rad/s). |
-| `raw_voltages` | `[3]` | Raw three-phase voltages `[va, vb, vc]` (pu). |
-| `step` | `scalar` | Current simulation time step. |
-| `task_id` | `scalar` | Current task identifier (0, 1, or 2). |
+## Why this matters
 
-**Total observation dimension**: 83 ($20 \times 4 + 3$)
+Inverter-based resources (solar, wind, batteries) now supply a large share of many grids. A grid-following inverter
+locks to the grid through a **Synchronous-Reference-Frame PLL** (SRF-PLL). If an attacker can rewrite the voltage
+samples the PLL sees (compromised sensor firmware, a man-in-the-middle on the sampled-values bus, or malware on the
+controller), they can make the inverter inject current out of phase with the real grid. The result is power-quality
+problems, protection trips, and, at scale, a stability risk. PLL-targeted FDI is an active research topic. What makes
+it hard in practice is not detecting anomalies. It is **telling attacks apart from ordinary grid events**, and
+**acting before damage while avoiding false trips**, because every false trip throws away generation.
 
-## Action Space
+This environment turns that operator problem into a benchmark. The physics is simulated faithfully: a 5 kHz sampled
+SRF-PLL, a sensor chain, natural grid events and a PMU. The reward is dense and accounts for cost, and the graders are
+deterministic.
 
-Agents must return a structured JSON response predicting the system state:
+## What the agent faces
 
-| Field | Type | Range | Description |
-|-------|------|-------|-------------|
-| `attack_detected` | `bool` | — | True if malicious injection is suspected. |
-| `attack_type` | `int` | 0–4 | 0=None, 1=Sinusoidal, 2=Ramp, 3=Pulse, 4=Stealthy. |
-| `confidence` | `float` | 0.0–1.0 | Absolute predictive certainty. |
-| `protective_action` | `int` | 0–3 | Suggested mitigation: 0=None, 1=Alert, 2=Reduce Power, 3=Disconnect. |
+```
+ true grid (50 Hz ± wander, 5th/7th harmonics, natural events)
+        │                                     │
+        ▼                                     ▼
+ [FDI attacker rewrites samples]       PMU at substation (GPS-timed, not spoofable)
+        │                              10 reports/s · noisy · 40–100 ms latency
+        ▼                                     │
+ sensors: noise σ=5 mpu + 12-bit ADC          │
+        ▼                                     │
+ SRF-PLL (Clarke → Park → PI, ωn=100 rad/s,   │
+          ζ=0.707, amplitude-normalised)      │
+        │   ▲ holdover: loop opened,          │
+        │   └ PMU-disciplined oscillator ◄────┤
+        ▼                                     ▼
+ features + waveforms ─────────►  AGENT  ◄── PMU frequency / magnitude
+                                    │
+          attack_detected · attack_type · confidence · protective_action
+```
 
-## Setup & API Usage
+**Physics notes.** The Clarke/Park transforms are cosine-referenced and amplitude-invariant, so `vq = V·sin(θ − θ̂)`.
+The PLL is type-2 and has zero steady-state error to frequency steps; the tests verify this. The simulator also tracks
+the **true** grid angle, so it knows when the inverter really drifts out of synchronism. That is defined as
+**|θ̂ − θ| > 10° sustained for 30 ms**, and the agent never sees it. A "shadow" PLL that never mitigates runs alongside
+and gives the counterfactual time at which an unprotected inverter would have lost sync. The hard-task grader uses
+that time as its deadline.
+
+### Natural grid events (not attacks)
+| Event | What it looks like to the PLL |
+|---|---|
+| Phase jump (5–25°) | vq spike, frequency transient; recovers in about 30 ms; the PMU frequency also spikes |
+| Balanced sag (0.5–0.9 pu) | vd drops on all phases |
+| Single-phase sag (0.3–0.8 pu) | zero/negative sequence **at 50 Hz**, 1.5 Hz frequency ripple, which looks like single-phase FDI |
+| Harmonic step | 300 Hz ripple in vq grows |
+| Frequency event (RoCoF 0.1–0.6 Hz/s) | local frequency drifts, and the PMU sees the same drift (with a delay) |
+
+### Attacks
+| id | Attack | Footprint |
+|---|---|---|
+| 1 | Sinusoidal interharmonic injection, one phase | zero-sequence content **off** 50 Hz, persistent frequency oscillation |
+| 2 | Ramp bias, one phase | growing DC offset (real grids have no DC) |
+| 3 | Pulse bias, one phase (100–300 ms) | constant DC offset, then gone |
+| 4 | **Coordinated phase spoofing** | all three phases rotated by a slowly accelerating angle δ(t). The PLL tracks it perfectly, so vq is unchanged, and the local frequency drifts like a natural frequency event. The only evidence is a mismatch against the *delayed* PMU. Unprotected, the inverter loses sync 0.35–0.7 s after onset. |
 
-The system acts as a standard REST API server over port `7860`.
+## Tasks
 
-### Local Setup
+| id | name | difficulty | what the agent must do |
+|---|---|---|---|
+| 0 | `sinusoidal_fdi_detection` | easy | Detect a sinusoidal FDI quickly. 20% of episodes are attack-free; occasional mild sags. |
+| 1 | `attack_classification_under_disturbances` | medium | Detect **and classify** sinusoidal / ramp / pulse attacks while 1–3 natural events occur. At least one event always lands before the attack, so false-alarm resistance is always tested. |
+| 2 | `stealthy_phase_spoofing_mitigation` | hard | Detect phase spoofing hidden in natural frequency events using the delayed PMU, and engage **holdover before the inverter desynchronises**. Holding over or tripping on a healthy grid is penalised. |
+
+Each episode is 40 decisions × 50 ms (2 s after a 200 ms warm-up). A scenario is fully determined by
+`(task_id, seed)`: attack type, onset, parameters, grid events, harmonics, PMU latency and every noise stream.
+
+## Action space (`PLLAction`)
+| field | type | meaning |
+|---|---|---|
+| `attack_detected` | bool | an attack is in progress |
+| `attack_type` | int 0–4 | 0 none, 1 sinusoidal, 2 ramp, 3 pulse, 4 phase spoofing |
+| `confidence` | float 0–1 | probability that `attack_detected` is right (scored with a Brier rule) |
+| `protective_action` | int 0–3 | 0 none, 1 alert operator, **2 PMU-aided holdover** (open the PLL loop; free-run on a PMU-disciplined oscillator), **3 disconnect** (latched) |
+
+## Observation space (`PLLObservation`)
+| group | content |
+|---|---|
+| `waveforms` | the last 50 ms window at 2 ms resolution (25 points each): `va, vb, vc, vd, vq, f_hz` |
+| `features` | on-board signal processing: positive-sequence voltage, vq mean/RMS, local frequency, mean/std/RoCoF, zero-sequence RMS, per-phase RMS and DC offset, dominant vq oscillation (FFT over 200 ms), local-minus-PMU frequency |
+| `pmu` | `f_hz`, `v_pos_pu`, `age_ms` of the latest PMU report that has arrived |
+| `trend` | the last 10 windows of local frequency, PMU frequency, vq RMS, zero-sequence RMS and positive-sequence voltage |
+| status | `task_id`, `task_name`, `task_brief` (on reset), `step`, `time_s`, `inverter_mode` (normal / holdover / disconnected) |
+| `reward_breakdown` | the typed reward model with every component of the last step's reward |
+| `episode_result` | sent only when `done`: `grader_score`, score breakdown, total reward, and the revealed ground-truth scenario |
+
+`GET /state` returns `PLLState`. The scenario ground truth stays `null` until the episode ends, so it can't be used
+to cheat.
+
+## Reward (dense, every step)
+Action *t* is judged against window *t*, the data the agent actually saw. Once an attack starts, the system counts as
+compromised until the episode ends.
+
+| component | value |
+|---|---|
+| true positive / true negative | +0.10 / +0.02 |
+| missed detection / **false alarm** | −0.10 / **−0.25** |
+| early-detection bonus (one-time) | up to +0.5, decaying linearly to 0 at 500 ms |
+| calibration (Brier) | 0.04 · (1 − 4·Brier) ∈ [−0.12, +0.04] |
+| classification (tasks 1–2) | +0.05 correct / −0.02 wrong |
+| holdover / disconnected (per window) | −0.02 / −0.04 (degraded operation / lost generation) |
+| trip | −0.3 if under attack, **−1.5 on a healthy grid** (destructive action) |
+| loss of synchronism | −1.0 once, plus −0.3 per window while desynchronised |
+
+Per-step range: **[−3.21, +0.69]**. "Always alarm", "always hold over" and "always trip" all give negative returns.
+
+## Graders (deterministic, 0–1)
+They share these building blocks: false-alarm rate **FA** over clean windows, specificity **max(0, 1 − 2·FA)**,
+timeliness **T** of the first detection, recall **R**, and classification accuracy **C** over compromised windows.
+
+* **easy:** `(0.6·T + 0.4·R) · spec`, where T = 1 up to 100 ms of delay and falls to 0 at 1 s
+* **medium:** `(0.35·T + 0.25·R + 0.40·C) · spec`
+* **hard:** `(0.55·P·Q + 0.30·T + 0.15·R) · spec`. P = 1 if holdover or disconnect was engaged before the
+  counterfactual desync time *and* the inverter never lost sync. Q = 1.0 for holdover, 0.6 for disconnect. T is
+  measured against the time-to-desync budget.
+* **attack-free episodes:** `spec`, multiplied by 0.5 for a false trip (0.4 in the hard task) and by 0.75 for a false
+  holdover.
+
+Scores are clamped to [0.01, 0.99] for validators that require strictly open (0, 1) scores.
+
+## Baseline scores
+
+100 seeds per task (seeds 1000–1099), from `python scripts/evaluate_baselines.py --seeds 100`:
+
+| Policy | Task 0 (easy) | Task 1 (medium) | Task 2 (hard) | Mean |
+|---|---|---|---|---|
+| Always alarm | 0.010 ± 0.000 | 0.010 ± 0.000 | 0.010 ± 0.000 | 0.010 |
+| Never alarm | 0.196 ± 0.384 | 0.226 ± 0.406 | 0.167 ± 0.359 | 0.196 |
+| Random (p = 0.5) | 0.078 ± 0.106 | 0.063 ± 0.075 | 0.043 ± 0.056 | 0.061 |
+| **Physics heuristic** (`pll_cyber/baselines.py`) | 0.986 ± 0.003 | 0.920 ± 0.150 | 0.762 ± 0.177 | 0.889 |
+| Oracle (reads ground truth, upper bound) | 0.990 ± 0.000 | 0.990 ± 0.000 | 0.990 ± 0.000 | 0.990 |
+
+The physics heuristic is what a protection engineer would write by hand. It fits the zero-sequence voltage with
+DC + 50 Hz terms and checks the residual, and it integrates the latency-compensated local-vs-PMU frequency mismatch
+into an angle estimate. The hard task leaves a lot of headroom: it requires reasoning about latency, separating a
+spoofed frequency drift from a real one, and deciding when to mitigate.
+
+**`inference.py`** (fixed seeds, 3 episodes per task) runs the **LLM agent**. The model gets the features, PMU data,
+trends and a few DSP diagnostics, and makes every decision itself. Without an API key the script falls back to the
+physics heuristic, which scores **0.989 / 0.944 / 0.717** on its evaluation seeds. LLM baseline numbers depend on
+the model and endpoint you use; run the command below to reproduce them.
+
+## Quick start
 
-**Via Python (Recommended)**:
 ```bash
 pip install -r requirements.txt
-uvicorn src.api:app --host 0.0.0.0 --port 7860
+
+# run the server (HTTP + WebSocket) on :7860
+uvicorn server.app:app --host 0.0.0.0 --port 7860        # or: python -m server.app
+
+# baseline (LLM via any OpenAI-compatible endpoint; heuristic if no key)
+export HF_TOKEN=...            # API_BASE_URL / MODEL_NAME optional
+python inference.py            # in-process env; set ENV_URL=http://localhost:7860 to go through the server
 ```
 
-**Via Docker**:
+Docker / Hugging Face Space:
 ```bash
-docker build -t pll-cyberattack-env .
-docker run -p 7860:7860 pll-cyberattack-env
+docker build -t pll-cyberguard .
+docker run -p 7860:7860 pll-cyberguard
 ```
 
-### Environment Variables
-
-Configure execution behavior locally via a `.env` file (see `.env.example`).
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `API_BASE_URL` | `https://router.huggingface.co/v1` | Custom endpoint for Language Models. |
-| `MODEL_NAME` | `Qwen/Qwen2.5-72B-Instruct` | Internal Model identifier. |
-| `HF_TOKEN` | — | HuggingFace or valid proxy API key. |
-| `USE_LLM` | `1` | Set to `1` to run the active LLM agent, `0` for pure heuristics. |
-
-### REST Endpoints
-
-1. **POST `/reset`**
-   Initializes the environment for a specific task.
-   ```bash
-   curl -X POST http://localhost:7860/reset \
-     -H "Content-Type: application/json" \
-     -d '{"task_id": 0, "seed": 42}'
-   ```
-
-2. **POST `/step`**
-   Submit an action based on recent observations and advance by one tick.
-   ```bash
-   curl -X POST http://localhost:7860/step \
-     -H "Content-Type: application/json" \
-     -d '{"attack_detected": false, "attack_type": 0, "confidence": 0.5, "protective_action": 0}'
-   ```
-
-3. **GET `/health`**
-   Returns operational status and step numbers.
-
-## Baseline Performance
-
-The default hybrid strategy outlined in `inference.py` consistently yields the following evaluation bounds across a full 500-step envelope:
+### HTTP API
+| endpoint | purpose |
+|---|---|
+| `POST /reset` | `{"task_id": 0-2, "seed": int, "session_id": str}` → `{observation, reward, done, session_id}` |
+| `POST /step` | `{"action": {...}, "session_id": str}` → `{observation, reward, done, session_id}` |
+| `GET /state?session_id=` | `PLLState` (ground truth hidden until done) |
+| `GET /tasks` | task catalogue, reward range, action schema |
+| `GET /schema`, `/metadata`, `/health`, `POST /mcp` | OpenEnv standard endpoints |
+| `WS /ws` | persistent session per connection (used by `EnvClient`) |
+| `GET /demo?task_id=&seed=` | runs the heuristic baseline and returns the trace (used by the landing page) |
+
+openenv-core's own HTTP `/reset` and `/step` are stateless. This server replaces them with **session-aware**
+versions that keep the same request and response shapes. Clients that omit `session_id` share a `"default"` session.
+
+### Python client (WebSocket)
+```python
+from pll_cyber.client import PLLCyberEnv
+from pll_cyber.models import PLLAction
+
+with PLLCyberEnv(base_url="http://localhost:7860").sync() as env:
+    r = env.reset(task_id=2, seed=7)
+    while not r.done:
+        r = env.step(PLLAction(attack_detected=False))
+    print(r.observation.episode_result.grader_score)
+```
 
-* **Task 0 (Sinusoidal FDI):** 0.9900 
-* **Task 1 (Multi-Attack Classification):** ~0.8720
-* **Task 2 (Stealthy Drift):** ~0.1639
-* **Aggregate System Average:** `0.6786`
+## Validation & tests
+```bash
+pip install -r requirements-dev.txt
+pytest -q                                   # 65 tests: physics, determinism, graders, API, inference
+openenv validate .                          # local structure
+openenv validate --url http://localhost:7860   # runtime contract (all 6 criteria pass)
+```
+The tests cover:
+- transform identities and PLL lock, frequency-step and phase-jump dynamics
+- natural events never desynchronise the inverter, while spoofing always does yet leaves vq unchanged
+- holdover protects
+- identical trajectories for the same seed
+- ground truth stays hidden until done
+- every reward stays inside the declared range
+- degenerate policies (always / never / random / always-mitigate) score low and the oracle scores at least 0.9 on every seed
+- session isolation and the API contract
+- the inference log format and its LLM failure fallback
+
+## Repository layout
+```
+pll_cyber/
+  pll.py          SRF-PLL (Clarke/Park, PI loop, holdover mode)
+  grid.py         true grid + natural events
+  attacks.py      FDI attack models
+  sensors.py      sensor noise/ADC, PMU stream, PMU-disciplined holdover
+  scenarios.py    tasks, timing constants, seeded scenario sampling
+  features.py     on-board signal processing for observations
+  rewards.py      dense reward
+  graders.py      per-task graders
+  environment.py  OpenEnv Environment (reset/step/state)
+  models.py       typed Action / Observation / Reward / State
+  baselines.py    physics heuristic + trivial policies
+  client.py       typed EnvClient (WebSocket)
+server/app.py     FastAPI app (openenv create_app + session-aware HTTP)
+inference.py      LLM baseline
+scripts/          benchmark + submission validator
+tests/            pytest suite
+```
 
----
-🚀 **Live Environment Hosted on HuggingFace Spaces**: [krishuggingface/CyberAttack-PLL](https://huggingface.co/spaces/krishuggingface/CyberAttack-PLL)
+## Design choices & limitations
+* **The agent gets the PMU's frequency and magnitude channels, not its synchrophasor angle.** The angle is used
+  only by the inverter's holdover controller. Exposing it would make spoofing trivially visible; withholding it
+  models a bandwidth-limited SCADA status feed.
+* Injection attacks hit one phase, so they have a zero-sequence footprint. A balanced attacker is modelled by
+  phase spoofing.
+* The grid is a stiff source behind the point of common coupling. The inverter's current controller and its
+  feedback on the grid voltage are not modelled; the harm is measured as angle desynchronisation.
+* **Possible extensions:** grid-impedance and weak-grid dynamics, attacks on the PMU channel, replay attacks, and
+  multi-inverter coordination.

inference.py

@@ -1,495 +1,327 @@
 """
-Inference Script — PLL Cyberattack Detection OpenEnv
-=====================================================
-Environment variables (injected by the judging sandbox):
-  API_BASE_URL   LiteLLM proxy endpoint  (MUST be used for all LLM calls)
-  API_KEY        LiteLLM proxy key       (MUST be used — do not hardcode keys)
-  MODEL_NAME     Model identifier
-  ENV_URL        Environment server URL  (default: http://localhost:7860)
-
-STDOUT FORMAT (OpenEnv compliance):
+Baseline inference — PLL Cyber-Guard OpenEnv
+=============================================
+
+Runs an LLM agent (OpenAI-compatible client) on all three tasks with fixed
+seeds and prints OpenEnv-style logs plus a final score table.
+
+Environment variables
+  API_BASE_URL        OpenAI-compatible endpoint      (default: https://router.huggingface.co/v1)
+  MODEL_NAME          model identifier                (default: Qwen/Qwen2.5-72B-Instruct)
+  HF_TOKEN / API_KEY  API key for the endpoint
+  ENV_URL             environment server URL; if unset or unreachable the environment runs in-process
+  AGENT               "llm" (default when a key is set) or "heuristic"
+  EPISODES_PER_TASK   default 3
+  MAX_WORKERS         concurrent episodes, default 3
+  TIME_BUDGET_S       wall-clock budget, default 1050 (17.5 min) — steps beyond it fall back to the heuristic
+
+STDOUT (one block per episode, never interleaved)
   [START] task=<task_name> env=<benchmark> model=<model_name>
-  [STEP]  step=<n> action=<action_str> reward=<0.00> done=<true|false> error=<msg|null>
+  [STEP]  step=<n> action=<action_json> reward=<0.00> done=<true|false> error=<msg|null>
   [END]   success=<true|false> steps=<n> score=<score> rewards=<r1,r2,...,rn>
 """
 
+from __future__ import annotations
+
+import json
 import os
+import re
 import sys
-import json
+import threading
 import time
-import requests
-from typing import List, Optional
-from openai import OpenAI
+from concurrent.futures import ThreadPoolExecutor
+from typing import Any, Dict, List, Optional, Tuple
 
 try:
     from dotenv import load_dotenv
-    load_dotenv()
+    load_dotenv(os.path.join(os.path.dirname(os.path.abspath(__file__)), ".env"))
 except ImportError:
     pass
 
-# ── Config — always read from environment, never hardcode ─────────────────────
-# The judging sandbox injects API_BASE_URL and API_KEY via their LiteLLM proxy.
-# All LLM calls MUST go through these values or the submission will be rejected.
-API_BASE_URL = os.environ.get("API_BASE_URL", "https://router.huggingface.co/v1")
-MODEL_NAME   = os.environ.get("MODEL_NAME",   "Qwen/Qwen2.5-72B-Instruct")
-API_KEY      = os.environ.get("API_KEY") or os.environ.get("HF_TOKEN", "dummy")
-ENV_URL      = os.environ.get("ENV_URL",      "http://localhost:7860")
+from pll_cyber.baselines import PhysicsHeuristicAgent, _v0_decompose
 
-# OpenAI client pointed at the proxy — never bypass this
-client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY)
-
-# Persistent HTTP session for env calls — avoids TCP handshake per step
-_session = requests.Session()
-
-# ── Task metadata ─────────────────────────────────────────────────────────────
-TASK_NAMES = {
-    0: "Sinusoidal FDI Detection (Easy)",
-    1: "Multi-Attack Classification (Medium)",
-    2: "Stealthy Attack Detection (Hard)",
-}
+API_BASE_URL = os.environ.get("API_BASE_URL", "https://router.huggingface.co/v1")
+MODEL_NAME = os.environ.get("MODEL_NAME", "Qwen/Qwen2.5-72B-Instruct")
+API_KEY = os.environ.get("HF_TOKEN") or os.environ.get("API_KEY") or os.environ.get("OPENAI_API_KEY")
+ENV_URL = os.environ.get("ENV_URL", "").rstrip("/")
+AGENT = os.environ.get("AGENT", "llm" if API_KEY else "heuristic").lower()
+EPISODES_PER_TASK = int(os.environ.get("EPISODES_PER_TASK", "3"))
+MAX_WORKERS = int(os.environ.get("MAX_WORKERS", "3"))
+TIME_BUDGET_S = float(os.environ.get("TIME_BUDGET_S", "1050"))
+LLM_TIMEOUT_S = float(os.environ.get("LLM_TIMEOUT_S", "25"))
 
 BENCHMARK = "pll-cyberattack-detection"
-
-DEFAULT_ACTION = {
-    "attack_detected": False,
-    "attack_type": 0,
-    "confidence": 0.5,
-    "protective_action": 0,
+TASK_NAMES = {0: "sinusoidal_fdi_detection", 1: "attack_classification_under_disturbances",
+              2: "stealthy_phase_spoofing_mitigation"}
+# Fixed evaluation seeds → reproducible scenarios. Chosen so the first three per task already cover
+# the attack types and an attack-free episode (see scripts/evaluate_baselines.py for 100-seed results).
+EVAL_SEEDS = {
+    0: [1, 8, 2, 3, 9, 4],      # sinusoidal, none, sinusoidal, ...
+    1: [1, 3, 4, 7, 2, 5],      # sinusoidal, pulse, ramp, none, ...
+    2: [1, 4, 2, 3, 5, 6],      # spoofing, none, spoofing, ...
 }
 
-# ── System prompt ─────────────────────────────────────────────────────────────
-SYSTEM_PROMPT = """You are an AI agent monitoring a power grid inverter's Phase-Locked Loop (PLL).
-You receive time-windowed sensor readings each step and must detect cyberattacks.
-
-vq_window: q-axis voltage error (should be ~0 when healthy)
-vd_window: d-axis voltage
-omega_window: estimated frequency (normalized, nominal=0)
-omega_deviation_window: frequency deviation from nominal in rad/s (useful for detecting slow phase drift)
-raw_voltages: [va, vb, vc] at current step
-task_id: 0=detect only, 1=classify type, 2=detect stealthy attack
-
-For task_id=0: Focus on detecting any attack (attack_detected=True/False).
-For task_id=1: Also classify the attack type (1=sinusoidal, 2=ramp, 3=pulse).
-For task_id=2: Detect very subtle attacks before the PLL loses lock. Look for slow drifts in omega_deviation and vq.
-
-Analysis tips:
-- In healthy state, vq values should be near 0 and stable.
-- Sinusoidal attacks cause oscillating patterns in vq.
-- Ramp attacks cause steadily increasing vq magnitude.
-- Pulse attacks cause sudden step changes in vq.
-- Stealthy attacks cause very slow, gradual drift in omega_deviation_window.
-- Look at trends across the full window, not just the latest value.
-
-Respond ONLY with valid JSON, no explanation:
-{
-  "attack_detected": <bool>,
-  "attack_type": <int 0-4>,
-  "confidence": <float 0.0-1.0>,
-  "protective_action": <int 0-3>
-}"""
-
-# ── Logging helpers ───────────────────────────────────────────────────────────
-
-def log_start(task: str, env: str, model: str) -> None:
-    print(f"[START] task={task} env={env} model={model}", flush=True)
-
-
-def log_step(step: int, action: dict, reward: float, done: bool, error) -> None:
-    action_str = json.dumps(action, separators=(',', ':'))
-    error_val  = error if error else "null"
-    print(
-        f"[STEP] step={step} action={action_str} "
-        f"reward={reward:.2f} done={str(done).lower()} error={error_val}",
-        flush=True,
-    )
-
-
-def log_end(success: bool, steps: int, score: float, rewards: List[float]) -> None:
-    rewards_str = ",".join(f"{r:.2f}" for r in rewards)
-    print(
-        f"[END] success={str(success).lower()} steps={steps} "
-        f"score={score:.3f} rewards={rewards_str}",
-        flush=True,
-    )
-
-# ── Detector Agent & Smart Blending ───────────────────────────────────────────
-
-def detector_agent(prev_info: dict) -> Optional[dict]:
-    """Reads the environment's intrinsic physics-based detector output."""
-    det = prev_info.get("detector", {})
-    if not det or "attack_detected" not in det:
-        return None
-        
-    return {
-        "attack_detected": det.get("attack_detected", False),
-        "attack_type": det.get("attack_type", 0),
-        "confidence": det.get("confidence", 0.5),
-        "protective_action": det.get("protective_action", 0),
-    }
-
-def smart_blend_agent(obs: dict, prev_info: dict) -> dict:
-    """Uses detector if confident, else falls back to robust heuristic."""
-    heur_action = heuristic_agent(obs)
-    det_action = detector_agent(prev_info)
-    
-    if not det_action:
-        return heur_action
-    if det_action["confidence"] < 0.5:
-        return heur_action
-    
-    return det_action
+_t0 = time.monotonic()
+_print_lock = threading.Lock()
 
 
-# ── Rule-Based Heuristic Agent ────────────────────────────────────────────────
+def _err(msg: str) -> None:
+    print(msg, file=sys.stderr, flush=True)
 
-class HeuristicState:
-    """Tracks running state for the heuristic agent across steps."""
 
-    def __init__(self):
-        self.reset()
+# ---------------------------------------------------------------------------
+# Environment adapters (HTTP session or in-process)
+# ---------------------------------------------------------------------------
+class HTTPEnv:
+    def __init__(self, base_url: str, session_id: str):
+        import requests
+        self.s = requests.Session()
+        self.url = base_url
+        self.sid = session_id
 
-    def reset(self):
-        self.vq_history        = []
-        self.omega_dev_history = []
-        self.attack_detected   = False
-        self.predicted_type    = 0
-        self.settled_baseline  = None
-        self.peak_vq           = 0.0
+    def reset(self, task_id: int, seed: int) -> Dict[str, Any]:
+        r = self.s.post(f"{self.url}/reset", json={"task_id": task_id, "seed": seed, "session_id": self.sid}, timeout=60)
+        r.raise_for_status()
+        return r.json()["observation"]
 
+    def step(self, action: Dict[str, Any]) -> Tuple[Dict[str, Any], float, bool]:
+        r = self.s.post(f"{self.url}/step", json={"action": action, "session_id": self.sid}, timeout=60)
+        r.raise_for_status()
+        j = r.json()
+        return j["observation"], float(j["reward"] or 0.0), bool(j["done"])
 
-_hstate = HeuristicState()
 
+class LocalEnv:
+    def __init__(self):
+        from pll_cyber.environment import PLLCyberEnvironment
+        self.env = PLLCyberEnvironment()
 
-def heuristic_agent(obs: dict) -> dict:
-    """Rule-based fallback — only called when the LLM request fails."""
-    global _hstate
-
-    vq        = obs["vq_window"]
-    omega_dev = obs["omega_deviation_window"]
-    task_id   = obs["task_id"]
-    step      = obs["step"]
-
-    if step == 0:
-        _hstate.reset()
-
-    vq_abs         = [abs(v) for v in vq]
-    vq_mean        = sum(vq_abs) / len(vq_abs)
-    vq_max         = max(vq_abs)
-    omega_dev_abs  = [abs(v) for v in omega_dev]
-    omega_dev_mean = sum(omega_dev_abs) / len(omega_dev_abs)
-
-    _hstate.vq_history.append(vq_mean)
-    _hstate.omega_dev_history.append(omega_dev_mean)
-    _hstate.peak_vq = max(_hstate.peak_vq, vq_mean)
+    def reset(self, task_id: int, seed: int) -> Dict[str, Any]:
+        from pll_cyber.evaluation import obs_to_dict
+        return obs_to_dict(self.env.reset(task_id=task_id, seed=seed))
 
-    if step == 50:
-        _hstate.settled_baseline = omega_dev_mean
+    def step(self, action: Dict[str, Any]) -> Tuple[Dict[str, Any], float, bool]:
+        from pll_cyber.evaluation import obs_to_dict
+        from pll_cyber.models import PLLAction
+        o = self.env.step(PLLAction(**action))
+        return obs_to_dict(o), float(o.reward or 0.0), bool(o.done)
 
-    detected = False if step < 25 else (vq_mean > 0.01 or vq_max > 0.025)
-    if detected:
-        _hstate.attack_detected = True
 
-    # ── Task 0: binary detection ──────────────────────────────────────────────
-    if task_id == 0:
-        return {
-            "attack_detected": _hstate.attack_detected,
-            "attack_type": 1 if _hstate.attack_detected else 0,
-            "confidence": min(1.0, vq_mean * 50) if _hstate.attack_detected else 0.8,
-            "protective_action": 1 if _hstate.attack_detected else 0,
-        }
+def _server_up(url: str) -> bool:
+    if not url:
+        return False
+    try:
+        import requests
+        return requests.get(f"{url}/health", timeout=5).status_code == 200
+    except Exception:
+        return False
+
+
+# ---------------------------------------------------------------------------
+# LLM agent
+# ---------------------------------------------------------------------------
+SYSTEM_PROMPT = """You are the cyber-security guard of a grid-connected solar inverter. Every 50 ms you receive
+measurements from its Synchronous-Reference-Frame Phase-Locked Loop (SRF-PLL) and must decide whether a
+False-Data-Injection (FDI) attack on the voltage sensors is in progress.
+
+PHYSICS CHEAT-SHEET
+- Healthy grid: vd ≈ 1 pu, vq ≈ 0 (rms < 0.012), frequency ≈ 50 Hz (natural wander ±0.03 Hz), small 5th/7th harmonics.
+- Natural (NOT attacks): phase jumps (short vq spike), balanced sags (all phases dip), single-phase sags
+  (one phase dips; zero-sequence appears AT 50 Hz), harmonic steps, frequency events (RoCoF up to 0.6 Hz/s that the
+  trusted PMU also sees). These last ≤ 0.25 s or are mirrored by the PMU.
+- Attack 1 sinusoidal: interharmonic injection on one phase → zero-sequence content NOT at 50 Hz
+  (diagnostics.v0_offfund_rms high), persistent oscillation of local frequency.
+- Attack 2 ramp: growing DC offset on one phase (phase_dc_offset magnitude keeps increasing).
+- Attack 3 pulse: constant DC offset on one phase for 0.1-0.3 s, then gone. Real grids never have DC offsets.
+- Attack 4 phase spoofing: all three phases rotated together; vq stays clean, but the LOCAL frequency drifts away
+  from the PMU frequency. The PMU is trustworthy but delayed (see pmu.age_ms; it reports the average of the
+  100 ms ending age_ms ago), so compare the PMU with your local frequency from the same moment in trend.f_local_hz.
+  diagnostics.spoof_angle_est_deg integrates that mismatch: |angle| growing past ~3 deg is an attack.
+  The inverter desynchronises (damage) at 10 deg, typically 0.35-0.7 s after the attack starts.
+
+TASKS
+- task 0: detect a sinusoidal FDI (attack_type 1). task 1: detect AND classify (1/2/3) amid natural events.
+- task 2: detect phase spoofing (attack_type 4) and engage protective_action=2 (PMU-aided holdover) in time.
+Once an attack has started the system stays compromised: keep attack_detected=true for the rest of the episode.
+False alarms are heavily penalised; protective_action 2/3 on a healthy grid is costly; 3 (disconnect) is latched.
+
+Reply with ONLY a JSON object:
+{"attack_detected": bool, "attack_type": 0-4, "confidence": 0.0-1.0, "protective_action": 0-3, "reason": "<10 words"}"""
+
+
+def _diagnostics(obs: Dict[str, Any], tracker: PhysicsHeuristicAgent) -> Dict[str, float]:
+    """DSP helpers an engineer would compute on the raw observation (no ground truth involved)."""
+    d = _v0_decompose(obs)
+    tracker.act(obs)            # updates the latency-compensated spoofing angle estimate
+    return {"v0_dc_pu": round(d["dc"], 5), "v0_fundamental_pu": round(d["amp50"], 5),
+            "v0_offfund_rms_pu": round(d["resid_rms"], 5),
+            "spoof_angle_est_deg": round(tracker.angle_est_deg, 2)}
+
+
+def _user_message(obs: Dict[str, Any], diag: Dict[str, float], memory: List[str]) -> str:
+    payload = {
+        "task_id": obs["task_id"], "step": obs["step"], "time_s": obs["time_s"],
+        "inverter_mode": obs["inverter_mode"],
+        "features": obs["features"], "pmu": obs["pmu"], "trend": obs["trend"],
+        "diagnostics": diag, "your_recent_decisions": memory[-6:],
+    }
+    return json.dumps(payload, separators=(",", ":"))
 
-    # ── Task 1: classification ────────────────────────────────────────────────
-    if task_id == 1:
-        if not _hstate.attack_detected:
-            return {
-                "attack_detected": False,
-                "attack_type": 0,
-                "confidence": 0.7,
-                "protective_action": 0,
-            }
 
-        n_elevated = sum(1 for v in _hstate.vq_history if v > 0.01)
+_JSON_RE = re.compile(r"\{.*\}", re.S)
 
-        if n_elevated < 5:
-            attack_type = 1
-        else:
-            elevated = [v for v in _hstate.vq_history if v > 0.005]
-            recent   = elevated[-min(20, len(elevated)):]
-
-            current_vs_peak = vq_mean / _hstate.peak_vq if _hstate.peak_vq > 0 else 0
-            zero_crossings  = sum(1 for i in range(1, len(vq)) if vq[i] * vq[i - 1] < 0)
-
-            if len(recent) >= 6:
-                first_third = sum(recent[: len(recent) // 3]) / (len(recent) // 3)
-                last_third  = sum(recent[-len(recent) // 3 :]) / (len(recent) // 3)
-                growth      = last_third / first_third if first_third > 0.001 else 1.0
-            else:
-                growth = 1.0
-
-            if current_vs_peak < 0.15 and _hstate.peak_vq > 0.05:
-                attack_type = 3
-            elif current_vs_peak < 0.4 and n_elevated > 30:
-                attack_type = 3
-            elif zero_crossings >= 2 and growth < 1.5:
-                attack_type = 1
-            elif growth > 1.3:
-                attack_type = 2
-            elif zero_crossings >= 1:
-                attack_type = 1
-            else:
-                vq_diffs = [vq[i] - vq[i - 1] for i in range(1, len(vq))]
-                neg = sum(1 for d in vq_diffs if d < 0)
-                attack_type = 3 if neg > 14 else 1
-
-            _hstate.predicted_type = attack_type
-
-        return {
-            "attack_detected": True,
-            "attack_type": _hstate.predicted_type,
-            "confidence": 0.8,
-            "protective_action": 1,
-        }
-
-    # ── Task 2: stealthy attack ───────────────────────────────────────────────
-    if task_id == 2:
-        drift_detected = False
-        confidence     = 0.3
-
-        if step > 50 and _hstate.settled_baseline is not None:
-            baseline = _hstate.settled_baseline
-            ratio    = omega_dev_mean / baseline if baseline > 0.01 else omega_dev_mean * 100
-
-            if len(_hstate.omega_dev_history) > 10:
-                recent_10  = _hstate.omega_dev_history[-10:]
-                old_10     = (_hstate.omega_dev_history[-20:-10]
-                              if len(_hstate.omega_dev_history) > 20
-                              else _hstate.omega_dev_history[:10])
-                recent_avg = sum(recent_10) / len(recent_10)
-                old_avg    = sum(old_10) / len(old_10)
-                rising     = recent_avg > old_avg * 1.1
-            else:
-                rising = False
-
-            if ratio > 2.0:
-                drift_detected, confidence = True, 0.9
-            elif ratio > 1.3 and rising:
-                drift_detected, confidence = True, 0.8
-            elif rising and vq_mean > 0.1:
-                drift_detected, confidence = True, 0.6
-            elif vq_mean > 0.2:
-                drift_detected, confidence = True, 0.5
-
-        if drift_detected:
-            _hstate.attack_detected = True
 
+def parse_action(text: str) -> Optional[Dict[str, Any]]:
+    m = _JSON_RE.search(text or "")
+    if not m:
+        return None
+    try:
+        p = json.loads(m.group(0))
+        det = p.get("attack_detected", False)
+        det = det if isinstance(det, bool) else str(det).lower() == "true"
         return {
-            "attack_detected": drift_detected,
-            "attack_type": 4 if drift_detected else 0,
-            "confidence": confidence,
-            "protective_action": 2 if drift_detected else 0,
+            "attack_detected": det,
+            "attack_type": max(0, min(4, int(p.get("attack_type", 0)))),
+            "confidence": max(0.0, min(1.0, float(p.get("confidence", 0.5)))),
+            "protective_action": max(0, min(3, int(p.get("protective_action", 0)))),
         }
+    except (ValueError, TypeError, json.JSONDecodeError):
+        return None
 
-    return DEFAULT_ACTION.copy()
-
-# ── LLM agent (PRIMARY — always called first) ─────────────────────────────────
 
-def parse_llm_response(response_text: str) -> dict:
-    try:
-        text = response_text.strip()
-        if text.startswith("```"):
-            lines      = text.split("\n")
-            in_block   = False
-            json_lines: List[str] = []
-            for line in lines:
-                if line.strip().startswith("```") and not in_block:
-                    in_block = True
-                    continue
-                elif line.strip().startswith("```") and in_block:
+class LLMAgent:
+    _consecutive_failures = 0
+    _disabled = False
+    _lock = threading.Lock()
+
+    def __init__(self, client):
+        self.client = client
+        self.tracker = PhysicsHeuristicAgent()
+        self.fallback = PhysicsHeuristicAgent()
+        self.memory: List[str] = []
+        self.stats = {"llm": 0, "fallback": 0}
+
+    def act(self, obs: Dict[str, Any]) -> Tuple[Dict[str, Any], Optional[str]]:
+        diag = _diagnostics(obs, self.tracker)
+        fb = self.fallback.act(obs)
+        action, error = None, None
+        if not LLMAgent._disabled and time.monotonic() - _t0 < TIME_BUDGET_S:
+            for attempt in range(2):
+                try:
+                    resp = self.client.chat.completions.create(
+                        model=MODEL_NAME, temperature=0.0, max_tokens=120, seed=0, timeout=LLM_TIMEOUT_S,
+                        messages=[{"role": "system", "content": SYSTEM_PROMPT},
+                                  {"role": "user", "content": _user_message(obs, diag, self.memory)}],
+                    )
+                    action = parse_action(resp.choices[0].message.content)
+                    if action is None:
+                        error = "unparseable_llm_output"
+                    with LLMAgent._lock:
+                        LLMAgent._consecutive_failures = 0
                     break
-                elif in_block:
-                    json_lines.append(line)
-            text = "\n".join(json_lines)
-
-        parsed = json.loads(text)
-        return {
-            "attack_detected":   bool(parsed.get("attack_detected", False)),
-            "attack_type":       max(0, min(4, int(parsed.get("attack_type", 0)))),
-            "confidence":        max(0.0, min(1.0, float(parsed.get("confidence", 0.5)))),
-            "protective_action": max(0, min(3, int(parsed.get("protective_action", 0)))),
-        }
-    except (json.JSONDecodeError, KeyError, TypeError, ValueError):
-        return DEFAULT_ACTION.copy()
-
-
-def format_observation(obs: dict) -> str:
-    return "\n".join([
-        f"Step: {obs['step']}",
-        f"Task: {obs['task_id']}",
-        f"vq_window (last 20): {[round(v, 6) for v in obs['vq_window']]}",
-        f"vd_window (last 20): {[round(v, 6) for v in obs['vd_window']]}",
-        f"omega_window (last 20): {[round(v, 6) for v in obs['omega_window']]}",
-        f"omega_deviation_window (last 20): {[round(v, 6) for v in obs['omega_deviation_window']]}",
-        f"raw_voltages: {[round(v, 6) for v in obs['raw_voltages']]}",
-    ])
-
-_llm_disabled = False  # circuit breaker — flips True after first LLM failure
-
-
-def llm_agent(obs: dict, prev_info: dict) -> dict:
-    """Primary agent — calls the LLM through the injected proxy.
-    Falls back to smart blending if the API call itself raises an exception.
-    Uses a circuit breaker: after the first failure, all future calls skip the
-    network request and go straight to blending (restoring ~10s runtime).
-    """
-    global _llm_disabled
-    if _llm_disabled:
-        return smart_blend_agent(obs, prev_info)
-
-    try:
-        completion = client.chat.completions.create(
-            model=MODEL_NAME,
-            messages=[
-                {"role": "system", "content": SYSTEM_PROMPT},
-                {"role": "user",   "content": format_observation(obs)},
-            ],
-            temperature=0.1,
-            max_tokens=200,
-            timeout=10.0,
-        )
-        return parse_llm_response(completion.choices[0].message.content or "")
-    except Exception as e:
-        print(f"[WARN] LLM error ({type(e).__name__}: {e}), disabling LLM for remaining steps", file=sys.stderr, flush=True)
-        _llm_disabled = True
-        return smart_blend_agent(obs, prev_info)
+                except Exception as e:  # network / auth / rate limit
+                    error = f"{type(e).__name__}"
+                    if getattr(e, "status_code", None) in (401, 402, 403):   # permanent: bad key / no credits
+                        with LLMAgent._lock:
+                            if not LLMAgent._disabled:
+                                LLMAgent._disabled = True
+                                _err(f"[WARN] LLM unavailable (HTTP {e.status_code}); using heuristic fallback")
+                        break
+                    time.sleep(1.5 * (attempt + 1))
+            if action is None and error and error != "unparseable_llm_output":
+                with LLMAgent._lock:
+                    LLMAgent._consecutive_failures += 1
+                    if LLMAgent._consecutive_failures >= 5 and not LLMAgent._disabled:
+                        LLMAgent._disabled = True
+                        _err(f"[WARN] LLM disabled after repeated failures ({error}); using heuristic fallback")
+        elif LLMAgent._disabled:
+            error = "llm_disabled_fallback"
+        else:
+            error = "time_budget_fallback"
+        if action is None:
+            action = fb
+            self.stats["fallback"] += 1
+        else:
+            self.stats["llm"] += 1
+        self.memory.append(f"t={obs['time_s']:.2f}:det={int(action['attack_detected'])},type={action['attack_type']},"
+                           f"prot={action['protective_action']}")
+        return action, error
 
-# ── Episode runner ────────────────────────────────────────────────────────────
 
-def run_episode(task_id: int) -> float:
-    task_name = TASK_NAMES[task_id]
+class HeuristicAgentAdapter:
+    def __init__(self):
+        self.agent = PhysicsHeuristicAgent()
+        self.stats = {"llm": 0, "fallback": 0}
 
-    log_start(task=task_name, env=BENCHMARK, model=MODEL_NAME)
+    def act(self, obs: Dict[str, Any]) -> Tuple[Dict[str, Any], Optional[str]]:
+        return self.agent.act(obs), None
 
-    # Reset heuristic state before every episode so stale data from a previous
-    # task never bleeds into the next one (also covers the LLM fallback path).
-    _hstate.reset()
 
-    step_count   = 0
-    grader_score = 0.0
+# ---------------------------------------------------------------------------
+# Episode runner
+# ---------------------------------------------------------------------------
+def run_episode(task_id: int, seed: int, make_env, make_agent, label: str) -> Dict[str, Any]:
+    lines = [f"[START] task={TASK_NAMES[task_id]} env={BENCHMARK} model={label}"]
     rewards: List[float] = []
-    success      = False
-
+    score, success, steps = 0.0, False, 0
+    agent = make_agent()
     try:
-        reset_resp = _session.post(
-            f"{ENV_URL}/reset",
-            json={"task_id": task_id},
-            timeout=60,
-        )
-        reset_resp.raise_for_status()
-        obs = reset_resp.json()
-
-        done         = False
-        total_reward = 0.0
-        info         = {}
-
+        env = make_env(f"t{task_id}-s{seed}")
+        obs = env.reset(task_id, seed)
+        done = False
         while not done:
-            # Frame skipping: call LLM every 10 steps, heuristic in between.
-            # This caps LLM calls at ~150 total across 3 tasks, keeping runtime
-            # well under the 20-min judging limit even with 3s/call latency.
-            if step_count % 10 == 0:
-                action = llm_agent(obs, info)
-            else:
-                action = smart_blend_agent(obs, info)
-
-            step_resp = _session.post(
-                f"{ENV_URL}/step",
-                json=action,
-                timeout=60,
-            )
-            step_resp.raise_for_status()
-            result = step_resp.json()
-
-            obs         = result["observation"]
-            reward      = result["reward"]
-            done        = result["done"]
-            info        = result.get("info", {})
-            error       = result.get("error", None)
-
-            step_reward   = reward["total"] if isinstance(reward, dict) else float(reward)
-            total_reward += step_reward
-            rewards.append(step_reward)
-
-            step_count += 1
-            log_step(step=step_count, action=action, reward=step_reward, done=done, error=error)
-
-            if step_count % 50 == 0:
-                print(
-                    f"[DEBUG] step={step_count} cumulative_reward={total_reward:+.4f} "
-                    f"detected={action['attack_detected']} type={action['attack_type']}",
-                    file=sys.stderr, flush=True,
-                )
-
-        grader_score = info.get("grader_score", 0.01)
-        grader_score = max(0.01, min(0.99, grader_score))  # strict (0, 1)
-        success      = grader_score > 0.01
-
-    except Exception as exc:
-        print(f"[DEBUG] Episode error: {type(exc).__name__}: {exc}", file=sys.stderr, flush=True)
-        success = False
-    except BaseException as exc:
-        print(f"[DEBUG] Critical interruption: {type(exc).__name__}: {exc}", file=sys.stderr, flush=True)
-        success = False
-        raise
-
-    finally:
-        log_end(success=success, steps=step_count, score=grader_score, rewards=rewards)
-
-    return grader_score
-
-# ── Server Check ──────────────────────────────────────────────────────────────
-
-def wait_for_server(env_url: str, timeout: int = 60) -> bool:
-    print(f"[DEBUG] Waiting for environment server at {env_url} to start...", file=sys.stderr, flush=True)
-    start_t = time.time()
-    while time.time() - start_t < timeout:
-        try:
-            resp = _session.get(f"{env_url}/health", timeout=2)
-            if resp.status_code == 200:
-                print("[DEBUG] Environment server is up!", file=sys.stderr, flush=True)
-                return True
-        except Exception:
-            pass
-        time.sleep(1)
-    print(f"[DEBUG] Environment server failed to start within {timeout}s.", file=sys.stderr, flush=True)
-    return False
-
-# ── Entry point ───────────────────────────────────────────────────────────────
-
-def main() -> None:
-    print(f"[DEBUG] PLL Cyberattack Detection — model={MODEL_NAME} env={ENV_URL}", file=sys.stderr, flush=True)
-
-    if not wait_for_server(ENV_URL):
-        print("[DEBUG] Exiting due to server unavailable.", file=sys.stderr, flush=True)
-        return
+            action, error = agent.act(obs)
+            obs, reward, done = env.step(action)
+            steps += 1
+            rewards.append(reward)
+            lines.append(f"[STEP] step={steps} action={json.dumps(action, separators=(',', ':'))} "
+                         f"reward={reward:.2f} done={str(done).lower()} error={error or 'null'}")
+        res = obs.get("episode_result") or {}
+        score = float(res.get("grader_score", 0.0))
+        success = True
+    except Exception as e:
+        _err(f"[DEBUG] episode task={task_id} seed={seed} failed: {type(e).__name__}: {e}")
+    lines.append(f"[END] success={str(success).lower()} steps={steps} score={score:.3f} "
+                 f"rewards={','.join(f'{r:.2f}' for r in rewards)}")
+    with _print_lock:
+        print("\n".join(lines), flush=True)
+    return {"task_id": task_id, "seed": seed, "score": score, "success": success, "stats": agent.stats}
 
-    start_time = time.time()
-    scores: List[float] = []
 
-    try:
-        for task_id in range(3):
-            try:
-                score = run_episode(task_id)
-            except Exception as exc:
-                print(f"[DEBUG] run_episode({task_id}) crashed: {exc}", file=sys.stderr, flush=True)
-                score = 0.0
-            scores.append(score)
-            print(f"[DEBUG] task={task_id} score={score:.4f}", file=sys.stderr, flush=True)
-    except BaseException as exc:
-        print(f"[DEBUG] Process interrupted: {type(exc).__name__}: {exc}", file=sys.stderr, flush=True)
-
-    elapsed = time.time() - start_time
-    avg     = sum(scores) / len(scores) if scores else 0.0
-    print(f"[DEBUG] avg_score={avg:.4f} elapsed={elapsed:.1f}s", file=sys.stderr, flush=True)
+def main() -> None:
+    use_http = _server_up(ENV_URL)
+    if ENV_URL and not use_http:
+        _err(f"[WARN] ENV_URL={ENV_URL} unreachable — running the environment in-process")
+    make_env = (lambda sid: HTTPEnv(ENV_URL, sid)) if use_http else (lambda sid: LocalEnv())
+
+    if AGENT == "llm" and API_KEY:
+        from openai import OpenAI
+        client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY)
+        make_agent = lambda: LLMAgent(client)  # noqa: E731
+        label = MODEL_NAME
+    else:
+        if AGENT == "llm":
+            _err("[WARN] No HF_TOKEN/API_KEY set — running the physics-heuristic baseline instead of the LLM")
+        make_agent = HeuristicAgentAdapter
+        label = "physics-heuristic"
+
+    jobs = [(t, s) for t in range(3) for s in EVAL_SEEDS[t][:EPISODES_PER_TASK]]
+    _err(f"[DEBUG] agent={label} env={'http ' + ENV_URL if use_http else 'in-process'} episodes={len(jobs)}")
+    with ThreadPoolExecutor(max_workers=max(1, MAX_WORKERS)) as pool:
+        results = list(pool.map(lambda j: run_episode(j[0], j[1], make_env, make_agent, label), jobs))
+
+    print("\n=== Baseline scores ===", flush=True)
+    overall = []
+    for t in range(3):
+        sc = [r["score"] for r in results if r["task_id"] == t]
+        mean = sum(sc) / len(sc)
+        overall.append(mean)
+        print(f"task {t} ({TASK_NAMES[t]}): mean={mean:.3f} scores={[round(x, 3) for x in sc]}", flush=True)
+    llm_steps = sum(r["stats"]["llm"] for r in results)
+    fb_steps = sum(r["stats"]["fallback"] for r in results)
+    print(f"overall mean={sum(overall) / 3:.3f}  agent={label}  llm_steps={llm_steps} fallback_steps={fb_steps} "
+          f"elapsed={time.monotonic() - _t0:.1f}s", flush=True)
 
 
 if __name__ == "__main__":
-    main()
+    main()

inference_test.py

@@ -1,501 +0,0 @@
-"""
-Inference Script — PLL Cyberattack Detection OpenEnv
-=====================================================
-MANDATORY environment variables:
-  API_BASE_URL   The API endpoint for the LLM
-  MODEL_NAME     The model identifier to use
-  HF_TOKEN       Your Hugging Face / API key
-
-Uses a HYBRID approach:
-  - A fast rule-based heuristic agent runs by default (no LLM needed)
-  - The heuristic analyzes vq/omega_deviation windows to detect attacks
-  - Set USE_LLM=1 env var to use the LLM instead (slower, may fail)
-
-Must be named inference.py and placed at the project root.
-Uses OpenAI client for LLM calls when enabled.
-"""
-
-import os
-import json
-from typing import List, Optional
-import time
-import math
-import requests
-from openai import OpenAI
-
-API_BASE_URL = os.getenv("API_BASE_URL", "https://router.huggingface.co/v1")
-MODEL_NAME = os.getenv("MODEL_NAME", "Qwen/Qwen2.5-72B-Instruct")
-HF_TOKEN = os.getenv("HF_TOKEN")
-ENV_URL = os.getenv("ENV_URL", "https://krishuggingface-cyberattack-pll.hf.space")
-USE_LLM = os.environ.get("USE_LLM", "0") == "1"
-
-client = OpenAI(base_url=API_BASE_URL, api_key=HF_TOKEN)
-
-SYSTEM_PROMPT = """You are an AI agent monitoring a power grid inverter's Phase-Locked Loop (PLL).
-You receive time-windowed sensor readings each step and must detect cyberattacks.
-
-vq_window: q-axis voltage error (should be ~0 when healthy)
-vd_window: d-axis voltage
-omega_window: estimated frequency (normalized, nominal=0)
-omega_deviation_window: frequency deviation from nominal in rad/s (useful for detecting slow phase drift)
-raw_voltages: [va, vb, vc] at current step
-task_id: 0=detect only, 1=classify type, 2=detect stealthy attack
-
-For task_id=0: Focus on detecting any attack (attack_detected=True/False).
-For task_id=1: Also classify the attack type (1=sinusoidal, 2=ramp, 3=pulse).
-For task_id=2: Detect very subtle attacks before the PLL loses lock. Look for slow drifts in omega_deviation and vq.
-
-Analysis tips:
-- In healthy state, vq values should be near 0 and stable.
-- Sinusoidal attacks cause oscillating patterns in vq.
-- Ramp attacks cause steadily increasing vq magnitude.
-- Pulse attacks cause sudden step changes in vq.
-- Stealthy attacks cause very slow, gradual drift in omega_deviation_window.
-- Look at trends across the full window, not just the latest value.
-
-Respond ONLY with valid JSON, no explanation:
-{
-  "attack_detected": <bool>,
-  "attack_type": <int 0-4>,
-  "confidence": <float 0.0-1.0>,
-  "protective_action": <int 0-3>
-}"""
-
-TASK_NAMES = {
-    0: "Sinusoidal FDI Detection (Easy)",
-    1: "Multi-Attack Classification (Medium)",
-    2: "Stealthy Attack Detection (Hard)",
-}
-
-DEFAULT_ACTION = {
-    "attack_detected": False,
-    "attack_type": 0,
-    "confidence": 0.5,
-    "protective_action": 0,
-}
-
-
-# =====================================================================
-# Logging Helpers (OpenEnv compliance)
-# =====================================================================
-
-def log_start(task: str, env: str, model: str) -> None:
-    print(f"[START] task={task} env={env} model={model}", flush=True)
-
-
-def log_step(step: int, action: dict, reward: float, done: bool, error) -> None:
-    action_str = json.dumps(action, separators=(',', ':'))
-    error_val = error if error else "null"
-    print(f"[STEP] step={step} action={action_str} reward={reward:.2f} done={str(done).lower()} error={error_val}", flush=True)
-
-
-def log_end(success: bool, steps: int, score: float, rewards: list) -> None:
-    rewards_str = ",".join(f"{r:.2f}" for r in rewards)
-    print(f"[END] success={str(success).lower()} steps={steps} score={score:.3f} rewards={rewards_str}", flush=True)
-
-
-# =====================================================================
-# Detector-Based Agent
-# =====================================================================
-
-def detector_agent(prev_info: dict) -> Optional[dict]:
-    """Reads the environment's adaptive detector output from the previous step."""
-    det = prev_info.get("detector", {})
-    if not det or "attack_detected" not in det:
-        return None
-        
-    return {
-        "attack_detected": det.get("attack_detected", False),
-        "attack_type": det.get("attack_type", 0),
-        "confidence": det.get("confidence", 0.5),
-        "protective_action": det.get("protective_action", 0),
-    }
-
-
-# =====================================================================
-# Rule-Based Heuristic Agent
-# =====================================================================
-
-class HeuristicState:
-    """Tracks running state for the heuristic agent across steps."""
-    def __init__(self):
-        self.reset()
-
-    def reset(self):
-        self.vq_history = []           # all vq_mean(abs) values
-        self.omega_dev_history = []    # all omega_dev_mean(abs) values
-        self.attack_detected = False   # latched detection flag
-        self.predicted_type = 0        # latched classification
-        self.settled_baseline = None   # omega_dev baseline when PLL settles
-        self.peak_vq = 0.0            # highest vq_mean seen
-
-
-_hstate = HeuristicState()
-
-
-def heuristic_agent(obs: dict) -> dict:
-    """
-    Rule-based attack detector using cumulative state tracking.
-    No LLM needed — runs instantly.
-
-    The key insight is that the PLL's closed-loop response transforms
-    attack signals, so we track statistics over time rather than
-    trying to classify from a single 20-step vq window shape.
-    """
-    global _hstate
-    vq = obs["vq_window"]
-    omega_dev = obs["omega_deviation_window"]
-    task_id = obs["task_id"]
-    step = obs["step"]
-
-    if step == 0:
-        _hstate.reset()
-
-    # --- Compute per-step features ---
-    vq_abs = [abs(v) for v in vq]
-    vq_mean = sum(vq_abs) / len(vq_abs)
-    vq_max = max(vq_abs)
-    vq_latest = abs(vq[-1])
-
-    omega_dev_abs = [abs(v) for v in omega_dev]
-    omega_dev_mean = sum(omega_dev_abs) / len(omega_dev_abs)
-
-    # Track history
-    _hstate.vq_history.append(vq_mean)
-    _hstate.omega_dev_history.append(omega_dev_mean)
-    _hstate.peak_vq = max(_hstate.peak_vq, vq_mean)
-
-    # Record baseline around step 45-50 (PLL settled)
-    if step == 50:
-        _hstate.settled_baseline = omega_dev_mean
-
-    # -----------------------------------------------------------------
-    # Detection: is vq significantly elevated?
-    # After PLL warm-start settles (~step 20-30), healthy vq < 0.005
-    # -----------------------------------------------------------------
-    if step < 25:
-        # PLL still settling, don't detect
-        detected = False
-    else:
-        detected = vq_mean > 0.01 or vq_max > 0.025
-
-    # Latch detection on
-    if detected:
-        _hstate.attack_detected = True
-
-    # -----------------------------------------------------------------
-    # Task 0: Binary detection only
-    # -----------------------------------------------------------------
-    if task_id == 0:
-        return {
-            "attack_detected": _hstate.attack_detected,
-            "attack_type": 1 if _hstate.attack_detected else 0,
-            "confidence": min(1.0, vq_mean * 50) if _hstate.attack_detected else 0.8,
-            "protective_action": 1 if _hstate.attack_detected else 0,
-        }
-
-    # -----------------------------------------------------------------
-    # Task 1: Classification using cumulative patterns
-    # -----------------------------------------------------------------
-    if task_id == 1:
-        if not _hstate.attack_detected:
-            return {
-                "attack_detected": False,
-                "attack_type": 0,
-                "confidence": 0.7,
-                "protective_action": 0,
-            }
-
-        # Classify using cumulative vq_history
-        # Only classify after enough attack data (10+ steps of elevated vq)
-        n_elevated = sum(1 for v in _hstate.vq_history if v > 0.01)
-
-        if n_elevated < 5:
-            # Not enough data yet, use simple guess
-            attack_type = 1
-        else:
-            # Get recent vq trend (last 10 elevated values)
-            elevated = [v for v in _hstate.vq_history if v > 0.005]
-            recent = elevated[-min(20, len(elevated)):]
-
-            # Feature 1: Is vq currently high or has it decayed?
-            current_vs_peak = vq_mean / _hstate.peak_vq if _hstate.peak_vq > 0 else 0
-
-            # Feature 2: How many zero crossings in current window
-            zero_crossings = sum(1 for i in range(1, len(vq)) if vq[i] * vq[i-1] < 0)
-
-            # Feature 3: Is vq growing or shrinking over recent history
-            if len(recent) >= 6:
-                first_third = sum(recent[:len(recent)//3]) / (len(recent)//3)
-                last_third = sum(recent[-len(recent)//3:]) / (len(recent)//3)
-                growth = last_third / first_third if first_third > 0.001 else 1.0
-            else:
-                growth = 1.0
-
-            # Classification logic:
-            # Sinusoidal: persistent oscillation, zero crossings, stable amplitude
-            # Ramp: growing vq over time (growth > 1)
-            # Pulse: high initial vq that decays to near zero (current_vs_peak < 0.3)
-
-            if current_vs_peak < 0.15 and _hstate.peak_vq > 0.05:
-                # vq has decayed significantly from peak → pulse (ended)
-                attack_type = 3
-            elif current_vs_peak < 0.4 and n_elevated > 30:
-                # vq decayed after a long time → pulse
-                attack_type = 3
-            elif zero_crossings >= 2 and growth < 1.5:
-                # Active oscillation without growing → sinusoidal
-                attack_type = 1
-            elif growth > 1.3:
-                # Growing signal → ramp
-                attack_type = 2
-            elif zero_crossings >= 1:
-                # Some oscillation → sinusoidal
-                attack_type = 1
-            else:
-                # Default: if mono-decrease, pulse; else sinusoidal
-                vq_diffs = [vq[i] - vq[i-1] for i in range(1, len(vq))]
-                neg = sum(1 for d in vq_diffs if d < 0)
-                if neg > 14:  # 14/19 = 73% decreasing
-                    attack_type = 3
-                else:
-                    attack_type = 1
-
-            _hstate.predicted_type = attack_type
-
-        return {
-            "attack_detected": True,
-            "attack_type": _hstate.predicted_type,
-            "confidence": 0.8,
-            "protective_action": 1,
-        }
-
-    # -----------------------------------------------------------------
-    # Task 2: Stealthy attack — detect omega_dev rising above baseline
-    # -----------------------------------------------------------------
-    if task_id == 2:
-        drift_detected = False
-        confidence = 0.3
-
-        if step > 50 and _hstate.settled_baseline is not None:
-            baseline = _hstate.settled_baseline
-
-            # Compare current to baseline
-            ratio = omega_dev_mean / baseline if baseline > 0.01 else omega_dev_mean * 100
-
-            # Check if omega_dev is rising relative to recent history
-            if len(_hstate.omega_dev_history) > 10:
-                recent_10 = _hstate.omega_dev_history[-10:]
-                old_10 = _hstate.omega_dev_history[-20:-10] if len(_hstate.omega_dev_history) > 20 else _hstate.omega_dev_history[:10]
-                recent_avg = sum(recent_10) / len(recent_10)
-                old_avg = sum(old_10) / len(old_10)
-                rising = recent_avg > old_avg * 1.1
-            else:
-                rising = False
-
-            if ratio > 2.0:
-                drift_detected = True
-                confidence = 0.9
-            elif ratio > 1.3 and rising:
-                drift_detected = True
-                confidence = 0.8
-            elif rising and vq_mean > 0.1:
-                drift_detected = True
-                confidence = 0.6
-            elif vq_mean > 0.2:
-                drift_detected = True
-                confidence = 0.5
-
-        if drift_detected:
-            _hstate.attack_detected = True
-
-        return {
-            "attack_detected": drift_detected,
-            "attack_type": 4 if drift_detected else 0,
-            "confidence": confidence,
-            "protective_action": 2 if drift_detected else 0,
-        }
-
-    return DEFAULT_ACTION.copy()
-
-
-# =====================================================================
-# LLM Agent (optional, set USE_LLM=1)
-# =====================================================================
-
-def parse_llm_response(response_text: str) -> dict:
-    """Parse LLM response JSON, returning default action on failure."""
-    try:
-        text = response_text.strip()
-        if text.startswith("```"):
-            lines = text.split("\n")
-            json_lines = []
-            in_block = False
-            for line in lines:
-                if line.strip().startswith("```") and not in_block:
-                    in_block = True
-                    continue
-                elif line.strip().startswith("```") and in_block:
-                    break
-                elif in_block:
-                    json_lines.append(line)
-            text = "\n".join(json_lines)
-
-        parsed = json.loads(text)
-        action = {
-            "attack_detected": bool(parsed.get("attack_detected", False)),
-            "attack_type": max(0, min(4, int(parsed.get("attack_type", 0)))),
-            "confidence": max(0.0, min(1.0, float(parsed.get("confidence", 0.5)))),
-            "protective_action": max(0, min(3, int(parsed.get("protective_action", 0)))),
-        }
-        return action
-    except (json.JSONDecodeError, KeyError, TypeError, ValueError):
-        return DEFAULT_ACTION.copy()
-
-
-def format_observation(obs: dict) -> str:
-    """Format observation dict into a concise string for the LLM."""
-    parts = [
-        f"Step: {obs['step']}",
-        f"Task: {obs['task_id']}",
-        f"vq_window (last 20): {[round(v, 6) for v in obs['vq_window']]}",
-        f"vd_window (last 20): {[round(v, 6) for v in obs['vd_window']]}",
-        f"omega_window (last 20): {[round(v, 6) for v in obs['omega_window']]}",
-        f"omega_deviation_window (last 20): {[round(v, 6) for v in obs['omega_deviation_window']]}",
-        f"raw_voltages: {[round(v, 6) for v in obs['raw_voltages']]}",
-    ]
-    return "\n".join(parts)
-
-
-def llm_agent(obs: dict) -> dict:
-    """Call the LLM to decide an action. Falls back to heuristic on error."""
-    try:
-        obs_text = format_observation(obs)
-        completion = client.chat.completions.create(
-            model=MODEL_NAME,
-            messages=[
-                {"role": "system", "content": SYSTEM_PROMPT},
-                {"role": "user", "content": obs_text},
-            ],
-            temperature=0.1,
-            max_tokens=200,
-        )
-        llm_response = completion.choices[0].message.content
-        return parse_llm_response(llm_response)
-    except Exception as e:
-        print(f"    LLM error ({type(e).__name__}: {e}), falling back to heuristic")
-        return heuristic_agent(obs)
-
-
-# =====================================================================
-# Episode Runner
-# =====================================================================
-
-def run_episode(task_id: int) -> float:
-    log_start(task=TASK_NAMES[task_id], env="pll-cyberattack-detection", model=MODEL_NAME if USE_LLM else "rule-based-heuristic")
-
-    print(f"\n{'='*60}")
-    print(f"Task {task_id}: {TASK_NAMES[task_id]}")
-    print(f"Agent: {'LLM (' + MODEL_NAME + ')' if USE_LLM else 'Rule-Based Heuristic'}")
-    print(f"{'='*60}")
-
-    step_count = 0
-    grader_score = 0.0
-    rewards = []
-
-    try:
-        # Reset environment
-        reset_response = requests.post(
-            f"{ENV_URL}/reset",
-            json={"task_id": task_id},
-            timeout=30,
-        )
-        reset_response.raise_for_status()
-        obs = reset_response.json()
-
-        done = False
-        total_reward = 0.0
-        prev_info = {}
-
-        while not done:
-            # Choose agent
-            if USE_LLM:
-                action = llm_agent(obs)
-            else:
-                if step_count == 0:
-                    action = DEFAULT_ACTION.copy()
-                else:
-                    det_action = detector_agent(prev_info) if "detector" in prev_info else None
-                    heur_action = heuristic_agent(obs)
-                    
-                    if not det_action:
-                        action = heur_action
-                    elif det_action["confidence"] < 0.5:
-                        action = heur_action
-                    else:
-                        action = heur_action
-
-            # Step environment
-            step_response = requests.post(
-                f"{ENV_URL}/step",
-                json=action,
-                timeout=30,
-            )
-            step_response.raise_for_status()
-            result = step_response.json()
-
-            obs = result["observation"]
-            reward = result["reward"]
-            done = result["done"]
-            info = result["info"]
-            total_reward += reward["total"]
-            rewards.append(reward["total"])
-            log_step(step=step_count, action=action, reward=reward["total"], done=done, error=None)
-
-            prev_info = info
-            step_count += 1
-
-            # Print progress every 50 steps
-            if step_count % 50 == 0:
-                print(f"  Step {step_count:3d} | Reward: {reward['total']:+.4f} | "
-                      f"Cumulative: {total_reward:+.4f} | "
-                      f"Detected: {action['attack_detected']} | "
-                      f"Type: {action['attack_type']}")
-
-        # Extract grader score
-        grader_score = info.get("grader_score", 0.0)
-        print(f"\n  Episode complete: {step_count} steps")
-        print(f"  Total reward: {total_reward:+.4f}")
-        print(f"  Grader score: {grader_score:.4f}")
-    finally:
-        log_end(success=grader_score > 0.0, steps=step_count, score=grader_score, rewards=rewards)
-
-    return grader_score
-
-
-if __name__ == "__main__":
-    agent_name = f"LLM ({MODEL_NAME})" if USE_LLM else "Rule-Based Heuristic"
-    print("PLL Cyberattack Detection — Agentic Inference")
-    print(f"Agent: {agent_name}")
-    print(f"Environment: {ENV_URL}")
-    if not USE_LLM:
-        print("(Set USE_LLM=1 to use LLM agent instead of heuristic)")
-
-    start_time = time.time()
-    scores = []
-
-    for task_id in range(3):
-        score = run_episode(task_id)
-        print(f"Task {task_id} score: {score:.4f}")
-        scores.append(score)
-
-    elapsed = time.time() - start_time
-
-    print(f"\n{'='*60}")
-    print("FINAL RESULTS")
-    print(f"{'='*60}")
-    for i, score in enumerate(scores):
-        print(f"  Task {i} ({TASK_NAMES[i]}): {score:.4f}")
-    print(f"\n  Average score: {sum(scores)/len(scores):.4f}")
-    print(f"  Total time: {elapsed:.1f}s ({elapsed/60:.1f} min)")
-    print(f"{'='*60}")

openenv.yaml

@@ -1,65 +1,47 @@
+spec_version: 1
 name: pll-cyberattack-detection
-version: 1.0.0
+type: space
+runtime: fastapi
+app: server.app:app
+port: 7860
+
+# ---- Environment card ---------------------------------------------------
+version: 2.0.0
 description: >
-  Real-world OpenEnv for cyberattack detection on SRF-based 
-  Phase-Locked Loops in grid-connected inverters. An agent monitors 
-  sensor streams and detects False Data Injection attacks before 
-  they cause loss of grid synchronization.
-tags: [power-systems, cybersecurity, control-systems, openenv, pll, fdi]
-reward_range: [-2.2, 1.15]
-observation_space:
-  type: continuous
-  shape: [83]
+  Cyber-guard for a grid-connected inverter. An agent watches the signals of a
+  5 kHz-simulated SRF phase-locked loop and a delayed, trusted PMU stream, and must
+  detect, classify and mitigate false-data-injection attacks on the voltage sensors
+  without alarming on natural grid disturbances (phase jumps, sags, single-phase
+  faults, harmonic steps, frequency events).
+tags: [openenv, power-systems, cybersecurity, smart-grid, control-systems, pll, fdi, intrusion-detection]
+episode:
+  steps: 40
+  decision_interval_ms: 50
+  simulation_rate_hz: 5000
+reward_range: [-3.21, 0.69]
+action:
+  model: pll_cyber.models.PLLAction
   fields:
-    - name: vq_window
-      shape: [20]
-      description: q-axis voltage error signal (pu)
-    - name: vd_window
-      shape: [20]
-      description: d-axis voltage (pu)
-    - name: omega_window
-      shape: [20]
-      description: normalized frequency deviation from nominal
-    - name: omega_deviation_window
-      shape: [20]
-      description: frequency deviation from nominal (rad/s)
-    - name: raw_voltages
-      shape: [3]
-      description: raw three-phase voltages [va, vb, vc] (pu)
-action_space:
-  type: mixed
-  fields:
-    - name: attack_detected
-      type: bool
-    - name: attack_type
-      type: int
-      range: [0, 4]
-    - name: confidence
-      type: float
-      range: [0.0, 1.0]
-    - name: protective_action
-      type: int
-      range: [0, 3]
+    attack_detected: {type: bool}
+    attack_type: {type: int, range: [0, 4], meaning: "0 none, 1 sinusoidal, 2 ramp, 3 pulse, 4 phase spoofing"}
+    confidence: {type: float, range: [0.0, 1.0], scored_with: brier}
+    protective_action: {type: int, range: [0, 3], meaning: "0 none, 1 alert, 2 PMU-aided holdover, 3 disconnect (latched)"}
+observation:
+  model: pll_cyber.models.PLLObservation
+  groups: [waveforms, features, pmu, trend, reward_breakdown, episode_result]
 tasks:
-  - id: sinusoidal_fdi
+  - id: 0
+    name: sinusoidal_fdi_detection
     difficulty: easy
-    numeric_id: 0
-    grader: time_to_detection
-    max_score: 1.0
-    episode_length: 500
-    description: Detect sinusoidal FDI attack within 100 steps of attack start
-  - id: multi_attack_classification
+    grader: pll_cyber.graders.grade_easy
+    score_range: [0.0, 1.0]
+  - id: 1
+    name: attack_classification_under_disturbances
     difficulty: medium
-    numeric_id: 1
-    grader: classification_accuracy
-    max_score: 1.0
-    episode_length: 500
-    description: Classify attack type from observation window
-  - id: stealthy_attack_detection
+    grader: pll_cyber.graders.grade_medium
+    score_range: [0.0, 1.0]
+  - id: 2
+    name: stealthy_phase_spoofing_mitigation
     difficulty: hard
-    numeric_id: 2
-    grader: pre_lock_loss_detection
-    max_score: 1.0
-    episode_length: 500
-    description: Detect stealthy low-amplitude attack before PLL loss-of-lock
-episode_length: 500
+    grader: pll_cyber.graders.grade_hard
+    score_range: [0.0, 1.0]

pll_cyber/__init__.py

@@ -0,0 +1,7 @@
+"""PLL Cyber-Guard: an OpenEnv environment for cyber-attack detection on grid-inverter PLLs."""
+
+__version__ = "2.0.0"
+
+from .models import PLLAction, PLLObservation, PLLState, RewardBreakdown  # noqa: E402
+
+__all__ = ["PLLAction", "PLLObservation", "PLLState", "RewardBreakdown", "__version__"]

pll_cyber/attacks.py

@@ -0,0 +1,107 @@
+"""
+False Data Injection (FDI) attacks on the PLL's voltage-measurement channel.
+
+The attacker sits between the voltage transducers / ADC and the controller
+(compromised sensor firmware, a man-in-the-middle on the sampled-values bus,
+or malware on the control board) and rewrites the samples the PLL sees.
+
+    1  sinusoidal      — interharmonic injection on one phase: excites a
+                         persistent oscillation in the PLL frequency estimate
+    2  ramp            — slowly growing DC bias on one phase (sensor-offset
+                         style): a 50 Hz ripple in vq that grows over time
+    3  pulse           — step bias on one phase for a short burst
+    4  phase_spoofing  — coordinated, balanced manipulation of all three
+                         phases that rotates the measured voltage vector by a
+                         slowly accelerating angle δ(t). The PLL tracks the
+                         spoofed angle perfectly (vq stays ≈ 0) so classic
+                         residual detectors are blind, while the inverter
+                         drifts out of synchronism with the real grid. Its
+                         only local footprint is a small frequency deviation
+                         that is indistinguishable from a natural grid
+                         frequency event — unless it is cross-checked against
+                         a trusted reference (the PMU stream).
+
+Types 1–3 are "loud but harmless-ish"; type 4 is quiet and destructive.
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import asdict, dataclass, field
+from typing import Any, Dict, Optional
+
+import numpy as np
+
+from .pll import TWO_PI
+
+ATTACK_TYPES: Dict[int, str] = {
+    0: "none",
+    1: "sinusoidal",
+    2: "ramp",
+    3: "pulse",
+    4: "phase_spoofing",
+}
+ATTACK_IDS: Dict[str, int] = {v: k for k, v in ATTACK_TYPES.items()}
+
+
+@dataclass
+class AttackSpec:
+    kind: str = "none"
+    t_start: float = math.inf
+    params: Dict[str, float] = field(default_factory=dict)
+
+    @property
+    def type_id(self) -> int:
+        return ATTACK_IDS[self.kind]
+
+    @property
+    def present(self) -> bool:
+        return self.kind != "none"
+
+    def to_dict(self) -> Dict[str, Any]:
+        d = asdict(self)
+        d["type_id"] = self.type_id
+        if not math.isfinite(d["t_start"]):
+            d["t_start"] = None
+        return d
+
+
+class Attack:
+    """Evaluates an AttackSpec on a block of sample times."""
+
+    def __init__(self, spec: AttackSpec):
+        self.spec = spec
+
+    def angle_offset(self, t: np.ndarray) -> Optional[np.ndarray]:
+        """Spoofed angle δ(t) in rad added to all three measured phases, or None."""
+        s = self.spec
+        if s.kind != "phase_spoofing":
+            return None
+        tau = np.clip(t - s.t_start, 0.0, None)
+        r, df_max, sign = s.params["rocof_hz_s"], s.params["max_offset_hz"], s.params["direction"]
+        t_sat = df_max / r
+        # Frequency offset ramps at r until df_max, then holds; δ = 2π ∫ Δf dτ
+        ramp_part = 0.5 * r * np.minimum(tau, t_sat) ** 2
+        hold_part = df_max * np.clip(tau - t_sat, 0.0, None)
+        return sign * TWO_PI * (ramp_part + hold_part)
+
+    def additive(self, t: np.ndarray) -> Optional[np.ndarray]:
+        """Additive injection, shape (3, n), or None."""
+        s = self.spec
+        if s.kind in ("none", "phase_spoofing"):
+            return None
+        tau = t - s.t_start
+        active = tau >= 0.0
+        if not active.any():
+            return None
+        sig = np.zeros_like(t)
+        p = s.params
+        if s.kind == "sinusoidal":
+            sig = np.where(active, p["amplitude_pu"] * np.sin(TWO_PI * p["freq_hz"] * tau + p["phase"]), 0.0)
+        elif s.kind == "ramp":
+            sig = np.where(active, np.clip(p["rate_pu_s"] * tau, -p["cap_pu"], p["cap_pu"]), 0.0)
+        elif s.kind == "pulse":
+            sig = np.where(active & (tau < p["duration_s"]), p["magnitude_pu"], 0.0)
+        out = np.zeros((3, t.size))
+        out[int(p["phase_index"])] = sig
+        return out

pll_cyber/baselines.py

@@ -0,0 +1,195 @@
+"""
+Reference policies.
+
+`PhysicsHeuristicAgent` is a transparent, physics-informed baseline that a
+protection engineer could write by hand. It works on plain observation
+dicts, so the same code drives in-process, HTTP and WebSocket rollouts.
+
+  * Single-phase injection (tasks 0/1). A genuine grid has no DC component
+    and its only zero-sequence content comes from single-phase faults, which
+    sit at the 50 Hz fundamental. The agent fits v0 = (va+vb+vc)/3 with
+    DC + 50 Hz terms:
+        DC                   → bias attack (ramp if growing, pulse if not)
+        residual off 50 Hz   → sinusoidal interharmonic injection
+        50 Hz only           → single-phase sag (benign)
+  * Phase spoofing (task 2). The local frequency is compared with the PMU
+    frequency at the *same* instant (latency-compensated using the agent's
+    own history of local frequency). The mismatch is integrated into an
+    estimate of the accumulated angle error; holdover is engaged before it
+    approaches the 10° limit.
+
+Also included: trivial policies used by tests to prove the graders cannot
+be gamed.
+"""
+
+from __future__ import annotations
+
+import math
+import random
+from typing import Any, Dict, List, Optional
+
+import numpy as np
+
+WAVE_DT_S = 0.002          # exported waveform resolution
+F0 = 50.0
+
+
+def _v0_decompose(obs: Dict[str, Any]) -> Dict[str, float]:
+    w = obs["waveforms"]
+    v0 = (np.asarray(w["va"]) + np.asarray(w["vb"]) + np.asarray(w["vc"])) / 3.0
+    t = np.arange(v0.size) * WAVE_DT_S
+    X = np.column_stack([np.ones_like(t), np.cos(2 * np.pi * F0 * t), np.sin(2 * np.pi * F0 * t)])
+    coef, *_ = np.linalg.lstsq(X, v0, rcond=None)
+    resid = v0 - X @ coef
+    return {"dc": float(coef[0]), "amp50": float(math.hypot(coef[1], coef[2])),
+            "resid_rms": float(np.sqrt(np.mean(resid ** 2)))}
+
+
+class PhysicsHeuristicAgent:
+    # Thresholds (pu unless noted); noise floor of v0 residual is ≈0.003 pu
+    RESID_THR = 0.0075
+    DC_THR = 0.012
+    SPOOF_ANGLE_ALARM_DEG = 2.5
+    SPOOF_ANGLE_HOLDOVER_DEG = 3.5
+    SPOOF_RESID_HZ = 0.025
+    CONFIRM = 2
+
+    def __init__(self):
+        self.reset()
+
+    def reset(self) -> None:
+        self.latched = False
+        self.latched_type = 0
+        self.streak = 0
+        self.dc_hist: List[float] = []
+        self.resid_hist: List[float] = []
+        self.pulse_ended = False
+        self.f_hist: List[tuple] = []        # (t_center_s, f_local)
+        self.last_pmu: Optional[tuple] = None
+        self.angle_est_deg = 0.0
+        self.spoof_streak = 0
+
+    # ------------------------------------------------------------------
+    def act(self, obs: Dict[str, Any]) -> Dict[str, Any]:
+        if obs.get("step", 0) == 0:
+            self.reset()
+        task = obs["task_id"]
+        if task == 2:
+            return self._act_spoofing(obs)
+        return self._act_injection(obs, task)
+
+    # ------------------------------------------------------------------
+    def _act_injection(self, obs: Dict[str, Any], task: int) -> Dict[str, Any]:
+        d = _v0_decompose(obs)
+        thr_dc = self.DC_THR / 3.0                  # v0 carries one third of a single-phase offset
+        bias_now = abs(d["dc"]) > thr_dc
+        resid_now = d["resid_rms"] > self.RESID_THR
+        hit = bias_now or resid_now
+
+        self.streak = self.streak + 1 if hit else 0
+        if hit or self.latched:
+            self.dc_hist.append(d["dc"])
+            self.resid_hist.append(d["resid_rms"])
+        if not self.latched and self.streak >= self.CONFIRM:
+            self.latched = True
+        if self.latched and task == 1:
+            self.latched_type = self._classify(bias_now)
+
+        if not self.latched:
+            conf = 0.9 if self.streak == 0 else 0.6
+            return {"attack_detected": False, "attack_type": 0, "confidence": conf, "protective_action": 0}
+        atype = 1 if task == 0 else self.latched_type
+        return {"attack_detected": True, "attack_type": atype, "confidence": 0.9, "protective_action": 1}
+
+    def _classify(self, bias_now: bool) -> int:
+        """sinusoidal ↔ bias (ramp/pulse) from the v0 history since the alarm started."""
+        if self.latched_type == 3 and self.pulse_ended:
+            return 3
+        thr_dc = self.DC_THR / 3.0
+        active = [x for x in self.dc_hist if abs(x) > thr_dc]
+        signs = {np.sign(x) for x in active}
+        if len(signs) > 1:
+            return 1                                 # offset changes sign → oscillation, not a bias
+        if np.median(self.resid_hist) > self.RESID_THR:
+            return 1
+        if not bias_now and len(active) >= 1:
+            self.pulse_ended = True                  # a bias that disappears was a pulse
+            return 3
+        mags = [abs(x) for x in active]
+        if len(mags) >= 2:
+            growth = mags[-1] - mags[0]
+            if growth > 0.25 * max(mags):
+                return 2
+            if self.latched_type == 2 and len(mags) > 4:
+                return 2                             # ramp that has reached its saturation cap
+            return 3
+        return self.latched_type or 2
+
+    # ------------------------------------------------------------------
+    def _act_spoofing(self, obs: Dict[str, Any]) -> Dict[str, Any]:
+        f = obs["features"]
+        pmu = obs["pmu"]
+        t_now = obs["time_s"]
+        interval = obs.get("decision_interval_ms", 50.0) / 1000.0
+        self.f_hist.append((t_now - interval / 2.0, f["f_local_hz"]))
+        self.f_hist = self.f_hist[-20:]
+
+        # Only learn from a *new* PMU report
+        t_rep = t_now - pmu["age_ms"] / 1000.0 - 0.05     # report = average over its 100 ms interval
+        new_report = self.last_pmu is None or abs(t_rep - self.last_pmu[0]) > 1e-6
+        if new_report and len(self.f_hist) >= 3 and obs["inverter_mode"] == "normal":
+            ts = np.array([x[0] for x in self.f_hist])
+            fs = np.array([x[1] for x in self.f_hist])
+            if ts[0] <= t_rep:
+                # average local frequency over the same 100 ms interval the PMU measured
+                grid = np.linspace(t_rep - 0.05, t_rep + 0.05, 5)
+                f_loc = float(np.interp(grid, ts, fs).mean())
+                resid = f_loc - pmu["f_hz"]
+                dt = 0.1 if self.last_pmu is None else max(0.05, t_rep - self.last_pmu[0])
+                self.angle_est_deg = 0.9 * self.angle_est_deg + 360.0 * resid * dt
+                same_sign = np.sign(resid) == np.sign(self.angle_est_deg)
+                if abs(resid) > self.SPOOF_RESID_HZ and same_sign:
+                    self.spoof_streak += 1
+                else:
+                    self.spoof_streak = 0
+        if new_report:
+            self.last_pmu = (t_rep, pmu["f_hz"])
+
+        angle = abs(self.angle_est_deg)
+        if not self.latched and (angle > self.SPOOF_ANGLE_ALARM_DEG or self.spoof_streak >= 2):
+            self.latched = True
+        if not self.latched:
+            return {"attack_detected": False, "attack_type": 0, "confidence": 0.8, "protective_action": 0}
+        holdover = angle > self.SPOOF_ANGLE_HOLDOVER_DEG or self.spoof_streak >= 2 or obs["inverter_mode"] == "holdover"
+        return {"attack_detected": True, "attack_type": 4, "confidence": 0.85,
+                "protective_action": 2 if holdover else 1}
+
+
+# ---------------------------------------------------------------------------
+# Trivial policies (exploit checks)
+# ---------------------------------------------------------------------------
+
+class ConstantAgent:
+    def __init__(self, detected: bool, attack_type: int = 0, protective_action: int = 0):
+        self.a = {"attack_detected": detected, "attack_type": attack_type,
+                  "confidence": 1.0, "protective_action": protective_action}
+
+    def reset(self) -> None:
+        pass
+
+    def act(self, obs: Dict[str, Any]) -> Dict[str, Any]:
+        return dict(self.a)
+
+
+class RandomAgent:
+    def __init__(self, seed: int = 0, p_detect: float = 0.5):
+        self.rng = random.Random(seed)
+        self.p = p_detect
+
+    def reset(self) -> None:
+        pass
+
+    def act(self, obs: Dict[str, Any]) -> Dict[str, Any]:
+        d = self.rng.random() < self.p
+        return {"attack_detected": d, "attack_type": self.rng.randint(0, 4) if d else 0,
+                "confidence": self.rng.random(), "protective_action": self.rng.choice([0, 0, 1, 2])}

pll_cyber/client.py

@@ -0,0 +1,34 @@
+"""
+Typed OpenEnv client (persistent WebSocket session).
+
+    from pll_cyber.client import PLLCyberEnv
+    from pll_cyber.models import PLLAction
+
+    with PLLCyberEnv(base_url="http://localhost:7860").sync() as env:
+        result = env.reset(task_id=2, seed=7)
+        while not result.done:
+            result = env.step(PLLAction(attack_detected=False))
+        print(result.observation.episode_result.grader_score)
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict
+
+from openenv.core import EnvClient
+from openenv.core.client_types import StepResult
+
+from .models import PLLAction, PLLObservation, PLLState
+
+
+class PLLCyberEnv(EnvClient[PLLAction, PLLObservation, PLLState]):
+    def _step_payload(self, action: PLLAction) -> Dict[str, Any]:
+        return action.model_dump(exclude={"metadata"})
+
+    def _parse_result(self, payload: Dict[str, Any]) -> StepResult[PLLObservation]:
+        obs = PLLObservation(**payload.get("observation", {}),
+                             done=payload.get("done", False), reward=payload.get("reward"))
+        return StepResult(observation=obs, reward=payload.get("reward"), done=payload.get("done", False))
+
+    def _parse_state(self, payload: Dict[str, Any]) -> PLLState:
+        return PLLState(**payload)

pll_cyber/environment.py

@@ -0,0 +1,323 @@
+"""
+PLL Cyber-Guard environment (OpenEnv `Environment` implementation).
+
+One episode = 40 decisions × 50 ms of a grid-connected inverter. Each step:
+
+    1. the agent's action a_t is scored against window t (what it observed)
+    2. the protective action is applied (normal / holdover / disconnect)
+    3. the next 50 ms window is simulated at 5 kHz:
+         grid → FDI attack → sensors+ADC → SRF-PLL → features
+       alongside a "shadow" PLL that never mitigates, which gives the
+       counterfactual time at which the unprotected inverter would have
+       lost synchronism with the real grid
+    4. physical harm (sustained >10° desynchronisation) is charged
+"""
+
+from __future__ import annotations
+
+import math
+import secrets
+import uuid
+from collections import deque
+from typing import Any, Dict, List, Optional
+
+import numpy as np
+from openenv.core.env_server.interfaces import Environment
+from openenv.core.env_server.types import EnvironmentMetadata
+
+from . import __version__
+from .attacks import Attack, ATTACK_TYPES
+from .features import FeatureExtractor
+from .graders import grade_episode
+from .grid import GridModel
+from .models import (EpisodeResult, PLLAction, PLLObservation, PLLState, PMUReading,
+                     RewardBreakdown, SignalFeatures, Trend, Waveforms)
+from .pll import F_NOMINAL, SRFPLL
+from .rewards import StepOutcome, compute_reward
+from .scenarios import (ADC_LSB_PU, DECISION_INTERVAL_S, DT, MAX_STEPS, PMU_F_NOISE_HZ,
+                        PMU_REPORT_INTERVAL_S, PMU_V_NOISE_PU, SAMPLE_RATE_HZ,
+                        SAMPLES_PER_DECISION, SENSOR_NOISE_PU, TASKS, WARMUP_S,
+                        Scenario, sample_scenario)
+from .sensors import DisciplinedHoldover, PMUStream, measure
+
+DESYNC_THRESHOLD_DEG = 10.0
+DESYNC_SUSTAIN_S = 0.030
+_DESYNC_RAD = math.radians(DESYNC_THRESHOLD_DEG)
+_DESYNC_SAMPLES = int(round(DESYNC_SUSTAIN_S * SAMPLE_RATE_HZ))
+_INTERVAL_MS = DECISION_INTERVAL_S * 1000.0
+
+
+class _DesyncTracker:
+    """Detects sustained loss of synchronism: |θ̂ − θ| > 10° for ≥ 30 ms."""
+
+    def __init__(self):
+        self.run = 0
+        self.first_step: Optional[int] = None
+        self.max_err_deg = 0.0
+
+    def update(self, theta_hat: List[float], theta_true: np.ndarray, step: int, active: bool = True) -> bool:
+        err = np.abs(np.angle(np.exp(1j * (np.asarray(theta_hat) - theta_true))))
+        if not active:
+            self.run = 0
+            return False
+        self.max_err_deg = max(self.max_err_deg, float(np.degrees(err.max())))
+        hit = False
+        run = self.run
+        for above in (err > _DESYNC_RAD):
+            run = run + 1 if above else 0
+            if run >= _DESYNC_SAMPLES:
+                hit = True
+        self.run = run
+        if hit and self.first_step is None:
+            self.first_step = step
+        return hit
+
+
+class PLLCyberEnvironment(Environment[PLLAction, PLLObservation, PLLState]):
+    SUPPORTS_CONCURRENT_SESSIONS = True
+
+    def __init__(self):
+        super().__init__()
+        self._sc: Optional[Scenario] = None
+        self._state = PLLState(episode_id=None, step_count=0)
+
+    # ------------------------------------------------------------------
+    # OpenEnv API
+    # ------------------------------------------------------------------
+    def reset(self, seed: Optional[int] = None, episode_id: Optional[str] = None,
+              task_id: Optional[int] = None, task: Optional[str] = None, **kwargs: Any) -> PLLObservation:
+        tid = self._resolve_task(task_id, task)
+        if seed is None:
+            seed = secrets.randbelow(2 ** 31)
+        seed = int(seed)
+
+        sc = sample_scenario(tid, seed)
+        self._sc = sc
+        ss = np.random.SeedSequence([seed, 2000 + tid])
+        rng_sensor, rng_pmu = (np.random.default_rng(s) for s in ss.spawn(2))
+        self._rng_sensor = rng_sensor
+
+        self._grid = GridModel(sc.grid, theta0=sc.theta0)
+        self._attack = Attack(sc.attack)
+        self._pll = SRFPLL(DT, theta0=sc.theta0)
+        self._shadow = SRFPLL(DT, theta0=sc.theta0)
+        self._pmu = PMUStream(PMU_REPORT_INTERVAL_S, sc.pmu_latency_s, PMU_F_NOISE_HZ,
+                              PMU_V_NOISE_PU, rng_pmu)
+        self._fx = FeatureExtractor(SAMPLE_RATE_HZ, SAMPLES_PER_DECISION)
+        self._desync = _DesyncTracker()
+        self._desync_shadow = _DesyncTracker()
+        self._holdover = DisciplinedHoldover()
+        self._theta_hist: deque = deque(maxlen=6)     # (t0, θ̂ samples) of recent windows
+
+        self._mode = "normal"
+        self._step = 0
+        self._done = False
+        self._had_tp = False
+        self._log: List[Dict[str, Any]] = []
+        self._cum_reward = 0.0
+        self._result: Optional[EpisodeResult] = None
+
+        # Warm-up: let the PLL settle; the last warm-up window is observation 0.
+        n_warm = int(round(WARMUP_S / DECISION_INTERVAL_S))
+        for k in range(n_warm):
+            window = self._simulate_window(step=k - n_warm + 1, warmup=True)
+
+        self._state = PLLState(
+            episode_id=episode_id or str(uuid.uuid4()), step_count=0, task_id=tid,
+            task_name=TASKS[tid].name, seed=seed, done=False, time_s=0.0,
+            inverter_mode="normal", cumulative_reward=0.0,
+        )
+        obs = self._observation(window, reward=None, breakdown=None)
+        obs.task_brief = TASKS[tid].brief
+        obs.message = f"Episode started: task {tid} ({TASKS[tid].difficulty}), seed {seed}."
+        return obs
+
+    def step(self, action: PLLAction, timeout_s: Optional[float] = None, **kwargs: Any) -> PLLObservation:
+        if self._sc is None:
+            raise RuntimeError("Call reset() before step().")
+        if self._done:
+            obs = self._observation(self._last_window, reward=0.0, breakdown=RewardBreakdown())
+            obs.message = "Episode already finished — call reset() to start a new one."
+            return obs
+
+        sc = self._sc
+        t = self._step
+        onset = sc.onset_step
+        label = onset is not None and t >= onset
+        detected = bool(action.attack_detected)
+        first_tp = label and detected and not self._had_tp
+        self._had_tp = self._had_tp or (label and detected)
+
+        tripped_now = action.protective_action == 3 and self._mode != "disconnected"
+        if self._mode != "disconnected":
+            self._mode = {3: "disconnected", 2: "holdover"}.get(action.protective_action, "normal")
+
+        window = self._simulate_window(step=t + 1)
+
+        outcome = StepOutcome(
+            label_attack=label, true_type=sc.attack.type_id,
+            steps_since_onset=(t - onset) if label else 0,
+            detected=detected, reported_type=int(action.attack_type),
+            confidence=float(action.confidence), first_true_positive=first_tp,
+            classify=sc.task_id >= 1, tripped_now=tripped_now, mode_next=self._mode,
+            desync_in_next_window=window["desync"], first_desync=window["first_desync"],
+            decision_interval_ms=_INTERVAL_MS,
+        )
+        breakdown = compute_reward(outcome)
+        self._cum_reward += breakdown.total
+
+        self._log.append({
+            "step": t, "label": label, "detected": detected,
+            "attack_type": int(action.attack_type), "confidence": float(action.confidence),
+            "protective_action": int(action.protective_action), "reward": breakdown.total,
+        })
+        self._step += 1
+        self._done = self._step >= MAX_STEPS
+        if self._done:
+            self._finish()
+
+        self._state.step_count = self._step
+        self._state.done = self._done
+        self._state.time_s = round(self._step * DECISION_INTERVAL_S, 3)
+        self._state.inverter_mode = self._mode
+        self._state.cumulative_reward = round(self._cum_reward, 6)
+
+        obs = self._observation(window, reward=breakdown.total, breakdown=breakdown)
+        if self._done:
+            obs.message = f"Episode finished. grader_score={self._result.grader_score:.3f}"
+        return obs
+
+    @property
+    def state(self) -> PLLState:
+        return self._state
+
+    def get_metadata(self) -> EnvironmentMetadata:
+        return EnvironmentMetadata(
+            name="pll-cyberattack-detection",
+            description=("Cyber-guard for a grid-connected inverter: detect, classify and mitigate false-data-"
+                         "injection attacks on an SRF-PLL, without alarming on natural grid disturbances."),
+            version=__version__,
+            author="Kris Keshav",
+            documentation_url="https://github.com/KrisKeshav/PLL-Cyberattack-Detection-OpenEnv",
+        )
+
+    # ------------------------------------------------------------------
+    # Internals
+    # ------------------------------------------------------------------
+    @staticmethod
+    def _resolve_task(task_id: Optional[int], task: Optional[str]) -> int:
+        if task is not None:
+            for tid, spec in TASKS.items():
+                if task in (spec.name, spec.difficulty, str(tid)):
+                    return tid
+            raise ValueError(f"Unknown task '{task}'. Options: {[s.name for s in TASKS.values()]}")
+        tid = 0 if task_id is None else int(task_id)
+        if tid not in TASKS:
+            raise ValueError(f"task_id must be one of {sorted(TASKS)}")
+        return tid
+
+    def _simulate_window(self, step: int, warmup: bool = False) -> Dict[str, Any]:
+        n = SAMPLES_PER_DECISION
+        grid, attack = self._grid, self._attack
+        t_next = grid.t + DT * np.arange(1, n + 1)
+        out = grid.generate(n, DT, attack.angle_offset(t_next))
+        v = out["v"]
+        add = attack.additive(out["t"])
+        if add is not None:
+            v = v + add
+        vm = measure(v, self._rng_sensor, SENSOR_NOISE_PU, ADC_LSB_PU)
+
+        self._pmu.ingest(out["t"], out["theta"], out["v_pos"])
+        now = float(out["t"][-1])
+        rep = self._pmu.latest(now)
+
+        holdover = self._mode == "holdover"
+        if holdover:
+            # θ̂ held by the PLL is the angle for the next sample instant, out["t"][0]
+            self._update_holdover(t_next=float(out["t"][0]))
+        elif self._holdover.active:
+            self._holdover.release()
+        r = self._pll.run(vm[0], vm[1], vm[2], holdover=holdover,
+                          holdover_freq_hz=self._holdover.f_hold,
+                          holdover_rocof_hz_s=self._holdover.rocof)
+        if holdover:      # carry the extrapolated frequency ramp into the next window
+            self._holdover.f_hold += self._holdover.rocof * n * DT
+        rs = self._shadow.run(vm[0], vm[1], vm[2])
+        self._theta_hist.append((float(out["t"][0]), r.theta_hat))
+
+        connected = self._mode != "disconnected"
+        had_desync = self._desync.first_step is not None
+        desync = False
+        if not warmup:
+            desync = self._desync.update(r.theta_hat, out["theta"], step, active=connected)
+            self._desync_shadow.update(rs.theta_hat, out["theta"], step)
+
+        pmu_f = rep.f_hz if rep is not None else F_NOMINAL
+        fx = self._fx.update(vm, r.vd, r.vq, r.f_hat, pmu_f)
+        window = {
+            **fx,
+            "pmu": {
+                "f_hz": round(pmu_f, 4),
+                "v_pos_pu": round(rep.v_pos_pu if rep is not None else 1.0, 4),
+                "age_ms": round((now - rep.t_end) * 1000.0, 1) if rep is not None else 999.0,
+            },
+            "desync": desync,
+            "first_desync": desync and not had_desync,
+        }
+        self._last_window = window
+        return window
+
+    def _theta_hat_at(self, t: float) -> Optional[float]:
+        for t0, th in reversed(self._theta_hist):
+            i = int(round((t - t0) / DT))
+            if 0 <= i < len(th):
+                return th[i]
+        return None
+
+    def _update_holdover(self, t_next: float) -> None:
+        """Engage / steer the PMU-disciplined oscillator before simulating a window."""
+        if not self._holdover.active:
+            self._holdover.engage(t_next)
+        self._holdover.update(self._pll, self._pmu, t_next)
+
+    def _finish(self) -> None:
+        sc = self._sc
+        graded = grade_episode(
+            sc.task_id, self._log, sc.onset_step, _INTERVAL_MS,
+            true_type=sc.attack.type_id,
+            unmitigated_desync_step=self._desync_shadow.first_step,
+            actual_desync_step=self._desync.first_step,
+        )
+        scenario = sc.to_dict()
+        scenario["unmitigated_desync_step"] = self._desync_shadow.first_step
+        scenario["actual_desync_step"] = self._desync.first_step
+        scenario["max_sync_error_deg"] = round(self._desync.max_err_deg, 3)
+        scenario["unmitigated_max_sync_error_deg"] = round(self._desync_shadow.max_err_deg, 3)
+        scenario["attack_type_name"] = ATTACK_TYPES[sc.attack.type_id]
+        self._result = EpisodeResult(
+            grader_score=graded["score"], score_breakdown=graded,
+            total_reward=round(self._cum_reward, 6), scenario=scenario,
+        )
+        self._state.grader_score = graded["score"]
+        self._state.scenario = scenario
+
+    def _observation(self, window: Dict[str, Any], reward: Optional[float],
+                     breakdown: Optional[RewardBreakdown]) -> PLLObservation:
+        sc = self._sc
+        return PLLObservation(
+            done=self._done,
+            reward=None if reward is None else round(reward, 6),
+            task_id=sc.task_id,
+            task_name=TASKS[sc.task_id].name,
+            step=self._step,
+            max_steps=MAX_STEPS,
+            time_s=round(self._step * DECISION_INTERVAL_S, 3),
+            decision_interval_ms=_INTERVAL_MS,
+            inverter_mode=self._mode,
+            waveforms=Waveforms(**window["waveforms"]),
+            features=SignalFeatures(**window["features"]),
+            pmu=PMUReading(**window["pmu"]),
+            trend=Trend(**window["trend"]),
+            reward_breakdown=breakdown,
+            episode_result=self._result if self._done else None,
+        )

pll_cyber/evaluation.py

@@ -0,0 +1,48 @@
+"""In-process rollout helpers (used by tests, the benchmark script and inference.py)."""
+
+from __future__ import annotations
+
+from typing import Any, Callable, Dict, Iterable, List
+
+import numpy as np
+
+from .environment import PLLCyberEnvironment
+from .models import PLLAction
+
+
+def obs_to_dict(obs) -> Dict[str, Any]:
+    return obs.model_dump(exclude={"metadata"})
+
+
+def run_episode(agent, task_id: int, seed: int, env: PLLCyberEnvironment | None = None,
+                on_step: Callable[[int, Dict[str, Any], Any], None] | None = None) -> Dict[str, Any]:
+    env = env or PLLCyberEnvironment()
+    obs = env.reset(seed=seed, task_id=task_id)
+    if hasattr(agent, "reset"):
+        agent.reset()
+    rewards: List[float] = []
+    while not obs.done:
+        action = agent.act(obs_to_dict(obs))
+        obs = env.step(PLLAction(**action))
+        rewards.append(float(obs.reward or 0.0))
+        if on_step:
+            on_step(len(rewards), action, obs)
+    res = obs.episode_result
+    return {"task_id": task_id, "seed": seed, "score": res.grader_score,
+            "total_reward": res.total_reward, "rewards": rewards,
+            "breakdown": res.score_breakdown, "scenario": res.scenario}
+
+
+def evaluate(make_agent: Callable[[], Any], task_ids: Iterable[int], seeds: Iterable[int]) -> Dict[int, Dict[str, float]]:
+    env = PLLCyberEnvironment()
+    out: Dict[int, Dict[str, float]] = {}
+    seeds = list(seeds)
+    for tid in task_ids:
+        scores, rets = [], []
+        for s in seeds:
+            r = run_episode(make_agent(), tid, s, env)
+            scores.append(r["score"])
+            rets.append(r["total_reward"])
+        out[tid] = {"mean": float(np.mean(scores)), "std": float(np.std(scores)),
+                    "return": float(np.mean(rets)), "n": len(seeds)}
+    return out

pll_cyber/features.py

@@ -0,0 +1,108 @@
+"""
+On-board signal processing: turns raw per-sample signals of a decision window
+into the compact, physically meaningful features exposed to the agent.
+"""
+
+from __future__ import annotations
+
+from collections import deque
+from typing import Deque, Dict, List
+
+import numpy as np
+
+from .pll import F_NOMINAL
+
+TREND_LEN = 10
+FFT_BUFFER_S = 0.200
+DECIMATE = 10          # 5 kHz → 500 Hz for the exported waveforms (25 points / window)
+
+
+def _r(x: float, nd: int = 5) -> float:
+    return float(round(float(x), nd))
+
+
+def _rl(xs, nd: int = 4) -> List[float]:
+    return [float(round(float(x), nd)) for x in xs]
+
+
+class FeatureExtractor:
+    def __init__(self, sample_rate_hz: float, samples_per_window: int):
+        self.fs = sample_rate_hz
+        self.n = samples_per_window
+        fft_len = int(FFT_BUFFER_S * sample_rate_hz) // 5          # vq decimated ×5 → 1 kHz
+        self.vq_buf: Deque[float] = deque(maxlen=fft_len)
+        self.f_means: Deque[float] = deque(maxlen=TREND_LEN)
+        self.pmu_f: Deque[float] = deque(maxlen=TREND_LEN)
+        self.vq_rms: Deque[float] = deque(maxlen=TREND_LEN)
+        self.v0_rms: Deque[float] = deque(maxlen=TREND_LEN)
+        self.vpos: Deque[float] = deque(maxlen=TREND_LEN)
+
+    def update(self, v_meas: np.ndarray, vd: List[float], vq: List[float], f_hat: List[float],
+               pmu_f: float) -> Dict[str, object]:
+        vd_a, vq_a, f_a = np.asarray(vd), np.asarray(vq), np.asarray(f_hat)
+
+        self.vq_buf.extend(vq_a[::5].tolist())
+        vq_rms = float(np.sqrt(np.mean(vq_a ** 2)))
+        v0 = v_meas.sum(axis=0) / 3.0
+        v0_rms = float(np.sqrt(np.mean(v0 ** 2)))
+        f_mean = float(f_a.mean())
+        vpos = float(vd_a.mean())
+
+        self.f_means.append(f_mean)
+        self.pmu_f.append(pmu_f)
+        self.vq_rms.append(vq_rms)
+        self.v0_rms.append(v0_rms)
+        self.vpos.append(vpos)
+
+        # RoCoF from the slope of window-mean frequency over the last 4 windows
+        fm = np.asarray(self.f_means)[-4:]
+        if fm.size >= 2:
+            rocof = float(np.polyfit(np.arange(fm.size) * (self.n / self.fs), fm, 1)[0])
+        else:
+            rocof = 0.0
+
+        # Dominant vq oscillation (Hann-windowed FFT over the last 200 ms at 1 kHz)
+        buf = np.asarray(self.vq_buf)
+        osc_f, osc_a = 0.0, 0.0
+        if buf.size >= 32:
+            x = (buf - buf.mean()) * np.hanning(buf.size)
+            spec = np.abs(np.fft.rfft(x)) * 2.0 / (np.hanning(buf.size).sum())
+            freqs = np.fft.rfftfreq(buf.size, d=5.0 / self.fs)
+            band = (freqs >= 3.0) & (freqs <= 400.0)
+            if band.any():
+                i = int(np.argmax(np.where(band, spec, 0.0)))
+                osc_f, osc_a = float(freqs[i]), float(spec[i])
+
+        two_cycles = int(round(2.0 * self.fs / F_NOMINAL))
+        dc = v_meas[:, -two_cycles:].mean(axis=1)
+
+        features = {
+            "v_pos_seq_pu": _r(vpos),
+            "vq_mean_pu": _r(vq_a.mean()),
+            "vq_rms_pu": _r(vq_rms),
+            "f_local_hz": _r(f_mean),
+            "f_local_std_hz": _r(f_a.std()),
+            "rocof_local_hz_s": _r(rocof, 4),
+            "v_zero_seq_rms_pu": _r(v0_rms),
+            "phase_rms_pu": _rl(np.sqrt(np.mean(v_meas ** 2, axis=1)), 5),
+            "phase_dc_offset_pu": _rl(dc, 5),
+            "vq_osc_freq_hz": _r(osc_f, 1),
+            "vq_osc_amp_pu": _r(osc_a),
+            "f_local_minus_pmu_hz": _r(f_mean - pmu_f),
+        }
+        waveforms = {
+            "va": _rl(v_meas[0, DECIMATE - 1::DECIMATE]),
+            "vb": _rl(v_meas[1, DECIMATE - 1::DECIMATE]),
+            "vc": _rl(v_meas[2, DECIMATE - 1::DECIMATE]),
+            "vd": _rl(vd_a[DECIMATE - 1::DECIMATE]),
+            "vq": _rl(vq_a[DECIMATE - 1::DECIMATE]),
+            "f_hz": _rl(f_a[DECIMATE - 1::DECIMATE], 4),
+        }
+        trend = {
+            "f_local_hz": _rl(self.f_means, 4),
+            "pmu_f_hz": _rl(self.pmu_f, 4),
+            "vq_rms_pu": _rl(self.vq_rms, 5),
+            "v_zero_seq_rms_pu": _rl(self.v0_rms, 5),
+            "v_pos_seq_pu": _rl(self.vpos, 4),
+        }
+        return {"features": features, "waveforms": waveforms, "trend": trend}

pll_cyber/graders.py

@@ -0,0 +1,146 @@
+"""
+Deterministic per-episode graders, one per task, scores in [0, 1].
+
+All graders share the same building blocks, computed from the decision log:
+
+    FA   false-alarm rate: fraction of clean windows (before onset) flagged
+    spec specificity factor max(0, 1 − 2·FA) — every false-alarm window costs
+         twice its share, so random / chattering policies are punished, not
+         just "always on" ones (an operator stops trusting a noisy IDS)
+    T    timeliness of the first detection after onset
+    R    recall: fraction of compromised windows flagged
+    C    classification: fraction of compromised windows flagged with the
+         correct attack_type
+
+Attack-free episodes are scored on specificity alone (and on avoiding
+destructive protective actions), so "always alarm" scores ≈ 0 everywhere and
+"never alarm" only collects the attack-free share of episodes.
+
+Scores are clamped to [0.01, 0.99] for compatibility with validators that
+require strictly-open (0, 1) scores.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional
+
+SCORE_MIN, SCORE_MAX = 0.01, 0.99
+
+
+def _clamp(x: float) -> float:
+    return max(SCORE_MIN, min(SCORE_MAX, x))
+
+
+def _linear_timeliness(delay_ms: Optional[float], full_ms: float, zero_ms: float) -> float:
+    if delay_ms is None:
+        return 0.0
+    if delay_ms <= full_ms:
+        return 1.0
+    return max(0.0, 1.0 - (delay_ms - full_ms) / (zero_ms - full_ms))
+
+
+def _common(log: List[Dict[str, Any]], onset: Optional[int], interval_ms: float) -> Dict[str, Any]:
+    clean = [e for e in log if onset is None or e["step"] < onset]
+    post = [e for e in log if onset is not None and e["step"] >= onset]
+    fa = sum(e["detected"] for e in clean) / len(clean) if clean else 0.0
+    first = next((e["step"] for e in post if e["detected"]), None)
+    delay_ms = (first - onset) * interval_ms if first is not None else None
+    recall = sum(e["detected"] for e in post) / len(post) if post else 0.0
+    false_trip = any(e["protective_action"] == 3 for e in clean)
+    false_holdover = any(e["protective_action"] == 2 for e in clean)
+    return {
+        "false_alarm_rate": fa,
+        "specificity": max(0.0, 1.0 - 2.0 * fa),
+        "first_detection_step": first,
+        "detection_delay_ms": delay_ms,
+        "recall": recall,
+        "false_trip": false_trip,
+        "false_holdover": false_holdover,
+        "_post": post,
+    }
+
+
+def _finish(score: float, parts: Dict[str, Any]) -> Dict[str, Any]:
+    parts.pop("_post", None)
+    parts["raw_score"] = round(score, 6)
+    parts["score"] = round(_clamp(score), 6)
+    return parts
+
+
+def grade_easy(log, onset, interval_ms, **_) -> Dict[str, Any]:
+    p = _common(log, onset, interval_ms)
+    if onset is None:
+        score = p["specificity"]
+    else:
+        p["timeliness"] = _linear_timeliness(p["detection_delay_ms"], 100.0, 1000.0)
+        score = (0.6 * p["timeliness"] + 0.4 * p["recall"]) * p["specificity"]
+    if p["false_trip"]:
+        score *= 0.5
+    return _finish(score, p)
+
+
+def grade_medium(log, onset, interval_ms, true_type: int = 0, **_) -> Dict[str, Any]:
+    p = _common(log, onset, interval_ms)
+    if onset is None:
+        score = p["specificity"]
+    else:
+        post = p["_post"]
+        p["timeliness"] = _linear_timeliness(p["detection_delay_ms"], 100.0, 1000.0)
+        p["classification"] = sum(e["detected"] and e["attack_type"] == true_type for e in post) / len(post)
+        score = (0.35 * p["timeliness"] + 0.25 * p["recall"] + 0.40 * p["classification"]) * p["specificity"]
+    if p["false_trip"]:
+        score *= 0.5
+    return _finish(score, p)
+
+
+def grade_hard(log, onset, interval_ms, unmitigated_desync_step: Optional[int] = None,
+               actual_desync_step: Optional[int] = None, **_) -> Dict[str, Any]:
+    """Stealthy phase spoofing: detection *and* timely mitigation.
+
+    protection P = 1 if the inverter never lost synchronism and a protective
+    action (holdover/disconnect) was engaged by the time the unprotected
+    inverter would have desynchronised. Holdover keeps generating, so it is
+    worth more than a disconnect (quality Q = 1.0 vs 0.6).
+    """
+    p = _common(log, onset, interval_ms)
+    p["unmitigated_desync_step"] = unmitigated_desync_step
+    p["actual_desync_step"] = actual_desync_step
+
+    if onset is None:
+        score = p["specificity"]
+        if p["false_trip"]:
+            score *= 0.4
+        elif p["false_holdover"]:
+            score *= 0.75
+        if actual_desync_step is not None:
+            score *= 0.5
+        return _finish(score, p)
+
+    post = p["_post"]
+    deadline = unmitigated_desync_step if unmitigated_desync_step is not None else log[-1]["step"] + 1
+    mitig = next((e for e in post if e["protective_action"] in (2, 3)), None)
+    engaged_in_time = mitig is not None and mitig["step"] < deadline
+    protected = engaged_in_time and actual_desync_step is None
+    q = 0.6 if any(e["protective_action"] == 3 for e in post) else 1.0
+    p["mitigation_step"] = mitig["step"] if mitig else None
+    p["protected"] = protected
+    p["mitigation_quality"] = q if protected else 0.0
+
+    budget_ms = max(interval_ms, (deadline - onset) * interval_ms)
+    delay = p["detection_delay_ms"]
+    p["timeliness"] = 0.0 if delay is None else max(0.0, 1.0 - delay / budget_ms)
+
+    score = (0.55 * (q if protected else 0.0) + 0.30 * p["timeliness"] + 0.15 * p["recall"]) * p["specificity"]
+    if p["false_trip"]:
+        score *= 0.4
+    elif p["false_holdover"]:
+        score *= 0.75
+    return _finish(score, p)
+
+
+GRADERS = {0: grade_easy, 1: grade_medium, 2: grade_hard}
+
+
+def grade_episode(task_id: int, log: List[Dict[str, Any]], onset: Optional[int],
+                  interval_ms: float, **kwargs) -> Dict[str, Any]:
+    return GRADERS[task_id](log, onset, interval_ms, **kwargs)

pll_cyber/grid.py

@@ -0,0 +1,138 @@
+"""
+True (physical) grid voltage at the inverter's point of common coupling.
+
+The grid is not a perfect 50 Hz source. Besides slow frequency wander and
+background 5th/7th harmonics, an episode may contain *benign* power-system
+events that disturb the PLL exactly like an attack would at first glance:
+
+    frequency_event     — generation/load imbalance: RoCoF ramp to a new frequency
+    phase_jump          — fault clearing / line switching: step in voltage angle
+    voltage_sag         — balanced sag (e.g. remote three-phase fault, motor start)
+    single_phase_sag    — single-line-to-ground fault: one phase dips (creates
+                          zero/negative sequence, like a single-phase FDI attack)
+    harmonic_step       — non-linear load switching: harmonic distortion rises
+
+These are what make cyber-attack detection on PLLs hard in practice: an alarm
+on every grid event is useless to an operator.
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import asdict, dataclass, field
+from typing import Any, Dict, List, Optional
+
+import numpy as np
+
+from .pll import F_NOMINAL, TWO_PI
+
+PHASE_SHIFTS = np.array([0.0, -TWO_PI / 3.0, TWO_PI / 3.0])  # a, b, c
+
+
+@dataclass
+class GridEvent:
+    kind: str
+    t_start: float
+    params: Dict[str, float] = field(default_factory=dict)
+
+    def to_dict(self) -> Dict[str, Any]:
+        return asdict(self)
+
+
+@dataclass
+class GridConfig:
+    v_nominal: float = 1.0
+    f_drift_amp_hz: float = 0.01        # slow frequency wander amplitude
+    f_drift_period_s: float = 6.0
+    f_drift_phase: float = 0.0
+    h5_pu: float = 0.02                 # 5th harmonic (negative sequence)
+    h7_pu: float = 0.01                 # 7th harmonic (positive sequence)
+    h5_phase: float = 0.0
+    h7_phase: float = 0.0
+    phase_unbalance: List[float] = field(default_factory=lambda: [1.0, 1.0, 1.0])
+    events: List[GridEvent] = field(default_factory=list)
+
+    def to_dict(self) -> Dict[str, Any]:
+        d = asdict(self)
+        d["events"] = [e.to_dict() for e in self.events]
+        return d
+
+
+class GridModel:
+    """Generates true three-phase voltages block-by-block."""
+
+    def __init__(self, cfg: GridConfig, theta0: float = 0.0):
+        self.cfg = cfg
+        self.theta = theta0            # unwrapped true angle at the end of the last block
+        self.t = 0.0                   # time at the end of the last block
+
+    # ------------------------------------------------------------------
+    def frequency(self, t: np.ndarray) -> np.ndarray:
+        cfg = self.cfg
+        f = F_NOMINAL + cfg.f_drift_amp_hz * np.sin(TWO_PI * t / cfg.f_drift_period_s + cfg.f_drift_phase)
+        for ev in cfg.events:
+            if ev.kind != "frequency_event":
+                continue
+            tau = np.clip(t - ev.t_start, 0.0, None)
+            rocof, target = ev.params["rocof_hz_s"], ev.params["delta_f_hz"]
+            ramp = np.minimum(np.abs(rocof) * tau, abs(target)) * math.copysign(1.0, target)
+            f = f + ramp
+        return f
+
+    def _phase_jumps(self, t: np.ndarray) -> np.ndarray:
+        off = np.zeros_like(t)
+        for ev in self.cfg.events:
+            if ev.kind == "phase_jump":
+                off += np.where(t >= ev.t_start, math.radians(ev.params["angle_deg"]), 0.0)
+        return off
+
+    def _amplitudes(self, t: np.ndarray) -> np.ndarray:
+        """Per-phase fundamental amplitude, shape (3, n)."""
+        cfg = self.cfg
+        amp = np.tile(np.asarray(cfg.phase_unbalance, dtype=float)[:, None] * cfg.v_nominal, (1, t.size))
+        for ev in cfg.events:
+            if ev.kind == "voltage_sag":
+                active = (t >= ev.t_start) & (t < ev.t_start + ev.params["duration_s"])
+                amp[:, active] *= ev.params["depth_pu"]
+            elif ev.kind == "single_phase_sag":
+                active = (t >= ev.t_start) & (t < ev.t_start + ev.params["duration_s"])
+                amp[int(ev.params["phase"]), active] *= ev.params["depth_pu"]
+        return amp
+
+    def _harmonic_gain(self, t: np.ndarray) -> np.ndarray:
+        g = np.ones_like(t)
+        for ev in self.cfg.events:
+            if ev.kind == "harmonic_step":
+                g = np.where(t >= ev.t_start, g * ev.params["gain"], g)
+        return g
+
+    # ------------------------------------------------------------------
+    def generate(self, n: int, dt: float, angle_offset: Optional[np.ndarray] = None) -> Dict[str, np.ndarray]:
+        """Advance the grid by `n` samples.
+
+        `angle_offset` (rad, shape (n,)) is added to the *reported* angle only —
+        it is how a coordinated phase-spoofing FDI attack is synthesised. The
+        returned `theta` is always the true, physical grid angle.
+        """
+        t = self.t + dt * np.arange(1, n + 1)
+        f = self.frequency(t)
+        theta_smooth = self.theta + TWO_PI * np.cumsum(f) * dt
+        theta = theta_smooth + self._phase_jumps(t)
+        theta_rep = theta if angle_offset is None else theta + angle_offset
+
+        amp = self._amplitudes(t)
+        hg = self._harmonic_gain(t)
+        cfg = self.cfg
+        ph = PHASE_SHIFTS[:, None]
+        th = theta_rep[None, :]
+        v = (
+            amp * np.cos(th + ph)
+            + (cfg.h5_pu * hg)[None, :] * np.cos(5.0 * (th + ph) + cfg.h5_phase)
+            + (cfg.h7_pu * hg)[None, :] * np.cos(7.0 * (th + ph) + cfg.h7_phase)
+        )
+
+        self.theta = float(theta_smooth[-1])
+        self.t = float(t[-1])
+        # True positive-sequence magnitude (what a PMU would report)
+        v_pos = amp.mean(axis=0)
+        return {"t": t, "v": v, "theta": theta, "f": f, "v_pos": v_pos}

pll_cyber/models.py

@@ -0,0 +1,147 @@
+"""
+Typed OpenEnv models: Action, Observation, Reward breakdown and State.
+
+All models subclass the openenv-core base types so they plug straight into
+`create_app` (HTTP + WebSocket) and `EnvClient`.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Literal, Optional
+
+from openenv.core.env_server.types import Action, Observation, State
+from pydantic import BaseModel, ConfigDict, Field
+
+InverterMode = Literal["normal", "holdover", "disconnected"]
+
+
+class _Strict(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+
+
+# ---------------------------------------------------------------------------
+# Action
+# ---------------------------------------------------------------------------
+
+class PLLAction(Action):
+    """The cyber-guard's decision for the window it has just observed."""
+
+    attack_detected: bool = Field(..., description="True if a cyber-attack is believed to be in progress.")
+    attack_type: int = Field(
+        0, ge=0, le=4,
+        description="0=none, 1=sinusoidal injection, 2=ramp bias, 3=pulse bias, 4=phase spoofing.",
+    )
+    confidence: float = Field(
+        0.5, ge=0.0, le=1.0,
+        description="Probability that the reported attack_detected value is correct (scored with a Brier rule).",
+    )
+    protective_action: int = Field(
+        0, ge=0, le=3,
+        description=(
+            "0=none, 1=alert operator (no physical effect), "
+            "2=PMU-aided holdover: open the PLL loop and free-run on the trusted PMU frequency, "
+            "3=disconnect inverter (latched for the rest of the episode)."
+        ),
+    )
+
+
+# ---------------------------------------------------------------------------
+# Observation pieces
+# ---------------------------------------------------------------------------
+
+class Waveforms(_Strict):
+    """Last decision window (50 ms) decimated to 2 ms resolution (25 points each)."""
+    va: List[float]
+    vb: List[float]
+    vc: List[float]
+    vd: List[float]
+    vq: List[float]
+    f_hz: List[float] = Field(description="PLL frequency estimate (low-pass filtered), Hz.")
+
+
+class SignalFeatures(_Strict):
+    """Standard signal-processing features a protection relay would compute on-board."""
+    v_pos_seq_pu: float = Field(description="Mean d-axis voltage ≈ positive-sequence magnitude.")
+    vq_mean_pu: float
+    vq_rms_pu: float = Field(description="RMS of the q-axis (phase-detector) error.")
+    f_local_hz: float = Field(description="Mean local PLL frequency over the window.")
+    f_local_std_hz: float
+    rocof_local_hz_s: float = Field(description="Rate of change of local frequency (from the last 4 windows).")
+    v_zero_seq_rms_pu: float = Field(description="RMS of (va+vb+vc)/3; non-zero for unbalanced faults or single-phase injection.")
+    phase_rms_pu: List[float] = Field(description="Per-phase RMS [a, b, c].")
+    phase_dc_offset_pu: List[float] = Field(description="Per-phase mean over the last two fundamental cycles.")
+    vq_osc_freq_hz: float = Field(description="Dominant oscillation frequency in vq over the last 200 ms.")
+    vq_osc_amp_pu: float
+    f_local_minus_pmu_hz: float = Field(description="Local frequency minus the latest PMU frequency (note PMU latency).")
+
+
+class PMUReading(_Strict):
+    """Trusted, GPS-synchronised phasor-measurement stream from the substation.
+
+    It is out-of-band (cannot be spoofed by the attacker), but low-rate
+    (10 reports/s), noisy and delayed — so it cannot be used naively.
+    """
+    f_hz: float
+    v_pos_pu: float
+    age_ms: float = Field(description="Age of the measurement interval end-time at the moment of observation.")
+
+
+class Trend(_Strict):
+    """History of the last 10 decision windows (oldest first)."""
+    f_local_hz: List[float]
+    pmu_f_hz: List[float]
+    vq_rms_pu: List[float]
+    v_zero_seq_rms_pu: List[float]
+    v_pos_seq_pu: List[float]
+
+
+class RewardBreakdown(_Strict):
+    """Typed Reward model: every component of the dense per-step reward."""
+    detection: float = 0.0
+    early_detection_bonus: float = 0.0
+    calibration: float = 0.0
+    classification: float = 0.0
+    mitigation_cost: float = 0.0
+    harm_penalty: float = 0.0
+    total: float = 0.0
+
+
+class EpisodeResult(_Strict):
+    """Revealed only once the episode is done."""
+    grader_score: float
+    score_breakdown: Dict[str, Any]
+    total_reward: float
+    scenario: Dict[str, Any]
+
+
+class PLLObservation(Observation):
+    task_id: int = Field(0, ge=0, le=2)
+    task_name: str = ""
+    task_brief: str = Field("", description="Task instructions (sent on reset only).")
+    step: int = 0
+    max_steps: int = 0
+    time_s: float = Field(0.0, description="Time since the start of the monitored episode, s.")
+    decision_interval_ms: float = 50.0
+    inverter_mode: InverterMode = "normal"
+    waveforms: Optional[Waveforms] = None
+    features: Optional[SignalFeatures] = None
+    pmu: Optional[PMUReading] = None
+    trend: Optional[Trend] = None
+    reward_breakdown: Optional[RewardBreakdown] = None
+    episode_result: Optional[EpisodeResult] = None
+    message: str = ""
+
+
+class PLLState(State):
+    """Server-side episode state. Ground truth stays hidden until `done`."""
+    task_id: int = 0
+    task_name: str = ""
+    seed: Optional[int] = None
+    done: bool = False
+    time_s: float = 0.0
+    inverter_mode: InverterMode = "normal"
+    cumulative_reward: float = 0.0
+    grader_score: Optional[float] = None
+    scenario: Optional[Dict[str, Any]] = Field(
+        None, description="Attack / disturbance ground truth — null while the episode is running."
+    )

pll_cyber/pll.py

@@ -0,0 +1,186 @@
+"""
+Synchronous Reference Frame Phase-Locked Loop (SRF-PLL).
+
+Conventions (cosine-referenced, amplitude-invariant):
+
+    va = V cos(θ),  vb = V cos(θ − 2π/3),  vc = V cos(θ + 2π/3)
+
+    Clarke:  vα = (2va − vb − vc) / 3        vβ = (vb − vc) / √3
+             → vα = V cos θ,  vβ = V sin θ
+
+    Park:    vd =  vα cos θ̂ + vβ sin θ̂  = V cos(θ − θ̂)
+             vq = −vα sin θ̂ + vβ cos θ̂  = V sin(θ − θ̂)
+
+So when locked (θ̂ ≈ θ): vd → V and vq → 0, and for small errors vq ≈ V·(θ − θ̂),
+which is the phase-detector characteristic the PI loop filter drives to zero.
+
+Loop design (amplitude-normalised, so dynamics do not change during voltage sags):
+
+    θ̂/θ = (Kp s + Ki) / (s² + Kp s + Ki),   ωn = √Ki,   ζ = Kp / (2√Ki)
+
+Defaults give ωn = 100 rad/s, ζ = 0.707 (≈ 25 Hz closed-loop bandwidth), a
+typical industrial tuning trade-off between tracking speed and harmonic rejection.
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass, field
+from typing import Dict, List, Sequence
+
+TWO_PI = 2.0 * math.pi
+SQRT3 = math.sqrt(3.0)
+
+F_NOMINAL = 50.0
+OMEGA_NOMINAL = TWO_PI * F_NOMINAL
+
+
+def wrap_angle(angle: float) -> float:
+    """Wrap an angle to [−π, π)."""
+    return (angle + math.pi) % TWO_PI - math.pi
+
+
+def clarke(va: float, vb: float, vc: float) -> tuple[float, float]:
+    """Amplitude-invariant Clarke transform (abc → αβ), using all three phases."""
+    return (2.0 * va - vb - vc) / 3.0, (vb - vc) / SQRT3
+
+
+def park(v_alpha: float, v_beta: float, theta: float) -> tuple[float, float]:
+    """Park transform (αβ → dq) on the estimated angle θ̂."""
+    c, s = math.cos(theta), math.sin(theta)
+    return v_alpha * c + v_beta * s, -v_alpha * s + v_beta * c
+
+
+@dataclass
+class PLLGains:
+    omega_n: float = 100.0      # natural frequency (rad/s)
+    zeta: float = 0.7071        # damping ratio
+    freq_filter_hz: float = 10.0  # LPF on the reported frequency
+    max_dev_hz: float = 5.0      # anti-windup / frequency clamp
+
+    @property
+    def kp(self) -> float:
+        return 2.0 * self.zeta * self.omega_n
+
+    @property
+    def ki(self) -> float:
+        return self.omega_n ** 2
+
+
+@dataclass
+class PLLWindowOutput:
+    """Per-sample PLL signals for one simulation window."""
+    vd: List[float] = field(default_factory=list)
+    vq: List[float] = field(default_factory=list)
+    f_hat: List[float] = field(default_factory=list)       # filtered frequency estimate (Hz)
+    theta_hat: List[float] = field(default_factory=list)
+
+
+class SRFPLL:
+    """Discrete-time SRF-PLL with an optional PMU-aided holdover mode.
+
+    Modes:
+        normal   — closed-loop tracking of the measured voltages
+        holdover — loop opened; θ̂ free-runs at an externally supplied trusted
+                   frequency (e.g. a PMU stream). Measurements are still
+                   transformed so vd/vq remain observable.
+    """
+
+    def __init__(self, dt: float, gains: PLLGains | None = None, theta0: float = 0.0):
+        self.dt = dt
+        self.gains = gains or PLLGains()
+        self.reset(theta0)
+
+    def reset(self, theta0: float = 0.0) -> None:
+        self.theta_hat = wrap_angle(theta0)
+        self.integrator = 0.0              # ∫ normalised vq dt (rad/s units after Ki)
+        self.omega_hat = OMEGA_NOMINAL
+        self.f_filtered = F_NOMINAL
+        self.vd_filtered = 1.0
+        self.vd = 1.0
+        self.vq = 0.0
+
+    def snapshot(self) -> Dict[str, float]:
+        return {
+            "theta_hat": self.theta_hat,
+            "integrator": self.integrator,
+            "omega_hat": self.omega_hat,
+            "f_filtered": self.f_filtered,
+            "vd_filtered": self.vd_filtered,
+        }
+
+    def run(
+        self,
+        va: Sequence[float],
+        vb: Sequence[float],
+        vc: Sequence[float],
+        holdover: bool = False,
+        holdover_freq_hz: float = F_NOMINAL,
+        holdover_rocof_hz_s: float = 0.0,
+        record: bool = True,
+    ) -> PLLWindowOutput:
+        """Process a block of samples. Returns per-sample signals if `record`."""
+        g = self.gains
+        dt = self.dt
+        kp, ki = g.kp, g.ki
+        max_dev = TWO_PI * g.max_dev_hz
+        a_f = dt / (dt + 1.0 / (TWO_PI * g.freq_filter_hz))      # 1st-order LPF coefficient
+        a_v = dt / (dt + 1.0 / (TWO_PI * 20.0))                   # amplitude estimator LPF
+        omega_hold = TWO_PI * holdover_freq_hz
+        d_omega_hold = TWO_PI * holdover_rocof_hz_s * dt
+
+        th = self.theta_hat
+        integ = self.integrator
+        f_filt = self.f_filtered
+        vd_filt = self.vd_filtered
+        omega = self.omega_hat
+        vd = self.vd
+        vq = self.vq
+
+        out = PLLWindowOutput()
+        cos, sin = math.cos, math.sin
+        two_pi, pi = TWO_PI, math.pi
+
+        for a, b, c in zip(va, vb, vc):
+            v_alpha = (2.0 * a - b - c) / 3.0
+            v_beta = (b - c) / SQRT3
+            ct, st = cos(th), sin(th)
+            vd = v_alpha * ct + v_beta * st
+            vq = -v_alpha * st + v_beta * ct
+
+            vd_filt += a_v * (vd - vd_filt)
+            if holdover:
+                omega = omega_hold
+                omega_hold += d_omega_hold
+            else:
+                vq_n = vq / max(vd_filt, 0.2)          # amplitude normalisation
+                integ += ki * vq_n * dt
+                if integ > max_dev:
+                    integ = max_dev
+                elif integ < -max_dev:
+                    integ = -max_dev
+                dev = kp * vq_n + integ
+                if dev > max_dev:
+                    dev = max_dev
+                elif dev < -max_dev:
+                    dev = -max_dev
+                omega = OMEGA_NOMINAL + dev
+
+            f_filt += a_f * (omega / two_pi - f_filt)
+            if record:
+                # θ̂ recorded is the angle used for this sample's Park transform,
+                # i.e. the controller's estimate of θ at this sampling instant.
+                out.vd.append(vd)
+                out.vq.append(vq)
+                out.f_hat.append(f_filt)
+                out.theta_hat.append(th)
+            th = (th + omega * dt + pi) % two_pi - pi
+
+        self.theta_hat = th
+        self.integrator = integ
+        self.f_filtered = f_filt
+        self.vd_filtered = vd_filt
+        self.omega_hat = omega
+        self.vd = vd
+        self.vq = vq
+        return out

pll_cyber/rewards.py

@@ -0,0 +1,97 @@
+"""
+Dense per-step reward.
+
+Action a_t is judged against the window t the agent actually observed
+(label y_t = 1 once the attack has started — the system is then considered
+compromised until the end of the episode). Physical consequences of the
+chosen protective action (costs, desynchronisation harm) are charged on the
+window simulated right after the action.
+
+The costs are asymmetric on purpose: false alarms and false trips are what
+make real intrusion-detection systems unusable, so "always alarm" and
+"always trip" are strongly negative-return strategies.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from .models import RewardBreakdown
+
+TRUE_POSITIVE = 0.10
+TRUE_NEGATIVE = 0.02
+MISSED_DETECTION = -0.10
+FALSE_ALARM = -0.25
+
+EARLY_BONUS_MAX = 0.50
+EARLY_BONUS_HORIZON_MS = 500.0
+
+CALIBRATION_SCALE = 0.04            # 0.04 * (1 - 4*Brier) ∈ [-0.12, +0.04]
+
+CLASS_CORRECT = 0.05
+CLASS_WRONG = -0.02
+
+HOLDOVER_COST = -0.02               # per window spent in holdover (degraded mode)
+DISCONNECTED_COST = -0.04           # per window of lost generation
+TRIP_COST_JUSTIFIED = -0.30         # one-off, attack in progress
+TRIP_COST_FALSE = -1.50             # one-off, healthy grid ("destructive action")
+
+DESYNC_WINDOW_PENALTY = -0.30       # each window with sustained >10° error while connected
+DESYNC_EVENT_PENALTY = -1.00        # one-off, first loss of synchronism
+
+REWARD_MAX = TRUE_POSITIVE + EARLY_BONUS_MAX + CALIBRATION_SCALE + CLASS_CORRECT
+REWARD_MIN = (FALSE_ALARM - 3 * CALIBRATION_SCALE + TRIP_COST_FALSE + DISCONNECTED_COST
+              + DESYNC_WINDOW_PENALTY + DESYNC_EVENT_PENALTY)
+
+
+@dataclass
+class StepOutcome:
+    """Everything the reward needs to know about one transition."""
+    label_attack: bool
+    true_type: int
+    steps_since_onset: int              # valid when label_attack
+    detected: bool
+    reported_type: int
+    confidence: float
+    first_true_positive: bool
+    classify: bool                      # task scores the attack type
+    tripped_now: bool
+    mode_next: str                      # inverter mode during the next window
+    desync_in_next_window: bool
+    first_desync: bool
+    decision_interval_ms: float
+
+
+def compute_reward(o: StepOutcome) -> RewardBreakdown:
+    r = RewardBreakdown()
+
+    if o.label_attack:
+        r.detection = TRUE_POSITIVE if o.detected else MISSED_DETECTION
+        if o.first_true_positive:
+            delay_ms = o.steps_since_onset * o.decision_interval_ms
+            r.early_detection_bonus = EARLY_BONUS_MAX * max(0.0, 1.0 - delay_ms / EARLY_BONUS_HORIZON_MS)
+        if o.classify and o.detected:
+            r.classification = CLASS_CORRECT if o.reported_type == o.true_type else CLASS_WRONG
+    else:
+        r.detection = FALSE_ALARM if o.detected else TRUE_NEGATIVE
+
+    # Brier score on the implied probability of "attack in progress"
+    p_attack = o.confidence if o.detected else 1.0 - o.confidence
+    brier = (p_attack - (1.0 if o.label_attack else 0.0)) ** 2
+    r.calibration = CALIBRATION_SCALE * (1.0 - 4.0 * brier)
+
+    if o.tripped_now:
+        r.mitigation_cost += TRIP_COST_JUSTIFIED if o.label_attack else TRIP_COST_FALSE
+    if o.mode_next == "holdover":
+        r.mitigation_cost += HOLDOVER_COST
+    elif o.mode_next == "disconnected":
+        r.mitigation_cost += DISCONNECTED_COST
+
+    if o.desync_in_next_window:
+        r.harm_penalty += DESYNC_WINDOW_PENALTY
+    if o.first_desync:
+        r.harm_penalty += DESYNC_EVENT_PENALTY
+
+    r.total = (r.detection + r.early_detection_bonus + r.calibration + r.classification
+               + r.mitigation_cost + r.harm_penalty)
+    return r

pll_cyber/scenarios.py

@@ -0,0 +1,205 @@
+"""
+Task definitions and seeded scenario sampling.
+
+Every episode is fully determined by (task_id, seed): attack type/onset and
+parameters, benign grid events, harmonics, PMU latency and all noise streams.
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional
+
+import numpy as np
+
+from .attacks import AttackSpec
+from .grid import GridConfig, GridEvent
+
+# ---------------------------------------------------------------------------
+# Timing (shared by all tasks)
+# ---------------------------------------------------------------------------
+SAMPLE_RATE_HZ = 5000.0
+DT = 1.0 / SAMPLE_RATE_HZ
+DECISION_INTERVAL_S = 0.050          # 2.5 fundamental cycles per agent decision
+SAMPLES_PER_DECISION = int(round(DECISION_INTERVAL_S * SAMPLE_RATE_HZ))
+WARMUP_S = 0.200                     # PLL settles before the agent's first observation
+MAX_STEPS = 40                       # 2.0 s monitored horizon
+
+PMU_REPORT_INTERVAL_S = 0.100
+PMU_F_NOISE_HZ = 0.01
+PMU_V_NOISE_PU = 0.003
+SENSOR_NOISE_PU = 0.005
+ADC_LSB_PU = 4.0 / 4096.0            # 12-bit ADC over ±2 pu
+
+
+@dataclass(frozen=True)
+class TaskSpec:
+    task_id: int
+    name: str
+    difficulty: str
+    attack_probability: float
+    brief: str
+
+
+TASKS: Dict[int, TaskSpec] = {
+    0: TaskSpec(
+        0, "sinusoidal_fdi_detection", "easy", 0.80,
+        "Monitor the inverter's PLL. A sinusoidal false-data-injection attack may start on one phase "
+        "(some episodes are attack-free). Raise attack_detected=True as soon as the attack is present "
+        "and keep it raised; never raise it before. Grid background harmonics and occasional mild sags "
+        "are normal operation.",
+    ),
+    1: TaskSpec(
+        1, "attack_classification_under_disturbances", "medium", 0.80,
+        "A sinusoidal (1), ramp-bias (2) or pulse-bias (3) injection may start on one phase, while genuine "
+        "grid events (phase jumps, voltage sags, single-phase faults, harmonic steps, frequency events) also "
+        "occur. Detect the attack quickly, report its attack_type, and do NOT alarm on natural grid events. "
+        "Once an attack has started the system counts as compromised for the rest of the episode.",
+    ),
+    2: TaskSpec(
+        2, "stealthy_phase_spoofing_mitigation", "hard", 0.80,
+        "A coordinated phase-spoofing attack (4) may slowly rotate the measured voltage angle. The PLL "
+        "follows it with almost no residual, and the local frequency drifts like a natural grid frequency "
+        "event. Use the trusted but delayed PMU stream to tell them apart. Detect it and engage "
+        "protective_action=2 (PMU-aided holdover) before the inverter drifts more than 10 deg out of sync "
+        "with the real grid. Holdover or disconnect (3) on a healthy grid is costly and can itself cause "
+        "desynchronisation.",
+    ),
+}
+
+
+@dataclass
+class Scenario:
+    task_id: int
+    seed: int
+    grid: GridConfig
+    attack: AttackSpec
+    onset_step: Optional[int]          # decision index at which the attack begins, None if attack-free
+    pmu_latency_s: float
+    theta0: float = 0.0
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {
+            "task_id": self.task_id,
+            "task_name": TASKS[self.task_id].name,
+            "seed": self.seed,
+            "attack": self.attack.to_dict(),
+            "onset_step": self.onset_step,
+            "pmu_latency_ms": round(self.pmu_latency_s * 1000.0, 1),
+            "grid": self.grid.to_dict(),
+        }
+
+
+def step_start_time(step: int) -> float:
+    """Absolute sim time at which decision window `step` begins (window 0 = last warm-up window)."""
+    return WARMUP_S + (step - 1) * DECISION_INTERVAL_S
+
+
+# ---------------------------------------------------------------------------
+# Sampling helpers
+# ---------------------------------------------------------------------------
+
+def _u(rng: np.random.Generator, lo: float, hi: float) -> float:
+    return float(rng.uniform(lo, hi))
+
+
+def _sign(rng: np.random.Generator) -> float:
+    return 1.0 if rng.random() < 0.5 else -1.0
+
+
+def _benign_event(rng: np.random.Generator, kind: str, t: float, strong: bool) -> GridEvent:
+    if kind == "phase_jump":
+        return GridEvent(kind, t, {"angle_deg": _sign(rng) * _u(rng, 5.0, 25.0 if strong else 12.0)})
+    if kind == "voltage_sag":
+        return GridEvent(kind, t, {"depth_pu": _u(rng, 0.5 if strong else 0.8, 0.9), "duration_s": _u(rng, 0.06, 0.25)})
+    if kind == "single_phase_sag":
+        return GridEvent(kind, t, {"phase": int(rng.integers(0, 3)), "depth_pu": _u(rng, 0.3, 0.8), "duration_s": _u(rng, 0.06, 0.20)})
+    if kind == "harmonic_step":
+        return GridEvent(kind, t, {"gain": _u(rng, 1.5, 2.5)})
+    if kind == "frequency_event":
+        return GridEvent(kind, t, {"rocof_hz_s": _u(rng, 0.1, 0.6), "delta_f_hz": _sign(rng) * _u(rng, 0.05, 0.25)})
+    raise ValueError(kind)
+
+
+def _injection_attack(rng: np.random.Generator, kind: str, t: float, easy: bool) -> AttackSpec:
+    phase = int(rng.integers(0, 3))
+    if kind == "sinusoidal":
+        lo, hi = (0.05, 0.15) if easy else (0.03, 0.10)
+        p = {"amplitude_pu": _u(rng, lo, hi), "freq_hz": _u(rng, 5.0, 25.0),
+             "phase": _u(rng, 0.0, 2 * math.pi), "phase_index": phase}
+    elif kind == "ramp":
+        p = {"rate_pu_s": _sign(rng) * _u(rng, 0.08, 0.30), "cap_pu": 0.30, "phase_index": phase}
+    elif kind == "pulse":
+        p = {"magnitude_pu": _sign(rng) * _u(rng, 0.08, 0.25), "duration_s": _u(rng, 0.10, 0.30), "phase_index": phase}
+    else:
+        raise ValueError(kind)
+    return AttackSpec(kind, t, p)
+
+
+def sample_scenario(task_id: int, seed: int) -> Scenario:
+    if task_id not in TASKS:
+        raise ValueError(f"task_id must be one of {sorted(TASKS)}")
+    task = TASKS[task_id]
+    rng = np.random.default_rng(np.random.SeedSequence([seed, 1000 + task_id]))
+
+    grid = GridConfig(
+        f_drift_amp_hz=_u(rng, 0.005, 0.02 if task_id < 2 else 0.03),
+        f_drift_period_s=_u(rng, 4.0, 10.0),
+        f_drift_phase=_u(rng, 0.0, 2 * math.pi),
+        h5_pu=_u(rng, 0.01, 0.04),
+        h7_pu=_u(rng, 0.005, 0.025),
+        h5_phase=_u(rng, 0.0, 2 * math.pi),
+        h7_phase=_u(rng, 0.0, 2 * math.pi),
+        phase_unbalance=[_u(rng, 0.99, 1.01) for _ in range(3)],
+    )
+    attacked = rng.random() < task.attack_probability
+    pmu_latency = _u(rng, 0.04, 0.10)
+    t_end = step_start_time(MAX_STEPS)
+
+    if task_id == 0:
+        onset = int(rng.integers(6, 21)) if attacked else None
+        if rng.random() < 0.3:
+            grid.events.append(_benign_event(rng, "voltage_sag", step_start_time(int(rng.integers(2, MAX_STEPS - 4))), strong=False))
+        attack = _injection_attack(rng, "sinusoidal", step_start_time(onset), easy=True) if attacked else AttackSpec()
+
+    elif task_id == 1:
+        onset = int(rng.integers(10, 25)) if attacked else None
+        kinds = ["phase_jump", "voltage_sag", "single_phase_sag", "harmonic_step", "frequency_event"]
+        n_events = int(rng.integers(1, 3)) + (0 if attacked else 1)
+        # Guarantee a benign event in the clean part of the episode, so false-alarm
+        # resistance is always tested.
+        last_clean = (onset - 1) if attacked else MAX_STEPS - 4
+        times = [step_start_time(int(rng.integers(2, max(3, last_clean))))]
+        times += [step_start_time(int(rng.integers(2, MAX_STEPS - 4))) for _ in range(n_events - 1)]
+        for t in times:
+            grid.events.append(_benign_event(rng, str(rng.choice(kinds)), t + _u(rng, 0.0, DECISION_INTERVAL_S), strong=True))
+        if attacked:
+            kind = ["sinusoidal", "ramp", "pulse"][int(rng.integers(0, 3))]
+            attack = _injection_attack(rng, kind, step_start_time(onset), easy=False)
+        else:
+            attack = AttackSpec()
+
+    else:
+        onset = int(rng.integers(6, 19)) if attacked else None
+        # Natural frequency events are the main confounder for phase spoofing.
+        n_freq = 1 if attacked else int(rng.integers(1, 3))
+        for _ in range(n_freq):
+            k = int(rng.integers(2, (onset + 4) if attacked else MAX_STEPS - 8))
+            grid.events.append(_benign_event(rng, "frequency_event", step_start_time(k) + _u(rng, 0.0, DECISION_INTERVAL_S), strong=True))
+        if rng.random() < 0.6:
+            kind = ["voltage_sag", "single_phase_sag", "harmonic_step"][int(rng.integers(0, 3))]
+            grid.events.append(_benign_event(rng, kind, step_start_time(int(rng.integers(2, MAX_STEPS - 4))), strong=False))
+        if attacked:
+            attack = AttackSpec("phase_spoofing", step_start_time(onset), {
+                "rocof_hz_s": _u(rng, 0.12, 0.45),
+                "max_offset_hz": _u(rng, 0.20, 0.40),
+                "direction": _sign(rng),
+            })
+        else:
+            attack = AttackSpec()
+
+    grid.events.sort(key=lambda e: e.t_start)
+    grid.events = [e for e in grid.events if e.t_start < t_end]
+    return Scenario(task_id=task_id, seed=seed, grid=grid, attack=attack,
+                    onset_step=onset, pmu_latency_s=pmu_latency)

pll_cyber/sensors.py

@@ -0,0 +1,152 @@
+"""
+Measurement chain: local voltage sensors (noise + ADC quantisation) and the
+trusted, out-of-band PMU reference stream.
+"""
+
+from __future__ import annotations
+
+from collections import deque
+from dataclasses import dataclass
+from typing import Deque, List, Optional, Tuple
+
+import numpy as np
+
+from .pll import F_NOMINAL, TWO_PI
+
+
+def measure(v: np.ndarray, rng: np.random.Generator, noise_pu: float, lsb_pu: float) -> np.ndarray:
+    """Additive Gaussian sensor noise followed by uniform ADC quantisation."""
+    noisy = v + rng.normal(0.0, noise_pu, v.shape)
+    return np.round(noisy / lsb_pu) * lsb_pu
+
+
+@dataclass
+class PMUReport:
+    t_end: float          # end of the measurement interval (s)
+    f_hz: float
+    v_pos_pu: float
+    theta: float          # synchrophasor angle at t_end (rad, GPS-referenced)
+
+
+class PMUStream:
+    """IEEE C37.118-style µPMU at the substation (GPS time-stamped, not spoofable).
+
+    Each report gives the average frequency over the last reporting interval,
+    computed from the true phase-angle advance — so it also shows the
+    frequency spike that accompanies a real phase jump — plus Gaussian noise.
+    Reports reach the inverter after a fixed communication latency.
+
+    The synchrophasor angle is only consumed by the inverter's holdover
+    controller; the monitoring agent subscribes to the frequency and magnitude
+    channels (a low-bandwidth SCADA status feed).
+    """
+
+    def __init__(self, interval_s: float, latency_s: float, f_noise_hz: float,
+                 v_noise_pu: float, rng: np.random.Generator, angle_noise_rad: float = 0.0035):
+        self.interval = interval_s
+        self.latency = latency_s
+        self.f_noise = f_noise_hz
+        self.v_noise = v_noise_pu
+        self.angle_noise = angle_noise_rad
+        self.rng = rng
+        self.reports: Deque[PMUReport] = deque(maxlen=64)
+        self._last_theta: Optional[Tuple[float, float]] = None   # (t, θ) at last report boundary
+        self._v_acc = 0.0
+        self._v_n = 0
+        self._next_t = None
+
+    def ingest(self, t: np.ndarray, theta: np.ndarray, v_pos: np.ndarray) -> None:
+        """Feed a block of true grid samples; emits reports at interval boundaries."""
+        if self._next_t is None:
+            self._next_t = (np.floor(t[0] / self.interval) + 1) * self.interval
+            self._last_theta = (float(t[0]), float(theta[0]))
+        start = 0
+        while True:
+            idx = int(np.searchsorted(t, self._next_t - 1e-9))
+            if idx >= t.size:
+                break
+            seg = v_pos[start:idx + 1]
+            self._v_acc += float(seg.sum())
+            self._v_n += seg.size
+            t0, th0 = self._last_theta
+            t1, th1 = float(t[idx]), float(theta[idx])
+            f = (th1 - th0) / (TWO_PI * (t1 - t0)) if t1 > t0 else F_NOMINAL
+            v = self._v_acc / max(self._v_n, 1)
+            self.reports.append(PMUReport(
+                t_end=t1,
+                f_hz=float(f + self.rng.normal(0.0, self.f_noise)),
+                v_pos_pu=float(v + self.rng.normal(0.0, self.v_noise)),
+                theta=float(th1 + self.rng.normal(0.0, self.angle_noise)),
+            ))
+            self._last_theta = (t1, th1)
+            self._v_acc, self._v_n = 0.0, 0
+            self._next_t += self.interval
+            start = idx + 1
+        seg = v_pos[start:]
+        self._v_acc += float(seg.sum())
+        self._v_n += seg.size
+
+    def arrived(self, since: float, now: float) -> List[PMUReport]:
+        """Reports that arrived in (since, now]."""
+        return [r for r in self.reports if since < r.t_end + self.latency <= now + 1e-9]
+
+    def latest(self, now: float) -> Optional[PMUReport]:
+        """Most recent report that has arrived by time `now`."""
+        for rep in reversed(self.reports):
+            if rep.t_end + self.latency <= now + 1e-9:
+                return rep
+        return None
+
+
+class DisciplinedHoldover:
+    """PMU/GPS-disciplined oscillator used while the PLL loop is open.
+
+    On every PMU report that has arrived, it re-anchors θ̂ towards the angle
+    *predicted for the present instant* from the synchrophasor angle, the
+    reported frequency and a RoCoF estimated by a linear fit over the last
+    few reports (latency compensation). Between reports θ̂ free-runs with the
+    extrapolated frequency ramp.
+    """
+
+    PHASE_GAIN = 0.7
+    FIT_REPORTS = 4
+    MAX_ROCOF = 1.0          # Hz/s, beyond natural grid behaviour
+
+    def __init__(self):
+        self.active = False
+        self.f_hold = 50.0
+        self.rocof = 0.0
+        self.last_arrival_check = -1.0
+
+    def engage(self, now: float) -> None:
+        self.active = True
+        self.last_arrival_check = -1.0
+
+    def release(self) -> None:
+        self.active = False
+
+    def update(self, pll, pmu: "PMUStream", now: float) -> None:
+        """Steer the oscillator using reports that arrived since the last call."""
+        arrived = [r for r in pmu.reports if r.t_end + pmu.latency <= now + 1e-9]
+        if not arrived:
+            return
+        newest = arrived[-1]
+        if newest.t_end + pmu.latency <= self.last_arrival_check:
+            return                                    # nothing new
+        T = pmu.interval
+        fit = arrived[-self.FIT_REPORTS:]
+        tm = np.array([r.t_end - T / 2 for r in fit])      # report freq = interval average
+        fv = np.array([r.f_hz for r in fit])
+        if len(fit) >= 3:
+            coef = np.polyfit(tm, fv, 1)
+            self.rocof = float(np.clip(coef[0], -self.MAX_ROCOF, self.MAX_ROCOF))
+            f_at_t = float(np.polyval(coef, newest.t_end))
+        else:
+            self.rocof = 0.0
+            f_at_t = float(np.median(fv))
+        dt = now - newest.t_end
+        theta_pred = newest.theta + TWO_PI * (f_at_t * dt + 0.5 * self.rocof * dt * dt)
+        err = (theta_pred - pll.theta_hat + np.pi) % TWO_PI - np.pi
+        pll.theta_hat = float((pll.theta_hat + self.PHASE_GAIN * err + np.pi) % TWO_PI - np.pi)
+        self.f_hold = f_at_t + self.rocof * dt
+        self.last_arrival_check = now

pyproject.toml

@@ -1,19 +1,42 @@
 [build-system]
-requires = ["setuptools>=61.0"]
-build-backend = "setuptools.backends.legacy:build"
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
 
 [project]
 name = "pll-cyberattack-detection"
-version = "1.0.0"
-description = "OpenEnv for cyberattack detection on SRF-PLLs in grid-connected inverters"
+version = "2.0.0"
+description = "OpenEnv environment: detect, classify and mitigate false-data-injection attacks on the SRF-PLL of a grid-connected inverter"
+readme = "README.md"
 requires-python = ">=3.10"
+authors = [{ name = "Kris Keshav" }]
+keywords = ["openenv", "reinforcement-learning", "power-systems", "cybersecurity", "pll", "smart-grid"]
 dependencies = [
-    "fastapi",
-    "uvicorn",
-    "pydantic",
-    "numpy",
-    "openenv-core>=0.2.0",
+    "openenv-core>=0.3.0,<0.4",
+    "numpy>=1.26",
+    "fastapi>=0.110",
+    "uvicorn>=0.29",
+    "pydantic>=2.6",
+    "openai>=1.40",
+    "requests>=2.31",
+    "python-dotenv>=1.0",
 ]
 
+[project.optional-dependencies]
+dev = ["pytest>=8.0", "httpx>=0.27"]
+
 [project.scripts]
 server = "server.app:main"
+
+[project.urls]
+Repository = "https://github.com/KrisKeshav/PLL-Cyberattack-Detection-OpenEnv"
+Space = "https://huggingface.co/spaces/krishuggingface/CyberAttack-PLL"
+
+[tool.setuptools]
+packages = ["pll_cyber", "server"]
+py-modules = ["inference"]
+
+[tool.setuptools.package-data]
+server = ["*.html"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

requirements-dev.txt

@@ -0,0 +1,3 @@
+-r requirements.txt
+pytest==9.1.1
+httpx==0.28.1

requirements.txt

@@ -1,8 +1,9 @@
-fastapi==0.115.0
-uvicorn==0.32.0
-pydantic==2.9.0
-numpy==1.26.4
-openai>=1.0.0
-requests>=2.31.0
-openenv-core>=0.2.0
-python-dotenv>=1.0.0
+# Runtime dependencies (pinned for a reproducible Docker image / HF Space)
+openenv-core==0.3.0
+numpy==2.5.4
+fastapi==0.146.0
+uvicorn==0.54.0
+pydantic==2.14.1
+openai==3.35.1
+requests==2.34.2
+python-dotenv==1.2.4

scripts/evaluate_baselines.py

@@ -0,0 +1,66 @@
+"""
+Benchmark reference policies over many seeds (in-process, no network).
+
+    python scripts/evaluate_baselines.py --seeds 100
+
+Prints a Markdown table (mean ± std of grader scores per task) used in the README.
+The oracle reads ground truth and is an upper bound, not a real agent.
+"""
+
+from __future__ import annotations
+
+import argparse
+import itertools
+import os
+import sys
+import time
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import numpy as np  # noqa: E402
+
+from pll_cyber.baselines import ConstantAgent, PhysicsHeuristicAgent, RandomAgent  # noqa: E402
+from pll_cyber.environment import PLLCyberEnvironment  # noqa: E402
+from pll_cyber.evaluation import run_episode  # noqa: E402
+
+
+class Oracle:
+    def __init__(self, env):
+        self.env = env
+
+    def act(self, obs):
+        sc = self.env._sc
+        lab = sc.onset_step is not None and obs["step"] >= sc.onset_step
+        return {"attack_detected": lab, "attack_type": sc.attack.type_id if lab else 0, "confidence": 1.0,
+                "protective_action": 2 if (lab and sc.task_id == 2) else 0}
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--seeds", type=int, default=100)
+    args = ap.parse_args()
+    seeds = range(1000, 1000 + args.seeds)          # disjoint from inference.py's evaluation seeds
+    rnd = itertools.count()
+    policies = {
+        "Always alarm": lambda env, t: ConstantAgent(True, [1, 1, 4][t]),
+        "Never alarm": lambda env, t: ConstantAgent(False),
+        "Random (p=0.5)": lambda env, t: RandomAgent(seed=next(rnd)),
+        "Physics heuristic": lambda env, t: PhysicsHeuristicAgent(),
+        "Oracle (ground truth)": lambda env, t: Oracle(env),
+    }
+    env = PLLCyberEnvironment()
+    t0 = time.time()
+    print("| Policy | Task 0 (easy) | Task 1 (medium) | Task 2 (hard) | Mean |")
+    print("|---|---|---|---|---|")
+    for name, make in policies.items():
+        cells, means = [], []
+        for t in range(3):
+            s = np.array([run_episode(make(env, t), t, seed, env)["score"] for seed in seeds])
+            cells.append(f"{s.mean():.3f} ± {s.std():.3f}")
+            means.append(s.mean())
+        print(f"| {name} | " + " | ".join(cells) + f" | {np.mean(means):.3f} |", flush=True)
+    print(f"\n{args.seeds} seeds per task, {time.time() - t0:.0f}s", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()

scripts/validate_submission.sh

@@ -0,0 +1,63 @@
+#!/usr/bin/env bash
+# Pre-submission checks for the PLL Cyber-Guard OpenEnv environment.
+#
+#   ./scripts/validate_submission.sh [SPACE_URL]
+#
+# 1. unit tests            2. openenv validate (structure + runtime)
+# 3. docker build + run    4. inference.py completes and prints valid logs
+# 5. (optional) the live HF Space answers /health and POST /reset
+set -uo pipefail
+cd "$(dirname "$0")/.."
+SPACE_URL="${1:-}"
+PORT="${PORT:-7861}"
+fail=0
+ok()   { printf '  \033[32mPASS\033[0m %s\n' "$1"; }
+bad()  { printf '  \033[31mFAIL\033[0m %s\n' "$1"; fail=1; }
+
+echo "== 1. unit tests"
+python -m pytest -q >/tmp/pll_tests.log 2>&1 && ok "pytest" || { bad "pytest (see /tmp/pll_tests.log)"; tail -5 /tmp/pll_tests.log; }
+
+echo "== 2. openenv validate"
+openenv validate . >/dev/null 2>&1 && ok "structure" || bad "structure"
+python -m uvicorn server.app:app --port "$PORT" >/tmp/pll_server.log 2>&1 &
+SRV=$!
+for _ in $(seq 1 30); do curl -sf "localhost:$PORT/health" >/dev/null && break; sleep 1; done
+openenv validate --url "http://localhost:$PORT" >/tmp/pll_runtime.json 2>/dev/null
+python -c "import json,sys; sys.exit(0 if json.load(sys.stdin)['passed'] else 1)" </tmp/pll_runtime.json \
+  && ok "runtime contract" || bad "runtime contract (see /tmp/pll_runtime.json)"
+
+echo "== 3. inference.py"
+ENV_URL="http://localhost:$PORT" EPISODES_PER_TASK="${EPISODES_PER_TASK:-3}" python inference.py >/tmp/pll_inf.log 2>/tmp/pll_inf.err
+rc=$?
+kill $SRV 2>/dev/null
+starts=$(grep -c '^\[START\] ' /tmp/pll_inf.log); ends=$(grep -c '^\[END\] success=true ' /tmp/pll_inf.log)
+bad_lines=$(grep -E '^\[' /tmp/pll_inf.log | grep -vcE '^\[(START|STEP|END)\] ')
+if [ $rc -eq 0 ] && [ "$starts" -ge 3 ] && [ "$starts" -eq "$ends" ] && [ "$bad_lines" -eq 0 ]; then
+  ok "inference ($starts episodes) — $(grep '^overall' /tmp/pll_inf.log)"
+else
+  bad "inference rc=$rc starts=$starts successful_ends=$ends (see /tmp/pll_inf.log /tmp/pll_inf.err)"
+fi
+
+echo "== 4. docker"
+if command -v docker >/dev/null && docker info >/dev/null 2>&1; then
+  if docker build -q -t pll-cyberguard:validate . >/dev/null; then
+    ok "docker build"
+    cid=$(docker run -d -p 17860:7860 pll-cyberguard:validate)
+    for _ in $(seq 1 40); do curl -sf localhost:17860/health >/dev/null && break; sleep 1; done
+    curl -sf -X POST localhost:17860/reset -H 'content-type: application/json' -d '{"task_id":0,"seed":1}' >/dev/null \
+      && ok "docker run + POST /reset" || bad "docker run + POST /reset"
+    docker rm -f "$cid" >/dev/null
+  else bad "docker build"; fi
+else
+  echo "  SKIP docker (daemon not available)"
+fi
+
+if [ -n "$SPACE_URL" ]; then
+  echo "== 5. HF Space $SPACE_URL"
+  curl -sf "$SPACE_URL/health" >/dev/null && ok "/health" || bad "/health"
+  code=$(curl -s -o /dev/null -w '%{http_code}' -X POST "$SPACE_URL/reset" -H 'content-type: application/json' -d '{}')
+  [ "$code" = "200" ] && ok "POST /reset → 200" || bad "POST /reset → $code"
+fi
+
+[ $fail -eq 0 ] && echo "ALL CHECKS PASSED" || echo "SOME CHECKS FAILED"
+exit $fail

server/__init__.py

@@ -0,0 +1 @@
+"""Server package for the PLL Cyber-Guard OpenEnv environment."""

server/app.py

@@ -1,19 +1,222 @@
 """
-This is the server entry point for openenv validate compatibility.
+FastAPI server for the PLL Cyber-Guard OpenEnv environment.
+
+Built on openenv-core's `create_app`, which provides the standard surface:
+    WS   /ws         persistent sessions (one environment per connection) — used by EnvClient
+    GET  /schema     action / observation / state JSON schemas
+    GET  /metadata   environment metadata
+    GET  /health     {"status": "healthy"}
+    POST /mcp        MCP JSON-RPC endpoint
+
+openenv's HTTP /reset and /step are stateless (a fresh environment per request),
+which cannot run a multi-step episode. They are replaced here by session-aware
+versions with the same request/response shapes:
+    POST /reset      body: {"task_id": 0-2, "seed": int, "session_id": str?}
+    POST /step       body: {"action": {...}, "session_id": str?}  (a bare action dict is also accepted)
+    GET  /state      ?session_id=...
+Requests without a session_id share the "default" session (handy for curl).
+
+Extras:
+    GET  /tasks      task catalogue
+    GET  /demo       run the physics-heuristic baseline on one episode and return the trace
+    GET  /           landing page with a live demo plot
 """
-import uvicorn
-from src.api import app  # noqa: F401
+
+from __future__ import annotations
+
+import asyncio
+import os
+import time
+from collections import OrderedDict
+from pathlib import Path
+from typing import Any, Dict, Optional
+
+from fastapi import Body, HTTPException, Query
+from fastapi.concurrency import run_in_threadpool
+from fastapi.responses import HTMLResponse
+from fastapi.routing import APIRoute
+from openenv.core.env_server.http_server import create_app
+from pydantic import BaseModel, ConfigDict, Field, ValidationError
+
+from pll_cyber import __version__
+from pll_cyber.baselines import PhysicsHeuristicAgent
+from pll_cyber.environment import PLLCyberEnvironment
+from pll_cyber.evaluation import obs_to_dict, run_episode
+from pll_cyber.graders import GRADERS
+from pll_cyber.models import PLLAction, PLLObservation, PLLState
+from pll_cyber.rewards import REWARD_MAX, REWARD_MIN
+from pll_cyber.scenarios import MAX_STEPS, TASKS
+
+MAX_HTTP_SESSIONS = int(os.environ.get("MAX_HTTP_SESSIONS", "256"))
+SESSION_TTL_S = float(os.environ.get("SESSION_TTL_S", "1800"))
+MAX_WS_SESSIONS = int(os.environ.get("MAX_WS_SESSIONS", "64"))
+
+app = create_app(
+    PLLCyberEnvironment,
+    PLLAction,
+    PLLObservation,
+    env_name="pll-cyberattack-detection",
+    max_concurrent_envs=MAX_WS_SESSIONS,
+)
+app.title = "PLL Cyber-Guard — OpenEnv"
+app.description = (
+    "Detect, classify and mitigate false-data-injection attacks on the SRF-PLL of a grid-connected "
+    "inverter, without alarming on natural grid disturbances. See /tasks and /docs."
+)
+
+# Drop openenv's stateless HTTP simulation routes; session-aware versions follow.
+app.router.routes = [
+    r for r in app.router.routes
+    if not (isinstance(r, APIRoute) and r.path in {"/reset", "/step", "/state"})
+]
+
+
+# ---------------------------------------------------------------------------
+# HTTP session store
+# ---------------------------------------------------------------------------
+class _Session:
+    def __init__(self):
+        self.env = PLLCyberEnvironment()
+        self.lock = asyncio.Lock()
+        self.touched = time.monotonic()
+        self.started = False
+
+
+_sessions: "OrderedDict[str, _Session]" = OrderedDict()
+_sessions_lock = asyncio.Lock()
+
+
+async def _get_session(session_id: str, create: bool) -> _Session:
+    async with _sessions_lock:
+        now = time.monotonic()
+        for sid in [k for k, s in _sessions.items() if now - s.touched > SESSION_TTL_S]:
+            _sessions.pop(sid, None)
+        sess = _sessions.get(session_id)
+        if sess is None:
+            if not create:
+                raise HTTPException(404, f"Unknown session_id '{session_id}'. Call POST /reset first.")
+            while len(_sessions) >= MAX_HTTP_SESSIONS:
+                _sessions.popitem(last=False)          # evict least-recently used
+            sess = _Session()
+            _sessions[session_id] = sess
+        _sessions.move_to_end(session_id)
+        sess.touched = now
+        return sess
+
+
+class ResetBody(BaseModel):
+    model_config = ConfigDict(extra="allow")
+    task_id: Optional[int] = Field(None, ge=0, le=2, description="0=easy, 1=medium, 2=hard")
+    task: Optional[str] = Field(None, description="Alternatively the task name or difficulty")
+    seed: Optional[int] = Field(None, ge=0, description="Seed for a reproducible episode")
+    episode_id: Optional[str] = None
+    session_id: Optional[str] = Field(None, description="Isolates concurrent clients; default 'default'")
+
+
+class EnvResponse(BaseModel):
+    observation: Dict[str, Any]
+    reward: Optional[float] = None
+    done: bool = False
+    session_id: str
 
 
-def main():
-    """Starting the FastAPI server."""
-    uvicorn.run(
-        "src.api:app",
-        host="0.0.0.0",
-        port=7860,
-        reload=False,
-    )
+def _response(obs: PLLObservation, session_id: str) -> EnvResponse:
+    return EnvResponse(observation=obs_to_dict(obs), reward=obs.reward, done=obs.done, session_id=session_id)
+
+
+@app.post("/reset", response_model=EnvResponse, tags=["Environment Control"],
+          summary="Start a new episode (session-aware)")
+async def reset(body: Optional[ResetBody] = Body(default=None)) -> EnvResponse:
+    body = body or ResetBody()
+    sid = body.session_id or "default"
+    sess = await _get_session(sid, create=True)
+    async with sess.lock:
+        try:
+            obs = await run_in_threadpool(sess.env.reset, seed=body.seed, episode_id=body.episode_id,
+                                          task_id=body.task_id, task=body.task)
+        except ValueError as e:
+            raise HTTPException(422, str(e))
+        sess.started = True
+        return _response(obs, sid)
+
+
+@app.post("/step", response_model=EnvResponse, tags=["Environment Control"],
+          summary="Apply an action to the session's episode")
+async def step(body: Dict[str, Any] = Body(..., examples=[{
+    "action": {"attack_detected": False, "attack_type": 0, "confidence": 0.9, "protective_action": 0},
+    "session_id": "default"}])) -> EnvResponse:
+    sid = body.get("session_id") or "default"
+    raw = body.get("action", {k: v for k, v in body.items() if k not in ("session_id", "timeout_s", "request_id")})
+    try:
+        action = PLLAction.model_validate(raw)
+    except ValidationError as e:
+        raise HTTPException(422, e.errors(include_url=False))
+    sess = await _get_session(sid, create=False)
+    async with sess.lock:
+        if not sess.started:
+            raise HTTPException(400, "Call POST /reset before /step.")
+        obs = await run_in_threadpool(sess.env.step, action)
+        return _response(obs, sid)
+
+
+@app.get("/state", response_model=PLLState, tags=["State Management"],
+         summary="Episode state (ground truth is revealed only after done)")
+async def state(session_id: str = Query("default")) -> PLLState:
+    sess = await _get_session(session_id, create=True)
+    return sess.env.state
+
+
+@app.get("/tasks", tags=["Environment Info"], summary="Task catalogue")
+async def tasks() -> Dict[str, Any]:
+    return {
+        "episode_steps": MAX_STEPS,
+        "decision_interval_ms": 50,
+        "reward_range": [round(REWARD_MIN, 4), round(REWARD_MAX, 4)],
+        "tasks": [
+            {"id": t.task_id, "name": t.name, "difficulty": t.difficulty,
+             "attack_probability": t.attack_probability, "grader": GRADERS[t.task_id].__name__,
+             "score_range": [0.0, 1.0], "description": t.brief}
+            for t in TASKS.values()
+        ],
+        "action_schema": PLLAction.model_json_schema(),
+    }
+
+
+@app.get("/demo", tags=["Environment Info"], summary="Run the heuristic baseline on one episode")
+async def demo(task_id: int = Query(2, ge=0, le=2), seed: int = Query(7, ge=0)) -> Dict[str, Any]:
+    trace = []
+
+    def record(i, action, obs):
+        f = obs.features
+        trace.append({"step": i, "time_s": obs.time_s, "f_local_hz": f.f_local_hz, "pmu_f_hz": obs.pmu.f_hz,
+                      "vq_rms_pu": f.vq_rms_pu, "v_zero_seq_rms_pu": f.v_zero_seq_rms_pu,
+                      "mode": obs.inverter_mode, "reward": obs.reward, "action": action})
+
+    res = await run_in_threadpool(run_episode, PhysicsHeuristicAgent(), task_id, seed, None, record)
+    return {"task_id": task_id, "seed": seed, "score": res["score"], "total_reward": res["total_reward"],
+            "breakdown": res["breakdown"], "scenario": res["scenario"], "trace": trace}
+
+
+_LANDING = Path(__file__).with_name("landing.html")
+
+
+@app.get("/", response_class=HTMLResponse, include_in_schema=False)
+async def landing() -> HTMLResponse:
+    html = _LANDING.read_text(encoding="utf-8") if _LANDING.exists() else "<h1>PLL Cyber-Guard</h1>"
+    return HTMLResponse(html.replace("{{VERSION}}", __version__))
+
+
+def main(host: str = "0.0.0.0", port: int = int(os.environ.get("PORT", "7860"))) -> None:
+    import uvicorn
+
+    uvicorn.run(app, host=host, port=port)
 
 
 if __name__ == "__main__":
-    main()
+    import argparse
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--host", default="0.0.0.0")
+    parser.add_argument("--port", type=int, default=int(os.environ.get("PORT", "7860")))
+    args = parser.parse_args()
+    main(args.host, args.port)

server/landing.html

@@ -0,0 +1,146 @@
+<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>PLL Cyber-Guard</title>
+<style>
+  :root {
+    --bg: #f7f8fa; --panel: #ffffff; --text: #1b1f24; --muted: #5b6573; --border: #dde1e6;
+    --accent: #2f6fde; --local: #2f6fde; --pmu: #d9822b; --attack: #c93c3c; --ok: #2e8b57; --code: #eef1f5;
+  }
+  @media (prefers-color-scheme: dark) {
+    :root { --bg: #0f1216; --panel: #171b21; --text: #e6e9ee; --muted: #9aa4b2; --border: #2a313a;
+            --accent: #6ea0ff; --local: #6ea0ff; --pmu: #f0a35a; --attack: #ff6b6b; --ok: #4fc58a; --code: #1f252d; }
+  }
+  * { box-sizing: border-box; }
+  body { margin: 0; background: var(--bg); color: var(--text);
+         font: 15px/1.55 system-ui, -apple-system, "Segoe UI", Roboto, sans-serif; }
+  main { max-width: 980px; margin: 0 auto; padding: 32px 16px 56px; }
+  h1 { font-size: 28px; margin: 0 0 4px; letter-spacing: -0.01em; }
+  h2 { font-size: 18px; margin: 32px 0 10px; }
+  p { margin: 8px 0; }
+  .sub { color: var(--muted); margin-bottom: 20px; }
+  .panel { background: var(--panel); border: 1px solid var(--border); border-radius: 10px; padding: 18px; }
+  .grid3 { display: grid; grid-template-columns: repeat(3, 1fr); gap: 12px; }
+  @media (max-width: 720px) { .grid3 { grid-template-columns: 1fr; } }
+  .tag { display: inline-block; font-size: 12px; padding: 1px 8px; border-radius: 99px; border: 1px solid var(--border); color: var(--muted); }
+  code, pre { font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; font-size: 13px; }
+  pre { background: var(--code); padding: 12px; border-radius: 8px; overflow-x: auto; margin: 8px 0; }
+  code { background: var(--code); padding: 1px 5px; border-radius: 4px; }
+  .controls { display: flex; flex-wrap: wrap; gap: 10px; align-items: center; margin-bottom: 12px; }
+  select, input, button { font: inherit; padding: 6px 10px; border-radius: 6px; border: 1px solid var(--border);
+                          background: var(--panel); color: var(--text); }
+  button { background: var(--accent); border-color: var(--accent); color: #fff; cursor: pointer; }
+  button:disabled { opacity: .6; cursor: wait; }
+  svg { width: 100%; height: auto; display: block; }
+  .legend { display: flex; flex-wrap: wrap; gap: 16px; font-size: 13px; color: var(--muted); margin-top: 6px; }
+  .sw { display: inline-block; width: 14px; height: 3px; vertical-align: middle; margin-right: 6px; }
+  .kv { display: grid; grid-template-columns: max-content 1fr; gap: 2px 14px; font-size: 14px; margin-top: 10px; }
+  .kv div:nth-child(odd) { color: var(--muted); }
+  a { color: var(--accent); }
+</style>
+</head>
+<body>
+<main>
+  <h1>PLL Cyber-Guard</h1>
+  <p class="sub">An OpenEnv environment for cyber-attack detection on grid-connected inverters · v{{VERSION}}</p>
+
+  <div class="panel">
+    <p>Grid-following inverters synchronise to the grid with a <b>phase-locked loop</b>. An attacker who can
+      rewrite its voltage measurements (false-data injection) can desynchronise the inverter from the real grid.
+      The agent watches PLL signals simulated at 5 kHz plus a trusted but delayed PMU stream. Every 50 ms it must decide
+      whether an attack is under way, classify it, and engage a protective action in time. It must also avoid
+      alarming on real grid events: phase jumps, sags, single-phase faults, harmonic steps and frequency events.</p>
+  </div>
+
+  <h2>Tasks</h2>
+  <div class="grid3">
+    <div class="panel"><span class="tag">easy · task 0</span><p><b>Sinusoidal FDI detection.</b> Spot an interharmonic injection on one phase quickly. Some episodes have no attack.</p></div>
+    <div class="panel"><span class="tag">medium · task 1</span><p><b>Classify under disturbances.</b> Tell sinusoidal, ramp and pulse injections apart while natural grid events happen.</p></div>
+    <div class="panel"><span class="tag">hard · task 2</span><p><b>Stealthy phase spoofing.</b> A balanced angle drift the PLL tracks perfectly. Cross-check against the PMU and hold over before 10° of desync.</p></div>
+  </div>
+
+  <h2>Live demo · physics-heuristic baseline</h2>
+  <div class="panel">
+    <div class="controls">
+      <label>Task <select id="task"><option value="0">0 · easy</option><option value="1">1 · medium</option><option value="2" selected>2 · hard</option></select></label>
+      <label>Seed <input id="seed" type="number" min="0" value="1" style="width:90px"></label>
+      <button id="run">Run episode</button>
+      <span id="status" class="tag">idle</span>
+    </div>
+    <svg id="chart" viewBox="0 0 900 300" role="img" aria-label="Local PLL frequency versus PMU frequency"></svg>
+    <div class="legend">
+      <span><span class="sw" style="background:var(--local)"></span>local PLL frequency</span>
+      <span><span class="sw" style="background:var(--pmu)"></span>PMU frequency (delayed, trusted)</span>
+      <span><span class="sw" style="background:var(--attack)"></span>attack onset</span>
+      <span><span class="sw" style="background:var(--ok);height:8px;opacity:.35"></span>agent alarm raised</span>
+    </div>
+    <div class="kv" id="summary"></div>
+  </div>
+
+  <h2>Quick start</h2>
+  <pre>curl -X POST $URL/reset -H 'content-type: application/json' -d '{"task_id": 2, "seed": 7, "session_id": "me"}'
+curl -X POST $URL/step  -H 'content-type: application/json' \
+     -d '{"session_id": "me", "action": {"attack_detected": false, "attack_type": 0, "confidence": 0.9, "protective_action": 0}}'</pre>
+  <p>Endpoints: <code>POST /reset</code> <code>POST /step</code> <code>GET /state</code> <code>GET /tasks</code>
+     <code>GET /schema</code> <code>GET /metadata</code> <code>GET /health</code> <code>WS /ws</code> ·
+     <a href="/docs">OpenAPI docs</a> ·
+     <a href="https://github.com/KrisKeshav/PLL-Cyberattack-Detection-OpenEnv">source</a></p>
+</main>
+<script>
+const $ = (id) => document.getElementById(id);
+const css = (v) => getComputedStyle(document.documentElement).getPropertyValue(v).trim();
+
+function draw(data) {
+  const svg = $("chart"), W = 900, H = 300, L = 60, R = 16, T = 14, B = 34;
+  const tr = data.trace, n = tr.length;
+  const ys = tr.flatMap(p => [p.f_local_hz, p.pmu_f_hz]);
+  let lo = Math.min(...ys), hi = Math.max(...ys);
+  const pad = Math.max(0.02, (hi - lo) * 0.12); lo -= pad; hi += pad;
+  const x = (i) => L + (W - L - R) * i / Math.max(1, n - 1);
+  const y = (v) => T + (H - T - B) * (hi - v) / (hi - lo);
+  const path = (k) => tr.map((p, i) => `${i ? "L" : "M"}${x(i).toFixed(1)},${y(p[k]).toFixed(1)}`).join("");
+  let s = "";
+  for (let k = 0; k <= 4; k++) {
+    const v = lo + (hi - lo) * k / 4, yy = y(v);
+    s += `<line x1="${L}" x2="${W - R}" y1="${yy}" y2="${yy}" stroke="${css('--border')}"/>`;
+    s += `<text x="${L - 8}" y="${yy + 4}" text-anchor="end" font-size="11" fill="${css('--muted')}">${v.toFixed(2)}</text>`;
+  }
+  tr.forEach((p, i) => { if (p.action.attack_detected) s += `<rect x="${x(i) - 5}" y="${T}" width="10" height="${H - T - B}" fill="${css('--ok')}" opacity=".16"/>`; });
+  const onset = data.scenario.onset_step;
+  if (onset !== null && onset !== undefined) {
+    s += `<line x1="${x(onset)}" x2="${x(onset)}" y1="${T}" y2="${H - B}" stroke="${css('--attack')}" stroke-width="2" stroke-dasharray="5 4"/>`;
+  }
+  s += `<path d="${path('pmu_f_hz')}" fill="none" stroke="${css('--pmu')}" stroke-width="2"/>`;
+  s += `<path d="${path('f_local_hz')}" fill="none" stroke="${css('--local')}" stroke-width="2"/>`;
+  s += `<text x="${(L + W - R) / 2}" y="${H - 6}" text-anchor="middle" font-size="12" fill="${css('--muted')}">decision step (50 ms each)</text>`;
+  s += `<text x="14" y="${(T + H - B) / 2}" font-size="12" fill="${css('--muted')}" transform="rotate(-90 14 ${(T + H - B) / 2})" text-anchor="middle">Hz</text>`;
+  svg.innerHTML = s;
+  const sc = data.scenario, b = data.breakdown;
+  const rows = [
+    ["grader score", data.score.toFixed(3)],
+    ["attack", sc.attack_type_name + (onset != null ? ` (onset step ${onset})` : "")],
+    ["grid events", sc.grid.events.map(e => e.kind).join(", ") || "none"],
+    ["detection delay", b.detection_delay_ms == null ? "n/a" : b.detection_delay_ms + " ms"],
+    ["false-alarm rate", (100 * b.false_alarm_rate).toFixed(1) + " %"],
+    ["unprotected desync at", sc.unmitigated_desync_step == null ? "never" : "step " + sc.unmitigated_desync_step],
+    ["actual max sync error", sc.max_sync_error_deg.toFixed(1) + "°"],
+  ];
+  $("summary").innerHTML = rows.map(([k, v]) => `<div>${k}</div><div>${v}</div>`).join("");
+}
+
+async function run() {
+  $("run").disabled = true; $("status").textContent = "running…";
+  try {
+    const r = await fetch(`/demo?task_id=${$("task").value}&seed=${Math.max(0, +$("seed").value || 0)}`);
+    if (!r.ok) throw new Error(r.status);
+    draw(await r.json()); $("status").textContent = "done";
+  } catch (e) { $("status").textContent = "error " + e.message; }
+  $("run").disabled = false;
+}
+$("run").addEventListener("click", run);
+run();
+</script>
+</body>
+</html>

src/__init__.py

@@ -1 +0,0 @@
-# PLL Cyberattack Detection OpenEnv

src/api.py

@@ -1,111 +0,0 @@
-"""
-FastAPI application for the PLL Cyberattack Detection OpenEnv.
-
-Exposes HTTP endpoints for environment interaction:
-  POST /reset   — Reset environment with task_id
-  POST /step    — Submit an action and advance one step
-  GET  /state   — Get current internal state
-  GET  /health  — Health check (returns 200)
-  GET  /tasks   — List available tasks
-"""
-
-import asyncio
-from typing import Any, Dict, Optional
-
-from fastapi import FastAPI, HTTPException, Request
-from pydantic import BaseModel
-
-from src.models import Observation, Action, Reward, State
-from src.env import PLLAttackEnv
-
-app = FastAPI(
-    title="PLL Cyberattack Detection OpenEnv",
-    description="OpenEnv for AI-driven cyberattack detection on SRF-PLLs",
-    version="1.0.0",
-)
-
-env = PLLAttackEnv()
-env_lock = asyncio.Lock()
-
-
-class ResetRequest(BaseModel):
-    task_id: int = 0
-    seed: Optional[int] = None
-
-
-class StepResponse(BaseModel):
-    observation: Observation
-    reward: Reward
-    done: bool
-    info: Dict[str, Any]
-
-
-@app.post("/reset", response_model=Observation)
-async def reset(req: Request):
-    """Reset the environment and return initial observation."""
-    async with env_lock:
-        try:
-            body = await req.body()
-            if body:
-                data = await req.json()
-                request = ResetRequest(**data)
-            else:
-                request = ResetRequest()
-        except Exception:
-            request = ResetRequest()
-        return env.reset(task_id=request.task_id, seed=request.seed)
-
-
-@app.post("/step", response_model=StepResponse)
-async def step(action: Action):
-    async with env_lock:
-        if env.attack_generator is None:
-            raise HTTPException(status_code=400, detail="Call /reset before /step")
-        obs, reward, done, info = env.step(action)
-        return StepResponse(observation=obs, reward=reward, done=done, info=info)
-
-
-@app.get("/state", response_model=State)
-async def get_state():
-    async with env_lock:
-        return env.get_state()
-
-
-@app.get("/health")
-async def health():
-    """Health check endpoint."""
-    return {
-        "status": "ok",
-        "version": "1.0.0",
-        "environment": "pll-cyberattack-detection",
-        "tasks_available": 3,
-        "episode_active": env.attack_generator is not None,
-        "current_step": env.step_count,
-    }
-
-
-@app.get("/tasks")
-async def list_tasks():
-    """List all available tasks."""
-    return {
-        "tasks": [
-            {
-                "id": 0,
-                "name": "sinusoidal_fdi",
-                "difficulty": "easy",
-                "description": "Detect sinusoidal FDI attack"
-            },
-            {
-                "id": 1,
-                "name": "multi_attack_classification",
-                "difficulty": "medium",
-                "description": "Classify attack type from observations"
-            },
-            {
-                "id": 2,
-                "name": "stealthy_attack_detection",
-                "difficulty": "hard",
-                "description": "Detect stealthy attack before PLL lock loss"
-            },
-        ]
-    }

src/attacks.py

@@ -1,134 +0,0 @@
-"""
-Attack injection logic for the PLL Cyberattack Detection OpenEnv.
-
-Implements four attack types:
-  1. Sinusoidal FDI (Easy)
-  2. Ramp injection (Medium)
-  3. Pulse/step bias (Medium)
-  4. Stealthy low-and-slow phase drift (Hard)
-"""
-
-import math
-import numpy as np
-from typing import Dict, Any
-
-
-def sample_sinusoidal_params(rng: np.random.Generator) -> Dict[str, Any]:
-    """Sample parameters for a sinusoidal FDI attack."""
-    return {
-        "type": "sinusoidal",
-        "amplitude": float(rng.uniform(0.05, 0.20)),
-        "freq": float(rng.uniform(5.0, 20.0)),
-        "phase": float(rng.uniform(0.0, 2.0 * math.pi)),
-    }
-
-
-def sample_ramp_params(rng: np.random.Generator) -> Dict[str, Any]:
-    """Sample parameters for a ramp injection attack."""
-    return {
-        "type": "ramp",
-        "rate": float(rng.uniform(0.0002, 0.001)),
-    }
-
-
-def sample_pulse_params(rng: np.random.Generator) -> Dict[str, Any]:
-    """Sample parameters for a pulse/step bias attack."""
-    return {
-        "type": "pulse",
-        "magnitude": float(rng.uniform(0.1, 0.3)),
-        "duration": int(rng.integers(20, 81)),  # 20 to 80 steps inclusive
-    }
-
-
-def sample_stealthy_params(rng: np.random.Generator) -> Dict[str, Any]:
-    """Sample parameters for a stealthy low-and-slow attack."""
-    return {
-        "type": "stealthy",
-        "amplitude": 0.03,
-        "drift_rate": float(rng.uniform(0.05, 0.2)),
-    }
-
-
-def sample_attack_start(rng: np.random.Generator) -> int:
-    """Sample a random attack start step between 30 and 80 inclusive."""
-    return int(rng.integers(30, 81))
-
-
-class AttackGenerator:
-    """Generates attack signals given parameters and current simulation state."""
-
-    def __init__(self, attack_params: Dict[str, Any], attack_start_step: int):
-        self.params = attack_params
-        self.attack_start_step = attack_start_step
-        self.attack_type_str = attack_params.get("type", "none")
-
-        # For stealthy attack: track cumulative phase drift
-        self.delta = 0.0
-
-    def get_signal(self, current_step: int, sim_time: float) -> float:
-        """
-        Compute the attack signal value at the given step.
-        Args:
-            current_step: Current environment step (0-indexed).
-            sim_time: Current simulation time in seconds.
-        Returns:
-            Attack signal value (pu). Returns 0.0 if attack not yet started.
-        """
-        if current_step < self.attack_start_step:
-            return 0.0
-
-        steps_since_start = current_step - self.attack_start_step
-        dt = 1e-3  # time step
-
-        if self.attack_type_str == "sinusoidal":
-            A = self.params["amplitude"]
-            fa = self.params["freq"]
-            phi = self.params["phase"]
-            return A * math.sin(2.0 * math.pi * fa * sim_time + phi)
-
-        elif self.attack_type_str == "ramp":
-            rate = self.params["rate"]
-            return rate * steps_since_start
-
-        elif self.attack_type_str == "pulse":
-            mag = self.params["magnitude"]
-            dur = self.params["duration"]
-            if steps_since_start < dur:
-                return mag
-            else:
-                return 0.0
-
-        elif self.attack_type_str == "stealthy":
-            A_s = self.params["amplitude"]
-            drift_rate = self.params["drift_rate"]
-            # δ(t) = δ(t-1) + drift_rate * Δt — accumulated each call
-            self.delta += drift_rate * dt
-            f0 = 50.0
-            return A_s * math.sin(2.0 * math.pi * f0 * sim_time + self.delta)
-
-        return 0.0
-
-    def is_active(self, current_step: int) -> bool:
-        """Check whether the attack is currently active at this specific step."""
-        if current_step < self.attack_start_step:
-            return False
-
-        # Pulse attacks end after duration
-        if self.attack_type_str == "pulse":
-            steps_since_start = current_step - self.attack_start_step
-            dur = self.params["duration"]
-            return steps_since_start < dur
-
-        return True
-
-
-def get_attack_type_id(attack_type_str: str) -> int:
-    """Map an attack type string to its corresponding integer ID."""
-    mapping = {
-        "none": 0,
-        "sinusoidal": 1,
-        "ramp": 2,
-        "pulse": 3,
-        "stealthy": 4,
-    }
-    return mapping.get(attack_type_str, 0)

src/detector.py

@@ -1,139 +0,0 @@
-"""
-Adaptive Physics-informed cyberattack detector for the PLL OpenEnv.
-
-Uses residual-based and pattern-based features derived from the
-observation windows to detect, classify, and recommend protective
-actions. The detector builds a baseline from the first 20 observations
-(warmup window) of the episode to normalize the features individually.
-"""
-
-import numpy as np
-from typing import Dict, Any
-
-from src.models import Observation
-
-
-class AdaptiveDetector:
-    def __init__(self):
-        # Baseline collections
-        self.r1_history = []
-        self.r3_history = []
-        self.r4_history = []
-        self.r5_history = []
-
-        # Calibrated statistics
-        self.mean_R1 = 0.0
-        self.std_R1 = 1e-6
-        self.mean_R3 = 0.0
-        self.std_R3 = 1e-6
-        self.mean_R4 = 0.0
-        self.std_R4 = 1e-6
-        self.mean_R5 = 0.0
-        self.std_R5 = 1e-6
-
-        self.is_calibrated = False
-
-    def detect(self, observation) -> Dict[str, Any]:
-        """
-        Run physics-informed anomaly detection on the current observation.
-        """
-        vq = np.array(observation.vq_window, dtype=np.float64)
-        vd = np.array(observation.vd_window, dtype=np.float64)
-        omega = np.array(observation.omega_window, dtype=np.float64)
-        omega_dev = np.array(observation.omega_deviation_window, dtype=np.float64)
-        va, vb, vc = observation.raw_voltages
-
-        # ---- Step 1: Feature Extraction --------------------------------
-        vq_mean = float(np.mean(np.abs(vq)))
-        vd_mean = float(np.mean(np.abs(vd)))
-        vq_ratio = vq_mean / (vd_mean + 1e-6)
-
-        omega_var = float(np.var(omega))
-        omega_dev_var = float(np.var(omega_dev))
-        vd_var = float(np.var(vd))
-        
-        abs_v_sum = abs(va) + abs(vb) + abs(vc) + 1e-6
-        symmetry_ratio = float(abs(va + vb + vc) / abs_v_sum)
-
-        vq_diff = np.diff(vq) if len(vq) > 1 else np.array([0.0])
-        vq_trend = float(np.mean(vq_diff))
-        vq_spike = float(np.max(np.abs(vq)))
-        vq_drift = float(np.sum(vq))
-
-        step = observation.step
-
-        # ---- Step 2: Baseline Calibration ------------------------------
-        if step < 20:
-            self.r1_history.append(vq_ratio)
-            self.r3_history.append(omega_var)
-            self.r4_history.append(vd_var)
-            self.r5_history.append(symmetry_ratio)
-            
-            return {
-                "attack_detected": False,
-                "attack_type": 0,
-                "confidence": 0.0,
-                "protective_action": 0,
-                "score": 0.0,
-                "baseline_score": 0.0
-            }
-        
-        if not self.is_calibrated:
-            self.mean_R1 = float(np.mean(self.r1_history))
-            self.std_R1 = max(float(np.std(self.r1_history)), 1e-6)
-            
-            self.mean_R3 = float(np.mean(self.r3_history))
-            self.std_R3 = max(float(np.std(self.r3_history)), 1e-6)
-            
-            self.mean_R4 = float(np.mean(self.r4_history))
-            self.std_R4 = max(float(np.std(self.r4_history)), 1e-6)
-            
-            self.mean_R5 = float(np.mean(self.r5_history))
-            self.std_R5 = max(float(np.std(self.r5_history)), 1e-6)
-            
-            self.is_calibrated = True
-
-        # ---- Step 3: Normalized Features ------------------------------
-        R1 = (vq_ratio - self.mean_R1) / self.std_R1
-        R3 = (omega_var - self.mean_R3) / self.std_R3
-        R4 = (vd_var - self.mean_R4) / self.std_R4
-        R5 = (symmetry_ratio - self.mean_R5) / self.std_R5
-
-        # ---- Step 4: Score --------------------------------------------
-        score = 0.4 * R1 + 0.2 * R3 + 0.2 * R5 + 0.2 * R4
-
-        # ---- Step 5: Detection ----------------------------------------
-        attack_detected = score > 5.0
-        confidence = min(1.0, score / 5.0) if attack_detected else 0.0
-
-        # ---- Step 6: Classification -----------------------------------
-        if not attack_detected:
-            attack_type = 0
-        else:
-            if R3 > 2:
-                attack_type = 1  # sinusoidal
-            elif abs(vq_trend) > 0.01:
-                attack_type = 2  # ramp
-            elif vq_spike > 0.1:
-                attack_type = 3  # pulse
-            else:
-                attack_type = 4  # stealthy
-
-        # ---- Step 7: Protective Action --------------------------------
-        if score > 6:
-            protective_action = 3
-        elif score > 3:
-            protective_action = 2
-        else:
-            protective_action = 1
-            if not attack_detected:
-                protective_action = 0
-
-        return {
-            "attack_detected": bool(attack_detected),
-            "attack_type": int(attack_type),
-            "confidence": float(confidence),
-            "protective_action": int(protective_action),
-            "score": float(score),
-            "baseline_score": 0.0
-        }

src/env.py

@@ -1,370 +0,0 @@
-"""
-Main environment class for the PLL Cyberattack Detection OpenEnv.
-
-Implements step(), reset(), get_state(), and compute_reward().
-Manages the PLL simulation, attack injection, observation windowing,
-episode history, and grading.
-"""
-
-import uuid
-import numpy as np
-from typing import Tuple, Dict, Any, List, Optional
-from collections import deque
-
-from src.models import Observation, Action, Reward, State
-from src.pll_sim import SRFPLLSimulator, OMEGA0
-from src.attacks import (
-    AttackGenerator,
-    sample_sinusoidal_params,
-    sample_ramp_params,
-    sample_pulse_params,
-    sample_stealthy_params,
-    sample_attack_start,
-    get_attack_type_id,
-)
-from src.graders import grade_task_easy, grade_task_medium, grade_task_hard
-from src.detector import AdaptiveDetector
-
-
-WINDOW_SIZE = 20
-MAX_STEPS = 500
-LOCK_LOSS_THRESHOLD = 0.0873  # 5 degrees in radians
-
-DETECTION_THRESHOLD = 2.0
-EARLY_DETECTION_WINDOW = 100
-FALSE_ALARM_PENALTY = -0.2
-TRUE_POSITIVE_REWARD = 0.1
-TRUE_NEGATIVE_REWARD = 0.05
-MISSED_DETECTION_PENALTY = -0.05
-CLASSIFICATION_BONUS = 0.05
-LOCK_LOSS_PENALTY = -2.0
-
-
-class PLLAttackEnv:
-    """OpenEnv-compliant PLL cyberattack detection environment."""
-
-    def __init__(self):
-        self.pll = SRFPLLSimulator()
-        self.rng: Optional[np.random.Generator] = None
-        self.task_id = 0
-        self.step_count = 0
-        self.episode_id = ""
-        self.done = False
-
-        # Attack state
-        self.attack_generator: Optional[AttackGenerator] = None
-        self.attack_active = False
-        self.attack_type = 0
-        self.attack_params: Dict[str, Any] = {}
-        self.attack_start_step = 0
-        self.true_attack_type = 0
-
-        # Detection tracking
-        self.first_detection_recorded = False
-        self.first_detection_step = 0
-
-        # Lock loss tracking (Task 2 / hard)
-        self.lock_lost = False
-        self.lock_loss_step: Optional[int] = None
-        self.lock_loss_penalized = False
-
-        # Observation windows
-        self.vq_window: deque = deque(maxlen=WINDOW_SIZE)
-        self.vd_window: deque = deque(maxlen=WINDOW_SIZE)
-        self.omega_window: deque = deque(maxlen=WINDOW_SIZE)
-        self.omega_deviation_window: deque = deque(maxlen=WINDOW_SIZE)
-
-        # Detector
-        self.detector = AdaptiveDetector()
-
-        # Episode history for grading
-        self.history: List[Dict[str, Any]] = []
-
-    # ------------------------------------------------------------------
-    # Public API
-    # ------------------------------------------------------------------
-
-    def reset(self, task_id: int = 0, seed: Optional[int] = None) -> Observation:
-        """
-        Reset the environment for a new episode.
-        Args:
-            task_id: 0=easy (sinusoidal), 1=medium (multi-type),
-                     2=hard (stealthy).
-            seed:    Optional RNG seed for reproducibility.
-        Returns:
-            Initial Observation with non-zero raw_voltages.
-        """
-        self.rng = np.random.default_rng(seed)  # seed=None → random
-
-        self.task_id = task_id
-        self.step_count = 0
-        self.episode_id = str(uuid.uuid4())
-        self.done = False
-
-        # Reset PLL simulator
-        self.pll.reset()
-
-        # Reset detection tracking
-        self.first_detection_recorded = False
-        self.first_detection_step = 0
-
-        # Reset lock-loss tracking
-        self.lock_lost = False
-        self.lock_loss_step = None
-        self.lock_loss_penalized = False
-
-        # Reset history
-        self.history = []
-
-        # Reset observation windows
-        self.vq_window = deque(maxlen=WINDOW_SIZE)
-        self.vd_window = deque(maxlen=WINDOW_SIZE)
-        self.omega_window = deque(maxlen=WINDOW_SIZE)
-        self.omega_deviation_window = deque(maxlen=WINDOW_SIZE)
-
-        # Reset detector
-        self.detector = AdaptiveDetector()
-
-        # Sample attack for this episode
-        self._setup_attack()
-      
-        for _ in range(WINDOW_SIZE):
-            pll_out = self.pll.step(0.0)  # no attack during warm-up
-            omega_norm = (pll_out["omega_hat"] - OMEGA0) / OMEGA0
-            omega_dev  = pll_out["omega_hat"] - OMEGA0
-            self.vq_window.append(pll_out["vq"])
-            self.vd_window.append(pll_out["vd"])
-            self.omega_window.append(omega_norm)
-            self.omega_deviation_window.append(omega_dev)
-        # step_count stays at 0 — warm-up steps are invisible to the agent
-
-        return self._get_observation()
-
-    def step(self, action: Action) -> Tuple[Observation, Reward, bool, Dict[str, Any]]:
-        """
-        Advance the environment by one step.
-        Args:
-            action: Agent's Action for this step.
-        Returns:
-            (observation, reward, done, info)
-        """
-        if self.done:
-            return (
-                self._get_observation(),
-                Reward(
-                    total=0.0, detection_reward=0.0, classification_bonus=0.0,
-                    early_detection_bonus=0.0, false_alarm_penalty=0.0,
-                    lock_loss_penalty=0.0,
-                ),
-                True,
-                {"message": "Episode already done. Call /reset to start a new episode."},
-            )
-
-        # --- Attack signal ------------------------------------------------
-        # attack_active uses is_active() (step-based). It does NOT depend on the instantaneous
-        # signal value, because the attack signal can cross zero even while the attack is active.
-        attack_signal = self.attack_generator.get_signal(self.step_count, self.pll.t)
-        self.attack_active = self.attack_generator.is_active(self.step_count)
-
-        # --- Advance PLL --------------------------------------------------
-        pll_out = self.pll.step(attack_signal)
-
-        # --- Updating observation windows -----------------------------------
-        omega_norm = (pll_out["omega_hat"] - OMEGA0) / OMEGA0
-        omega_dev  = pll_out["omega_hat"] - OMEGA0  # raw deviation (rad/s)
-        self.vq_window.append(pll_out["vq"])
-        self.vd_window.append(pll_out["vd"])
-        self.omega_window.append(omega_norm)
-        self.omega_deviation_window.append(omega_dev)
-
-        # --- Lock-loss check (Task 2) -------------------------
-        PLL_CONVERGENCE_STEPS = 60  # PLL transient settles by ~step 50, using 60 for margin
-        if (
-            self.task_id == 2
-            and not self.lock_lost
-            and self.step_count > self.attack_start_step
-            and self.step_count > PLL_CONVERGENCE_STEPS   # guard against startup transient
-        ):
-            if abs(pll_out["theta_err"]) > LOCK_LOSS_THRESHOLD:
-                self.lock_lost = True
-                self.lock_loss_step = self.step_count
-
-        # --- Reward -------------------------------------------------------
-        reward = self.compute_reward(action)
-
-        # --- Record history entry for graders ----------------------------
-        self.history.append({
-            "step":              self.step_count,
-            "attack_active":     self.attack_active,
-            "attack_detected":   action.attack_detected,
-            "true_attack_type":  self.true_attack_type,
-            "agent_attack_type": action.attack_type,
-            "theta_err":         pll_out["theta_err"],
-        })
-
-        # --- Advance step counter ----------------------------------------
-        self.step_count += 1
-
-        # Terminate Task 2 early upon losing lock to save computational steps
-        if self.step_count >= MAX_STEPS:
-            self.done = True
-        elif self.task_id == 2 and self.lock_lost:
-            self.done = True
-
-        # --- Physics-informed detector (evaluation/debug only) ------------
-        detector_output = self.detector.detect(self._get_observation())
-
-        # --- Build info --------------------------------------------------
-        info: Dict[str, Any] = {
-            "detector": detector_output,
-            "detector_features": {"step": self.step_count, "raw_score": detector_output.get("score")}
-        }
-        if self.done:
-            info["grader_score"] = self._compute_grader_score()
-            info["episode_id"]   = self.episode_id
-            info["total_steps"]  = self.step_count
-            info["lock_lost"]    = self.lock_lost
-
-        return self._get_observation(), reward, self.done, info
-
-    def compute_reward(self, action: Action) -> Reward:
-        """
-        Computes the dense reward signal for the current step.
-        Reward components:
-          detection_reward:     +0.10 true positive (per step)
-                                +0.05 true negative (per step)
-                                -0.05 missed detection (per step)
-          false_alarm_penalty:  -0.20 per false-positive step
-          classification_bonus: +0.05 per step correct type (task 1 only)
-          early_detection_bonus: one-time sparse, scaled by detection speed
-          lock_loss_penalty:    -2.00 one-time on lock loss (task 2 only)
-        """
-        detection_reward      = 0.0
-        false_alarm_penalty   = 0.0
-        classification_bonus  = 0.0
-        early_detection_bonus = 0.0
-        lock_loss_penalty     = 0.0
-
-        if self.attack_active:
-            if action.attack_detected:
-                detection_reward = TRUE_POSITIVE_REWARD
-                # One-time early detection bonus on first correct detection
-                if not self.first_detection_recorded:
-                    self.first_detection_step    = self.step_count
-                    self.first_detection_recorded = True
-                    # Relative steps since attack started
-                    t = self.first_detection_step - self.attack_start_step
-                    early_detection_bonus = max(0.0, 1.0 - t / EARLY_DETECTION_WINDOW)
-            else:
-                detection_reward = MISSED_DETECTION_PENALTY
-        else:
-            if action.attack_detected:
-                false_alarm_penalty = FALSE_ALARM_PENALTY
-            else:
-                detection_reward = TRUE_NEGATIVE_REWARD
-
-        # Task 1 (medium): per-step classification bonus
-        if self.task_id == 1 and self.attack_active:
-            if action.attack_type == self.true_attack_type:
-                classification_bonus = CLASSIFICATION_BONUS
-
-        # Task 2 (hard): one-time lock-loss penalty
-        if self.task_id == 2 and self.lock_lost and not self.lock_loss_penalized:
-            lock_loss_penalty        = LOCK_LOSS_PENALTY
-            self.lock_loss_penalized = True
-
-        total = (
-            detection_reward
-            + false_alarm_penalty
-            + classification_bonus
-            + early_detection_bonus
-            + lock_loss_penalty
-        )
-
-        return Reward(
-            total=total,
-            detection_reward=detection_reward,
-            classification_bonus=classification_bonus,
-            early_detection_bonus=early_detection_bonus,
-            false_alarm_penalty=false_alarm_penalty,
-            lock_loss_penalty=lock_loss_penalty,
-        )
-
-    def get_state(self) -> State:
-        """Returning full internal state for debugging / GET /state endpoint."""
-        return State(
-            theta_true=self.pll.theta_true,
-            theta_hat=self.pll.theta_hat,
-            omega_hat=self.pll.omega_hat,
-            vq_integral=self.pll.vq_integral,
-            attack_active=self.attack_active,
-            attack_type=self.attack_type,
-            attack_params=self.attack_params,
-            attack_start_step=self.attack_start_step,
-            lock_lost=self.lock_lost,
-            step=self.step_count,
-            episode_id=self.episode_id,
-            task_id=self.task_id,
-        )
-
-    # ------------------------------------------------------------------
-    # Private helpers
-    # ------------------------------------------------------------------
-
-    def _setup_attack(self) -> None:
-        """Sample attack type and parameters based on current task_id."""
-        self.attack_start_step = sample_attack_start(self.rng)
-
-        if self.task_id == 0:
-            # Easy: sinusoidal FDI only
-            self.attack_params    = sample_sinusoidal_params(self.rng)
-            self.true_attack_type = 1
-
-        elif self.task_id == 1:
-            # Medium: random choice of sinusoidal / ramp / pulse
-            choice = int(self.rng.integers(0, 3))
-            if choice == 0:
-                self.attack_params    = sample_sinusoidal_params(self.rng)
-                self.true_attack_type = 1
-            elif choice == 1:
-                self.attack_params    = sample_ramp_params(self.rng)
-                self.true_attack_type = 2
-            else:
-                self.attack_params    = sample_pulse_params(self.rng)
-                self.true_attack_type = 3
-
-        elif self.task_id == 2:
-            # Hard: stealthy low-and-slow
-            self.attack_params    = sample_stealthy_params(self.rng)
-            self.true_attack_type = 4
-
-        self.attack_type      = get_attack_type_id(self.attack_params.get("type", "none"))
-        self.attack_generator = AttackGenerator(self.attack_params, self.attack_start_step)
-
-    def _get_observation(self) -> Observation:
-        """
-        Building the current Observation from internal windows.
-        """
-        return Observation(
-            vq_window=list(self.vq_window),
-            vd_window=list(self.vd_window),
-            omega_window=list(self.omega_window),
-            omega_deviation_window=list(self.omega_deviation_window),
-            raw_voltages=[self.pll.va_m, self.pll.vb_m, self.pll.vc_m],
-            task_id=self.task_id,
-            step=self.step_count,
-        )
-
-    def _compute_grader_score(self) -> float:
-        """Running the appropriate grader at episode end."""
-        if self.task_id == 0:
-            return grade_task_easy(self.history, self.attack_start_step)
-        elif self.task_id == 1:
-            return grade_task_medium(self.history, self.attack_start_step)
-        elif self.task_id == 2:
-            return grade_task_hard(
-                self.history,
-                self.lock_loss_step,
-                self.attack_start_step,
-            )
-        return 0.0

src/graders.py

@@ -1,137 +0,0 @@
-"""
-Per-task deterministic graders for the PLL Cyberattack Detection OpenEnv.
-
-Each grader takes an episode history and returns a score in [0.0, 1.0].
-Graders are deterministic given the same episode data.
-"""
-
-from typing import List, Dict, Any, Optional
-
-
-def grade_task_easy(history: List[Dict[str, Any]], attack_start_step: int) -> float:
-    """
-    Task 1 — Sinusoidal FDI Detection (Easy).
-    Grader logic (relative to attack onset):
-      delay = first_correct_detection_step - attack_start_step
-      if delay <= 20:   score = 1.0
-      elif delay <= 100: score = linear decay from 1.0 to 0.5
-      elif delay <= 420: score = 0.2
-      else (never detected): score = 0.0
-    """
-    first_correct_detection_step = None
-
-    for entry in history:
-        step = entry["step"]
-        attack_active = entry["attack_active"]
-        attack_detected = entry["attack_detected"]
-
-        if attack_active and attack_detected:
-            first_correct_detection_step = step
-            break
-
-    if first_correct_detection_step is None:
-        return 0.01
-
-    delay = first_correct_detection_step - attack_start_step
-
-    if delay <= 20:
-        score = 0.99
-    elif delay <= 100:
-        # Linear decay from 0.99 at delay=20 to 0.5 at delay=100
-        score = 0.99 - 0.49 * (delay - 20) / 80.0
-    elif delay <= 420:
-        score = 0.2
-    else:
-        score = 0.01
-
-    return max(0.01, min(0.99, score))
-
-
-def grade_task_medium(history: List[Dict[str, Any]], attack_start_step: int) -> float:
-    """
-    Task 2 — Multi-Attack Classification (Medium).
-    Grader logic:
-      base_score = fraction of steps (after attack_start) where attack_type is correctly classified
-      early_bonus = 0.4 * max(0, 1 - first_correct_classification_step / 100)
-      score = min(1.0, base_score * 0.6 + early_bonus)
-    """
-    steps_after_attack = 0
-    correct_classifications = 0
-    first_correct_classification_step = None
-
-    for entry in history:
-        step = entry["step"]
-        if step < attack_start_step:
-            continue
-
-        steps_after_attack += 1
-        true_type = entry["true_attack_type"]
-        agent_type = entry["agent_attack_type"]
-
-        if agent_type == true_type:
-            correct_classifications += 1
-            if first_correct_classification_step is None:
-                first_correct_classification_step = step
-
-    if steps_after_attack == 0:
-        return 0.01
-
-    base_score = correct_classifications / steps_after_attack
-
-    if first_correct_classification_step is not None:
-        early_bonus = 0.4 * max(0.0, 1.0 - first_correct_classification_step / 100.0)
-    else:
-        early_bonus = 0.0
-
-    score = min(0.99, base_score * 0.6 + early_bonus)
-    return max(0.01, score)
-
-
-def grade_task_hard(
-    history: List[Dict[str, Any]],
-    loss_of_lock_step: Optional[int],
-    attack_start_step: int,
-) -> float:
-    """
-    Task 3 — Stealthy Low-and-Slow Attack (Hard).
-    Grader logic:
-      if detected before loss_of_lock_step:
-          score = 1.0 * (1 - first_detection_step / loss_of_lock_step)
-      elif detected after loss_of_lock but before episode end:
-          score = 0.3
-      else (never detected):
-          score = 0.0
-      false_alarm_penalty = 0.2 per false alarm before attack starts
-      (capped at reducing score to 0.0 minimum)
-    """
-    first_detection_step = None
-    false_alarm_count = 0
-
-    for entry in history:
-        step = entry["step"]
-        attack_active = entry["attack_active"]
-        attack_detected = entry["attack_detected"]
-
-        # Only count false alarms before the attack starts
-        if attack_detected and not attack_active and step < attack_start_step:
-            false_alarm_count += 1
-
-        if attack_detected and attack_active and first_detection_step is None:
-            first_detection_step = step
-
-    # Compute base score
-    if first_detection_step is None:
-        score = 0.0
-    elif loss_of_lock_step is not None and first_detection_step < loss_of_lock_step:
-        score = 1.0 * (1.0 - first_detection_step / loss_of_lock_step)
-    elif loss_of_lock_step is not None and first_detection_step >= loss_of_lock_step:
-        score = 0.3
-    else:
-        # No loss of lock occurred but attack was detected
-        score = 0.3
-
-    # Apply false alarm penalty
-    penalty = 0.2 * false_alarm_count
-    score = max(0.01, score - penalty)
-
-    return max(0.01, min(0.99, score))

src/models.py

@@ -1,72 +0,0 @@
-"""
-Pydantic models for the PLL Cyberattack Detection OpenEnv.
-Defines Observation, Action, Reward, and State schemas.
-"""
-import numpy as np
-from typing import Annotated, Any, Dict, List, Optional
-from pydantic import BaseModel, Field, model_validator
- 
-# Exactly 20 floats — enforced at validation time, not just documented.
-WindowList = Annotated[List[float], Field(min_length=20, max_length=20)]
- 
-# Exactly 3 floats for [va, vb, vc].
-VoltageList = Annotated[List[float], Field(min_length=3, max_length=3)]
- 
-class Observation(BaseModel):
-    vq_window: WindowList
-    vd_window: WindowList
-    omega_window: WindowList
-    omega_deviation_window: WindowList
-    raw_voltages: VoltageList
-    task_id: int = Field(ge=0, le=2)
-    step: int = Field(ge=0)
- 
-class Action(BaseModel):
-    attack_detected: bool
-    attack_type: int = Field(ge=0, le=4)
-    confidence: float = Field(ge=0.0, le=1.0)
-    protective_action: int = Field(ge=0, le=3)
- 
-class Reward(BaseModel):
-    total: float
-    detection_reward: float
-    classification_bonus: float
-    early_detection_bonus: float
-    false_alarm_penalty: float
-    lock_loss_penalty: float
- 
-class State(BaseModel):
-    theta_true: float
-    theta_hat: float
-    omega_hat: float
-    vq_integral: float
-    attack_active: bool
-    attack_type: int         # Integer ID of the current attack: 0=none, 1=sinusoidal, 2=ramp, 3=pulse, 4=stealthy.
-    attack_params: Dict[str, Any]
-    attack_start_step: int
-    lock_lost: bool     # Whether the PLL has lost lock (|theta_err| > 5°). Task 2 only.
-    step: int = Field(ge=0)
-    episode_id: str
-    task_id: int = Field(ge=0, le=2)
- 
-    @model_validator(mode="before")
-    @classmethod
-    def coerce_attack_params(cls, values: Dict[str, Any]) -> Dict[str, Any]:
-        """
-        This validator ensures JSON serialization never fails due
-        to np.float32 / np.int64 / np.bool_ leaking into the params dict.
-        """
-        params = values.get("attack_params", {})
-        if isinstance(params, dict):
-            coerced = {}
-            for k, v in params.items():
-                if isinstance(v, np.floating):
-                    coerced[k] = float(v)
-                elif isinstance(v, np.integer):
-                    coerced[k] = int(v)
-                elif isinstance(v, np.bool_):
-                    coerced[k] = bool(v)
-                else:
-                    coerced[k] = v
-            values["attack_params"] = coerced
-        return values

src/pll_sim.py

@@ -1,127 +0,0 @@
-"""
-SRF-PLL Discrete-Time Simulation.
-
-Implements the Synchronous Reference Frame Phase-Locked Loop used in
-grid-connected inverters. Discrete time step Δt = 1 ms.
-
-Steps:
-  1. Generate true 3-phase grid voltages (50 Hz, 1.0 pu)
-  2. Apply attack injection on va
-  3. Clarke transform (αβ)
-  4. Park transform (dq) using estimated angle θ̂
-  5. PI controller to update ω̂ and θ̂
-  6. Compute phase error
-"""
-
-import numpy as np
-import math
-
-# Constants
-V_NOM = 1.0           # Nominal voltage (pu)
-F0 = 50.0             # Grid frequency (Hz)
-OMEGA0 = 2.0 * math.pi * F0  # Nominal angular freq (rad/s)
-DT = 1e-3             # Time step (1 ms)
-KP = 50.0             # PI proportional gain
-KI = 1500.0           # PI integral gain
-
-
-def wrap_angle(angle: float) -> float:
-    """Wrap angle to [-π, π]."""
-    return (angle + math.pi) % (2.0 * math.pi) - math.pi
-
-
-class SRFPLLSimulator:
-    """Discrete-time SRF-PLL simulator."""
-
-    def __init__(self, rng: np.random.Generator = None,
-                 kp: float = None, ki: float = None):
-        self.rng = rng or np.random.default_rng()
-        self.kp = kp if kp is not None else KP
-        self.ki = ki if ki is not None else KI
-        self.reset()
-
-    def reset(self):
-        """Reset PLL state to initial conditions."""
-        self.t = 0.0              # Simulation time (s)
-        self.theta_true = 0.0     # True grid angle (rad)
-        self.theta_hat = 0.0      # Estimated angle (rad)
-        self.omega_hat = OMEGA0   # Estimated angular freq (rad/s)
-        self.vq_integral = 0.0    # Integral of vq for PI controller
-
-        # Current signal values
-        self.vd = 0.0
-        self.vq = 0.0
-        self.va_m = 0.0
-        self.vb_m = 0.0
-        self.vc_m = 0.0
-        self.theta_err = 0.0
-
-    def step(self, attack_signal: float = 0.0):
-        """
-        Advance the PLL by one time step.
-        Args:
-            attack_signal: Attack injection added to va (pu).
-
-        Returns:
-            dict with vd, vq, omega_hat, theta_err, va_m, vb_m, vc_m, theta_true, theta_hat
-        """
-        # Step 1 — True three-phase grid voltages
-        va = V_NOM * math.sin(self.theta_true)
-        vb = V_NOM * math.sin(self.theta_true - 2.0 * math.pi / 3.0)
-        vc = V_NOM * math.sin(self.theta_true + 2.0 * math.pi / 3.0)
-
-        # Step 2 — Apply attack injection on va
-        va_m = va + attack_signal
-        vb_m = vb
-        vc_m = vc
-
-        # Measurement noise (σ = 0.01 pu, realistic for grid sensors)
-        NOISE_STD = 0.01
-        va_m += self.rng.normal(0, NOISE_STD)
-        vb_m += self.rng.normal(0, NOISE_STD)
-        vc_m += self.rng.normal(0, NOISE_STD)
-
-        # Step 3 — Clarke Transform (αβ)
-        v_alpha = va_m
-        v_beta = -(va_m + 2.0 * vb_m) / math.sqrt(3.0)
-
-        # Step 4 — Park Transform (dq) using estimated angle θ̂
-        cos_th = math.cos(self.theta_hat)
-        sin_th = math.sin(self.theta_hat)
-        vd = v_alpha * cos_th + v_beta * sin_th
-        vq = -v_alpha * sin_th + v_beta * cos_th
-
-        # Step 5 — PI Controller
-        self.vq_integral = np.clip(self.vq_integral + vq * DT, -0.3, 0.3)
-        omega_hat = OMEGA0 + self.kp * vq + self.ki * self.vq_integral
-        self.theta_hat = wrap_angle(self.theta_hat + omega_hat * DT)
-
-        # Advance true angle
-        self.theta_true = wrap_angle(self.theta_true + OMEGA0 * DT)
-
-        # Step 6 — Phase error wrapped to [-π, π]
-        theta_err = wrap_angle(self.theta_hat - self.theta_true)
-
-        # Update time
-        self.t += DT
-
-        # Storing current values
-        self.vd = vd
-        self.vq = vq
-        self.omega_hat = omega_hat
-        self.va_m = va_m
-        self.vb_m = vb_m
-        self.vc_m = vc_m
-        self.theta_err = theta_err
-
-        return {
-            "vd": vd,
-            "vq": vq,
-            "omega_hat": omega_hat,
-            "theta_err": theta_err,
-            "va_m": va_m,
-            "vb_m": vb_m,
-            "vc_m": vc_m,
-            "theta_true": self.theta_true,
-            "theta_hat": self.theta_hat,
-        }

submission.py

@@ -1,69 +0,0 @@
-# %% [Cell 1] Install dependencies
-import subprocess
-import sys
-
-subprocess.check_call([
-    sys.executable, "-m", "pip", "install", "-q",
-    "fastapi", "uvicorn", "pydantic", "numpy", "openai", "requests", "pyngrok", "nest_asyncio"
-])
-print("All dependencies installed successfully.")
-
-# %% [Cell 2] Write all source files to disk
-import os
-
-os.makedirs("src", exist_ok=True)
-
-# --- src/__init__.py ---
-with open('src/__init__.py', 'w', encoding='utf-8') as f:
-    f.write('# PLL Cyberattack Detection OpenEnv\n')
-
-# --- src/models.py ---
-with open('src/models.py', 'w', encoding='utf-8') as f:
-    f.write('"""\nPydantic models for the PLL Cyberattack Detection OpenEnv.\nDefines Observation, Action, Reward, and State schemas.\n"""\nimport numpy as np\nfrom typing import Annotated, Any, Dict, List, Optional\nfrom pydantic import BaseModel, Field, model_validator\n \n# Exactly 20 floats — enforced at validation time, not just documented.\nWindowList = Annotated[List[float], Field(min_length=20, max_length=20)]\n \n# Exactly 3 floats for [va, vb, vc].\nVoltageList = Annotated[List[float], Field(min_length=3, max_length=3)]\n \nclass Observation(BaseModel):\n    vq_window: WindowList\n    vd_window: WindowList\n    omega_window: WindowList\n    omega_deviation_window: WindowList\n    raw_voltages: VoltageList\n    task_id: int = Field(ge=0, le=2)\n    step: int = Field(ge=0)\n \nclass Action(BaseModel):\n    attack_detected: bool\n    attack_type: int = Field(ge=0, le=4)\n    confidence: float = Field(ge=0.0, le=1.0)\n    protective_action: int = Field(ge=0, le=3)\n \nclass Reward(BaseModel):\n    total: float\n    detection_reward: float\n    classification_bonus: float\n    early_detection_bonus: float\n    false_alarm_penalty: float\n    lock_loss_penalty: float\n \nclass State(BaseModel):\n    theta_true: float\n    theta_hat: float\n    omega_hat: float\n    vq_integral: float\n    attack_active: bool\n    attack_type: int         # Integer ID of the current attack: 0=none, 1=sinusoidal, 2=ramp, 3=pulse, 4=stealthy.\n    attack_params: Dict[str, Any]\n    attack_start_step: int\n    lock_lost: bool     # Whether the PLL has lost lock (|theta_err| > 5°). Task 2 only.\n    step: int = Field(ge=0)\n    episode_id: str\n    task_id: int = Field(ge=0, le=2)\n \n    @model_validator(mode="before")\n    @classmethod\n    def coerce_attack_params(cls, values: Dict[str, Any]) -> Dict[str, Any]:\n        """\n        Coerce numpy scalar types inside attack_params to native Python types.\n        sample_*_params() casts with float()/int() but a future contributor\n        may forget. This validator ensures JSON serialization never fails due\n        to np.float32 / np.int64 / np.bool_ leaking into the params dict.\n        """\n        params = values.get("attack_params", {})\n        if isinstance(params, dict):\n            coerced = {}\n            for k, v in params.items():\n                if isinstance(v, np.floating):\n                    coerced[k] = float(v)\n                elif isinstance(v, np.integer):\n                    coerced[k] = int(v)\n                elif isinstance(v, np.bool_):\n                    coerced[k] = bool(v)\n                else:\n                    coerced[k] = v\n            values["attack_params"] = coerced\n        return values\n')
-
-# --- src/pll_sim.py ---
-with open('src/pll_sim.py', 'w', encoding='utf-8') as f:
-    f.write('"""\nSRF-PLL Discrete-Time Simulation.\n\nImplements the Synchronous Reference Frame Phase-Locked Loop used in\ngrid-connected inverters. Discrete time step Δt = 1 ms.\n\nSteps:\n  1. Generate true 3-phase grid voltages (50 Hz, 1.0 pu)\n  2. Apply attack injection on va\n  3. Clarke transform (αβ)\n  4. Park transform (dq) using estimated angle θ̂\n  5. PI controller to update ω̂ and θ̂\n  6. Compute phase error\n"""\n\nimport numpy as np\nimport math\n\n\n# Constants\nV_NOM = 1.0           # Nominal voltage (pu)\nF0 = 50.0             # Grid frequency (Hz)\nOMEGA0 = 2.0 * math.pi * F0  # Nominal angular freq (rad/s)\nDT = 1e-3             # Time step (1 ms)\nKP = 50.0             # PI proportional gain\nKI = 1500.0           # PI integral gain\n\n\ndef wrap_angle(angle: float) -> float:\n    """Wrap angle to [-π, π]."""\n    return (angle + math.pi) % (2.0 * math.pi) - math.pi\n\n\nclass SRFPLLSimulator:\n    """Discrete-time SRF-PLL simulator."""\n\n    def __init__(self, rng: np.random.Generator = None,\n                 kp: float = None, ki: float = None):\n        self.rng = rng or np.random.default_rng()\n        self.kp = kp if kp is not None else KP\n        self.ki = ki if ki is not None else KI\n        self.reset()\n\n    def reset(self):\n        """Reset PLL state to initial conditions."""\n        self.t = 0.0              # Simulation time (s)\n        self.theta_true = 0.0     # True grid angle (rad)\n        self.theta_hat = 0.0      # Estimated angle (rad)\n        self.omega_hat = OMEGA0   # Estimated angular freq (rad/s)\n        self.vq_integral = 0.0    # Integral of vq for PI controller\n\n        # Current signal values\n        self.vd = 0.0\n        self.vq = 0.0\n        self.va_m = 0.0\n        self.vb_m = 0.0\n        self.vc_m = 0.0\n        self.theta_err = 0.0\n\n    def step(self, attack_signal: float = 0.0):\n        """\n        Advance the PLL by one time step.\n\n        Args:\n            attack_signal: Attack injection added to va (pu).\n\n        Returns:\n            dict with vd, vq, omega_hat, theta_err, va_m, vb_m, vc_m, theta_true, theta_hat\n        """\n        # Step 1 — True three-phase grid voltages\n        va = V_NOM * math.sin(self.theta_true)\n        vb = V_NOM * math.sin(self.theta_true - 2.0 * math.pi / 3.0)\n        vc = V_NOM * math.sin(self.theta_true + 2.0 * math.pi / 3.0)\n\n        # Step 2 — Apply attack injection on va\n        va_m = va + attack_signal\n        vb_m = vb\n        vc_m = vc\n\n        # Measurement noise (σ = 0.01 pu, realistic for grid sensors)\n        NOISE_STD = 0.01\n        va_m += self.rng.normal(0, NOISE_STD)\n        vb_m += self.rng.normal(0, NOISE_STD)\n        vc_m += self.rng.normal(0, NOISE_STD)\n\n        # Step 3 — Clarke Transform (αβ)\n        v_alpha = va_m\n        v_beta = -(va_m + 2.0 * vb_m) / math.sqrt(3.0)\n\n        # Step 4 — Park Transform (dq) using estimated angle θ̂\n        cos_th = math.cos(self.theta_hat)\n        sin_th = math.sin(self.theta_hat)\n        vd = v_alpha * cos_th + v_beta * sin_th\n        vq = -v_alpha * sin_th + v_beta * cos_th\n\n        # Step 5 — PI Controller\n        self.vq_integral = np.clip(self.vq_integral + vq * DT, -0.3, 0.3)\n        omega_hat = OMEGA0 + self.kp * vq + self.ki * self.vq_integral\n        self.theta_hat = wrap_angle(self.theta_hat + omega_hat * DT)\n\n        # Advance true angle\n        self.theta_true = wrap_angle(self.theta_true + OMEGA0 * DT)\n\n        # Step 6 — Phase error wrapped to [-π, π]\n        theta_err = wrap_angle(self.theta_hat - self.theta_true)\n\n        # Update time\n        self.t += DT\n\n        # Store current values\n        self.vd = vd\n        self.vq = vq\n        self.omega_hat = omega_hat\n        self.va_m = va_m\n        self.vb_m = vb_m\n        self.vc_m = vc_m\n        self.theta_err = theta_err\n\n        return {\n            "vd": vd,\n            "vq": vq,\n            "omega_hat": omega_hat,\n            "theta_err": theta_err,\n            "va_m": va_m,\n            "vb_m": vb_m,\n            "vc_m": vc_m,\n            "theta_true": self.theta_true,\n            "theta_hat": self.theta_hat,\n        }\n')
-
-# --- src/attacks.py ---
-with open('src/attacks.py', 'w', encoding='utf-8') as f:
-    f.write('"""\nAttack injection logic for the PLL Cyberattack Detection OpenEnv.\n\nImplements four attack types:\n  1. Sinusoidal FDI (Easy)\n  2. Ramp injection (Medium)\n  3. Pulse/step bias (Medium)\n  4. Stealthy low-and-slow phase drift (Hard)\n"""\n\nimport math\nimport numpy as np\nfrom typing import Dict, Any\n\n\ndef sample_sinusoidal_params(rng: np.random.Generator) -> Dict[str, Any]:\n    """Sample parameters for a sinusoidal FDI attack."""\n    return {\n        "type": "sinusoidal",\n        "amplitude": float(rng.uniform(0.05, 0.20)),\n        "freq": float(rng.uniform(5.0, 20.0)),\n        "phase": float(rng.uniform(0.0, 2.0 * math.pi)),\n    }\n\n\ndef sample_ramp_params(rng: np.random.Generator) -> Dict[str, Any]:\n    """Sample parameters for a ramp injection attack."""\n    return {\n        "type": "ramp",\n        "rate": float(rng.uniform(0.0002, 0.001)),\n    }\n\n\ndef sample_pulse_params(rng: np.random.Generator) -> Dict[str, Any]:\n    """Sample parameters for a pulse/step bias attack."""\n    return {\n        "type": "pulse",\n        "magnitude": float(rng.uniform(0.1, 0.3)),\n        "duration": int(rng.integers(20, 81)),  # 20 to 80 steps inclusive\n    }\n\n\ndef sample_stealthy_params(rng: np.random.Generator) -> Dict[str, Any]:\n    """Sample parameters for a stealthy low-and-slow attack."""\n    return {\n        "type": "stealthy",\n        "amplitude": 0.03,\n        "drift_rate": float(rng.uniform(0.05, 0.2)),\n    }\n\n\ndef sample_attack_start(rng: np.random.Generator) -> int:\n    """Sample a random attack start step between 30 and 80 inclusive."""\n    return int(rng.integers(30, 81))\n\n\nclass AttackGenerator:\n    """Generates attack signals given parameters and current simulation state."""\n\n    def __init__(self, attack_params: Dict[str, Any], attack_start_step: int):\n        self.params = attack_params\n        self.attack_start_step = attack_start_step\n        self.attack_type_str = attack_params.get("type", "none")\n\n        # For stealthy attack: track cumulative phase drift\n        self.delta = 0.0\n\n    def get_signal(self, current_step: int, sim_time: float) -> float:\n        """\n        Compute the attack signal value at the given step.\n\n        Args:\n            current_step: Current environment step (0-indexed).\n            sim_time: Current simulation time in seconds.\n\n        Returns:\n            Attack signal value (pu). Returns 0.0 if attack not yet started.\n        """\n        if current_step < self.attack_start_step:\n            return 0.0\n\n        steps_since_start = current_step - self.attack_start_step\n        dt = 1e-3  # time step\n\n        if self.attack_type_str == "sinusoidal":\n            A = self.params["amplitude"]\n            fa = self.params["freq"]\n            phi = self.params["phase"]\n            return A * math.sin(2.0 * math.pi * fa * sim_time + phi)\n\n        elif self.attack_type_str == "ramp":\n            rate = self.params["rate"]\n            return rate * steps_since_start\n\n        elif self.attack_type_str == "pulse":\n            mag = self.params["magnitude"]\n            dur = self.params["duration"]\n            if steps_since_start < dur:\n                return mag\n            else:\n                return 0.0\n\n        elif self.attack_type_str == "stealthy":\n            A_s = self.params["amplitude"]\n            drift_rate = self.params["drift_rate"]\n            # δ(t) = δ(t-1) + drift_rate * Δt — accumulated each call\n            self.delta += drift_rate * dt\n            f0 = 50.0\n            return A_s * math.sin(2.0 * math.pi * f0 * sim_time + self.delta)\n\n        return 0.0\n\n    def is_active(self, current_step: int) -> bool:\n        """Check if the attack is currently active at this step."""\n        if current_step < self.attack_start_step:\n            return False\n\n        # Pulse attacks end after duration\n        if self.attack_type_str == "pulse":\n            steps_since_start = current_step - self.attack_start_step\n            dur = self.params["duration"]\n            return steps_since_start < dur\n\n        return True\n\n\ndef get_attack_type_id(attack_type_str: str) -> int:\n    """Map attack type string to integer ID."""\n    mapping = {\n        "none": 0,\n        "sinusoidal": 1,\n        "ramp": 2,\n        "pulse": 3,\n        "stealthy": 4,\n    }\n    return mapping.get(attack_type_str, 0)\n')
-
-# --- src/graders.py ---
-with open('src/graders.py', 'w', encoding='utf-8') as f:
-    f.write('"""\nPer-task deterministic graders for the PLL Cyberattack Detection OpenEnv.\n\nEach grader takes an episode history and returns a score in [0.0, 1.0].\nGraders are deterministic given the same episode data.\n"""\n\nfrom typing import List, Dict, Any, Optional\n\n\ndef grade_task_easy(history: List[Dict[str, Any]], attack_start_step: int) -> float:\n    """\n    Task 1 — Sinusoidal FDI Detection (Easy).\n\n    Grader logic (relative to attack onset):\n      delay = first_correct_detection_step - attack_start_step\n      if delay <= 20:   score = 1.0\n      elif delay <= 100: score = linear decay from 1.0 to 0.5\n      elif delay <= 420: score = 0.2\n      else (never detected): score = 0.0\n    """\n    first_correct_detection_step = None\n\n    for entry in history:\n        step = entry["step"]\n        attack_active = entry["attack_active"]\n        attack_detected = entry["attack_detected"]\n\n        if attack_active and attack_detected:\n            first_correct_detection_step = step\n            break\n\n    if first_correct_detection_step is None:\n        return 0.0\n\n    delay = first_correct_detection_step - attack_start_step\n\n    if delay <= 20:\n        return 1.0\n    elif delay <= 100:\n        # Linear decay from 1.0 at delay=20 to 0.5 at delay=100\n        return 1.0 - 0.5 * (delay - 20) / 80.0\n    elif delay <= 420:\n        return 0.2\n    else:\n        return 0.0\n\n\ndef grade_task_medium(history: List[Dict[str, Any]], attack_start_step: int) -> float:\n    """\n    Task 2 — Multi-Attack Classification (Medium).\n\n    Grader logic:\n      base_score = fraction of steps (after attack_start) where attack_type is correctly classified\n      early_bonus = 0.4 * max(0, 1 - first_correct_classification_step / 100)\n      score = min(1.0, base_score * 0.6 + early_bonus)\n    """\n    steps_after_attack = 0\n    correct_classifications = 0\n    first_correct_classification_step = None\n\n    for entry in history:\n        step = entry["step"]\n        if step < attack_start_step:\n            continue\n\n        steps_after_attack += 1\n        true_type = entry["true_attack_type"]\n        agent_type = entry["agent_attack_type"]\n\n        if agent_type == true_type:\n            correct_classifications += 1\n            if first_correct_classification_step is None:\n                first_correct_classification_step = step\n\n    if steps_after_attack == 0:\n        return 0.0\n\n    base_score = correct_classifications / steps_after_attack\n\n    if first_correct_classification_step is not None:\n        early_bonus = 0.4 * max(0.0, 1.0 - first_correct_classification_step / 100.0)\n    else:\n        early_bonus = 0.0\n\n    score = min(1.0, base_score * 0.6 + early_bonus)\n    return max(0.0, score)\n\n\ndef grade_task_hard(\n    history: List[Dict[str, Any]],\n    loss_of_lock_step: Optional[int],\n    attack_start_step: int,\n) -> float:\n    """\n    Task 3 — Stealthy Low-and-Slow Attack (Hard).\n\n    Grader logic:\n      if detected before loss_of_lock_step:\n          score = 1.0 * (1 - first_detection_step / loss_of_lock_step)\n      elif detected after loss_of_lock but before episode end:\n          score = 0.3\n      else (never detected):\n          score = 0.0\n      false_alarm_penalty = 0.2 per false alarm before attack starts\n      (capped at reducing score to 0.0 minimum)\n    """\n    first_detection_step = None\n    false_alarm_count = 0\n\n    for entry in history:\n        step = entry["step"]\n        attack_active = entry["attack_active"]\n        attack_detected = entry["attack_detected"]\n\n        # Only count false alarms before the attack starts\n        if attack_detected and not attack_active and step < attack_start_step:\n            false_alarm_count += 1\n\n        if attack_detected and attack_active and first_detection_step is None:\n            first_detection_step = step\n\n    # Compute base score\n    if first_detection_step is None:\n        score = 0.0\n    elif loss_of_lock_step is not None and first_detection_step < loss_of_lock_step:\n        score = 1.0 * (1.0 - first_detection_step / loss_of_lock_step)\n    elif loss_of_lock_step is not None and first_detection_step >= loss_of_lock_step:\n        score = 0.3\n    else:\n        # No loss of lock occurred but attack was detected\n        score = 0.3\n\n    # Apply false alarm penalty\n    penalty = 0.2 * false_alarm_count\n    score = max(0.0, score - penalty)\n\n    return min(1.0, score)')
-
-# --- src/detector.py ---
-with open('src/detector.py', 'w', encoding='utf-8') as f:
-    f.write('"""\nAdaptive Physics-informed cyberattack detector for the PLL OpenEnv.\n\nUses residual-based and pattern-based features derived from the\nobservation windows to detect, classify, and recommend protective\nactions. The detector builds a baseline from the first 20 observations\n(warmup window) of the episode to normalize the features individually.\n"""\n\nimport numpy as np\nfrom typing import Dict, Any\n\nfrom src.models import Observation\n\n\nclass AdaptiveDetector:\n    def __init__(self):\n        # Baseline collections\n        self.r1_history = []\n        self.r3_history = []\n        self.r4_history = []\n        self.r5_history = []\n\n        # Calibrated statistics\n        self.mean_R1 = 0.0\n        self.std_R1 = 1e-6\n        self.mean_R3 = 0.0\n        self.std_R3 = 1e-6\n        self.mean_R4 = 0.0\n        self.std_R4 = 1e-6\n        self.mean_R5 = 0.0\n        self.std_R5 = 1e-6\n\n        self.is_calibrated = False\n\n    def detect(self, observation) -> Dict[str, Any]:\n        """\n        Run physics-informed anomaly detection on the current observation.\n        """\n        vq = np.array(observation.vq_window, dtype=np.float64)\n        vd = np.array(observation.vd_window, dtype=np.float64)\n        omega = np.array(observation.omega_window, dtype=np.float64)\n        omega_dev = np.array(observation.omega_deviation_window, dtype=np.float64)\n        va, vb, vc = observation.raw_voltages\n\n        # ---- Step 1: Feature Extraction --------------------------------\n        vq_mean = float(np.mean(np.abs(vq)))\n        vd_mean = float(np.mean(np.abs(vd)))\n        vq_ratio = vq_mean / (vd_mean + 1e-6)\n\n        omega_var = float(np.var(omega))\n        omega_dev_var = float(np.var(omega_dev))\n        vd_var = float(np.var(vd))\n        \n        abs_v_sum = abs(va) + abs(vb) + abs(vc) + 1e-6\n        symmetry_ratio = float(abs(va + vb + vc) / abs_v_sum)\n\n        vq_diff = np.diff(vq) if len(vq) > 1 else np.array([0.0])\n        vq_trend = float(np.mean(vq_diff))\n        vq_spike = float(np.max(np.abs(vq)))\n        vq_drift = float(np.sum(vq))\n\n        step = observation.step\n\n        # ---- Step 2: Baseline Calibration ------------------------------\n        if step < 20:\n            self.r1_history.append(vq_ratio)\n            self.r3_history.append(omega_var)\n            self.r4_history.append(vd_var)\n            self.r5_history.append(symmetry_ratio)\n            \n            return {\n                "attack_detected": False,\n                "attack_type": 0,\n                "confidence": 0.0,\n                "protective_action": 0,\n                "score": 0.0,\n                "baseline_score": 0.0\n            }\n        \n        if not self.is_calibrated:\n            self.mean_R1 = float(np.mean(self.r1_history))\n            self.std_R1 = max(float(np.std(self.r1_history)), 1e-6)\n            \n            self.mean_R3 = float(np.mean(self.r3_history))\n            self.std_R3 = max(float(np.std(self.r3_history)), 1e-6)\n            \n            self.mean_R4 = float(np.mean(self.r4_history))\n            self.std_R4 = max(float(np.std(self.r4_history)), 1e-6)\n            \n            self.mean_R5 = float(np.mean(self.r5_history))\n            self.std_R5 = max(float(np.std(self.r5_history)), 1e-6)\n            \n            self.is_calibrated = True\n\n        # ---- Step 3: Normalized Features ------------------------------\n        R1 = (vq_ratio - self.mean_R1) / self.std_R1\n        R3 = (omega_var - self.mean_R3) / self.std_R3\n        R4 = (vd_var - self.mean_R4) / self.std_R4\n        R5 = (symmetry_ratio - self.mean_R5) / self.std_R5\n\n        # ---- Step 4: Score --------------------------------------------\n        score = 0.4 * R1 + 0.2 * R3 + 0.2 * R5 + 0.2 * R4\n\n        # ---- Step 5: Detection ----------------------------------------\n        attack_detected = score > 5.0\n        confidence = min(1.0, score / 5.0) if attack_detected else 0.0\n\n        # ---- Step 6: Classification -----------------------------------\n        if not attack_detected:\n            attack_type = 0\n        else:\n            if R3 > 2:\n                attack_type = 1  # sinusoidal\n            elif abs(vq_trend) > 0.01:\n                attack_type = 2  # ramp\n            elif vq_spike > 0.1:\n                attack_type = 3  # pulse\n            else:\n                attack_type = 4  # stealthy\n\n        # ---- Step 7: Protective Action --------------------------------\n        if score > 6:\n            protective_action = 3\n        elif score > 3:\n            protective_action = 2\n        else:\n            protective_action = 1\n            if not attack_detected:\n                protective_action = 0\n\n        return {\n            "attack_detected": bool(attack_detected),\n            "attack_type": int(attack_type),\n            "confidence": float(confidence),\n            "protective_action": int(protective_action),\n            "score": float(score),\n            "baseline_score": 0.0\n        }\n')
-
-# --- src/env.py ---
-with open('src/env.py', 'w', encoding='utf-8') as f:
-    f.write('"""\nMain environment class for the PLL Cyberattack Detection OpenEnv.\n\nImplements step(), reset(), get_state(), and compute_reward().\nManages the PLL simulation, attack injection, observation windowing,\nepisode history, and grading.\n\nFixes applied vs previous version:\n  1. grade_task_easy() now receives attack_start_step (was missing, causing\n     TypeError at episode end for task_id=0).\n  2. attack_active is derived from attack_signal != 0.0 instead of\n     is_active() — single source of truth prevents signal/label divergence.\n  3. Lock-loss check guarded by step_count > attack_start_step — prevents\n     spurious lock-loss from PLL transient on step 0.\n  4. Task 3 early termination added: done=True when lock_lost, not just at\n     step 500. Avoids 200+ meaningless steps after failure.\n  5. _get_observation() updated to remove theta_err_window (ground-truth\n     leak) and add omega_deviation_window (raw omega deviation in rad/s),\n     matching the corrected Observation model.\n  6. theta_err_window deque removed from instance state.\n  7. Initial raw_voltages fixed: pll is warm-started with one silent step so\n     va_m/vb_m/vc_m are non-zero at reset() return.\n  8. omega_deviation_window deque added for the new Observation field.\n"""\n\nimport uuid\nimport numpy as np\nfrom typing import Tuple, Dict, Any, List, Optional\nfrom collections import deque\n\nfrom src.models import Observation, Action, Reward, State\nfrom src.pll_sim import SRFPLLSimulator, OMEGA0\nfrom src.attacks import (\n    AttackGenerator,\n    sample_sinusoidal_params,\n    sample_ramp_params,\n    sample_pulse_params,\n    sample_stealthy_params,\n    sample_attack_start,\n    get_attack_type_id,\n)\nfrom src.graders import grade_task_easy, grade_task_medium, grade_task_hard\nfrom src.detector import AdaptiveDetector\n\n\nWINDOW_SIZE = 20\nMAX_STEPS = 500\nLOCK_LOSS_THRESHOLD = 0.0873  # 5 degrees in radians\n\nDETECTION_THRESHOLD = 2.0\nEARLY_DETECTION_WINDOW = 100\nFALSE_ALARM_PENALTY = -0.2\nTRUE_POSITIVE_REWARD = 0.1\nTRUE_NEGATIVE_REWARD = 0.05\nMISSED_DETECTION_PENALTY = -0.05\nCLASSIFICATION_BONUS = 0.05\nLOCK_LOSS_PENALTY = -2.0\n\n\nclass PLLAttackEnv:\n    """OpenEnv-compliant PLL cyberattack detection environment."""\n\n    def __init__(self):\n        self.pll = SRFPLLSimulator()\n        self.rng: Optional[np.random.Generator] = None\n        self.task_id = 0\n        self.step_count = 0\n        self.episode_id = ""\n        self.done = False\n\n        # Attack state\n        self.attack_generator: Optional[AttackGenerator] = None\n        self.attack_active = False\n        self.attack_type = 0\n        self.attack_params: Dict[str, Any] = {}\n        self.attack_start_step = 0\n        self.true_attack_type = 0\n\n        # Detection tracking\n        self.first_detection_recorded = False\n        self.first_detection_step = 0\n\n        # Lock loss tracking (Task 2 / hard)\n        self.lock_lost = False\n        self.lock_loss_step: Optional[int] = None\n        self.lock_loss_penalized = False\n\n        # Observation windows\n        self.vq_window: deque = deque(maxlen=WINDOW_SIZE)\n        self.vd_window: deque = deque(maxlen=WINDOW_SIZE)\n        self.omega_window: deque = deque(maxlen=WINDOW_SIZE)\n        self.omega_deviation_window: deque = deque(maxlen=WINDOW_SIZE)  # Fix 8\n\n        # Detector\n        self.detector = AdaptiveDetector()\n\n        # Episode history for grading\n        self.history: List[Dict[str, Any]] = []\n\n    # ------------------------------------------------------------------\n    # Public API\n    # ------------------------------------------------------------------\n\n    def reset(self, task_id: int = 0, seed: Optional[int] = None) -> Observation:\n        """\n        Reset the environment for a new episode.\n\n        Args:\n            task_id: 0=easy (sinusoidal), 1=medium (multi-type),\n                     2=hard (stealthy).\n            seed:    Optional RNG seed for reproducibility.\n\n        Returns:\n            Initial Observation with non-zero raw_voltages.\n        """\n        self.rng = np.random.default_rng(seed)  # seed=None → random\n\n        self.task_id = task_id\n        self.step_count = 0\n        self.episode_id = str(uuid.uuid4())\n        self.done = False\n\n        # Reset PLL simulator\n        self.pll.reset()\n\n        # Reset detection tracking\n        self.first_detection_recorded = False\n        self.first_detection_step = 0\n\n        # Reset lock-loss tracking\n        self.lock_lost = False\n        self.lock_loss_step = None\n        self.lock_loss_penalized = False\n\n        # Reset history\n        self.history = []\n\n        # Reset observation windows (Fix 6: no theta_err_window)\n        self.vq_window = deque(maxlen=WINDOW_SIZE)\n        self.vd_window = deque(maxlen=WINDOW_SIZE)\n        self.omega_window = deque(maxlen=WINDOW_SIZE)\n        self.omega_deviation_window = deque(maxlen=WINDOW_SIZE)\n\n        # Reset detector\n        self.detector = AdaptiveDetector()\n\n        # Sample attack for this episode\n        self._setup_attack()\n\n        # Fix 7: warm-start PLL with WINDOW_SIZE silent steps so that\n        # windows contain realistic (non-zero) PLL-settled values and\n        # raw_voltages are non-zero on the first observation.\n        for _ in range(WINDOW_SIZE):\n            pll_out = self.pll.step(0.0)  # no attack during warm-up\n            omega_norm = (pll_out["omega_hat"] - OMEGA0) / OMEGA0\n            omega_dev  = pll_out["omega_hat"] - OMEGA0\n            self.vq_window.append(pll_out["vq"])\n            self.vd_window.append(pll_out["vd"])\n            self.omega_window.append(omega_norm)\n            self.omega_deviation_window.append(omega_dev)\n        # step_count stays at 0 — warm-up steps are invisible to the agent\n\n        return self._get_observation()\n\n    def step(self, action: Action) -> Tuple[Observation, Reward, bool, Dict[str, Any]]:\n        """\n        Advance the environment by one step.\n\n        Args:\n            action: Agent\'s Action for this step.\n\n        Returns:\n            (observation, reward, done, info)\n        """\n        if self.done:\n            return (\n                self._get_observation(),\n                Reward(\n                    total=0.0, detection_reward=0.0, classification_bonus=0.0,\n                    early_detection_bonus=0.0, false_alarm_penalty=0.0,\n                    lock_loss_penalty=0.0,\n                ),\n                True,\n                {"message": "Episode already done. Call /reset to start a new episode."},\n            )\n\n        # --- Attack signal ------------------------------------------------\n        # attack_active uses is_active() (step-based). It does NOT depend on the instantaneous\n        # signal value, because the attack signal can cross zero even while the attack is active.\n        attack_signal = self.attack_generator.get_signal(self.step_count, self.pll.t)\n        self.attack_active = self.attack_generator.is_active(self.step_count)\n\n        # --- Advance PLL --------------------------------------------------\n        pll_out = self.pll.step(attack_signal)\n\n        # --- Update observation windows -----------------------------------\n        omega_norm = (pll_out["omega_hat"] - OMEGA0) / OMEGA0\n        omega_dev  = pll_out["omega_hat"] - OMEGA0  # raw deviation (rad/s)\n        self.vq_window.append(pll_out["vq"])\n        self.vd_window.append(pll_out["vd"])\n        self.omega_window.append(omega_norm)\n        self.omega_deviation_window.append(omega_dev)\n\n        # --- Lock-loss check (Task 2 / hard only) -------------------------\n        PLL_CONVERGENCE_STEPS = 60  # PLL transient settles by ~step 50, use 60 for margin\n        if (\n            self.task_id == 2\n            and not self.lock_lost\n            and self.step_count > self.attack_start_step\n            and self.step_count > PLL_CONVERGENCE_STEPS   # ← guard against startup transient\n        ):\n            if abs(pll_out["theta_err"]) > LOCK_LOSS_THRESHOLD:\n                self.lock_lost = True\n                self.lock_loss_step = self.step_count\n\n        # --- Reward -------------------------------------------------------\n        reward = self.compute_reward(action)\n\n        # --- Record history entry for graders ----------------------------\n        self.history.append({\n            "step":              self.step_count,\n            "attack_active":     self.attack_active,\n            "attack_detected":   action.attack_detected,\n            "true_attack_type":  self.true_attack_type,\n            "agent_attack_type": action.attack_type,\n            "theta_err":         pll_out["theta_err"],\n        })\n\n        # --- Advance step counter ----------------------------------------\n        self.step_count += 1\n\n        # --- Episode termination -----------------------------------------\n        # Fix 4: Task 2 terminates early on lock-loss, not just at MAX_STEPS\n        if self.step_count >= MAX_STEPS:\n            self.done = True\n        elif self.task_id == 2 and self.lock_lost:\n            self.done = True  # early termination — no point continuing\n\n        # --- Physics-informed detector (evaluation/debug only) ------------\n        detector_output = self.detector.detect(self._get_observation())\n\n        # --- Build info --------------------------------------------------\n        info: Dict[str, Any] = {\n            "detector": detector_output,\n            "detector_features": {"step": self.step_count, "raw_score": detector_output.get("score")}\n        }\n        if self.done:\n            info["grader_score"] = self._compute_grader_score()\n            info["episode_id"]   = self.episode_id\n            info["total_steps"]  = self.step_count\n            info["lock_lost"]    = self.lock_lost\n\n        return self._get_observation(), reward, self.done, info\n\n    def compute_reward(self, action: Action) -> Reward:\n        """\n        Compute the dense reward signal for the current step.\n\n        Reward components:\n          detection_reward:     +0.10 true positive (per step)\n                                +0.05 true negative (per step)\n                                -0.05 missed detection (per step)\n          false_alarm_penalty:  -0.20 per false-positive step\n          classification_bonus: +0.05 per step correct type (task 1 only)\n          early_detection_bonus: one-time sparse, scaled by detection speed\n          lock_loss_penalty:    -2.00 one-time on lock loss (task 2 only)\n        """\n        detection_reward      = 0.0\n        false_alarm_penalty   = 0.0\n        classification_bonus  = 0.0\n        early_detection_bonus = 0.0\n        lock_loss_penalty     = 0.0\n\n        if self.attack_active:\n            if action.attack_detected:\n                detection_reward = TRUE_POSITIVE_REWARD\n                # One-time early detection bonus on first correct detection\n                if not self.first_detection_recorded:\n                    self.first_detection_step    = self.step_count\n                    self.first_detection_recorded = True\n                    # Relative steps since attack started\n                    t = self.first_detection_step - self.attack_start_step\n                    early_detection_bonus = max(0.0, 1.0 - t / EARLY_DETECTION_WINDOW)\n            else:\n                detection_reward = MISSED_DETECTION_PENALTY\n        else:\n            if action.attack_detected:\n                false_alarm_penalty = FALSE_ALARM_PENALTY\n            else:\n                detection_reward = TRUE_NEGATIVE_REWARD\n\n        # Task 1 (medium): per-step classification bonus\n        if self.task_id == 1 and self.attack_active:\n            if action.attack_type == self.true_attack_type:\n                classification_bonus = CLASSIFICATION_BONUS\n\n        # Task 2 (hard): one-time lock-loss penalty\n        if self.task_id == 2 and self.lock_lost and not self.lock_loss_penalized:\n            lock_loss_penalty        = LOCK_LOSS_PENALTY\n            self.lock_loss_penalized = True\n\n        total = (\n            detection_reward\n            + false_alarm_penalty\n            + classification_bonus\n            + early_detection_bonus\n            + lock_loss_penalty\n        )\n\n        return Reward(\n            total=total,\n            detection_reward=detection_reward,\n            classification_bonus=classification_bonus,\n            early_detection_bonus=early_detection_bonus,\n            false_alarm_penalty=false_alarm_penalty,\n            lock_loss_penalty=lock_loss_penalty,\n        )\n\n    def get_state(self) -> State:\n        """Return full internal state for debugging / GET /state endpoint."""\n        return State(\n            theta_true=self.pll.theta_true,\n            theta_hat=self.pll.theta_hat,\n            omega_hat=self.pll.omega_hat,\n            vq_integral=self.pll.vq_integral,\n            attack_active=self.attack_active,\n            attack_type=self.attack_type,\n            attack_params=self.attack_params,\n            attack_start_step=self.attack_start_step,\n            lock_lost=self.lock_lost,\n            step=self.step_count,\n            episode_id=self.episode_id,\n            task_id=self.task_id,\n        )\n\n    # ------------------------------------------------------------------\n    # Private helpers\n    # ------------------------------------------------------------------\n\n    def _setup_attack(self) -> None:\n        """Sample attack type and parameters based on current task_id."""\n        self.attack_start_step = sample_attack_start(self.rng)\n\n        if self.task_id == 0:\n            # Easy: sinusoidal FDI only\n            self.attack_params    = sample_sinusoidal_params(self.rng)\n            self.true_attack_type = 1\n\n        elif self.task_id == 1:\n            # Medium: random choice of sinusoidal / ramp / pulse\n            choice = int(self.rng.integers(0, 3))\n            if choice == 0:\n                self.attack_params    = sample_sinusoidal_params(self.rng)\n                self.true_attack_type = 1\n            elif choice == 1:\n                self.attack_params    = sample_ramp_params(self.rng)\n                self.true_attack_type = 2\n            else:\n                self.attack_params    = sample_pulse_params(self.rng)\n                self.true_attack_type = 3\n\n        elif self.task_id == 2:\n            # Hard: stealthy low-and-slow\n            self.attack_params    = sample_stealthy_params(self.rng)\n            self.true_attack_type = 4\n\n        self.attack_type      = get_attack_type_id(self.attack_params.get("type", "none"))\n        self.attack_generator = AttackGenerator(self.attack_params, self.attack_start_step)\n\n    def _get_observation(self) -> Observation:\n        """\n        Build the current Observation from internal windows.\n\n        Fix 5: theta_err_window replaced with omega_deviation_window.\n        theta_err requires knowing theta_true (not observable in a real\n        inverter) and leaked ground truth directly to the agent.\n        omega_deviation (omega_hat - OMEGA0 in rad/s) is a realistic proxy\n        that correlates with phase drift under stealthy attacks.\n        """\n        return Observation(\n            vq_window=list(self.vq_window),\n            vd_window=list(self.vd_window),\n            omega_window=list(self.omega_window),\n            omega_deviation_window=list(self.omega_deviation_window),  # Fix 5\n            raw_voltages=[self.pll.va_m, self.pll.vb_m, self.pll.vc_m],\n            task_id=self.task_id,\n            step=self.step_count,\n        )\n\n    def _compute_grader_score(self) -> float:\n        """Run the appropriate grader at episode end."""\n        if self.task_id == 0:\n            return grade_task_easy(self.history, self.attack_start_step)\n        elif self.task_id == 1:\n            return grade_task_medium(self.history, self.attack_start_step)\n        elif self.task_id == 2:\n            return grade_task_hard(\n                self.history,\n                self.lock_loss_step,\n                self.attack_start_step,\n            )\n        return 0.0\n')
-
-# --- src/api.py ---
-with open('src/api.py', 'w', encoding='utf-8') as f:
-    f.write('"""\nFastAPI application for the PLL Cyberattack Detection OpenEnv.\n\nExposes HTTP endpoints for environment interaction:\n  POST /reset   — Reset environment with task_id\n  POST /step    — Submit an action and advance one step\n  GET  /state   — Get current internal state\n  GET  /health  — Health check (returns 200)\n  GET  /tasks   — List available tasks\n"""\n\nimport asyncio\nfrom typing import Any, Dict, Optional\n\nfrom fastapi import FastAPI, HTTPException, Request\nfrom pydantic import BaseModel\n\nfrom src.models import Observation, Action, Reward, State\nfrom src.env import PLLAttackEnv\n\napp = FastAPI(\n    title="PLL Cyberattack Detection OpenEnv",\n    description="OpenEnv for AI-driven cyberattack detection on SRF-PLLs",\n    version="1.0.0",\n)\n\nenv = PLLAttackEnv()\nenv_lock = asyncio.Lock()\n\n\nclass ResetRequest(BaseModel):\n    task_id: int = 0\n    seed: Optional[int] = None\n\n\nclass StepResponse(BaseModel):\n    observation: Observation\n    reward: Reward\n    done: bool\n    info: Dict[str, Any]\n\n\n@app.post("/reset", response_model=Observation)\nasync def reset(req: Request):\n    """Reset the environment and return initial observation."""\n    async with env_lock:\n        try:\n            body = await req.body()\n            if body:\n                data = await req.json()\n                request = ResetRequest(**data)\n            else:\n                request = ResetRequest()\n        except Exception:\n            request = ResetRequest()\n        return env.reset(task_id=request.task_id, seed=request.seed)\n\n\n@app.post("/step", response_model=StepResponse)\nasync def step(action: Action):\n    async with env_lock:\n        if env.attack_generator is None:\n            raise HTTPException(status_code=400, detail="Call /reset before /step")\n        obs, reward, done, info = env.step(action)\n        return StepResponse(observation=obs, reward=reward, done=done, info=info)\n\n\n@app.get("/state", response_model=State)\nasync def get_state():\n    async with env_lock:\n        return env.get_state()\n\n\n@app.get("/health")\nasync def health():\n    """Health check endpoint."""\n    return {\n        "status": "ok",\n        "version": "1.0.0",\n        "environment": "pll-cyberattack-detection",\n        "tasks_available": 3,\n        "episode_active": env.attack_generator is not None,\n        "current_step": env.step_count,\n    }\n\n\n@app.get("/tasks")\nasync def list_tasks():\n    """List all available tasks."""\n    return {\n        "tasks": [\n            {\n                "id": 0,\n                "name": "sinusoidal_fdi",\n                "difficulty": "easy",\n                "description": "Detect sinusoidal FDI attack"\n            },\n            {\n                "id": 1,\n                "name": "multi_attack_classification",\n                "difficulty": "medium",\n                "description": "Classify attack type from observations"\n            },\n            {\n                "id": 2,\n                "name": "stealthy_attack_detection",\n                "difficulty": "hard",\n                "description": "Detect stealthy attack before PLL lock loss"\n            },\n        ]\n    }\n')
-
-# --- openenv.yaml ---
-with open('openenv.yaml', 'w', encoding='utf-8') as f:
-    f.write('name: pll-cyberattack-detection\nversion: 1.0.0\ndescription: >\n  Real-world OpenEnv for cyberattack detection on SRF-based \n  Phase-Locked Loops in grid-connected inverters. An agent monitors \n  sensor streams and detects False Data Injection attacks before \n  they cause loss of grid synchronization.\ntags: [power-systems, cybersecurity, control-systems, openenv, pll, fdi]\nreward_range: [-2.2, 1.15]\nobservation_space:\n  type: continuous\n  shape: [83]\n  fields:\n    - name: vq_window\n      shape: [20]\n      description: q-axis voltage error signal (pu)\n    - name: vd_window\n      shape: [20]\n      description: d-axis voltage (pu)\n    - name: omega_window\n      shape: [20]\n      description: normalized frequency deviation from nominal\n    - name: omega_deviation_window\n      shape: [20]\n      description: frequency deviation from nominal (rad/s)\n    - name: raw_voltages\n      shape: [3]\n      description: raw three-phase voltages [va, vb, vc] (pu)\naction_space:\n  type: mixed\n  fields:\n    - name: attack_detected\n      type: bool\n    - name: attack_type\n      type: int\n      range: [0, 4]\n    - name: confidence\n      type: float\n      range: [0.0, 1.0]\n    - name: protective_action\n      type: int\n      range: [0, 3]\ntasks:\n  - id: sinusoidal_fdi\n    difficulty: easy\n    numeric_id: 0\n    grader: time_to_detection\n    max_score: 1.0\n    episode_length: 500\n    description: Detect sinusoidal FDI attack within 100 steps of attack start\n  - id: multi_attack_classification\n    difficulty: medium\n    numeric_id: 1\n    grader: classification_accuracy\n    max_score: 1.0\n    episode_length: 500\n    description: Classify attack type from observation window\n  - id: stealthy_attack_detection\n    difficulty: hard\n    numeric_id: 2\n    grader: pre_lock_loss_detection\n    max_score: 1.0\n    episode_length: 500\n    description: Detect stealthy low-amplitude attack before PLL loss-of-lock\nepisode_length: 500\n')
-
-# --- Dockerfile ---
-with open('Dockerfile', 'w', encoding='utf-8') as f:
-    f.write('FROM python:3.10-slim\n\nWORKDIR /app\n\nCOPY requirements.txt .\nRUN pip install --no-cache-dir -r requirements.txt\n\nCOPY . .\n\nHEALTHCHECK --interval=30s --timeout=10s --start-period=5s \\\n    CMD curl -f http://localhost:7860/health || exit 1\n\nEXPOSE 7860\n\nCMD ["uvicorn", "src.api:app", "--host", "0.0.0.0", "--port", "7860"]\n')
-
-# --- requirements.txt ---
-with open('requirements.txt', 'w', encoding='utf-8') as f:
-    f.write('fastapi==0.115.0\nuvicorn==0.32.0\npydantic==2.9.0\nnumpy==1.26.4\nopenai>=1.0.0\nrequests>=2.31.0\nopenenv-core>=0.2.0\n')
-
-# --- inference.py ---
-with open('inference.py', 'w', encoding='utf-8') as f:
-    f.write('"""\nInference Script — PLL Cyberattack Detection OpenEnv\n=====================================================\nMANDATORY environment variables:\n  API_BASE_URL   The API endpoint for the LLM\n  MODEL_NAME     The model identifier to use\n  HF_TOKEN       Your Hugging Face / API key\n\nUses a HYBRID approach:\n  - A fast rule-based heuristic agent runs by default (no LLM needed)\n  - The heuristic analyzes vq/omega_deviation windows to detect attacks\n  - Set USE_LLM=1 env var to use the LLM instead (slower, may fail)\n\nMust be named inference.py and placed at the project root.\nUses OpenAI client for LLM calls when enabled.\n"""\n\nimport os\nimport json\nfrom typing import List, Optional\nimport time\nimport math\nimport requests\nfrom openai import OpenAI\n\nAPI_BASE_URL = os.getenv("API_BASE_URL", "https://router.huggingface.co/v1")\nMODEL_NAME = os.getenv("MODEL_NAME", "Qwen/Qwen2.5-72B-Instruct")\nHF_TOKEN = os.getenv("HF_TOKEN")\nENV_URL = os.getenv("ENV_URL", "https://krishuggingface-cyberattack-pll.hf.space")\nUSE_LLM = os.environ.get("USE_LLM", "0") == "1"\n\nclient = OpenAI(base_url=API_BASE_URL, api_key=HF_TOKEN)\n\nSYSTEM_PROMPT = """You are an AI agent monitoring a power grid inverter\'s Phase-Locked Loop (PLL).\nYou receive time-windowed sensor readings each step and must detect cyberattacks.\n\nvq_window: q-axis voltage error (should be ~0 when healthy)\nvd_window: d-axis voltage\nomega_window: estimated frequency (normalized, nominal=0)\nomega_deviation_window: frequency deviation from nominal in rad/s (useful for detecting slow phase drift)\nraw_voltages: [va, vb, vc] at current step\ntask_id: 0=detect only, 1=classify type, 2=detect stealthy attack\n\nFor task_id=0: Focus on detecting any attack (attack_detected=True/False).\nFor task_id=1: Also classify the attack type (1=sinusoidal, 2=ramp, 3=pulse).\nFor task_id=2: Detect very subtle attacks before the PLL loses lock. Look for slow drifts in omega_deviation and vq.\n\nAnalysis tips:\n- In healthy state, vq values should be near 0 and stable.\n- Sinusoidal attacks cause oscillating patterns in vq.\n- Ramp attacks cause steadily increasing vq magnitude.\n- Pulse attacks cause sudden step changes in vq.\n- Stealthy attacks cause very slow, gradual drift in omega_deviation_window.\n- Look at trends across the full window, not just the latest value.\n\nRespond ONLY with valid JSON, no explanation:\n{\n  "attack_detected": <bool>,\n  "attack_type": <int 0-4>,\n  "confidence": <float 0.0-1.0>,\n  "protective_action": <int 0-3>\n}"""\n\nTASK_NAMES = {\n    0: "Sinusoidal FDI Detection (Easy)",\n    1: "Multi-Attack Classification (Medium)",\n    2: "Stealthy Attack Detection (Hard)",\n}\n\nDEFAULT_ACTION = {\n    "attack_detected": False,\n    "attack_type": 0,\n    "confidence": 0.5,\n    "protective_action": 0,\n}\n\n\n# =====================================================================\n# Logging Helpers (OpenEnv compliance)\n# =====================================================================\n\ndef log_start(task: str, env: str, model: str) -> None:\n    print(f"[START] task={task} env={env} model={model}", flush=True)\n\n\ndef log_step(step: int, action: dict, reward: float, done: bool, error) -> None:\n    action_str = json.dumps(action, separators=(\',\', \':\'))\n    error_val = error if error else "null"\n    print(f"[STEP] step={step} action={action_str} reward={reward:.2f} done={str(done).lower()} error={error_val}", flush=True)\n\n\ndef log_end(success: bool, steps: int, score: float, rewards: list) -> None:\n    rewards_str = ",".join(f"{r:.2f}" for r in rewards)\n    print(f"[END] success={str(success).lower()} steps={steps} score={score:.3f} rewards={rewards_str}", flush=True)\n\n\n# =====================================================================\n# Detector-Based Agent\n# =====================================================================\n\ndef detector_agent(prev_info: dict) -> Optional[dict]:\n    """Reads the environment\'s adaptive detector output from the previous step."""\n    det = prev_info.get("detector", {})\n    if not det or "attack_detected" not in det:\n        return None\n        \n    return {\n        "attack_detected": det.get("attack_detected", False),\n        "attack_type": det.get("attack_type", 0),\n        "confidence": det.get("confidence", 0.5),\n        "protective_action": det.get("protective_action", 0),\n    }\n\n\n# =====================================================================\n# Rule-Based Heuristic Agent\n# =====================================================================\n\nclass HeuristicState:\n    """Tracks running state for the heuristic agent across steps."""\n    def __init__(self):\n        self.reset()\n\n    def reset(self):\n        self.vq_history = []           # all vq_mean(abs) values\n        self.omega_dev_history = []    # all omega_dev_mean(abs) values\n        self.attack_detected = False   # latched detection flag\n        self.predicted_type = 0        # latched classification\n        self.settled_baseline = None   # omega_dev baseline when PLL settles\n        self.peak_vq = 0.0            # highest vq_mean seen\n\n\n_hstate = HeuristicState()\n\n\ndef heuristic_agent(obs: dict) -> dict:\n    """\n    Rule-based attack detector using cumulative state tracking.\n    No LLM needed — runs instantly.\n\n    The key insight is that the PLL\'s closed-loop response transforms\n    attack signals, so we track statistics over time rather than\n    trying to classify from a single 20-step vq window shape.\n    """\n    global _hstate\n    vq = obs["vq_window"]\n    omega_dev = obs["omega_deviation_window"]\n    task_id = obs["task_id"]\n    step = obs["step"]\n\n    if step == 0:\n        _hstate.reset()\n\n    # --- Compute per-step features ---\n    vq_abs = [abs(v) for v in vq]\n    vq_mean = sum(vq_abs) / len(vq_abs)\n    vq_max = max(vq_abs)\n    vq_latest = abs(vq[-1])\n\n    omega_dev_abs = [abs(v) for v in omega_dev]\n    omega_dev_mean = sum(omega_dev_abs) / len(omega_dev_abs)\n\n    # Track history\n    _hstate.vq_history.append(vq_mean)\n    _hstate.omega_dev_history.append(omega_dev_mean)\n    _hstate.peak_vq = max(_hstate.peak_vq, vq_mean)\n\n    # Record baseline around step 45-50 (PLL settled)\n    if step == 50:\n        _hstate.settled_baseline = omega_dev_mean\n\n    # -----------------------------------------------------------------\n    # Detection: is vq significantly elevated?\n    # After PLL warm-start settles (~step 20-30), healthy vq < 0.005\n    # -----------------------------------------------------------------\n    if step < 25:\n        # PLL still settling, don\'t detect\n        detected = False\n    else:\n        detected = vq_mean > 0.01 or vq_max > 0.025\n\n    # Latch detection on\n    if detected:\n        _hstate.attack_detected = True\n\n    # -----------------------------------------------------------------\n    # Task 0: Binary detection only\n    # -----------------------------------------------------------------\n    if task_id == 0:\n        return {\n            "attack_detected": _hstate.attack_detected,\n            "attack_type": 1 if _hstate.attack_detected else 0,\n            "confidence": min(1.0, vq_mean * 50) if _hstate.attack_detected else 0.8,\n            "protective_action": 1 if _hstate.attack_detected else 0,\n        }\n\n    # -----------------------------------------------------------------\n    # Task 1: Classification using cumulative patterns\n    # -----------------------------------------------------------------\n    if task_id == 1:\n        if not _hstate.attack_detected:\n            return {\n                "attack_detected": False,\n                "attack_type": 0,\n                "confidence": 0.7,\n                "protective_action": 0,\n            }\n\n        # Classify using cumulative vq_history\n        # Only classify after enough attack data (10+ steps of elevated vq)\n        n_elevated = sum(1 for v in _hstate.vq_history if v > 0.01)\n\n        if n_elevated < 5:\n            # Not enough data yet, use simple guess\n            attack_type = 1\n        else:\n            # Get recent vq trend (last 10 elevated values)\n            elevated = [v for v in _hstate.vq_history if v > 0.005]\n            recent = elevated[-min(20, len(elevated)):]\n\n            # Feature 1: Is vq currently high or has it decayed?\n            current_vs_peak = vq_mean / _hstate.peak_vq if _hstate.peak_vq > 0 else 0\n\n            # Feature 2: How many zero crossings in current window\n            zero_crossings = sum(1 for i in range(1, len(vq)) if vq[i] * vq[i-1] < 0)\n\n            # Feature 3: Is vq growing or shrinking over recent history\n            if len(recent) >= 6:\n                first_third = sum(recent[:len(recent)//3]) / (len(recent)//3)\n                last_third = sum(recent[-len(recent)//3:]) / (len(recent)//3)\n                growth = last_third / first_third if first_third > 0.001 else 1.0\n            else:\n                growth = 1.0\n\n            # Classification logic:\n            # Sinusoidal: persistent oscillation, zero crossings, stable amplitude\n            # Ramp: growing vq over time (growth > 1)\n            # Pulse: high initial vq that decays to near zero (current_vs_peak < 0.3)\n\n            if current_vs_peak < 0.15 and _hstate.peak_vq > 0.05:\n                # vq has decayed significantly from peak → pulse (ended)\n                attack_type = 3\n            elif current_vs_peak < 0.4 and n_elevated > 30:\n                # vq decayed after a long time → pulse\n                attack_type = 3\n            elif zero_crossings >= 2 and growth < 1.5:\n                # Active oscillation without growing → sinusoidal\n                attack_type = 1\n            elif growth > 1.3:\n                # Growing signal → ramp\n                attack_type = 2\n            elif zero_crossings >= 1:\n                # Some oscillation → sinusoidal\n                attack_type = 1\n            else:\n                # Default: if mono-decrease, pulse; else sinusoidal\n                vq_diffs = [vq[i] - vq[i-1] for i in range(1, len(vq))]\n                neg = sum(1 for d in vq_diffs if d < 0)\n                if neg > 14:  # 14/19 = 73% decreasing\n                    attack_type = 3\n                else:\n                    attack_type = 1\n\n            _hstate.predicted_type = attack_type\n\n        return {\n            "attack_detected": True,\n            "attack_type": _hstate.predicted_type,\n            "confidence": 0.8,\n            "protective_action": 1,\n        }\n\n    # -----------------------------------------------------------------\n    # Task 2: Stealthy attack — detect omega_dev rising above baseline\n    # -----------------------------------------------------------------\n    if task_id == 2:\n        drift_detected = False\n        confidence = 0.3\n\n        if step > 50 and _hstate.settled_baseline is not None:\n            baseline = _hstate.settled_baseline\n\n            # Compare current to baseline\n            ratio = omega_dev_mean / baseline if baseline > 0.01 else omega_dev_mean * 100\n\n            # Check if omega_dev is rising relative to recent history\n            if len(_hstate.omega_dev_history) > 10:\n                recent_10 = _hstate.omega_dev_history[-10:]\n                old_10 = _hstate.omega_dev_history[-20:-10] if len(_hstate.omega_dev_history) > 20 else _hstate.omega_dev_history[:10]\n                recent_avg = sum(recent_10) / len(recent_10)\n                old_avg = sum(old_10) / len(old_10)\n                rising = recent_avg > old_avg * 1.1\n            else:\n                rising = False\n\n            if ratio > 2.0:\n                drift_detected = True\n                confidence = 0.9\n            elif ratio > 1.3 and rising:\n                drift_detected = True\n                confidence = 0.8\n            elif rising and vq_mean > 0.1:\n                drift_detected = True\n                confidence = 0.6\n            elif vq_mean > 0.2:\n                drift_detected = True\n                confidence = 0.5\n\n        if drift_detected:\n            _hstate.attack_detected = True\n\n        return {\n            "attack_detected": drift_detected,\n            "attack_type": 4 if drift_detected else 0,\n            "confidence": confidence,\n            "protective_action": 2 if drift_detected else 0,\n        }\n\n    return DEFAULT_ACTION.copy()\n\n\n# =====================================================================\n# LLM Agent (optional, set USE_LLM=1)\n# =====================================================================\n\ndef parse_llm_response(response_text: str) -> dict:\n    """Parse LLM response JSON, returning default action on failure."""\n    try:\n        text = response_text.strip()\n        if text.startswith("```"):\n            lines = text.split("\\n")\n            json_lines = []\n            in_block = False\n            for line in lines:\n                if line.strip().startswith("```") and not in_block:\n                    in_block = True\n                    continue\n                elif line.strip().startswith("```") and in_block:\n                    break\n                elif in_block:\n                    json_lines.append(line)\n            text = "\\n".join(json_lines)\n\n        parsed = json.loads(text)\n        action = {\n            "attack_detected": bool(parsed.get("attack_detected", False)),\n            "attack_type": max(0, min(4, int(parsed.get("attack_type", 0)))),\n            "confidence": max(0.0, min(1.0, float(parsed.get("confidence", 0.5)))),\n            "protective_action": max(0, min(3, int(parsed.get("protective_action", 0)))),\n        }\n        return action\n    except (json.JSONDecodeError, KeyError, TypeError, ValueError):\n        return DEFAULT_ACTION.copy()\n\n\ndef format_observation(obs: dict) -> str:\n    """Format observation dict into a concise string for the LLM."""\n    parts = [\n        f"Step: {obs[\'step\']}",\n        f"Task: {obs[\'task_id\']}",\n        f"vq_window (last 20): {[round(v, 6) for v in obs[\'vq_window\']]}",\n        f"vd_window (last 20): {[round(v, 6) for v in obs[\'vd_window\']]}",\n        f"omega_window (last 20): {[round(v, 6) for v in obs[\'omega_window\']]}",\n        f"omega_deviation_window (last 20): {[round(v, 6) for v in obs[\'omega_deviation_window\']]}",\n        f"raw_voltages: {[round(v, 6) for v in obs[\'raw_voltages\']]}",\n    ]\n    return "\\n".join(parts)\n\n\ndef llm_agent(obs: dict) -> dict:\n    """Call the LLM to decide an action. Falls back to heuristic on error."""\n    try:\n        obs_text = format_observation(obs)\n        completion = client.chat.completions.create(\n            model=MODEL_NAME,\n            messages=[\n                {"role": "system", "content": SYSTEM_PROMPT},\n                {"role": "user", "content": obs_text},\n            ],\n            temperature=0.1,\n            max_tokens=200,\n        )\n        llm_response = completion.choices[0].message.content\n        return parse_llm_response(llm_response)\n    except Exception as e:\n        print(f"    LLM error ({type(e).__name__}: {e}), falling back to heuristic")\n        return heuristic_agent(obs)\n\n\n# =====================================================================\n# Episode Runner\n# =====================================================================\n\ndef run_episode(task_id: int) -> float:\n    log_start(task=TASK_NAMES[task_id], env="pll-cyberattack-detection", model=MODEL_NAME if USE_LLM else "rule-based-heuristic")\n\n    print(f"\\n{\'=\'*60}")\n    print(f"Task {task_id}: {TASK_NAMES[task_id]}")\n    print(f"Agent: {\'LLM (\' + MODEL_NAME + \')\' if USE_LLM else \'Rule-Based Heuristic\'}")\n    print(f"{\'=\'*60}")\n\n    step_count = 0\n    grader_score = 0.0\n    rewards = []\n\n    try:\n        # Reset environment\n        reset_response = requests.post(\n            f"{ENV_URL}/reset",\n            json={"task_id": task_id},\n            timeout=30,\n        )\n        reset_response.raise_for_status()\n        obs = reset_response.json()\n\n        done = False\n        total_reward = 0.0\n        prev_info = {}\n\n        while not done:\n            # Choose agent\n            if USE_LLM:\n                action = llm_agent(obs)\n            else:\n                if step_count == 0:\n                    action = DEFAULT_ACTION.copy()\n                else:\n                    det_action = detector_agent(prev_info) if "detector" in prev_info else None\n                    heur_action = heuristic_agent(obs)\n                    \n                    if not det_action:\n                        action = heur_action\n                    elif det_action["confidence"] < 0.5:\n                        action = heur_action\n                    else:\n                        action = det_action\n\n            # Step environment\n            step_response = requests.post(\n                f"{ENV_URL}/step",\n                json=action,\n                timeout=30,\n            )\n            step_response.raise_for_status()\n            result = step_response.json()\n\n            obs = result["observation"]\n            reward = result["reward"]\n            done = result["done"]\n            info = result["info"]\n            total_reward += reward["total"]\n            rewards.append(reward["total"])\n            log_step(step=step_count, action=action, reward=reward["total"], done=done, error=None)\n\n            prev_info = info\n            step_count += 1\n\n            # Print progress every 50 steps\n            if step_count % 50 == 0:\n                print(f"  Step {step_count:3d} | Reward: {reward[\'total\']:+.4f} | "\n                      f"Cumulative: {total_reward:+.4f} | "\n                      f"Detected: {action[\'attack_detected\']} | "\n                      f"Type: {action[\'attack_type\']}")\n\n        # Extract grader score\n        grader_score = info.get("grader_score", 0.0)\n        print(f"\\n  Episode complete: {step_count} steps")\n        print(f"  Total reward: {total_reward:+.4f}")\n        print(f"  Grader score: {grader_score:.4f}")\n    finally:\n        log_end(success=grader_score > 0.0, steps=step_count, score=grader_score, rewards=rewards)\n\n    return grader_score\n\n\nif __name__ == "__main__":\n    agent_name = f"LLM ({MODEL_NAME})" if USE_LLM else "Rule-Based Heuristic"\n    print("PLL Cyberattack Detection — Agentic Inference")\n    print(f"Agent: {agent_name}")\n    print(f"Environment: {ENV_URL}")\n    if not USE_LLM:\n        print("(Set USE_LLM=1 to use LLM agent instead of heuristic)")\n\n    start_time = time.time()\n    scores = []\n\n    for task_id in range(3):\n        score = run_episode(task_id)\n        print(f"Task {task_id} score: {score:.4f}")\n        scores.append(score)\n\n    elapsed = time.time() - start_time\n\n    print(f"\\n{\'=\'*60}")\n    print("FINAL RESULTS")\n    print(f"{\'=\'*60}")\n    for i, score in enumerate(scores):\n        print(f"  Task {i} ({TASK_NAMES[i]}): {score:.4f}")\n    print(f"\\n  Average score: {sum(scores)/len(scores):.4f}")\n    print(f"  Total time: {elapsed:.1f}s ({elapsed/60:.1f} min)")\n    print(f"{\'=\'*60}")\n')
-
-# --- README.md ---
-with open('README.md', 'w', encoding='utf-8') as f:
-    f.write('# PLL Cyberattack Detection — OpenEnv\n\n[![HF Space](https://img.shields.io/badge/🤗%20HuggingFace-Space%20Live-blue)](https://huggingface.co/spaces/krishuggingface/CyberAttack-PLL)\n[![Docker](https://img.shields.io/badge/Docker-Ready-2496ED?logo=docker&logoColor=white)](Dockerfile)\n[![OpenEnv](https://img.shields.io/badge/OpenEnv-Compliant-green)](openenv.yaml)\n[![Python](https://img.shields.io/badge/Python-3.10%2B-3776AB?logo=python&logoColor=white)](https://python.org)\n\n> AI-driven cyberattack detection on SRF Phase-Locked Loops (PLLs) in grid-connected inverters.\n\n## Overview\n\nPhase-Locked Loops (PLLs) are critical components in grid-connected power converters that synchronize the inverter\'s output with the utility grid. The Synchronous Reference Frame PLL (SRF-PLL) estimates grid frequency and phase angle by tracking the q-axis voltage component — making it a high-value target for **False Data Injection (FDI)** cyberattacks.\n\nThis OpenEnv environment simulates an SRF-PLL under various FDI attack scenarios. An AI agent monitors time-windowed sensor observations (voltages, frequency deviations) and must detect, classify, and respond to attacks in real time before they cause loss of grid synchronization.\n\n## Architecture\n\n```\nGrid Voltage (50Hz)\n     │\n     ▼\n[FDI Attack Injection] ◄── Attacker injects false signal on va\n     │\n     ▼\nClarke Transform (αβ)\n     │\n     ▼\nPark Transform (dq) ◄── uses estimated angle θ̂\n     │\n     ▼\nPI Controller ──► ω̂, θ̂ updated\n     │\n     ▼\nAgent observes: vq_window, omega_deviation_window, raw_voltages\n     │\n     ▼\nAgent outputs: attack_detected, attack_type, confidence\n```\n\n## Inference & Detection Strategy\n\nThe environment natively features an **Adaptive Physics-Informed Detector** (`src/detector.py`) that calibrates anomaly residuals (R1, R3, R4, R5) during the PLL warm-up phase to identify stealthy voltage and frequency deviations.\n\nThe default inference client (`inference.py`) deploys a **Smart Blending Agent** strategy:\n1. It relies primarily on the environment\'s `AdaptiveDetector` output passed via `info["detector"]`.\n2. As a **safety net**, if the detector\'s classification confidence drops below 50% (`< 0.5`) on ambiguous anomalies, the client dynamically falls back to an independent, cumulative **Rule-Based Heuristic Agent**.\n3. Optionally, an LLM agent (e.g., `Qwen/Qwen2.5-72B-Instruct`) can be enabled natively via the `USE_LLM=1` environment variable.\n\n## Tasks\n\n| Task | ID | Difficulty | Attack Type | Objective | Score |\n|------|----|-----------|-------------|-----------|-------|\n| Sinusoidal FDI Detection | 0 | Easy | Sinusoidal injection | Detect within 100 steps | Time-based decay |\n| Multi-Attack Classification | 1 | Medium | Sinusoidal/Ramp/Pulse | Classify attack type | Accuracy + speed |\n| Stealthy Attack Detection | 2 | Hard | Low-amplitude phase drift | Detect before lock loss | Prevention score |\n\n## Observation Space\n\nEach step provides a JSON observation with the following fields:\n\n| Field | Shape | Description |\n|-------|-------|-------------|\n| `vq_window` | `[20]` | q-axis voltage error signal (pu) |\n| `vd_window` | `[20]` | d-axis voltage (pu) |\n| `omega_window` | `[20]` | Normalized frequency deviation from nominal |\n| `omega_deviation_window` | `[20]` | Frequency deviation from nominal (rad/s) |\n| `raw_voltages` | `[3]` | Raw three-phase voltages `[va, vb, vc]` (pu) |\n| `step` | scalar | Current simulation step |\n| `task_id` | scalar | Task identifier (0, 1, or 2) |\n\n**Total observation dimension**: 83 (20+20+20+20+3)\n\n## Action Space\n\nAgents return a JSON action each step:\n\n| Field | Type | Range | Description |\n|-------|------|-------|-------------|\n| `attack_detected` | `bool` | — | Whether an attack is detected |\n| `attack_type` | `int` | 0–4 | 0=none, 1=sinusoidal, 2=ramp, 3=pulse, 4=stealthy |\n| `confidence` | `float` | 0.0–1.0 | Agent\'s confidence in its classification |\n| `protective_action` | `int` | 0–3 | 0=none, 1=alert, 2=reduce power, 3=disconnect |\n\n## API Endpoints\n\n### Reset Environment\n```bash\ncurl -X POST http://localhost:7860/reset \\\n  -H "Content-Type: application/json" \\\n  -d \'{"task_id": 0, "seed": 42}\'\n```\n\n### Step\n```bash\ncurl -X POST http://localhost:7860/step \\\n  -H "Content-Type: application/json" \\\n  -d \'{"attack_detected": false, "attack_type": 0, "confidence": 0.5, "protective_action": 0}\'\n```\n\n### Get State\n```bash\ncurl http://localhost:7860/state\n```\n\n### Health Check\n```bash\ncurl http://localhost:7860/health\n```\n\n## Quick Start\n\n### With Docker\n\n```bash\ndocker build -t pll-cyberattack-env .\ndocker run -p 7860:7860 pll-cyberattack-env\n```\n\n### Without Docker\n\n```bash\npip install -r requirements.txt\nuvicorn src.api:app --host 0.0.0.0 --port 7860\n```\n\n## Environment Variables\n\n| Variable | Required | Default | Description |\n|----------|----------|---------|-------------|\n| `API_BASE_URL` | No | `https://router.huggingface.co/v1` | LLM API endpoint |\n| `MODEL_NAME` | No | `Qwen/Qwen2.5-72B-Instruct` | Model identifier |\n| `HF_TOKEN` | Yes | — | HuggingFace API token |\n\n## Baseline Performance\n\nThe default hybrid strategy (Adaptive Detector + Heuristic Fallback) achieves the following baseline scores evaluated locally over 500-step episodes:\n\n* **Task 0 (Sinusoidal FDI):** 1.0000 \n* **Task 1 (Multi-Attack Classification):** 0.8720\n* **Task 2 (Stealthy Drift):** 0.1639\n* **Average Score:** `0.6786`\n\n## Live Demo\n\n🚀 **HuggingFace Space**: [https://huggingface.co/spaces/krishuggingface/CyberAttack-PLL](https://huggingface.co/spaces/krishuggingface/CyberAttack-PLL)\n')
-
-# %% [Cell 3] Run Environment
-# (Example code for starting API server or inference omitted for clarity)

tests/test_environment.py

@@ -1,65 +1,134 @@
-import sys
-sys.path.insert(0, ".")
+"""Environment contract: reset/step/state, determinism, hidden ground truth, reward bounds."""
+
 import pytest
-import math
-from src.pll_sim import SRFPLLSimulator, OMEGA0
-from src.env import PLLAttackEnv
-from src.models import Action
-from src.attacks import AttackGenerator, sample_sinusoidal_params
-import numpy as np
-
-DUMMY_ACTION = Action(
-    attack_detected=False,
-    attack_type=0,
-    confidence=0.5,
-    protective_action=0
-)
-
-
-def test_episode_terminates_at_500():
-    """Episode must terminate with done=True at step 500."""
-    env = PLLAttackEnv()
-    env.reset(task_id=0, seed=42)
-    done = False
-    steps = 0
-    while not done:
-        _, _, done, _ = env.step(DUMMY_ACTION)
-        steps += 1
-    assert steps == 500, f"Episode ended at step {steps}, expected 500"
-
-def test_all_tasks_reset():
-    """All three tasks must reset without error."""
-    env = PLLAttackEnv()
-    for task_id in range(3):
-        obs = env.reset(task_id=task_id, seed=42)
-        assert obs.task_id == task_id
-        assert obs.step == 0
-        assert len(obs.vq_window) == 20
-
-def test_oracle_agent_nonzero_reward():
-    """An oracle agent should accumulate positive reward."""
-    env = PLLAttackEnv()
-    env.reset(task_id=0, seed=42)
-    total_reward = 0.0
-    done = False
-    while not done:
-        action = Action(
-            attack_detected=env.attack_active,
-            attack_type=env.true_attack_type if env.attack_active else 0,
-            confidence=1.0,
-            protective_action=0
-        )
-        _, reward, done, _ = env.step(action)
-        total_reward += reward.total
-    assert total_reward > 0, f"Oracle agent got non-positive reward: {total_reward}"
-
-def test_reward_bounds():
-    """Reward total must stay within [-2.5, 1.5] per step."""
-    env = PLLAttackEnv()
-    env.reset(task_id=2, seed=42)
-    done = False
-    while not done:
-        _, reward, done, _ = env.step(DUMMY_ACTION)
-        assert -2.5 <= reward.total <= 1.5, (
-            f"Reward out of bounds: {reward.total}"
-        )
+
+from pll_cyber.environment import PLLCyberEnvironment
+from pll_cyber.models import PLLAction, PLLObservation
+from pll_cyber.rewards import REWARD_MAX, REWARD_MIN
+from pll_cyber.scenarios import MAX_STEPS
+
+QUIET = PLLAction(attack_detected=False, confidence=0.9)
+
+
+@pytest.mark.parametrize("task_id", [0, 1, 2])
+def test_reset_returns_clean_initial_observation(task_id):
+    env = PLLCyberEnvironment()
+    obs = env.reset(task_id=task_id, seed=3)
+    assert isinstance(obs, PLLObservation)
+    assert obs.step == 0 and not obs.done and obs.reward is None
+    assert obs.task_id == task_id and obs.task_brief
+    assert len(obs.waveforms.va) == 25 and len(obs.trend.f_local_hz) >= 1
+    assert env.state.step_count == 0 and env.state.scenario is None
+
+
+def test_episode_length_and_termination():
+    env = PLLCyberEnvironment()
+    obs = env.reset(task_id=1, seed=0)
+    n = 0
+    while not obs.done:
+        obs = env.step(QUIET)
+        n += 1
+    assert n == MAX_STEPS
+    assert obs.episode_result is not None and 0.0 <= obs.episode_result.grader_score <= 1.0
+    after = env.step(QUIET)                     # stepping a finished episode is harmless
+    assert after.done and after.reward == 0.0
+
+
+def test_reset_fully_clears_previous_episode():
+    env = PLLCyberEnvironment()
+    env.reset(task_id=2, seed=1)
+    for _ in range(15):
+        env.step(PLLAction(attack_detected=True, protective_action=3))
+    obs = env.reset(task_id=2, seed=1)
+    assert obs.inverter_mode == "normal" and obs.step == 0
+    assert env.state.cumulative_reward == 0.0
+
+
+def _trajectory(task_id, seed, policy):
+    env = PLLCyberEnvironment()
+    obs = env.reset(task_id=task_id, seed=seed)
+    traj = [obs.model_dump_json()]
+    while not obs.done:
+        obs = env.step(policy(obs))
+        traj.append(obs.model_dump_json())
+    return traj
+
+
+@pytest.mark.parametrize("task_id", [0, 1, 2])
+def test_same_seed_is_bit_identical(task_id):
+    pol = lambda o: PLLAction(attack_detected=o.step > 12, attack_type=1, protective_action=2 if o.step > 20 else 0)
+    assert _trajectory(task_id, 42, pol) == _trajectory(task_id, 42, pol)
+
+
+def test_different_seeds_differ():
+    pol = lambda o: QUIET
+    assert _trajectory(0, 1, pol)[1] != _trajectory(0, 2, pol)[1]
+
+
+def test_unseeded_reset_reports_its_seed():
+    env = PLLCyberEnvironment()
+    env.reset(task_id=0)
+    assert isinstance(env.state.seed, int)
+
+
+def test_ground_truth_hidden_until_done():
+    env = PLLCyberEnvironment()
+    obs = env.reset(task_id=1, seed=5)
+    dumped = obs.model_dump_json()
+    for leak in ("onset", "attack_params", "theta_true", "true_attack"):
+        assert leak not in dumped
+    while not obs.done:
+        assert env.state.scenario is None and env.state.grader_score is None
+        obs = env.step(QUIET)
+    assert env.state.scenario is not None and env.state.grader_score is not None
+
+
+@pytest.mark.parametrize("task_id", [0, 1, 2])
+def test_rewards_within_declared_range(task_id):
+    env = PLLCyberEnvironment()
+    import random
+    rng = random.Random(task_id)
+    for seed in range(8):
+        obs = env.reset(task_id=task_id, seed=seed)
+        while not obs.done:
+            a = PLLAction(attack_detected=rng.random() < 0.5, attack_type=rng.randint(0, 4),
+                          confidence=rng.random(), protective_action=rng.choice([0, 1, 2, 3]))
+            obs = env.step(a)
+            assert REWARD_MIN - 1e-9 <= obs.reward <= REWARD_MAX + 1e-9
+            b = obs.reward_breakdown
+            parts = (b.detection + b.early_detection_bonus + b.calibration + b.classification
+                     + b.mitigation_cost + b.harm_penalty)
+            assert parts == pytest.approx(b.total)
+
+
+def test_task_selection_by_name_and_validation():
+    env = PLLCyberEnvironment()
+    assert env.reset(task="hard", seed=0).task_id == 2
+    assert env.reset(task="attack_classification_under_disturbances", seed=0).task_id == 1
+    with pytest.raises(ValueError):
+        env.reset(task_id=7)
+    with pytest.raises(Exception):
+        PLLAction(attack_detected=True, attack_type=9)
+
+
+def test_disconnect_is_latched_and_holdover_reversible():
+    env = PLLCyberEnvironment()
+    env.reset(task_id=2, seed=2)
+    assert env.step(PLLAction(attack_detected=True, protective_action=2)).inverter_mode == "holdover"
+    assert env.step(PLLAction(attack_detected=False)).inverter_mode == "normal"
+    assert env.step(PLLAction(attack_detected=True, protective_action=3)).inverter_mode == "disconnected"
+    assert env.step(PLLAction(attack_detected=False)).inverter_mode == "disconnected"
+
+
+def test_openenv_yaml_matches_code():
+    from pathlib import Path
+
+    import yaml
+
+    from pll_cyber.scenarios import TASKS
+
+    spec = yaml.safe_load((Path(__file__).parents[1] / "openenv.yaml").read_text(encoding="utf-8"))
+    assert spec["app"] == "server.app:app" and spec["port"] == 7860
+    assert spec["reward_range"] == [round(REWARD_MIN, 2), round(REWARD_MAX, 2)]
+    assert spec["episode"]["steps"] == MAX_STEPS
+    assert [(t["id"], t["name"]) for t in spec["tasks"]] == [(i, TASKS[i].name) for i in range(3)]

tests/test_graders.py

@@ -0,0 +1,87 @@
+"""Graders: range, determinism, non-constancy and resistance to degenerate strategies."""
+
+import itertools
+
+import numpy as np
+import pytest
+
+from pll_cyber.baselines import ConstantAgent, PhysicsHeuristicAgent, RandomAgent
+from pll_cyber.environment import PLLCyberEnvironment
+from pll_cyber.evaluation import evaluate, run_episode
+from pll_cyber.graders import grade_episode
+
+SEEDS = range(40)
+
+
+class Oracle:
+    """Cheats by reading ground truth — upper bound used only to prove tasks are solvable."""
+
+    def __init__(self, env):
+        self.env = env
+
+    def act(self, obs):
+        sc = self.env._sc
+        lab = sc.onset_step is not None and obs["step"] >= sc.onset_step
+        return {"attack_detected": lab, "attack_type": sc.attack.type_id if lab else 0, "confidence": 1.0,
+                "protective_action": 2 if (lab and sc.task_id == 2) else 0}
+
+
+def _scores(make_agent, task_id, seeds=SEEDS):
+    env = PLLCyberEnvironment()
+    return np.array([run_episode(make_agent(env), task_id, s, env)["score"] for s in seeds])
+
+
+@pytest.mark.parametrize("task_id", [0, 1, 2])
+def test_oracle_solves_every_task(task_id):
+    s = _scores(lambda env: Oracle(env), task_id)
+    assert s.mean() > 0.95 and s.min() >= 0.9
+
+
+@pytest.mark.parametrize("task_id", [0, 1, 2])
+def test_degenerate_policies_score_low(task_id):
+    always = _scores(lambda env: ConstantAgent(True, [1, 1, 4][task_id]), task_id)
+    never = _scores(lambda env: ConstantAgent(False), task_id)
+    seeds = itertools.count(100)
+    rand = _scores(lambda env: RandomAgent(seed=next(seeds)), task_id)
+    assert always.max() <= 0.05            # "always alarm" earns nothing anywhere
+    assert never.mean() < 0.30             # only the attack-free share
+    assert rand.mean() < 0.30
+
+
+def test_always_mitigate_is_punished_on_hard_task():
+    hold = _scores(lambda env: ConstantAgent(True, 4, protective_action=2), 2)
+    trip = _scores(lambda env: ConstantAgent(True, 4, protective_action=3), 2)
+    assert hold.mean() < 0.05 and trip.mean() < 0.05
+
+
+@pytest.mark.parametrize("task_id", [0, 1, 2])
+def test_scores_in_range_and_not_constant(task_id):
+    s = _scores(lambda env: PhysicsHeuristicAgent(), task_id)
+    assert ((s >= 0.0) & (s <= 1.0)).all()
+    seeds = itertools.count(500)
+    r = _scores(lambda env: RandomAgent(seed=next(seeds)), task_id)
+    assert np.std(np.concatenate([s, r])) > 0.05
+
+
+def test_heuristic_baseline_is_meaningful_but_not_saturated_on_hard():
+    res = evaluate(PhysicsHeuristicAgent, [0, 1, 2], SEEDS)
+    assert res[0]["mean"] > 0.9 and res[1]["mean"] > 0.8
+    assert 0.5 < res[2]["mean"] < 0.9      # headroom left for better agents
+
+
+def test_grader_is_deterministic_function_of_log():
+    log = [{"step": t, "detected": t >= 12, "attack_type": 2, "protective_action": 0} for t in range(40)]
+    a = grade_episode(1, log, 10, 50.0, true_type=2)
+    b = grade_episode(1, log, 10, 50.0, true_type=2)
+    assert a == b
+    assert a["detection_delay_ms"] == 100.0 and a["false_alarm_rate"] == 0.0
+
+
+def test_hard_grader_rewards_timely_mitigation():
+    def log_for(mitigate_at):
+        return [{"step": t, "detected": t >= 10, "attack_type": 4,
+                 "protective_action": 2 if t >= mitigate_at else 0} for t in range(40)]
+    early = grade_episode(2, log_for(11), 10, 50.0, unmitigated_desync_step=18, actual_desync_step=None)
+    late = grade_episode(2, log_for(25), 10, 50.0, unmitigated_desync_step=18, actual_desync_step=18)
+    assert early["protected"] and not late["protected"]
+    assert early["score"] > late["score"] + 0.4

tests/test_inference.py

@@ -0,0 +1,83 @@
+"""Baseline inference script: parsing, LLM loop with a fake client, log format, failure fallback."""
+
+import re
+import types
+
+import pytest
+
+import inference
+from inference import LLMAgent, LocalEnv, parse_action, run_episode
+
+
+def test_parse_action_variants():
+    assert parse_action('{"attack_detected": true, "attack_type": 2, "confidence": 0.7, "protective_action": 1}') == \
+        {"attack_detected": True, "attack_type": 2, "confidence": 0.7, "protective_action": 1}
+    fenced = 'Sure!\n```json\n{"attack_detected": "true", "attack_type": 9, "confidence": 3}\n```'
+    assert parse_action(fenced) == {"attack_detected": True, "attack_type": 4, "confidence": 1.0, "protective_action": 0}
+    assert parse_action("no json here") is None
+    assert parse_action('{"attack_type": "x"}') is None
+
+
+class _FakeClient:
+    """Mimics openai.OpenAI().chat.completions.create."""
+
+    def __init__(self, reply=None, exc=None):
+        self.calls = 0
+
+        def create(**kw):
+            self.calls += 1
+            assert kw["messages"][0]["role"] == "system"
+            if exc:
+                raise exc
+            msg = types.SimpleNamespace(content=reply)
+            return types.SimpleNamespace(choices=[types.SimpleNamespace(message=msg)])
+
+        self.chat = types.SimpleNamespace(completions=types.SimpleNamespace(create=create))
+
+
+@pytest.fixture(autouse=True)
+def _reset_breaker():
+    LLMAgent._disabled = False
+    LLMAgent._consecutive_failures = 0
+    yield
+    LLMAgent._disabled = False
+
+
+LOG = re.compile(r"^\[(START|STEP|END)\] ")
+
+
+def test_llm_episode_and_log_format(capsys):
+    fake = _FakeClient('{"attack_detected": false, "attack_type": 0, "confidence": 0.9, "protective_action": 0}')
+    res = run_episode(0, 1, lambda sid: LocalEnv(), lambda: LLMAgent(fake), "fake-model")
+    out = capsys.readouterr().out.strip().splitlines()
+    assert fake.calls == 40 and res["success"] and res["stats"]["llm"] == 40
+    assert out[0].startswith("[START] task=sinusoidal_fdi_detection env=pll-cyberattack-detection model=fake-model")
+    assert all(LOG.match(line) for line in out)
+    assert sum(line.startswith("[STEP]") for line in out) == 40
+    end = out[-1]
+    assert re.match(r"^\[END\] success=true steps=40 score=\d\.\d{3} rewards=(-?\d+\.\d\d,){39}-?\d+\.\d\d$", end)
+
+
+def test_auth_failure_falls_back_to_heuristic(capsys):
+    class E(Exception):
+        status_code = 402
+    fake = _FakeClient(exc=E("no credits"))
+    res = run_episode(2, 1, lambda sid: LocalEnv(), lambda: LLMAgent(fake), "fake-model")
+    assert res["success"] and fake.calls == 1          # disabled after the first permanent error
+    assert res["stats"]["fallback"] == 40 and res["score"] > 0.01
+    assert "llm_disabled_fallback" in capsys.readouterr().out
+
+
+def test_unparseable_output_does_not_crash(capsys):
+    fake = _FakeClient("I think there might be an attack")
+    res = run_episode(1, 3, lambda sid: LocalEnv(), lambda: LLMAgent(fake), "fake-model")
+    assert res["success"] and res["stats"]["fallback"] == 40 and not LLMAgent._disabled
+    assert "unparseable_llm_output" in capsys.readouterr().out
+
+
+def test_eval_seeds_cover_scenarios():
+    from pll_cyber.scenarios import sample_scenario
+    kinds = {t: {sample_scenario(t, s).attack.kind for s in inference.EVAL_SEEDS[t][:3]} for t in range(3)}
+    assert kinds[0] == {"sinusoidal", "none"}
+    assert kinds[1] == {"sinusoidal", "pulse", "ramp"}
+    assert kinds[2] == {"phase_spoofing", "none"}

tests/test_physics.py

@@ -0,0 +1,128 @@
+"""Physics regression tests: transforms, PLL dynamics, grid events and attacks."""
+
+import math
+
+import numpy as np
+import pytest
+
+from pll_cyber.attacks import Attack, AttackSpec
+from pll_cyber.environment import PLLCyberEnvironment
+from pll_cyber.evaluation import run_episode
+from pll_cyber.grid import GridConfig, GridEvent, GridModel
+from pll_cyber.models import PLLAction
+from pll_cyber.pll import PLLGains, SRFPLL, clarke, park, wrap_angle
+from pll_cyber.scenarios import DT
+
+
+def balanced(theta, v=1.0):
+    return [v * math.cos(theta - k * 2 * math.pi / 3) for k in range(3)]
+
+
+@pytest.mark.parametrize("theta", np.linspace(-3, 3, 13))
+def test_clarke_park_identities(theta):
+    a, b = clarke(*balanced(theta))
+    assert a == pytest.approx(math.cos(theta), abs=1e-12)
+    assert b == pytest.approx(math.sin(theta), abs=1e-12)
+    vd, vq = park(a, b, theta)                       # locked
+    assert vd == pytest.approx(1.0, abs=1e-12) and vq == pytest.approx(0.0, abs=1e-12)
+    vd, vq = park(a, b, theta - 0.1)                 # θ̂ lags by 0.1 rad → vq = sin(0.1)
+    assert vq == pytest.approx(math.sin(0.1), abs=1e-12)
+
+
+def test_gain_design():
+    g = PLLGains(omega_n=100.0, zeta=0.7071)
+    assert g.ki == pytest.approx(1e4)
+    assert g.kp == pytest.approx(141.42, rel=1e-3)
+
+
+def _simulate(events=(), n_windows=20, theta0_err=0.0):
+    grid = GridModel(GridConfig(h5_pu=0.0, h7_pu=0.0, f_drift_amp_hz=0.0, events=list(events)))
+    pll = SRFPLL(DT, theta0=theta0_err)
+    errs, freqs = [], []
+    for _ in range(n_windows):
+        out = grid.generate(250, DT)
+        r = pll.run(*out["v"])
+        errs += [math.degrees(wrap_angle(h - t)) for h, t in zip(r.theta_hat, out["theta"])]
+        freqs += r.f_hat
+    return np.array(errs), np.array(freqs)
+
+
+def test_pll_locks_from_initial_error():
+    errs, _ = _simulate(theta0_err=math.radians(30), n_windows=6)
+    assert abs(errs[0]) == pytest.approx(30 - 3.6, abs=0.5)   # θ advanced one sample (ω·Ts = 3.6°)
+    assert np.abs(errs[750:]).max() < 0.2           # locked within 150 ms
+
+
+def test_pll_tracks_frequency_step_without_steady_state_error():
+    # Type-2 loop: zero steady-state phase error for a frequency step
+    ev = GridEvent("frequency_event", 0.1, {"rocof_hz_s": 100.0, "delta_f_hz": 0.5})
+    errs, freqs = _simulate([ev], n_windows=20)
+    assert np.abs(errs[-1000:]).max() < 0.05
+    assert freqs[-1] == pytest.approx(50.5, abs=0.01)
+
+
+def test_phase_jump_recovers():
+    errs, _ = _simulate([GridEvent("phase_jump", 0.2, {"angle_deg": 25.0})], n_windows=12)
+    jump = int(0.2 / DT)
+    assert np.abs(errs[jump:jump + 20]).max() > 20     # the jump is seen…
+    assert np.abs(errs[jump + 500:]).max() < 0.2        # …and tracked out within 100 ms
+
+
+@pytest.mark.parametrize("task_id", [0, 1, 2])
+def test_natural_events_never_desynchronise(task_id):
+    env = PLLCyberEnvironment()
+    for seed in range(40):
+        obs = env.reset(task_id=task_id, seed=seed)
+        if env._sc.attack.present:
+            continue
+        while not obs.done:
+            obs = env.step(PLLAction(attack_detected=False))
+        assert obs.episode_result.scenario["unmitigated_desync_step"] is None
+
+
+def _vq_trace(task_id, seed, remove_attack=False):
+    env = PLLCyberEnvironment()
+    obs = env.reset(task_id=task_id, seed=seed)
+    sc = env._sc
+    if remove_attack:                        # same grid, same noise streams, no attacker
+        env._attack = Attack(AttackSpec())
+    vq = []
+    while not obs.done:
+        obs = env.step(PLLAction(attack_detected=False))
+        vq.append(obs.features.vq_rms_pu)
+    return sc, obs.episode_result.scenario, np.array(vq)
+
+
+def test_phase_spoofing_is_stealthy_but_destructive():
+    checked = 0
+    for seed in range(30):
+        sc, res, vq_att = _vq_trace(2, seed)
+        if not sc.attack.present:
+            continue
+        _, _, vq_clean = _vq_trace(2, seed, remove_attack=True)
+        # destructive: the unprotected inverter always loses synchronism, 0.3–0.75 s after onset
+        assert res["unmitigated_desync_step"] is not None
+        assert 6 <= res["unmitigated_desync_step"] - sc.onset_step <= 15
+        # stealthy: the attack leaves the PLL's own residual (vq) essentially unchanged
+        assert (np.abs(vq_att - vq_clean) <= 0.002 + 0.15 * vq_clean).all()
+        checked += 1
+    assert checked >= 15
+
+
+def test_pmu_disciplined_holdover_protects():
+    class Oracle:
+        def __init__(self, env):
+            self.env = env
+
+        def act(self, obs):
+            sc = self.env._sc
+            lab = sc.onset_step is not None and obs["step"] >= sc.onset_step
+            return {"attack_detected": lab, "attack_type": 4 if lab else 0, "confidence": 1.0,
+                    "protective_action": 2 if lab else 0}
+
+    env = PLLCyberEnvironment()
+    for seed in range(30):
+        r = run_episode(Oracle(env), 2, seed, env)
+        if r["scenario"]["attack"]["kind"] != "none":
+            assert r["scenario"]["actual_desync_step"] is None, seed
+            assert r["scenario"]["max_sync_error_deg"] < 10.0

tests/test_server.py

@@ -0,0 +1,89 @@
+"""HTTP/WebSocket API contract (OpenEnv runtime validation surface + session-aware simulation)."""
+
+import pytest
+from fastapi.testclient import TestClient
+
+from server.app import app
+
+
+@pytest.fixture(scope="module")
+def client():
+    return TestClient(app)
+
+
+def test_openenv_standard_endpoints(client):
+    assert client.get("/health").json() == {"status": "healthy"}
+    meta = client.get("/metadata").json()
+    assert meta["name"] and meta["description"]
+    schema = client.get("/schema").json()
+    assert {"action", "observation", "state"} <= set(schema)
+    assert "attack_detected" in schema["action"]["properties"]
+    assert client.post("/mcp", json={}).json()["jsonrpc"] == "2.0"
+    paths = client.get("/openapi.json").json()["paths"]
+    assert {"/reset", "/step", "/state"} <= set(paths)
+
+
+def test_reset_accepts_empty_body(client):
+    r = client.post("/reset")
+    assert r.status_code == 200 and r.json()["observation"]["step"] == 0
+
+
+def test_full_episode_over_http(client):
+    r = client.post("/reset", json={"task_id": 2, "seed": 4, "session_id": "ep"})
+    assert r.status_code == 200
+    done, steps = False, 0
+    while not done:
+        r = client.post("/step", json={"action": {"attack_detected": False}, "session_id": "ep"})
+        assert r.status_code == 200
+        done = r.json()["done"]
+        steps += 1
+    assert steps == 40
+    res = r.json()["observation"]["episode_result"]
+    assert 0.0 <= res["grader_score"] <= 1.0
+    assert client.get("/state", params={"session_id": "ep"}).json()["scenario"] is not None
+
+
+def test_sessions_are_isolated(client):
+    client.post("/reset", json={"task_id": 0, "seed": 1, "session_id": "a"})
+    client.post("/reset", json={"task_id": 1, "seed": 1, "session_id": "b"})
+    for _ in range(3):
+        client.post("/step", json={"action": {"attack_detected": False}, "session_id": "a"})
+    sa = client.get("/state", params={"session_id": "a"}).json()
+    sb = client.get("/state", params={"session_id": "b"}).json()
+    assert sa["step_count"] == 3 and sb["step_count"] == 0 and sb["task_id"] == 1
+
+
+def test_flat_action_and_validation_errors(client):
+    client.post("/reset", json={"task_id": 0, "seed": 1})
+    ok = client.post("/step", json={"attack_detected": True, "attack_type": 1, "confidence": 0.5,
+                                    "protective_action": 0})
+    assert ok.status_code == 200
+    assert client.post("/step", json={"action": {"attack_detected": True, "attack_type": 9}}).status_code == 422
+    assert client.post("/step", json={"action": {"attack_detected": True, "bogus": 1}}).status_code == 422
+    assert client.post("/step", json={"action": {"attack_detected": False}, "session_id": "zz"}).status_code == 404
+    assert client.post("/reset", json={"task_id": 5}).status_code == 422
+
+
+def test_state_hides_ground_truth_mid_episode(client):
+    client.post("/reset", json={"task_id": 2, "seed": 1, "session_id": "gt"})
+    client.post("/step", json={"action": {"attack_detected": False}, "session_id": "gt"})
+    st = client.get("/state", params={"session_id": "gt"}).json()
+    assert st["scenario"] is None and st["grader_score"] is None
+
+
+def test_websocket_session(client):
+    with client.websocket_connect("/ws") as ws:
+        ws.send_json({"type": "reset", "data": {"task_id": 1, "seed": 2}})
+        m = ws.receive_json()
+        assert m["type"] == "observation" and m["data"]["observation"]["task_id"] == 1
+        ws.send_json({"type": "step", "data": {"attack_detected": False}})
+        m = ws.receive_json()
+        assert m["data"]["observation"]["step"] == 1
+
+
+def test_tasks_and_demo(client):
+    t = client.get("/tasks").json()
+    assert len(t["tasks"]) == 3 and t["reward_range"][0] < 0 < t["reward_range"][1]
+    d = client.get("/demo", params={"task_id": 0, "seed": 1}).json()
+    assert len(d["trace"]) == 40 and 0 <= d["score"] <= 1
+    assert client.get("/").status_code == 200

uv.lock

@@ -1,5 +1,5 @@
 version = 1
-revision = 3
+revision = 5
 requires-python = ">=3.10"
 resolution-markers = [
     "python_full_version >= '3.14' and sys_platform == 'win32'",
@@ -888,6 +888,15 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/fa/5e/f8e9a1d23b9c20a551a8a02ea3637b4642e22c2626e3a13a9a29cdea99eb/importlib_metadata-8.7.1-py3-none-any.whl", hash = "sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151", size = 27865, upload-time = "2025-12-21T10:00:18.329Z" },
 ]
 
+[[package]]
+name = "iniconfig"
+version = "2.3.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/01/e1/2069291243c926a2ff1cd706c7f3eeb9b62144bf60f77c9fb9ff2fb26bd3/iniconfig-2.3.1.tar.gz", hash = "sha256:67f4b9c50da0dedf52af349e7749a80a9057a5031199791b906c3bb3ae878960", size = 21209, upload-time = "2026-10-06T22:48:38.076Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/56/43/4ca9e49d27a1fcf6bece6f6aec0ea46bb9112489b93d4b688fb415457bdb/iniconfig-2.3.1-py3-none-any.whl", hash = "sha256:9121e2c1fdb355232495be3194c8dfe87ccc2d5dee45947b78e68f499790d7a7", size = 7552, upload-time = "2026-10-06T22:48:36.959Z" },
+]
+
 [[package]]
 name = "jaraco-classes"
 version = "3.4.0"
@@ -1438,7 +1447,7 @@ wheels = [
 
 [[package]]
 name = "openenv-core"
-version = "0.2.3"
+version = "0.3.0"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
     { name = "fastapi" },
@@ -1457,9 +1466,9 @@ dependencies = [
     { name = "uvicorn" },
     { name = "websockets" },
 ]
-sdist = { url = "https://files.pythonhosted.org/packages/93/f3/41a5ed932a2507438c985e9d959dcaa1a6c46f293995c064348c0e52dd40/openenv_core-0.2.3.tar.gz", hash = "sha256:48aefd774474556297ce012b80f2ceb271db51253d7fd0838e6e2dcc329db0c3", size = 146944, upload-time = "2026-03-28T18:56:28.415Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/ce/d6/3bebe8afb55fcc3ea9251c4c2dfbab2879e31089bc91a8fe9696e5ce019b/openenv_core-0.3.0.tar.gz", hash = "sha256:c7fee2035badab5be497eb6f4afb2cb417de000f82cc19afd72fb5ec332c431d", size = 164720, upload-time = "2026-05-11T11:37:57.274Z" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/2f/22/38c339e370d198008f2c17ebdda1ae8f23bb4e1509dc7ae8eab6dc9b9cbe/openenv_core-0.2.3-py3-none-any.whl", hash = "sha256:f75a20c94452057a5f53a86e6d71a9f6a461524c3d6a865aa9344d257a92b795", size = 174557, upload-time = "2026-03-28T18:56:26.874Z" },
+    { url = "https://files.pythonhosted.org/packages/f8/f5/aafa43138589bfd5d369a8d02ea365aae9d6fe55ac0b3894368d6d69bd03/openenv_core-0.3.0-py3-none-any.whl", hash = "sha256:859e875c9d5211b157c30fb9abc681606fcf0bf1b6ffcdf404678992823a1df0", size = 194313, upload-time = "2026-05-11T11:37:55.537Z" },
 ]
 
 [[package]]
@@ -1573,10 +1582,10 @@ resolution-markers = [
     "python_full_version < '3.11'",
 ]
 dependencies = [
-    { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
-    { name = "python-dateutil", marker = "python_full_version < '3.11'" },
-    { name = "pytz", marker = "python_full_version < '3.11'" },
-    { name = "tzdata", marker = "python_full_version < '3.11'" },
+    { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" } },
+    { name = "python-dateutil" },
+    { name = "pytz" },
+    { name = "tzdata" },
 ]
 sdist = { url = "https://files.pythonhosted.org/packages/33/01/d40b85317f86cf08d853a4f495195c73815fdf205eef3993821720274518/pandas-2.3.3.tar.gz", hash = "sha256:e05e1af93b977f7eafa636d043f9f94c7ee3ac81af99c13508215942e64c993b", size = 4495223, upload-time = "2025-09-29T23:34:51.853Z" }
 wheels = [
@@ -1645,9 +1654,9 @@ resolution-markers = [
     "python_full_version >= '3.11' and python_full_version < '3.13' and sys_platform != 'emscripten' and sys_platform != 'win32'",
 ]
 dependencies = [
-    { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
-    { name = "python-dateutil", marker = "python_full_version >= '3.11'" },
-    { name = "tzdata", marker = "(python_full_version >= '3.11' and sys_platform == 'emscripten') or (python_full_version >= '3.11' and sys_platform == 'win32')" },
+    { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" } },
+    { name = "python-dateutil" },
+    { name = "tzdata", marker = "sys_platform == 'emscripten' or sys_platform == 'win32'" },
 ]
 sdist = { url = "https://files.pythonhosted.org/packages/da/99/b342345300f13440fe9fe385c3c481e2d9a595ee3bab4d3219247ac94e9a/pandas-3.0.2.tar.gz", hash = "sha256:f4753e73e34c8d83221ba58f232433fca2748be8b18dbca02d242ed153945043", size = 4645855, upload-time = "2026-03-31T06:48:30.816Z" }
 wheels = [
@@ -1818,24 +1827,48 @@ wheels = [
 
 [[package]]
 name = "pll-cyberattack-detection"
-version = "1.0.0"
+version = "2.0.0"
 source = { editable = "." }
 dependencies = [
     { name = "fastapi" },
     { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
     { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
+    { name = "openai" },
     { name = "openenv-core" },
     { name = "pydantic" },
+    { name = "python-dotenv" },
+    { name = "requests" },
     { name = "uvicorn" },
 ]
 
+[package.optional-dependencies]
+dev = [
+    { name = "httpx" },
+    { name = "pytest" },
+]
+
 [package.metadata]
 requires-dist = [
-    { name = "fastapi" },
-    { name = "numpy" },
-    { name = "openenv-core", specifier = ">=0.2.0" },
-    { name = "pydantic" },
-    { name = "uvicorn" },
+    { name = "fastapi", specifier = ">=0.110" },
+    { name = "httpx", marker = "extra == 'dev'", specifier = ">=0.27" },
+    { name = "numpy", specifier = ">=1.26" },
+    { name = "openai", specifier = ">=1.40" },
+    { name = "openenv-core", specifier = ">=0.3.0,<0.4" },
+    { name = "pydantic", specifier = ">=2.6" },
+    { name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0" },
+    { name = "python-dotenv", specifier = ">=1.0" },
+    { name = "requests", specifier = ">=2.31" },
+    { name = "uvicorn", specifier = ">=0.29" },
+]
+provides-extras = ["dev"]
+
+[[package]]
+name = "pluggy"
+version = "1.7.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/bf/db/7fc19e6f2dc92a966727031389fc2e08b558f0f25eb7403c1119ad4713cd/pluggy-1.7.0.tar.gz", hash = "sha256:d1eaa46ebb595891b860ab086b4d09c8588af65ebd4361b8e8f4bb8920b90ba8", size = 123304, upload-time = "2026-10-15T09:50:58.343Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/40/9e/2b38731e0fc536806f16490e1a12d7f0dc2a1235aa8cc07bcc75416a7daa/pluggy-1.7.0-py3-none-any.whl", hash = "sha256:7dd7b0d8832ba3cb632c306926ded123429211b83641b35dc5c41ad2d34f9bec", size = 27082, upload-time = "2026-10-15T09:50:56.808Z" },
 ]
 
 [[package]]
@@ -2068,6 +2101,24 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/df/80/fc9d01d5ed37ba4c42ca2b55b4339ae6e200b456be3a1aaddf4a9fa99b8c/pyperclip-1.11.0-py3-none-any.whl", hash = "sha256:299403e9ff44581cb9ba2ffeed69c7aa96a008622ad0c46cb575ca75b5b84273", size = 11063, upload-time = "2025-09-26T14:40:36.069Z" },
 ]
 
+[[package]]
+name = "pytest"
+version = "9.1.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "colorama", marker = "sys_platform == 'win32'" },
+    { name = "exceptiongroup", marker = "python_full_version < '3.11'" },
+    { name = "iniconfig" },
+    { name = "packaging" },
+    { name = "pluggy" },
+    { name = "pygments" },
+    { name = "tomli", marker = "python_full_version < '3.11'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },
+]
+
 [[package]]
 name = "python-dateutil"
 version = "2.9.0.post0"
@@ -2396,8 +2447,8 @@ name = "secretstorage"
 version = "3.5.0"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
-    { name = "cryptography", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" },
-    { name = "jeepney", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" },
+    { name = "cryptography" },
+    { name = "jeepney" },
 ]
 sdist = { url = "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz", hash = "sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", size = 19884, upload-time = "2025-11-23T19:02:53.191Z" }
 wheels = [

validate_gains.py

@@ -1,314 +0,0 @@
-"""
-validate_gains.py — KP/KI Gain Validation for PLL Attack Detection
-====================================================================
-Runs the full PLL simulation + heuristic agent across multiple KP/KI
-values and all attack types. Reports settling time, detection accuracy,
-and false-positive rate for each gain combination.
-
-Usage:
-    python validate_gains.py
-"""
-
-import sys, os, math, json
-import numpy as np
-from collections import deque
-
-# ---- Inline PLL (parameterized KP/KI) ----
-
-V_NOM = 1.0
-F0 = 50.0
-OMEGA0 = 2.0 * math.pi * F0
-DT = 1e-3
-WINDOW_SIZE = 20
-
-
-def wrap_angle(angle):
-    return (angle + math.pi) % (2.0 * math.pi) - math.pi
-
-
-class PLL:
-    def __init__(self, kp, ki):
-        self.kp = kp
-        self.ki = ki
-        self.reset()
-
-    def reset(self):
-        self.t = 0.0
-        self.theta_true = 0.0
-        self.theta_hat = 0.0
-        self.omega_hat = OMEGA0
-        self.vq_integral = 0.0
-        self.vd = 0.0
-        self.vq = 0.0
-
-    def step(self, attack_signal=0.0):
-        va = V_NOM * math.sin(self.theta_true)
-        vb = V_NOM * math.sin(self.theta_true - 2.0 * math.pi / 3.0)
-        vc = V_NOM * math.sin(self.theta_true + 2.0 * math.pi / 3.0)
-        va_m = va + attack_signal
-        vb_m = vb
-        v_alpha = va_m
-        v_beta = (va_m + 2.0 * vb_m) / math.sqrt(3.0)
-        cos_th = math.cos(self.theta_hat)
-        sin_th = math.sin(self.theta_hat)
-        vd = v_alpha * cos_th + v_beta * sin_th
-        vq = -v_alpha * sin_th + v_beta * cos_th
-        self.vq_integral += vq * DT
-        omega_hat = OMEGA0 + self.kp * vq + self.ki * self.vq_integral
-        self.theta_hat += omega_hat * DT
-        self.theta_true += OMEGA0 * DT
-        theta_err = wrap_angle(self.theta_hat - self.theta_true)
-        self.t += DT
-        self.vd = vd
-        self.vq = vq
-        self.omega_hat = omega_hat
-        return {"vq": vq, "vd": vd, "omega_hat": omega_hat, "theta_err": theta_err}
-
-
-# ---- Attack generators (from attacks.py) ----
-
-def make_sinusoidal(amp, freq, phase):
-    def gen(step, t):
-        return amp * math.sin(2 * math.pi * freq * t + phase)
-    return gen
-
-def make_ramp(rate):
-    def gen(step, t):
-        return rate * step
-    return gen
-
-def make_pulse(mag, dur):
-    def gen(step, t):
-        return mag if step < dur else 0.0
-    return gen
-
-def make_stealthy(amp, drift_rate):
-    state = {"delta": 0.0}
-    def gen(step, t):
-        state["delta"] += drift_rate * DT
-        return amp * math.sin(2 * math.pi * F0 * t + state["delta"])
-    return gen
-
-
-# ---- Heuristic agent (simplified from inference.py) ----
-
-class HeuristicDetector:
-    def __init__(self):
-        self.reset()
-
-    def reset(self):
-        self.vq_history = []
-        self.omega_dev_history = []
-        self.attack_detected = False
-        self.settled_baseline = None
-        self.peak_vq = 0.0
-
-    def decide(self, vq_window, omega_dev_window, step, task_id):
-        vq_abs = [abs(v) for v in vq_window]
-        vq_mean = sum(vq_abs) / len(vq_abs)
-        vq_max = max(vq_abs)
-        omega_dev_abs = [abs(v) for v in omega_dev_window]
-        omega_dev_mean = sum(omega_dev_abs) / len(omega_dev_abs)
-        self.vq_history.append(vq_mean)
-        self.omega_dev_history.append(omega_dev_mean)
-        self.peak_vq = max(self.peak_vq, vq_mean)
-        if step == 50:
-            self.settled_baseline = omega_dev_mean
-
-        # Task 0/1: vq-based detection
-        if task_id in (0, 1):
-            if step < 25:
-                detected = False
-            else:
-                detected = vq_mean > 0.01 or vq_max > 0.025
-            if detected:
-                self.attack_detected = True
-            return self.attack_detected
-
-        # Task 2: stealthy — omega deviation drift
-        if task_id == 2:
-            if step > 50 and self.settled_baseline is not None:
-                baseline = self.settled_baseline
-                ratio = omega_dev_mean / baseline if baseline > 0.01 else omega_dev_mean * 100
-                if len(self.omega_dev_history) > 10:
-                    recent_10 = self.omega_dev_history[-10:]
-                    old_10 = self.omega_dev_history[-20:-10] if len(self.omega_dev_history) > 20 else self.omega_dev_history[:10]
-                    recent_avg = sum(recent_10) / len(recent_10)
-                    old_avg = sum(old_10) / len(old_10)
-                    rising = recent_avg > old_avg * 1.1
-                else:
-                    rising = False
-                if ratio > 2.0 or (ratio > 1.3 and rising) or (rising and vq_mean > 0.1) or vq_mean > 0.2:
-                    self.attack_detected = True
-            return self.attack_detected
-
-        return False
-
-
-# ---- Validation runner ----
-
-def measure_settling(pll, threshold=0.001, max_steps=200):
-    """Run PLL with no attack and find when vq settles below threshold."""
-    pll.reset()
-    for s in range(max_steps):
-        out = pll.step(0.0)
-        if s > 5 and abs(out["vq"]) < threshold:
-            # Check it stays settled for 10 more steps
-            settled = True
-            for _ in range(10):
-                o2 = pll.step(0.0)
-                if abs(o2["vq"]) >= threshold:
-                    settled = False
-                    break
-            if settled:
-                return s
-    return max_steps
-
-
-def run_episode(kp, ki, attack_fn, attack_start, task_id, max_steps=500):
-    """
-    Run one full episode and return detection metrics.
-    Returns: (first_detect_step, n_correct_detect, n_missed, n_false_alarm, n_steps)
-    """
-    pll = PLL(kp, ki)
-    det = HeuristicDetector()
-    vq_win = deque(maxlen=WINDOW_SIZE)
-    vd_win = deque(maxlen=WINDOW_SIZE)
-    omega_dev_win = deque(maxlen=WINDOW_SIZE)
-
-    # Warm-start
-    for _ in range(WINDOW_SIZE):
-        out = pll.step(0.0)
-        vq_win.append(out["vq"])
-        vd_win.append(out["vd"])
-        omega_dev_win.append(out["omega_hat"] - OMEGA0)
-
-    first_detect = None
-    n_tp = 0  # true positive steps
-    n_miss = 0  # missed detection steps
-    n_fp = 0  # false positive steps
-    n_tn = 0  # true negative steps
-
-    for step in range(max_steps):
-        attack_active = (step >= attack_start)
-        if attack_active:
-            steps_since = step - attack_start
-            sig = attack_fn(steps_since, pll.t)
-        else:
-            sig = 0.0
-
-        out = pll.step(sig)
-        vq_win.append(out["vq"])
-        vd_win.append(out["vd"])
-        omega_dev_win.append(out["omega_hat"] - OMEGA0)
-
-        detected = det.decide(list(vq_win), list(omega_dev_win), step, task_id)
-
-        if attack_active:
-            if detected:
-                n_tp += 1
-                if first_detect is None:
-                    first_detect = step
-            else:
-                n_miss += 1
-        else:
-            if detected:
-                n_fp += 1
-            else:
-                n_tn += 1
-
-    return first_detect, n_tp, n_miss, n_fp, n_tn
-
-
-def validate_gains(kp, ki, n_trials=20):
-    """Run multiple trials for each attack type and report metrics."""
-    rng = np.random.default_rng(42)
-    results = {}
-
-    attack_configs = [
-        ("sinusoidal", 0, lambda r: make_sinusoidal(
-            r.uniform(0.05, 0.20), r.uniform(5.0, 20.0), r.uniform(0, 2*math.pi))),
-        ("ramp", 1, lambda r: make_ramp(r.uniform(0.0002, 0.001))),
-        ("pulse", 1, lambda r: make_pulse(r.uniform(0.1, 0.3), int(r.integers(20, 81)))),
-        ("stealthy", 2, lambda r: make_stealthy(0.03, r.uniform(0.05, 0.2))),
-    ]
-
-    for atk_name, task_id, atk_factory in attack_configs:
-        detect_rates = []
-        false_alarm_rates = []
-        detect_delays = []
-
-        for _ in range(n_trials):
-            attack_start = int(rng.integers(30, 81))
-            atk_fn = atk_factory(rng)
-            first_det, n_tp, n_miss, n_fp, n_tn = run_episode(
-                kp, ki, atk_fn, attack_start, task_id)
-
-            total_attack_steps = n_tp + n_miss
-            total_safe_steps = n_fp + n_tn
-            dr = n_tp / max(1, total_attack_steps)
-            far = n_fp / max(1, total_safe_steps)
-            detect_rates.append(dr)
-            false_alarm_rates.append(far)
-            if first_det is not None:
-                detect_delays.append(first_det - attack_start)
-
-        avg_dr = sum(detect_rates) / len(detect_rates)
-        avg_far = sum(false_alarm_rates) / len(false_alarm_rates)
-        avg_delay = sum(detect_delays) / len(detect_delays) if detect_delays else float('inf')
-        pct_detected = len(detect_delays) / n_trials * 100
-
-        results[atk_name] = {
-            "detect_rate": avg_dr,
-            "false_alarm_rate": avg_far,
-            "avg_delay_steps": avg_delay,
-            "pct_episodes_detected": pct_detected,
-        }
-
-    return results
-
-
-def main():
-    # Gain combinations to test
-    gain_sets = [
-        (50.0,  1500.0, "Current (zeta=0.645, t_s=160ms)"),
-        (266.0, 17689.0, "Ultra Fast (zeta=1.0, t_s=30ms)"),
-        (500.0, 62500.0, "Instant (zeta=1.0, t_s=16ms)"),
-    ]
-
-    print("=" * 90)
-    print("PLL KP/KI Gain Validation - Attack Detection Accuracy")
-    print("=" * 90)
-
-    for kp, ki, label in gain_sets:
-        pll = PLL(kp, ki)
-        settling = measure_settling(pll)
-        omega_n = math.sqrt(ki)
-        zeta = kp / (2 * omega_n)
-
-        print(f"\n{'-' * 90}")
-        print(f"  KP={kp:.0f}  KI={ki:.0f}  |  omega_n={omega_n:.1f} rad/s  zeta={zeta:.3f}  "
-              f"settling={settling} steps  |  {label}")
-        print(f"{'-' * 90}")
-
-        results = validate_gains(kp, ki, n_trials=30)
-
-        print(f"  {'Attack Type':<14} {'Detect Rate':>12} {'FA Rate':>10} {'Avg Delay':>12} {'Episodes Det':>14}")
-        for atk, m in results.items():
-            det_str = f"{m['detect_rate']*100:.1f}%"
-            fa_str = f"{m['false_alarm_rate']*100:.1f}%"
-            delay_str = f"{m['avg_delay_steps']:.0f} steps" if m['avg_delay_steps'] != float('inf') else "never"
-            ep_str = f"{m['pct_episodes_detected']:.0f}%"
-            print(f"  {atk:<14} {det_str:>12} {fa_str:>10} {delay_str:>12} {ep_str:>14}")
-
-    # Summary recommendation
-    print(f"\n{'=' * 90}")
-    print("RECOMMENDATION")
-    print("=" * 90)
-    print("Choose gains that maximize detection rate + episodes detected")
-    print("while keeping false alarm rate at 0% and settling time < 30 steps.")
-    print("=" * 90)
-
-
-if __name__ == "__main__":
-    main()