Upload README.md with huggingface_hub

README.md

@@ -14,81 +14,176 @@ tags:
 
 # GridOps — Community Microgrid Bridge Operator
 
-An OpenEnv reinforcement learning environment where an AI agent operates a **100-home community microgrid** in an Indian city during summer. The agent must balance solar generation, battery storage, diesel backup, and grid trade to keep the lights on while minimizing cost and emissions.
+> Keep the lights on for 100 homes. Don't go broke. Don't pollute.
 
-## Why This Matters
+An OpenEnv RL environment where an AI agent operates a **community microgrid in an Indian city during summer**. Every hour for 3 days, the agent decides how to use the battery, whether to run diesel, and whether to ask residents to cut usage — while the national grid automatically covers whatever is left over (up to its 200 kW limit).
 
-Community microgrid operation is a real job in India under the RDSS (Revamped Distribution Sector Scheme). IEX prosumer bidding is live. The tension between local energy independence and national grid economics creates genuine multi-day planning challenges that simple heuristics cannot solve.
+**Live dashboard**: [77ethers-gridops.hf.space/dashboard/](https://77ethers-gridops.hf.space/dashboard/)
 
-## Action Space (3 continuous dimensions)
+---
+
+## Why This Environment Exists
+
+Community microgrid operation is a **real job** in India under the [RDSS](https://rdss.gov.in/) (Revamped Distribution Sector Scheme). IEX prosumer bidding is live. Over 50 million Indian homes will have rooftop solar by 2030, and someone needs to manage the battery-grid-diesel tradeoff in real time.
+
+This environment captures the core tension: **sell surplus solar during expensive evening peaks for profit — but if a heatwave continues tomorrow and your battery is empty, the neighborhood goes dark.**
+
+Simple heuristics ("always discharge at peak", "always run diesel when low") provably fail. The agent must learn multi-hour planning, price forecasting, and constraint management.
+
+---
+
+## The Problem at a Glance
 
-| Action | Range | Description |
+**You have:**
+- **Solar panels** — 250 kW peak, free, but only during daylight
+- **Community battery** — 500 kWh storage, 100 kW max charge/discharge
+- **Diesel generator** — 100 kW, but Rs 25/kWh + Rs 100 startup cost
+- **National grid** — auto-imports/exports as slack (capped at 200 kW)
+
+**You control (3 continuous actions):**
+
+| Action | Range | What it does |
 |--------|-------|-------------|
-| `battery_dispatch` | -1.0 to +1.0 | Charge (-100 kW) or discharge (+100 kW) the community battery |
-| `diesel_dispatch` | 0.0 to 1.0 | Diesel generator output (0-100 kW). Rs 100 startup cost. |
-| `demand_shedding` | 0.0 to 1.0 | Request residents reduce usage (0-20%). 50% rebounds next hour. |
-
-**The grid is NOT an action** — it automatically absorbs the residual (capped at ±200 kW). If demand exceeds all supply sources, that's a blackout.
-
-## Observation Space
-
-| Field | Type | Description |
-|-------|------|-------------|
-| `hour` | float | Current hour in episode (0-72) |
-| `demand_kw` | float | Current aggregate demand (kW) |
-| `solar_kw` | float | Current solar generation (kW) |
-| `battery_soc` | float | Battery state-of-charge (0-1) |
-| `grid_price` | float | Current IEX price (Rs/kWh) |
-| `diesel_fuel_remaining` | float | Diesel fuel level (0-1) |
-| `diesel_is_on` | bool | Whether diesel was running last step |
-| `demand_forecast_4h` | list[4] | Noisy demand forecast (±15%) |
-| `solar_forecast_4h` | list[4] | Noisy solar forecast |
-| `price_forecast_4h` | list[4] | Noisy price forecast |
-| `cumulative_blackout_kwh` | float | Total unmet demand |
-| `cumulative_cost` | float | Net cost so far (Rs) |
-| `day_of_episode` | int | Current day (1-3) |
-
-## Episode Structure
-
-- **Duration**: 3 days (72 hours, 1-hour steps)
-- **Why 3 days**: Day 1 = learn the cycle. Day 2 = heatwave hits. Day 3 = multi-day planning tested.
-- **Determinism**: Seeded RNG → identical episodes per seed.
-
-## Tasks (Easy → Medium → Hard)
-
-| Task | Conditions | Challenge |
-|------|-----------|-----------|
-| **Task 1: Normal Summer** | Clear skies, ~100 kW avg demand, Rs 3-12 prices | Battery arbitrage + evening peak management |
-| **Task 2: Heatwave + Clouds** | Day 2-3 heatwave (+30% demand), intermittent clouds, price spikes Rs 18 | Multi-day battery planning under uncertainty |
-| **Task 3: Extreme Crisis** | Full 3-day heatwave, -30% solar, +50% demand, Rs 8-20, limited diesel | Survival mode — all resources constrained |
+| `battery_dispatch` | -1 to +1 | Charge (-100 kW) or discharge (+100 kW). Rs 2.5/kWh degradation. |
+| `diesel_dispatch` | 0 to 1 | Diesel output (0-100 kW). Rs 25/kWh + Rs 100 startup if was off. |
+| `demand_shedding` | 0 to 1 | Ask residents to cut 0-20% usage. **100% rebounds next hour.** Rs 40/kWh penalty. |
+
+**You do NOT control the grid.** It automatically absorbs whatever energy gap remains after your decisions. If the gap exceeds 200 kW, that's a **blackout** (Rs 150/kWh penalty).
+
+---
+
+## The Critical Bottleneck
+
+At **8 PM every evening**, demand hits **250 kW** but the grid maxes out at **200 kW** and solar is zero.
+
+The **50 kW gap** must come from your battery. If you discharged it for profit during the day, the neighborhood goes dark.
+
+On a heatwave day (Task 2-3), demand spikes to **325-375 kW**. Now the gap is **125-175 kW** — you need battery + diesel + shedding just to survive. And in Task 3, the grid goes down entirely for 6 hours.
+
+---
+
+## What the Agent Sees (Observation)
+
+| Field | Description |
+|-------|-------------|
+| `hour` | Current hour in episode (0-72, starting 6 AM) |
+| `demand_kw` | What the 100 homes need right now |
+| `solar_kw` | Free solar power available (0 at night, up to 250 kW midday) |
+| `battery_soc` | Battery charge level (0-1, i.e. 0-500 kWh) |
+| `grid_price` | Current IEX electricity price (Rs 3-20/kWh) |
+| `diesel_fuel_remaining` | Diesel tank level (0-1) |
+| `diesel_is_on` | Was diesel running last step? (startup cost if turning on) |
+| `demand_forecast_4h` | Noisy 4-hour demand forecast (+-15%) |
+| `solar_forecast_4h` | Noisy 4-hour solar forecast |
+| `price_forecast_4h` | Noisy 4-hour price forecast |
+| `cumulative_blackout_kwh` | Total blackout energy so far |
+| `cumulative_cost` | Total money spent so far (Rs) |
+| `flow_*` | Detailed energy flows (solar, grid import/export, battery in/out, diesel, demand) |
+
+**Partial observability**: forecasts have +-15% Gaussian noise. The agent cannot perfectly predict heatwave intensity, cloud cover, or price spikes.
+
+---
+
+## 3 Tasks (Each Tests a Different RL Capability)
+
+### Task 1: Normal Summer (Easy) — *Tests basic arbitrage*
+- Clear skies, standard demand (~100 kW avg, 250 kW peak)
+- Grid prices Rs 3-12 with clear cheap night / expensive evening pattern
+- **What the agent must learn**: charge battery at night (cheap grid), discharge during evening peak (expensive grid), let solar cover midday
+
+### Task 2: Heatwave + Price Spike (Medium) — *Tests temporal planning*
+- Day 2-3 heatwave (+30% demand), intermittent clouds
+- **Rs 20 price spike** on Day 2 evening — visible in 4-hour forecast
+- **What the agent must learn**: read the forecast, hold battery charge for the spike instead of greedily discharging early. A greedy policy discharges mid-afternoon; an RL agent that reads the forecast holds until 6 PM.
+
+### Task 3: Extreme Crisis + Grid Outage (Hard) — *Tests constraint management*
+- Full 3-day heatwave, -30% solar from haze, +50% demand
+- Limited diesel (33% tank = ~8 hours at full power)
+- **6-hour grid outage** on Day 2 afternoon — grid cap drops to 0 kW
+- **What the agent must learn**: aggressively pre-charge battery before the outage, ration diesel across the outage window, shed demand strategically to stretch resources. This is true microgrid islanding.
+
+---
 
 ## Grading (0.0 - 1.0)
 
 ```
-score = 0.50 × cost_efficiency + 0.25 × reliability + 0.25 × green_score
+score = 0.50 x cost_efficiency + 0.25 x reliability + 0.25 x green_score
 ```
 
-- **Cost efficiency**: How much cheaper than a dumb "max grid import, no intelligence" baseline
-- **Reliability**: Fraction of demand met (blackouts penalized via Rs 150/kWh VoLL)
-- **Green score**: 1 - (diesel_used / total_demand)
+| Component | Formula | What it rewards |
+|-----------|---------|----------------|
+| **Cost efficiency** (50%) | `1 - (agent_cost / baseline_cost)` | Spending less than a dumb "max grid import" baseline |
+| **Reliability** (25%) | `(demand_met - blackout) / demand_met` | Keeping the lights on |
+| **Green score** (25%) | `1 - (diesel_used / total_demand)` | Minimizing diesel emissions |
+
+**Baseline**: "import max grid every hour, no battery/diesel/shedding" — physically possible, but expensive and suffers blackouts during peak hours and grid outages.
+
+**VoLL (Value of Lost Load)**: Rs 150/kWh blackout penalty. This is a smooth gradient — no hard reliability cliff. The agent always gets signal for reducing blackouts incrementally.
+
+---
+
+## Why Heuristics Fail
+
+| Strategy | Why it fails |
+|----------|-------------|
+| "Always discharge battery" | Empty by evening peak. 50 kW gap = blackout. Score collapses. |
+| "Always run diesel" | Rs 25/kWh vs Rs 5 grid at night. Hemorrhages money. Green score = 0. |
+| "Shed demand whenever short" | Rs 40/kWh cost + 100% rebounds next hour. More expensive than diesel. |
+| "Discharge when price > X" | Ignores battery state. Drains SOC before the real peak. |
+| "Do nothing" | Grid alone can't cover evening peak. 3.6% blackout rate. |
+
+The oracle (rule-based, time-of-day + price-aware) scores 0.70-0.81. There's a clear **0.20-0.35 gap** between heuristics and the oracle, proving the environment has real optimization headroom.
+
+---
+
+## Anti-Gaming Design
+
+The environment has 5 mechanisms that prevent reward hacking:
+
+1. **Shedding is expensive** — Rs 40/kWh + 100% rebound. Costlier than diesel. True emergency only.
+2. **Battery degradation** — Rs 2.5/kWh throughput. Prevents infinite cycling for tiny arbitrage.
+3. **Diesel startup cost** — Rs 100 per on-switch. Prevents on/off toggling.
+4. **VoLL is smooth** — Rs 150/kWh with no cliff. Agent can't exploit a binary gate.
+5. **Grid is capped** — 200 kW max (0 during outages). Can't just buy everything.
+
+---
 
 ## Baseline Scores
 
 | Strategy | Task 1 | Task 2 | Task 3 |
 |----------|--------|--------|--------|
-| **Oracle** | 0.81 | 0.82 | 0.77 |
-| Do-Nothing | 0.58 | 0.51 | 0.46 |
-| Always-Discharge | 0.58 | 0.51 | 0.47 |
-| Always-Diesel | 0.42 | 0.42 | 0.45 |
+| **Oracle (rule-based)** | **0.79** | **0.81** | **0.70** |
+| Do-Nothing (grid only) | 0.58 | 0.51 | 0.45 |
+| Always-Discharge | 0.59 | 0.51 | 0.45 |
+| Always-Diesel | 0.42 | 0.42 | 0.44 |
+
+- **Deterministic**: identical scores across 3 runs (seeded RNG)
+- **Oracle ceiling is < 1.0**: proves the environment has real physics constraints, not inflated scores
+- **Clear separation**: oracle >> heuristics on every task
+
+---
 
 ## Key Physics
 
-- **Battery**: 500 kWh, 100 kW max, 90% round-trip efficiency, **Rs 2.5/kWh degradation cost**
-- **Diesel**: 100 kW, Rs 25/kWh, **Rs 100 startup cost** (penalizes on-off cycling)
-- **Demand shedding**: Up to 20%, but **50% rebounds next hour** (no free lunch)
-- **VoLL**: Rs 150/kWh blackout penalty (smooth gradient, no hard gate)
+| Component | Spec | Cost |
+|-----------|------|------|
+| **Solar** | 250 kW peak, bell curve 6 AM - 6 PM | Free |
+| **Battery** | 500 kWh, 100 kW max, 90% round-trip (sqrt each way) | Rs 2.5/kWh degradation |
+| **Diesel** | 100 kW max | Rs 25/kWh + Rs 100 startup |
+| **Grid** | 200 kW max import/export (slack variable) | Market price Rs 3-20/kWh |
+| **Blackout** | Unmet demand when all sources exhausted | Rs 150/kWh VoLL penalty |
+| **Shedding** | Up to 20% demand reduction | Rs 40/kWh + 100% rebound next hour |
 
-## Setup
+**Energy balance every step:**
+```
+supply  = solar + grid_import + battery_discharge + diesel
+consume = effective_demand + grid_export + battery_charge
+```
+Supply always equals consumption. Any unmet demand beyond grid cap = blackout.
+
+---
+
+## Setup & Usage
 
 ```bash
 # Install
@@ -97,13 +192,13 @@ pip install -e .
 # Run server
 uvicorn gridops.server.app:app --port 8000
 
-# Dashboard
+# Interactive dashboard
 open http://localhost:8000/dashboard/
 
-# Oracle test
+# Validate oracle + determinism
 python scripts/oracle_test.py
 
-# Inference
+# Run LLM baseline
 export API_BASE_URL="https://router.huggingface.co/v1"
 export HF_TOKEN="your-token"
 export MODEL_NAME="meta-llama/Llama-3.3-70B-Instruct"
@@ -117,26 +212,57 @@ docker build -t gridops .
 docker run -p 8000:8000 gridops
 ```
 
+## OpenEnv Validation
+
+```bash
+# Local structure check
+openenv validate
+
+# Runtime check (against live server)
+openenv validate --url http://localhost:8000
+```
+
+---
+
 ## Project Structure
 
 ```
 gridops/
-├── inference.py                # LLM baseline (uses OpenAI client)
-├── openenv.yaml                # OpenEnv manifest
-├── Dockerfile
-├── server/app.py               # Root entry point (OpenEnv validate)
+├── inference.py                 # LLM baseline (API_BASE_URL, MODEL_NAME, HF_TOKEN)
+├── openenv.yaml                 # OpenEnv manifest
+├── Dockerfile                   # Docker deployment
+├── server/app.py                # Root entry point (openenv validate)
 ├── gridops/
-│   ├── models.py               # GridOpsAction, GridOpsObservation (Pydantic)
+│   ├── models.py                # GridOpsAction, GridOpsObservation (Pydantic)
 │   ├── simulation/
-│   │   ├── physics.py          # Energy balance, battery, VoLL, degradation
-│   │   └── scenarios.py        # Demand/solar/price curve generators
+│   │   ├── physics.py           # Energy balance, battery, VoLL, degradation, outages
+│   │   └── scenarios.py         # Demand/solar/price curve generators
 │   ├── tasks/
-│   │   ├── definitions.py      # 3 task configs
-│   │   └── graders.py          # 0-1 grading with cost/reliability/green
+│   │   ├── definitions.py       # 3 task configs (normal, heatwave, crisis+outage)
+│   │   └── graders.py           # 0-1 scoring: cost + reliability + green
 │   └── server/
-│       ├── app.py              # FastAPI + OpenEnv create_app
-│       ├── environment.py      # OpenEnv Environment class
-│       └── static/index.html   # Interactive dashboard
+│       ├── app.py               # FastAPI + OpenEnv create_app
+│       ├── environment.py       # OpenEnv Environment class
+│       └── static/index.html    # Interactive dashboard with energy flows
 └── scripts/
-    └── oracle_test.py          # Oracle validation + determinism check
+    └── oracle_test.py           # Oracle + heuristic validation + determinism check
 ```
+
+---
+
+## API Endpoints
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/health` | GET | Health check |
+| `/schema` | GET | Action/observation/state JSON schemas |
+| `/metadata` | GET | Environment name and description |
+| `/reset` | POST | Reset environment (OpenEnv standard) |
+| `/step` | POST | Execute action (OpenEnv standard) |
+| `/state` | GET | Current state (OpenEnv standard) |
+| `/ws` | WebSocket | Persistent session (OpenEnv standard) |
+| `/api/reset` | POST | Stateful reset (dashboard) |
+| `/api/step` | POST | Stateful step (dashboard) |
+| `/api/state` | GET | Stateful state (dashboard) |
+| `/tasks` | GET | List available tasks |
+| `/dashboard/` | GET | Interactive web UI |