docs/SPEC.md

# EcoGrid-OpenEnv — Technical Specification

## 1. Overview

EcoGrid-OpenEnv is a production-grade Reinforcement Learning environment for the
OpenEnv framework that simulates sustainable energy grid management. An agent
controls energy distribution across renewable sources (solar + wind), fossil fuels,
and battery storage to meet variable demand while minimising cost and carbon emissions.

**Author:** Team DD  
**Hackathon:** Scaler School of Technology × Meta PyTorch Finale  
**Theme:** #3 — World Modeling (Professional Tasks) + Mercor Sub-theme

---

## 2. File Tree

```
eco-grid-openenv/
├── env/
│   ├── __init__.py           # Package init, re-exports EcoGridEnv
│   ├── environment.py        # Core EcoGridEnv class (reset/step/state)
│   ├── dynamics.py           # Deterministic physics helpers (solar/wind/demand/battery)
│   ├── reward.py             # Multi-objective reward function
│   └── tasks.py              # 3 task graders (easy/medium/hard)
├── models/
│   ├── __init__.py           # Package init, re-exports all schemas
│   └── schemas.py            # Pydantic v2 typed models
├── tests/
│   ├── test_environment.py   # Environment unit tests
│   ├── test_reward.py        # Reward function tests
│   └── test_graders.py       # Grader determinism tests
├── docs/
│   └── SPEC.md               # This file
├── logs/                     # Runtime logs (gitignored)
├── lora_adapter/             # Trained LoRA weights (gitignored)
├── openenv.yaml              # OpenEnv specification file
├── baseline.py               # Heuristic + LLM inference script
├── train_unsloth.py          # Unsloth GRPO training script
├── app.py                    # Streamlit dashboard for HF Spaces
├── Dockerfile                # Multi-stage production Dockerfile
├── docker-compose.yml        # Local dev compose config
├── requirements.txt          # Pinned Python dependencies
├── README.md                 # Full hackathon README
├── BLOG.md                   # HuggingFace blog post (≤400 words)
└── .gitignore                # Ignore logs/, lora_adapter/, __pycache__
```

---

## 3. State Space

| Field | Type | Range | Unit | Description |
|-------|------|-------|------|-------------|
| demand | float | [0, 200] | MWh | Energy demand this timestep |
| solar_capacity | float | [0, 1] | ratio | Available solar generation capacity |
| wind_capacity | float | [0, 1] | ratio | Available wind generation capacity |
| battery_level | float | [0, 1] | ratio | Battery state of charge |
| grid_stability | float | [0, 1] | ratio | Grid frequency stability indicator |
| carbon_budget_remaining | float | [0, 1000] | kgCO₂ | Remaining carbon emission budget |
| price_signal | float | [0, 300] | $/MWh | Current spot electricity price |
| time_step | int | [0, 96] | step | Current simulation timestep |

---

## 4. Action Space

| Field | Type | Range | Constraint | Description |
|-------|------|-------|-----------|-------------|
| renewable_ratio | float | [0, 1] | sum ≤ 1.0 | Fraction of demand met by renewables |
| fossil_ratio | float | [0, 1] | sum ≤ 1.0 | Fraction of demand met by fossil fuels |
| battery_action | float | [-1, 1] | — | Charge (+1) or discharge (-1) battery |

**Constraint:** `renewable_ratio + fossil_ratio ≤ 1.0`  
The remaining fraction `1 - renewable_ratio - fossil_ratio` is unmet demand (blackout risk).

---

## 5. Reward Function

### Terms (summing to [0, 1])

| Term | Weight | Formula |
|------|--------|---------|
| cost_savings | 0.30 | `1 - normalised(fossil_cost + grid_cost)` |
| carbon_score | 0.30 | `1 - normalised(carbon_emission)` |
| stability_score | 0.25 | `1 - blackout_risk` |
| renewable_bonus | 0.15 | `renewable_ratio × (1 - blackout_risk)` |

### Penalties (subtracted before clip)

| Penalty | Trigger | Value |
|---------|---------|-------|
| blackout_penalty | demand unmet > 20% | -0.5 |
| carbon_overrun | carbon_budget < 0 (hard only) | -0.8 |

**Final:** `clip(weighted_sum - penalties, 0, 1)`

---

## 6. Task Specifications

### Easy: BasicGridBalance
- **Episode length:** 48 steps
- **Conditions:** Predictable solar (low noise), stable demand, no battery
- **Objective:** Minimise cost
- **Grader criteria:** avg_cost_reduction > 30%, no blackouts, avg_renewable_ratio > 0.5
- **Score:** total_reward / max_possible_reward

### Medium: RenewableVariability
- **Episode length:** 96 steps
- **Conditions:** Noisy solar+wind, demand spikes, small battery (capacity=0.3)
- **Objective:** Avoid blackouts while maintaining renewable usage
- **Grader criteria:** blackout_episodes < 3, avg_renewable_ratio > 0.6
- **Score:** 0.4×(renewable) + 0.4×(stability) + 0.2×(cost)

### Hard: CarbonConstrained
- **Episode length:** 96 steps
- **Conditions:** Strict carbon cap (500 kgCO₂), high volatility (2× noise), limited storage
- **Objective:** Never exceed carbon cap, maintain stability > 0.7
- **Grader criteria:** carbon_budget ≥ 0 at all steps, grid_stability > 0.7
- **Score:** 0 if any carbon overrun; else 0.5×(carbon) + 0.3×(cost) + 0.2×(stability)

---

## 7. Physics Model

All functions are **deterministic** given a seed (use `np.random.default_rng(seed)`).

### Solar Output
```
solar = 0.5 + 0.5 × sin(2π × time_step / 24) + noise × N(0, noise_level)
clamped to [0, 1]
```

### Wind Output
```
wind = random_walk(previous_wind, step_size=0.05) + noise × N(0, noise_level)
clamped to [0, 1]
```

### Demand Curve
```
base = 80 MWh
morning_peak = 30 × exp(-((t - 8)² / 8))
evening_peak = 40 × exp(-((t - 18)² / 8))
spike = random_spike_probability × spike_magnitude
demand = base + morning_peak + evening_peak + spike
```

### Battery
```
new_level = clamp(level + action × charge_rate, 0, 1)
```

### Carbon Emission
```
emission = fossil_ratio × demand × emission_factor (kg CO₂/MWh)
```

---

## 8. Mercor Sub-theme

The reward function is designed so that **more tokens of agent reasoning correspond
to better reward scaling**. Specifically:
- Actions that consider multiple objectives (cost + carbon + stability) score higher
- Chain-of-thought prompts in baseline.py encourage structured reasoning
- GRPO training rewards longer, more considered outputs with higher task scores

---

## 9. Performance Targets

| Metric | Target |
|--------|--------|
| Episodes/second (numpy sim) | > 10,000 |
| Random baseline score (easy) | 0.15–0.25 |
| Heuristic baseline score (easy) | 0.68–0.74 |
| Trained agent score (easy) | 0.75–0.85 |
| Seed determinism | 100% reproducible |