# Benchmark Notes

## Benchmark Goal

This benchmark is designed to test whether an LLM-driven controller can operate a small but nontrivial power grid safely under progressively harder disturbance settings. The benchmark is implemented as an OpenEnv environment backed by Grid2Op, with task-specific resets, shaped environment rewards, and separate per-task graders. [1][2][3]

Core implementation:
- task setup: [server/tasks.py](../server/tasks.py)
- environment and shaped rewards: [server/environment.py](../server/environment.py)
- official submission scoring: [server/graders.py](../server/graders.py)
- verified-candidate inference and evaluation: [ft_inference.py](../ft_inference.py)

## Tasks And Variants

The benchmark currently contains four tasks:
- `single_fault`
- `n_minus_1`
- `cascade_prevent`
- `multi_stage_cascade`

Current benchmark variants:
- `single_fault_easy`
- `single_fault_moderate`
- `single_fault_severe`
- `n_minus_1_fixed`
- `cascade_prevent_easy`
- `cascade_prevent_medium`
- `cascade_prevent_hard`
- `cascade_prevent_extreme`
- `multi_stage_cascade_expert`

This gives the benchmark both breadth across tasks and controlled difficulty variation within tasks.

## Why The Benchmark Is Varied

Each task tests a different operational skill:

- `single_fault`: congestion relief under a short redispatch-only horizon
- `n_minus_1`: secure operation after a fixed contingency, including safe reconnection
- `cascade_prevent`: time-critical prevention of auto-trips
- `multi_stage_cascade`: load preservation across stage boundaries under guaranteed degradation

These are not cosmetic variants. They differ in:
- reset structure
- allowed action families
- horizon length
- prompt guidance
- reward shape
- grading logic

That is why the same model can succeed on one task and still struggle on another.

## Why The Benchmark Is Robust

### 1. Same evaluation pipeline for every model

All compared models use the same verified-candidate inference pipeline:
- simulate legal actions
- prompt the model with verified outcomes
- require a valid `GridAction`
- require exact match to a verified candidate
- execute and grade

This makes base vs SFT vs GRPO comparisons much more defensible.

### 2. Strong protection against invalid-action wins

A model does not get credit for:
- invalid JSON
- malformed actions
- invented actions outside the verified set

This is one of the benchmark’s strongest robustness features.

### 3. Separate train-time reward and benchmark score

The environment has shaped rewards, but the official task score comes from dedicated graders. This prevents us from treating a convenient training reward as the benchmark truth.

### 4. Seen-seed and unseen-seed evaluation

We did not only test on one small seed block. The current evaluation story includes:
- seed block `0..4`
- unseen seed block `100..102`

This does not prove full generalization, but it is much stronger than a single-block claim.

### 5. Distinct task-specific scoring

The graders are different on purpose:
- target completion for `single_fault`
- emergency/steady-state/reconnection for `n_minus_1`
- containment/recovery for `cascade_prevent`
- stage completion/load preservation/island quality for `multi_stage_cascade`

This makes it harder for one generic exploit to score well across the entire benchmark.

## Current Results

Best completed submission model:
- `outputs/models/grid2op-qwen3-4b-sft-3k-v1`

Main seed block `0..4`, `5` episodes per task:
- `single_fault`: `0.856`
- `n_minus_1`: `0.990`
- `cascade_prevent`: `0.990`
- `multi_stage_cascade`: `0.9156444`
- failures: `0`

Unseen seed block `100..102`, `3` episodes per task:
- `single_fault`: `0.830`
- `n_minus_1`: `0.9222223`
- `cascade_prevent`: `0.990`
- `multi_stage_cascade`: `0.9069863`
- failures: `0`

These results show that the benchmark is strong enough to separate:
- an unreliable base model
- a strong SFT model
- a safe but flat GRPO follow-up

## What The Benchmark Still Does Not Claim

We should be careful about overclaiming.

This benchmark does not prove:
- utility-grade deployment readiness
- full contingency coverage
- universal transfer to much larger grids

What it does provide is:
- a reproducible four-task control suite
- realistic Grid2Op dynamics
- verified-candidate action enforcement
- task-specific grading
- seen and unseen seed evaluation

That is a strong benchmark package for a hackathon submission.

## References

[1] OpenEnv integration for training and evaluation:  
https://huggingface.co/docs/trl/openenv

[2] Learning to run a power network challenge for training topology controllers:  
https://www.sciencedirect.com/science/article/abs/pii/S0378779620304387

[3] RL2Grid benchmark paper:  
https://huggingface.co/papers/2503.23101

[4] Local implementation:  
[server/tasks.py](../server/tasks.py), [server/environment.py](../server/environment.py), [server/graders.py](../server/graders.py), [ft_inference.py](../ft_inference.py)