Spaces:
Sleeping
Sleeping
| # Task 4 — `multi_stage_cascade` | |
| ## Purpose | |
| Task 4 is the hardest benchmark task. Three lines are disconnected, load is increased, and the task is designed around guaranteed multi-stage degradation rather than full prevention. The goal is to preserve as much load as possible across stage boundaries over `30` steps. | |
| Current task definition in code: | |
| - description and scenario setup: [server/tasks.py](../server/tasks.py) | |
| - reward: [server/environment.py](../server/environment.py) | |
| - grader: [server/graders.py](../server/graders.py) | |
| ## What We Changed | |
| This task changed the most during development. | |
| Main changes: | |
| - made it explicitly stage-aware in both prompt and reward logic | |
| - added stage boundary assessment and island-availability tracking | |
| - added controlled-islanding and redispatch guidance to the prompt | |
| - changed the reward to penalize load loss at stage boundaries instead of only rewarding short-term survival | |
| - changed the grader to score stage completion, load preservation, island quality, and stage-wise stabilization speed | |
| These changes were motivated by multi-stage cascading-failure literature, where greedy single-stage action selection is known to miss later-stage consequences. [1][2] | |
| ## Current Implementation | |
| Reset: | |
| - three lines disconnected | |
| - load increase around the expert benchmark setting | |
| - overflow window tightened for faster propagation | |
| - `30` steps split into three conceptual stages | |
| Prompt-side task rules now include: | |
| - explicit stage context such as `stage_1_of_3` | |
| - steps remaining to the next boundary | |
| - `available_load_ratio` | |
| - `available_island_ratio` | |
| - guidance to avoid stranding load in islands without enough generation | |
| - separate controlled-islanding and redispatch guidance blocks | |
| Current reward shape: | |
| - generation cost penalty | |
| - positive term for available-island convergence | |
| - load-loss penalty at stage boundaries | |
| - heavy penalties for blackout or convergence failure | |
| - positive win reward at the horizon if enough load is preserved | |
| Current grader: | |
| - `30%` stage completion | |
| - `40%` load preservation | |
| - `20%` island quality | |
| - `10%` speed of stabilization within each stage | |
| ## What We Observed | |
| This is the task where the SFT model most clearly outperformed the base model while staying safe. | |
| Final SFT result, seed block `0..4`: | |
| - `0.9156444` | |
| Final SFT result, unseen seeds `100..102`: | |
| - `0.9069863` | |
| Base-model result on seed block `0..4`: | |
| - `0.000` | |
| This task also exposed the limit of the original offline GRPO setup: | |
| - completed GRPO runs preserved the SFT behavior | |
| - they did not improve the `multi_stage_cascade` score | |
| ## Current Limitation | |
| The task is intentionally constrained by verified candidate actions and a fixed benchmark horizon. That makes it robust and reproducible, but it also means some genuine long-horizon improvements may be hard to expose unless the candidate set itself contains them. | |
| ## References | |
| [1] Mitigating Multi-Stage Cascading Failure by Reinforcement Learning (ISGT Asia 2019): | |
| https://vbn.aau.dk/en/publications/mitigating-multi-stage-cascading-failure-by-reinforcement-learnin/ | |
| [2] Deep Reinforcement Learning for Power Grid Multi-Stage Cascading Failure Mitigation: | |
| https://www.climatechange.ai/papers/iclr2025/1 | |
| [3] Local implementation: | |
| [server/tasks.py](../server/tasks.py), [server/environment.py](../server/environment.py), [server/graders.py](../server/graders.py), [inference.py](../inference.py) | |