---
title: ShockMarket OpenEnv
emoji: 🔋
colorFrom: green
colorTo: blue
sdk: docker
app_port: 8000
tags:
  - openenv
base_path: /web
---

#  ShockMarket: Agglomeration 2.0 (AG56)

### *Where Deep Reinforcement Learning Meets Deterministic Optimization*

---

##  OpenEnv Specification Compliant
This repository implements the full **OpenEnv** interface for a real-world industrial task: Battery Energy Arbitrage.

### Environment Description
The agent manages a 20 MWh battery to maximize profit across two simultaneous markets: Energy Arbitrage (buying low, selling high) and Ancillary Services (FCR).

### Action Space (Pydantic Model)
- `market_choice` (float [0.0, 1.0]): Probability/Decision threshold. `>=0.6` selects Ancillary, `<0.6` selects Arbitrage.
- `p_fraction` (float [-1.0, 1.0]): Power fraction. Positive means discharge (sell), negative means charge (buy).

### Observation Space (Pydantic Model)
- `energy_price` (float): Current energy market price in /MWh.
- `fcr_price` (float): Current Frequency Containment Reserve price in /MW.
- `soc` (float): Current State of Charge (MWh).

### Tasks and Graders
The environment provides 3 programmatic tasks with deterministic graders (0.0-1.0 score):
1. **Easy**: Maximize profit on a stable day with artificially high ancillary service prices.
2. **Medium**: Arbitrage during high volatility with grid stress.
3. **Hard**: Maximize revenue under strict terminal SoC constraints and degraded battery efficiency.

### Baseline Scores
Running the baseline OpenAI API inference script (`backend/baseline.py`) with `gpt-4o-mini` yields the following reproducible scores:
- **Easy**: 0.85 / 1.00
- **Medium**: 0.45 / 1.00
- **Hard**: 0.12 / 1.00

---

##  The Vision

**ShockMarket** is a high-performance framework designed to solve the ultimate puzzle in modern energy grids: **How do we maximize the value of a Battery Energy Storage System (BESS) across competing markets?**

We don't just solve a math problem; we build an intelligent agent that navigates the volatile dance between **Energy Arbitrage** and **Frequency Reserve Markets (FCR & aFRR)**.

---

##  System Architecture

The project is structured as a multi-service application to provide a full-stack experience for energy traders and data scientists:

| Service | Port | Description |
| :--- | :--- | :--- |
| **Frontend** | `3000` | React (Vite) interface for strategy visualization. |
| **Backend API** | `8080` | FastAPI server managing RL inference and data orchestration. |
| **Dashboard** | `8501` | Live Streamlit interface for deep-dive technical analytics. |

---

##  Key Features

* **Deterministic Co-Optimization:** Perfect-foresight day-ahead scheduling using Convex Optimization (CVXPY).
* **Multi-Market Synergy:** Simultaneous bidding in French Day-Ahead (DA), FCR (Symmetric), and aFRR (UP/DOWN) products.
* **Industrial Battery Modeling:** Realistic SoC dynamics including efficiency losses (90%) and linear throughput degradation penalties (~15 /MWh).
* **Hybrid Intelligence:** An advanced **Reinforcement Learning (RL)** layer (TD3/DDPG) sitting atop the MILP formulation to refine decision-making under uncertainty.

---

##  The Hybrid Architecture: RL + MILP

While the core of the repository uses **Linear Programming (MILP via CVXPY)** to find the mathematical optimum for a given set of prices, the **Agglomeration 2.0** update introduces a **Reinforcement Learning (RL)** layer to transform this from a static solver into a dynamic strategy agent.

### How RL Enhances the Optimization

The RL agent (implemented via `Stable-Baselines3`) acts as the "Grand Strategist" atop the "Tactical Solver" (MILP).

#### 1. Adaptive Market Switching

The RL agent learns the **probability of market profitability**. It makes the high-level decision:
* *Should we commit to the Ancillary market (FCR/aFRR) for the capacity payment?*
* *Or should we keep the battery empty to catch a predicted price spike in the Arbitrage market?*

#### 2. Feature-Rich Observation Space
The agent observes Market Signals (DA, FCR, aFRR), Physical State (SoC), and Temporal Context (15-min intervals).

#### 3. The Reward Function: PnL + Sustainability
The agent is trained to maximize PnL while being "nudged" by a **Terminal SoC Penalty** to return the battery to a target level by midnight.

---

##  Technical Specifications

| Parameter | Specification |
| --- | --- |
| **Time Grid** | 15-minute intervals (96 steps/day) |
| **Battery Power** | 10 MW |
| **Battery Energy** | 20 MWh |
| **SoC Limits** | 10% - 90% (Safety Buffer) |
| **Optimization Solver** | ECOS (Embedded Conic Solver) |
| **RL Algorithm** | TD3 / DDPG (Actor-Critic) |

---

##  Setup & Installation

### Option 1: Docker (Recommended for Full Stack)
The easiest way to run the entire project (Frontend, Backend, and Dashboard) in sync.

```bash
# 1) Build and start all services
docker-compose up --build

# 2) Access the services:
# - Frontend: http://localhost:3000
# - Backend API: http://localhost:8080
# - Dashboard: http://localhost:8501
```

### Option 2: Local Python Setup (`uv` optimized)
Best for core RL model development and training.

```bash
cd backend

# 1) Create and activate environment
uv venv .venv
source .venv/bin/activate  # Or .venv\Scripts\activate on Windows

# 2) Synchronize dependencies
uv sync

# 3) Run the main training & simulation pipeline
uv run main.py
```

### Option 3: Local Frontend Development
```bash
cd frontend
npm install
npm run dev
```

---

##  Project Structure

The project follows a clean service-oriented architecture:

###  [Backend](file:///backend)
The powerhouse of RL training, data orchestration, and API services.
- `src/`, `rl/`, `sim/`, `twin/`: Domain logic and optimization modules.
- `dashboards/`: Technical Streamlit dashboards for live data monitoring.
- `data/`: Market price data repository.
- `models/`: Persistent storage for trained RL model weights.
- `app.py`: FastAPI backend entry point.
- `main.py`: Entry point for training and simulation.
- `Dockerfile`: Container configuration for the backend.

###  [Frontend](file:///frontend)
React-based visualization UI for intuitive strategy monitoring.
- `src/`: React components and UI logic.
- `Dockerfile`: Multi-stage build for production-ready nginx serving.
- `nginx.conf`: Custom routing configuration.

---

##  Project Context & Credits

This framework is based on industrial-standard battery models and the reference thesis:

> *"Optimizing Residential Battery Energy Storage Systems Across Frequency Regulation Markets and Energy Arbitrage"*  **Elias Schuhmacher & Eric Rosen (2025)**.

Developed under the **Agglomeration 2.0 - AG56** initiative.