README.md

metadata

title: EnergyStock BESS-RL
emoji: ⚡
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
pinned: false

⚡ BESS-RL: Battery Energy Storage System Reinforcement Learning

An industrial-grade, OpenEnv-compliant reinforcement learning environment for optimizing Battery Energy Storage System (BESS) dispatch. Develop agents that manage grid-scale batteries to maximize revenue across multiple simultaneous streams using real-world PJM electricity market data.

---

🌍 Environment Overiew

The BESS-RL environment simulates the hourly operation of a battery connected to the PJM (Pennsylvania-New Jersey-Maryland Interconnection) grid. Agents must solve a complex co-optimization problem involving three distinct revenue and reliability objectives:

1. Energy Arbitrage (EA): Capitalizing on price volatility by charging during low-price periods and discharging when prices peak.
2. Frequency Regulation (FR): Providing grid stability by following the PJM RegD (Dynamic Regulation) signal in real-time.
3. Peak Shaving (PS): Managing local grid stress by ensuring net load remains below critical thresholds to avoid heavy demand charges.

Key Features

• Real Data: Driven by authentic PJM hourly Locational Marginal Prices (LMP), RegD signals, and load profiles.
• Physics-Informed: Includes battery degradation (cycle costs), efficiency losses, and SOC (State of Charge) constraints.
• Task Tiers: Built-in curriculum from single-objective arbitrage to full multi-objective co-optimization.

---

📊 Observation Space

The agent receives a 6-dimensional vector at each timestep representing the current state of the market and the battery.

Index	Field	Range	Units	Description
0	hour_of_day	0 – 23	Hours	Current hour (normalized to float)
1	soc	0.0 – 1.0	%	Battery State of Charge (0% to 100%)
2	price_lmp	0 – 200+	$/MWh	Current Locational Marginal Price
3	p_avg	0 – 200+	$/MWh	24-hour rolling average LMP
4	freq_regd	-1.0 – 1.0	Signal	PJM RegD frequency regulation signal
5	load_mw	0 – 50+	MW	Local grid load demand

---

🕹️ Action Space

The action space is a continuous vector of 3 values, each bounded in [-1.0, 1.0].

Index	Component	Range	Physical Meaning
0	a_PS	[-1, 1]	Peak Shaving dispatch request
1	a_EA	[-1, 1]	Energy Arbitrage dispatch request
2	a_FR	[-1, 1]	Frequency Regulation dispatch request

The environment implements a "clipping aggregator": a_final = clip(a_PS + a_EA + a_FR, -1.0, 1.0). +1.0 represents Max Charge Power, while -1.0 represents Max Discharge Power.

---

🏆 Multi-Objective Tasks

Task	Level	Objectives	Learning Focus
easy	Beginner	EA	Price forecasting and temporal arbitrage timing.
medium	Intermediate	EA + FR	Balancing high-frequency signal tracking with market trends.
hard	Advanced	EA + FR + PS	Multi-constraint optimization under grid stress.

---

🛠️ Setup & Installation

Option 1: Docker (Recommended)

The project is containerized for seamless deployment.

# Build and start both the backend server and React dashboard
docker-compose up --build

The application will be available at http://localhost:7860.

Option 2: Manual Setup

If you prefer to run the components locally for development:

# 1. Install dependencies
pip install -r backend/requirements.txt
pip install openai torch numpy pandas pydantic python-dotenv

# 2. Configure environment
cp .env.example .env
# Open .env and add your HF_TOKEN and other API keys

# 3. Start the FastAPI server
python backend/main.py

---

🚀 Usage

Running the Evaluation Harness

The inference.py script runs a full evaluation of the SAC agent across all task tiers and outputs OpenEnv-compliant logs.

# Requires HF_TOKEN, MODEL_NAME, and API_BASE_URL env variables
python inference.py

Training Your Own Agent

We use Soft Actor-Critic (SAC) to train the dispatch models.

# Train on the hard task for 500 episodes
python train/trainer.py --task hard --episodes 500

AI-Powered Evaluation (llm-evaluator)

Run the AI-powered evaluation to get a qualitative verdict and a robust performance score from the LLM evaluator.

# Evaluate specific task tiers
python train/llm_evaluate.py --task easy
python train/llm_evaluate.py --task medium
python train/llm_evaluate.py --task hard

---

🛠️ Environment Configuration

Variable	Description
API_BASE_URL	Endpoint for the LLM advisory layer (OpenAI compatible)
MODEL_NAME	The model identifier for inference (e.g., meta-llama/Llama-3.1-8B-Instruct)
HF_TOKEN	Your Hugging Face API key for inference and model syncing

---

✅ Submission Validation

Before submitting your project, you can use the validate-submission.sh script to ensure everything is configured correctly.

What it validates:

1. Connectivity: Checks if your HuggingFace Space is live and the /reset endpoint is responsive.
2. Containerization: Verifies that your Dockerfile builds successfully (typically within a 10-minute timeout).
3. OpenEnv Compliance: Runs openenv validate to ensure your repository structure and environment meet the OpenEnv standards.

How to run:

You can run the script locally or directly via curl.

Prerequisites:

• Docker: Install Docker
• OpenEnv: pip install openenv-core

Run locally:

chmod +x validate-submission.sh
./validate-submission.sh <your_hf_space_url>

Run via curl:

curl -fsSL https://raw.githubusercontent.com/PuttaVenkataSai/PowerGrid/main/validate-submission.sh | bash -s -- <your_hf_space_url>

Example: ./validate-submission.sh https://saiteja020-powergrid.hf.space

---

🏗️ Architecture

• Agent: SAC with Twin Critics and automatic entropy tuning for robust policy learning.
• Frontend: React + Tailwind CSS dashboard for real-time visualization of BESS dispatch.
• Backend: FastAPI high-performance server exposing OpenEnv standard endpoints (/reset, /step).
• Validation: Internal validate-submission.sh script to ensure HF Space and OpenEnv compliance.